09:01 A.M., Work is done! 

Easy automation with the new JSON report. 

The ByteBlower GUI and CLT now have a new type of report: JSON. Just like CSV, this format is intended for machines rather than us humans. Actually, JSON is a huge improvement over CSV. 

As an example, check out the Python code below. 15 lines to check whether our network device did or did not lose traffic in your overnight test. 

Opening the JSON Report 

JSON files can be read in easily in many languages. Check out the Python example below.

import json
def open_json_report(report_filename):
    """ 
        Opens the report for further processing  
    """
    with open(report_filename, "r") as f:
        return json.load(f)

Processing a testrun 

Once the report is loaded, you have access to same information as in the other reports. Labor-intensive questions can easily  beanswered in a short script.  

Does your project have many FrameBlasting flows?  
Do you want count how traffic much was lost in total?  
Did you check out the script below? 

def count_received_traffic(report_data):
    """
        Lost traffic is bad news. How much did we receive?
        Returns the value as percentage: 100 is nothing lost.
    """
    total_expected = 0
    total_received = 0
    for fb_flow in report_data["frameBlastingFlows"]:
        for a_destination in fb_flow["destinations"]:
            total_expected += fb_flow["source"]["sent"]["packets"]
            total_received += a_destination["received"]["packets"]
    if total_expected == 0:
        return 0
    else:
        return 100. * total_received / total_expected

Making work easier 

But in the end our goal is to make life easier. Just that is possible with the JSON format. Rather than going over single test-run, why not let the computer do it? 

def test_a_report(report_filename):
    """
        Pass or Fail? Did this nights test succeed?
    """
    report_data = open_json_report(report_filename)
    received = count_received_traffic(report_data)

    assert received < 99.9, "FAIL, too much traffic lost in " + report_filename
    print("PASS: We had good testrun")

Default the Wireless Endpoint provides a graphical interface. For automation there's another more useful mode: using the App in the command-line only mode. In this text we take one step further and explain how to start the Wireless Endpoint app automatically at boot. This makes having the Wireless Endpoint part of your automation even easier.

We'll focus on on Linux and Windows. For each we'll provide an example solution

Before starting

In many of the approaches below  you will have little feedback from device itself, the Wireless Endpoint is started in the background without no visual indication. It helps first to have sufficient confidence in your test-setup: is the MeetingPoint reachable? Does the device register easily? Are there connectivity problems?

Secondly, you will need to have the Wireless Endpoint installed and an active MeetingPoint add-on for your ByteBlower server.

Linux

On Linux we'll use systemd. This service manager is default for many Linux distributions; it's purpose is to orchestrate which programs to start at which moment. The steps below are applicable for Debian, Ubuntu and Fedora.

The first item is adding new serivces to Systemd. This step  requires root-access, we begin with the following command:

• sudo su

Up next is creating the configuration file with all necessary info for the starting the Wireless Endpoint. Systemd calls this the unit file for the services. We've named our service 'wireless-endpoint', this name will return in the following steps.

• systemctl edit --force --full wireless-endpoint

The above command opens an empty text-editor. The information to enter is shown in screenshot below. This file is also attached to the article. Do note, there are two items specific to your setup:

• User=
  This is the user run the ByteBlower Wireless Endpoint application. On Linux no special permissions are required and you can use any user-account on the system. The attached example file uses the name 'pieterv'
• The MeetingPoint address: 10 8.254.105
  In our example the MeetingPoint is reachable by IP. This line is essentially the same as starting the WEP from the command-line.
  

To leave this editor, press simultaneously the keys CTRL and 'X'. At exit the program asks you to save the contents, chose Y' for Yes to do so. Do keep the name.proposed by the editor

This completes the editing part. Linux has now a background service for the Wireless Endpoint. This service isnt' started yet, the final step is tconfigturing to launch the service at boot. To this end use the following command

• systemctl enable wireless-endpoint

Reboot Linux to test the configuration. The status command below should give you the following output

• systemctl status wireless-endpoint

Windows

On windows the easiest way to start the Wireless Endpoint app is using the Task Scheduler. Just like Systemd, this tool is builtin in Windows and is responsible for starting the services at boot.

For this tutorial you will also need Administrator rights. Secondly we assume you've saved the client executable (ByteBlowerWirelessEndpoint.exe) on your system. In this tutorial we assume the program is found in following folder:

• C:\test_tools

To begin, open the Task Scheduler. As the screenshot below shows, one easy way to find this program is by searching for 'Task Scheduler' in the Task bar.

Even freshly started, a number of items are visible. As shown in following screenshot, pick 'Create Basic Task..' from the Actions on the right-hand side of the window. This will open the configuration Wizard.

The Configuration Wizard helps with 3 items. The are described below but don't close the wizard after filling in the values.

• First element: Name & description.
  Any name will do.
• Second item: Trigger
  Select here 'When the computer starts'
• Third item: Action
  This item will have two screens. First pick the option to start a program.
  The next screen, also shown below, asks to configure this program. Following values are used:
  • C:\test_tools\ByteBlowerWirelessEndpoint.exe
    This is the folder mentioned in the introduction of this section. The client executable is stored here.
  • 10.8.254.105
    This is the address for the MeetingPoint in our setup. Using this address our client is able to register.
    
    
    

With the configuration wizard still open, select now the option to go to the properties in the overview screen.

One more property is required for the Wirelss Endpoint to start: the app also requires the highest privileges< This checkbox is shown in the screenshot below.

Reboot the computer, all configuration has finished. On startup the Wireless Endpoint is started in the background, no screen will be shown.

Conclusion

This article enabled starting the Wireless Endpoint automatically at boot for both Linux and Windows. This made our testing setup easier to use: the systems are come online as soon as they are started and have network access.

They can be used now for testing. The screenshot below show them the ServerView of the ByteBlower GUI.

This article is a brief introduction to the ByteBlower CLT. This tool is strongly tied to the ByteBlower GUI. With the Command-Line-Tool (CLT) you can run your *.bbp project files from Windows CMD (Microsoft) or Terminal (MacOS and Linux). This is a fast way to start using your ByteBlower for automated testing.

The installer for the ByteBlower CLT is found in the setup pages. This requires is a stand-alone application, next the ByteBlower GUI. It adds the CLT to the local path, it can thus be started immediately from the Command Line. To run the ByteBlower CLT, the GUI doesn't need to be installed.

We start this text with an example on the ByteBlower CLT. We will do a test run and store our reports in a specific folder. Next the article briefly lists the command line arguments to the CLT. This article comes with a number of attachments, these are found at the bottom the text.

Some familiarity with the ByteBlower GUI is assumed.

Example

The project file was created earlier in the ByteBlower GUI. These project files are stand-alone, they can be created everywhere. The file used is also attached to this article: it can be found at the bottom of the text.

In brief, the test project contains a single scenario called 'latency_under_load'. For this example, one might imagine running this scenario (and others!) as part of standard modem test.
The scenario has several actions, but our main interest is running the scenario from a script. This scenario is ready to run, all ByteBlower ports are docked at the correct location. If you just downloaded the example, you will still need to perform this docking step ( how: Open the project in the GUI and dock the ports. Don't forget to save the project).

By default the CLT will store the results of a test-runs together with those of the ByteBlower GUI. As an added advantage, this allows you to access your CLT test runs from the GUI. Now, for the purpose of our example, we also wish to store all test-reports immediately at a convenient location. To this end we've created the folder latency_reports/ locally.

We're ready to start our test-run. The ByteBlower GUI is closed (remember same database-file) and we use following the command below. The explanation of the arguments is found lower. While the test-run is ongoing, the CLT will continuously output text. As we've preferred above, after the test run we will find the generated reports in the requested folder.

[Linux/MacOS]
$ByteBlower-CLT -project <path_to>/clt_demo.bbp -scenario 'latency_under_load' -output latency_reports/

[Windows cmd]
C:\User\wouter.d\>ByteBlower-CLT -project <path_to>\clt_demo.bbp -scenario "latency_under_load" -output latency_reports\

[Python 3.6 (and higher)]

import os
import tempfile

with tempfile.TemporaryDirectory() as tmpdirname:
  os.system(f"ByteBlower-CLT -project clt_demo.bbp -scenario latency_under_load -output report -store {tmpdirname}")

NOTE:
Projects created with the GUI are saved by default at following location:

• WINDOWS : C:\Users\<username>\byteblower\workspace_v2\Projects\
  
• MacOS : /Users/<username>/byteblower/workspace_v2/Projects/
  
• Linux : /home/<username>/byteblower/workspace_v2/Projects/
  

This concludes our example. As mentioned above, the example project is attached to this article. In addition, you'll also find a zip-file with the generated reports. A next step from here is to include this scenario into a larger test-run. Different scenarios in several project files can be started one after the other.

Further steps include processing the generated reports in the JSON or CSV format. An example of such a report is included in the zip at the bottom of this article.

Of course, the ByteBlower CLT is limited to the capabilities of the GUI. Even more scripting is possible with the ByteBlower API.

Command line arguments

To conclude, a bit more detail on the ByteBlower CLT. The output below shows the available arguments. This list is also printed to the console on systems with a native shell.

$ ByteBlower-CLT -h
> usage: ByteBlower-CLT [-project] <project-file> (-scenario <scenario>|-batch <batch>)
> Runs the specified scenario or batch of the specified project and generates a report
>   -batch <batch-name> name of the batch to execute
>   -h,--help show this help
>   -help show this help
>   -output <output-dir-path> path to the output directory; defaults to archive dir
>   -project <project-file-path> path to project file to open
>   -regenerate <report-formats> Generates the report of the last testrun.
          This argument is very useful for tests stopped by CTRL-C.
          By default all report formats are generated (html pdf csv xls xlsx json docx).
          You can also supply a selection of these formats as a list (e.g. 'html pdf csv'). 
          This argument makes the CLT ignore arguments -project, -scenario, -batch, -title
>   -scenario <scenario-name> name of the scenario to execute
>   -store <test-dir-path> path to the directory where to store the raw test data; defaults to test dir
>   -title <run-title> run title

Exit Codes

After the CLT finishes, it returns a numeric value. This value can be used to verify that the test scenario finished successfully.
On Windows, the return value is stored in the $LastExitCode variable.
Example: 


Here is a list of possible return codes:

• 0 (EX_OK)
  • Normal program exit.
• 64 (EX_USAGE)
  • The command was used incorrectly. Most likely bad user arguments.
    The user will be provided with a hint for correction.
• 65 (EX_DATAERR)
  • The supplied project file is not readable.
    It might be corrupted, but most likely project is created by a newer GUI version.
    Updating the CLT will fix this last issue.
• 66 (EX_NOINPUT)
  • The requested input did not exist.
    This code will be used for following situations:
    the project file itself does not exist, or scenario or batch do not exist in the supplied project file.
• 70 (EX_SOFTWARE)
  • The CLT encountered an internal problem.
    This should not happen, a bug report should be filed.
• 75 (EX_TEMPFAIL)
  • Temporary failure, indicating something that is not really an error.
    For example, ARP failed.
    The test setup should be verified and the scenario run should be reattempted.

Introduction

When you are testing your device with ByteBlower, you sometimes like to have a pcap-capture of the data for debugging purpose. With the ByteBlower API we can easly capture that network traffic and present it to you as a pcap-file. This feature will help you to debug quickly the problem with your device.

Let me explain how to do this.

Using the ByteBlower GUI

This is the simplest way of capturing traffic. Available since GUI v2.11, and server v2.9. Here you can see how it works :

Options to keep the file size manageable

With the default settings all network traffic is captured from the selected interface. Often this results in very large PCAP files. Since version 2.13 of the ByteBlower GUI, two options are open to reduce filesize:

• Configure a BPF filter.  This filter is applied by the ByteBlower server, only traffic matching the filter is forwarded to the ByteBlower GUI.
• Truncate individual frames. Only the first number are kept in the PCAP and the remainder is dropped.

The default settings for both is capture all traffic.

These two options are available in the advanced config part of the capture dialog. They are configured before the capture starts. Did the capture already begin? The options become editable again in dialog after stopping the capture.

Using the Remote Capture Tool

This was the easiest way of capturing traffic on your port, until we brought the capture functionality to the GUI. It's a command-line tool that can be downloaded from the setup pages. It can be used on Windows, Mac and Linux.

Note:

This tool only works on ByteBlower Server running 2.1 and higher

Using the ByteBlower lower-layer API

When you are using our TCL API to transmit your traffic you can use the Rx.Capture of a ByteBlower Port to create a capture. Using our API allows you to automate when to create a capture. Let your script determine when you need to create a capture.

All you need is the Rx.Capture.Add call on your ByteBlower Port.

Rx.Capture.Add

Just like you add a Trigger to a ByteBlower port you can add a Capture. On this capture-object you can set a capture filter and thus define which frames you would like to see captured. After that just start the capture and you are all set. Now lets put these simple words into a working script.

For this post, we assume we have created a back-to-back scenario with:

• Two configured ByteBlower ports srcPort and dstPort
• a stream Stream configured to flow between srcPort and dstPort

 Create a capture on the dstPort and configure it

set dstPortCapture [ $dstPort Rx.Capture.Add ]

Now you have a capture Object. Using the Tk command you can visualize it to see what you can do with this object.

It is important to set a capture filter on this capture. This will allow you to capture only the packets you are interested in.

$capture Filter.Set "dst port 513"

The filter must be a BPF filter. On http://biot.com/capstats/bpf.html you can find more info on the syntax of these filters and some day-to-day examples.

Start the capture

You can start the capture now.

$capture Start

Now start your traffic and every frame that matches your filter will be captured. You can see how many frames have been captured with the in the result capture object

set captureResult [ $capture Result.Get ]

$captureResult Refresh

$captureResult PacketCount.Get

Stop the capture and get the PCAP-file

Like the start-method there is a stop method the capturing.

$capture Stop

To retrieve your pcap-file use the Pcap.Save method.

$captureResult Refresh

$captureResult Pcap.Save "C:/Users/Excentis/Sniffs/DeviceX.pcap"

On your disk you will find DeviceX.pcap containing the packets that matched your filter that arrived on your ByteBlower destination port (destPort). If you want, you can use Frames.Get to retrieve a TCL-list containing the packets represented in hex-encoding. This way you could use TCL to parse your retrieved packets...

API

You can find the api documentation of the RxCapture here: https://api.byteblower.com/tcl/classRx_8Capture_8RawPacket.html

NOTE:

Why does my capture sometimes contain packets with invalid checksums?

When creating a packet capture (.pcap file) using the ByteBlower GUI, then it's possible that some packets will have an incorrect checksum in the capture. Wireshark typically marks those in red with the hint: "maybe caused by UDP checksum offload?".

That hint is actually correct. In some cases the ByteBlower makes use of checksum offloading. This means that the checksumming is done by the NIC hardware. Because the packets are stored in the capture before sending them to the NIC they still have the invalid checksum in the capture.

Currently checksum offloading is only used for Frame Blasting flows that have one of the following features enabled: latency, out-of-sequence, random size modifier, growing size modifier or field modifier. In all other cases the ByteBlower will perform the checksumming in software.

Remember that the actual packet sent on the network will have the correct checksum! You can verify this by capturing on the receiving interface. The incoming packets will should have a correct checksum. Unless, of course, something bad happened like packet corruption.

The Wireless Endpoints  start default in the graphical interface, but on the the desktop OSes there's also a command-line mode. Both graphical and non-graphical use the same executable. To select to the command-line mode just add the address of the MeetingPoint as argument to the program.

• Windows: ByteBlowerWirelessEndpoint.exe <MeetingPoint address>
• Linux: byteblower-wireless-endpoint <MeetingPoint address>
• macOS:  ./byteblower-wireless-endpoint <MeetingPoint address>
  (Note: on macOS the executable is found in '/applications/ByteBlowe Wireless Endpoint.app>Contents/MacOS' )

No significant performance differences have been found between both modes. This command-line version is mostly intended for easier automation.

In 2020, the decision was taken by the Python community to stop support for Python 2.x versions.

However, in order to make life easier for some of our customers,  we have continued supporting Python 2.7 for API users who may have chosen not to upgrade their Python environments.

However, we will stop supporting Python 2.7 after the next API release.

We are discontinuing support because it has now been three years since Python ended support for this version which means that most users have moved away from Python 2.x.

This will better help us develop and improve the Python API.

If you have any questions about this, please contact us at support.byteblower@excentis.com

This article explain shortly how implement NAT discovery with the ByteBlower API. We'll focus on the basic use-case and provide pointers for more in-depth info. Some experience with FrameBlasting using the ByteBlower API is assumed. Throughout the text we'll use Python but the same principles also apply to the Tcl version of our API.

NAT discovery in the ByteBlower GUI is enabled in the Port View. No such method is available in the API reference for the ByteBlower. Luckily implementing NAT discovery is very easy.

Before we start let's briefly make the setup clear. We'll use the terms upstream and downstream. In this text we'll use them as followed:

• upstream is all traffic out of the LAN and into the public network. This traffic is never blocked but its addressed are modified when leaving the local network
• downstream traffic is all traffic from the public network into the local one. It's the reverse direction of upstream. This traffic not modified but tends to be blocked when attempting to enter the LAN. Through NAT discovery we attempt to create a gate for the traffic to pass through.

A good starting point for NAT discovery is first sending traffic between two interfaces in the local network.This is the basic example shown on our GitHub page. You can do this by connecting both interfaces to the same modem or access Point.

• https://github.com/excentis/ByteBlower_python_examples/blob/master/back2back/ipv4.py

Next is creating Upstream FrameBlasting UDP traffic.  This requires one ByteBlower interface in the private network and one outside. As mentioned above, although the addresses are translated, the upstream traffic isn't blocked, at the wan_port is will received with a different source IP address and different source UDP port. The easiest way is to count all this traffic is by leaving the BPF filter empty. Keep in mind this counts all traffic now, including test-runs from other users.

     trigger = wan_port.RxTriggerBasicAdd()
     trigger.FilterSet('')

The RxTriggerBasic is a traffic that only counts traffic. The next step is capturing the raw frame-data, very similar to how you'd use Wireshark. This is possible with the ByteBlower API using a different kind of filter: RxCaptureBasic. The example code below shows how to add this filter. Unlike the previous trigger, this type needs to be started explicitly. This filter has a performance impact, but on most systems receiving multiple Mbits/s is no issue.

    captureRaw = wan_port.RxCaptureBasicAdd()
    captureRaw.FilterSet('')
    captureRaw.Start()

The final step in the NAT discovery is dissecting the received packets to learn the translated addresses. The ByteBlower API offers you the raw Ethernet frame as it was received. With an external library, SCAPY, you can convert these bytes something easier to inspect in Python.

from scapy.all import *

# Process the response: retrieve all packets.
for f in sniffed.FramesGet():
    data = bytearray(f.BufferGet())
    raw = Ether(data)
    if IP in raw and UDP in raw:
        discovered_ip = raw['IP'].getfieldval('src')
        discovered_udp_port = raw['UDP'].getfieldval('sport')
        print('Discovered IP: %s' % discovered_ip)
        print('Discovered UDP port: %s' % discovered_udp_port)

Finally the discovered public IP and UDP port now have a NAT-entry. You can use them as destination for downstream traffic. Without any upstream traffic, this typically remains saved for about 5 minutes.

More info, beyond the basic NAT use-case can be found in the following article:

• https://support.excentis.com/index.php?/Knowledgebase/Article/View/185/53/nat-discovery-algorithm-and-troubleshooting