Knowledge base : Knowledge Base > ByteBlower > Examples

This article is a brief introduction to the ByteBlower CLT. This tool is strongly tied to the ByteBlower GUI. With the Command-Line-Tool (CLT) you can run your *.bbp project files from Windows CMD (Microsoft) or Terminal (MacOS and Linux). This is a fast way to start using your ByteBlower for automated testing.

The installer for the ByteBlower CLT is found in the setup pages. This requires is a stand-alone application, next the ByteBlower GUI. It adds the CLT to the local path, it can thus be started immediately from the Command Line. To run the ByteBlower CLT, the GUI doesn't need to be installed.

We start this text with an example on the ByteBlower CLT. We will do a test run and store our reports in a specific folder. Next the article briefly lists the command line arguments to the CLT. This article comes with a number of attachments, these are found at the bottom the text.

Some familiarity with the ByteBlower GUI is assumed.

Example

The project file was created earlier in the ByteBlower GUI. These project files are stand-alone, they can be created everywhere. The file used is also attached to this article: it can be found at the bottom of the text.

In brief, the test project contains a single scenario called 'latency_under_load'. For this example, one might imagine running this scenario (and others!) as part of standard modem test.
The scenario has several actions, but our main interest is running the scenario from a script. This scenario is ready to run, all ByteBlower ports are docked at the correct location. If you just downloaded the example, you will still need to perform this docking step ( how: Open the project in the GUI and dock the ports. Don't forget to save the project).

By default the CLT will store the results of a test-runs together with those of the ByteBlower GUI. As an added advantage, this allows you to access your CLT test runs from the GUI. Now, for the purpose of our example, we also wish to store all test-reports immediately at a convenient location. To this end we've created the folder latency_reports/ locally.

We're ready to start our test-run. The ByteBlower GUI is closed (remember same database-file) and we use following the command below. The explanation of the arguments is found lower. While the test-run is ongoing, the CLT will continuously output text. As we've preferred above, after the test run we will find the generated reports in the requested folder.

[Linux/MacOS]
$ByteBlower-CLT -project <path_to>/clt_demo.bbp -scenario 'latency_under_load' -output latency_reports/

[Windows cmd]
C:\User\wouter.d\>ByteBlower-CLT -project <path_to>\clt_demo.bbp -scenario "latency_under_load" -output latency_reports\

[Python 3.6 (and higher)]

import os
import tempfile

with tempfile.TemporaryDirectory() as tmpdirname:
  os.system(f"ByteBlower-CLT -project clt_demo.bbp -scenario latency_under_load -output report -store {tmpdirname}")


NOTE:
Projects created with the GUI are saved by default at following location:

  • WINDOWS : C:\Users\<username>\byteblower\workspace_v2\Projects\
  • MacOS : /Users/<username>/byteblower/workspace_v2/Projects/
  • Linux : /home/<username>/byteblower/workspace_v2/Projects/

This concludes our example. As mentioned above, the example project is attached to this article. In addition, you'll also find a zip-file with the generated reports. A next step from here is to include this scenario into a larger test-run. Different scenarios in several project files can be started one after the other.

Further steps include processing the generated reports in the JSON or CSV format. An example of such a report is included in the zip at the bottom of this article.

 

Of course, the ByteBlower CLT is limited to the capabilities of the GUI. Even more scripting is possible with the ByteBlower API.

Command line arguments

To conclude, a bit more detail on the ByteBlower CLT. The output below shows the available arguments. This list is also printed to the console on systems with a native shell.

$ ByteBlower-CLT -h
> usage: ByteBlower-CLT [-project] <project-file> (-scenario <scenario>|-batch <batch>)
> Runs the specified scenario or batch of the specified project and generates a report
>   -batch <batch-name> name of the batch to execute
>   -h,--help show this help
>   -help show this help
>   -output <output-dir-path> path to the output directory; defaults to archive dir
>   -project <project-file-path> path to project file to open
>   -regenerate <report-formats> Generates the report of the last testrun.
          This argument is very useful for tests stopped by CTRL-C.
          By default all report formats are generated (html pdf csv xls xlsx json docx).
          You can also supply a selection of these formats as a list (e.g. 'html pdf csv'). 
          This argument makes the CLT ignore arguments -project, -scenario, -batch, -title
>   -scenario <scenario-name> name of the scenario to execute
>   -store <test-dir-path> path to the directory where to store the raw test data; defaults to test dir
>   -title <run-title> run title

Exit Codes

After the CLT finishes, it returns a numeric value. This value can be used to verify that the test scenario finished successfully.
On Windows, the return value is stored in the $LastExitCode variable.
Example: 


Here is a list of possible return codes:

  • 0 (EX_OK)
    • Normal program exit.
  • 64 (EX_USAGE)
    • The command was used incorrectly. Most likely bad user arguments.
      The user will be provided with a hint for correction.
  • 65 (EX_DATAERR)
    • The supplied project file is not readable.
      It might be corrupted, but most likely project is created by a newer GUI version.
      Updating the CLT will fix this last issue.
  • 66 (EX_NOINPUT)
    • The requested input did not exist.
      This code will be used for following situations:
      the project file itself does not exist, or scenario or batch do not exist in the supplied project file.
  • 70 (EX_SOFTWARE)
    • The CLT encountered an internal problem.
      This should not happen, a bug report should be filed.
  • 75 (EX_TEMPFAIL)
    • Temporary failure, indicating something that is not really an error.
      For example, ARP failed.
      The test setup should be verified and the scenario run should be reattempted.

The ByteBlower Wireless Endpoint is software that turns your mobile devices into ByteBlower Ports.
With minimal effort, you can integrate iPhones, iPads, Windows phones, Android devices, but also Linux machines, MacBooks, Windows PCs, ... into your network test scenarios.

Requirements

The Wireless Endpoint runs on your SmartPhone, tablet or Laptop; it's the software that generates the testing traffic. You need to install the Wireless Endpoint on all devices you want to test. This Wireless Endpoint software is available for free on all major app stores. You can find the download links collected in the website below.

Tests are organized from the ByteBlower chassis. For the Wireless Endpoint to become active it needs to reach the management connection of the ByteBlower. When possible, we suggest connecting the second management interface into the Access Point. This is the Lan only option explained in the the link below:

On the ByteBlower chassis, it's the Meeting Point software that does the interaction with the Wireless Endpoint. It can be installed on any ByteBlower server when you have a valid license for it.

Basic Workflow

Setting up a Wireless Endpoint is identical on all supported devices.

Start the Wireless Endpoint

Enter the IP address of your Meeting Point. This is required the first time only. Next time, the app will fill in the previously used address by default.

Push the button to go to the next page to connect with the Meeting Point.
The Wireless Endpoint connects with the Meeting Point, and reaches the "Registered" state.

States

On top of the screen, the Wireless Endpoint displays its current state. There are five possible states

  • Ready - waiting until you enter the IP address of the Meeting Point.
  • Contacting - establishing a connection with the Meeting Point.
  • Registered - initial handshake with the Meeting Point succeeded. From now on, the Wireless Endpoint can be controlled using your GUI. The Wireless Endpoint starts sending heartbeat messages to the Meeting Point, to signal that it is still alive.
  • Armed - when you run a test scenario using the GUI, the entire set of instructions for this device is transmitted at the beginning. When all instructions are received, the device goes into the Armed state. Now, the Wireless Endpoint becomes quiet, and will no longer send heartbeat messages, so that the test traffic is not disturbed.
  • Running - when the start time has come, the Wireless Endpoint will begin sending/receiving network traffic.

At the bottom of the screen, there is an Abort button. When you push this button, the Wireless Endpoint will go back from the Registered to the Ready state. When in the Armed or Running, the Abort button is disabled.

For more information, have a look at this article: Wireless Endpoint States

GUI

From now on, you can sit back and control the entire test process using your GUI.
All you need to do is adding the Meeting Point in the Server View.


All connected Wireless Endpoints become visible.


Now you can dock ByteBlower Ports on your Wireless Endpoints to integrate them into your test scenarios.

Running Test Scenarios

When you start a test scenario using Wireless Endpoints, the Meeting Point will automatically initialize all Wireless Endpoints.
All Wireless Endpoints go into the "Armed" State.


While the actual test is running, there is complete radio silence between the Meeting Point and the Wireless Endpoints.
This way, the network test traffic itself is not influenced by any unwanted signals.
When you look at the Wireless Endpoint while a test is running, you can see the Current Speed.


After the test finishes, the Meeting Point gathers all results from the Wireless Endpoints.
And then the report is generated.

Have a great time using our Wireless Endpoints !

This article collects a number of example BPF filters. The filters are grouped by the OSI layer.

These BPF filters are used throughout the ByteBlower API. Both for counting traffic, as for selecting the traffic to save in PCAP capture.

Layer 3 Network: IP, IP6, ARP, ...

Example Description
ip Captures all TCP traffic regardless of lower transport layers
ip6 Selects all IP6 traffic. This is not a default filter but has been added to ByteBlower.
arp Captures only messages from the adress resolution protocol.
icmp All ICMP traffic of which ping is the most known application.
igmp IPv4 Multicast traffic

Layer 4 Transport: TCP and UDP

This is the transport layer, ByteBlower supports the two major families: TCP and UDP. Both protocols can be used over IPv4 or IPv6. When  the network layer isn't explicitly mentioned in the filter all traffic from both is captured.

In the examples below we'll use tcp by default, but the last 3 example filters can also be used with udp.

Example Description
tcp Captures all TCP traffic regardless of lower layers transport layers
udp Same as above, but for UDP
ip and tcp Capture only traffic over IP
ip6 and tcp Selects for ip6 and TCP. This not a default BPF filter.
(ip or ip6) and tcp Same behavior as the first filter. This line is only included as an example, prefer to use the first one.

Filter on ports

Both TCP and UDP use ports. In the examples below, the filters are shown for both protocols.

TCP UDP Description
tcp port 80 udp port 80 Captures all traffic from or to port 80. Both directions are collected.
This mainly useful for TCP.
tcp src port 80 udp src port 80 Capture only traffic transmitted from port 80.
tcp dst port 80 udp dst port 80 The inverse of the one above, captures all traffic with port 80 as destination

Filter on TCP flags

The filters below are TCP specific. They filter on specific flags. This makes the following filters very useful to capture only the start of traffic, the end or any abnormal behavior.

This type of filters use the array operators of BPF. The filter tcp[13:1] fetches a single byte at offset 13; i.e. the fourteenth byte of the TCP header. This can be written even easier using the tcpflags shorthand. That is used most heavily in the filters below.

Other shorthands are those for those to select the individual bits of the tcp-flags: tcp-fin (= 0x01),

  • tcp-fin, with value 0x01
  • tcp-syn, 0x02
  • tcp-rst, 0x04
  • tcp-push, 0x08
  • tcp-ack, 0x10
  • tcp-urg, 0x20

The BPF syntax has no such definitions for the slightly newer TCP flags: ECE (0x40), CWR (0x80) and NS (0x100). These flags are related Congestion Notification (rfc3168, rfc3540). Especially this last one poses a challenge, it's the final example in the list below.

TCP Description
tcp[tcpflags] == tcp-syn Selects the initial  SYN packet. This is the initiation of the session by the TCP client.
tcp[tcpflags] == (tcp-syn + tcp-ack) This SYN+ACK response from the TCP server.
(tcp[tcpflags] & tcp-syn) > 0 All packets with the SYN flag enabled. This filter matches the traffic of the above two.
This filter uses the bitwise and operator ('&') to select the the Syn bit out of the TCP flags.
(tcp[tcpflags] & tcp-fin) > 0 All packets with a Fin flag. Similar to the filter directly above, this also selects packets with Fin + Ack.
tcp[tcpflags] == tcp-rst Selects all TCP Resets
(tcp[tcpflags] & tcp-fin) > 0 All packets with a Fin flag.
(tcp[tcpflags] & 0x40)  > 0 This filters captures all packets with the ECE flag enabled. Both the initial negation is thus captured, as well as a TCP session echoing congestion on the network.
(tcp[tcpflags] & (0x40 + tcp-syn))  == (0x40 + tcp-syn) Like the above, this filter selects packets with ECE flag enabled. Extra in this filter is to only select the initial handshake. This is done by adding the tcp-syn value.
In other words, this filters all session that capable to react to Explicit Congestion Notifications.
(tcp[tcpflags] & (0x40 + tcp-syn))  == 0x40 The filter is the inverse of the above. It also packets with the ECE flags but also requires for these packets to have the tcp-syn flag disabled.
This filters thus on actual cases where the session reactes to an explicit congestion notification.
(tcp[12:2] & 0x100) > 0 As promised, in the last filter the NS flag, or ECN-nonce is used.
This flags is experimental, and makes the filter much complex.

The NS flag has value 0x100, this larger the usual single byte for the TCP flags.

As a result, 12:2 is used instead of tcpflags. This expression selects 2 bytes at an offset of 12 bytes in the TCP header.

The remainder of this BPF filter is similar to the above.

GitHub Projects - Frame Blasting Modifiers

Following the previous article (Insert Link), we are going to explore some modifications to the frame and frameblasting flows. This will enable us to utilize more of the GUI's features; in particular the time and frame-size modifiers. These features create variations in the throughput which simulate non-uniform traffic flows.

 

We will look at 3 scenarios: 

  • Growing frame size - Frame increases size up to a maximum.

  • Random size

  • Bursty traffic - Frames sent in bursts  (1 second intervals).   

  

Each one of the tests will run vs a default frame for comparison purposes.

  

Let's go!

                                                                                     

 

Part 1. Docking the Ports

When you open the .bbp file you will see the following displayed:

 

  • 2 ports configured with DHCP

  • Port 2 is NAT enabled

 

1.1 Connect the Ports to the Trunk Interface.

 

Port 1 and Port 2 are set are configured for DHCP so they should be docked onto trunk interfaces compatible with DHCP.

You will see that Port_2 has been NAT enabled but this is not strictly necessary.

 

 

  • In this example, trunk 1-3 and 1-4 are such interfaces.

 

 

 

Part 2. Frame Modifiers

 

In this part we will look at the different frames and frame blasting flows with their modifications. Frames can be modified in terms of their size (number of bytes) as well as adding a modifier at the level 4 layer. This can mean adding a payload offset. Another feature is changing the UDP source and destination ports at the level 4 layer (see below).

2.1 Frames

 

  • Two frames have been created with the default size of 1024 bytes. 

  • Going to the layer 4 setting we can see that Frame_1 and Frame_2 have different source and destination ports.

 

 

 

2.2 Frame Blasting Flows

 

In this step, we will look at the different frame blasting flows and how they transmit the data frames. The settings of the 'Frame Blasting' tab enable the user to adjust the thoughput speed as well as change the uniformity of the throughput. A frame blasting flow will take the frame defined in the previous step and repeatedly 'blast' it from source to destination.

 

In this example:

 

  • The growing size frame will produce a sawtooth throughput.

  • The random size frame modifier will generate random frame sizes between a minimum (60 bytes) and a maximum (1514 bytes).

  • The bursty traffic frame uses a time frame modifier to produce bursts of traffic containing 5000 frames with an interburst gap of 1.2 seconds.

  

 

  

2.3 Flow

 

The flow is the traffic that will be sent by the ByteBlower server. Here we can define the source and destination of the generated traffic.

 

  • Select the source and destination ports. 

  • In this demonstration we will send the frames form Port 1 → Port 2 which are connected back-to-back.

  • A latency distribution has been included.