Introduction

The WAN-LAN performance is one of the primary performance parameters of a Home Gateway. This test will verify the throughput and latency for different WAN and LAN configurations, both in function of the frame size.

First, the maximum throughput performance is determined. This is done for both directions ( up and downs ) seperately. A combination of upstream and downstream is not considered.

Test Setup

Test Configuration

Each trial for each frame size has a duration of 1 minute. For the GUI project, we created different test scenarios with different loads for 4 different frames sizes: 66, 512, 1024 and 1518 bytes.

For the Tcl script, we use our RFC2544 implementation to determine the maximum speed. When this speed is determined, we perform a latency measurement at that speed.

The Home Gateway should be configured with the WAN interface:

• a single, fixed IP address (10.0.0.1/24)
• full duplex, auto-negotiation
• routed forwarding mode
• NAPT function activated
• firewall deactivated

The LAN interfaces must be configured this way:

• a fixed single IP address ( 192.168.1.1/24)
• full duplex, auto-negotiation

By default, the UDP destination port is configured to be port 20005.

ByteBlower Implementation

ByteBlower GUI project

Download

The ByteBlower project file can be downloaded here.

Execution

The test execution through the GUI is done in two fases: we first do a rough estimation. In a second stage, we perform a more fine-tuned test. We can continue this last step until we have a result which has the required resolution. In this example, we will use a 10Mbps precision.

Estimation

To perform an estimation, we can run the 'Estimate' batch from the example project. This will send traffic for each frame size ( 66, 512, 1024 and 1514 ) at 100Mbit up to 1Gbit, increasing with 100Mbit each iteration. It will generate a report for each frame size like this:

So, in this example, the forward performance of the Home Gateway for 512 bytes frames is between 200 and 300 Mbps. We can now create a new batch which performs in the best range for each frame size in each direction:

Fine-tuning

Fine-tuning is done by creating a batch based on the result of the estimation. For our device under test, this batch looks like this:

We can run this test and we will have a report for each frame size, telling us the best performance with 0% of loss, and the latency for such a test, like this:

This report tells us the maximum downstream performance for 66byte frames is 40Mbps. At that point, the device will have an average latency of 522us and a jitter of 180us.

Implementation

The ByteBlower GUI allows the easy creation of scenarios running flows at different speeds. The GUI uses the throughput wizard for this purpose. This wizard can be found at menu:
Wizard > Generic > Throughput

Through this wizard, you can create a scenario running a frame of a certain length at different rates. If we already have an idea of the performance of the device under test, we can directly start in that range and tune the minimum and maximum throughput which will be used. If we can, we can create a first scenario performing a throughput test at different rates which differ a lot, to have a first indication. Depending on the device under test, sending at a high speed and verify the throughput also provides a good ball point throughput.

Once we have a global idea of the throughput, we can use a second scenario for fine-tuning. We can repeat this wizard for different frame sizes and have different scenarios for each frame size. To run an automated test, we can then combine those scenarios in a batch:

Latency

Because we are not only interested in the throughput, but also in the latency at maximum lossless speed, we want to run a latency test too. If you are on a ByteBlower 2100 or better, you can combine a throughput flow with a latency test. Older systems, such as the ByteBlower 1000 series will not be able to run a latency test at high speeds. In that case, a second flow can be added to the throughput test, running at low speed and performing the latency test.

Because latency measurements overwrite the last 8 bytes of a frame, we must make sure that frame are different at a different location in the frame. The throughput wizard in GUI version 1.8.32 and later will generate good frames, olders versions may generate a difference at the end of the frame.

A second important note is that latency measurements can generate bad UDP checksums (see this KB). Our example project uses a UDP checksum of zero to work around this issue.

NAPT

Because our Home Gateway will be performing address and port translation, we must make sure we configure our ports accordingly: