netperf

How to install netperf

  1. Download netperf from ftp://ftp.netperf.org/netperf/netperf-2.6.0.tar.gz

  2. Install netperf from source code

1
2
3
4
5
# tar -xvzf netperf-xxxxx
# cd netperf-xxxxx
# ./configure --enable-burst
# make
# make install

Features and Functions

Netperf is designed around a basic client-server model. There are two executables - netperf and netserver. Generally you will only execute the netperf program, with the netserver program being invoked by the remote system’s inetd or having been previously started as its own standalone daemon.

When you execute netperf it will establish a “control connection” to the remote system. This connection will be used to pass test configuration information and results to and from the remote system. Regardless of the type of test to be run, the control connection will be a TCP connection using BSD sockets. The control connection can use either IPv4 or IPv6.

Once the control connection is up and the configuration information has been passed, a separate “data” connection will be opened for the measurement itself using the API’s and protocols appropriate for the specified test. When the test is completed, the data connection will be torn-down and results from the netserver will be passed-back via the control connection and combined with netperf’s result for display to the user.

Netperf places no traffic on the control connection while a test is in progress. Certain TCP options, such as SO_KEEPALIVE, if set as your systems’ default, may put packets out on the control connection while a test is in progress. Generally speaking this will have no effect on the results.

How to use netperf

For xen performance testing, we only test the following type of test:

  • TCP_STREAM - is the default test in netperf. It is quite simple, transferring some quantity of data from the system running netperf to the system running netserver.
  • TCP_RR - A TCP_RR test can be thought-of as a user-space to user-space ping with no think time - it is by default a synchronous, one transaction at a time, request/response test.
  • TCP_CRR - A TCP_CRR test is like a merger of a TCP_RR and TCP_CC test which measures the performance of establishing a connection, exchanging a single request/response transaction, and tearing-down that connection.
  • UDP_STREAM - A UDP_STREAM test is similar to a TCP_STREAM test except UDP is used as the transport rather than TCP.
  • UDP_RR - It is very much the same as a TCP_RR test except UDP is used rather than TCP.

Multiple instances of TCP_STREAM

Test scenarios

  1. Host ->External Host
  2. Host -> Guest
  3. Guest -> Host
  4. Guest -> Guest
  5. External Host -> Guest
  6. Guest -> External Host

Note:
A straight attachment link two machine
For secnario 5 & 6 better in 10Gbit

Command

  1. Run netserver on remote host

  2. On local host, run:

e.g instances = 4

1
2
3
4
5
6
7
8
#!/bin/bash
for a in 32 64 128 256 512 1024 1460 2048 4096 8192  9000 16384 32768  65495 65507
do
    for b in 1 2 3 4
    do
        netperf -t TCP_STREAM -f m -H $remote_server_ip -P 0 -l 10 -- -m $a &
    done
done

Multiple instances of UDP_STREAM

Test scenarios

  1. Host ->External Host
  2. Host -> Guest
  3. Guest -> Host
  4. Guest -> Guest
  5. External Host -> Guest
  6. Guest -> External Host

Note:
A straight attachment link two machine
For secnario 5 & 6 better in 10Gbit

Command

  1. Run netserver on remote host

  2. On local host, run:

e.g instances = 4

1
2
3
4
5
6
7
8
#!/bin/bash
for a in 32 64 128 256 512 1024 1460 2048 4096 8192  9000 16384 32768  65495 65507
do
    for b in 1 2 3 4
    do
        netperf -t UDP_STREAM -f m -H $remote_server_ip -P 0 -l 10 -- -m $a &
    done
done

Multiple instances of TCP_RR

Test scenarios

  1. Host ->External Host
  2. Host -> Guest
  3. Guest -> Host
  4. Guest -> Guest
  5. External Host -> Guest
  6. Guest -> External Host

Note:
A straight attachment link two machine
For secnario 5 & 6 better in 10Gbit

Command

  1. Run netserver on remote host

  2. On local host, run:

e.g instances = 1/25/50/100

1
2
3
4
for b in 1 25 50 100
do
    netperf -v 0 -P 0 -t TCP_RR -B ": VM1 -b $b -r 256,256" -H $remote_server_ip -- -r 256,256 -b $b -D
done

Multiple instances of UDP_RR

Test scenarios

  1. Host ->External Host
  2. Host -> Guest
  3. Guest -> Host
  4. Guest -> Guest
  5. External Host -> Guest
  6. Guest -> External Host

Note:
A straight attachment link two machine
For secnario 5 & 6 better in 10Gbit

Command

  1. Run netserver on remote host

  2. On local host, run:

e.g instances = 1/25/50/100

1
2
3
4
for b in 1 25 50 100
do
    netperf -v 0 -P 0 -t UDP_RR -B ": VM1 -b $b -r 256,256" -H $remote_server_ip -- -r 256,256 -b $b -D
done

Multiple instances of TCP_CRR

Test scenarios

  1. Host ->External Host
  2. Host -> Guest
  3. Guest -> Host
  4. Guest -> Guest
  5. External Host -> Guest
  6. Guest -> External Host

Note:
A straight attachment link two machine
For secnario 5 & 6 better in 10Gbit

Command

  1. Run netserver on remote host

  2. On local host, run:

e.g instances = 1/25/50/100

1
2
3
4
for b in 1 25 50 100
do
    netperf -v 0 -P 0 -t TCP_CRR  -H $remote_server_ip -- -r 256,256 -b 1 -D
done

Note: There also some example scripts used to test TCP_STREAM, TCP_RR, UDP_STREAM, UDP_RR under netperf-xxxxx/doc/examples/, you can modify it as necessary.

Results analysis

For TCP_STREAM test

The results will be in the following format:

1
87380  16384     64    10.00     688.1
  • the first field - remote (netserver) send and/or receive socket buffer sizes for the data connection.
  • the second field - local (netperf) send and receive socket buffer sizes for the data connection.
  • the third field - the size of the buffer passed-in to the “send” calls of a _STREAM test.
  • the fourth field - the length of the test (default 10 seconds).
  • the fifth field - Throughput

We should focus on the Throughput, the larger the better.

For UDP_STREAM test

The results will be in the following format:

1
2
212992      64   10.00      659319      0      33.76
212992           10.00      610884             31.28

The first line is the results of local host.
The second is the results of remote server.

  • the first field - remote (netserver) send and/or receive socket buffer sizes for the data connection.
  • the second field - local (netperf) send and receive socket buffer sizes for the data connection.
  • the third field - the size of the buffer passed-in to the “send” calls of a _STREAM test.
  • the fourth field - the length of the test (default 10 seconds).
  • the fifth field - Throughput

We should focus on the Throughput, the larger the better.

For TCP_RR test

The results will be in the following format:

1
31464.49 : VM1 -25  -r 256,256
  • the first field - Transaction rate

We should focus on the Transaction rate, the larger the better.

For TCP_CRR

The results will be in the following format:

1
151.40
  • the first field - Transaction rate

We should focus on the Transaction rate, the larger the better.

For UDP_RR

The results will be in the following format:

1
33748.49 : VM1 -b  -r 256,256
  • the first field - Transaction rate

We should focus on the Transaction rate, the larger the better.

The main parameters

Netperf command format:

  1. sets both global and test-specific options:
    netperf <global> -- <test-specific>

  2. sets just global options and:
    netperf <global>

  3. sets just global options and:
    netperf -- <test-specific>

Global Options

  • -B - This option will cause to be appended to the brief (see -P) output of netperf.
  • -H name|ip - Set the hostname (or IP address) and address family to use to establish the control connection to the remote system.
  • -b - This option is only present when netperf has been configure with –enable-intervals=yes prior to compilation. It sets the size of the burst of send calls in a _STREAM test. When used in conjunction with the -w option it can cause the rate at which data is sent to be “paced.”
  • -i - This option enables the calculation of confidence intervals and sets the minimum and maximum number of iterations to run in attempting to achieve the desired confidence interval.
  • -t testname - Specify the test to perform. As of this writing, valid values for testname include:
    • TCP_STREAM, TCP_MAERTS, TCP_SENDFILE, TCP_RR, TCP_CRR, TCP_CC
    • UDP_STREAM, UDP_RR
    • XTI_TCP_STREAM, XTI_TCP_RR, XTI_TCP_CRR, XTI_TCP_CC
    • XTI_UDP_STREAM, XTI_UDP_RR
    • SCTP_STREAM, SCTP_RR
    • DLCO_STREAM, DLCO_RR, DLCL_STREAM, DLCL_RR
    • LOC_CPU, REM_CPU
    • OMNI
  • -l testlen - Specify the length of the test (default 10 seconds).
  • -4 - Use AF_INET (aka IPv4) addressing for the control and possibly data connections.
  • -6 - Use AF_INET6 (aka IPv6) addressing for the control and possibly data connections.
  • -v verbosity - This option controls how verbose netperf will be in its output, and is often used in conjunction with the -P option.
  • -P 0|1 - A value of “1” for the -P option will enable display of the test banner. A value of “0” will disable display of the test banner.

Test Specific Options(TCP_STREAM, UDP_STREAM)

  • -s - This option sets the local (netperf) send and receive socket buffer sizes for the data connection to the value(s) specified. Often, this will affect the advertised and/or effective TCP or other window, but on some platforms it may not. By default the units are bytes, but suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -S - This option sets the remote (netserver) send and/or receive socket buffer sizes for the data connection to the value(s) specified. Often, this will affect the advertised and/or effective TCP or other window, but on some platforms it may not. By default the units are bytes, but suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -m bytes - Set the size of the buffer passed-in to the “send” calls of a STREAM test. Note that this may have only an indirect effect on the size of the packets sent over the network, and certain Layer 4 protocols do _not preserve or enforce message boundaries, so setting -m for the send size does not necessarily mean the receiver will receive that many bytes at any one time. By default the units are bytes, but suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -M bytes - Set the size of the buffer passed-in to the “recv” calls of a _STREAM test. This will be an upper bound on the number of bytes received per receive call. By default the units are bytes, but suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -D - This option will set TCP_NODELAY on the data connection on those systems where TCP_NODELAY is defined. This disables something known as the Nagle Algorithm, which is intended to make the segments TCP sends as large as reasonably possible.

Test Specific Options(TCP_RR, TCP_CRR, UDP_RR)

  • -r - This option sets the request (first value) and/or response (second value) sizes for an _RR test. By default the units are bytes, but a suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -s - This option sets the local (netperf) send and receive socket buffer sizes for the data connection to the value(s) specified. Often, this will affect the advertised and/or effective TCP or other window, but on some platforms it may not. By default the units are bytes, but a suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -S - This option sets the remote (netserver) send and/or receive socket buffer sizes for the data connection to the value(s) specified. Often, this will affect the advertised and/or effective TCP or other window, but on some platforms it may not. By default the units are bytes, but a suffix of “G,” “M,” or “K” will specify the units to be 2^30 (GB), 2^20 (MB) or 2^10 (KB) respectively. A suffix of “g,” “m” or “k” will specify units of 10^9, 10^6 or 10^3 bytes respectively.
  • -D - This option will set TCP_NODELAY on the data connection on those systems where TCP_NODELAY is defined. This disables something known as the Nagle Algorithm, which is intended to make the segments TCP sends as large as reasonably possible.

Reference

Technology