LMbench

How to install LMbench

  1. Download LMbench from http://www.bitmover.com/lmbench/lmbench3.tar.gz

  2. Install LMbench

1
2
3
# tar -xvzf lmbench3.tar.gz
# cd lmbench3
# make

Note: You may get the following error:

1
2
3
4
5
6
7
8
gmake[2]: *** No rule to make target `../SCCS/s.ChangeSet', needed by `bk.ver'.  Stop.
gmake[2]: Leaving directory `/root/lmbench3/src'
make[1]: *** [lmbench] Error 2
make[1]: Leaving directory `/root/lmbench3/src'
make: *** [build] Error 2
Just ignore the errors, and there is also a method to avoid the errors:
# mkdir SCCS
# touch SCCS/s.ChangeSet

What is LMbench?

  • Suite of simple, portable benchmarks
  • Compares different systems performance
  • Results available for most major vendors (SUN, HP, IBM, DEC, SGI, PCs including 200 Mhz P6’s)
  • Free software, covered by the GNU General Public License.
  • Bandwidth benchmarks
    • Cached file read
    • Memory copy (bcopy)
    • Memory read
    • Memory write
    • Pipe
    • TCP
  • Latency benchmarks
    • Context switching.
    • Networking: connection establishment, pipe, TCP, UDP, and RPC hot potato
    • File system creates and deletes.
    • Process creation.
    • Signal handling
    • System call overhead
    • Memory read latency
  • Miscellanious
    • Processor clock rate calculation

Why do I want yet another benchmark?

Things that other people seem to like about LMbench include the following:

  • Portable test of operating system primitives
    The benchmarks are all in C, fairly portable (although they do prefer to be compiled with GCC). This is useful for generating a set of apples to apples systems comparisons.
  • Keeping up with the Jones
    LMbench is very useful for motivating action. When confronted with numbers that prove that BloatOS is 4 times slower than all of the competition, resources tend to get allocated to fix the problems.
  • Database of results
    The database of results includes runs from most all of the major computer workstation manufactors.
  • Memory latency results
    The memory latency test shows the latency of all of the system (data) caches, i.e., level 1, 2, and 3, if present, as well as main memory and TLB miss latency. In addition the sizes of the caches can be read off of a properly plotted set of results. The hardware folks like this. This benchmark has found bugs in operating system page coloring schemes.
  • Context switching results
    Everybody seems to love context switching numbers. This particular benchmark is quite careful not to just quote the ``in cache’’ numbers. It varies both the number and size of the procesess and plots the results in such a way that it is easy to see when you don’t fit in the cache. You can also see the real costs of a cold cache context switch.
  • Regression testing
    Sun & SGI have used these benchmarks to find and fix performance problems.
    Intel used them during P6 development.
    Linus uses them to do performance tuning of Linux.
  • New benchmarks
    The source is small, readable, and easy to extend. It is routinely massaged into different form to measure something else. For example, the networking metrics include libraries to handle connection establishment, server shutdowns, etc.

How do I run LMbench?

You will need:

  1. the benchmark sources, see the section before this.
  2. a computer with at probably 16MB (or more) memory.
  3. disk space for the sources and binaries: about 2MB.
  4. disk space for working files during the benchmark: about 16MB.

After you finished install LMbench:

1
2
# cd lmbench3
# make results

Note: make results will let you do some choose, for xen memory performance testing, you should take care the following items(Other options you can just press the enter key to use the default value):

  • ‘MB [default 693]’ - You should choose a proper value for ‘MB [default 693]’

  • ‘Processor mhz [default 2748 MHz, 0.3639 nanosec clock]’ - The default value for ‘Processor mhz’ maybe wrong, you need to input the correct value according to your cpu frequency.

  • ‘Mail results [default yes]’ - We do not need to email the results, so just set no

More details see Configure your test section.

The benchmark takes about a half hour (or more on slow machines) to run. When the test finished, run make see to check the results.

The benchmark binaries will be in bin/your_os_name.

The results will be in results/your_os_name/your_host_name.

Go to the results and look at the Makefile for more information.

Results analysis

Run make see, you will find the following lines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
......
*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------------------------
Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                             UNIX      reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
dhcp-66-7 Linux 3.10.0- 2191 2348 1572 4079.6 5207.5 4187.6 3172.6 3510 5083.
dhcp-66-7 Linux 3.10.0- 2188 2365 1645 3753.9 5164.9 4172.5 3167.3 3421 5492.
dhcp-66-7 Linux 3.10.0- 2182 2344 1555 2862.8 5900.7 3846.8 3140.0 3403 4958.

Memory latencies in nanoseconds - smaller is better
    (WARNING - may not be correct, check graphs)
------------------------------------------------------------------------------
Host                 OS   Mhz   L1 $   L2 $    Main mem    Rand mem    Guesses
--------- -------------   ---   ----   ----    --------    --------    -------
dhcp-66-7 Linux 3.10.0-  2393 1.5980 4.1780   40.6       108.2
dhcp-66-7 Linux 3.10.0-  1579 1.5970 4.1910   33.9       109.4
dhcp-66-7 Linux 3.10.0-  1710 1.5950 4.1590   37.2       125.7
......

It include Memory copy (bcopy), Memory read, Memory write bandwidth(MB/s), the bigger the better.
Also it include Memory read latency, the smaller the better.

The results is also saved in results/your_os_name/your_host_name, you can check it and do some analysis based on it.

Configure your test

When you run make results, LMbench will let you do some choose to configure your test. Here are each items mean and some suggest.

  • MULTIPLE COPIES [default 1]

If you are running on a SMP machine, LMbench will allow to run multiple copies of lmbench in parallel, choose the default value(1) should be OK.

  • Job placement selection:

You must choose a This item allow to choose LMbench how to schedule the test. If you don’t know what the seven option mean, just choose 1).

  • MB [default 695]

Several benchmarks operate on a range of memory. This memory should be sized such that it is at least 4 times as big as the external cache[s] on your system. It should be no more than 80% of your physical memory.

  • SUBSET (ALL|HARWARE|OS|DEVELOPMENT) [default all]

LMbench measures a wide variety of system performance, and the full suite of benchmarks can take a long time on some platforms. Consequently, LMbench offer the capability to run only predefined subsets of benchmarks, one for operating system specific benchmarks and one for hardware specific benchmarks. LMbench also offer the option of running only selected benchmarks which is useful during operating system development. If you don’t know which to choose, just choose the default value(ALL).

  • FASTMEM [default no]

This benchmark measures, by default, memory latency for a number of different strides. That can take a long time and is most useful if you are trying to figure out your cache line size or if your cache line size is greater than 128 bytes. For xen performance testing, i suggest to choose the default value(no).

  • SLOWFS [default no]

This benchmark measures, by default, file system latency. That can take a long time on systems with old style file systems (i.e., UFS, FFS, etc.). Linux’ ext2fs and Sun’s tmpfs are fast enough that this test is not painful. For xen performance testing, i suggest to choose the default value(no).

  • DISKS [default none]

This option is used to determine whether to test disk performance. If you want to skip the disk tests, hit return below. If you want to include disk tests, then specify the path to the disk device, such as /dev/sda. For xen testing, we only use LMbench to test guest’s memory performance, no disk testing include, so choose the default value(none).

  • REMOTE [default none]

This option is used to determine whether to test network performance. If you want to skip the network tests, hit return below. If you want to include network tests, then specify the ip address of other host. For xen testing, we only use LMbench to test guest’s memory performance, no network testing include, so choose the default value(none).

  • Processor mhz [default 1579 MHz, 0.6333 nanosec clock]

The default value for ‘Processor mhz’ maybe wrong, you need to input the correct value according to your cpu frequency.

  • FSDIR [default /usr/tmp]

The directory to store a test file as well as create and delete a large number of small files. For xen testing, just choose the default value(/usr/tmp/).

  • Status output file [default /dev/tty]

LMbench outputs status information as it runs various benchmarks. For xen testing, just choose the default value(/dev/tty).

  • Mail results [default yes]

There is a database of benchmark results that is shipped with new releases of lmbench. Your results can be included in the database if you wish. For xen testing, there is no need to send the results, so set the value to no.

Reference

Technology