Gelato@UNSW Benchmark Box
Contents
cassata is the benchmark box (only available internally)
Sysadmin details
login benchmark password benchmark
Local profile should be sourced from /home/benchmark/environment.sh; this should set up everything required (benchmark user profile does this by default)
System
The system is based on CentOS 4 which is more or less RHEL4
Installing software
The easiest way is with yum
$ yum install package
to install a locally downloaded RPM
$ yum localinstall ./file.rpm
Installing benchmarks
Feel free to install your own benchmarks.
The idea is that we can (pretty much) tar up /home/benchmark and move it anywhere, so please try and keep it self-contained. That means don't put things in /usr/local etc if you don't have to, and use the ~/environment.sh file to do any setup you require.
Please update this wiki page with basic instructions on how to run the benchmark.
Kernels
Place your kernel for network boot in paulaner:/tftpboot/gelato/cassata/vmlinux-description and modify paulaner://tftpboot/gelato/cassata.conf to point to it (follow the instructions).
Note the system has been modified to work without and initrd.
Using zx1_defconfig should "just work"; modules are generally a pain and best avoided
Compilers
The intel compiler is installed as icc, the intel fortran compiler as ifort
source/install files
src/install files for most everything not included with the system is located in ~benchmark/src
local space
Please create a /usr/src/username directory for yourself, and keep anything local in there. This is not backed up, and although it shouldn't disappear, don't rely on that (this box does run a lot of experimental kernels where anything could happen!)
Benchmarks and instructions
volano mark
We have the JRockit 5.0 R26 JDK installed locally. To run VolanoMark with this JVM
$ cd vmark2.5.0.9 $ ./loopserver.sh gelato & $ ./loopclient.sh gelato
contest
contest is a kernel compile benchmark which provides a pretty good view of overall system performance.
It can run with a number of different "jobs" in the background, which do things like hit disk or create memory pressure.
Our version is slightly modified to work correctly with IA64 kernels.
To run it
$ cd contest/linux-2.6.18
Firstly, remove any old log files with rm *.log
The quickest way to build is probably just a simulator build, which can be setup via make sim_defconfig.
You can then run contest with the repeats and background tasks, e.g.
$ contest -n 1 noload
makes for a nifty straight kernel compile time output. Contest can also read the log files it outputs to give you a summary with -r, e.g.
[benchmark@cassata linux-2.6.18]$ contest -r no_load: Kernel [runs] Time CPU% Loads LCPU% Ratio 2.6.18 2 212 190.6 0.0 0.0 1.00
Note CPU is for SMP, so divide by CPU's
lmbench
lmbench 2 and 3 are installed
dbench
dbench is used by contest; add note on standalone
reaim 7
reaim7 is installed, with some local patches (beware: paths are hard-coded into the binaries)
reaim should be run in a private directory, e.g.
$ cd /usr/src/ianw/reaim
The general way to run it is
$ reaim -g -l /usr/src/ianw/reaim/reaim-`uname -r` -c ~/share/reaim/reaim.config -f ~/share/reaim/workfile.compute
-g runs until the "cross-over point"; where creating more jobs is showing decreasing performance
-l puts a "prefix" on the output files -- this should include the path
-c is the config. This uses /tmp for disk space; if you are benchmarking disks you will need to change this
-f is the workfile. There are a number in the ~/share/reaim directory aimed at different things, from compute, multiuser, database and disk. Each tweaks the way the test performs.
results
AIM generally works by increasing the number of worker tasks steadily. As the system scales, the number of jobs per minute increases as more tasks are added. The system will peak; a perfectly behaved system would sustain the peak load as even more tasks are added, but in reality contention for resources kicks in and performance declines.
Running with -g uses a simple algorithm to pick this peak point.
For a given number of tasks, a higher jobs per minute is better.
TODO: scripts to graph results, etc.
SPEC CPU2000
The SPEC CPU2000 binaries have been built by IanWienand with the Intel 9.1 compiler. They are fairly brute force and do not use PGO, etc, but should be sufficient for systems work.
The gelato config file is /home/benchmark/SPECcpu2000/configs/gelato-intel
To run a SPEC test, firstly get the spec environment
$ [benchmark@cassata ~]$ cd SPECcpu2000/ $ [benchmark@cassata SPECcpu2000]$ . ./shrc
You can cleanup old run directories with
$ rm -Rf ./benchspec/C*/*/run $ rm -rf ./results
Running under spec
$ runspec --config=gelato-intel --size=test --noreportable --iterations=1 testname
--size=test uses only a small test input to make things fast (useful for testing it works); --size=ref should be used if you want a result
testname can be one of the tests, or int for integer tests, fp for floating point, or all for everything
--iterations saves saves time
- config file automatically turns off base/peak numbers
Results live in results subdir -- others may clear these out on you so make sure you copy out important results
Please don't use the runspec --action=nuke or other things that destroy the binaries (though they should rebuild ok)
Running single tests
Firstly, clear out old runs and things with the rm commands above
Then do a run of the test you wish to do by hand as per the previous section
Then you can do
$ go testname
to get to the test directory (e.g. go gcc). Then go to the run subdirectory, and pick the latest number, e.g.
[benchmark@cassata 176.gcc]$ cd run [benchmark@cassata run]$ ls 00000001 list [benchmark@cassata run]$ cd 00000001/
Now you can run specinvoke -n and spec will show you what it runs for the test, e.g.
[benchmark@cassata 00000001]$ specinvoke -n # Use another -n on the command line to see chdir commands # Starting run for user #0 ../00000001/cc1_base.intel91-linux 166.i -o 166.s > 166.out 2>> 166.err # Starting run for user #0 ../00000001/cc1_base.intel91-linux 200.i -o 200.s > 200.out 2>> 200.err # Starting run for user #0 ../00000001/cc1_base.intel91-linux expr.i -o expr.s > expr.out 2>> expr.err # Starting run for user #0 ../00000001/cc1_base.intel91-linux integrate.i -o integrate.s > integrate.out 2>> integrate.err # Starting run for user #0 ../00000001/cc1_base.intel91-linux scilab.i -o scilab.s > scilab.out 2>> scilab.err
You can now run that by hand, possibly under caliper etc.
OSDB
OSDB stresses a database.
OSDB seems to be a bit flakey, for example the mysql test fails with SQL errors and the postgres test needs to be compiled with gcc4 otherwise it tends to segfault (i think this is a linker bug, but didn't track it down).
The postgres database is setup sufficiently to run the test.
$ osdb-pg --datadir ~/osdb/data-40mb
You can created different data sets (see --generate-files, or use the smaller 4mb data set as well. AFAIK faster is better.
orabm
orabm benchmarks Oracle which is installed on the box. It has a separate information page - orabm
Performance Monitoring
HP Caliper
HP Caliper is installed in /opt/hp-caliper; it is available in the default environment as caliper
See http://www.hp.com/go/caliper for more information
