How do I measure performance of a virtual server?

Question

I've got a VPS running Ubuntu. Being a virtual server, I understand that it shares resources with unknown number of other servers, and I'm noticing that it's considerably slower than my desktop machine.

Is there some tool to measure the performance of the virtual machine? I'd be curious to see some approximate measure similar to bogomips, possibly for CPU (operations/sec), memory and disk read/write speed. I'd like to be able to compare those numbers to my desktop machine.

I'm not interested in the specs of the actual physical machine my VPS is running on - by doing cat /proc/cpuinfo I can see that it's a nice quad-core Xeon machine, but it doesn't matter to me. I'm basically interested in how fast a program would run in my VPS - how many CPU operations it can make in a second, how many bytes to write to RAM or to disk.

I only have ssh access to the machine so the tool need to be command-line.

I could write a script which, say, does some calculations in a loop for a second and counts how many loops it was able to do, or something similar to measure disk and RAM performance. But I'm sure something like this already exists.

Solution

Well, since nobody wants to answer... :)

Searching Synaptic for "bench" finds a lot of benchmarking suites capable of testing different aspects of a machine. The only one I heard about previously is phoronix-test-suite, which I'm sure is very comprehensive although my short attention span didn't allow me to figure out how to use it.

Then I found UnixBench, which is described as

UnixBench is the original BYTE UNIX benchmark suite, updated and revised by many people over the years.

The purpose of UnixBench is to provide a basic indicator of the performance of a Unix-like system; ... These test results are then compared to the scores from a baseline system to produce an index value, which is generally easier to handle than the raw scores.

Multi-CPU systems are handled. ... The tests compare Unix systems by comparing their results to a set of scores set by running the code on a benchmark system, which is a SPARCstation 20-61 (rated at 10.0).

UnixBench is mentioned by Linode as a tool for VM performance testing in this blog post:

Using identical hardware, KVM Linodes are much faster compared to Xen. For example, in our UnixBench testing a KVM Linode scored 3x better than a Xen Linode.

The test suite is NOT in Ubuntu repositories, but it is trivial to download and compile it:

wget https://github.com/kdlucas/byte-unixbench/archive/master.zip
unzip ./master.zip
cd ./byte-unixbench-master/UnixBench
./Run

The tests take a while to finish. The output looks like

------------------------------------------------------------------------
Benchmark Run: Mon Oct 15 2012 23:55:22 - 00:23:16
4 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       12015218.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     2214.8 MWIPS (10.1 s, 7 samples)
Execl Throughput                                896.9 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks         58968.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           14578.6 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        422068.2 KBps  (30.0 s, 2 samples)
Pipe Throughput                               70993.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  16001.5 lps   (10.0 s, 7 samples)
Process Creation                               1861.8 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   2525.5 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    737.8 lpm   (60.1 s, 2 samples)
System Call Overhead                         432496.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   12015218.4   1029.6
Double-Precision Whetstone                       55.0       2214.8    402.7
Execl Throughput                                 43.0        896.9    208.6
File Copy 1024 bufsize 2000 maxblocks          3960.0      58968.3    148.9
File Copy 256 bufsize 500 maxblocks            1655.0      14578.6     88.1
File Copy 4096 bufsize 8000 maxblocks          5800.0     422068.2    727.7
Pipe Throughput                               12440.0      70993.3     57.1
Pipe-based Context Switching                   4000.0      16001.5     40.0
Process Creation                                126.0       1861.8    147.8
Shell Scripts (1 concurrent)                     42.4       2525.5    595.6
Shell Scripts (8 concurrent)                      6.0        737.8   1229.7
System Call Overhead                          15000.0     432496.2    288.3
                                                                   ========
System Benchmarks Index Score                                         249.7

------------------------------------------------------------------------
Benchmark Run: Tue Oct 16 2012 00:23:16 - 00:51:20
4 CPUs in system; running 4 parallel copies of tests

Dhrystone 2 using register variables       42619039.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     8274.0 MWIPS (10.4 s, 7 samples)
Execl Throughput                               3398.5 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks         68332.4 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           21462.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        718205.6 KBps  (30.0 s, 2 samples)
Pipe Throughput                              149713.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  61968.3 lps   (10.0 s, 7 samples)
Process Creation                               5321.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   5957.1 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    812.6 lpm   (60.1 s, 2 samples)
System Call Overhead                        1557391.5 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   42619039.2   3652.0
Double-Precision Whetstone                       55.0       8274.0   1504.4
Execl Throughput                                 43.0       3398.5    790.4
File Copy 1024 bufsize 2000 maxblocks          3960.0      68332.4    172.6
File Copy 256 bufsize 500 maxblocks            1655.0      21462.9    129.7
File Copy 4096 bufsize 8000 maxblocks          5800.0     718205.6   1238.3
Pipe Throughput                               12440.0     149713.5    120.3
Pipe-based Context Switching                   4000.0      61968.3    154.9
Process Creation                                126.0       5321.7    422.4
Shell Scripts (1 concurrent)                     42.4       5957.1   1405.0
Shell Scripts (8 concurrent)                      6.0        812.6   1354.3
System Call Overhead                          15000.0    1557391.5   1038.3
                                                                   ========
System Benchmarks Index Score                                         592.5

Which means that the VPS in question has a score of 249.7 for single task and 592.5 for parallel processing.

My desktop machine, while having similar or lower specs to the physical machine my VPS is running on, produced a score of 1409.7 for single task and 5156.3 for parallel processing. Exactly the kind of metric I was looking for.

Another important metric is network speed. I've found a script which downloads test files from different locations and measures download speed. The script can be run with

wget freevps.us/downloads/bench.sh -O - -o /dev/null|bash

(although it probably would be safer to download the script and inspect its contents before running)

To monitor disk I/O latency there is ioping utility which can be installed from Ubuntu repositories:

# ioping . -c 10
4096 bytes from . (ext4 /dev/disk/...): request=1 time=16.4 ms
4096 bytes from . (ext4 /dev/disk/...): request=2 time=16.1 ms
...

Comments

[Sergey]

Ahh, I think you mis-understood me - I'm not talking about the underlying physical machine. I've updated my question.

[Sergey]

Ok, an example: I gzip a 1Gb file on my netbook and measure time it takes to perform the task. Then I gzip the same file on my desktop - turns out my desktop does it 3 times faster than the netbook, so I give them "gzip rating" of 100 and 300, respectively. Then I compress the same file on the VPS and find, say, that at the time of the test it is 1.5 times faster than the netbook but still 2 times slower than the desktop - so it's "gzip factor" of 150. I can devise something like this myself, but surely measuring performance is a common problem

[Jo-Erlend Schinstad]

Measuring performance is fine, as long as you know which computer you're measuring. With a VPS, you don't necessarily know that. It might be moved between lots of completely different computers without your knowledge, processors might be added and removed, RAM added and removed, etc. Or another VPS starts using lots of CPU for 25 seconds, making your VPS slower. There's too many variables.

[justingrif]

Sounds like there may be a niche for a vps performance monitor tool.

[Jo-Erlend Schinstad]

First you must come up with some sort of definition about what a VPS is. And one single VPS might be different from one moment to another, and it might not be possible for you to know that. If you cannot provide the question, then it is impossible to provide an answer.

[m0j0]

UnixBench is now on Github: github.com/kdlucas/byte-unixbench

[Sergey]

@m0j0: I've modified the answer to reflect that it's hosted at GitHub now, thanks

[Hi-Angel]

These 2 commands wget https://github.com/kdlucas/byte-unixbench/archive/master.zip && unzip ./master.zip could be replaced with just git clone https://github.com/kdlucas/byte-unixbench.

[Lenne]

VPS are sharing the CPU(s) with other VPS. You don't know how many. You might start out being the only client on a new server, 3 months later you might at one time be fighting for cycles with the others, at another time they may all be idling. You might be allocated many cycles for the first minute, after that, you may be reduced so you don't hog all the ressources. (Yes, I know this is an old question, but it's still relevant.)

[abulka]

I also had to sudo apt install make gcc