Performance Study on Virtual Machine Hypervisors

Xen’s report (1)Comparison of 3 popular VMMs: Xen, VMware and UML

Benchmark: The SPEC CPU suite contains a series of long-running

computationally-intensive applications intended to measure the performance of a system's processor, memory system, and compiler quality.

A full build of the default configuration of Linux 2.4.22 on local disk PostgreSQL running the OSDB multi-user Information Retrieval (IR)

benchmark PostgreSQL running the OSDB multi-user On-Line Transaction

Processing (OLTP) benchmark requires many synchronous disk operations, resulting in many protection domain transitions.

The dbench 2.0 file system single user benchmark, It emulates the load placed on a file server by Windows 95 clients.

SPEC WEB99 is a complex application-level benchmark for evaluating Web servers and the systems that host them

Xen’s report (2)

native Linux (L) Xen/Linux (X) VMware Workstation 3.2 (V) User Mode Linux (U)

Linux Build time SPEC INT 2000 OSDB-IR OSDB-OLTP DBench SPEC WEB99

VMware’ report (1)2 VMMs: VMware ESX Server 3.0.1 and Xen 3.0.3

Benchmark The integer component of the SPECcpu2000 benchmark suite,

available from SPEC® (Standard Performance Evaluation Corporation), was chosen to represent CPU-intensive applications.

Passmark, a synthetic suite of benchmarks intended to isolate various aspects of workstation performance, was selected to represent desktop-oriented workloads.

Netperf was used to simulate the network usage in a datacenter. The SPECjbb2005 benchmark suite from SPEC was used to

represent the Java applications typically used in the datacenters. A compile workload — build SPECcpu2000 INT package — was also

added to capture typical IT development and test usage in datacenters.

VMware’s report (2): Spec2000 INT

VMware’s report (3): Passmark

VMware’s report (4): Compile

VMware’s report (5): Netperf

VMware’s report (6): Specjob 2005

DiscussionWhy different benchmarks lead different results?

Why multiple virtual machine products lead different results?

Performance is really important?

What else could evaluate VMM?