1 kansas city cmg, 2005 ethan bolker and yiping ding october, 2005 virtual performance won't do...

1 Kansas City CMG, 2005

Ethan Bolker and Yiping DingOctober, 2005

Virtual performance won't Virtual performance won't dodo: : Capacity planning for virtual systemsCapacity planning for virtual systems

2Kansas City CMG 05

Everything we think we know is virtualEverything we think we know is virtual

What we can measure and verify is limited

3Kansas City CMG 05

What Analysts are Saying?What Analysts are Saying?

“Enterprises that do not leverage virtualization technologies will pay up to 40 percent more in acquisition costs by 2008, and roughly 20 percent more in administrative costs than enterprises that leverage virtualization technologies”

T. Bittman, Gartner Research – July 2003

What we see is not Reality.It is our Perception.

Virtualization is the Perception.Perception is the Business.

4Kansas City CMG 05

The History of Computing Is

A History of Virtualization

5Kansas City CMG 05

What is the “Performance” Model for that?What is the “Performance” Model for that?

1

S

1Throughput

Service time

Utilization = Throughput x Service timeUtilization: % of system busy

6Kansas City CMG 05

One job may take days to complete …One job may take days to complete …

1

Days

Throughput

Service time

Utilization = Throughput x Service timeU = 1 job / 30 days x 3 days / job = 10%Response time almost equals service time

7Kansas City CMG 05

Multiprogramming: nontrivial performance modeling

x

R

xMultics

Utilization = Throughput x Service timeU = 8 job / 1 min. x 0.1 min / job = 80%

Throughput

Response time = service time / (1 – U) = 0.1 / 0.2 = 0.5 min

8Kansas City CMG 05

Advanced ChipsAdvanced Chips

x

R

x

9Kansas City CMG 05

A Basic Computer System without VirtualizationA Basic Computer System without Virtualization

Applications Operating System

Processors

NetworkInterface

I/O Subsystem

Memory

Hardware

Sum of app utilization = processor utilizationCapture Ratio: SUM of U(Ai) / U(P)

U(Ai)

U(P)

10Kansas City CMG 05

Memory Processors

Hardware

Operating System

I/O Subsystem Network Interface

Memory Processors

Hardware


Virtualized Layer

OS

Applications

OS

Applications

App

licat

ion

A Virtualized System


Three Basic Architectures of VirtualizationThree Basic Architectures of Virtualization

Virtualization Layer below the OS Virtualization Layer above the OS (Virtualization Layer both below and above the OS)


Examples of Virtualization ProductsExamples of Virtualization Products

Vendor Below / Above OS

Hyper-threaded

Processor Intel Below

VMware ESX Server VMware (EMC) Below

VMware GSX Server VMware (EMC) Above

Virtual MachineTechnology

Microsoft Above

AIX Micropartition IBM Below

Sun N1 SUN Above and Below

nPar, vPar HP Below

PR/SM IBM Below


VMware ESX Server: V Layer is below OSVMware ESX Server: V Layer is below OS


VMware GSX Server: VMware GSX Server: V Layer is both above and below the OSV Layer is both above and below the OS


Microsoft Virtual Machine TechnologyMicrosoft Virtual Machine Technology

Virtual Machine Operating System and Applications


IA-32 Server

Windows Server 2003

Virtual Server 2005

Virtual Hardware Virtual Hardware

Virtual Server is above and below the OS, like VMWare GSX


Model with Virtual Layer above OSModel with Virtual Layer above OS

When Virtual Layer is above an OS, Virtual Layer is an extension of the OS



IA-32 Server

Windows Server 2003

Virtual Server 2005

Virtual Hardware Virtual Hardware

A New OS with additional measurements

Any HW presented by an OS is virtual HW

Applicationsrunning on the new OS with OS like measurements.(ie, treat it as an application.)


Model with V Layer below OS

•When V Layer is below OS, V Layer serves as a new OS

A new OS withnew Measurements

An application with OS like measurements


Virtualization layer below operating systemVirtualization layer below operating system

Memory Processors

Hardware

Operating System

Applications


Memory Processors

Hardware


Virtualized Layer

New OS

New App


Sun N1: Virtualization above and below OS

OS

SeparatelyAdministrableSolaris Instances


Definitions of utilizationDefinitions of utilization

Vj(P) = (virtual) processor utilization measured by guest j = sum( Vj(Ai) )

Uj(P) = real processor utilization charged to guest j by the virtualization manager

U0(P) = real processor utilization the virtualization manager uses to do its workCentral question: does Vj(P) = Uj(P) ?


Guests and Virtualization ManagerGuests and Virtualization Manager

Each guest runs its own OS The OS could be different for different guests Each guest knows nothings about the others Performance data is collected at each guest Only Virtualization Manager knows all and keeps track of the resource consumption of each virtual machine on the physical system The Virtualization Manager schedule access to real physical resources to support each guest


Stretch-out!Stretch-out!

We expect to see Vj(P) > Uj(P)… the fraction of time the guest run queue is not empty is larger than the time charged to guest j by the manager, which may award the CPU cycles to some other guest, forcing guest j to wait even while it thinks it is processing work


Two virtual machines on one physical systemTwo virtual machines on one physical system

Memory

Processors

Hardware

Operating System

Applications


Memory Processors, V1(P)

Hardware


Virtualized Layer

Hardware

Operating System

Applications

Memory Processors, V2(P)


Virtualized Layer

Virtualization Manager

V2(P) = sum V2(Ai)

U(P) = sum Uj(P)

V1(P) = sum V1(Ai)


sum Uj(P) = U(P)


How to Allocate Resource among GuestsHow to Allocate Resource among Guests

Each guest consumes as much of the processing power as it wishes, provided U(P) < 1(no shares assigned) Assign each guest a (fraction) share f(j), which is interpreted as either a cap or a guarantee

When shares are caps, each guest owns its fraction of processing power, but no more than that

When shares are guarantees, each guest could consume more than its share when other guests are idle

In each of the three cases, we must know how to interpret the measurements in each guest


Easiest Case to Understand: Shares as CapsEasiest Case to Understand: Shares as Caps

Each guest is unaffected by other guests The relationship between the guest Virtual utilization and the guest physical utilization is simple: Vj(P) = Uj(P) / f(j) Vj(P) approaches 100% as Uj(P) approaches the cap f(j)


No Shares AssignedNo Shares Assigned

Vi(P) = Ui(P) / [1 – (U(P) – Ui(P))]

Where Ui(P) ≤ U(P) ≤ 1 Ui(P) ≤ Vi(P) ≤ 1

If Ui(P) = U(P) then processor is busy for guest i only (and there is no overhead), thus Vi(P) = Ui(P)

Vi(P) is usually greater than Ui(P)

Each guest consumes as much of the processing power as it wishes, provided U(P) < 1


0

0.10.2

0.30.4

0.5

0.60.7

0.80.9

1

1 2 3 4 5 6 7 8

Util

izat

ion

U Ui Vi

No Shares AssignedNo Shares Assigned

Vi(P) = Ui(P) / [1 – (U(P) – Ui(P))]


Shares as GuaranteesShares as Guarantees

When shares are guarantees, each guest can consume more than its share when other guests are idle

Vi = F(f1, …, fn, U1,…, Un)

Virtual utilization depends on share andutilization of each guest


Processors,

Hardware

Windows 2000

Guest 1: Bermuda

Processors

Hardware

Virtualized Layer

Hardware

Windows 2000

Guest 2: Largo

Virtualized Layer

VMware experimentsVMware experiments


U(Bermuda) = 25% U(Largo) = 20% - 40%

U(P) = 45% - 65%

V(Bermuda) = ?

Processors

V(Largo) = ?


Processors, U(P)

Hardware

Windows 2000

Guest 1: Bermuda

Hardware

Virtualized Layer

Hardware

Windows 2000

Guest 2: Largo

Virtualized Layer

VMware experiments: Guest BermudaVMware experiments: Guest Bermuda


U(Largo) = 20% - 40%

Processors, V(Largo)

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput

utilization (guest view)

utilization (manager view)

response time

total utilization (manager view)

V(Bermuda)

Processors

U(Bermuda) = 25%


Processors

Hardware

Windows 2000

Guest 1: Bermuda

Processors, V(Bermuda)

Hardware

Virtualized Layer

Hardware

Windows 2000

Guest 2: Largo

Virtualized Layer

VMware experiments: Guest LargoVMware experiments: Guest Largo


U(Bermuda) = 25%

Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Processors

V(Largo)

U(Largo) = 20% - 40%


VMware measurementsVMware measurements

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Same total utilization


Observations of VMware experimentsObservations of VMware experiments

Guest’s utilization (Vi) was larger than the utilization attributed to it by the manager (Ui)

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Vi

Ui



The proportional utilization stretching is roughly constant for each guest, but different for the two guests

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time




The response time on each machine depends on the total utilization

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Total utilization

Response Time


Observations of VMware experimentsObservations of VMware experimentsWhen both guests ran at the same (approximate) load (Experiment 2), job response time was essentially the same on each

Bermuda

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time


Largo

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7

throughput



response time



Main points / SummaryMain points / Summary showed processor utilizations measured by the guest and by the virtualization manager need not agree discussed the relationship between those utilization measurements when no shares have been assigned proposed a methodology for computing how activity in one guest can affect the performance in others suggested the value of using throughput rather than utilization as the independent variable when attempting to answer what-if questions about transaction response time


Future WorkFuture Work

find a virtualization system that allows us to specify “no shares” so that we can validate the model introduced in the paper continue our experiments on VMware and other systems in order to understand share allocation semantics develop a reasonably generic methodology for modeling at least the simplest of the share allocation semantics

1 kansas city cmg, 2005 ethan bolker and yiping ding october, 2005 virtual performance won't do...

Documents