effect of virtualization on os interference

IBM Research

(c) 2005 IBM Corporation1 The Effect of Virtualization on OS Interference 10/10/05

PROSE Approach

Kernel <-> Hypervisor Interface

Logical Partition

Hardware Platform

Hypervisor

Logical Partition Logical Partition Logical Partition

Hardware <-> Hypervisor Interface

DB2lib OS lib OS

GUPS

9P9P

Con

trol

ler

Con

trol

ler

App 9p

Run applications in stand-alone partition

Enable execution environment which makes starting a partition as easy as starting an application

Development environment allowing creation of specialized kernels as easy as developing an application (library-OS)

Resource sharing between library-OS partitions and traditional partitions keeping library-OS kernels simple and reliable

Extensions to allow bridging resource sharing and management across the entire cluster.

Unified communication protocol for resource sharing and control with built-in failure detection and recovery.

* Details available at http://www.research.ibm.com/prose

IBM Research

(c) 2005 IBM Corporation2 The Effect of Virtualization on OS Interference 10/10/05

Noise Control w/PROSE & Hypervisors

Allow strict control of percentage of CPU devoted

to application versus system daemons and I/O

requests

Can eliminate jitter associated with interrupt

service routines

Provides a higher degree of determinism that

vanilla Linux, but does so at a performance cost

IBM Research

(c) 2005 IBM Corporation3 The Effect of Virtualization on OS Interference 10/10/05

Measured OS Interference Comparison w/FWQ

98

99

100

101

102

103

104

105

106

107

108

Time

FWQ

Com

ple

tion T

ime (

ms)

203

204

205

206

207

208

209

210

211

212

213

Time

FWQ

Com

ple

tion

Tim

e (

ms)

125

126

127

128

129

130

131

132

133

134

135

Time

FWQ

Com

ple

tion

Tim

e (m

s)

125

126

127

128

129

130

131

132

133

134

135

Time

FW

Q C

om

ple

tion

Tim

e (

ms)

Idle Loaded

Linux

PROSE

IBM Research

(c) 2005 IBM Corporation4 The Effect of Virtualization on OS Interference 10/10/05

BACKUP SLIDES

IBM Research

(c) 2005 IBM Corporation5 The Effect of Virtualization on OS Interference 10/10/05

HDEC Sensitivity

125

126

127

128

129

130

131

132

133

134

135

Time

FW

Q C

om

ple

tion

Tim

e (

ms)

IBM Research

(c) 2005 IBM Corporation6 The Effect of Virtualization on OS Interference 10/10/05

Phase Scheduling Noise

FWQ aren't aligned to

scheduler quanta

Noise is exacerbated by

fixed length scheduling

slots.

Fixed noise ratio based

on HDEC length

...

IBM Research

(c) 2005 IBM Corporation7 The Effect of Virtualization on OS Interference 10/10/05

Acknowledgments

This work would not be possible without the contributions

of Jimi Xenidis, Michal Ostrowski, Orran Krieger, and

the rest of the rHype team. This work was supported in

part by the Defense Advanced Research Projects

Agency under contract no. NBCH30390004.

IBM Research

(c) 2005 IBM Corporation8 The Effect of Virtualization on OS Interference 10/10/05

GUPS Performance Slide

04/08/1900 08/29/1900 05/26/1901 03/02/1903 08/14/1906 06/20/1913 03/03/1927 08/10/195402/04/1900 04/11/1900 08/30/1900 08/05/1902 02/19/1907 11/23/1916 10/06/1936 10/06/1976

0

5000

10000

15000

20000

25000

30000

1 2 4 8 16 32 64 128

1 2 4 8 16 32 64 128

Size (MB)

Tim

e (m

s)

IBM Research

(c) 2005 IBM Corporation9 The Effect of Virtualization on OS Interference 10/10/05

Applibc

App

MPI

myOS

Transparent Application Development Process

Original Application

PROSE Application

Custom OS Library

ApplibOS

IBM Research

(c) 2005 IBM Corporation10 The Effect of Virtualization on OS Interference 10/10/05

Partitioning Resources as Files

HardwareDevices

SystemServices

ApplicationServices

Disk

Network

TCP/IP Stack Database

GUI

/dev/hda1

/dev/hda2

tun/tap driver

/dev/eth0

/dev/tap0

/dev/tap1

/net /arp

/udp/tcp

/clone/stats/0/1

/ctl/data/listen/local/remote/status

File System/mnt/9p_root/mnt/common_fs/mnt/remote_nfs

/sql/clone/0

/query/result

/1

/win/clone/0/1

/ctl/data/opengl/refresh

/2

IBM Research

(c) 2005 IBM Corporation11 The Effect of Virtualization on OS Interference 10/10/05

PROSE I/O

u9fsdevcons

Privatenamespace

in channel

out channel

Shared Memory

libfs

application

openreadwriteclose

Network

tcp/ip

mpifsEthernet

DiskPartition

FileSystem

netfs

IBM Research

(c) 2005 IBM Corporation12 The Effect of Virtualization on OS Interference 10/10/05

Future Work

Experiment with dynamic scheduling policy which adapts slot-

scheduler based on idle yielding.

Repeat experiments with different interrupt service policies.

Repeat experiments with different virtualization implementations

(Xen, VMware, IBM Virtualization Engine, etc.)

Repeat experiments with a standard benchmark w/ I/O dependencies

instead of relying on microbenchmarks.

SMP studies.

IBM Research

(c) 2005 IBM Corporation13 The Effect of Virtualization on OS Interference 10/10/05

Experimental Setup

arlx112 arlx113

Both• IBM JS20 Blade• SLOF Firmware• 4 GB DRAM Memory• Single* 1.66 GHZ 970

• Linux 2.6.10 Controller Partition• Linux 2.6.10• 64 MB of memory

PROSE Partition• Application + lib-os• 1 GB of memory• Console & Time over 9P

for(i=0; i<num_samples; i++) {start = mftb();for(w=0; w<work_len; w++);stop = mtftb();delta[i] = stop-start;

}

IBM Research

(c) 2005 IBM Corporation14 The Effect of Virtualization on OS Interference 10/10/05

Results - Linux

98

99

100

101

102

103

104

105

106

107

108

Time

FWQ

Com

ple

tion T

ime (

ms)

203

204

205

206

207

208

209

210

211

212

213

Time

FWQ

Co

mp

leti

on

Tim

e (

ms)

Idle Loaded

IBM Research

(c) 2005 IBM Corporation15 The Effect of Virtualization on OS Interference 10/10/05

Results - PROSE

125

126

127

128

129

130

131

132

133

134

135

Time

FWQ

Com

ple

tion

Tim

e (m

s)

125

126

127

128

129

130

131

132

133

134

135

Time

FW

Q C

om

ple

tion

Tim

e (

ms)

Idle Loaded