applications of virtual machine monitors for scalable, reliable, and interactive internet services

Applications of Virtual Machines For Scalable, Reliable, And Interactive Internet Services

Ph.D. Oral ExaminationAmr A. AwadallahJune 24th, 2006 [email protected]

http://www.awadallah.com/vmatrix

mailto:[email protected]

Stanford University – CSL – 7/24/2006 – Amr A. Awadallah PhD Oral Exam 2

What is The vMatrix?


Virtual Machine Monitors

Virtual Machine Monitor (aka Hypervisor)

Real Machine (CPU, Memory, Disks, Network)

Virtual Machine 1:vCPU, vMem, vDisk, vNet

OS1: Windows 2000

Virtual Machine 2:vCPU, vMem, vDisk, vNet

OS2: Linux

MySQL, ApacheOracle, IIS


Live Demo of VMware player:


The vMatrix!

A network for delivering virtual machines (VMs) between real machines (RMs) running the virtual machine monitor (VMM) software.


Contribution

An overlay network of VMMs (the vMatrix) enables backward-compatible improvement of the scalability, reliability, and interactivity of Internet services.

Applications:1. Dynamic Content Distribution2. Server Switching3. Fair placement of Game Servers


First Application: Dynamic Content Distribution

Motivation: To enable distribution of dynamic content. (40% of web requests)

Definition:Dynamic content is web pages which are constructed by programs that execute on the server at the time a request is made. (e.g. http://maps.yahoo.com)


Today Is Static Mirroring


Dynamic Content Distribution


Two Tier Architecture


Main Problem

It is very hard to copy services due to all the dependencies that code has on system libraries, third-party modules, operating systems, and server hardware.


Challenges: Mobility (NAT)

VMM

RMVM Agent

VM1 VM2192.168.1.10

NAT/LBInternet64.58.77.28

DNSmaps.yahoo.com 64.58.77.28


Challenges: Security (VPNs)

VMM

VM1 VM2

RMVM Agent

VPNIntranet

DNS

192.168.1.10172.21.162.9

c009.proxy.yahoo.com

maps.yahoo.com 64.58.77.28

NAT/LBInternet64.58.77.28


Two Tier Challenges• Response Time

FRONT END BACK END

N1

FRONT END

N1

BACK END

N2

BACK END

• Perception!


Two Tier Challenges• Availability

FRONT END BACK END


The AJAX Solution


Advantages• Faster Response Time

• Higher Availability

• Network Bandwidth Savings

• Backward Compatibility

Disadvantage• VM files are very large (order of

gigabytes)!


Second Application: Server Multiplexing

Motivation: To allow statistical sharing of servers between services.


Static Servers (ala Circuit Switching)

FINANCE SPORTS

Load Balancer


Server Switching (ala Packet Switching)

FINANCE & SPORTS

Load Balancer

vMatrix Oracle

Key: Peak of Sums < Sum of Peaks


What makes Switching hard ?

• Software Dependencies

• Hardware Dependencies

• People Dependencies


vMatrix Framework:

The Oracle: This is the main control program

RM

VMM

VM

Hibernation Nest: VM files are suspended and stored here

for later retrieval.

Production Cluster: Live operational

VMs, typically only one VM per RM

Loading Chambers:

Development and Maintenance

Cluster, there could be more than one VM per RM here.

VM VM

VM VM

VM VM

VM VM RM

VMM

VM VM VM

RM

VMM

VM VM VM

RM

VMM

VM

RM

VMM

VM


Lifecycle of a server VM

Loading Chambers

Hibernation Nest

Instantiated

1. A large number of virtual servers are stored as dormant files on a SAN or Netapps

2. A number of virtual servers are activated in a shared RM so that developers & system admins can upgrade, debug, or patch them.

3. A virtual server is activated on a dedicated RM and exposed to live load.


Sample Implementations:

Real Machine (PIII-550MHz, 640MB RAM, 9GB hard disk)

VMware ESX VMM Server (consumes 184MB RAM, 3.5GB hard disk and 5% CPU)

Virtual Machine exposes a PIII-550MHz with 512MB RAM and 5.5GB hard disk.

Operating System: Red Hat Linux 9

Apache Web Server MySQL Database

PHP (Hyper Text Processor)

PHP-Nuke and osCommerce Internet Services




Operating System: Yahoo FreeBSD 4.8

YLIB (Yahoo C/C++ Libraries)

YSS (Yahoo Structured Search)

Yahoo! Autos Search Network API


Advantages• Presents developers with isolation model they expect

• Backward Compatibility

• Efficient Resource Utilization (People and Power)

• Peak Absorption

• Faster Recovery

Disadvantages• VM files are very large (order of gigabytes)!

• VM Software is not free

• Operating System Licensing Issues


Third Application: Equi-Ping Game Server Placement For Pre-Arranged First-Person-Shooter Multiplayer Matches

Motivation: To place game server hosts optimally such that all participating players have a fairer game experience.


Delay Sensitivity of Online Games:

Extremely Lag sensitive(on the order of 50ms)

First Person Shooters (FPS)(Halo, Battlefield2, Doom, CounterStrike)

Moderate sensitivity to lag (on the order of 500ms)

MMORPG and RTS(World of Warcraft, Everquest, Generals)

Not sensitive to lag (on the order of seconds)

Turn Based Strategy(Civilization 4, Risk, Chess, Pool)


Screenshot from Halo PC:


The Problem: Unfair Lag

Clan A Clan B

130ms

30ms

GameServer


The Solution: Equi-Ping Server

Clan A Clan B

80msReal Machine

80ms

VMM

Virtual Game Server in VM


Equi-ping placement problem• Red dots are servers, green dots are players, we need to pick the

Red dot that is equi-distant from all players.


S = set of m servers available to host the match.

P = set of n players participating in this match.

RTTs,p = round-trip-time (ping) from server s to player p

The Algorithm has 3 main steps:

1. Disregard all servers that have any RTTs,p larger than 180ms

6. For each server s, we compute a closeness factor Cs representing the average differential ping between the players if server s is picked:

10. Pick the server with smallest Cs

The order of complexity of this algorithm is O(n2m) ≈ O(m) since m » n

Equi-ping placement Algorithm

n

RTTRTT

C Pp Piisps

s 2

,,


Halo PC VM Server Stack




Operating System: Windows XP (1.8GB)

Halo PC Game Server (278MB)


Advantages:• Equi-ping placement of game servers to

minimize round-trip-time difference between participants

• Backward compatibility leading to zero cost for converting existing

• Economies of scale: this network can be shared among many games


Applications Summary• Dynamic Content Distribution which

improves interactivity for Internet services.

• Server Switching which improves resource utilization, availability and enables peek absorption.

• Equi-ping server placement for game host servers which is very important for the fairness of online FPS games.


Related WorkCommon drawback: no backward compatibility.

• Disk Imaging / Ghosting, (no concurrent VMs).

• Application Servers: IBM WebSphere, ATG Dynamo, BEA WebLogic, JBoss, (no strict isolation, interface violations).

• Java Virtual Machine (J#/C#/.Net), (needs recoding, low performance).

• Packagers: Debian APT, Redhat RPM, PRC, (no isolation).

• Light Weight OSes (Denali), (new OS API, needs recoding).

• OS Virtualization (Ensim, Ejasent, EXETender, Chrooting), (tied to OS).

• Active Proxy Caches (ICAP, Active cache, OPES)


Papers"The vMatrix: A Network of Virtual Machine Monitors for Dynamic

Content Distribution", Amr Awadallah and Mendel Rosenblum, 7th International Workshop on Web Content Caching and Distribution (WCW 2002), Boulder, Colorado, August 2002

"The vMatrix: Server Switching", Amr A. Awadallah and Mendel Rosenblum, IEEE 10th International Workshop on Future Trends in Distributed Computing Systems (IEEE FTDCS 2004), Suzhou, China, May 2004.

"The vMatrix: Equi-Ping Game Server Placement For Pre-Arranged First-Person-Shooter Multiplayer Matches", Amr A. Awadallah and Mendel Rosenblum, The 4th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA 2006), Dubai/Sharjah, UAE, March 2006


Other Papers

"TCP-BFA: Buffer Fill Avoidance", Amr A. Awadallah and Chetan Rai, IFIP High Performance Networking Conference, Vienna, Austria, September 1998.

"Analysis of a packet switch with memories running slower than the line-rate", Sundar Iyer, Amr A. Awadallah, and Nick McKeown, IEEE INFOCOM 2000, Tel-Aviv, Israel, March 2000.


ConclusionThe vMatrix is an overlay network of virtual

machine monitors enabling delivery of virtual machines between real machines.

The vMatrix is a practical backward-compatible solution for improving the scalability, reliability, and interactivity of existing Internet applications.


Thank You


Appendix: VMware Perl APIuse VMware::Control;

use VMware::Control::Server;

use VMware::Control::VM;

my $VMMserver = VMware::Control::Server ::new($hostname, $port, $user, $password);

$VMMserver->connect();

my $VM = VMware::Control::VM ::new($server, $vmconfig);

$VM->connect();

# To get a list of all VMs on a server

my @vmlist = $VMMserver->enumerate();

# To register a new VM on a server

$VMMserver->register($vmconfig);

# To start and stop a VM

$VM->start();

$VM->stop();

# To suspend and resume a VM

$VM->suspend();

$VM->resume();

# To get CPU, Memory, Net, IO stats

$VM->get(“Status.Stats.vm.cpuUsage”, 5*60);

# Check if VM is running (heart-beat)

$VM->get(“Status.Power”);


Two Tier Challenges• Replication

BACK ENDFRONT END

FRONT END

FRONT END

applications of virtual machine monitors for scalable, reliable, and interactive internet services

Technology