DotSlash – or how to DotSlash – or how to deal with 15 minutes deal with 15 minutes

of fame of fame

Weibin ZhaoHenning Schulzrinne

Columbia University

CATT/WICAT Annual Research ReviewNovember 14, 2003

11/14/2003 DotSlash 2

Web HotspotsWeb Hotspots A sudden, dramatic surge of request rate

Hard to predict – e.g., external reference to web site

Build up quickly Large magnitude

Short-term overload Long delay or no service

One-time event “15 minutes of fame” Small web sites, e.g., on DSL or cable modem Thus, traditional solutions (Akamai, etc.) not

economical

11/14/2003 DotSlash 3

Existing MechanismsExisting Mechanisms Capacity planning

high-end servers, LAN clusters, WAN mirrors

Fixed capacity, cost CDNs

Advance set-up, cost Only effective for large servers

Caching Not fully controlled by the server

Admission control No service for some clients

11/14/2003 DotSlash 4

DotSlash ApproachDotSlash Approach Enable a site to utilize spare capacity at

other sites via dynamic collaboration Rescue Extending capacity dynamically as needed Usage models: mainly for mutual aid, could

be commercial Focusing on small sites

Advantages Client transparent: no browser changes Low cost: based on existing systems, no

additional hardware cost

11/14/2003 DotSlash 5

Rescue Rescue Example-1:Example-1:

origin.comrescue.com

Client1

www.origin.com1.2.3.4

www.rescue.com5.6.7.8

(1) www.origin.com

(2) 1.2.3.4

(3) (4) HTTP redirect alias1.rescue.com

(8) Reverse proxy

(5) alias1.rescue.com

(6) 5.6.7.8

(9)

(10)(7)Dynamic

DNS

Dynamic DNS

DNS RR

origin.comDNS

rescue.comDNS

cache

HTTP redirect &HTTP redirect &Cache missCache miss

11/14/2003 DotSlash 6

Rescue Example-Rescue Example-2:2:

origin.comrescue.com

www.origin.com1.2.3.4

www.rescue.com5.6.7.8

Dynamic DNS

Dynamic DNS

DNS RR

origin.comDNS

rescue.comDNS

cache

Client2 (2) 5.6.7.8

(1) www.origin.com

(4)

(3)

DNS RR &DNS RR &Cache hitCache hit

11/14/2003 DotSlash 7

DotSlash FunctionsDotSlash Functions Workload monitoring Rescue server discovery

Enhanced SLP: mSLP + Select/Sort extensions

Request distribution and redirect DNS RR + HTTP redirect

Dynamic replication of content Reverse caching proxy

Rescue relationship management Allocate/release rescue servers

11/14/2003 DotSlash 8

Basic ArchitectureBasic Architecture Distributed system + dynamic virtual hosting

Distributed: multiple servers for one site Virtual hosting: one server for multiple sites

Components Apache module (Mod_dots), DotSlash daemon (Dotsd),

dynamic DNS (BIND), service discovery (enhanced SLP)

BIND mSLP

HTTPSHM

SLPDNS

TCP OtherDotsdClient Apache

DotsdMod_dots

11/14/2003 DotSlash 9

Server StatusServer Status

Normal

Rescue

SOS

Allocate rescue servers

Release rescue servers

As a rescue server

Not as a rescue server

Origin server Ask help from others

Rescue server Provide help to others

Regular server

Otherwise

11/14/2003 DotSlash 10

Workload MonitoringWorkload Monitoring Bottlenecks

Network bandwidth CPU Memory

Metrics Output data rate

(HTTP) KBytes/second

CPU load average Num concurrent conns

MaxClients

CPUNet Mem

Overload

Normal

OR Gate

AND Gate

11/14/2003 DotSlash 11

Origin ServerOrigin Server Workload monitoring HTTP redirect

Use rescue server aliases Don’t redirect requests from rescue

servers Redirect Policies

WRR based on rescue server capacity (load) URL Hashing (URL locality) for large files

DNS round-robin Add/remove rescue server IP addresses

11/14/2003 DotSlash 12

Rescue ServerRescue Server Workload monitoring Reverse caching proxy

Memory caching: concurrency control among threads

Check Host: header: A regular request A reverse proxy request to origin server (active

rescue) An HTTP redirect to origin server (expired rescue)

Dynamic DNS update Register new aliases

11/14/2003 DotSlash 13

Rescue Server Rescue Server ManagementManagement Allocation

Initial allocation: based on origin server load Additional allocation: based on rescue server

load Policy: double current capacity

Release When load is consistently low Release one rescue server at a time Partial release: the min-capacity rescue

server Final release: the last rescue server

11/14/2003 DotSlash 14

Other ComponentsOther Components Testing tool

Use httperf from HP Labs, and extend it to handle HTTP redirects automatically

Dynamic DNS Use BIND 9.2.2, set up a testing domain:

dotslash.cs.columbia.edu, and only allow updates from specific hosts

Service discovery Use SLP with mesh-enhancement (mSLP),

and Select, Sort and Attribute List extensions

11/14/2003 DotSlash 15

Testing EnvironmentTesting Environment Apache

2.0.47 With proxy and caching modules Enable: caching and mod_dots

OS Redhat 9, Linux kernel 2.4.20-20.9,

hosts from Clic Lab Redhat 7.3, Linux kernel 2.4.19-

6_planetlab, nodes from PlanetLab

11/14/2003 DotSlash 16

ConclusionConclusion

Preliminary version implemented Basic measurement and control flow Working on service location

On-going work Performance measurement Apply to dynamic contents

LAMP: Linux+ Apache + MySQL + PHP