engineering a public affairs project donna liu, executive director uchannel, princeton university...
TRANSCRIPT
Engineering A Public Affairs Project
Donna Liu, Executive DirectorUChannel, Princeton University
Vivek Pai, Associate ProfessorCS Department, Princeton
UniversityCopyright Vivek Pai 2007. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
October 26, 2007 - Educause 2
BBB.COM
WWW Service
client
October 26, 2007 - Educause 3
Content Distribution Networks
BBB.COM
client server surrogate
B
B
B
B
B
B
cache
A
A
A
A
A
A
AAA.COM
C
C
C
C
C
C
CCC.COM
redirector
October 26, 2007 - Educause 4
Commercial CDN Companies
Many playersAkamai, Limelight, Level 3, …
CostsMonthly commitment(High) Premium over BWPossibly extra for spikes
October 26, 2007 - Educause 5
Enter PlanetLab
ConsortiumUniversities, research labs, companies
ResourcesEach member contributes machines, BW
AccessBy joining, you get accounts everywhere
ProjectsExperiments, long-running services
October 26, 2007 - Educause 6
PlanetLab Overview819 Nodes at 406 Sites
October 26, 2007 - Educause 7
CoDeeN: Toward a Self-Healing Web
Goals:Improve all levels of Web infrastructureInteroperate where possibleDrive with real trafficFeed back measurementsRouting
Name Lookup
Data Transfer
Replication
Client
Security
October 26, 2007 - Educause 8
CoBlitz Project
On-demand replication of large filesNo limit on file size
Using HTTP – standard clients & serversNo plugins, fewer support callsNo BitTorrent blocking, etc.
No preprocessing/staging neededInstant-on, immediate overload protection
October 26, 2007 - Educause 9
CoBlitz
Agent CDNClient
CDN reverse caches the chunks!
CDN
CDNCDN
CDN ClientAgent
CDN
chunk1
chun
k1chunk2
chunk3
chunk2
chunk5
chunk5
chunk1
chunk1
chunk4 chunk4 chunk5
chun
k5
chunk1 chunk2
chunk3 chunk3
chunk4 chunk5
CDN = Redirector + Reverse Proxy
October 26, 2007 - Educause 10
How To Use CoBlitz
Start with a regular URLNo password protectionAny standard Web serverOnly static content – no CGI, etcIdeally, unchanging
Add CoBlitz prefix – coblitz.codeen.orghttp://www.example.com/bigfilehttp://coblitz.codeen.org/www.example.com/bigfile
October 26, 2007 - Educause 11
CoBlitz Content Restrictions
Domain ends in .eduNo restrictions
General public, not whitelistedNo html, images, audio, video – copyright issuesStill useful for ISOs, PDFs, etc.
Whitelisted – contact usNo restrictions
October 26, 2007 - Educause 12
0
0.5
1
1.5
2
2.5
3
3.5
Direct BitTorrent CoBlitz
Do
wn
load
Rat
e (M
bp
s) Slowest
Median
CoBlitz Performance
CoBlitz users:UChannelCiteSeer digital libraryFedora Core LinuxMozarteumPlanetLab projects
October 26, 2007 - Educause 13
UChannel CoBlitz Usage
Served directly
Served by CoBlitz
October 26, 2007 - Educause 14
Browsers Versus RSS
Regular browsersDiurnal scheduleFlash crowds possible
RSS (Really Simple Syndication)Publishing via “feeds”“Feed readers” poll & grab new itemsPoll frequency ~1hr for always-on
October 26, 2007 - Educause 15
UChannel Bandwidth Usage – 5min
Extremely sharp peaks
October 26, 2007 - Educause 16
UChannel Bandwidth Usage – 1hr
Reasonable non-peak BW only 25 Mbps
October 26, 2007 - Educause 17
UChannel Usage Statistics
October 26, 2007 - Educause 18
Production QuestionsAre you stable?
Running 3 years, mostly 24/7No centralization – runs even when Princeton disconnected
How much BW available?Function of demand & participationMore PL members more BWMore CoBlitz usage less extra capacity
Service Level Agreements?Not on public PlanetLab
October 26, 2007 - Educause 19
Fedora Core 6 Release – Public PL
Released 10am 10/24/061.4 Gbps peak, sustained 800+ Mbps
October 26, 2007 - Educause 20
Limits on Current Deployment
Possibly outlast welcome (bandwidth)Keep going until PL members complainRegularly the #1 BW consumerSometimes more than rest combined
Exact BWs not under our controlShare resources on (overloaded) PLNo Africa coverage, spotty Australia
October 26, 2007 - Educause 21
Polish TV Results – Private PL
New record: 3.3 Gbps aggregateAll links near 100% utilizationNow, weekly trials starting
October 26, 2007 - Educause 22
Growth & Sustainability
Nonprofit usersCost-recovery modelBuy BW in bulk, pay for developersCheaper than CDNs, private BW purchase
For-profitsSpinning out tech as PlanetWorks LLCPrivate managed CoBlitz appliancesCost-effective beyond 10 Mbps
October 26, 2007 - Educause 23
Example Cost-Recovery
$25K/month (aggregate) can buy1Gbps bandwidth8 locations, 3 machines/location1 support person
If there are 5-10 projects interested,let’s get in touch
October 26, 2007 - Educause 24
Questions
[email protected]://codeen.cs.princeton.edu/
coblitz/
[email protected]://uc.princeton.edu