planetlab: an open laboratory for introducing disruptive technology into the internet
DESCRIPTION
PlanetLab: An Open Laboratory for Introducing Disruptive Technology into the Internet. David Culler University of California, Berkeley http://www.cs.berkeley.edu/~culler. Q: From where will the next 10 major advances in the internet emerge?. Innovator’s Dilemma. - PowerPoint PPT PresentationTRANSCRIPT
March 9, 2004 1
PlanetLab: An Open Laboratory for Introducing Disruptive Technology
into the Internet
David CullerUniversity of California, Berkeley
http://www.cs.berkeley.edu/~culler
March 9, 2004 3
Innovator’s Dilemma• The Internet is an enormous success
story– commercially– impact on our daily lives– global reach
• Success has an unexpected cost: ossification– difficult to deploy disruptive technologies
correct vulnerabilities introduce new capabilities
March 9, 2004 5
…PlanetLab
Intel BerkeleyIntel BerkeleyICIRICIR
MITMIT
PrincetonPrincetonCornellCornell
DukeDuke
UTUT
ColumbiaColumbiaUCSBUCSBUCBUCB
UCSDUCSDUCLAUCLA
UWUW
Intel SeattleIntel Seattle
KYKY
CambridgeCambridge
HarvardHarvard
GITGIT
UppsalaUppsalaCopenhagenCopenhagen
CMUCMU
UPennUPennWIWIChicagoChicagoUtahUtah
Intel ORIntel OR
UBCUBC
WashuWashu
ISIISI
IntelIntel
RiceRice
BolognaBologna
LancasterLancaster
St. LouisSt. Louis
UAUA
CanterburyCanterbury
SydneySydney
BerlinBerlinBelasBelas
TorontoToronto
CUHKCUHK
HUJHUJ
UCLUCL
TaiwanTaiwan
NizhnyNizhny
Towards 1,000 widely-distributed nodes– today: 360 machines, 148 sites, 20 countries, – 110 networks– at edge sites and network cross-roads– Universities, Companies, Nat’l labs
March 9, 2004 6
Today’s Internet
Limitations– the Internet is “opaque”
making it difficult to adapt to network conditions
– applications typically split into two pieces: client and server
Best-Effort Packet
Delivery Service
March 9, 2004 8
Tomorrow’s Internet
Opportunities– multiple vantage points
anomaly detection, robust routing Adapt to load, failures, $
– Proximity: close wherever you are low latency, high bandwidth, predictable, reliable content distribution, data fusion
– multiple, independent domains survivable storage
Collection ofPlanetary-ScaleServices
March 9, 2004 9
Towards Planetary-Scale Services
• CDN and P2P just tip of the iceberg
• Research Community developing the architectural building blocks to enable many kinds of distributed services–scalable translation, –dist. storage, –dist. events, –instrumentation, –management
March 9, 2004 10
Key missing element – real hands-on experience
• Researchers had no vehicle to try out their next n great ideas in this space
• Lot’s of simulations • Lot’s of emulation on large clusters
– emulab, millennium, modelnet
• Lot’s of folks calling their 17 friends before the next deadline
– RON testbed
• but not the surprises and frustrations of experience at scale to drive innovation
March 9, 2004 11
Quick Growth• “Underground” meeting March 2002• Intel seeds effort
– First 100 nodes, Operational support
• First node up July 2002• By SOSP (March 2003) 25% of accepted
papers refer to PlanetLab• Each following conference
has seen dramatic load– OSDI– NDSI
March 9, 2004 14
Washington - ScriptRoute
• Internet measurement tool• open, unauthenticated, community measurement
infrastructure vs closed, engineered service
UWGoogle
March 9, 2004 15
Princeton: CoDeeNOpen Content Distribution Network
BBB
B
B
B
B
A
A
A
A
AA
C
C
C
C
CCC
March 9, 2004 16
Berkeley – Internet Indirection Infrastructure
• Multicast, anycast, mobility, …
src
dstPut v,IDGet ID
March 9, 2004 17
NetBait Serendipity• Brent Chun built a simple http server on port 80 to
explain what planetlab was about and to direct inquiries to planet-lab.org
• It also logged requests• Sitting just outside the firewall of ~40 universities...• the worlds largest honey pot• the number of worm probes from compromised
machines was shocking• imagine the the epidemiology
• see netbait.planet-lab.org
March 9, 2004 18
One example
• The monthly code-red cycle in the large?• What happened in March 2003?
0
50
100
150
200
250
1/5/
2003
1/10
/200
3
1/15
/200
3
1/20
/200
3
1/25
/200
3
1/30
/200
3
2/4/
2003
2/9/
2003
2/14
/200
3
2/19
/200
3
2/24
/200
3
3/1/
2003
3/6/
2003
3/11
/200
3
3/16
/200
3
Pro
be
s p
er
da
y
Code Red
Nimda
March 9, 2004 19
No, not Iraq
• A new voracious worm appeared and displaced the older Code Red
0
200
400
600
800
1000
1200
1400
3/1/
2003
3/2/
2003
3/3/
2003
3/4/
2003
3/5/
2003
3/6/
2003
3/7/
2003
3/8/
2003
3/9/
2003
3/10
/200
3
3/11
/200
3
3/12
/200
3
3/13
/200
3
3/14
/200
3
3/15
/200
3
3/16
/200
3
3/17
/200
3
3/18
/200
3
3/19
/200
3
3/20
/200
3
Pro
be
s p
er
da
y
Code Red
Nimda
Code Red II.F
March 9, 2004 24
Evolving the Internet• Add a new layer to the network architecture
– overlay networks
• Challenge– how to innovate & deploy at scale
purpose-built virtual networks that use the existing Internet for transmission
the Internet was once deployed as an overlay on
top of the telephony network
overlay
Internet
March 9, 2004 25
PlanetLab is…
Intel BerkeleyIntel BerkeleyICIRICIR
MITMIT
PrincetonPrincetonCornellCornell
DukeDuke
UTUT
ColumbiaColumbiaUCSBUCSBUCBUCB
UCSDUCSDUCLAUCLA
UWUW
Intel SeattleIntel Seattle
KYKY
CambridgeCambridge
HarvardHarvard
GITGIT
UppsalaUppsalaCopenhagenCopenhagen
CMUCMU
UPennUPennWIWIChicagoChicagoUtahUtah
Intel ORIntel OR
UBCUBC
WashuWashu
ISIISI
IntelIntel
RiceRice
BolognaBologna
LancasterLancaster
St. LouisSt. Louis
UAUA
CanterburyCanterbury
SydneySydney
BerlinBerlinBelasBelas
TorontoToronto
CUHKCUHK
HUJHUJ
UCLUCL
TaiwanTaiwan
NizhnyNizhny
More than a bunch of machines spread around the world
March 9, 2004 26
PlanetLab is…
A common software architecture• OS running on each node• Remote installation mechanism• Remote management capability• Central account management service
March 9, 2004 27
PlanetLab is…
A network testbed • 450 active research projects
– experiment at scale– experiment under real-world conditions– potential for real workloads and users
A deployment platform• 15 continuously running services
design deploy
measure
March 9, 2004 28
PlanetLab is…
A microcosm of the next Internet
• Fold services back into PlanetLab– evolve core technologies to support overlays and
slices– discover common sub-services
• Long-term goals– become the way users interact with the Internet– define standards that support multiple
“PlanetLabs”
March 9, 2004 29
Software Architecture
• Distributed virtualization– slice a network of virtual machines– isolation
isolate services from each other protect the Internet from PlanetLab
• Unbundled Management– OS defines only local (per-node) behavior
global (network-wide) behavior implemented by services
– multiple competing services running in parallel shared, unprivileged interfaces
March 9, 2004 32
Virtualization• Hypervisors (e.g., VMWare)
– don’t scale well– don’t need multi-OS functionality
• Paravirtualization (e.g., Xen, Denali)– not yet mature
• Virtualize at system call interface (e.g., Jail, Vservers)– reasonable compromise– doesn’t provide the isolation that hypervisors do
• Unix processes– isolation is problematic
• Java Virtual Machine– too high-level
March 9, 2004 33
PlanetLab VMM• Linux: significant mind-share• Vservers: virtualizes at the system call
interface– each vserver runs in its own security context
private UID/GID name space limited superuser capabilities (e.g., no CAP_NET_RAW)
– uses chroot for file system isolation– scales to 1000 vservers per node (29MB each)
• Plkmod: enforces isolation– processor and link schedulers, disk quotas– network virtualization
safe raw sockets (UDP, TCP, ICMP, GRE) rate limits exceptional packets, unique addrs/ports
March 9, 2004 34
Infrastructure Services• Brokerage Services
– buy, sell, trade, pool resources– least mature / greatest potential (market-based)
• Environment Services– create / maintain programming environments– challenge: space efficiency
• Monitoring Services– resource discovery– network / service health– adaptive applications
• Routing Underlay– discover network topology
March 9, 2004 35
PLC: Brokerage Service
create_slice(name, credentials)
delete_slice(name, credentials)
assign_owners(name, ssh_keys[ ], credentials)
set_state(name, boot_script, credentials)
set_resources(name, share, credentials)
instantiate_slice(name, nodes[ ], credentials)
rcap = acquire(rspec)
bind(name, rcap)
March 9, 2004 36
Stork: Environment Service
• Defines a boot_script to be run when VM inits– invokes Stork
• Maintains a repository of packages– those registered by a client service + dependencies
• Shared package area on each node– retrieves one copy of each required package– installation scripts run in context of client VM
• Optionally…– manages user accounts for client slices– establishes tunnels to neighboring nodes
March 9, 2004 37
Monitoring Service – Sophia, Pier,…
S S S… A A A…
Information Plane
observe analyze react
Distributed query processor
March 9, 2004 38
Pluto: Routing Underlay
Service Overlay Networks
Library of Routing Services
Topology Probing Kernel
Raw Topology Information
Primitives
Expense
Scope
March 9, 2004 40
A Rich Research Agenda
• Global System Architecture– Slices, management, distribution,
• Network measurement– Scriptroute, PlanetProbe, I3, etc.
• Application-level multicast– ESM, Scribe, TACT, etc.
• Distributed Hash Tables– Chord, Tapestry, Pastry, Bamboo,
etc.• Wide-area distributed storage
– Oceanstore, SFS, CFS, Palimpsest, IBP
• Resource allocation– Sharp, Slices, XenoCorp,
Automated contracts• Distributed query processing
– PIER, IrisLog, Sophia, etc.
• Content Dist. Networks– CoDeeN, ESM, UltraPeer emulation,
Gnutella mapping• Management and Monitoring
– Ganglia, InfoSpect, Scout Monitor, BGP Sensors, etc.
• Overlay Networks– RON, ROM++, ESM, XBone, ABone,
etc.• Virtualization and Isolation
– Xen, Denali, VServers, SILK, Mgmt VMs, etc.
• Router Design implications– NetBind, Scout, NewArch, Icarus,
etc. • Testbed Federation
– NetBed, RON, XenoServers• Etc., etc., etc.
March 9, 2004 41
Confluence of Technologies • Cluster-based scalable distribution, remote execution,
management, monitoring tools– UCB Millennium, OSCAR, ..., Utah Emulab, ...
• CDNS and P2Ps– Gnutella, Kazaa, ...
• Proxies routine• Virtual machines & Sandboxing
– VMWare, Janos, Denali,... web-host slices (EnSim)• Overlay networks becoming ubiquitous
– xBone, RON, Detour... Akamai, Digital Island, .... • Service Composition Frameworks
– yahoo, ninja, .net, websphere, Eliza• Established internet ‘crossroads’ – colos• Web Services / Utility Computing• Authentication infrastructure (grid)• Packet processing (layer 7 switches, NATs, firewalls)• Internet instrumentation
March 9, 2004 42
Current Institutions (partial)Academia Sinica, TaiwanBoston UniversityCaltechCarnegie Mellon UniversityChinese Univ of Hong KongColumbia UniversityCornell UniversityDatalogisk Institut CopenhagenDuke UniversityGeorgia TechHarvard UniversityHP LabsIntel ResearchJohns HopkinsLancaster UniversityLawrence Berkeley LaboratoryMITMichigan State UniversityNational Tsing Hua Univ.New York University
Northwestern University
Princeton UniversityPurdue UniversityRensselaer Polytechnic Inst.Rice UniversityRutgers UniversityStanford UniversityTechnische Universitat BerlinThe Hebrew Univ of JerusalemUniversity College LondonUniversity of ArizonaUniversity of BaselUniversity of BolognaUniversity of British ColumbiaUC BerkeleyUCLAUC San DiegoUC Santa BarbaraUniversity of CambridgeUniversity of CanterburyUniversity of ChicagoUniversity of Illinois
University of KansasUniversity of KentuckyUniversity of MarylandUniversity of MassachusettsUniversity of MichiganUniversity of North CarolinaUniversity of PennsylvaniaUniversity of RochesterUSC / ISIUniversity of Technology SydneyUniversity of TennesseeUniversity of Texas
University of Toronto
University of Utah
University of Virginia
University of Washington
University of Wisconsin
Uppsala University, Sweden
Washington University in St Louis
Wayne State University
March 9, 2004 43
What Planet-Lab is about?• Create the open infrastructure for invention of the
next generation of wide-area (“planetary scale”) services– post-cluster, post-yahoo, post-CDN, post-P2P, ...
• Potentially, the foundation on which the next Internet can emerge– think beyond TCP/UDP/IP + DNS + BGP + OSPF... as to what the net
provides– building-blocks upon which services and applications will be based– “the next internet will be created as an overlay in the current one” (NRC)
• A different kind of network testbed– not a collection of pipes and giga-pops– not a distributed supercomputer – geographically distributed network services– alternative network architectures and protocols
• Focus and Mobilize the Network / Systems Research Community to define the emerging internet
March 9, 2004 44
Join the fun ... • It is just beginning
– towards a representative sample of the internet (1,000s)
• PlanetLab consortium recently formed– Princeton, UCB, UWash– Intel, HP, Google, …
• Hands-on experience with wide-area services at scale is mothering tremendous innovation
– nothing “just works” in the wide-area at scale– Joining forces we all learn more
www.planet-lab.org