Intel Research @ Berkeleyand
Extreme Networked Systems
www.intel-research.net/berkeley
David Culler
8/12/2002
8/12/2002 IRB/XIS 2
Where this presentation might go...
aka Outline
• new models of industry/academic research collaboration
• vast networks of tiny devices in the physical world
• open infrastructure for emerging planetary-scale services
8/12/2002 IRB/XIS 3
New model for ind/acad collaboration
• Key challenges ahead in EECS are fundamentally problems of scale
– require level of investigation and engineering beyond what is sustainable within the university and beyond what a company can commit outside product scope
– industry possesses key technology and expertise– requires insights from many perspectives
• A new lab stucture built around deep research collaboration and intimate ties to the EECS department
– industry contributes substantial effort of high quality– projects span boundaries– faculty co-direct lab– student / faculty cycles drive the continuous motion
• Operate in uniquely open fashion
8/12/2002 IRB/XIS 4
Intel Network of Lablets Concept
• Network of small labs working closely with top computer science departments around the world on deeply collaborative projects.
– Berkeley – extreme network systems– Washington – HCI– CMU – distributed storage– Cambridge
• Complement the corporate labs– explore off the roadmap, long range, high risk
• Complement the external-research council– drive projects of significant scale and impact
• Expand the channel– Bi-directional transfer of people, ideas, technology
8/12/2002 IRB/XIS 5
lablet mission
• Leadership role in emerging and important areas
• Combining the unique strengths of Intel and Univ.
• Bi-directional exchange of breakthough ideas, technology and people
Lablet
Novel componenttechnology
SRPs
University
Intel Labs
Advanced Applications
Advance of theresearch ecosystem
8/12/2002 IRB/XIS 6
Berkeley Emphasis
• Cross-cutting problems of scale. • Extreme Interconnected Systems
• “endonets”– dense, fine-grain networked systems deeply embedded in or
interacting with physical environment– sensor networks– ubiquitous computing architectures– computational fabrics, surfaces, structures
• “exonets”– broad coverage networked systems at societal scale– world-wide storage systems– composable infrastructure services– massive servers for millions of users
8/12/2002 IRB/XIS 7
Scale and structure
Active day-to-day involvement
• ~20 full-time Intel Researchers and Engineers– currently 13
• ~5 part-time Intel folks
• 20 faculty, students, visitors, research consultants
Two-in-a-box co-directors
• University Director + Intel Director
• Report to David Tennenhouse, VP Research
Project focused
• ~6-year projects starting about every two years
8/12/2002 IRB/XIS 8
Two Major Lab Projects
• Define and Develop complete ‘network system stack’ for deeply embedded sensor/effector networks
– enabling technology– create the community– core architecture, OS, networking, service foundations– demonstrate revolutionary applications
• Create an Open Laboratory for Widely-distributed “Planetary Scale” Services to explore architecture, services and applications
– enabling resource catalyzes community– distributed development effort– foundations: scalable, secure slice-able platform– infra and service design trade-offs (DHT, Dist-storage)
8/12/2002 IRB/XIS 9
Open Collaborative Research Agreement
• Master Agreement states – intent: Open
– terms, conditions (IP addendum)
• Research Project Descriptions– what, who, where
• scope of work defines boundary of openness!– an openness agreement is all about defining reach-through
8/12/2002 IRB/XIS 10
System Stack for Deeply Embedded Networks
8/12/2002 IRB/XIS 11
Bridging the Technology-Appln Gap
mg
mt
/ dia
g /
deb
ug
alg
ori
thm
/ th
eoryservice
network
system
architecture
pro
g /
dat
a m
od
el
data mgmt
application
Monitoring & Managing Spaces and Things
technology
8/12/2002 IRB/XIS 12
Deeply Embedded Networks
• # nodes >> # people
• sensor/actuator data stream
• unattended
• inaccessible
• prolonged deployment
• energy constrained
• operate in aggregate
• in-network processing necessary
• what they do changes over time
=> must be programmed over the network
8/12/2002 IRB/XIS 13
Project Activities• Core Platform
– architecture, TinyOS, Networking– simulation and debugging tools
• Programming Support– NesC (TinyOS modularity and concurrency)– Cooperating FSMs, atomicity– Macroprogramming
• Sensor-Network databases– streaming, noisy data, with in-network query processing
• Delay Tolerant Networking– overlay for diverse, challenged internets
• Interactive Environments and Things– ambient displays, remote physical communication– context-aware tools for the handicapped
• Habitat and Environmental Monitoring– dense sensor networks in the hands of life scientists
• Generic Sensor Kit
8/12/2002 IRB/XIS 14
Platform Architecture
• Goal– create a small wireless device that would enable us to explore
the system design space, applns to be attempted, and a new research community
– develop the architecture in response to observed system design
• Approach– joined in the series of UCB COTS mote designs
» WeC -> Rene -> iDot -> MICA– look to silicon for full architecture
• New ideas– rich interfaces allow radical system optimizations
» analog wake-up, Tx-Rx time synch– federation of accelerators, not dedicate protocol proc.– HW/SW multithreading for low power, passive vigilance
service
network
system
architecture
data mgmt
application
technology
8/12/2002 IRB/XIS 15
Berkeley Wireless Sensor ‘Motes’
Mote Type WeC Rene Rene2 Dot Mica
Date Sep-99 Oct-00 Jun-01 Aug-01 Feb-02
Microcontroller (4MHz)
Type AT90LS8535 ATMega163 ATMega103/128
Prog. Mem. (KB) 8 16 128
RAM (KB) 0.5 1 4
Communication
Radio RFM TR1000
Rate (Kbps) 10 10/40
Modulation Type OOK OOK/ASK
8/12/2002 IRB/XIS 16
TinyOS Application Graph
RFM
Radio byte
Radio Packet
UART
Serial Packet
ADC
Temp photo
Active Messages
clocks
bit
by
tep
ac
ke
t
Route map router sensor appln
ap
pli
ca
tio
n
HW
SWExample: self-organized ad- hoc, multi-hop routing of photo sensor readings
3450 B code 226 B data
Graph of cooperatingstate machines on shared stack
8/12/2002 IRB/XIS 17
It is a noisy world after all...
• Get to rethink each of the layers in a new context
– coding, framing– mac– routing– transport, – rate control– discovery– multicast– aggregation– naming– security– ...
• Resource constrained, power aware, highly variable, ...
• Every node is also a router• No entrenched ‘dusty packets’
probability of reception from center node vs xmit strength
8/12/2002 IRB/XIS 18
Example “epidemic” tree formation
8/12/2002 IRB/XIS 19
Habitat Monitoring
Acadia National ParkMt. Desert Island, ME
Great Duck IslandNature Conservancy
Ongoing research
WAN(satcast)
LAN
sensor nets
http://www.greatduckisland.net
8/12/2002 IRB/XIS 20
Cross-cutting issues?
application
service
network
system
architecture
technology
mg
mt
/ dia
g /
deb
ug
alg
ori
thm
/ th
eory
pro
g /
dat
a m
od
el
• Programming environments
• Deep & scalable simulation
• Algorithm behavior at scale
• Operating on prob. distributions
• Fine-Grain Inverse problems
• Pseudo-imaging
• Constructive foundations of self-organization
data mgmt
8/12/2002 IRB/XIS 21
The Other Extreme - Planetary Scale Services
www.planet-lab.org
8/12/2002 IRB/XIS 22
Motivation
• A new class of services & applications is emerging that spread over a sizable fraction of the web
– CDNs as the first examples
– Peer-to-peer, ...
• Architectural components are beginning to emerge– Distributed hash tables to provide scalable translation
– Distributed storage, caching, instrumentation, mapping, events ...
• The next internet will be created as an overlay on the current one
– as did the last one
– it will be defined by its services, not its transport
» translation, storage, caching, event notification, management
• There will soon be vehicle to try out the next n great ideas in this area
8/12/2002 IRB/XIS 23
Confluence of Technologies
• Cluster-based scalable distribution, remote execution, management, monitoring tools
– UCB Millennium, OSCAR, ..., Utah Emulab, ModelNet...
• CDNS and P2Ps– Gnutella, Kazaa, ... ,Pastry, Chord, CAN, Tapestry
• Proxies routine• Virtual machines & Sandboxing
– VMWare, Janos, Denali,... web-host slices (EnSim)
• Overlay networks becoming ubiquitous– XBONE, RON, Detour... Akamai, Digital Island, ....
• Service Composition Frameworks– yahoo, ninja, .net, websphere, Eliza
• Established internet ‘crossroads’ – colos• Web Services / Utility Computing• Grid authentication infrastructure• Packet processing,
– Anets, .... layer 7 switches, NATs, firewalls
• Internet instrumentation
The Time is NOW
8/12/2002 IRB/XIS 24
Guidelines (1)
• Thousand viewpoints on “the cloud” is what matters– not the thousand servers– not the routers, per se– not the pipes, per se
8/12/2002 IRB/XIS 25
Guidelines (2)
• and you miust have the vantage points of the crossroads– primarily co-location centers
8/12/2002 IRB/XIS 26
Guidelines (3)
• Each service needs an overlay covering many points
– logically isolated
• Many concurrent services and applications– must be able to slice nodes => VM per service– service has a slice across large subset
• Must be able to run each service / app over long period to build meaningful workload
– traffic capture/generator must be part of facility
• Consensus on “a node” more important than “which node”
8/12/2002 IRB/XIS 27
Guidelines (4)
• Test-lab as a whole must be up a lot– global remote administration and management
» mission control
– redundancy within
• Each service will require its own remote management capability
• Testlab nodes cannot “bring down” their site– generally not on main forwarding path
– proxy path
– must be able to extend overlay out to user nodes?
• Relationship to firewalls and proxies is key
Management, Management, Management
8/12/2002 IRB/XIS 28
Guidelines (5)
• Storage has to be a part of it– edge nodes have significant capacity
• Needs a basic well-managed capability– but growing to the seti@home model should be considered at
some stage
– may be essential for some services
8/12/2002 IRB/XIS 29
Initial Researchers (mar 02)
WashingtonTom Anderson
Steven Gribble
David Wetherall
MITFrans Kaashoek
Hari Balakrishnan
Robert Morris
David Anderson
BerkeleyIon Stoica
Joe Helerstein
Eric Brewer
John Kubi
Intel ResearchDavid CullerTimothy RoscoeSylvia RatnasamyGaetano BorrielloSatyaMilan Milenkovic
DukeAmin VadatJeff Chase
PrincetonLarry PetersonRandy WangVivek Pai
Rice Peter Druschel
UtahJay Lepreau
CMUSrini SeshanHui Zhang
UCSDStefan Savage
ColumbiaAndrew
CampbellICIR
Scott ShenkerMark HandleyEddie Kohler
http://www.planet-lab.org/
8/12/2002 IRB/XIS 30
Initial Planet-Lab Candidate Sites
Intel BerkeleyIntel BerkeleyICIRICIR
MITMIT
PrincetonPrincetonCornellCornell
DukeDuke
UTUT
ColumbiaColumbiaUCSBUCSBUCBUCB
UCSDUCSDUCLAUCLA
UWUW
Intel SeattleIntel Seattle
KYKY
MelbourneMelbourne
CambridgeCambridge
HarvardHarvard
GITGIT
UppsalaUppsalaCopenhagenCopenhagen
CMUCMU
UPennUPennWIWI
ChicagoChicagoUtahUtah
Intel ORIntel OR
UBCUBC
WashuWashu
ISIISI
IntelIntel
RiceRice
BeijingBeijingTokyoTokyo
BarcelonaBarcelona
AmsterdamAmsterdamKarlsruheKarlsruhe
St. LouisSt. Louis
8/12/2002 IRB/XIS 31
Approach:Service-Centric Virtualization
• Virtual Machine Technology has re-emerged for hosting complete desktop environments on non-native OS’s and potentially on machine monitors.
– ex. VMWare, ...
• Sandboxing has emerged to emulate multiple virtual machines per server with limited /bin, (no /dev)
– ex. ENSim web hosting
• Network Services require fundamentally simpler virtual machines, can be made far more scalable (VMs per PM), focused on service requirements
– ex. Jail, Denali, scalable and fast, but no full legacy OS – access to overlays (controlled access to raw sockets)– allocation & isolation
» proportional scheduling across resource container - CPU, net, disk– foundation of security model– fast packet/flow processing puts specific design pressures
• Instrumentation and management are additional virtualized ‘slices’
– distributed workload generation, data collection
8/12/2002 IRB/XIS 32
Hard problems/challenges
• “Slice-ability” – multiple experimental services deployed over many nodes
– Distributed Virtualization– Isolation & Resource Containment– Proportional Scheduling– Scalability
• Security & Integrity - remotely accessed and fully exposed– Authentication / Key Infrastructure proven, if only systems were bug free– Build secure scalable platform for distributed services
» Narrow API vs. Tiny Machine Monitor
• Management – Resource Discovery, Provisioning, Overlay->IP– Create management services (not people) and environment for innovation
in management» Deal with many as if one
• Building Blocks and Primitives– Ubiquitous overlays
• Instrumentation
8/12/2002 IRB/XIS 33
Emerging Extreme Internet
Wide-Area Broad-Coverage Services
Traditional pt-pt Internet Deeply-EmbeddedNetworks
8/12/2002 IRB/XIS 34
backup
8/12/2002 IRB/XIS 35
Mission for the Network of Labs
• Bold new form of Industry-University collaboration that reflects the changing nature of the information age.
• Conduct the highest quality research in emerging, important areas of CS and IT.
• Join the unique strengths of Universities and the company in concurrent, collaborative efforts that are both broad in scope and deeply penetrating in exploration.
• Operate in a uniquely open fashion, promoting a powerful, bidirectional exchange of groundbreaking ideas, technology, and people.
• Leadership role in the creation of new research ecosystems spanning the continuum from academic study to product development.
• Labs will be project-focused with an active, constantly evolving agenda involving Intel researchers, University researchers, and members of the larger research community
8/12/2002 IRB/XIS 36
Berkeley Focus
Extreme Interconnected Systems
• Invent, develop, explore, analyze, and understand highly interconnected systems at the extremes of the computing and networking spectrum - the very large, the very small, and the very numerous
• Do leading-edge Computer Science on problems of scale, cutting across traditional areas of architecture, operating systems, networks, and languages to enable a wide range of explorations in ubiquitous computing, both embedded in the environment or carried easily on moving objects and people
8/12/2002 IRB/XIS 37
Current Research Team
• Hans Mulder – co-director, IA64• Kevin Fall: UCSD, ISI, UCB,
NetBoost, Intel– high speed ip networking
• Alan Mainwaring: TMC, UCB, Sun, Intel
– virtual networks, deep scalable network systems
• Anind Dey: Georgia Tech, aware house
– framework for context aware applns, ubicom
• David Gay: UCB– Prog. Lang. design/Imp for novel
comm. layers
• Wei Hong, UCB, Illustra, Cohera, PeopleSoft
– Federated databases
• Su Ping: Intel– Software Engineering, embedded
systems
• Eric Paulos: UCB– HCI, robotics, ubicomp
• Timothy Roscoe: Cambridge, Sprint
– Operating systems, Distributed Computing, Infrastructure Services
• Brent Chun: UCB, CIT– cluster systems, resource
management
• Matt Welsh, UCB (Post Doc)– Operating Systems, internet service
design
• Phil Buonodonna, UCB (abd intern)
– Storage Area Networks, networks
• Silvia Ratnasamy, UCB/ICSI (abd)– Networking, P2P
• Justin Tomilson, Part Time– optimization, IEOR PhD Student
• Earl Hines – operations mgr
8/12/2002 IRB/XIS 38
Additional Researchers
• Joe Hellerstein, Faculty Consultant (next AD)– streaming database, sensor database, P2P
• Eric Brewer, Faculty Consultant – systems, language design
• Larry Peterson, Consultant/Sabattical
• Deborah Estrin, Faculty consultant– internet, multicast, rsvp,...sensor nets
• Paul Wright, Former Faculty consultant– infopad, BWRC, cybercut
8/12/2002 IRB/XIS 39
Current Faculty Research Associates
• James Demmel large-scale comp. sci• Michael Franklin Sensor Databases• Steven Glaser structural dynamics• Joe HellersteinStreaming Databases• John Kubiatowicz planetary storage• James Landay HCI• David A Patterson Architecture• Kris Pister MEMS, Smart Dust• Jan Rabaey Low power systems• Satish Rao Distr. Systems Theory• Ion Stoika Networking• Vivek Subramanian Disposable devices• David Wagner Security• Kathy Yelick Parallel Languages• Jennifer Mankoff HCI• Shankar Sastry Distributed Robotics