11/12/04cens seminar1 adopting ideas from interplanetary networking for sensor network applications...
TRANSCRIPT
11/12/04 CENS Seminar 1
Adopting Ideas from Interplanetary Networking for Sensor Network Applications
Andrew Parker, UCLA
Scott Burleigh, JPL
Richard Guy, UCLA
Deborah Estrin, UCLA
11/12/04 CENS Seminar 2
Interplanetary Network Vision
• Picture of sensor networks in space
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
* V. Cerf, InterPlanetary Internet, 2004
11/12/04 CENS Seminar 3
Why We Care
11/12/04 CENS Seminar 4
Outline
• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks
11/12/04 CENS Seminar 5
Mexico Deployment
• People: Paul Davis, Deborah Estrin, Richard Guy, Martin Lukac, John Wallace, Monica Kohler, Ramesh Govindan, Igor Stubailo, Allen Husker, Katie Mika, John Propst, Sam Irvine, Jeremy Elson, et al.
• Seismic array of 50 nodes• 5 km apart• Spanning 250 km• Through jungle, mountain,
and urban environments• Targeting end of Q1 2005
11/12/04 CENS Seminar 6
Data Story
• Every bit is important. Can’t lose data.
• 24 bits / sample, 100 samples / sec, 3 channels = 900 bytes/sec, 3.24 MB/hour, 1.2-1.8 MB/s compressed, per Node
• 1 GB Flash will hold 20 days of compressed data
• Conventional methods of retrieval
– Satellite uplink (expensive)
– Data mule (graduate student)
• Is there a better way?
CENS Data CommunicationsController
Guralp Seismometer
Q330 24 bit Digitizer
Stargate
11/12/04 CENS Seminar 7
A Better Way: Networking the Array
• Goal: Multihop data from the edge into Mexico City• Mexico deployment is different from typical sensor network
applications– Every bit of data is important
– Radio connectivity is directional • Due to distance, power, and regulations, we’re using directional
antennas
– Routing is over a linear topology
– Radio communication is not the greatest power consumer
11/12/04 CENS Seminar 8
Braided String
• Introduces some path redundancy• 802.11b
– 200 mw radio (SMC PCMCIA card)
• Yagi antenna– 20 dbi– 30 degree spread– Tested with 2-way splitter– 5 - 10 km in LA area– 4 Mb/s - 150 kb/s
• Parabolic antenna– 24 dbi– 5 - 10 degree spread– 4 way splitter– Tested to ~ 25 km– Closer to 1 Mb/s
11/12/04 CENS Seminar 9
Routing is Hard to Do Right
• Possible approach: Static Routes - topology is linear and nodes are immobile
• Problem: Still as brittle as a single string -- can’t back track around breaks
• Possible approach: Use AODV, DSR, Roofnet, etc.• Problem: Flapping links will make establishing end-to-end
routes nearly impossible
• What to do, what to do…?
11/12/04 CENS Seminar 10
Outline
• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks
11/12/04 CENS Seminar 11
Interplanetary Networking: Motes in Space
• Huge transmission delays– Several seconds to
the Moon
– 5 - 20 minutes to Mars
– 1 hour to Jupiter
– 7 hours to Pluto
• Lossy links• Long disconnects
– Sometimes predictable
11/12/04 CENS Seminar 12
Mars Communication
http://www.astrosurf.com/lombry/qsl-mars-communication3.htm
11/12/04 CENS Seminar 13
Mars Connectivity
QuickTime™ and aCinepak decompressor
are needed to see this picture.
11/12/04 CENS Seminar 14
Oops!
11/12/04 CENS Seminar 15
Properties of Interplanetary Networking
•End to end paths unreliable•Interactive / chatty protocols break•Need to communicate across varying network technologies, including non-IP networks
•Links are lossy and high delay–Very high round trip times,
–Even for single hop
•Links are often disrupted–Node mobility
–Node powered down
•Asymmetric links•Asymmetric node capabilities
11/12/04 CENS Seminar 16
Delay Tolerant Network Architecture
• Messaging service (Like Email) – Insulates applications from
network behavior
– Asynchronous, deferred transmission
– Non-interactive end-to-end
You’veGotDTN!
11/12/04 CENS Seminar 17
Delay Tolerant Networking
• Bundles are Basic Unit of Transport– Written as files onto persistent store
– Resilient across server / node restarts
• Custody Transfer of Bundles– Represents a QOS agreement: custodian tries real hard to transfer
custody and not delete until this has happened
– Conceptually moves the “end” of an end to end transaction
– Custody is asserted, rather than given.
• Overlay routing of Bundles among Custodians– Custodians may be separated by intermediate nodes.
11/12/04 CENS Seminar 18
Delay Tolerant, Not Run-Over-by-a-Bus Tolerant
• Bad things can still happen with Custody Transfer• Brings up the question of trust. When should a node trust
another more than it itself? • Risk vs. Resource trade-off
VS
11/12/04 CENS Seminar 19
Bundle Agents and Clients
• Client nodes register with Bundle Agents to send and receive Bundles on their behalf
– Email analogy: Pop or IMAP client connecting to a server– Based on the destination name, delivery is made to the corresponding
Bundle Agent.– At this point it’s considered to be done. Again, similar to Email.
• Naming scheme allows for late binding and separation of names from nodes
– Determination of which node(s) receive the message may be deferred
• Connects different domains via gateways– Late binding names allow different domains to communicate (through
gateways). – Connects loosely coupled “Internets”– Official DTN specification uses Tuples:
• (Interdomain Label, Opaque Intradomain Label)
11/12/04 CENS Seminar 20
Outline
• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks
11/12/04 CENS Seminar 21
DTN vs. Email
DTN? Email?What’s the difference?
Professor Culler
11/12/04 CENS Seminar 22
DTN vs. Email
Cool
Professor Culler
Biggist Difference: DTN Overlay
DTN is able to make progress towards the destination, even when no contemporaneous route exists
11/12/04 CENS Seminar 23
Outline
• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks
11/12/04 CENS Seminar 24
Mexico Deployment: Routing
• DTN has the “Tuple” address: (Interdomain, Intradomain)– Doesn’t really apply to Mexico
– Single domain
• Hybrid Routing Approach– You know where you want to get to, but not how
– Overlay static on top of mesh routing• Using Roofnet
– link-quality aware
– minmizes number of transmissions
– Use mesh routing to tell you the next hop towards the furthest reachable downstream candidate
11/12/04 CENS Seminar 25
Hybrid Examples
• Normal case• Simple break• Partition• Reconnect
11/12/04 CENS Seminar 26
Hybrid Examples
• Normal case• Simple break• Partition• Reconnect
11/12/04 CENS Seminar 27
Hybrid Examples
• Normal case• Simple break• Partition• Reconnect
11/12/04 CENS Seminar 28
Hybrid Examples
• Normal case• Simple break• Partition• Reconnect
11/12/04 CENS Seminar 29
Hybrid Examples
• Normal case• Simple break• Partition• Reconnect
11/12/04 CENS Seminar 30
Mexico Deployment: Handling Data
• DTN suggests exchanging Bundles (Files)• Divide data into one-hour segments• Augment with meta-data
– To, From, Data, Size, etc.
• 1.2 - 1.8 MB compressed• Store as files on disk (even on intermediate hops)• Easy to recover from server and node restarts• Human manageable
11/12/04 CENS Seminar 31
Node Software Architecture
BundleBundle
Bundle
Outbox
Duiker
Bundle Forwarder
Bundle Receiver
RoutingDataBundle Sender
Seismic Data(Q330)Upstream
Nodes
DownstreamNode
BundleBundle
Bundle
Inbox
Data FileData File
Data File
Drafts
CENS NODE
11/12/04 CENS Seminar 32
Managing Space
• Nodes are equipped with 2 - 4 GB of storage: 40 - 80 days of data
• A Bundle is deleted when ACK is received from sink for locally generated data– ACK is just another bundle
– Application-level ACK
• When space gets low– Node refuses to accept transient data (“route around me”). Better
than accept and drop.
– Delete transient data when space gets low
• No intermediate custody transfer• This policy delays the deletion of original data the longest
11/12/04 CENS Seminar 33
Transmission Priority
• Transmission priority affects performance– Want to avoid starvation
• Priority can be based on:– Age
– Transient vs. Local
– Number of hops traversed
– Number of times sent
– Some other measure of how hard it was to get this far
11/12/04 CENS Seminar 34
What about Off the Shelf DTN?
• There is an official Bundle specification
• There are 1 1/2 reference implementations of the Bundle specification
• Why not use it?
• Because…– Only reason why Mars worked, and only way Mexico will work, is because
scenario specific information was used. You MUST do this.– It’s impossible to do a good job in all scenarios– The more generic you try to be, the more it looks like flooding.– The more options and switches you support, the more brittle and complex it
becomes• See Sendmail configuration file
• How do we remain flexible and customizable, yet stay SIMPLE?
11/12/04 CENS Seminar 35
A Modular DTN Software Architecture
• Like Click
• Modular software IP router
• Easy to plug modules together
• Easy to create new modules
• Results in an optimized router
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
// Declare three elements…src :: FromDevice(eth0);ctr :: Counter;sink :: Discard;// .. Connect them togethersrc -> ctr;ctr -> sink;
11/12/04 CENS Seminar 36
Click’s Elegance
• Click modules consume and produce the same data structure
– IP packets
• Same is nice and simple. Easy to compose modules if they input/output the same thing
• IP is nice too (as opposed to something obscure or too generic)
– Strong foundation in specifications
– Many compliant implementations as examples
– People are already familiar with certain features and behaviors of IP routers
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
11/12/04 CENS Seminar 37
Click and DTN - High Level
• IP packets• IP specifications• Lots of existing IP routers• Lots of specific IP
features/behavior to choose from
• Bundles• Bundle specifications
– Experimental and evolving
• Very few Bundle servers• Lots of Bundle features and behavior
to choose from– In this area, Bundles have a richer
set of potential behavior than IP
Click DTN
11/12/04 CENS Seminar 38
Introducing this Approach to Sensor Networks
• EmStar– Component based framework for sensor network applications (on Linux)– Lots of services and applications already existing– Simulation / Emulation / Deployed Modes– Heterogeneous applications (Stargates / Motes)– Debugging, monitoring, and visualization– Growing user-base
• UCLA, USC, MIT, Umass Amherst, Ohio State, etc.• Intel Research, Xerox PARC, Microsoft Research
Emulation Array
Node001 …
Emulation Mode
HostMote Protocol
Tra
nsce
iver
Mot
e
Tra
nsce
iver
Mot
e
Tra
nsce
iver
Mot
e
Tra
nsce
iver
Mot
e
MN MN MN MN
Node002
Node003
NodeN
Simulation Frameworkwith real RF channels
Visualization Tools
Client Server
kfusd.o
/dev/fusd/dev/servicename
Kernel
User
Robust multi-process,microkernel architecture
11/12/04 CENS Seminar 39
Relationships Between EmStar Modules Unclear
How to bring Click’s readability and ease-of-use to EmStar?
include link/link.run &link_udp(udp0); &link_linkstats(udp0,ls0,show="leds:core"); &link_neighbors(ls0,show="hide"); process mdiff { type = once; noclean; cmd = "devel/microdiff/mdiff --uses ls0"; waitfor = ls0; }
process mdiff_test { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_test"; } process mdiff_filter { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_filter"; }
Example EmRun Configuration File
11/12/04 CENS Seminar 40
Bolt: Architecture Description Language for EmStar
• Makes relationships between modules of an EmStar application explicit
• Began as a class project with Eddie Kohler and Todd Millstein last spring
• Implemented in CIL and Ocaml
11/12/04 CENS Seminar 41
udp0 = udp() ;ls0 = linkstatsd();nd = neighbord();mdiff = Mdiff();
mdiff_simple_app [packet_dev] <-> [app_packet_opts] mdiff;mdiff_simple_filter [packet_dev] <-> [filter_packet_opts] mdiff;
mdiff [lu_opts] <----------------------> [lp_opts] ls0 [lu_opts] <-> [opts] udp0;mdiff [sc_opts] <- [s_opts] nd [ls_opts] <-- [s_opts] ls0; nd [lu_opts] <-> [lp_opts] ls0;
Bolt Config.: White Board to Emacs Buffer
Instantiating componentsData FlowConnecting component devices
11/12/04 CENS Seminar 42
BUT WAIT, THERE’S MORE!…
11/12/04 CENS Seminar 43
Click Modules vs. EmStar Modules
• Click modules are C++ classes• EmStar modules are separate processes• Click modules communicate via function calls• EmStar modules communicate by passing bits or text via
device files• Since Click modules are compiled, they benefit from basic
compiler checks and optimizations. Click itself does higher level checks and optimizations
• Wouldn’t it be nice if EmStar modules enjoyed similar benefits?
11/12/04 CENS Seminar 44
Bolt: Compiler-like Analysis and Optimization for EmStar
• It Slices! It Dices!
• It does static analysis and optimizations not otherwise possible across EmStar modules
• No user changes or annotation required of EmStar code– Though it could help
• Bolt statically infers type-safety violations between two functions in different processes (EmStar modules)
– Despite type-obscuring function calls and heavy use of function pointers
• Bolt uses subgraph isomorphism to identify and swap out combinations of modules with more efficient combinations that perform the same function
• Analysis not completely sound nor complete, but hard given the circumstances
11/12/04 CENS Seminar 45
Implementing DTN Using Bolt
• DTN looks like any other set of EmStar modules
• What exactly are the EmStar modules that make up the DTN suite?– Click has benefited from the existence of numerous IP router
implementations– DTN must build up experience from real deployments (Mexico and others)– Identify and extract reusable DTN related components, for example:
• Managing a large number of concurrent file transfers between a pair of nodes
• Despite large delay and disconnections
• Recover across restarts
• LTP (S. Burleigh) & File Mover (A. Parker)
– Goal is not necessarily to be able to build a complete DTN solution for your particular situation
– Rather, to reuse the ones that make sense, and build from there
11/12/04 CENS Seminar 46
Outline
• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks
11/12/04 CENS Seminar 47
References
• DTN: http://www.dtnrg.org• IPN: http://www.ipnsig.org• Click: http://www.pdos.lcs.mit.edu/click• EmStar: http://cvs.cens.ucla.edu/emstar• Bolt: http://lecs.cs.ucla.edu/~adparker/Bolt