MaDMAN: A Middlewarefor Delay-Tolerant Mobile

Ad-Hoc Networks

TR-UTEDGE-2010-011

Agoston Petz Alex Bednarczyk

Nicholas PaineDrew Stovall

Christine Julien

The University of Texas at AustinMobile and Pervasive Computing Group

© Copyright 2010The University of Texas at Austin

MaDMAN: A Middleware for Delay-TolerantMobile Ad-Hoc Networks

Agoston Petz, Alex Bednarczyk, Nicholas Paine, Drew Stovall, Christine JulienMobile and Pervasive Computing Group

University of Texas at Austin{agoston, bednarczyk , npaine, dstovall, c.julien}@mail.utexas.edu

ABSTRACTAs delay-tolerant networks (DTNs), dynamic networks inwhich nodes are not always connected to one another, gaintraction in both the research community and in real-worlddeployments, research must explore how DTN protocols cancoexist with traditional mobile ad-hoc networking proto-cols. This is a requirement for future deployments in whichchanging networking conditions dictate that delay-tolerantnetworking protocols be available, but may not always be thebest choice given dynamic environmental conditions. In thispaper we present an adaptive middleware (MaDMAN) fordelay tolerant networks that seamlessly and dynamically mi-grates applications’ communications sessions between dif-ferent network and routing protocols in response to changingnetworking conditions without dropping the connection orbreaking application-layer connectivity. We present the ar-chitecture of our middleware, an implementation of it withinthe Click Modular Router, and provide an analysis using areal-world mobile ad-hoc delay-tolerant network comprisedof autonomous robots with x86 hardware and commodity802.11 wireless cards.

Categories and Subject DescriptorsC.2.1 [Network Architecture and Design]: Storeand forward networks

General TermsDesign, Experimentation, Performance

KeywordsDelay Tolerant Networks, Adaptive, Architecture, Click,Mobile Computing

1. INTRODUCTIONSince the connectivity among applications on devices

in a delay-tolerant network (DTN) is widely varying andunpredictable, it is necessary to integrate DTN solu-tions with traditional mobile computing solutions to en-sure the availability of the best communication supportpossible at any given moment. Applications executingin DTN environments would like their connections to

be supported by the network technology best suited tothe combination of the communication session’s require-ments (e.g., for throughput and delay) and the instan-taneous network context. In this paper, we describe amiddleware for delay-tolerant mobile ad-hoc networks(MaDMAN) that enables a symbiotic network architec-ture in which applications’ connections can seamlesslymigrate from one communication style to another in re-sponse to changing network or application conditions.

DTNs are networks of intermittently connected nodeswhose topologies are subject to constant change dueto mobility or adverse networking conditions. Becausethe endpoints of a communication session in a DTNmay never be connected in the traditional sense, com-munication in a DTN is often supported through theferrying of messages by mobile nodes that opportunis-tically encounter sources and destinations. In a sense,DTNs are an extreme variant of mobile ad hoc networks(MANETs). In the latter, high-levels of node mobilitycause the network’s routing topology to change, but ap-plications in MANETs can generally assume that someconnected path exists from the source to the destinationat any point in time. Communication in DTNs, on theother hand, often relies on temporal connectivity—theidea that a source node may encounter a node that will,in the future, encounter the destination.

DTN architectures potentially apply in many appli-cation instances. In an urban setting, commuter trans-portation systems can be used to carry messages fromone region of a city to another [6]. A remote village maybe well-connected internally but less reliably connectedto the wider world [23]. In a military network, deviceswithin a post or base camp may have good connectivityto one another but may be connected to devices in otherregions only by roving UAVs or convoys that relay mes-sages from time to time. Researchers moving amongtagged herds of zebras can collect and share informa-tion about zebra behavior and movement [10]. Pocketsof well-connected sensor networks can be periodicallyconnected via roving mobile robots [22].

Figure 1 shows an abstraction of the common oper-ating environment shared by these disparate applica-

1

Region 2

A

BRegion 1

Region 3

Figure 1: A generic DTN target environment

tions. The figure shows three well-connected regionsthat are interconnected transiently via delay-tolerantlinks (dashed lines in the figure). In the deployments de-picted, nodes can break off from the well-connected re-gions and transit between them (such as node A, whichis transiting between regions 1 and 2, and node B, whichhas just arrived at region 3).

Research issues relating to routing and understandingnetwork topologies are well-studied [3, 8, 13, 16], andgreat strides have been made with respect to support-ing the new paradigm of mobile computing that DTNsexhibit. In this paper, we study a more practical sys-tems issue, namely the ability of networked devices inDTNs to adapt their connections to the changing op-erating environment. While previous work discussedin Section 2 has investigated choosing the best imple-mentation strategy for a communication session at itsinception, the more difficult problem of adapting an on-going communication session is largely unexplored. Sev-eral aspects make this a challenging problem, includ-ing 1) providing seamless transitions for applications(i.e., an application should not need to be aware of thechange in underlying implementation); 2) enabling in-telligent use of cross-layer context information to adaptthe communication implementation only when it makessense and as soon as it makes sense; and 3) exchangingstate among vastly differing communication strategiesto ensure the lowest overhead, highest throughput per-formance possible.

To address these challenges, we introduce the MaD-MAN middleware architecture that allows one networkstack to be dynamically swapped in for another with-out experiencing (from the application level) a disrup-tion in connectivity. The middleware model we presentin Section 3 builds on our initial effort in such a dy-namic DTN architecture [25] and allows applications’

ongoing connections to seamlessly migrate between twodifferent communication stacks: a traditional mobile adhoc networking stack (for use in well-connected regionssuch as those shown in Figure 1) and a dedicated DTNstack (for use in transient communication situations).In defining the MaDMAN middleware model, we focuson the mechanics of this transition and the systems andcommunications issues that must be resolved to enableseamless transitions.

We have implemented MaDMAN’s stack-switchingcapabilities using the Click modular router [11]; we de-scribe this implementation in detail in Section 4. Theuse of Click will enable future integration with a widevariety of network context measurement capabilities thatcan influence MaDMAN’s understanding of the dynamicenvironment; this added intelligence will enhance MaD-MAN’s ability to respond to changes in its environ-ment. We have evaluated this implementation usingthe Pharos testbed [32], which includes a set of au-tonomous robots that execute the MaDMAN middle-ware while they move in a DTN (Section 5). The MaD-MAN middleware we present in this paper representsan essential step in realizing the integration of emergingdelay-tolerant networks with existing mobile computingcapabilities in actual deployments.

2. MOTIVATIONIn this section, we explore the motivating conditions

for the use of delay-tolerant networks in more detail.We then look at approaches that have explored similarproblems, using this work as a springboard to preciselydefine the problem MaDMAN seeks to solve.

2.1 A Motivating ScenarioDelay-tolerant networks are relevant to a wide variety

of applications. To further explore the requirements ofapplications in DTNs, we describe a particular examplethat fits the model shown in Figure 1—that of usingDTNs to connect remote villages to each other and tothe Internet through distant hubs [5, 23]. In such anapplication, devices are deployed within small remotevillages (e.g., via the one laptop per child project [18])where the available infrastructure is severely limited.Within the village, the devices have good connectiv-ity among themselves, supported by traditional mobilead hoc networking protocols or other wireless grid ap-proaches. Connectivity to the wider world, however, issignificantly more difficult; without infrastructure net-works, long-range wireless communication is too energyintensive for the villages’ constrained resources. How-ever, people transit between villages and larger citiesto deliver goods and supplies and to visit family andfriends. Some of this movement is on a predictableschedule; other movements are more spontaneous. Byleveraging the storage and communication capabilities

2

of devices traveling with people among villages, com-puter users in remote areas can achieve access to thewider networked world.

This form of network access, however, is not of thesame nature as the Internet access most people en-joy. Connections are asynchronous, demanding an en-tirely new network architecture to support these de-vices. Delay-tolerant networks have emerged as the ob-vious candidate; in such networks, applications that cantolerate significant latencies can be supported even un-der the duress of the conditions described above. How-ever, much work remains to be done to provide applica-tions the same kind of abstractions they are used to intraditional networks given this vastly different underlay.

2.2 Related ApproachesOur overarching goal is to provide the semblance of

a communication session in a delay-tolerant network,even when no end-to-end connection is possible. Inaddition, we aim to maximize application performanceby dynamically selecting the best connection approachbased on applications’ requirements, the communicat-ing parties, and the network conditions.

Existing application semantics commonly stress a viewof end-to-end connectivity that may be unreasonablein dynamic or unpredictable environments. Transportlayer extensions have been developed that enable end-to-end connectivity in cases when it would otherwisebe impossible, for example by predicting when link fail-ures will occur in mobile networks and mitigating theirimpact [2] or by using dynamic DNS updates to sup-port TCP connection migration [31]. These approachesare not suitable for the types of networks envisioned indelay-tolerant networking as applications’ connectionsstill timeout if they experience extended disconnections.Other approaches use proxies as delegates to shield ap-plications from disconnection [15, 20]. Such approachesassume every application will eventually reconnect toa particular central service, making them unsuitableto the highly dynamic and unpredictable delay-tolerantnetworking environment, where a transitive connectionmay never exist between two communicating parties.

In response to the need to evolve network architec-tures to suit emerging applications, architectures thatmaintain interfaces for existing applications but exposenew functionality have been developed. The Over-lay Convergence Architecture for Legacy Applications(OCALA) [9] defines an overlay architecture that al-lows existing (legacy) applications to continue to func-tion, even if the underlying network architecture andimplementation are fundamentally changed. OCALAis similar in spirit to our approach but offers a differentend goal, i.e., to enable evaluating new network over-lays’ support of existing applications. The Overlay Ac-cess System for Internetworked Services (Oasis) [14] also

attempts to enable legacy applications to continue tofunction on top of overlays that offer more expressive in-terfaces to newly introduced applications. Oasis insertsa layer that manages available overlays for applications,allowing the “best” performing overlay to be dynami-cally selected at the beginning of a connection. The Bil-liards framework creates a notion of a communicationsession using information about the application’s com-munication requirements and the instantaneously avail-able network resource and context information (suchas node mobility, network queue lengths, etc.) In con-trast to OCALA, Oasis, and Billiards, applications suchas the one described previously require the ability forthe network architecture to dynamically reevaluate thechoice of underlying implementation, adapting not onlyto the application’s stated requirements but also to thechanging network situation and the changing networkpath from source to destination. The fundamental dif-ference is one of changing the communication support ofan ongoing communication session in addition to select-ing the best technologies when the session is initialized.

Some DTN solutions define an API that allows appli-cations to choose “traditional” end-to-end connectionsor DTN-based communication protocols [24]; this choicecan be based on information about the availability ofend-to-end connectivity [21]. This approach intimatelyintertwines the two communication approaches, whichhas the potential to significantly increase the overheadof communication. The Haggle project identified aset of architectural principles underlying the design ofpocket-switched networks, which encounter many of thesame challenges as DTNs [29]. Haggle constructs anunlayered network architecture that focuses on applica-tion messages and requirements, enabling applicationsto be communication protocol agnostic [33]. Applica-tions running on Haggle hand off application data unitsto the middleware to be forwarded by the best pos-sible forwarding algorithm for the particular messageat the particular time. In Haggle the redesign of thenetwork “stack” requires revisiting protocol implemen-tations and specifically tailoring them to be includedin Haggle (the prototype described in [33] includes “di-rect” and “flooding” communication). We argue thata layered approach to integrating multiple communica-tion approaches makes it easier to add new functionality“off-the-shelf.”

ParaNets matches our vision, defining a protocol treeinstead of a single stack in which the application, asthe root of the tree, can be supported (potentially si-multaneously) by multiple stacks with different capa-bilities and characteristics [7]. We argue that not onlyshould these networking technologies be run in paral-lel but they should be capable of being dynamicallyswapped in support of a single application’s long-livedconversation, complete with state commonly associated

3

with network communication sessions. Our approachis also similar in spirit to the MANETKit project [28],which componentize the behavior of MANET routingprotocols and allows multiple protocols to execute in asingle network in parallel. MANETKit allows applica-tions in MANETs to dynamically switch the protocolused to support a communication task based on the ap-plication’s operating conditions.

2.3 Problem DefinitionDesign considerations relating to the impact of a delay-

tolerant network environment on end-to-end applica-tion semantics have begun to be explored [19]. Wemove a step further by looking at a specific technicalchallenge within these new requirements; specifically,we enable the underlying network implementation totransition intelligently between a traditional MANETimplementation that can support TCP and a less re-liable DTN implementation that cannot. The goal isto provide a single “session” interface to the applica-tion and dynamically adapt the connection based on1) the conditions and quality of the network and thepath(s) to the destination and 2) the application’s re-quirements and behavior. Existing applications shouldbe able to use the resulting network API without mod-ification; “DTN-aware” applications should be able totailor their interactions to take full advantage of theadded, context-aware functionality.

In realizing our goals, we make a few assumptions.We delegate destination discovery to the underlying net-work protocols (be they traditional MANET routingprotocols or DTN routing protocols). We instead focuson the systems issues involved in being able to dynam-ically and robustly switch an ongoing connection fromone style of communication to another. In the remain-der of this paper, we will speak mostly in terms of a sin-gle “application,” which comprises two endpoints andthe (potentially dynamic) connection between them. Inrelation to Figure 1, interactions between devices lo-cated in the same “region” can be serviced by morereliable end-to-end connections supported by TCP andMANET routing; interactions between devices locatedin different regions will be serviced by the DTN sup-port. As nodes move between regions, their connectionsto other nodes will adapt by transitioning the commu-nication session from one modality to another. Thismigration occurs transparently to the application; on-going communication is not interrupted (although itsquality of service may change).

3. MADMAN MIDDLEWARE MODELTo enable the dynamic selection and reselection of

protocol support for applications in intermittently con-nected delay-tolerant networks, we define MaDMAN, amiddleware for delay-tolerant mobile ad hoc networks.

MaDMAN’s architecture is based on the idea that thedynamic environments that define delay tolerant net-works are best utilized by a combination of traditionalmobile ad-hoc network protocols and delay tolerant net-work protocols. Furthermore, in Section 5 we demon-strate that the appropriate choice depends not only onthe properties of a particular communication sessionbut also on the dynamic operating conditions. This isbecause different communication styles exhibit vastlydifferent performance under differing network condi-tions. We have architected MaDMAN to adapt tochanges in the network and to swap the protocol struc-ture supporting communication dynamically, even inthe middle of an open application session, without drop-ping the application session. In essence, the middlewareis composing a possibly unique and dynamic “networkstack” for each connection based on network context, al-lowing applications to maximize their throughput usingtraditional protocols when networking conditions aregood (e.g., when end-to-end delay for the connectionis low), and to continue to function even when network-ing conditions are poor (e.g., when end-to-end delay ishigh because of disconnections due to node mobility oradverse channel characteristics).

Due to the continuous emergence of new and bet-ter approaches to communication, it is also essentialthat MaDMAN enable simple and intuitive integrationof new physical, media access, network, transport, andapplication layer protocols, without requiring these ap-proaches to be redesigned or reimplemented to fit insidethe middleware. To this end, we designed MaDMAN tooperate within the Click Modular Router [11], a flexible,modular, and stable framework for developing routers.Click has a large base of available network protocolsand is itself a good framework for implementing exper-imental protocols. We further discuss our reasons forselecting Click in Section 4.1

Figure 2 shows the components of the MaDMANarchitecture. The grayed components in the figure(with dotted outlines) are components that already ex-ist within the Click implementation. The main com-ponents of the middleware are the Context Aggregatorand the Session Manager. The former handles MaD-MAN’s understanding of the operating environment’sconditions and presents the context information to theSession Manager, which uses the context informationto dynamically select and reselect the best communi-cation strategies. MaDMAN’s Session Framework in-cludes its various application interfaces and the SessionManager itself, which manages all of the application ses-sion resources. The application interfaces themselves(Socket Interface, Bundle Interface, etc.) implementthe inter-process communication between applicationsand MaDMAN (accomplished with the help of variousAPI libraries). In the remainder of this section, we pro-

4

App App …dl l

App App …pp pp

Socket APIBundle Protocol

Bundle InterfaceSocket Interface

Userspace

Bundle InterfaceSocket Interface

Queue

tor

Session FrameworkQueue

Session Manager

…

ext  Ag

grega

Transport/Network FrameworkUDPDTNTCP …

IP AddressingClick

UDP

EpidemicBack PressureAODV IP RoutingCo

nte

Routing Framework…BackPressureMAC802.11 … Data Link Framework

Physical NetworkKernel + Hardware

y

Figure 2: The High-Level MaDMAN Architecture

vide a conceptual overview of MaDMAN’s functionalityin four parts: 1) the application interface, 2) the con-nection logic, 3) the transport, network, and routingcomponents, and 4) the context aggregator.

3.1 Application InterfacesMaDMAN’s application interface consists of several

separate components (Socket Interface, and Bundle In-terface in the figure). These serve various types ofapplications, allowing non-Click code to interface withthe Click-implemented MaDMAN modules. For exam-ple, the Socket Interface allows application program-mers access to MaDMAN through the use of a stan-dard user library (SocketAPI in Figure 2) that providesa socket-style interface in the form that network ap-plications programmers are accustomed to. The Bun-dle Interface provides an interface specific to the bun-dle protocol and serves as a convergence layer for thebundle protocol to allow interoperability with emerg-ing delay-tolerant network applications [30]. These ap-plication interfaces and their associated API librariesexpose a small amount of context acquisition to theapplication programmer to facilitate the developmentof delay-tolerant “aware” applications. Through thisinterface, applications can optionally provide data pri-ority information as context, which MaDMAN’s con-nection intelligence can use to guide decisions to allo-cate resources to the connection and to optimize theavailable resources among connections. MaDMAN canalso selectively schedule low-priority connections on a

lower reliability network stack to free up resources forhigher priority connections. Applications can also pro-vide information about future bandwidth requirementsand the application’s desires for rescheduling transmis-sions to further help optimize data delivery. The MaD-MAN application interface includes a SOCKS proxy toenable legacy applications to entirely bypass the SocketAPI user library [12]. While this prevents these ap-plications from sharing context information with themiddleware, it enables reuse and interoperability.

3.2 Connection LogicThe connection logic lives inside of the Session Man-

ager in Figure 2. As part of the Session Framework,it sits above the transport layer and is responsible forengaging the best combination of lower-layer protocolsto use for a given connection. The module then passesthe application data along to the correct combinationof Click modules to establish communication. The Ses-sion Manager tracks each connection, demultiplexinginbound data to the correct application stream. It alsoreconfigures the protocol modules used to support eachcommunication session when changes in the environ-mental or network context dictate an adaptation. Theintelligence that uses context information to determinethe best combination of network modules to use at anygiven time resides in the Context Aggregator (describedbelow), but the Session Manager handles the mechanicsof closing and opening remote connections and main-taining and sharing state among protocols’ queues.

5

3.3 Transport, Network, and RoutingMaDMAN’s network stack, in the traditional sense,

consists of all possible transport, network, routing, anddata link layers available on a device. There are manypossible compositions of these elements into a workingstack, only some of which are functional and useful. Thepossibilities are enumerated by the connection manager,and the elements that can work together to constitutevarious stacks are determined before MaDMAN appli-cations run. Adding new functionality in these layerssimply requires adjusting this enumeration, which is dis-cussed further in Section 4.2.5. A validation frameworkfor checking the composition of network stack compo-nents is left for future work; MaDMAN currently relieson the user to define valid combinations.

MaDMAN’s key differentiator is its ability to exchangethe network stacks in the middle of an application’scommunication session. To support this, MaDMANtransfers state between various transport protocols tomaintain connection state in the transition process. Forexample, well-connected networks will rely heavily onTCP connections for reliability and its throughput scal-ing characteristics. When the connectivity becomes lessreliable, MaDMAN will exchange this TCP connectionfor a less reliable delay-tolerant connection based on, forexample, epidemic routing. While this exchange losesTCP’s reliability guarantees, MaDMAN does maintainsome of the TCP connection’s state (e.g., the last ac-knowledged data packet, the TCP window size, the cur-rent sequence number). This information is made avail-able through the Session Manager to the other com-munication protocols. Conversely, when the connectionmigrates back to the reliable network stack, similar con-text information can be used to bootstrap the new TCPconnection with a buffer that reflects the data that wasdelivered while TCP was unavailable. We envision cre-ating a datagram protocol for delay-tolerant networksto achieve these goals (dubbed UDPDTN in Figure 2), butits design and implementation are beyond the scope ofthis paper. The decision to switch from one stack to theother can be initiated by either end of the connection.It is a local decision per node, and thus we can end upwith nodes communicating using asymmetric protocolstacks. One node could use a certain combination ofprotocols to transfer data to another node, but receiveits responses using an entirely different set. The Ses-sion Manager is responsible for sorting the data into asingle stream to send to the application. Out-or-orderdelivery is a question of policy for the Session Managerand can be selected per application session.

3.4 Context AggregatorThe Context Aggregator is a cross-layer component re-

sponsible for gathering network performance data fromthe various elements in the network stack. Examples

include node connectivity, mobility statistics, instanta-neous and averaged throughput, latency, and protocol-specific parameters (e.g., TCP window size). The Con-text Aggregator also contains decision processes that usethis context information and the aforementioned config-uration file to determine the best combination of low-level modules to use for the applications’ active con-nections. The Context Aggregator’s intelligence can bebased on complex user-specified policies dependent onthe wide variety of available context information; forthe purposes of demonstrating MaDMAN in this pa-per, we rely on a simple timing based decision processdescribed in Section 4.2.4.

4. MADMAN IMPLEMENTATIONWe have implemented the model described in the pre-

vious section using the Click modular router. In thissection, we describe our implementation in detail, be-ginning with some background information on Click.

4.1 BackgroundThe MaDMAN middleware, with the exception of

the external application library, is implemented as acollection of Click elements within the Click ModularRouter [11], whose software architecture is well suitedto our goals of adaptability and dynamic reconfigurabil-ity. Click is written in C/C++, runs on Linux and BSD,and includes numerous components to manipulate thenetwork stack from the hardware drivers to the trans-port layer. This frees us from re-implementing commonprotocols yet offers the flexibility of easily swappingin alternatives or modifying existing protocols withoutbreaking the host operating system’s network layer.

Click “routers” consist of elements with explicitly de-fined input and output ports that are “wired” togetherby configuration files to define a network stack. MaD-MAN runs entirely within Click and is thus composedfrom Click elements using a configuration file. Figure 3shows the configuration of MaDMAN elements in ourcurrent implementation. Every element, with the ex-ception of the context aggregator, operates on packetsmoving up or down the stack. Adjacent modules areconnected to each other and can pass packets back andforth. Where drawn, arrows explicitly define the direc-tions in which packets can move; solid arrows representthe passing of packets between modules, and dotted ar-rows represent element handler calls between elements.For the purposes of this paper, we have defined two sep-arate network stacks, a standard TCP/IP stack and adelay-tolerant stack. In the remainder of this section,we describe these MaDMAN middleware components.

4.2 Middleware ComponentsFigure 3 highlights the specific Click middleware com-

ponents we implemented to demonstrate and validate

6

Source

Socket InterfaceQueue

Session Manager

Connection Manager

Epidemic

UDP/IP Encap

DTN Stac

TCP Socket

TCP Magic

TCP Stac

Beaconer

Aggregator

/ p

Ether Encap

Wifi Encap

ck C ag c ck

Context  

Wifi

Classifier

Figure 3: MaDMAN Middleware Components

the MaDMAN middleware. In comparison to Figure 2,this figure explicitly separates the elements of the TCPStack and the DTN Stack. In addition, the functionalityprovided by the components labeled Session Frameworkin Figure 2 is collected into a single Click element, theConnection Manager. We discuss these components,the Context Aggregator, and the wiring of MaDMANelements in the remainder of this section.

4.2.1 TCP StackOur TCP stack uses Click’s Socket element to open

and close standard TCP/IP connections. Although wewould have preferred composing a TCP stack out ofClick elements to enable reading and writing of internalTCP state, these modules are not currently available inthe standard Click distribution. Our TCP connectionsare handled by the Linux kernel’s TCP implementation,and, as a result, MaDMAN’s TCP stack is the stan-dard Linux TCP stack. Since we needed to collect stateinformation from the TCP connections (window sizes,sequence numbers, acknowledgments, etc.), we wrotea Click element (TCP Magic) that parses incoming andoutgoing TCP packets and provides state information tothe Connection Manager. TCP Magic is a passive ele-ment and does not affect the TCP connections. To trackpacket deliveries, TCP Magic correlates incoming TCPACKs with outbound data packets to determine howmuch of the outbound data was delivered successfully.This information is used by the Connection Managerto avoid sending data that has already been transferredover an old connection. Connections operating on theTCP stack use standard IP Routing. This stack pro-

vides the reliable connectivity that, in our conceptualmodel, is used when a source and destination have anactual MANET path connecting them (e.g., betweendevices in the same region in Figure 1).

4.2.2 DTN StackThe delay-tolerant stack consists of our implementa-

tion of epidemic routing [34], a popular routing pro-tocol for delay-tolerant networks given its effectivenessand simplicity. In epidemic routing, when two nodesconnect, they share message digests that contain in-formation about the packets each node carries. Nodesexamine the digests and specifically request the pack-ets they have not yet received. In this way, packetsare spread around as nodes come into contact with oneanother, and mobile nodes can thus ferry data to thedisconnected portions of the network as they move.

Our implementation consists of two Click elements,Beaconer and Epidemic. Beaconer sends and receivesapplication layer beacon packets on a user-configurableinterval to build a picture of which other nodes are cur-rently connected. The beacon packets are broadcastvia UDP, and Beaconer issues responses to establishwhether functional two-way communication is possible.When it is, a node considers the remote node connectedand makes this information available to the ConnectionManager via a handler call. When Beaconer missesa set number of beacons in a row (given by a user-configurable time-out interval), it logically disconnectsthe node. This is our first attempt at an intelligentconnection protocol; we plan to experiment with MAClayer beacons to accomplish the same thing but with-out the overhead of sending extra packets. However,this does have the advantage of establishing application-layer connectivity, and it provides a better guaranteethat when a node is logically connected, it is actuallycapable of sending and receiving packets.

The Epidemic element has its own internal queueof packets and holds on to all of the packets it re-ceives in case it encounters a node that has not yetreceived them. When Beaconer discovers a new con-nection, the Epidemic element initiates a message ex-change. For the duration of the connection, any newdata that the node creates is directly delivered to allof the connected nodes, in addition to being bufferedfor possible delivery to new contacts. In the future, wewill add “immunity”-style delivery confirmations [17]and data freshness timeouts to intelligently clear un-necessary data from the network and help free cacheresources. The delay-tolerant stack encapsulates all ofthe epidemic routing packets in UDP/IP (which we areusing for the time being until we develop or have accessto a proper delay-tolerant datagram protocol as men-tioned in Section 3.3).

7

4.2.3 Connection ManagerThe Connection Manager implements the logic de-

scribed in Section 3.2 and is responsible for multiplexingincoming and outgoing connections to the right networkstack. It relies on cues from the Context Aggregatorto know when to swap connections between stacks. TheConnection Manager tracks delivered data for all of itsconnections, which it uses to “prime” new stacks sothat they continue delivering data where the previousstack left off. In our first iteration of MaDMAN, theConnection Manager has two possibilities available toit: the TCP stack and the DTN stack described above,although we foresee having many combinations of lowerlayer protocols. When triggered to swap from the TCPstack to the DTN stack, the Connection Manager startssending data to the epidemic routing protocol’s databuffer at the point where TCP last received an ACK.When triggered to swap back to TCP, the solution issomewhat trickier since epidemic routing has no deliv-ery confirmations. Instead, the Connection Manageron the receiving end sends a re-establishment messagewith the sequence number of the last received epidemicpacket, thus allowing the remote node to restart theTCP connection where the DTN stack left off. Thereis a possibility here for data loss since epidemic routingdoes not provide guaranteed delivery, but should a re-liable delay-tolerant transport protocol be desired, themissing data chunks are enumerated by the ConnectionManager and could easily be requested from the sourceso long as it is still buffering the data.

4.2.4 Context AggregatorIn our current implementation, the Context

Aggregator is a simple element that provides cuesfor the Connection Manager to swap from the TCPstack to the DTN stack on a user-provided time-basedtrigger schedule. This first iteration of the ContextAggregator does not collate network statistics or pro-vide an intelligent stack swapping algorithm (hence thelack of handler connections to it in Figure 3), though itis sufficient for the purposes of demonstrating the fea-sibility of our approach. When swapping from DTNto TCP, the context aggregator waits for available con-nections. When the Beaconer element reports a directpath to the destination, the Context Aggregator initi-ates a swap back to the TCP stack. A better algorithmwould be to trigger the switch from TCP to DTN assoon as the average throughput drops below a certainfigure, based on application requirements and data gen-eration rate; the exploration of such smarter connectionswitching algorithms remains for future work.

4.2.5 Wiring of MaDMAN Elements

Click itself takes a router configuration file as inputto determine the types and wiring of all of the elementsthat comprise a “router,” or in our case, the MaDMANmiddleware. The advantage is that new configurationsare very easy to test, and functionally identical modulescan be swapped out without recompiling. For example,in Figure 3 the {TCP Socket / TCP Magic} combina-tion can be swapped for a plain Unix Socket should theuser wish to test higher layer functionality using a localdata stream instead of a full network stack. The changeinvolves two lines of the router configuration files whichmust be changed from:

source–> Socket()–> TCPMagic()

to:

source–> UnixSocket()

Click checks the gates of all elements to make sure thatall gates are connected and nothing is connected twice;nevertheless, it is possible to wire elements together inan order that Click will allow, but that will not func-tion as intended. A validation framework for checkingMaDMAN configurations is left for future work.

5. EVALUATIONTo evaluate MaDMAN, we used a combination of

emulation in a tailor-made wireless network emulatorand actual execution of the MaDMAN middleware ona mobile computing testbed. The former allowed usto determine the feasibility and potential of the MaD-MAN middleware in a controlled environment; the lat-ter demonstrates that these benefits transition to realworld wireless environments. In this section, we presentthese results; we present the emulation studies first1 andthen describe the testbed studies in detail.

5.1 A Feasibility Study in EmulationOur initial emulation study was designed to demon-

strate the feasibility of dynamically exchanging the com-munication technologies underlying an application’s com-munication session and to gauge the potential benefits.To study these effects, we constructed an emulator simi-lar to NIST Net [4] that allows us to introduce arbitrarydelays and packet drops. The “network” is effectively asimple delay buffer that intentionally delays packets fora random amount of time chosen from a probabilistic1The emulation studies presented in this paper also ap-peared in [25]. They used an initial prototype of our middle-ware design that was integrated with the Linux kernel; fromthis prototype, we discovered the need for a more modularand extensible middleware, and thus MaDMAN was born.

8

(a) Emulation Results (tcp only)

(b) Emulation Results (adaptive)

Figure 4: Results from Emulation

delay distribution based on the parameters of a par-ticular experiment. The distribution is given as an in-put and can be thought of as a delay vs. time curvethat captures the properties of the delay-tolerant net-work from the perspective of an end-to-end connection.The emulator does not model a particular topology nora particular routing protocol. We simply emulate theend-to-end behavior of a generic delay-tolerant networkto better understand the performance benefits of dy-namically swapping protocol implementations.

Figures 4(a) and 4(b) show the results of two experi-ments in our emulator. In this scenario, we introducedlong delays for packets in the middle of the experiment.Specifically, our delay vs. time curve models a situ-ation where the data source starts reliably connectedto its sink. It later leaves the well-connected portionof the network for 10 minutes, operating in the oppor-tunistic delay-tolerant network stack in this time, wherethe packet delays average two minutes. The data sourcethen later returns to the reliably connected network. Inthe figures, the gray ×’s represent the amount of delayintroduced by our delay buffer at different instants oftime in the emulation (in milliseconds, on a log scale).In Figure 4(a), the dark triangles represent the num-ber of packets delivered by the reliable TCP connec-tion at a particular instant in time. In Figure 4(b),the dark circles represent the number of packets deliv-ered by the adaptive combination of the reliable TCP

connection and the unreliable DTN protocol at a par-ticular instant in time (in the emulator, we used a sim-ple UDP protocol for these packets). As the figuresshow, the disconnected phase of the experiment (in themiddle of the figure) is marked by a dramatic drop inTCP throughput. Specifically, the adaptive version im-proves the overall throughput by 90%. These results ledus to believe that the adaptive network stack approachhas the potential to provide significant benefits to ap-plications in mixed delay-tolerant mobile ad hoc net-works. However, our initial implementation integratedinto Linux was brittle and not easily extensible withnew delay-tolerant networking protocols; we thereforereimplemented the middleware to create MaDMAN asdescribed in this paper and evaluated in a live testbedas described below.

5.2 Evaluation in the Pharos TestbedTo fully evaluate the Click-implemented MaDMAN

described in Section 4, we used the Pharos testbed [26],a highly capable mobile ad hoc network testbed at theUniversity of Texas at Austin that consists of Proteus [27]nodes, autonomous mobile robots with x86 hardwareand off-the-shelf 802.11 wireless radios.2

5.2.1 Testbed Experiment SetupWe used three of the Proteus nodes shown in Fig-

ure 5, whose mobility is provided by a customizedTraxxas Stampede, a high performance remote con-trol car chassis with Ackerman steering and 4-wheel-independent suspension. Each robot runs Linux v.2.6and is equipped with (among other things) an x86 moth-erboard, an Intel 802.11bg wireless card, and a Freescalemicrocontroller that interfaces with the steering andmotor controllers to navigate the robot. Mobility com-mands are issued by a user-level application using thePlayer/Stage API [1] running on the x86 computer.Although the robots can navigate using GPS, we per-formed these experiments indoors on a smaller scale forcontrol and reproducibility. We relied on the Proteus’sability to move (fairly) straight for short distances in-doors.

We used three nodes to create a delay-tolerant net-work; the spatial and timing characteristics of the ex-perimental environment are shown in Figure 6. To en-sure node disconnections (and reconnections) in a 40foot hallway, we had to attenuate the radios’ transmitpower. We accomplished this through a combinationof a modified ipw2200 Linux Intel wifi driver that al-lowed us to set the transmit power of the wireless cardsto 3dBm and aluminum foil around the antennae. Thecombination gave us a connection range of between five

2The Click MaDMAN implementation and the configurationfiles we used for these experiments are available at http://mpc.ece.utexas.edu/madman.

9

t = 15 sec.arrives t = 55 sec.

40 ft.

approx. radio range sink

sourcet = 175 sec.

t = 135 sec. arrives t = 175 sec.

arrives t = 215 sec. 12.5 ft.

data mule

Figure 6: Experiment Scenario

Figure 5: The Proteus Mobile Node

Figure 7: The Experimental Setup

and fifteen feet, depending on the channel characteris-tics; for our experiments, we were able to reliably estab-lish connections within five feet and to guarantee a nodemoving down the hallway would eventually disconnectfrom one at the end of the hallway. Each experimentconsists of a stationary data sink, a mobile data source,and a mobile data mule. Only the mobile data sourcecreates data packets. Specifically, MaDMAN’s Sourcemodule (see Figure 3) generated 1448B packets at anexperiment-specific rate. In all experiments, epidemicbeacon packets of size 64B were sent every 200 millisec-onds, and the epidemic disconnection time-out intervalwas set to 700 milliseconds.

We used the same mobility pattern in all of the re-sults we discuss in this paper. At the beginning of theexperiment, both the mobile data source and the sta-tionary sink nodes are next to each other and connected.In this situation, MaDMAN chooses to send all of thedata packets generated by the Source application viaTCP. At t = 15 seconds, the sink node begins movingtowards the data mule at the other end of the hallway ata speed of 1 foot/second. It eventually disconnects fromthe sink node; due to the reduced quality of data de-livery, MaDMAN swaps the connection underpinningsfrom using TCP over a reliable routing stack to usingUDP over an epidemic routing stack. Around t = 55seconds, the mobile data source arrives at the other endof the hallway and connects to the mobile data mule.The source node generates packets at the specified ratefor the entire duration of the experiment; because theapplication’s connection is engaged in epidemic rout-ing, the mobile data source sends queued packets to themobile data mule with the optimistic hope that it willencounter the data sink before the source is connectedto it again. At t = 135 seconds, the mobile data mulebegins to move in the direction of the stationary datasink, again at a speed of 1 foot/second. The mobilesource node follows 40 seconds later at t = 175. When

10

the mobile data source reconnects to the sink directly,MaDMAN responds by switching the connection’s sup-porting protocol stack back to the reliable TCP one.The state from the delay-tolerant network stack is usedto bootstrap the new TCP connection, which ultimatelycontinues the application’s communication session fromthe last packet the stationary data sink received fromthe mobile data mule.

5.2.2 Testbed Experiment ResultsIn this section, we provide results for two sets of ex-

periments executed using the setup described above.These two experiments differ in the rate with whichpackets are generated by the Source module; the firstexperiment generated 1448B packets at a fixed rate of10 packets per second, while the second experiment gen-erated 1448B packets at a fixed rate of 100 packets persecond. For simplicity in viewing the results, we showresults from only one run each of these experiments; re-sults were consistent across multiple runs. Each experi-ment was run once using only the reliable TCP networkstack without any intelligent swapping of the underlyingcommunication protocol structure and once with MaD-MAN’s stack swapping capabilities in place, with theconnection support changing as described in the previ-ous section. Figure 8 shows the results of these exper-iments. In the figures, the diamonds represent pack-ets delivered by the reliable TCP-based stack, and thesquares are packets delivered by the delay-tolerant UDPstack with epidemic routing.

MaDMAN enables “early” delivery of discon-nected data. In the first experiment, the data ratewas slow enough that both approaches were able to de-liver all of the data the application generated by theend of the experiment. In this case the adaptive na-ture of MaDMAN does not improve the packet deliv-ery ratio or the throughput as measured over the entireexperiment window. However, as Figure 8(c) shows,MaDMAN’s adaptive stacks do deliver a set of packetsearly, when the mobile data mule reaches the stationarydata sink in advance of the mobile data source’s return.Specifically, the data mule delivered 1035 data pack-ets approximately 40 seconds in advance of the mobiledata source’s return to connectivity with the stationarydata sink (as indicated by the peak in the adaptive dataaround 150 seconds into the experiment). In general,this result demonstrates that, while using a reliable con-nectivity infrastructure is ideal when possible, the useof an opportunistic communication structure when thereliable one becomes unavailable can improve the delayincurred in the delivery of application data packets.

MaDMAN improves application-level through-put through opportunistic delivery. In the sec-ond experiment, the Source module in MaDMAN gen-erated approximately 150KB of data per second (100

packets of size 1448B per second). This data rate washigh enough that the TCP connection alone could notreliably deliver all of the generated application data.Figure 8(b) shows the results when only the reliableTCP stack was employed; in this experiment, the sinknode received only 38% of the total generated applica-tion data. The remainder of the data was lost almostexclusively due to buffer overflows resulting from thehigh rate of data generation. On the other hand, Fig-ure 8(d) shows the results when MaDMAN adapted thecommunication implementation in response to the ex-periment’s changing operating conditions. In this case,the two modes of communication combined to deliver atotal of 65% of the total generated application data, asignificant improvement over TCP alone. The degree towhich epidemic delivery of packets by a data mule canimprove throughput is heavily dependent on the sizesof the epidemic data buffers, which in the case of thisexperiment were not a performance-limiting factor. Inthe future, we plan to vary the epidemic buffer sizes andexperimental setup in order to characterize the trade-off between buffer sizes and performance under variousdelay-tolerant network configurations. In general, thisexperiment’s results show that intelligently swappingthe communication implementation in a mixed delay-tolerant mobile ad hoc network can have a significantimpact on the total throughput of application data.

5.3 DiscussionThe combination of our evaluation of MaDMAN in

both emulation and in the Pharos testbed has shownthat, in both idealized and realistic environments, thebenefits of dynamically adjusting the communicationparadigm that supports an application’s ongoing com-munication sessions include decreasing the overall delayand increasing throughput. In this section, we discuss afew points that arose in the execution of our evaluation.

First, we have demonstrated that our Click-based im-plementation of the MaDMAN middleware can easilyrun on commodity hardware. We have made our imple-mentation available for use and extension. MaDMAN’smodular design ensures its independence from any par-ticular protocol implementations; the middleware mustbe made aware of them through the configuration filesdescribed in Section 4.2.5. We have demonstrated themiddleware using Click’s interface to the TCP imple-mentation in the Linux stack and our own implemen-tation of the popular delay-tolerant epidemic routingprotocol. Other protocols implemented within Click orwith interfaces to Click can be integrated in a similarmanner.

At a more detailed level, we found that when wepushed the packet generation rate much beyond 100packets per second, the results became inconsistent be-tween runs. We hypothesize that MaDMAN is send-

11

200

300

400

500

600ckets D

elivered

/Sec

TCP

0

100

200

300

400

500

600

0 50 100 150 200 250 300 350

Packets D

elivered

/Sec

Time (Sec)

TCP

(a) TCP-only (10 packets/second)

300

400

500

600

700

800

900

1000

ckets D

elivered

/Sec

TCP

0

100

200

300

400

500

600

700

800

900

1000

0 50 100 150 200 250 300

Packets D

elivered

/Sec

Time (Sec)

TCP

(b) TCP-only (100 packets/second)

200

300

400

500

600

ckets D

elivered

/Sec

TCP Adaptive

0

100

200

300

400

500

600

0 50 100 150 200 250

Packets D

elivered

/Sec

Time (Sec)

TCP Adaptive

(c) Adaptive (10 packets/second)

300

400

500

600

700

800

900

1000

ckets D

ilvered

/Sec

TCP Adaptive

0

100

200

300

400

500

600

700

800

900

1000

0 50 100 150 200 250

Packets D

ilvered

/Sec

Time (Sec)

TCP Adaptive

(d) Adaptive (100 packets/second)

Figure 8: Packets delivered vs. Time

ing packets so fast that the wireless channel often be-comes unusable, causing the wireless cards to disconnectat the data-link layer. Further study with the Pharostestbed should quantify the capabilities of the Proteusnodes and their radios so we could later have a con-crete understanding of the communication limits of thetestbed. We also found a wide variation in the timeit took for two nodes to reconnect to each other oncethey were within communication range. In some ex-treme cases, the nodes required up to a minute afterentering each others radio range to establish link-levelcommunication. We are confident that this is not a bugin the MaDMAN implementation, but further investi-gation with the nodes and their radio may enable us tomitigate these unpredictable connectivity challenges.

In the testbed evaluation, our experiments showedimprovement with respect to two metrics: 1) we enabledearly delivery of application data that was generated inthe disconnected state; and 2) we increased the totalthroughput in high traffic situations by opportunisti-cally delivering data when the source and sink werenot directly connected. We were able to show thesesignificant benefits even given the simplistic nature of

the experimental setup. Specifically, the mobile datasource remained disconnected from the stationary datasink only for about three and a half minutes; it is easyto imagine situations where the source node is operat-ing in the opportunistic mode for much longer, in whichcase MaDMAN’s benefits will be amplified. In addition,we used only a single data mule to deliver opportunisticdata. In larger mixed delay-tolerant mobile ad hoc net-works, it is likely that several mobile nodes will be ableto ferry opportunistic data among sources and sinks,which would also increase the benefits of the MaDMANmiddleware.

6. CONCLUSIONThis paper presented the MaDMAN middleware ar-

chitecture that enables an adaptive communication in-frastructure in mixed delay-tolerant and mobile ad hocnetworks. To handle and take advantage of the chang-ing operating conditions that applications in these dy-namic environments experience, MaDMAN enables theintelligent exchange of the protocol stacks that imple-ment a communication session in the middle of an on-going session. The design of MaDMAN ensures that

12

such an adaptation is seamless from the application’sperspective, uses context information sensed from theDTN environment, and exchanges state among the pro-tocol stacks to ensure that data delivery is both usefuland as timely as possible.

In the paper, we described our implementation ofthe MaDMAN middleware model using the Click mod-ular router. To support the middleware, we also im-plemented the epidemic routing algorithm, which as aresult of the modular nature of Click, was a straight-forward and relatively simple process. Defining a pre-cise methodology for integrating new protocols into MaD-MAN and testing our claims of its ease and simplicityremains for future work, however given our experienceimplementing epidemic routing, we are confident thatit can be done. We also provided a two-phase evalu-ation: an emulation that demonstrated the potentialgains from such intelligent adaptation and a testbedevaluation that showed the gains applications receivefrom MaDMAN on delay-tolerant networks implementedwith commodity hardware. We showed marked im-provements using a simple algorithm for determiningwhen to adapt the communication infrastructure; theaddition of more context-awareness and execution inlarger-scale networks have the potential to significantlyamplify these performance gains. MaDMAN is an es-sential first step in integrating DTN communication im-plementations with more traditional mobile computingsolutions so that they can be used to support real worldapplication deployments.

7. ACKNOWLEDGEMENTSThe authors thank Vidur Bhargava for his work on

the transmit power control module and Sanjay Shakkot-tai and Sriram Vishwanath for their project guidance.The authors would also like to thank the Center forExcellence in Distributed Global Environments for pro-viding research facilities and the collaborative environ-ment. This research was funded in part by the DoD.The views and conclusions herein are those of the au-thors and do not necessarily reflect the views of thesponsoring agencies.

8. REFERENCES

[1] M. Anderson, L. Thaete, and N. Wiegand.Player/Stage: A unifying paradigm to improverobotics education delivery. In Proceedings of theWorkshop on Research in Robots for Education,2007.

[2] A. Baig, M. Hassan, and L. Libman.Prediction-based recovery from link outages inon-board mobile communication networks. InProceedings of IEEE Globecom, pages 1575–1579,2004.

[3] C. Boldrini, M. Conti, I. Iacopini, andA. Passarella. HiBOp: A history based routingprotocol for opportunisitic networks. InProceedings of the IEEE International Symposiumon a World of Wireless, Mobile and MultimediaNetworks, June 2007.

[4] M. Carson and D. Santay. NIST Net: Alinux-based network emulation tool. ACMSIGCOMM Computer Communications REview,33(3):111–126, 2003.

[5] A. Doria, M. Uden, and D. Pandey. Providingconnectivity to the Saami nomadic community. InProceedings of the 2nd International Conferenceon Open Collaborative Design for SustainableDevelopment, 2002.

[6] K. Harras, K. Almeroth, and E. Belding-Royer.Delay tolerant mobile networks (DTMNs):Controlled flooding in sparse mobile networks. InProceedings of the 4th International IFIP-TC6Networking Conference, pages 1180–1192, May2005.

[7] K. A. Harras, M. P. Wittie, K. C. Almeroth, andE. M. Belding. ParaNets: A parallel networkarchitecture for challenged networks. InProceedings of the 8th IEEE Workshop on MobileComputing Systems and Applications, pages73–78, March 2007.

[8] S. Jain, K. Fall, and R. Patra. Routing in a delaytolerant network. ACM SIGCOMM ComputerCommunication Review, 34(4):145–158, October2004.

[9] D. Joseph, J. Kannan, A. Kubota,K. Lakshminarayanan, I. Stoica, and K. Wehrle.OCALA: An architecture for supporting legacyapplications over overlays. In Proceedings of the3rd Symposium on Networked Systems Design andImplementation, pages 267–280, 2006.

[10] P. Juang, H. Oki, W. Want, M. Martonosi,L. Peh, and D. Rubenstein. Energy-efficientcomputing for wildlife tracking: Design tradeoffsand early experiences with zebranet. ACMSIGPLAN Notices, 37(10):96–107, October 2002.

[11] E. Kohler, R. Morris, B. Chen, J. Jannotti, andM. Kaashoek. The Click modular router. ACMTransactions on Computer Systems,18(3):263–297, 2000.

[12] M. Leech, M. Ganis, Y. Lee, R. Kuris, D. Koblas,and L. Jones. SOCKS protocol version 5, IETFrequest for comments: 1928.http://tools.ietf.org/htmlrfc1928, March1996.

[13] A. Lindgren, A. Doria, and O. Schelen.Probabilistic routing in intermittently connectednetworks. In Proceedings of the 1st InternationalWorkshop on Service Assurance with Partial and

13

Intermittent Resources, pages 239–254, 2004.[14] H. Madhyastha, A. Venkataramani,

A. Krishnamurthy, and T. Anderson. Oasis: Anoverlay-aware network stack. SIGOPS OperatingSystems Review, 40(1):41–48, January 2006.

[15] Y. Mao, B. Knutsson, H. Lu, and J. Smith.DHARMA: Distributed home agent for robustmobile access. In Proceedings of the 24th AnnualJoint Conference of the IEEE Computer andCommunications Societies, pages 1196–1206,2005.

[16] T. Matsuda and T. Takine. (p,q)-Epidemicrouting for sparsely populated mobile ad hocnetworks. IEEE Journal on Selected Areas inCommunications, 26(5):783–793, June 2008.

[17] P. Mundur, M. Seligman, and G. Lee. Epidemicrouting with immunity in delay tolerant networks.In Proceedings of the IEEE MilitaryCommunications Conference, pages 1–7,November 2008.

[18] One laptop per child. http://laptop.org, 2000.[19] J. Ott. Application protocol design considerations

for a mobile internet. In Proceedings of the 1st

ACM/IEEE International Workshop on Mobilityin the Evolving Internet Architecture, pages75–80, 2006.

[20] J. Ott and D. Kutscher. A disconnection-toleranttransport for drive-thru internet environments. InProceedings of the 24th Annual Joint Conferenceof the IEEE Computer and CommunicationsSocieties, pages 1849–1862, 2005.

[21] J. Ott, D. Kutscher, and C. Dwertmann.Integrating DTN and MANET routing. InProceedings of the 2006 SIGCOMM Workshop onChallenged Networks, pages 221–228, 2006.

[22] P. Pathirana, N. Bulusu, A. Savkin, and S. Jha.Node localization using mobile robots indelay-tolerant sensor networks. IEEETransactions on Mobile Computing, 4(3):285–296,May–June 2005.

[23] A. Pentland, R. Fletcher, and A. Hasson. DakNet:Rethinking connectivity in developing nations.IEEE Computer, 37(1):78–83, January 2004.

[24] C. Perkins and E. Royer. Ad-hoc on-demanddistance vector routing. In Proceedings of the 2nd

IEEE Workshop on Mobile Computer Systemsand Applications, pages 90–100, 1999.

[25] A. Petz and C. Julien. An adaptive architectureto support delay tolerant networking. InProceedings of the 7th Workshop on Adaptive andReflective Middleware, December 2008.

[26] http://mpc.ece.utexas.edu/pharos/.[27] http//proteus.ece.utexas.edu.[28] R. Ramdhany and G. Coulson. MANETKit:

Supporting the dynamic deployment and

reconfiguration of ad-hoc routing protocols. InProceedings of the ACM/IFIP/USENIX 10th

International Middleware Conference, December2009.

[29] J. Scott, P. Hui, J. Crowcroft, and C. Diot.Haggle: A networking architecture designedaround mobile users. In Proceedings of the 3rd

Annual IFIP Conference on Wireless On-DemandNetwork Systems and Services, January 2006.(invited paper).

[30] K. Scott and S. Burleigh. Bundle protocolspecification, IETF request for comments: 5050.http://www.ietf.org/rfc/rfc5050.txt,November 2007.

[31] A. Snoeren and H. Balakrishnan. An end-to-endapproach to host mobility. In Proceedings of the6th Annual International Conference on MobileComputing and Networking, pages 155–166, 2000.

[32] D. Stovall, N. Paine, A. Petz, J. Enderle,C. Julien, and S. Vishwanath. Pharos: Anapplication-oriented testbed for heterogeneouswireless networking environments. TechnicalReport TR-UTEDGE-2009-006, The Center forExcellence in Distributed Global Environments,The University of Texas at Austin, 2009.

[33] J. Su, J. Scott, P. Hui, J. Crowcroft, E. de Lara,C. Diot, A. Goel, M. H. Lim, and E. Upton.Haggle: Seamless networking for mobileapplications. In Proceedings of the 9th

International Conference on UbiquitousComputing, pages 391–408, September 2007.

[34] A. Vahdat and D. Becker. Epidemic routing forpartially connected ad hoc networks. TechnicalReport CS-200006, Duke University, April 2000.

14