An Adaptive Architecture to Support Delay TolerantSupport Delay Tolerant

Networking

Agoston PetzChristine Julien

TR-UTEDGE-2008-002

© Copyright 2008The University of Texas at Austin

An Adaptive Architecture to Support Delay Tolerant Networking

Agoston Petz and Christine JulienDepartment of Electrical and Computer Engineering

University of Texas at Austinagoston.petz@gmail.com, c.julien@mail.utexas.edu

Abstract

Delay Tolerant Networks (DTNs) are emerging as a newform of networks in which sending and receiving nodes maynot be reliably connected to each other in a traditionalsense but must instead rely on mobile nodes to ferry mes-sages through the network. Future mobile computing de-ployments are likely to require such DTN deployments tooperate alongside more traditional mobile ad hoc network-ing approaches. Applications operating in these dynamicmixed environments would like their connections to be sup-ported by the network technology best suited to the com-bination of the communication session’s requirements andinstantaneous network context. In this paper, we explore thesystems issues related to enabling such a symbiotic networkarchitecture. Specifically, we develop an adaptive architec-ture that enables connections to seamlessly migrate fromone communication style to another in response to changingnetwork or application conditions. We delineate the proper-ties of the architecture, describe an initial implementation,and provide a simulation analysis that demonstrates that,given perfect context-awareness, significant performanceimprovements can be made by using such an adaptive ar-chitecture.

1 Introduction

Evolving mobile computing environments consist of un-predictable mobile entities connected via wireless networkswhose topologies are subject to constant change. The useof these networks in both commercial and military deploy-ments necessitates a new paradigm of communication andmessage exchange. Delay tolerant networking (DTN) hasevolved in which protocols asynchronously route messagesfrom a source to a destination over networks that lack con-tinuous connectivity. In these networks, connections comeand go dynamically, the source and destination may neverbe directly connected, and nodes’ movements are unpre-dictable.

Region 2

A

BRegion 1

Region 3

Figure 1. A generic DTN target environment

DTN architectures potentially apply in many applicationinstances. In an urban setting, commuter transportation sys-tems can be used as storage to carry messages from oneregion of a city to another [5]. A remote village may bewell connected internally, but less reliably connected to thewider world [20]. In a military network, devices within apost or base camp may have good connectivity to one an-other but may be connected to devices in other regions onlyby jeeps or convoys that relay messages from time to time.Researchers moving among tagged herds of zebras can col-lect and share information about zebra behavior and move-ment [10]. Pockets of well-connected sensor networks canbe periodically connected via roving mobile robots [19].

Figure 1 shows an abstraction of the common operatingenvironment shared by these disparate applications. Thefigure shows three well-connected regions that are intercon-nected transiently via delay-tolerant links (dashed lines inthe figure). In the deployments depicted, nodes can breakoff from the well-connected regions and transit betweenthem (such as node A, which is transiting between regions1 and 2, and node B, which has just arrived at region 3).

Research issues relating to routing [8, 11, 25] and opti-

2

mizing topologies are well-studied, and great strides havebeen made with respect to supporting this new paradigm ofcommunication. In this paper, we study a more practicalsystems issue, namely the ability of networked devices toadapt their network interactions to the changing operatingenvironment. Specifically, we present a network architec-ture and implementation that allows one network stack tobe dynamically swapped in for another while not experienc-ing (from the application level) a disruption in connectivity.With an aim to support networks such as the one depicted inFigure 1, we focus on transitioning between two stacks: atraditional stack (for use within the connected regions in thefigure) and a DTN stack (for use in transient communica-tion situations). We focus on enabling the mechanics of thistransition and the systems and communication issues thatmust be resolved to enable seamless transitions. For now,we ignore issues related to what context information shouldbe used to determine when to transition and the functionsthat relate the context to the transition decision and focusinstead on demonstrating that, given good knowledge of theconnection state, it is beneficial to make transitions in thefirst place.

The remainder of this paper is organized as follows. Inthe next section, we describe the targeted problem and pro-vide background information. Section 3 describes the ar-chitecture for the entire system, while Section 4 discussesthe implementation details of our prototype system. Sec-tion 5 provides some evaluation of our approach throughemulation, demonstrating the potential benefits of swappingstacks in particular situations. Section 6 concludes.

2 Background and Problem Definition

In this section, we first detail a single example appli-cation that falls in the general category depicted in Fig-ure 1. We then look at related approaches and use themas a springboard to precisely define the problem we addressin this paper.

2.1 A Motivating Scenario

As touched upon in the previous section, delay tolerantnetworks are relevant to a wide variety of applications. Tofurther explore the requirements of applications for delaytolerant networks, we hone in on a particular example thatfits the model shown in Figure 1, specifically using DTNsto connect remote villages to each other and to the Inter-net through distant hubs [3, 20]. In such an application,devices are deployed within small remote villages (e.g.,via the one laptop per child project [16]) where the avail-able infrastructure is severely limited. Within the village,the devices have good connectivity among themselves, sup-ported by traditional mobile ad hoc networking protocols or

other wireless grid approaches. Connectivity to the widerworld, however, is more difficult; without infrastructure net-works, long-range wireless communication is too energy in-tensive for the villages’ constrained resources. However,people transit between villages and larger cities to delivergoods and supplies and to visit family and friends. Someof this movement is on a predictable schedule; other move-ments are more spontaneous. By leveraging the storage andcommunication capabilities of devices traveling with peopleamong villages, computer users in remote areas can achieveaccess to the wider networked world.

This network access, however, is not of the same natureas the Internet access most people enjoy. Connections areasynchronous, demanding an entirely new network archi-tecture to support these devices. Delay tolerant networkshave emerged as the obvious candidate; in such networks,applications that can tolerate significant latencies can besupported even under the duress of the conditions describedabove. However, much work remains to be done to provideapplications the same kinds of abstractions they are used toin traditional networks given this vastly different underlay.

2.2 Related Approaches

Given the application described previously, an overarch-ing goal of our work is to provide applications the sem-blance of a communication session in a delay tolerant net-work, even when no end-to-end connection is even possi-ble. In addition, we aim to maximize application perfor-mance by dynamically selecting the best end-to-end ap-proach based on applications’ (potentially competing) re-quirements, the communicating parties, and the networkconditions. Given these goals, related approaches can bedivided into two categories: approaches that seek to pro-vide end-to-end connection semantics in challenged envi-ronments and approaches that seek to abstract complex un-derlying semantics while still exposing expressive protocolbehavior.

End-to-End in Dynamic Environments. The natureof existing applications commonly stresses a view of end-to-end connectivity which can be unreasonable in dynamicor unpredictable environments. Transport layer extensionshave been developed that enable end-to-end connectivityin cases when it would otherwise be impossible, for ex-ample by predicting when link failures will occur in mo-bile networks and mitigating their impact [1] or by usingdynamic DNS updates to support TCP connection migra-tion [24]. These approaches are not suitable for the typesof networks envisioned in delay tolerant networking as ap-plications’ connections still timeout if they experience ex-tended disconnections. Other approaches use proxies asdelegates to shield applications from disconnection [14, 7].Such approaches assume every application will eventually

reconnect to a particular central service, making them un-suitable to the highly dynamic and unpredictable delay tol-erant networking environment.

Abstracting Diverse Protocols. In response to the needto evolve network architectures to suite emerging applica-tions, architectures that maintain interfaces for existing ap-plications but expose new functionality have been devel-oped. The Overlay Convergence Architecture for LegacyApplications (OCALA) [9] defines an overlay architecturethat allows existing (legacy) applications to continue tofunction, even if the underlying network architecture andimplementation are fundamentally changed. OCALA func-tions below the transport layer, presenting legacy applica-tions with a standard IP-like interface and hiding a sub-layerthat translates between these traditional network interac-tions and interactions in new network overlays. By design,OCALA enables applications utilizing different overlays tointeract with each other but must use well-defined gatewaysto transition between overlay types. OCALA is similar inspirit to our approach but offers a different end goal, i.e., toenable evaluating new network overlays’ support of existingapplications.

The Overlay Access System for Internetworked Services(Oasis) [12] also attempts to enable legacy applications tocontinue to function on top of overlays that offer more ex-pressive interfaces to newly introduced applications. Oasisinserts a layer that manages available overlays for applica-tions, allowing the “best” performing overlay to be dynam-ically selected at the beginning of a connection. In contrastto both OCALA and Oasis, DTN applications such as theone described previously require the ability for the networkarchitecture to dynamically reevaluate the choice of under-lying implementation, adapting not only to the application’sstated requirements but also to the changing network situa-tion and the changing network path from the source to thedestination.

An approach specific to delay tolerant networks effec-tively defines an API that allows an application to choosea “traditional” end-to-end connection (provided by theAODV MANET routing protocol [21]) or a DTN basedcommunication protocol based on information about theavailability of the end-to-end connectivity [18]. The un-derlying assumption is that, if possible, end-to-end seman-tics are always preferable. This approach exposes networkinformation (e.g., information about routing paths and orresponsible intermediaries) to the application to use in sub-sequent decision making. Because the approach intimatelyintertwines AODV and DTN routing, the DTN routing al-gorithm can take advantage of AODV knowledge in its for-warding decisions. However, this approach makes the twocommunication styles dependent on each other, which hasthe potential to significantly increase the overhead in com-parison to the individual protocols.

Device 1 Device 2Device 2

Application

Middleware Middleware

Application

Middleware

ApplicationDevice 2

Middleware

MANET DTN

Wi l N t k DTN I t t

MANET DTN

Middleware

Wi l N t k

MANET DTN

Middleware

Wireless Network DTN Interconnect Wireless Network

Figure 2. Operation of a dynamic delay toler-ant network architecture.

2.3 Problem Definition

Design considerations relating to the impact of a delaytolerant network underlays on end-to-end application se-mantics have begun to be explored [17]. We move a stepfurther by looking at a specific technical challenge withinthese new requirements; specifically, we enable the under-lying network implementation to vacillate intelligently be-tween a traditional MANET implementation and a less reli-able DTN implementation. The goal is to provide a single“session” interface to the application and dynamically fillin the nature of the connection based on 1) the conditionsand quality of the network and the path(s) to the destinationand 2) the application’s requirements. Existing applicationsshould be able to use the resulting network stack withoutmodification; “DTN-aware” applications should ultimatelybe able to tailor their interactions to take full advantage ofthe added, context-aware functionality.

In realizing the above goal, we make a few assumptions.First, we delegate destination discovery to the underlyingnetwork protocols (be they traditional MANET routing pro-tocols or DTN routing protocols). We instead focus on thesystems issues involved in being able to dynamically androbustly switch an ongoing connection from one style ofcommunication to another. We also assume that the ideal“DTN-aware” application can specify, as an application pa-rameter, a maximum tolerable delay. This information canbe used to determine whether the underlying systems shouldeven attempt to satisfy the request, given the current net-work conditions. In the remainder of this paper, we willspeak mostly in terms of a single “application” which com-prises two end points and the (potentially dynamic) connec-tion between them. Aspects of the ideal resulting architec-ture are depicted and described in Figure 2. As the figureshows, applications within the same local area (e.g., withinthe same village) use the underlying MANET communica-tion support (dashed gray connection). When the networkconditions change to make MANET communication unrea-sonable or impossible (e.g., Device 2 begins to move awayfrom the village), the communication session is automati-

cally migrated to the DTN network technology. This mi-gration occurs transparently to the application; the ongoingcommunication is not interrupted (though its quality of ser-vice may change).

3 An Architecture for Seamless DelayTolerant Networking

The architecture we propose is based on the idea thatthe capabilities of a delay tolerant network are best utilizedby a network stack specifically tailored for this new styleof network. This implies that DTN-specific PHY, MAC,network, and transport protocols will all need to coordinateto fully utilize the network. While applications for DTNscommonly experience extended periods of high delay, thismay not be the common operation, and it is definitely notthe only model of operation. For this reason, we designed amiddleware for delay tolerant applications that allows themto function when delays are high, using delay tolerant pro-tocols, but also maximizes their performance when delaysare not high, using more traditional means of communi-cation. Our middleware is architected to allow new andexperimental physical and MAC layer protocols, networklayer protocols, and transport and application layer proto-cols to easily plug into the middleware framework. It isdesigned to present a single, uniform communication in-terface to applications while at the same time allowing re-searchers to easily modify the internal components. In thisway, it is not dissimilar to OCALA [9]. What makes ourmiddleware unique is the overarching design decision toallow applications to use both a traditional network stack(e.g., TCP/IP) and the delay tolerant stack simultaneouslyand to dynamically swap application connections betweenthe stacks as dictated by changing network conditions. Themiddleware’s model incorporates the use of application andnetwork context to automatically determine the appropriatetime to switch stacks on a per-connection basis.

In the remainder of this section, we describe the mid-dleware’s components at a conceptual level. More detaileddiscussions of our particular implementation are handled inSection 4. The middleware has four distinct parts: (1) aservice daemon that implements most of the middleware’sfunctionality, (2) a user library that provides applicationswith an API to access the service daemon, (3) a delay tol-erant router and transport protocol, and (4) a context aggre-gator to gather information about the network to determinethe optimal stack to use for a given connection. The fol-lowing four subsections describe the functions of each ofthese subsystems, and Figure 3 illustrates their conceptualrelationships.

User Library

Application

re

DTN Service Daemon (and Proxy)

y

Routing and Transport

Context Aggregator

N M

iddlew

a

Routing and Transport

Physical Network

DTN

Figure 3. DTN Middleware Components

3.1 Service Daemon

The service daemon implements most of the functional-ity of the middleware, and should be viewed as the “core”of the system. It is the most complex component and thecomponent that communicates with all of the other piecesof the system. It is responsible for interfacing with appli-cations, managing dynamic resources, and negotiating out-bound and inbound connections. The service daemon is in-tended, but not required, to run on all nodes participating inthe delay tolerant network.

Our middleware architecture specifies that the servicedaemon should behave like a proxy and communicate withapplications using standard libraries (e.g., sockets or someother network capable IPC). We maintain that a standard-ized approach to IPC is a good idea for two reasons: (1) itallows user applications written in any language to easilyinterface with the middleware so long as they have an ap-propriate user library, and (2) it allows the middleware toexist on a different machine than the application. The sec-ond reason is important since the ultimate architecture of anetwork relying on our middleware may vary from a com-pletely distributed deployment to one that includes delaytolerant “gateways” [4, 6]. In the latter case, our middle-ware could be hosted on a gateway machine and could ser-vice a number of clients on the local network. An unfor-tunate drawback of our proxy-style architecture is that itbreaks end-to-end semantics since application data is ac-knowledged by the middleware before it is actually deliv-ered to the destination. This is also an advantage of the mid-dleware, since it can keep trying to deliver data long afterthe sender has closed the socket and given up on waiting forthe acknowledgement. A future implementation of the mid-dleware’s service daemon could incorporate a mechanismsimilar to that explored in TCP Splice [13] that provides alock-step implementation that preserves the full end-to-endsemantics, should such a strong semantics be required.

3.2 User Library

The user library provides the API by which applicationsinteract with the delay tolerant network. The user libraryis intended to be as small (and portable) as possible andfunctions like a wrapper for the sockets library. Our user li-brary only implements the necessary functionality to trans-late socket requests to service daemon requests. It doesthis by first packing the application socket request parame-ters into a separate request destined for the service daemon,sending that request to the daemon, and returning the resultto the application.

The user library also allows the application to provide asmall amount of context to the middleware. Currently, ap-plications can provide data priority as context, which canbe used to optimize the available bandwidth or to determinewhat data can be dropped or rescheduled for later transmis-sion if the network cannot currently handle the throughput.Priority hints are helpful since a low priority connection canbe selectively scheduled on the delay tolerant stack to freeup TCP/IP resources. We are also considering feedback-hints about future bandwidth requirements, application sug-gestions for re-scheduling transmissions, etc.

In the future, the middleware will also act as a SOCKSproxy so that legacy applications will be able to entirely by-pass the user library. This will obviously prevent the shar-ing of application context and hints since that functionalityis captured by the user library API but it allows existing ap-plications to coexist with new DTN-aware applications.

3.3 Routing and Transport Layer

The routing component implements the algorithms andprotocols inherent to the routing layer, and it is where vari-ous existing and future protocols can be plugged in. As de-scribed previously, much existing work focuses on creatinggood DTN routing protocols, and this component approachallows the middleware to selectively take advantage of thiswork. In an effort to make this easier, the middleware places“delay tolerant routers” in user-space, enabling them to in-teract with the kernel using Linux’s built-in netfilter user-space queue. This is how AODV-UU [15] and many otherrouting protocol implementations interact with the kernel.We have also intentionally separated routing from transport(see the more detailed Figure 4) to enable the incorporationof end-to-end semantics specific to the new DTN model ofcommunication. We have created a very simple first cut atthis transport protocol, which we refer to as the User Data-gram Protocol for Delay Tolerant Networks (UDPDTN). Animportant aspect of the use of a transport protocol in supportof a DTN connection within our middleware is the ability totransfer state between the DTN network stack and the tradi-tional network stack as applications’ connections transition

between the two. We briefly describe this transport protocolnext.

3.4 User Datagram Protocol forDelay Tolerant Networks

UDPDTN is a stateful but connectionless protocol andis intended to simplify the seamless transition between thetraditional network stack and a DTN-aware network stack.Because a running TCP connection contains a significantamount of state information, it is important for a DTN trans-port protocol to be able to take advantage of this informationin the best way possible. During a stack transition, the mid-dleware logs information available from one transport pro-tocol (e.g., the last acknowledged data, the window size, thesequence number, etc.) and inputs it into the connection’snew transport protocol. The protocol that has been swappedin can use this context information to adjust its own param-eters. Our initial cut at the UDPDTN protocol is simple anduses little of this information; future DTN transport proto-cols may rely more heavily on it. In the future, we alsohope to run the DTNRG bundle overlay protocol [23] overour enhanced UDPDTN “underlay”, and the added DTN-specific information in the transport layer will benefit thebundle protocol and other DTN overlay protocols.

3.5 Context Aggregator

The context aggregator shown in Figure 3 is responsi-ble for monitoring and processing network context to ac-curately characterize the state of the network. This infor-mation will be used to determine the optimal stack to usefor a given connection at a given time and will dependon some combination of network layer statistics, transportlayer statistics, and the current application activity. Exam-ples include connectivity, throughput, and latency, althoughdefining this context and providing an algorithm to deter-mine the “appropriateness” of using either stack is beyondthe scope of this paper.

4 Implementation

We have implemented our middleware in C++ on theLinux 2.6 kernel. The following subsections provide spe-cific implementation details for the various components andSection 4.5 provides a step by step example of how an ap-plication uses the middleware to set up a connection.

4.1 Service Daemon

The service daemon described in the previous section isimplemented as a multi-threaded user-space server, whereeach thread generally encompasses one component of the

Application

User LibraryUser Library

Legacy Application

TCP sockets

DTN Service Daemon (and Proxy)

Linux sockets library

Non‐DTN Application

TCP UDP DTNUDP etc…

TCP/IP Stack IP

AODV DSR DTN etc…

MAC/PHY L

TCP/IP Stack IP

MAC/PHY Layer

Figure 4. Architecture Overview

middleware. The service daemon pulls them all togetherto allow them to be managed in a unified space. Specificfunctions of these threads will be described in Section 4.5as part of the example. As depicted in Figure 3, the servicedaemon interacts with the application, the context aggrega-tor, and the routing and transport components. Through theuser library (discussed next), the daemon employs socketsto send and receive application messages. It then uses theLinux socket API to interact with the lower layers. The spe-cific functions of the context aggregator and the routing andtransport component are discussed in more detail below.

4.2 User Library

The user library is a C library that mimics the API ofthe C sockets library. Most of the available function callsare enhanced versions of standard socket calls, for exam-ple, write is augmented with optional priority informa-tion. Applications must currently link to our library to usethe middleware; future work will add a SOCKS proxy in-terface to the middleware for legacy application support.

4.3 DTN Router and Transport Layer

In our current implementation we are not concerned withany specific routing protocols, only in making sure that theappropriate hooks are in place to plug different routing pro-tocols into the middleware. As mentioned previously, weimplement a generic router using Linux’s libipq libraryto queue incoming messages in user-space where the routercomponent of the middleware can examine, modify, andselectively allow or drop packets. Existing MANET rout-ing protocols and specialized DTN routing protocols can beplugged in to provide different styles of service as the con-text aggregator deems necessary. With respect to the trans-port layer protocols, we use a standard TCP implementation

for the MANET stack, and our DTN transport layer proto-col does not currently implement any functionality beyondwhat UDP already provides. We refer to this implementa-tion as “UDPDTN.”

4.4 Context Aggregator

The context aggregation engine is implemented as a spe-cial thread in the service daemon. We are currently modify-ing the TCP/IP stack in the Linux kernel to export variousdata about the network and transport layers into a custom/proc file. This file will be read by the context aggre-gator and used to influence decisions about when to tran-sition from traditional protocols to DTN protocols and viceversa. We also plan on implementing feedback mechanismswhereby the context aggregator can modify the behavior ofthe TCP/IP and DTN stacks. The proc file system is themost logical means to accomplish this. In the implemen-tation used for our feasibility study in the next section, ourcontext aggregator is a very simple component with a god-like view of when transitions are best made.

4.5 Example connection setup and tear-down

The DTN middleware has two modes of operation, onefor handling outbound connections to remote DTN-enabledhosts (shown in Figure 5), and one for handling inboundconnections (shown in Figure 6). Applications and mid-dleware components are shown as rectangles, sockets areshown as circles, and arrows show directional data flowbetween the sockets and components. To explain how thecomponents of the middleware interact, we provide a step-by-step description of how an outbound connection is setup. The operations involved in processing the inbound con-nection are analogous.

1. The application requests an outbound socket using ouruser library.

2. The library translates the socket request into a socketrequest to the middleware (using configuration infor-mation such as the service daemon’s host and port),and sends the request. This socket (labeled ‘1’ in Fig-ure 5) is then used to communicate between the appli-cation and the middleware.

3. The middleware’s request processor determines thatthe request is for an outbound connection, spawns aproxy handler thread to handle the connection, andpasses the previously created socket to the handler.

4. The proxy handler negotiates the specifics of theconnection, which includes reserving middleware re-sources, connecting to the remote address, and deter-mining priority information.

Application Application Application

1 1 1

Request Processor

Proxy Handler

TCP Proxy Handler

DTN Proxy Handler

App Proxy Handler

2 3

Remote Host

Figure 5. Outbound (Proxied) Request

Service Service Service

1 12 2 2

Request Processor

Service Handler

TCP Listener DTN ListenerDTN 

Forwarder

3 4

Remote Host

Figure 6. Service Request

5. The proxy handler, using input from the context ag-gregator, opens either an outbound TCP or UDPDTNconnection to the remote host, and starts forwardingdata from the application. These connections are sock-ets ‘2’ and ‘3’ in Figure 5. The proxy handler alsoreceives feedback from the context aggregator over theduration of the connection (not shown) to determine ifand when to migrate the connection between stacks.

6. The proxy handler also spawns two threads to listento the sockets and forward incoming data back to theapplication. These threads are responsible for ensuringin-order delivery and generating ACKs. Future workwill explore a using a worker-thread model instead ofa thread per socket to increase connection efficiency.

Connection setup for listening services (Figure 6) happensin a similar fashion, except that for each incoming connec-tion (regardless of the type) the middleware opens a returnconnection to the service application.

5 Evaluation

To evaluate our middleware’s performance, but more im-portantly to discover whether it is beneficial to dynami-

cally swap the network stack out from under an applicationconnection, and to determine the network conditions underwhich it makes sense to do so, we created a DTN networkemulator. The emulator provides a model of a delay tolerantnetwork that we can use to perform end-to-end connectivitytests using real systems.

5.1 Network Emulator

Our network emulator is similar to NIST Net [2] andallows us to introduce arbitrary packet delays and packetdrops into the network. As in NIST Net, our emulator is auser-space program that interfaces with the kernel using thelibipq library. Incoming and outgoing packets are firstpassed to the emulator, which assigns a delay to the packetand schedules the packet for later transmission. This allowsus to capture and control the behavior of arbitrary DTN de-ployments in a single piece of software. Our implementa-tion is independent of any particular DTN routing algorithmor particular network topology. Instead, we apply a delay toevery packet using on a probabilistic distribution based ona given mean. These distributions are given as inputs to theemulator and can be thought of as delay vs. time curveswhich capture the properties of the delay tolerant networkfrom the perspective of an end-to-end connection.

5.1.1 Packet Scheduling

Each packet is assigned a delay when it enters the network,and the value of the delay is calculated from a delay vs. timecurve at a 1/4 second granularity. For example, if a packetenters the system at time t = 14 seconds, and the delay at 14seconds is given by the delay vs. time curve as a normallydistributed random number x with mean 10,000 ms and avariance of 500 ms, the packet will be queued and deliveredat t + x seconds. All packets that enter the network at thesame time are assigned a delay using the same distribution.At quarter-second intervals, the delay distribution changesto a different mean and variance, allowing us to control thenetwork characteristics at a relatively fine level of granular-ity. In this scheme, out of order delivery is entirely possiblesince adjacent delays are completely independent. Duringthe course of our simulations, we tried many different delayvs. time curves, and the motivation behind the curves weselected is provided below.

5.1.2 Packet Loss

To model packet drops, we incorporated a probabilisticpacket drop scheme in the emulator. Each delay vs. timecurve also has a drop probability vs. time curve associatedwith it, and any packet scheduled by the emulator also has achance to be dropped equal to this probability. In practice,our drop probability is simply a function of the delay. For

1000

10000vered

TCP

Delay

1

10

100

0 200 400 600 800 1000 1200 1400

Packets Deliv

Time (sec.)

10000

100000

1000000

ivered

TCP

Delay

1

10

100

1000

0 200 400 600 800 1000 1200 1400

Packets Del

Time(sec.)

(a) TCP-only over Scenario 1 (c) TCP-only over Scenario 2

1000

10000

vered

Dynamic

Delay

1

10

100

0 200 400 600 800 1000 1200 1400

Packets Deliv

Time (sec.)

10000

100000

1000000

elivered

Dynamic

Delay

1

10

100

1000

0 200 400 600 800 1000 1200 1400

Packets De

Time(sec.)

(b) Adaptive TCP/DTN over Scenario 1 (d) Adaptive TCP/DTN over Scenario 2

Figure 7. Throuput vs. Simulation Time for Scenarios 1 and 2. Each graph shows the delay (in gray)imposed on the packets and the measured throughput (in black) of either the TCP-only approach ((a)and (b)) or the mixed TCP/DTN approach ((c) and (d)).

the purposes of this simulation, we naı̈vely assume that, asdelays increase, so does the probability that a packet travers-ing the network will get lost, queue dropped, or run intosome other problem. Our drop probabilities vary between.01% to 5% and are linear functions of the delay. Since TCPis a reliable protocol, packet drops did not cause data loss,but as can be seen from the results, this reliability comes ata very high price in environments with hefty delays. UD-PDTN in its current iteration does not provide any retrans-mit capabilities and relies on the best-effort guarantees ofthe underlying network. As stated previously, this is not anideal mechanism but leads to low overhead and better per-formance in terms of throughput, as demonstrated below.

5.1.3 Scenario 1—Short Delay

In reporting our results, we use two specific delay scenariosto compare a mixed DTN/reliable delivery protocol swapwith a basic TCP-only approach. The first of these scenar-ios models a situation where a well-connected user wandersaway from the well-connected portion of the network andoperates in disconnected mode for a short period of time,then wanders back. This is modeled by a rapidly increas-ing delay from 0 ms to 5000 ms, followed by a period of

two minutes where the delay stays around 5000 ms, andthen comes back down. This cycle happens twice duringthe course of the simulation, which lasts 20 minutes. Thisdelay curve can be seen in Figure 7(a) and (b).

5.1.4 Scenario 2—Long Delay

Our second scenario models a situation where a user leavesthe well connected portion of the network for a longer pe-riod of time, and then comes back. The DTN in which theuser spends the interim time also experiences lengthier de-lays. The user spends 10 minutes operating in this high-delay mode where the packet delays average two minutes.This delay curve is depicted in Figure 7(c) and (d).

5.2 Results

For each scenario, we ran three different tests: one usingonly TCP between the end hosts, one using only UDP, andone using our middleware to swap stacks dynamically dur-ing the connection. In our tests, we make the assumptionthat both TCP and UDP can take advantage of the delay tol-erant routing protocol and can thus be routed into and outof disconnected portions of the network. Therefore TCP

connections do not break under conventional IP route fail-ures, and instead continue operating over the DTN. SinceUDP does not have flow control, we had to limit the datarate to avoid overflow behavior. For this limit, we choseto use the data rate of TCP when TCP is transmitting dur-ing the non-delayed portion of the simulation. Our reason-ing behind this is simple: TCP increases the data rate untilit maximizes the capacity of the network, and the averagemaximum capacity of the network during disconnected op-eration will not be higher than the average maximum capac-ity of the network during fully connected operation. In thisway, we run UDP at a very optimistic data rate, since we areassuming that the delay tolerant routing and network layerscan handle as much traffic as TCP and IP routing layers canhandle. The following sections present the results of bothtests.

5.2.1 Scenario 1—Short Delay

For the TCP throughput test, the TCP socket buffer was keptfull, and TCP was allowed to send as much data as the net-work allowed. Each packet contained 1kB of data. We usedthe standard TCP implementation in the Linux 2.6 kernel(TCP CUBIC [22]). The results of the tests are shown inFigure 7(a). As expected, the throughput of TCP decreasesdramatically during the delayed phases of the simulation,dropping from around 90 packets per second to 10-30 pack-ets per second. Figure 7(b) shows the results of switch-ing to the UDPDTN protocol when the throughput of TCPdrops and switching back when the throughput of TCP risesagain. The packet arrival rate is very bursty during the ‘dis-connected’ portion of the simulation, but it is easy to seethat a large number of packets still arrive during the delayedphases of the simulation. TCP was able to deliver around47,000 packets with no data loss during the 20 minute run.The middleware, using dynamic stack swapping, was ableto deliver around 85,000 packets during the same simula-tion with a data loss of around 950 packets. This is an 80%increase in throughput with 0.01% packet loss. We did notgraph the UDP-only results; they are largely similar to thedynamic results. However, UDP is only a best-effort proto-col, so in the periods of low-delay, it cannot take advantageof TCP’s added end-to-end guarantees. On the contrary, bydynamically switching between TCP and a UDP-like proto-col, UDPDTN garners the advantages of both.

5.2.2 Scenario 2—Long Delay

The results for the second scenario are shown in Fig-ures 7(c) and 7(d). The results are analogous to those forthe first scenario; the disconnected phase is marked by adramatic drop in TCP throughput. Once again, the dynamicstack switching outperforms TCP-only, this time resultingin 60,000 packets delivered vs. TCP’s 31,766. This is a

90% increase in throughput, and it comes at a cost of 1.74%packet loss, with all packet losses occurring during the dis-connected phase.

5.2.3 Summary

The adaptive TCP/DTN stack swapping greatly increasesthroughput in both of the above scenarios, and it is easyto see that in scenarios with longer delays, and more timespent operating over DTNs, the benefits of stack swappingcan only increase.

6 Conclusions

Future mobile computing network deployments will de-mand the ability to dynamically migrate application con-nections from one communications technology to another.Specifically, as Delay Tolerant Networks (DTNs) becomecommonplace, application components will move betweenperiods of good connectivity, where traditional networkingprotocols can be employed, and weaker connectivity, wheretaking advantage of emerging DTN protocols will boostperformance. In this paper, we adopted a systems perspec-tive on enabling applications’ connections to be seamlesslymoved from one communication style to another. By build-ing this architecture in a real operating system and emulat-ing its performance over a real network, we demonstratedthat it is not only reasonable, but potentially highly benefi-cial to employ such an adaptive network architecture. Incomparison to a reliable end-to-end protocol, a protocolstack that adapts can take advantage of the reliability whenfeasible but achieve better throughput in periods of long de-lay by temporarily switching to a best-effort style protocol.When the network improves, the protocol stack can switchagain, reaping the benefits of both styles of communication.

This work has focused on developing a modular soft-ware architecture for enabling adaptation between DTN-aware protocols and more traditional communication stylesand demonstrating that such an adaptive architecture can bebeneficial to applications. We have assumed an omniscientperspective with respect to the points at which the transitionbetween protocols should occur. Ongoing work is exploringhow less perfect information about the network conditionscan be incorporated to maintain the benefit of an adaptivenetwork stack. In addition, we are incorporating real, rep-resentative DTN transport and routing protocols into the ar-chitecture. We expect the performance of the architecture toimprove even further, as the performance of these tailoredprotocols should exceed the naı̈ve UDP-like implementa-tion we have used. In total, the architecture we have pre-sented in this paper is a fundamental first step in smoothlyintegrating DTN technologies and benefits into our existingmobile computing infrastructure.

Acknowledgements

The authors would like to thank the Center for Excel-lence in Distributed Global Environments for providing re-search facilities and the collaborative environment. This re-search was funded in part by the DoD. The views and con-clusions herein are those of the authors and do not necessar-ily reflect the views of the sponsoring agencies.

References

[1] A. Baig, M. Hassan, and L. Libman. Prediction-based recov-ery from link outages in on-board mobile communicationnetworks. In Proc. of IEEE Globecom 2004, pages 1575–1579, 2004.

[2] M. Carson and D. Santay. NIST Net: a Linux-based networkemulation tool. ACM SIGCOMM Computer CommunicationReview, 33(3):111–126, 2003.

[3] A. Doria, M. Uden, and D. Pandey. Providing connectiv-ity to the Saami nomadic community. In Proc. of the 2nd

Int’l. Conf. on Open Collaborative Design for SustainableDevelopment, 2002.

[4] K. Fall. A delay-tolerant network architecture for challengedinternets. Proceedings of the 2003 conference on Applica-tions, technologies, architectures, and protocols for com-puter communications, pages 27–34, 2003.

[5] K. Harras, K. Almeroth, and E. Belding-Royer. Delay tol-erant mobile networks (DTMNs): Controlled flooding insparse mobile networks. In Proc. of the 4th Int’l. IFIP-TC6Networking Conf., pages 1180–1192, May 2005.

[6] T. Hyyryläinen, T. Kärkkäinen, C. Luo, V. Jaspertas,J. Karvo, and J. Ott. Opportunistic email distribution andaccess in challenged heterogeneous environments. Pro-ceedings of the second workshop on Challenged networksCHANTS, pages 97–100, 2007.

[7] O. J and D. Kutscher. A disconnection-tolerant transportfor drive-thru internet environments. In Proc. of the 24th

Annual Joint Conf. of the IEEE Computer and Communica-tions Societies, pages 1849–1862, 2005.

[8] S. Jain, K. Fall, and R. Patra. Routing in a delay tolerant net-work. ACM SIGCOMM Computer Communication Review,34(4):145–158, October 2004.

[9] D. Joseph, J. Kannan, A. Kubota, K. Lakshminarayanan,I. Stoica, and K. Wehrle. OCALA: An architecture for sup-porting legacy applications over overlays. In Proc. of the3rd Symp. on Networked Systems Design and Implementa-tion, pages 267–280, 2006.

[10] P. Juang, H. Oki, W. Want, M. Maronosi, L. Peh, andD. Rubenstein. Energy-efficient computing for wildlifetracking: Design tradeoffs and early experiences with ze-branet. ACM SIGPLAN Notices, 37(10):96–107, October2002.

[11] A. Lindgren, A. Doria, and O. Schelen. Probabilistic rout-ing in intermittently connected networks. In Proc. of the 1st

Int’l. Wkshp. on Service Assurance with Partial and Inter-mittent Resources, pages 239–254, 2004.

[12] H. Madhyastha, A. Venkataramani, A. Krishnamurthy, andT. Anderson. Oasis: An overlay-aware network stack.SIGOPS Operating Systems Review, 40(1):41–48, January2006.

[13] D. Maltz and P. Bhagwat. MSOCKS: an architecture fortransport layer mobility. INFOCOM’98. Seventeenth AnnualJoint Conference of the IEEE Computer and Communica-tions Societies. Proceedings. IEEE, 3.

[14] Y. Mao, B. Knutsson, H. Lu, and J. Smith. DHARMA: Dis-tributed home agent for robust mobile access. In Proc. of the24th Annual Joint Conf. of the IEEE Computer and Commu-nications Societies, pages 1196–1206, 2005.

[15] E. Nordstrom and H. Lundgren. AODV-UU Implementationfrom Uppsala University.

[16] One laptop per child. http://laptop.org, 2007.[17] J. Ott. Application protocol design considerations for a mo-

bile internet. In Proc. of the 1st ACM/IEEE Int’l. Wkshp. onMobility in the Evolving Internet Architecture, pages 75–80,2006.

[18] J. Ott, D. Kutscher, and C. Dwertmann. Integrating DTN andMANET routing. In Proc. of the 2006 SIGCOMM Wkshp.on Challenged Networks, pages 221–228, 2006.

[19] P. Pathirana, N. Bulusu, A. Savkin, and S. Jha. Node lo-calization using mobile robots in delay-tolerant sensor net-works. IEEE Trans. on Mobile Computing, 4(3):285–296,May–June 2005.

[20] A. Pentland, R. Fletcher, and A. Hasson. DakNet: Rethink-ing connectivity in developing nations. IEEE Computer,37(1):78–83, January 2004.

[21] C. Perkins and E. Royer. Ad-hoc on-demand distance vectorrouting. In Proc. of the 2nd IEEE Wkshp. on Mobile Com-puter Systems and Applications, pages 90–100, 1999.

[22] I. Rhee and L. Xu. CUBIC: A New TCP-Friendly High-Speed TCP Variant. PFLDNet05, 2005.

[23] K. Scott and S. Burleigh. Bundle Protocol Specification.IETF Draft, draft-irtf-dtnrgbundle-spec-01. txt, October,2003.

[24] A. Snoeren and H. Balakrishnan. An end-to-end approachto host mobility. In Proc. of the 6th Annual Int’l. Conf. onMobile Computing and Networking, pages 155–166, 2000.

[25] A. Vahdat and D. Becker. Epidemic routing for partiallyconnected ad hoc networks. Technical Report CS-200006,Duke University, April 2000.