Technical Report: Design, Implementation andDeployment of Intermittency-aware Cellular Edge Services

for Rural areas

Talal Ahmad Alfred Afutu Kessir Adjaho Yaw Nyarko Lakshminarayanan Subramanian

ABSTRACTRural areas continue to be plagued by limited data connec-tivity due to poor and unreliable power infrastructure, poorbackhaul connectivity and lack of economic incentives fortelecom providers. These factors continue to be an imped-iment to providing reliable, highly available and bandwidthintensive mobile services and applications to function well insuch contexts. This paper presents the design, implementa-tion and deployment of GreenLinks, a new ground-up plat-form that enables intermittency-aware, reliable and avail-able cellular services and application support in rural con-texts under extreme operational environments with limitedpower and no existing cellular coverage. Unlike the conven-tional monolithic cellular network architecture, GreenLinksenables a distributed (and potentially disjoint) collection ofopen, programmable cellular base stations to offer a gamutof both conventional cellular services and new forms of dis-tributed edge services. GreenLinks offers new applicationprimitives to support different types of intermittency-awaredistributed mobile edge services including P2P transactions,voice-based social media applications, and distributed mo-bile sensing applications. Using a participatory sensing whitespaces approach, a GreenLinks base station can co-exist withother conventional cellular networks.

1. INTRODUCTIONCellular networks have achieved significant penetra-

tion levels in developing regions in the past decade.However, much of this growth has been predominantlycentered in urban regions, with relatively low rural pres-ence [25, 28, 15, 14, 33, 17]. The existing cellular con-nectivity model is not economically viable for rural set-tings due to the following fundamental challenges in ru-ral contexts:Power: Rural regions lack stable and reliable grid power.Cellular networks are inherently power hungry in ru-ral settings (to blanket large areas) and often rely ondiesel-powered generators for power, which is a highlyexpensive proposition [5].Lack of reliable backhauls and data services: Dataconnectivity is largely absent in rural settings; whereconnectivity is available, options are generally restricted

to voice and SMS messages [12, 24]. Backhaul has beena roadblock to the introduction of new data-driven ser-vices.

Reliability: Maintenance of rural wireless networksis very hard due to a myriad of reliability problems,power-related problems and lack of local expertise to fixthem. Lack of clean power is known to trigger frequentdevice failures in rural networks [33].

In this paper, we present the design, implementationand deployment of GreenLinks, a new ground-up cellu-lar network that enables basic communication and pro-vides support for intermittency-aware edge based ser-vices in rural contexts. The basic building of the Green-Links network is a Virtual Cellular Node (VCN) whichare open, programmable cellular base station platforms [29,2, 6, 19] that enable easy deployment of low-power soft-ware defined base stations that function effectively overan IP backplane. In addition, each VCN requires ac-cess to a backhaul link and needs to operate in licensedspectrum band without owning any spectrum. Green-Links enables a distributed collection of VCNs to inter-operate as a single cellular network that can offer re-liable, available and intermittency-aware cellular edgeservices to mobile devices. GreenLinks provides a setof identity management and distributed mobile serviceprimitives that enables conventional cellular services andnew forms of distributed edge services and mobile appli-cations. This paper makes the following contributions:

1. Whitespaces detection: GreenLinks reuses spec-trum allocated to cellular operators using a participa-tory sensing whitespaces approach to determine unusedchannels in order to work without interference with con-ventional cellular operators.

2. Reliable Backhauls: In areas which are at theboundaries of conventional cellular coverage (where sig-nals are negligible, but existent), we demonstrate howone can use a simple directional signal booster to builda reliable backhaul abstraction using parallel boosteddata links between a VCN and a conventional cellulartower.

3. Identity Management: To provide Internet andconventional cellular services, GreenLinks supports an

1

arX

iv:1

604.

0535

5v1

[cs

.NI]

18

Apr

201

6

identity management layer that different users acrossVCNs to join the network without requiring any modi-fications to the end-hosts.

4. Distributed Mobile Edge Services: Given the edgeprogrammability, GreenLinks provides application prim-itives to enable new forms of intermittency-aware, cloud-controlled, distributed, mobile edge services where ev-ery VCN maintains critical application state to enablemobile services to function in an intermittency-awaremanner in the face of unreliable backhauls and networkfailures. These services are also distributed in the sensethat users across VCNs can cooperate and run differ-ent types of peer-to-peer applications and community-specific applications.

We demonstrate the effectiveness of the GreenLinksmodel using a real-world 3-VCN deployment includinga solar-powered cell tower installation in rural Ghana.Our Ghana installation also supports different typesof distributed mobile services (as outlined earlier) thathave been used by rural users within the community.In summary, we believe that GreenLinks represents achange in paradigm in how we design cellular networks,and could enable ground-up innovation where severalbudding rural entrepreneurs could launch competingcellular network services in rural localities while seam-lessly co-existing with existing operators.

2. RELATED WORKSoftware-based microcells: Recent advances in

hardware and open-source software has made availableinexpensive cellular equipment broadly accessible. Forexample, OpenBTS is an open-source GSM base transceiverstation (BTS) implementation which has enabled a widerange of projects aimed towards building small-scaleVCNs. Heimerl et al.[17] demonstrated the viability ofindependently run, locally operated cellular networks.Similarly, Rhizomatica[3] has deployed several community-run cellular networks in Oaxaca, Mexico with a short-term experimental spectrum license. Zheleva et al. [36]deployed a similar system for purely local communica-tions in Zambia.

Backhauls and rural mesh networks: The de-sign of GreenLinks builds upon a large body of workacross rural wireless networks including Fractel [11],Digital Gangetic Plains [7, 13, 28], WiLDNet [25, 33].There has been immense around design of various dif-ferent Line of Sight(LOS) and Non Line of Sight Cellu-lar(NLOS) backhauls [34, 21].

Cognitive Radio and GSM whitespaces: Mostof the literature on whitespaces is in the space focuseson TV spectrum, our work is more closely related towork on re-use of cellular spectrum. Sankaranarayananet al. [31] propose reusing timeslots in a GSM cell foradhoc networks during periods when the GSM cell islightly utilized. Buddhikot et al. [8] describe a sys-

Cloud Controller

Commercial Cellular Networks

Internet

Low-BDR link

Egress Server

VCN 1

Local-IVRLocal-MarketplaceLocal-Sensing

VCN 2

Local-IVRLocal-MarketplaceLocal-Sensing

Cloud-IVRCloud-MarketplaceCloud-Sensing

Figure 1: GreenLinks System Architecture

tem for indoor femtocells to dynamically share spec-trum with incumbent carriers by operating over ultrawide bands. Yin et al. [35] proposes a similar sys-tem and provides measurement results which indicatethat unused spectrum (i.e., whitespace) exists even ina dense, urban environment (Beijing). In contrast tothese, we focus on reusing GSM whitespaces to provideGSM service by means of macrocells. The only workthat recycles GSM whitespaces for reuse by a VCN isNGSM [16]. Later in this paper we highlight how ourapproach for doing GSM whitespaces builds on top ofNGSM and performs better.

Intermittency-aware networks: Dealing with in-termittent failures has been studied in the context ofDelay Tolerant Networks [18, 32]. One of the first realworld DTN deployments was DakNet which provideslow-cost digital communication to rural areas using phys-ical transportation links [26]. There have been severalrouting DTN-based routing algorithms like Encounterbased routing [23] and MobySpace [22]. [9, 20, 10]are other works that use DTN centric “store and for-ward” algorithms to progressively forward messages un-til they reach their destination. The intermittency-awareness abstraction of our solution is fundamentallydifferent from the traditional DTN abstractions sinceGreenLinks is primarily tailored and designed for thecellular network context and primarily focuses on higherapplication-centric “store and forward” abstractions; theDTN routes for messages are preset by the networkstructure which changes infrequently.

3. GREENLINKSIn this section, we describe the GreenLinks architec-

ture and how it enables new forms of intermittency-aware, distributed cellular edge services. The Green-Links architecture is shown in Figure 1. The basicbuilding block of GreenLinks are Virtual Cellular Nodes

2

(VCNs) which leverage open, programmable cellular bases-tations with local compute capabilities. GreenLinksaims to connect multiple VCNs which can be locatedin different geographic localities to cohesively behaveas a single cellular provider. GreenLinks is specificallydesigned for rural contexts with limited or no existingcellular coverage and we assume that each VCN oper-ates with a backhaul link with highly constrained band-width and high latency. We define such backhaul linksas low Bandwidth-Delay Ratio (BDR) links. Examplesof such backhaul links are: existing 3G coverage withweak signals, low bandwidth satellite links, microwavelinks and long distance Wi-Fi links. The backhauls canfail arbitrarily due to loss in power or other exogenousfactors.

GreenLinks offers a combination of conventional cel-lular services (voice calls, messaging, voice messagesand data services) and new forms of intermittency awaredistributed applications. Despite constant power fail-ures and network failures which may be common inrural contexts, GreenLinks aims to provide high avail-ability of basic services and high interactivity for dis-tributed applications. To achieve this, GreenLinks of-fers an intermittency aware service abstraction whereevery cloud service or application offered to mobile usershas a cloud instance and several edge instances (alsoreferred as local instance). The local instance of eachapplication runs at the compute server attached to thebase station, while the cloud instance runs in a datacenter. We partition each application in such a waythat most of the application processing is done at thelocal component. In addition, each application needsan intermittency-aware synchronization mechanism be-tween the local instances running across VCNs and thecloud instance to ensure the consistency or availabil-ity guarantees required for various functions within theapplication. Figure 1 outlines three different exampledistributed applications that we have built on top of theGreenLinks service abstraction: (a) a Marketplace ap-plication for supporting P2P transactions across users;(b) an IVR based social media application; (c) dis-tributed data gathering and sensing. We describe theseapplications in greater detail in the paper. We also havean Egress server which is responsible for communicationbetween VCNs and the conventional cellular networks.

3.1 Our VCN deploymentOur deployment consisted of three VCNs located in

the USA, Ghana and UAE respectively, with one cen-tralized controller located in Germany. Figure 2 showsthe deployment we have in Ghana, at the edge of exist-ing cellular coverage by commercial telcos. Pictured onthe left side of Figure 2 is the cellular base station hard-ware used, a RapidCell manufactured by Range Net-works[29]. To the right side of Figure 2 is the backhaul

Figure 2: Our VCN setup in Kumawu, Ghana

and base station antennae, and the solar panels used topower the complete setup. We used this VCN to ex-tend coverage beyond the existing coverage area, withthe backhaul comprising of analog signal booster witha directional antenna (facing the 3G base station) toboost the weak 3G signal to create a backhaul for theVCN. We also deployed a whitespaces solution to makesure that we do not interfere with the primary operatorsin the area while we try to use the bands in spectrum noone else was using. We also deployed 3 applications ontop of the Ghana VCN and used synchronization prim-itives to enable applications to run across Ghana andUS VCNs.

We used two hardware platforms as Base TransceiverStation (BTS) nodes. While in Ghana we used a 1Watt RapidCell from Range Networks [29], in USA andUAE we used the 10 mWatt USRP B100 from EttusResearch. Freeswitch PBX was used for routing callsand messages in all VCNs. As the local component ofthe IVR application, we deployed an Asterisk PBX oncompute node to handle IVR calls. The local serverwas also running an HTTP server for the SMS basedand data sensing applications.

3.2 Identity ManagementA key building block for supporting different forms of

services in GreenLinks is an intermittency aware iden-tity management layer. GreenLinks supports three ba-sic types of identities: network identities, user identitiesand application identities. In GreenLinks, the commonuser identity is the the International Mobile SubscriberIdentity (IMSI) which is physically written on a SIMcard and the network identity is the identity issued byNetwork against the IMSI, this is commonly referred toas the phone number of the user. GreenLinks supportsa generalization of the definition of these network anduser identities, in that user identities can encompassother broad notions of identities and network identitiesadditionally encompass IP addresses. Given the basicnetwork setup of a disjoint collection of VCNs inter-connected over an IP backplane, the cloud instance ofGreenLinks is publicly reachable (with a public IP ad-dress) while each VCN may be positioned behind a

3

NAT. GreenLinks assumes a flat 2-level hierarchy whereeach VCN independently maintains a connection withthe cloud and all synchronization and information rout-ing is performed through the cloud instance.

The basic identity lookup process translates a useridentity to determine the network identity within theGreenLinks network. GreenLinks has a lookup serverinstance within each VCN and cloud instances. Thetraditional lookup would typically involve a local lookupwithin the VCN and if locally unavailable, a cloud lookupfor the same user identity. The link between the VCNand the cloud has a low BDR and is typically inter-mittent. This raises four challenges: availability, con-sistency,external names and mobility. First, to ensurehigh availability of identity lookup, GreenLinks sup-ports quick local authentication of user identities, forpre-registered users and two users within the same VCNcan communicate with each other without the need forbackhaul connectivity to the cloud. A user is pre-registeredif the user identity has been authenticated by the cloudinstance and the identity is active in the local cache.Second, to ensure consistency of identities, local serversat the VCN cannot assign global user identities but onlythe centralized name server (co-located with the cloudcontroller) issues identities to all VCNs to avoid nam-ing conflicts. Third, the GreenLinks network can oper-ate on two types of names: unique user identities as-signed by GreenLinks or unique user identities assignedby external trusted entities (cellular identities, etc.). Inthe former case, the central name server has completecontrol over the user identity space and controls theuniqueness of GreenLinks assigned names. To enhanceavailability of names, the central server can partitionthe name space and also provide local VCNs auton-omy to assign locally unique user identities. GreenLinksassigned names have no semantic meaning outside theGreenLinks network. In the case of user identities as-signed by external trusted identities, the GreenLinkscloud name server needs to communicate with the ap-propriate external entities to verify their authenticity.Finally, given the intermittency aware lookup process,GreenLinks is not best suited to support rapid mobilityof user identities across VCNs. In specific cases, whereneighboring VCNs can directly communicate with eachother using a local backhaul, then GreenLinks can sup-port limited forms of local mobility across VCNs.

GreenLinks also supports distributed applications todefine application specific identities, which refer to uniqueinformation state of the application that may be tiedwith one or a group of user identities. For example,consider the case where two users wish to perform aBUY/SELL transaction over GreenLinks across multi-ple VCNs; each item transacted is associated with aunique application identity and each buy/sell bid is amapping between application identities and user iden-

tities. Applications can specify their own consistency,availability and intermittency-aware tradeoffs across theirapplication defined identities. This state is used by thelocal instances of the application running at each VCNto selectively synchronize state with the cloud instanceof an application.

GreenLinks also supports intermittency-aware forward-ing of state for applications and user-directed messages.As a simple example, the messaging application in Green-Links enables users to leave voice messages or text mes-sages with a user identity (this can be a GreenLinksissued identity or even a globally unique mobile num-ber), which can be locally stored in a VCN and lazilysynchronized with the cloud instance.

3.3 Distributed Application PrimitivesGreenLinks supports a collection of basic application

primitives that enables developers to easily translateexisting distributed applications with minimal changesto operate in intermittency-aware conditions. The keyfocus of primitives is to enable an intermittency-awarekey-value store abstraction over the application iden-tity space that enables applications to specify the righttradeoff between consistency and availability. Since mostVCNs are located in low-BDR regions, we achieve par-tial availability by dividing each application into a largerlocal component and a smaller cloud component. Thelocal component of the application is to ensure sig-nificant local processing. This ensures responsiveness(or application liveness) by keeping local communica-tion and interaction functional even in the absence ofthe low-BDR backhaul link. The cloud component ismainly responsible for saving application data and fa-cilitating interaction between the local components ofvarious VCNs. The lack of programmability in conven-tional cellular networks forces application deploymentextensively in the cloud. This means that failure of abackhaul link between the base station and core net-work inhibits access to even basic applications like callsand SMS. We discuss some of the primitives we designedand implemented:

1. SLOWPUT(Identity i, Type t, Data d):User i communicates marshalled data d for applicationt. This primitive is used by the local component for lazycommunication with the cloud component. This opera-tion is lazy because data may not be immediately sentafter the primitive is executed; the local server deter-mines the right order to send the data based on band-width availability and priority (based on t) of the otherrequests in the queue. The SLOWPUT primitive is used byapplications for which the local component can provideavailability at the expense of strong consistency. For ex-ample if someone is just try to send an e-mail from VCNA to a user connected to VCN B, any inconsistency inthe ordering of e-mails at receiver is tolerable.

4

2. FASTGET(Identity i, Type t, Data d):This primitive is used by the local component of any ap-plication to send and receive data from the cloud com-ponent in an active manner. The particulars include auser i communicating data d for application t. Thisprimitive is usually called as a result of an applicationfunction when consistency and response time can notbe compromised. The communication is not queued,as in SLOWPUT primitive. Instead, this primitive is ex-ecuted instantaneously, provided the the backhaul linkis available. For example, any inconsistencies in thecase of a banking transaction are grave, and an strongconsistency has to be ensured even if the user does notget a response immediately. Such applications use theFASTGET primitive and marshall the transaction in thedata parameter, forwarding it to the central componentof the banking application.3. FASTSEARCH(Identity i, Type t, Data d):This primitive is used by the user i to search for datad of a particular application type t, and returns one ormore data points based on search from the cloud compo-nent’s data store. The communication encapsulated bythe FASTSEARCH primitive occurs instantaneously. TheFASTSEARCH primitive executes a read operation on thepersistent storage of the cloud component, therefore itcan be performed even if a FASTGET operation has awrite lock. An example use-case of the FASTSEARCH

primitive can be to search in the data store of cloudcomponent while a user is on hold with an IVR-basedapplication, and is expecting a response quickly.

3.4 Example ApplicationsIn this subsection, we give the examples of distributed

applications that we built using the primitives discussedabove. In particular, we highlight how a functional divi-sion is made to identify the functions that require con-sistency to those that require availability.

3.4.1 P2P TransactionsThe application provides a text-based interface (run-

ning on top of the SMS service) and enables tradersto provide customers the price and quantity of a com-modity (usually crops) they wish to sell. It also en-ables customers to search for a particular commodity,the price at which various traders are selling, and toplace a BUY request. The BUY request leads to exchangeof contact information between transacting parties, andis logged as a SELL. The text-based interface requiressmall phrases to commence transactions.

The lazy SLOWPUT is used send the data to the serveron sale of a commodity. This is because the orderin which SELL request is updated at the cloud doesnot require strong consistency, but requires availabil-ity. Therefore when a user tries to sell, we acknowledgethe sell has been received locally(by sending a message

back to user) and transfer the sell to cloud in a lazymanner. In case the user is interested in purchasing acommodity, FASTSEARCH is used, and the data returnedis sent to the user over one or more text messages. Thisis because we want strong consistency and do not wantto give user a response immediately without ensuringthat we are actually able to update the record in thecloud.

3.4.2 IVR-Based Social MediaThis application enables users to interact with the

system by pressing numbers on the phone’s keypad.It enables users to leave a message, or to listen to amessage left by others. For the Ghana VCN, the ap-plication provided an option to either use English orTwi, the local language in the region. The key functionis the ReceiveHandler at the local component, whichis invoked using a HTTP request on entering the IVRextension. After the initial HTTP request, the call isbridged from base station to the Asterisk IVR serverrunning as the local component of application. The As-terisk server logs each input of user as entered. If theuser is recording a message, the ReceiveHandler is ex-ecuted after its completion. The ReceiveHandler usesthe lazy SLOWPUT primitive to send the recorded file tothe cloud component because a strong consistency inthe ordering of messages received is not required. Incase a person wants to listen to a message, the func-tion first checks if the validity of the session. If thesession is valid the message is played locally. OtherwiseFASTSEARCH and FASTGET primitives are used to searchand fetch the most recent message from the cloud com-ponent, while the user is on hold. We use these oper-ations because user is on hold and we want to respondto the user quickly.

3.4.3 Distributed Data Gathering & SensingThe purpose of the application was to collect data

on farms and crops grown. The application providesan interface to create the map of any farm by walkingalong its edge. This data is sent to the local applicationcomponent, an HTTP server next to the base station.The local component then uses SLOWPUT to upload thisdata to the cloud component. Our choice in primitive isvalid as the application does not require strong consis-tency and we can afford to have unordered data at thecloud component. This application is relatively simplebecause data only needs to be uploaded to the cloudcomponent of the application, and there is no need fordownloading data.

4. OPERATIONAL CHALLENGES

4.1 Detecting WhitespacesGSM spectrum license is required for operating in a

5

Figure 3: Used and unused channels over one week inKumawu Ghana.

particular area. Most countries issue licenses to a hand-ful of national carriers and have no regulatory frame-work for smaller providers. For our VCN, we had nospectrum allocated to operate. This is true for mostcommunity based cellular deployments [16, 3]. Thoughlicensed in its entirety to primary operators, majority ofthe cellular channels remain unused in rural localities.These unused bands of spectrum are called GSM whites-paces. In order to identify and reuse these whitespaceswithout any collaboration from primary providers, webuild a sensing-based participatory model for collectingchannel energy information in the locality.

Our solution takes advantage of the fact that GSMhandsets continually measure spectrum occupancy intheir proximity and report these measurements to theirserving cell. Phones do this to enable the serving cellto negotiate a hand-over with a cell tower that has astronger signal. We utilize this functionality and pro-vide cellular devices with channels to scan. To phones,these are the channels on which neighboring base sta-tions (of our cellular network) are operating - so theylook for energy on those channels and report it. Al-though this approach is similar to NGSM [16], we makethe following improvements: (i) We do not necessitatetwo different base stations. Instead, we wait for theconnected calls to complete in case it is time to shift toa different channel, (ii) We increase transmission powerperiodically based on sensory feedback from phones toslowly expand coverage, unlike NGSM which starts atfull power on a random channel and has a greater pos-sibility of interfering with primary, and (iii) We havea voluntary model where we incentivize volunteers toinstall an application that periodically makes calls andsends messages to the base station thus ensuring peri-odic stream of measurement reports even in low activityperiods (discussed later in the section).

We modified the peering component of OpenBTS toadd fake neighbors, and add the corresponding chan-nel that we intend to scan to each fake neighbor. Westop OpenBTS from validating the fake neighbors us-ing an IP-based protocol. Once the fake neighbors aresuccessfully set, we wait for the base station to send thenew neighbor list out to the phones. Once the phones

(a) Channel 45 energy (b) Channel 45 frequency

(c) Channel 48 energy (d) Channel 48 frequency

Figure 4: Channel energy as reported by various mea-surement reports and frequency of reports at a 5 minutegranularity.

get the neighbor list, they start reporting the energy onthe fake neighbor channels. We constantly run tcpdump

to collect MeasurementReport packets being exchangedbetween phones and the base stations. For this, wemodify the code base [1] made available by the devel-opers of NGSM. In Figure 3, we present the detectedwhitespaces over the period of one week in 124 channelsof the P-GSM-900 band. It can be observed that only9 channels out of 120 were being used, while the rest ofthe channels were unoccupied.

Apart from this we also developed an Android appli-cation that was given to a few volunteers. This appli-cation helped us gather reports frequently by sendingSMS between volunteers and the base station. Morecommunication by phones guarantees that regular re-ports are exchanged. Figure 4a shows, in chronologicalorder, the various readings over a period of approxi-mately 4 hours. Figure 4b shows, for the same channel,the plot of how many reports were received every 5 min-utes. These two graphs highlight the fact that there isa constant need for sensing the channel because evenfor channels clearly being used by a neighboring basestation, there are a significant number of reports sug-gesting that the energy is zero (the horizontal line aty=0 in Figure 4a). Taking a closer look at reports per-taining to channel 48 in Figure 4c and Figure 4d, we ob-served that it can take thousands of zero energy packetsand several minutes before a packet reporting a positiveenergy on a channel is received. One possible reason ex-plaining this phenomenon is that the overlap radius isvery small and only a few mobile devices (maybe sin-

6

Airtel

Approximate 3G coverage radius of Airtel tower

Ghana-VCN

Figure 5: Location of the Ghana VCN and nearby con-ventional tower

(a) UDP (b) TCP

Figure 6: Performance of Airtel connection with andwithout boosted backhaul

gle device) were able to see energy on a channel andfor a very small amount of time. Because of this, weargue for having some phones as volunteers in the areawho have our application which keeps sending SMS inthe background to other volunteers and the base stationand produces regular report. In the case of channel 45,we have a relatively constant stream of packets acrosstime as seen in Figure 4b but while in case of channel48 in Figure 4d there are several minute gaps where noreport is received about the channel. This highlightsthe need for having more volunteers; we add multiplefake neighbors and there is no way of controlling a singlechannel from which we receive reports.

4.2 Boosted Cellular BackhaulDesigning reliable backhauls in rural contexts is a

challenging proposition. We outline a simple and poten-tially powerful approach for building reliable backhaulsusing directional signal boosters coupled with parallelcellular connections. Specifically in areas which are atthe edge of cellular connectivity, we believe that the par-allel boosted links represents an easy-to-deploy solutionfor building reliable backhauls.

In Figure 2, we have pointed out the backhaul’s an-tenna, which is a high gain directional receiving antennapointing towards the 3G base station. The rest of thebackhaul setup consists of a signal repeater attachedto an indoor omni-directional antenna. The weak sig-nal is received at the directional antenna and is fed tothe repeater using coaxial cables. The output from therepeater is transferred to the indoor antenna, again us-ing coaxial cables. The key challenge stems from the

fact that the Ghana-VCN was located at the edge ofcoverage as shown in Figure 5. By mounting the receiv-ing antenna high enough, we were able to get a decentboosted 3G signal. We had multiple phones containingSIM cards belonging to the Airtel Network next to theindoor antenna. We tethered these phones to our setupand had multiple parallel boosted links as backhauls.It is important to note that we were able to obtain 3boosted links with a single signal booster.

We then used approaches like Ubuntu Bonding [4] touse the multiple backhauls together in various configu-rations like active backup (where one link is used andthe other is backup) and load balancing (where outgo-ing traffic is divided into the available links equally).However, the major limitation of Ubuntu Bonding is itsinability to divide a single flow into multiple subflows.Multi-path TCP [27] is another option that can helpaggregate multiple links, but it requires changes to theLinux kernel and some support from middleboxes.

We used Phonetone K3B0918217CLB kit which com-prised of a 10 db directional receiving antenna, 9db in-door panel antenna, and 70db analogue booster capa-ble of working in 900/1800/2100 MHz GSM bands. Wewere able to capture the 3G signal from Airtel whichwas not available earlier in the vicinity of VCN. In Fig-ure 6, we present the performance of the signal boosterbased backhaul in Ghana, where the bandwidth gainin TCP, and packet loss and jitter improvement in thecase of UDP (both measured using iperf) is plottedfor a single boosted link against trials. The parenthe-ses next to each point in the graph contains the av-erage measured RSSI value. This value was measuredusing Android phones that utilized the boosted back-haul to the BTS node. The effect of signal boostingof the Airtel network is evident in Figure 6; we wereable to change the transmission rates from EDGE (de-noted by E in graph) to that of HSDPA (denoted byH in graph). Consequently, we were able to get signif-icantly more bandwidth as reported by Figure 6b. Wealso reduced the packet loss rate and jitter in UDP, asreported by Figure 6a.

5. APPLICATION EXPERIENCESAll the three distributed applications we developed

have been deployed and tested in the local rural com-munity in Kumawu, Ghana. We outline some of thetechnical experiences in running these applications onthe wild. Due to space considerations, we primarily fo-cus on the marketplace P2P transaction application.

Marketplace application: To test the local andcloud component of the Marketplace application, webuilt an Android application that enables phones to au-tomatically SELL and BUY items in a periodic manner.We had 4 phones sending SELL messages and 3 phonessending BUY and SEARCH messages. 5 of the 7 phones

7

(a) Ghana (b) New York

Figure 7: CDF of time it takes for the different functionsin the P2P Marketplace application

(a) (b)

Figure 8: CDF of time taken for the SLOWPUT of file vsSMS and a view of the queue in the P2P Marketplaceapplication

were connected to the BTS in Ghana and the otherswere connected to the BTS in New York. Simultaneouswith the above application we made calls manually anda script running at local server generated data to sim-ulate the marketplace server receiving data. The nodein UAE did not participate in these measurements.

We report the performance of the P2P transactionsapplication in Figure 7. The graph above shows theend-to-end latency between the local server and cloudcontroller for the BUY,SELL and SEARCH operations. Itis important to recall that the SELL operation trans-lates to the SLOWPUT primitive, while BUY and SEARCH

translate to FASTGET and FASTSEARCH primitives respec-tively. It can be observed in Figure 7a that BUY,SELLand SEARCH operations cause nearly the same end-to-end latency. However in Figure 7b, there is a clear dis-tinction between the time it takes for SELL, comparedto BUY and SEARCH. This is expected because SELL re-quests are queued and processed in a lazy manner. Thisdistinction is not visible in Figure 7a as that the band-width between cloud and local server is low while theNew York VCN was connected to the cloud over a highbandwidth Ethernet link. This results in similar latencymeasurements for all three operations. It is importantto note that no failures were experienced and all mes-sages sent from both nodes were received at the serverdespite the fact that Ghana node was using a signal

booster based backhaul. At a user level user got SMSback for all the functions they performed. They gotSMS for SELL immediately because the local componentof the application gives the response immediately andthen executes the SLOWPUT operation while the responsefor BUY operation is sent only after response from cloudto maintain consistency. It is important to note thateven though there were no failures the users in Ghanaexperienced high latency as is evident from the timegiven on x-axis in Figure 7a and Figure 7bSensing and Voice Applications: File upload is a

common operation of the voice-based or sensing-basedapplications. Figure 8a shows the CDF of time it takesfor different SLOWPUT requests on the New York-basedVCN. Figure 8b shows the queue and how requests areprocessed for the SLOWPUT primitive for the VCN inGhana. Figure 8 shows that SMS-based applicationswill work faster than the file based applications as theyare smaller than files. It also shows that file based ap-plications can slow down the SMS-based applications.This is true as both are placed in a common queue forprocessing, and servicing larger files is more time con-suming than servicing smaller SMSs.Summary of User Experiences: The P2P mar-

ketplace application has been tested with cohorts offarmers and traders from the local community to per-form BUY/SELL transactions. The distributed sensingapplication has been used extensively to map farmingboundaries and collect meta-data about farms in the ru-ral region. Using this application, we have successfullybuilt GIS agricultural maps of different agricultural pro-duce in the locality. The IVR based social media appli-cation was inspired by the success of voice based citi-zen journalism platforms like Polly [30] and shows howone can build such voice based applications to functioneffectively in rural contexts using intermittency-awareVCNs.

6. CONCLUSIONSThis paper presents our experience in the design,

implementation and deployment of GreenLinks, a newground-up platform that enables intermittency-aware,reliable and available cellular services and applicationsupport in rural contexts under extreme operationalenvironments with limited power. Greenlinks providesnew primitives for identity management and buildingnew distributed applications to operate effectively inhighly intermittent environments. We present our expe-riences building three example intermittency-aware dis-tributed applications and have used these applicationson the ground in a rural context in Ghana. In addition,to address operational challenges in Ghana, our imple-mentation supports mechanisms for building incremen-tally deployable reliable backhauls and easy detectionof whitespaces using participatory sensing.

8

7. REFERENCES

[1] Github-gsmws repository.https://github.com/shaddi/gsmws.git.Accessed: 2015-11-05.

[2] Openbts. www://www.openbts.org/.[3] Rhizomatica community base station.

http://rhizomatica.org/projects/

community-basestation/. Accessed: 2015-11-05.[4] Ubuntubonding.

https://help.ubuntu.com/community/UbuntuBonding.[5] T. Ahmad, S. Kalyanaraman, F. Amjad, and

L. Subramanian. Solar vs diesel: where to drawthe line for cell towers? In Proceedings of theSeventh International Conference on Informationand Communication Technologies andDevelopment, page 7. ACM, 2015.

[6] M. Bansal, J. Mehlman, S. Katti, and P. Levis.Openradio: a programmable wireless dataplane.In Proceedings of the first workshop on Hot topicsin software defined networks, pages 109–114.ACM, 2012.

[7] P. Bhagwat, B. Raman, and D. Sanghi. Turning802.11 Inside-out. ACM SIGCOMM CCR, 2004.

[8] M. M. Buddhikot, I. Kennedy, F. Mullany, andH. Viswanathan. Ultra-broadband femtocells viaopportunistic reuse of multi-operator andmulti-service spectrum. Bell Labs TechnicalJournal, 13(4):129–143, 2009.

[9] J. Burgess, B. Gallagher, D. Jensen, and B. N.Levine. Maxprop: Routing for vehicle-baseddisruption-tolerant networks. In INFOCOM,volume 6, pages 1–11, 2006.

[10] A. Chaintreau, P. Hui, J. Crowcroft, C. Diot,R. Gass, and J. Scott. Impact of human mobilityon opportunistic forwarding algorithms. MobileComputing, IEEE Transactions on, 6(6):606–620,2007.

[11] K. Chebrolu and B. Raman. FRACTEL: A FreshPerspective on (Rural) Mesh Networks. In NSDR2007.

[12] A. Dhananjay, A. Sharma, M. Paik, J. Chen,T. K. Kuppusamy, J. Li, and L. Subramanian.Hermes: data transmission over unknown voicechannels. MobiCom ’10.

[13] Digital Gangetic Plains.http://www.iitk.ac.in/mladgp/.

[14] V. Gabale, A. Chiplunkar, B. Raman, andP. Dutta. Delaycheck: Scheduling voice overmulti-hop multi-channel wireless mesh networks.In COMSNETS, 2011.

[15] V. Gabale, B. Raman, K. Chebrolu, andP. Kulkarni. Lit mac: Addressing the challengesof effective voice communication in a low cost, lowpower wireless mesh network. In Proceedings ofthe First ACM Symposium on Computing for

Development, ACM DEV ’10, pages 5:1–5:11,New York, NY, USA, 2010. ACM.

[16] S. Hasan, K. Heimerl, K. Harrison, K. Ali,S. Roberts, A. Sahai, and E. Brewer. Gsmwhitespaces: An opportunity for rural cellularservice. In Dynamic Spectrum Access Networks(DYSPAN), 2014 IEEE International Symposiumon, pages 271–282. IEEE, 2014.

[17] K. Heimerl, S. Hasan, K. Ali, E. Brewer, andT. Parikh. Local, sustainable, small-scale cellularnetworks. In Proceedings of the SixthInternational Conference on Information andCommunication Technologies and Development:Full Papers-Volume 1, pages 2–12. ACM, 2013.

[18] S. Jain, K. Fall, and R. Patra. Routing in a DelayTolerant Network. In ACM SIGCOMM, pages145–158, New York, NY, USA, 2004. ACM.

[19] X. Jin, L. E. Li, L. Vanbever, and J. Rexford.Softcell: Scalable and flexible cellular corenetwork architecture. In Proceedings of the ninthACM conference on Emerging networkingexperiments and technologies, pages 163–174.ACM, 2013.

[20] P. Juang, H. Oki, Y. Wang, M. Martonosi, L. S.Peh, and D. Rubenstein. Energy-efficientcomputing for wildlife tracking: Design tradeoffsand early experiences with zebranet. In ACMSigplan Notices, volume 37, pages 96–107. ACM,2002.

[21] Kumu Networks 2014. Kumu networks.http://kumunetworks.com/.

[22] J. Leguay, T. Friedman, and V. Conan.Evaluating mobility pattern space routing fordtns. arXiv preprint cs/0511102, 2005.

[23] S. C. Nelson, M. Bakht, and R. Kravets.Encounter-based routing in dtns. In INFOCOM2009, IEEE, pages 846–854. IEEE, 2009.

[24] M. Paik, A. Sharma, A. Meacham, G. Quarta,P. Smith, J. Trahanas, B. Levine, M. A. Hopkins,B. Rapchak, and L. Subramanian. A Case forSmartTrack. ICTD 2009.

[25] R. Patra, S. Nedevschi, S. Surana, A. Sheth,L. Subramanian, and E. Brewer. WiLDNet:Design and Implementation of High PerformanceWiFi Based Long Distance Networks. Proceedingsof NSDI 2007, 2007.

[26] A. Pentland, R. Fletcher, and A. Hasson. Daknet:Rethinking connectivity in developing nations.Computer, 37(1):78–83, 2004.

[27] C. Raiciu, C. Paasch, S. Barre, A. Ford,M. Honda, F. Duchene, O. Bonaventure, andM. Handley. How hard can it be? designing andimplementing a deployable multipath tcp. InProceedings of the 9th USENIX conference onNetworked Systems Design and Implementation,

9

pages 29–29. USENIX Association, 2012.[28] B. Raman and K. Chebrolu. Design and

Evaluation of a new MAC Protocol forLong-Distance 802.11 Mesh Networks. In ACMMOBICOM, Aug. 2005.

[29] Range Networks. Range networks.http://www.rangenetworks.com/.

[30] A. A. Raza, M. Pervaiz, C. Milo, S. Razaq,G. Alster, J. Sherwani, U. Saif, and R. Rosenfeld.Viral entertainment as a vehicle for disseminatingspeech-based services to low-literate users. InProceedings of the Fifth International Conferenceon Information and Communication Technologiesand Development, pages 350–359. ACM, 2012.

[31] S. Sankaranarayanan, P. Papadimitratos,A. Mishra, and S. Hershey. A bandwidth sharingapproach to improve licensed spectrumutilization. In New Frontiers in DynamicSpectrum Access Networks, 2005. DySPAN 2005.2005 First IEEE International Symposium on,pages 279–288. IEEE, 2005.

[32] A. Seth, D. Kroeker, M. Zaharia, S. Guo, andS. Keshav. Low-cost communication for ruralinternet kiosks using mechanical backhaul.Proceedings of the 12th annual internationalconference on Mobile computing and networking,pages 334–345, 2006.

[33] S. Surana, R. Patra, S. Nedevschi, M. Ramos,L. Subramanian, Y. Ben-David, and E. Brewer.Beyond Pilots: Keeping Rural Wireless NetworksAlive. Proceedings of NSDI 2008, 2008.

[34] Tarana Wireless 2014. Tarana wireless.http://www.taranawireless.com/.

[35] L. Yin, K. Wu, S. Yin, S. Li, and L. M. Ni. Reuseof gsm white space spectrum for cognitivefemtocell access. In Proceedings of the 2012 IEEE18th International Conference on Parallel andDistributed Systems, pages 1–7. IEEE ComputerSociety, 2012.

[36] M. Zheleva, A. Paul, D. L. Johnson, andE. Belding. Kwiizya: Local cellular networkservices in remote areas. In Proceeding of the 11thAnnual International Conference on MobileSystems, Applications, and Services, MobiSys ’13,pages 417–430, New York, NY, USA, 2013. ACM.

10