wireless network emulation for research on information ... · wireless network emulation for...

Wireless Network Emulation for Research onInformation-Centric Networking

Philipp Moll, Sebastian Theuermann, Hermann HellwagnerInstitute of Information TechnologyAlpen-Adria-Universität Klagenfurt

[email protected]

ABSTRACTWhen developing new approaches in networking research,one of the most important requirements is to evaluate thedegree of improvement of a new approach both realisticallyand cost-effectively. Wireless networks and their adequateemulation play an important role in evaluation, but emula-tion of wireless links and networks is still difficult to handle.In this paper, we present a low-cost, fixed-network testbedable to emulate the dynamically changing conditions of wire-less links caused by client mobility and physical phenomena.We extend the existing fixed-network testbed for the pur-pose of wireless network emulation using the Linux toolstc, iptables, and NetEm in sophisticated ways. Convenientfunction blocks are provided to configure wireless networktopologies as well as dynamic link and mobility conditionsto be emulated with modest efforts. We utilize the testbed’scapabilities to investigate the influence of different mobilitymodels on streaming SVC-encoded videos in Named DataNetworking (NDN), a novel Information-Centric Networkingarchitecture. Furthermore, we evaluate the benefits of usingearly loss detectionmechanisms for streaming inNDN, by im-plementing Wireless Loss Detection and Recovery (WLDR).Our results show that the extended fixed-network testbedcan precisely emulate wireless network conditions and usage.For instance, the emulation revealed that both the choice ofthe mobility model and the use of WLDR have a substantialinfluence on the resulting SVC video streaming performance.

CCS CONCEPTS• Networks → Network performance evaluation; Net-work architectures; Physical links; Wireless access networks;

Permission to make digital or hard copies of part or all of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contactthe owner/author(s).WiNTECH ’18, November 2, 2018, New Delhi, India© 2018 Copyright held by the owner/author(s).ACM ISBN 978-1-4503-5930-6/18/11.https://doi.org/10.1145/3267204.3267211

KEYWORDSWireless Link Emulation; Networking Testbed; Information-Centric NetworkingACM Reference Format:Philipp Moll, Sebastian Theuermann, Hermann Hellwagner. 2018.Wireless Network Emulation for Research on Information-CentricNetworking. In 12th International Workshop on Wireless NetworkTestbeds, Experimental Evaluation & Characterization (WiNTECH’18), November 2, 2018, New Delhi, India. ACM, New York, NY, USA,10 pages. https://doi.org/10.1145/3267204.3267211

1 INTRODUCTIONThe global Internet traffic increases year by year and no endof this trend is foreseeable. This development triggers re-search on improvements of our current Internet architecture,but also on the development of new networking architectures.In both cases, it is important to analyze newly developedapproaches and to check if they are suited to be used on theInternet.For analyzing newly developed approaches, multiple op-

tions are available. One option is theoretical analysis, whichcan be used to verify the correctness of an approach or tocalculate its theoretical performance. The results of sucha theoretical analysis can be validated by event-based net-work simulations, which enrich the theoretical analysis byadding the basic characteristics and influences of computernetworks. Nevertheless, event-based simulations only in-clude abstractions of the real world and do not considerunderlying physical phenomena or hardware restrictions. Anetwork emulation is conducted with real devices and com-munication over real networks and is thereby a valuableaddition to simulation.Wireless links are an essential part of today’s networks.

According to Cisco’s Visual Networking Index1, 43% of allnetworked devices will be mobile-connected by 2021. There-fore, an option to emulate wireless links is required, butdifficult to realize because the performance of wireless linksdepends on various factors, including, but not limited to, re-flections, interference, and clients’ movements. In this paper,

1https://www.cisco.com/c/en/us/solutions/service-provider/visual-networking-index-vni/index.html, last visited: 2018-06-25

Session: Testbeds, Measurements and Experimentation WiNTECH’18, November 2, 2018, New Delhi, India

46

https://doi.org/10.1145/3267204.3267211

https://doi.org/10.1145/3267204.3267211

we extend an existing emulation testbed with the aim to beable to emulate dynamic link properties and thereby enablethe testbed to be used for wireless network emulation.

Another ongoing trend is the change of the Internet’s us-age from host-to-host communication to content distribution,which is also reflected by current developments in network-ing research. Research on Information-Centric Networking(ICN) aims to better support this content-based communica-tion. In Named Data Networking (NDN) [14], a promisingICN architecture, each piece of information is identifiable bya system-wide unique name. Data is requested by emittingan Interest packet, carrying the name of the desired informa-tion. When the Interest reaches a node which holds a copyof the information, the information is packaged into a Datapacket and sent back to the requester on the reverse path ofthe Interest.

More and more real-world applications for NDN are devel-oped, but a proper way to test these applications in wirelessnetworks is still missing. Our testbed provides the possibil-ity to run real-world NDN applications and to test them inarbitrary networks. From previous work [8, 10], we alreadyknow that NDN is well suited for adaptive video streamingin fixed network environments. We assume that mobility hasa strong impact on streaming performance, which is whywe now use the presented testbed to analyze the influenceof mobility on video streaming quality.

The main contribution of this paper is the presentation ofour open-source wireless network emulation testbed and adetailed overview of the function and usage of the testbed.We further analyze the capabilities of the Linux tools tc andNetEm and show their suitability to emulate wireless links.Finally, the influence of mobility patterns on the performanceof video streaming in ICN networks is shown. In addition,we improve the streaming quality in lossy environments byincluding mechanisms such as Wireless Loss Detection andRecovery (WLDR) [4] and discuss the results.

The remainder of this paper is structured as follows. Sec-tion 2 discusses related testbeds. In Section 3 an overview ofour testbed’s architecture is presented. The approach usedfor emulating wireless links is described in Section 4. InSection 5 we use our testbed to evaluate the benefits ofWLDR when considering different movement patterns ofmultimedia streaming clients in wireless networks. Finally,we conclude with the findings of the paper in Section 6.

2 RELATEDWORKThe evaluation of new networking approaches is generallydone by employing one or multiple of the following methods:Simulations using event-based simulators are rather gen-

eral tools and allow for testing new ideas without side effectsresulting from physical phenomena or hardware restrictions.

The ns3/ndnSIM network simulator [6] is an example of acommonly used simulation tool for NDN-related research.

Emulation using virtualization is a more practical method,where multiple sandboxed runtime environments, either re-alized as virtualized operating systems or Linux contain-ers, are interconnected by an emulated network topology.Depending on the computational requirements, these envi-ronments can be started either on a single machine or on aserver cluster, and can be used to test sample applications onarbitrary network topologies. The vICN toolchain [11] forresearch on Community ICN (CICN) as well as Mini-NDN2

for NDN-based research are representatives of virtualizationapproaches. Although these approaches are closer to realisticscenarios than simulation, they suffer from the absence of areal network and real parallelization.Emulation using a dedicated testbed is the most realistic

evaluation method. This method utilizes real hardware aswell as real network connections. The most prominent rep-resentative of an ICN testbed is the NDN testbed3, whereresearch institutions from all around the world participate inorder to allow for experiments over the public Internet.Whenconducting experiments on this testbed, the ever-changingenvironment, resulting from changing traffic conditions onthe underlying IP network, has to be kept in mind and com-plicates achieving reproducible results. A more controlledenvironment is ensured by the low-cost NDN testbed [9],which is a local testbed for network emulation purposes. Itutilizes low-performance devices as network nodes, whichcan be interconnected by an easily configurable overlay net-work topology. Resulting from the use of real hardware, thistestbed allows, besides the analysis of network behavior, in-sights into the computational complexity of evaluated algo-rithms and applications by CPU load and power consumptionobservations.For the evaluation of wireless links, various testbeds uti-

lizing real wireless channels are available. The NITOS facil-ity4, the Orbit testbed5, and the PhantomNet infrastructure6are prominent examples of testbeds which allow to connecttestbed nodes via real wireless links and thereby allow theoptimization of the wireless channels proper, but also ofapplications using those channels. Although real wirelesslink properties can be studied, the access to these testbeds islimited in terms of time restrictions and limited control oftestbed nodes.

Analyzing and optimizing applications’ performance basedon the characteristics of a wireless network does not in-evitably require the use of real wireless channels and can2https://github.com/named-data/mini-ndn, last accessed: 2018-05-233https://named-data.net/ndn-testbed/, last accessed: 2018-05-234https://nitlab.inf.uth.gr/NITlab/nitos, last accessed: 2018-08-075http://www.orbit-lab.org/, last accessed: 2018-08-076https://www.phantomnet.org/, last accessed: 2018-08-07


47

be replaced by more flexible emulated wireless channels inmany cases. Although virtualization approaches provide so-lutions for wireless link emulation, currently no possibilitiesto emulate mobility are supported. This is why we aim atrealizing wireless network emulation, including the effectsof mobility, on real hardware. Besides providing a highlycustomizable and flexible testbed, we focus on a low-costsolution, allowing other researchers to easily set up theirown testbed.

3 TESTBED ARCHITECTUREThe testbed’s architecture, as visualized in Figure 1, can bestructured into two main components, namely hardwarecomponents and the execution engine. The hardware of thetestbed actually runs the experiments, while the executionengine allows for the easy configuration of experiments. Inthe following, details on the main components and theirconstituting parts are presented.

Wireless NetworkEmulation Testbed

Hardware

Gateway

Switches

... ...

...

Banana Pi Routers

Execution Engine

Synchronization

Topology Configuration

Command Scheduling

Artifact Management

Emulation Description Interface

Figure 1: Conceptual design and main components ofthe wireless network emulation testbed.

3.1 Testbed HardwareThe testbed’s hardware is based on the low-cost NDN testbedof [9]; that paper presents the structure and the configura-tion options of the testbed in detail. The purpose of thetestbed is to provide a low-budget option to validate the-oretical concepts and evaluate the performance of ICN onphysical hardware. Besides upgraded software components(Section 3.2), the low-cost testbed is used as described in [9].This section provides an overview of the testbed’s hardwareand explains the interplay of its components.The testbed is based on an IP network interconnecting

20 network nodes, realized as Banana Pi Routers (BPI-R1).These low-performance devices are used to run the actual

experiments and therefore operate the NDN network stackand all applications required for a specific experiment.Each node is connected to two physically separated net-

works. First, the management network, which is mainly usedfor logging, artifact management and command scheduling.Second, the emulation network, which is used for the em-ulation traffic itself. This separation is needed in order toensure that management activities do not influence the em-ulation activity. The emulation network, physically set upas a star topology, is converted into an arbitrary emulationnetwork topology by using Linux tools, such as traffic con-trol (tc), iptables and Network Emulator (NetEm). These toolsallow to modify the emulation topology without changingthe physical links between network nodes.The gateway server is used for managing emulations. It

provides all required artifacts, but also manages the execu-tion of emulations and collects log files produced duringthe emulations. Furthermore, it provides a Web interface forobserving the current state of the testbed nodes and networklinks.

3.2 Execution EngineBesides the hardware components, the newly developed Exe-cution Engine (EE) is the second important component of thetestbed. It is used for the automated execution of networkemulations, including all tasks for preparing the testbed andits nodes for the emulation process. In the following, a de-scription of all components of the EE, including the interplayof the components is presented.

Emulation Description Interface. The Emulation De-scription Interface (EDI) provides an easy way to de-scribe network emulations using the programminglanguage Python. A network emulation is conductedby executing low-level system commands at prede-fined points in time on different nodes in the network.This includes configuring a network topology, but alsostarting applications and managing files.The concept of the EDI’s function blocks combines suchlower-level tasks in self-contained containers, similarto the class concept in object-oriented programming.Thereby it becomes easily possible to define high-leveltasks, such as setting up an emulated WiFi connection,with only a few lines of code.

Synchronization. In order to time emulation events,such as the start of an emulation run, but also to beable to calculate the delay between sending a packeton the sender side and receiving it on the receiverside, the system clocks of all devices have to be syn-chronized. Therefore, the Network Time Protocol (NTP)implementation chrony7 is used. Investigations on the

7https://chrony.tuxfamily.org/, last accessed 2018-05-23


48

testbed showed that chrony is capable of keeping themaximum clock offset between two devices on theorder of tens of microseconds.

Topology Configuration. The topology configurationcomponent of the EE allows the easy deployment ofarbitrary network topologies on top of the emulationnetwork. In combination with function blocks of theEDI, it becomes possible to configure, e.g., randomnetworks or predefined topologies, such as the AbileneCore topology [1]. In the background, this componentuses the Linux tools tc and iptables to configure thedesired network behavior.

Command Scheduling. The command schedulingmod-ule allows to execute commands at predefined timesbefore and during emulation runs. Basically, commandscheduling can be used to schedule two types of com-mands. First, initialization commands are executed toprepare the testbed for an emulation run. This prepa-ration includes commands needed for the topologyconfiguration as well as the distribution of configu-ration files required by various applications. Second,emulation commands are used to schedule commandsduring an emulation. These could effect starting orstopping an application, but also changing parametersof a link, e.g., initiating scheduled link failures.Technically, the command scheduler works as follows.The EDI structures commands for each node into ini-tialization and emulation commands and passes themto the corresponding nodes. The nodes then executethe initialization commands and thereby prepare thetestbed for the next emulation run. After completionof the initialization phase, an emulation start time isset by the EE, which is then used as the reference timefor all scheduled emulation commands.

Artifact Management. Applications, configuration filesand log files can be seen as different artifacts whichneed to be managed during an emulation run. Thesevarious artifacts can be managed by using functionblocks of the EDI. The idea of artifact management isthat all required applications and configuration filesare distributed to the testbed nodes in the initializationphase and automatically cleaned up after gatheringlog files, when the emulation run is completed.

4 WIRLESS LINK EMULATIONThis section presents how the testbed is used to emulatewireless links. The goal of our work is to not only emulatestatic properties, such as a constant loss rate or link delay, butalso to emulate changing network conditions, resulting, e.g.,from moving network nodes. In reality, the characteristics

of wireless links change continuously over time. For emula-tion purposes, this continuity is approximated by periodicadjustments of link properties.

The continuous change of a wireless link can be traced intwo different ways. The first way is to model the properties ofthe wireless link by using well-established fading and shad-owing models, such as Nakagami-lognormal channels [13],and by combining these models with mobility using models,such as presented in [3]. The second possibility is to usereal-world data obtained in controlled experiments, such asprovided by [7]. Independent of the researcher’s preference,the flexibility of our system allows to use both possibilities.In the following, we model link characteristics by using thens-38 implementation of Nakagami-lognormal channels inorder to create network traces, which are then used as inputfor our emulation. Nevertheless, real-world traces can formthe basis of an emulation as well.The testbed uses the Linux tools tc and NetEm9 to con-

trol the properties of emulated wireless links. In order toapproximate the real continuous link changes as accurate aspossible, the intervals between periodic adjustments needto be as small as possible, but larger as the time neededfor configuring the adjustment. When making the intervalstoo small, we run the risk of overwhelming tc and NetEm,which might result in unwanted behavior. To prevent this,the capabilities of those Linux tools are analyzed; a represen-tative part of the results from this analysis is presented inthe following.Figure 2 visualizes the evaluation results of the switch-

ing lag when switching the link delay of a single link be-tween two testbed nodes. Switching lag is defined as the timeelapsing between initiating a change of a parameter and thechange becoming effective. In the case of modifying a link’sdelay, we define “becoming effective” such that the delays offive subsequent packets do not deviate more than 5% fromthe targeted delay. For evaluation, we used the delay valuesof the DASH Industry Forum (DASH-IF) test vector NP2j [2].In the upper chart of Figure 2, the blue line (mostly cov-ered) indicates the test vector; the green line visualizes themeasured link delay between the testbed nodes. The resultsindicate that NetEm is capable of adjusting the link delayvalues accurately with only small anomalies. The lower chartof Figure 2 visualizes the average switching lag for adaptingthe link delay as defined by the test vector, including 95%confidence intervals (CI) resulting from 30 emulation runs.It can be seen that the lag for applying the changes lies con-sistently below 50 ms and fluctuates only little. In addition,evaluations for bandwidth shaping and packet loss variation

8https://www.nsnam.org/, last accessed 2018-05-259The NetEm tools are used for controlling link delay and packet loss char-acteristics. For bandwidth shaping, the tc token bucket filter is used.


49

1 2 3 4 5 6 7 8Emulation-time [s]

0

5

10

15

20

25

Link

dela

y [m

s]

Target and actual link delayChange initiatedChange effectiveTest vectorActual delay

1 2 3 4 5 6 7 8Emulation-time [s]

42

44

46

48

Switc

hing

lag

[ms]

Time until delay change becomes effective

Figure 2: Evaluation of the switching lag for link delayvariation using a DASH-IF test vector.

were conducted. The results of these evaluations are verysimilar to delay variation; results are published along withthe source code of the testbed.Having established that the switching lag for adjusting

link conditions is around 50 ms, we are able to search for theoptimal switching interval for emulating wireless links. Asswitching interval, we define the time between consecutiveadjustments of link properties. The basis for this evaluationare network traces generated by the network simulator ns-3.The simulation scenario includes two nodes connected viaa 802.11g WiFi link, one walking straight away from theother and back, while consuming the available bandwidthby sending UDP packets in both directions. Based on thesimulation traces, values for packet loss, bandwidth and linkdelay are extracted and used as input for the wireless linkemulation. Changes of a link’s characteristics are appliedonce per switching interval. The single target value for a in-terval is calculated by averaging all measured delay, loss andbandwidth values in the corresponding switching interval.

Figure 3 visualizes the influence of the switching intervalon delay and packet loss deviation. The deviation is calcu-lated between the WiFi network trace from simulation andthe traces generated by sending traffic over the emulatedWiFi testbed link. The blue line visualizes the absolute de-lay difference in milliseconds, the green line the packet lossdifference in percent. In general, we can see that shorterswitching intervals lead to smaller deviations, as long as theswitching interval is well above the switching lag. With aswitching interval of 100 ms, a delay difference of less than

50 100 250 500 1000Interval-size [ms]

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Dela

y di

ffere

nce

[ms]

DelayPacket loss

4.0

4.5

5.0

5.5

6.0

6.5

7.0

7.5

8.0

Pack

et lo

ss d

iffer

ence

[%]

Influence of switching interval length on link emulation quality

Figure 3: Visualization of emulation accuracy whenemulatingwireless linkswith varying switching inter-vals.

2 ms can be achieved. Regarding packet loss, a deviation ofless than 5% is measured when setting the switching inter-val to 100 ms. Based on these results, we assume that theoptimal switching interval for emulating wireless links is100 ms, which is why this interval is used for further evalua-tions. Configuring a switching interval of less than 100 msmeans that the adjusted property is effective for just a fewmilliseconds before it is adjusted again, which leads to fewerdelivered packets with correct parameters and a higher de-viation, as visible when focusing on the 50 ms switchingintervals. Depicted error bars represent 95% CIs after 40 em-ulation runs.Besides the calculated deviations visualized in Figure 3,

we can examine the emulation precision visually in Figure 4.The emulation of the wireless link includes adjustments ofthe link delay and the packet loss rate. When packet lossis high, fewer packets arrive; when the delay is high, thetransmitted packets arrive later at the receiver. Combiningthese characteristics already leads to a reduced bandwidth,without the need of shaping it separately. Figure 4 visualizestarget and actual delay (upper chart) and packet loss (lowerchart) values, measured during a single emulation run basedon the previously explained simulated network trace. Theemulated distance between the two nodes increases untilsecond 50, where hardly any packets are transmitted suc-cessfully. From second 50 onwards, the distance between thenodes is reduced until second 100, which leads to reducedpacket loss and delay values.


50

0 20 40 60 80 100Emulation-time [s]

0

10

20

30

40

Dela

y [m

s]

TargetActual

0 20 40 60 80 100Emulation-time [s]

0

20

40

60

80

100

Loss

[%]

TargetActual

Comparison of target and actual link conditions

Figure 4: Progression of link delay and packet loss onan emulatedWiFi link including a trivialmobility pat-tern.

The underlying mobility of the simulated network traceused for the presented evaluations may seem artificial, butbesides the good suitability for demonstration purposes, thecharacteristics of this trace regarding the progression ofpacket loss and delay values follow the well-establishedmodel for Nakagami-lognormal channels [13]. Therefore,the results of this evaluation, showing that the testbed iscapable of emulating wireless links accurately, become validfor other scenarios as well. Changing the emulation scenariois trivial, the only required step is to create a network traceby simulation or by conducting real-world measurements.Besides the emulation of channel-specific properties, theflexibility of this trace-driven approach also allows to em-ulate different types of handovers as well as interferences,provided that a given network trace includes such events.

5 ADAPTIVE VIDEO STREAMING OVERICN IN WIRELESS NETWORKS

After having explained the structure of our testbed and theprocess of wireless link emulation in detail, in this sectionwe demonstrate the testbed in action. We investigate the in-fluence of mobility when streaming MPEG-DASH-compliantmultimedia content [12] encoded with the scalable videocodec (SVC). SVC allows to encode a video into multiple lay-ers, including a base layer and multiple enhancement layers.The base layer provides a low-quality representation of thevideo. Enhancement layers can be stacked upon the base

layer in order to improve the quality in terms of higher tem-poral resolution, spatial resolution, or visual quality (SNR).

Furthermore, we investigate the improvements achievableby the Wireless Loss Detection and Recovery (WLDR) tech-nique [4] in this streaming scenario. WLDR is a techniqueused for the early detection of packet loss on wireless links.When requesting video segments in NDN, a video segmentis split up into multiple Data packets, having the same nameprefix and increasing sequence numbers. If e.g. Interest pack-ets for sequence numbers 0–10 and an Interest packet forsequence number 12 are received on a WiFi access point, itis very likely that the Interest for sequence number 11 waslost. When using WLDR, the access point recognizes the gapin sequence numbers and forces the consumer to re-issuean Interest packet for the lost sequence number. The sameprinciple is used for lost Data packets: when a gap in thesequence numbers of incoming Data packets is recognized,a retransmission of the lost Data packet is initiated. Thebenefits of WLDR for dynamic adaptive streaming withoutscalable codecs and static client positions was already identi-fied by Samain et al. [10]. We now use the wireless networkemulation testbed to analyze whether these benefits alsohold true for scalable video codecs and analyze how differentmobility models influence the performance of WLDR.In Section 5.1, we show how the testbed, especially its

execution engine, can be used to set up an evaluation scenariousing predefined function blocks. Details on the evaluationscenario and results of our evaluation are given in Section 5.2.

5.1 Using the Testbed’s Function BlocksIn this section, we describe how the testbed can be used todefine an emulation scenario and to conduct evaluations. Weuse the video streaming scenario described in the previoussection to showcase the testbed’s capabilities.The Emulation Description Interface (EDI) allows to use

function blocks which combine logically connected low-leveltasks to form a larger task. One example of a function blockis the configuration of an emulatedWiFi connection betweentwo nodes. An emulated WiFi connection consists of twophysical nodes, including an access point (AP), and a mobileclient. The function block applies changes in the wirelesslink’s performance caused by client mobility, by periodic ad-justments of connection properties. Pre-recorded connectionproperties resulting from different mobility scenarios can beused to configure the wireless connection.

Besides theWiFi block, other function blocks are available,which together allow to define a complete emulation scenario.Figure 5 depicts the interplay of individual function blocksto define the structure of our video streaming scenario. Inthe following, this interplay is explained in greater detail.


51

Abilene Core Topology

NDN Stack

Node

WiFi

Node

DAS Client

Application deployment

Topology configuration

Configuration of arbitrary network stacks

WiFi

Different configurations for individual blocks supported

DAS Client

DAS Server

NodeProvides access to physical nodes

Figure 5: Structure of an emulation topology for videostreaming, created by combining function blocks.

The network topology is set up by combining two differenttypes of function blocks. The Abilene Core Topology block isused to form the Abilene Core network [1] between elevenphysical devices. The WiFi block connects two nodes via anemulated WiFi link. One of those nodes acts as an AP andgateway to the Abilene network. The second node representsa mobile client connected to the network via WiFi.

The NDN Stack function block can be used to manage theNDN stack on the devices of the emulation topology. Besidesstarting the required NDN services, forwarding informationis disseminated to all nodes. NDN is only one example of anICN or even a general network architecture that is deployableon the testbed.

The Node function block allows to access a single physicaldevice. The block can be used to copy files or to schedulecommands to be executed before the emulation starts, or atpredefined points in time during the emulation.Management of application life-cycle can also be per-

formed by using specialized function blocks. Besides startingthe streaming application, the DAS Client and DAS Serverblocks configure the applications and collect log files afterthe completion of the emulation. Thereby, consumer logscontaining information about Quality of Experience (QoE)are collected on a single place on the gateway server.

5.2 Influence of Consumer Mobility onVideo Streaming Quality

We now use the wireless network emulation capabilities ofour testbed to evaluate the influence of different mobilitypatterns on video streaming performance as well as the ben-efits of early loss detection provided by WLDR. Our findingsare based on the streaming quality reported by the clientapplications, as experienced in various settings involvingdifferent mobility patterns and deployments of WLDR.

5.2.1 Emulation Scenario. The underlying network topol-ogy used in the evaluation consists of the Abilene Core topol-ogy [1] which is extended by wireless access networks host-ing the video consumer applications. The wireless accessnetworks are realized as emulated 802.11g WiFi networks

containing a mobile consumer (client) and a static accesspoint which is a member of the Abilene network. The posi-tion of the access networks on the Abilene network as wellas the position of the video producer (server) is selected ran-domly. Wireless links are modeled as Nakagami-lognormalchannels [13]. A link’s performance is obtained by simulationusing ns-310 and depends on consumer mobility, followingthree well-established mobility models: Constant Position,Random Direction and Random Walk [3]. In case of ConstantPosition, the consumer is positioned directly below the ac-cess point and does not move during the emulation. In caseof Random Direction and RandomWalk, the consumer walksaround the access point in a pattern similar to those depictedin Figure 6 at a constant walking speed of 1.4 meters per sec-ond. Although these mobility models do not model humanbehavior accurately and have been criticized [3], they canbe used to demonstrate the testbed’s capability to emulatevarious mobile scenarios.

−40 −20 0 20 40−40

−30

−20

−10

0

10

20

30

40 Random Direction

−40 −20 0 20 40−40

−30

−20

−10

0

10

20

30

40 Random Walk

Figure 6: Consumer mobility patterns based on therandom direction and random walk mobility models.

The video consumer and producer applications use theNDN stack for the video streaming process. NDN routinginformation in the core network is based on the shortestpath; Interests are forwarded using the Best Route forwardingstrategy [6].During an emulation run, two clients stream the same

video published by a single producer. The video itself is a 21minute long concatenation of two animation videos, encodedin three representations with bitrates ranging from 640 kbpsfor the base layer, 1 Mbps for medium quality and 1.4 Mbpsfor the best quality, taken from the scalable video codingdataset of [5]. The consumer and producer applications arebased on the work done by Rainer et al. [8] and use thetherein mentioned buffer-based adaptation logic with anInterest lifetime of 500 milliseconds. For transmission, videosegments are divided into several Data packets, each carrying2048 bytes.10https://www.nsnam.org/, last accessed 2018-05-25


52

WLDR [4] is implemented by an additional service observ-ing all incoming packets logged by the NDN ForwardingDaemon (NFD). Once detected, packet loss is reported tothe sender via a notification using UDP. In emulations usingWLDR, the WLDR service is only running on the pairs ofnodes in the WiFi access networks.

5.2.2 Streaming Performance on WiFi Links. In order to ratethe streaming performance, we focus on QoE characteristicswhich are perceivable for users watching the multimediastreams. Besides the average played out representation, thenumber and length of stalling events as well as the number ofquality variations are of particular interest. A stalling eventoccurs if the video player does not receive the base layer ofa video segment in time and the playout of the video getssuspended. We define a quality variation as change of theconsumed video quality between two consecutive segments.When focusing on the influence of mobility, we see that

the consumer’s mobility has a strong impact on the stream-ing performance, as visualized in Figure 7. The left part of thefigure displays the average segment’s representation after 15emulation runs. The solid orange lines in the boxplots repre-sent the median values, the dashed green lines illustrate theaverage over all emulation runs. The upper and lower bor-ders of a box depict 25% and 75% quartiles, whiskers show thevariability outside the quartiles, individual points illustrateoutliers. The available representations range from Represen-tation 0, which requires the SVC base layer only, up to thehighest available bitrate with Representation 2, requiring thebase layer and both enhancement layers. In the case of Con-stant Position scenarios, which do not involve packet losson the WiFi channel, the highest average played out repre-sentation is achieved. The average consumed representationis about 1.5, which means that most segments were playedout in one of the two higher representations. With movingconsumers, the average representation drops significantlyto 0.6 in case of Random Direction mobility, and 0.8 in caseof Random Walk mobility.

ConstantPosition

RandomDirection

RandomWalk

0.6

0.8

1.0

1.2

1.4

1.6

Avg.

SVC

laye

r

Played out Representation (without WLDR)

ConstantPosition

RandomDirection

RandomWalk

0

10

20

30

40

Avg.

stal

ling

time

[s]

Per Client Stalling Time (without WLDR)

Figure 7: Average played out video representation andstalling time under different mobility models.

The right part of Figure 7 displays the average stalling timeof consumers, where again the best results were achieved

in the Constant Position scenario. In this case, virtually nostalls were observed at all. In case of Random Walk mobility,where it could happen that clients stay at the edge of theAP’s coverage area for a longer period of time, the stallingtime during playing the video sums up to a total of morethan 30 seconds per client in some cases. Consumers movingaccording to the Random Direction model perform signifi-cantly better. In this case, a maximum of 16 seconds stallingtime was observed. Nevertheless, none of the clients wasable to retrieve the video without playout interruption.Figure 8 shows the progression of the consumed repre-

sentations and the number of encountered stalling eventsper segment for different mobility scenarios. The greyscaleintensity indicates the average streamed representation ofa segment, while the number of clients stalling at a certainsegment is illustrated by the height of the red bars in thelower part of the charts.

0 100 200 300 400 500 600Segment

L0

L1

L2

StallsCons

tant

Pos

ition

0 100 200 300 400 500 600Segment

L0

L1

L2

StallsRand

om D

irect

ion

0 100 200 300 400 500 600Segment

L0

L1

L2

Stalls

Rand

om W

alk

Played out SVC layers without WLDR

Figure 8: Progression of retrieved SVC layers and num-ber of stalls per segment.

In the Constant Position scenario, all clients are able toretrieve Layer 0 and Layer 1 after a ramp-up phase at thestart of the stream, which is caused by the clients’ adaptationlogic11. The adaptation logic tries to fetch Layer 2, if theplayout buffers for the lower layers are sufficiently filled.This results in oscillations of Layer 2, which indicates a toolow bandwidth for fetching the best provided representation.Regarding the scenarios involving movement, we can seesmall bursts of stalls in case of Random Direction mobility.This differs from Random Walk mobility, where longer stallscan be noticed, which is because clients stay longer on theedge of the AP’s coverage area, where packet loss can exceed40%. At RandomDirection, only short time intervals are spentat the edge of the AP’s coverage area.11The adaptation logic is a client-side component deciding which layers toretrieve when streaming multimedia content.


53

5.2.3 Streaming Performance Benefits of using WLDR. Toquantify the benefits of WLDR, we enhance the wirelessaccess networks by adding the WLDR service and repeatthe evaluations from the previous section. Comparing theresults without WLDR in Figure 7 to the results includingWLDR in Figure 9, no significant improvement in the av-erage consumed representation is measured, although thenumber of stalls decreased significantly. The difference in theaverage consumed representation and stalling time betweenthe different mobility models persists. Examining the worstcase setting regarding stalls, Random Walk, we quickly seethat the use of WLDR reduced the maximum stalling timefrom almost 50 seconds to only 31 seconds. Besides the im-provements of the maximum stalling time in both scenariosconsidering mobility, the average stalling times were signifi-cantly reduced as well. In the case of Random Direction, bothclients are in some emulation runs even capable of streamingthe video without stalls when using WLDR.

Keeping the characteristics of the mobility models in mind,we find the use of WLDR particularly beneficial in situationsof prolonged higher packet loss such as in the Random Walkscenario. Short-term spikes in packet loss as prevalent in theRandom Direction scenario can be compensated using theclient’s playout buffer, diminishing the impact of WLDR.

ConstantPosition

RandomDirection

RandomWalk

0.6

0.8

1.0

1.2

1.4

1.6

Avg.

SVC

laye

r

Played out Representation with WLDR

ConstantPosition

RandomDirection

RandomWalk

0

5

10

15

20

25

30

Avg.

stal

ling

time

[s]

Per Client Stalling Time with WLDR

Figure 9: Average played out video quality and stallingtime under differentmobilitymodels and withWLDRenabled.

Comparing the results of this evaluation to the findings ofa previous evaluation [10], where the influence of WLDR onthe streaming quality for non-scalable codecs was investi-gated, we noticed slightly different results. Although WLDRimproved the quality of experience in both evaluations, [10]observed improvements in the average streaming qualityin terms of bitrate, while we observed an improvement inrespect to the stalling behavior. One possible reason for thisdifference is the layered quality improvement concept uti-lized by scalable codecs. While low quality representationswith non-scalable codecs require a low bitrate, the base layerin SVC already requires a relatively high bitrate compared tothe enhancement layers. In lossy environments, this couldresult in situations where not even the base layer can be

successfully retrieved, which makes it impossible to play outa higher quality. In these situations, WLDR helps to retrievethe base layer and thereby reduces stalling events.

6 CONCLUSIONThe main part of this paper focuses on the presentationof a testbed for wireless network emulation. We extendedan existing low-cost testbed for fixed networks with thefunctionality to change link properties dynamically. Thisallows to emulate wireless networks, as well as link failuresand recoveries or other events concerning links. In addition,we developed the testbed’s Execution Engine (EE), whichallows to perform network emulations as conveniently aswith event-based simulators.

We demonstrated the capabilities of the testbed by inves-tigating the influence of different mobility models on MPEG-DASH SVC video streaming performance and the benefitsof explicit loss detection and notification (WLDR) in lossywireless networks. Our results show the importance of con-sidering client mobility in investigations involving wirelessnetworks. Furthermore, the results underline the benefits ofearly loss detection in NDN, realized by WLDR.

For future research, we plan to use the testbed for evalua-tions involving real-world network traces featuring variousmobile scenarios, such as vehicular mobility as publishedin [7], complementary to evaluations with synthetic mobilitymodels. Furthermore, we plan to use the testbed’s executionengine without ICN deployment for investigations on fogcomputing. From a more technical point of view, the supportof containerized applications, abstracting the underlying soft-ware stack to amore general environment is planned. Besidesproviding a clean-slate environment, containerization wouldease the development and deployment of apps applicable ona broad range of different testbed environments.The ease of use of the testbed’s EE for setting up emula-

tions makes us believe that other researchers could benefitfrom using such a tool as well. Generic concepts, not focusingon specific hardware platforms, allow the use of the EE forarbitrary testbed platforms. To allow the community to adoptthe EE for their purposes, we published the open-source li-censed source code and provide installation instructions12.Along with the source code, we additionally publish the func-tion blocks and the emulation scenarios, which allows forthe reproduction of the presented results.

ACKNOWLEDGMENTSThis work was supported in part by the Austrian ScienceFund (FWF) under the CHIST-ERAproject CONCERT (projectno. I1402).

12http://icn.itec.aau.at/ndn-network/


54

REFERENCES[1] 2005. Abilene Network. Technical Report. 2 pages. https:

//web.archive.org/web/20120324103518/http://www.internet2.edu/pubs/200502-IS-AN.pdf

[2] 2017. Guidelines for Implementation: DASH - AVC/264 Testcases and Vectors. Technical Report Version 0.9. DASH In-dustry Forum. https://dashif.org/wp-content/uploads/2015/04/DASH-AVC-264-Test-Vectors-v09-CommunityReview.pdf

[3] T. Camp, J. Boleng, and V. Davies. 2002. A survey of mobility modelsfor ad hoc network research. Wireless Communications and MobileComputing 2, 5 (Aug. 2002), 483–502.

[4] G. Carofiglio, L. Muscariello, M. Papalini, N. Rozhnova, and X. Zeng.2016. Leveraging ICN In-network Control for Loss Detection andRecovery in Wireless Mobile networks. In Proceedings of the 3rd ACMConference on Information-Centric Networking - ACM-ICN ’16. ACM,50–59.

[5] C. Kreuzberger, D. Posch, and H. Hellwagner. 2015. A scalable videocoding dataset and toolchain for dynamic adaptive streaming overHTTP. In Proceedings of the 6th ACM Multimedia Systems Conference.ACM, 213–218.

[6] S. Mastorakis, A. Afanasyev, I. Moiseenko, and L. Zhang. 2016. ndnSIM2: An updated NDN simulator for NS-3. Technical Report NDN-0028,Revision 2. NDN.

[7] C. Mueller, S. Lederer, and C. Timmerer. 2012. An Evaluation of Dy-namic Adaptive Streaming over HTTP in Vehicular Environments. In

Proceedings of the Fourth Annual ACM SIGMM Workshop on MobileVideo (MoVid12). ACM, 37–42.

[8] B. Rainer, D. Posch, and H. Hellwagner. 2016. Investigating the Perfor-mance of Pull-based Dynamic Adaptive Streaming in NDN. Journalon Selected Areas in Communications (Aug. 2016), 11.

[9] B. Rainer, D. Posch, A. Leibetseder, S. Theuermann, and H. Hellwag-ner. 2016. A Low-Cost NDN Testbed on Banana Pi Routers. IEEECommunications Magazine 54, 9 (Sept. 2016), 105 – 111.

[10] J. Samain, G. Carofiglio, L. Muscariello, M. Papalini, M. Sardara, M.Tortelli, and D. Rossi. 2017. Dynamic Adaptive Video Streaming: To-wards a Systematic Comparison of ICN and TCP/IP. IEEE Transactionson Multimedia 19, 10 (Oct. 2017), 2166–2181.

[11] M. Sardara, L. Muscariello, J. Augé, M. Enguehard, A. Compagno, andG. Carofiglio. 2017. Virtualized ICN (vICN): towards a unified networkvirtualization framework for ICN experimentation. In Proceedings ofthe 4th ACM Conference on Information-Centric Networking - ICN ’17.ACM, 109–115.

[12] I. Sodagar. 2011. The MPEG-DASH Standard for Multimedia StreamingOver the Internet. IEEE Multimedia 18, 4 (Apr. 2011), 62–67.

[13] T.T. Tjhung and C.C. Chai. 1999. Fade statistics in Nakagami-lognormalchannels. IEEE Transactions on Communications 47, 12 (Dec. 1999),1769–1772.

[14] L. Zhang, A. Afanasyev, J. Burke, V. Jacobson, kc Claffy, P. Crowley, C.Papadopoulos, L. Wang, and B. Zhang. 2014. Named Data Networking.ACM SIGCOMM Comput. Commun. Rev. 44, 3 (July 2014), 66–73.


55

https://web.archive.org/web/20120324103518/http://www.internet2.edu/pubs/200502-IS-AN.pdf



https://dashif.org/wp-content/uploads/2015/04/DASH-AVC-264-Test-Vectors-v09-CommunityReview.pdf

https://dashif.org/wp-content/uploads/2015/04/DASH-AVC-264-Test-Vectors-v09-CommunityReview.pdf

wireless network emulation for research on information ... · wireless network emulation for...

Documents