measurement-based analysis of the performance of several...

MeasurementMeasurement--Based Analysis of theBased Analysis of thePerformance of several Wireless Performance of several Wireless

TechnologiesTechnologies

R. Cosmaa, A. Cabellos-Apariciob, J. Domenech-Benllochc, J. Gimenez-Guzmanc, J. Martinez-Bausetc, M. Cristiana, A. Fuentetajad, A. Lópezb, J. Domingo-Pascualb, J. Quemadad

a. Technical University of Cluj-Napoca, Romaniab. Universitat Politècnica de Catalunya, Barcelona, Spainc. Universidad Politécnica de Valencia, Spaind. Universidad Politécnica de Madrid, Spain

LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 1

*Research carried out within CEPS (Spanish-funded research project)

AgendaAgenda

I. IntroductionII. Layer 3 ParametersIII. Measurement MethodologyIV. Experimental ResultsV. Conclusions and Future Work


I. IntroductionI. IntroductionEvents with major implications for the Internet:◦ The advent of broadband connections (DSL,

Cable Modems, Wireless Access)◦ The rapid evolution of Wireless TechnologiesThe current paradigm shift:◦ High Bandwidth◦ Low Delay◦ Extended Area of Service◦ Mobility


Present SolutionsPresent SolutionsIEEE 802.11 (WiFi)◦ Reduced costs◦ Maximum theoretical debits of 54 Mbps (.g)◦ Short-range coverage / low mobilityUMTS / HSDPA◦ 3G Technology: Mobility issues have been solved◦ Standards that enhance the Data RatesIEEE 802.16 (WiMAX)◦ New Infrastructure◦ Mobility support in 802.16e◦ Economical benefits


II. Layer 3 ParametersII. Layer 3 ParametersStandard metrics for the performance of Internet data delivery services:◦ One-way Delay (OWD)◦ Interpacket Delay Variation (IPDV)◦ Packet Loss Ratio (PLR)Relevance:◦ Performance of Transport Layer protocols◦ Performance of Interactive/Real-time applications◦ Insights on the dynamics and effectiveness of Layer 2

protocols/implementations.


III. Measurement MethodologyIII. Measurement Methodology1. Deployment and configuration of the test-bed

scenario2. Clock Synchronization3. Generating the Probe Traffic4. Capture of Test Traffic at the relevant nodes5. Extraction of Layer 3 Metrics from the capture

files


Testbed ScenarioTestbed ScenarioUMTS

Provider

RedIris

Internet

NTP + Control

GPS AntennaStratum 1NTP Server

Capture Point

WiMAX BS

WiFi BS

Flowserver

802.3

802.16

802.11Fast Ethernet

UMTS192.168.0.137

84.88.40.26

84.88.40.28

10.1.24.2

138.4.24.109

147.83.130.52


Clock Synchronization Clock Synchronization -- NTPNTPThe Network Time Protocol◦ Hierarchy-based Architecture (Stratum Levels)◦ Time source differentiation algorithms◦ Information combining (agreement) algorithmsNTP Dedicated Network◦ Separate Network Interfaces◦ Stratum 1 Servers◦ GPS-based Time Source◦ High accuracy


Test traffic Test traffic -- IsabelIsabelWhy did we use a videoconferencing application?◦ Simultaneous traffic generation for all the clients◦ UDP advantages◦ Different nature of Audio and Video flowsIsabel:◦ High modularity◦ Appropriate logical topology


Test traffic Test traffic -- IsabelIsabel

Each client sends its data to the flowserver.Flow multiplexing takes place at the flowserver.

Access Network

Interactive Terminals

Flowserver

Physical Topology Logical Topology


Traffic Capture Traffic Capture -- WiresharkWiresharkRequirements:◦ Timestamping◦ Capture file format

Wireshark:◦ Platform-independent operation◦ Advanced GUI◦ PCAP file format


Layer 3 Metrics ExtractionLayer 3 Metrics ExtractionImplementation:◦ Perl programming language◦ Net::Pcap wrapperFunctionality:◦ Synchronization checking◦ RTP flows listing◦ Packet matching algorithm for metric

extraction


IV. Experimental ResultsIV. Experimental ResultsStructure:◦ Statistical descriptors of central tendency and

variability: min, max, mean, median, standard deviation

◦ Histogram and CDF plots for OWD/IPDV◦ Variation of metrics with respect to packet sizeObjectives:◦ Empirical evaluation of link performances◦ Influence of Layers 1 & 2◦ Link behavior when subject to CBR/VBR traffic


Test traffic overviewTest traffic overview

Link/SSRC Flow Packets Flow size (Mbytes) Throughput(kbps)

Ethernet (2800) Up

Audio 32261 9.04 78.15

Video 22805 16.38 142.59

UMTS (2799) Up

Audio - - -

Video 19117 13.29 115.62

WiFi (2798) Up

Audio 32864 9.21 78.39

Video 20470 14.37 122.37

WiMAX (2797) Up

Audio 22429 6.28 54.26

Video 21005 15.08 129.64


Results Results -- WiFiWiFiStatistical parameters

Metric (ms) Max Mean Median Min Dev PLR

Video Up

OWD 20.373 4.677 1.003 0.550 7.128 0.034%

IPDV 1.443 0.400 0.064 -3.416 2.143

Audio Up

OWD 3.320 1.081 0.587 0.400 1.777 0%

IPDV 0.982 0 0.015 -2.039 1.488

Video Down

OWD 2.849 1.278 0.895 0.516 2.072 0.879%

IPDV 0.686 0 0.059 -1.773 1.487

Audio Down

OWD 1.824 0.895 0.602 0.511 1.616 0.38%

IPDV 0.607 0 0.018 -1.571 1.406

Maximum OWD values due to ARQ mechanismWell-centered IPDV distributionsUplink / Downlink asymmetry due to channel access mechanism


Results Results –– WiFi UplinkWiFi Uplink

Metrics variation with packet size:◦ OWD: unaffected◦ IPDV: increased values for packet sizes > 400

bytes

200 400 600 800

2468

1012

Packet Length [bytes]

OW

D [m

sec]

200 400 600 800

2468

101214


IPD

V [m

sec]

2 4 6 8 100

0.2

0.4

OWD [msec]

Rel

ativ

e Fr

eque

ncy

-1 -0.5 0 0.5 10

0.02

0.04

0.06

0.08

0.1

IPDV [msec]

2 4 6 8 100

0.5

1

Cum

ulat

ive

Pro

babi

lity

-1 -0.5 0 0.5 10

0.2

0.4

0.6

0.8

1


Results Results -- WiMAXWiMAXStatistical parameters

Asymmetries: ◦ Uplink / Downlink – different power amplifier gains for

MS and BS◦ Audio (CBR) / Video (VBR)High Packet Loss Ratios:◦ Experimental MS equipment at Valencia

Metric (ms) Max Mean Median Min Dev PLR

Video Up

OWD 275.118 150.672 134.998 54.056 77.686 10.621%

IPDV 49.587 0 8.933 -80.051 35.005

Audio Up

OWD 244.949 107.593 81.300 42.960 67.957 3.237%

IPDV 39.185 400 µs 4.616 -62.378 27.819

Video Down

OWD 158.730 78.567 67.192 55.251 34.513 6.54%

IPDV 10.160 -0.300 0.570 -22.082 9.540

Audio Down

OWD 104.967 68.058 62.121 54.322 23.135 0.857%

IPDV 8.691 0 0.275 -16.799 8.514


Results Results –– WiMAX UplinkWiMAX Uplink

Optimal packet length: ~500 bytes.Discrete nature of IPDV distribution:◦ Frame size: 5ms◦ TDD◦ ARQ mechanism

Multiple retransmissions

200 400 600 800

100

150

200


OW

D [m

sec]

200 400 600 800

20

40

60

80

Packet Length [bytes]IP

DV

[mse

c]

100 150 200 2500

0.01

0.02

OWD [msec]

Rel

ativ

e Fr

eque

ncy

-80 -60 -40 -20 0 20 400

0.01

0.02

IPDV [msec]

100 150 200 2500

0.5

1

Cum

ulat

ive

Pro

babi

lity

-80 -60 -40 -20 0 20 400

0.5

1


Results Results –– WiMAX DownlinkWiMAX Downlink

IPDV unstable with small packet sizesDiscrete nature of IPDVLess frame retransmissions than Uplink

200 400 600 80060

70

80

90


OW

D [m

sec]

200 400 600 800

2

4

6

8


DV

[mse

c]

60 80 100 120 1400

0.05

0.1

OWD [msec]

Rel

ativ

e Fr

eque

ncy

-20 -10 0 100

0.01

0.02

IPDV [msec]

60 80 100 120 1400

0.5

1

Cum

ulat

ive

Pro

babi

lity

-20 -10 0 100

0.5

1

Results Results –– UMTS/HSDPAUMTS/HSDPA

Special setup imposed by the maximum uplinkbandwidthHigh maximum values due to congestion in the provider’s networkIPDV distributions better than WiMAXEfficient ARQ mechanisms

Statistical parametersMetric (ms) Max Mean Median Min Dev PLR

Video Up

OWD 258.251 153.877 95.385 75.903 103.481 0.250%

IPDV 27.792 -0.258 1.058 -66.573 30.228

Video Down

OWD 98.176 74.2441 67.500 50.047 43.524 0.242%

IPDV 11.751 0 1.825 -22.244 15.349


Results Results –– UMTS UplinkUMTS Uplink

Metric variation with packet size:◦ OWD stable◦ IPDV less stable,

increasing

Discrete IPDV nature:◦ Link temporization at

RLC layer◦ Loss recovery in integer

number of TTIs

100 150 200 2500

0.05

0.1

OWD [msec]

Rel

ativ

e Fr

eque

ncy

-60 -40 -20 0 200

0.02

0.04

IPDV [msec]

200 400 600 800

200

400

600

800

1000


OW

D [m

sec]

200 400 600 800

20406080

100120


DV

[mse

c]

100 150 200 2500

0.5

1

Cum

ulat

ive

Pro

babi

lity

-60 -40 -20 0 200

0.5

1


Results Results –– UMTS DownlinkUMTS Downlink

Lower OWD valuesIPDV and OWD stable with packet sizeHSDPA effects on IPDV discretization:◦ TTI limited to one radio

frame◦ Reduced radio frame

duration duration

200 400 600 8006080

100120140160180


OW

D [m

sec]

200 400 600 800

20

40

60

80

100


IPD

V [m

sec]

60 70 80 900

0.005

0.01

0.015

0.02

0.025

OWD [msec]

Rel

ativ

e Fr

eque

ncy

60 70 80 900

0.2

0.4

0.6

0.8

1

Cum

ulat

ive

Pro

babi

lity

-10 -5 0 5 100

0.02

0.04

IPDV [msec]-10 -5 0 5 10

0

0.5

1


V. ConclusionsV. ConclusionsSimple methodology for extracting Layer 3 Metrics for heterogeneous networksWiFi:◦ Good stability of metrics in respect to packet size◦ Extreme values caused by ARQ mechanism◦ Uplink / Downlink asymmetry due to the radio channel

disputing algorithmWiMAX:◦ High PLR – experimental equipment◦ Uplink / Downlink asymmetry◦ IPDV discretization due to ARQ mechanismsUMTS / HSDPA:◦ Efficient retransmission schemes◦ HSDPA effects on Downlink metricsBetter values for metrics associated with CBR (Audio) traffic


V. ConclusionsV. ConclusionsWiFi WiMAX UMTS

OWDIPDVPLRThroughputRange/Mobility

Best

Medium

Worst


V. Future WorkV. Future WorkDevelopment of a system for fingerprinting wireless links:◦ Based on the nature of OWD & IPDV

distributions◦ Network Layer could infer the access link w/o

employing cross-layering techniquesExtending the Perl application:◦ Real-time mode◦ Capture and processing of traffic other than

RTP over UDP


measurement-based analysis of the performance of several...

Documents