is something wrong with routing - university of california ... · red-pd fair red and malicious...

Long Le, Jay Aikat, Kevin Jeffay and F. Donelson Smith Department of Computer Science University of North Carolina at Chapel Hill

April 28, 2009 1

Presented by

Alexander Afanasyev

!! Is something wrong with routing performance?

"!Default FIFO queuing has a number of problems

"! Active Queue Management (AQM) can help, but has its costs

!!What are the AQM solutions? "! Random Early Dropping (RED, Adaptive RED)

"! Proportional Integrator (PI)

"! Random Exponential Marking (REM)

!!Does AQM really help? "! Some algorithms can improve standard TCP

performance in some environments.

"! AQM can reduce channel utilization or increase latency

2

!! Packets are dropped on queue overflow !! Potential cause of flow synchronization problem

!! Some congestion avoidance (CA) algorithms wouldn’t be aware until loss is detected

"! A lot of packets can be dropped due to congestion before CA detects congestion state

"! A lot of network resources will be wasted to retransmit lost packets

!! Increased response latency (RTT) and jitter in “near-congestion” state !! Big FIFO buffers are considered optimal

"! 2-4 times bandwidth-delay-product

"! 100ms x Link Capacity

!! Flows can usurp the bottleneck link

3 Problem – Solutions - Evaluations - Discussion

!! Routers know exact queues state of their links

"! They can send reports to the senders to limit rate

!!When link is congested? "! Queue is not empty?

"! Queue is almost full?

!!How router should react to congestion? "! Drop all new packets?

"! Drop a portion of new and old packets?

"! Just report (ECN) to the sender to slow-down


5

28CHAPTER3. CLASSIFICATIONOF SELECTEDACTIVEQUEUEMANAGEMENTMECHANISMS

AQM classification Goal of algorithm

ECN Report congestion

DT Solve congestion problem

RED Provide performance guarantees (delay, throughput)

ATM-RED RED optimized for cell-based architecture in ATM networks

ARED Adaptive trade-off between link utilization and delay

SRED Like RED: provide performance guarantees (delay, throughput)

FRED Make RED fair

RED-PD Fair RED and malicious flow detection: cut back to target bandwidth

CHOKe Stateless, fair, RED-based algorithm

E-RED Improved stability (at the cost of support for bursty traffic)

AVQ Robustness, stability and delay bounds. (Regulate the queue size)

PI Regulate the queue length to keep the delay small

PIP Minimize PI’s queue length variance

REM Decouple congestion measure and performance

BLUE Low loss rates and low queue length oscillation

Table 3.1: Comparing the goal of selected AQM algorithms

3.4 Classification according to fairness of the solution

The classification of the surveyed AQM mechanisms with respect to their fairness is pre-

sented in Table 3.4. There exist only two cases: Either the solutions provide fairness or

not. An exception to this is ATM-RED, which proposes various strategies to drop cells.

Depending on the chosen strategy ATM-RED is fair or not.

3.5 Classification according to malicious-awareness

AQM mechanisms that are fair, can be malicious-aware as well. In this case they punish

misbehaving flow to compensate for service that has been obtain by fraud. Only few AQM

solutions are malicious-aware. An overview on this classification can be found in Table

3.5.

* K.!Graffi, K.!Pussep, N.!Liebau, and R.!Steinmetz, “Taxonomy of Active Queue Management Strategies in Context of Peer-to-Peer Scenarios,” Technische Universitat Darmstadt, Germany, Tech. Rep, 2007

Problem – Solutions - Evaluations - Discussion

SRC DSTData Data

ACKACK

SRC DSTData Data, Notify

ACKACK, Notify

GW

GW

Data

Src

GW

1

D D+Sf GW

N

D+Sf!

Dst

A+Sf!A+Sf!+Sr

GW 1, GW N " Internet routers on the transmission path between source Src

and destination Dst

D, A " Data and acknowledgment messages

Sf, Sr " Forward and reverse path network channel state indicators

Network


o! If link is congested, routers set ECN bit in the IP packets (Sf) o! TCP receiver echoes back ECN information in ACK response (Sf’)

o! TCP sender treats ACK packets with ECN as congestion indication

7

55

AQM Algorithms ConsideredAQM Algorithms ConsideredThe original RED AlgorithmThe original RED Algorithm

TimeTime

MaxMaxqueue lengthqueue length

MinMinthresholdthreshold

Mark/DropMark/Dropprobabilityprobability

No mark/dropNo mark/drop

MaxMaxthresholdthreshold

Forced dropForced drop

ProbabilisticProbabilisticearly mark/dropearly mark/drop

Router queue lengthRouter queue length

Mark/Drop probabilityMark/Drop probability

WeightedWeightedAverageAverageQueue LengthQueue Length

100%100%

minminthth maxmaxthth

maxmaxpp

Weighted average queue lengthWeighted average queue length

66

AQM Algorithms ConsideredAQM Algorithms ConsideredAdaptive/Gentle RED (ARED)Adaptive/Gentle RED (ARED)

TimeTime




Mark/DropMark/DropProbabilityProbability


MaxMaxthresholdthreshold ProbabilisticProbabilistic

early mark/dropearly mark/drop


22!!MaxMaxthresholdthreshold ProbabilisticProbabilistic

““gentlegentle”” drop drop

Mark/Drop ProbabilityMark/Drop Probability

WeightedWeightedAverageAverageQueueQueueLengthLength

100%100%

minminthth 22!!maxmaxthth

maxmaxppmaxmaxthth

77

Control Theoretic AQMControl Theoretic AQMThe Proportional Integral (PI) controllerThe Proportional Integral (PI) controller

•• PI attempts to maintain an explicit target queue lengthPI attempts to maintain an explicit target queue lengthTimeTime


•• PI samples instantaneous queue length at fixed intervalsPI samples instantaneous queue length at fixed intervalsand computes a mark/drop probability at and computes a mark/drop probability at kkthth sample:sample:

– p(kT) = a ! (q(kT) – qref) – b ! (q((k-1)T) - qref) + p((k-1)T)–– aa, , bb, and , and TT depend on link capacity, maximum RTT and the depend on link capacity, maximum RTT and the

number of flows at a routernumber of flows at a router

TargetTargetQueueQueue

ReferenceReference(qref)

88

Control Theoretic AQMControl Theoretic AQMRandom Exponential Marking (REM)Random Exponential Marking (REM)

•• REM is similar to PI (though differs in details)REM is similar to PI (though differs in details)TimeTime


•• REM mark/drop probability depends on:REM mark/drop probability depends on:–– Difference between input and output rateDifference between input and output rate–– Difference between instantaneous queue length and targetDifference between instantaneous queue length and target– p(t) = p(t–1) + ! [" (q(t) – qref)) + x(t) – c]– prob(t) = 1 – # -p(t), # > 1 a constant



Problem – Solutions - Evaluations - Discussion 8

55


TimeTime











100%100%


maxmaxpp


66


TimeTime













100%100%


maxmaxppmaxmaxthth

77









88








9

55


TimeTime











100%100%


maxmaxpp


66


TimeTime













100%100%


maxmaxppmaxmaxthth

77









88








!! “Live simulation” in lab test-bed

!! Synthetic WEB traffic using UNC HTTP model [SIGMETRIC 2001, MASCOTS 2003] !! Randomization of request/reply sizes, user think time

!! Persistent connection emulation

!! FreeBSD router implementation

!! 1Gbps bottleneck calibration, 100Mbps simulation

10

99

ISP1ISP1Browsers/Browsers/ServersServers

ISP2ISP2Browsers/Browsers/ServersServers

Experimental MethodologyExperimental MethodologyOverviewOverview

EthernetEthernetSwitchSwitch


•• Evaluate AQM schemes through Evaluate AQM schemes through ““live simulationlive simulation””•• Emulate the browsing behavior of a large population ofEmulate the browsing behavior of a large population of

users surfing the web in a laboratoryusers surfing the web in a laboratory testbed testbed–– Construct a physical network emulating a congested peeringConstruct a physical network emulating a congested peering

link between two ISPslink between two ISPs

ISP 1 EdgeISP 1 EdgeRouterRouter

ISP 2 EdgeISP 2 EdgeRouterRouter

–– Generate synthetic HTTP requests and responses but transmitGenerate synthetic HTTP requests and responses but transmitover real TCP/IP stacks, network links, and switchesover real TCP/IP stacks, network links, and switches

… …

CongestedCongestedLinkLink

1010

Experimental MethodologyExperimental MethodologyHTTP traffic generationHTTP traffic generation

•• Synthetic web traffic generated using the UNC HTTPSynthetic web traffic generated using the UNC HTTPmodel [SIGMETRICS 2001, MASCOTS 2003]model [SIGMETRICS 2001, MASCOTS 2003]

REQREQ

RESPRESP

UserUser

ServerServer

REQREQ

RESPRESP

REQREQ

RESPRESP

REQREQ

RESPRESP

REQREQ

RESPRESP

TimeTime

•• Primary random variables:Primary random variables:–– Request sizes/Reply sizesRequest sizes/Reply sizes–– User think timeUser think time–– Persistent connection usagePersistent connection usage–– Nbr Nbr of objects per persistentof objects per persistent

connectionconnection

Response TimeResponse Time

–– Number of embedded images/pageNumber of embedded images/page–– Number of parallel connectionsNumber of parallel connections–– Consecutive documents per serverConsecutive documents per server–– Number of servers per pageNumber of servers per page

1111

Experimental MethodologyExperimental MethodologyTestbed Testbed emulating an ISP peering linkemulating an ISP peering link

FreeBSDFreeBSDRouterRouter

FreeBSDFreeBSDRouterRouter


ISP1ISP1Browsers/Browsers/

ServersServers

ISP2ISP2Browsers/Browsers/

ServersServers

100100MbpsMbps


1 Gbps1 Gbps 1 Gbps1 Gbps 100100MbpsMbps

100100MbpsMbps

•• AQM schemes implemented in FreeBSD routers usingAQM schemes implemented in FreeBSD routers usingALTQ kernel extensionsALTQ kernel extensions

10-150 10-150 msms RTT RTT

•• End-systems either a traffic generation client or serverEnd-systems either a traffic generation client or server–– Use Use dummynetdummynet to provide to provide per-flowper-flow propagation delays propagation delays–– Two-way traffic generated, equal load generated in eachTwo-way traffic generated, equal load generated in each

directiondirection

… …1212

Experimental MethodologyExperimental Methodology1 Gbps network calibration experiments1 Gbps network calibration experiments

•• Experiments run on a congested 100 Mbps linkExperiments run on a congested 100 Mbps link•• Primary simulation parameter: Number of simulatedPrimary simulation parameter: Number of simulated

browsing usersbrowsing users•• Run calibration experiments on anRun calibration experiments on an uncongested uncongested 1 1 GbpsGbps

link to relate simulated user populations to average linklink to relate simulated user populations to average linkutilizationutilization

–– (And to ensure offered load linear in the number of simulated(And to ensure offered load linear in the number of simulatedusers users —— i.e.i.e., that end-systems are not a bottleneck), that end-systems are not a bottleneck)


100 Mbps100 Mbps(experiments)(experiments)


11GbpsGbps

11GbpsGbps

100100MbpsMbps

100100MbpsMbps

… …

11 Gbps Gbps(calibration)(calibration)


11

1313


We run experiments at offered loadsof 80%, 90%, 98%, and 105% of the

capacity of the 100 Mbps link


capacity of the 100 Mbps linkEx: 98% load means a number of

simulated users sufficient to generate 98Mbps (on average) on the 1 Gbps network

Ex: 98% load means a number ofsimulated users sufficient to generate 98

Mbps (on average) on the 1 Gbps network

Generating 98 Mbps ofHTTP traffic requiressimulating 9,330 users


Users

Link

Thr

ough

put (

Mbp

s)

20

40

60

80

100

120

140

160

180

200

1414

Experimental MethodologyExperimental MethodologyExperimental planExperimental plan

•• Run experiments with ARED, PI, and REM using theirRun experiments with ARED, PI, and REM using theirrecommended parameter settings at different offered loadsrecommended parameter settings at different offered loads

drop-tailAREDPIREM

80% 90% 98% 105%loss rate

utilizationresponse times

completed requests

uncongested

•• Compare results with drop-tail FIFO at the same offeredCompare results with drop-tail FIFO at the same offeredloadsloads……

–– The The ““negativenegative”” baselines (the performance to beat) baselines (the performance to beat)……and compare with performance on the 1and compare with performance on the 1 Gbps Gbps networknetwork

–– The The ““positivepositive”” baseline (the performance to achieve) baseline (the performance to achieve)•• Redo the experiments with ECNRedo the experiments with ECN

1515

Experimental ResultsExperimental Results —— 80% Load80% LoadPerformance with packet dropsPerformance with packet drops

50% ofresponses…

50% ofresponses…

…complete in125 ms or less…complete in125 ms or less

1616


No benefit to using PI or REMover drop-tail at 80% load


ARED can actuallymake things worse



1313











Users

Link

Thr

ough

put (

Mbp

s)

20

40

60

80

100

120

140

160

180

200

1414



drop-tailAREDPIREM

80% 90% 98% 105%loss rate


completed requests

uncongested




1515


50% ofresponses…

50% ofresponses…


1616







13

1313











Users

Link

Thr

ough

put (

Mbp

s)

20

40

60

80

100

120

140

160

180

200

1414



drop-tailAREDPIREM

80% 90% 98% 105%loss rate


completed requests

uncongested




1515


50% ofresponses…

50% ofresponses…


1616






50% requests complete within 128ms


1717


Drop-tail, PI, & REMequivalent for shortest

80% of responses


80% of responses

PI with qref = 240best overall


ARED notcompetitiveARED notcompetitive

1818


PI again best but thistime with qref = 24


Drop-tail & REMlargely equivalentDrop-tail & REMlargely equivalent

1919

Performance With Packet DropsPerformance With Packet DropsSummarySummary

•• At offered loads of 80% or below there is no benefit toAt offered loads of 80% or below there is no benefit toemploying PI or REM over drop-tail FIFOemploying PI or REM over drop-tail FIFO

–– All give comparable response time performance, loss rates,All give comparable response time performance, loss rates,and link utilizationand link utilization

•• There is a negative effect to employing ARED (at allThere is a negative effect to employing ARED (at allloads)loads)

–– Our attempts to tune ARED performance were unsuccessfulOur attempts to tune ARED performance were unsuccessful

•• At 90% and 98% loads PI outperforms drop-tail andAt 90% and 98% loads PI outperforms drop-tail andREMREM

–– But best parameter settings are load sensitiveBut best parameter settings are load sensitive

2020

Experimental Results Experimental Results —— REM REMPerformance with/without ECN at 90% loadPerformance with/without ECN at 90% load

Huge improvementfor qref = 240


REM performance improvedwith ECN for qref = 24



15

1717



80% of responses


80% of responses




1818





1919








2020







1717



80% of responses


80% of responses




1818





1919








2020







17

1717



80% of responses


80% of responses




1818





1919








2020







2121

Experimental Results Experimental Results —— ARED AREDPerformance with/without ECN at 90% loadPerformance with/without ECN at 90% load

ECN has little impact onARED performance


2222

Experimental ResultsExperimental Results —— 90% Load90% LoadPerformance with ECNPerformance with ECN

PI & REM outperform drop-tailand approximate performance on

the uncongested network



2323

DiscussionDiscussion1. Why does ARED perform so poorly?1. Why does ARED perform so poorly?

•• PI and REM measurePI and REM measurequeue length in bytesqueue length in bytes

•• By default REDBy default REDmeasures in packetsmeasures in packets

–– But ARED does haveBut ARED does havea a ““byte modebyte mode””

ARED Performance w/,w/o ECN at 90% Load

•• Drop/Mark probability in PI/REM biased by packet sizeDrop/Mark probability in PI/REM biased by packet size–– SYNs SYNs and pure and pure ACKs ACKs have a lower drop probability in PI/REMhave a lower drop probability in PI/REM

2424


•• ARED bases mark/drop probability on the (weighted)ARED bases mark/drop probability on the (weighted)average queue lengthaverage queue length

•• PI, REM use instantaneous measures of queue lengthPI, REM use instantaneous measures of queue length•• AREDARED’’s s reliance on the average queue length limits itsreliance on the average queue length limits its

ability to react effectively in the face ofability to react effectively in the face of bursty bursty traffictraffic

TimeTime

RouterRouterqueuequeuelengthlength

TimeTime


Weighted Queue Length(RED)

Instantaneous Queue Length(PI/REM)


19

2121




2222






2323







2424





TimeTime


TimeTime





!!Weighted queue length doesn’t allow to react effectively in face of bursty traffic

!!ARED “packet mode” implies small packets (SYN, ACKs) to have same probability to be dropped as big ones. But this doesn’t explain “bad” behavior if ECN is employed.

20

2121




2222






2323







2424





TimeTime


TimeTime





21

2525

DiscussionDiscussion2. Why does ARED not benefit from ECN?2. Why does ARED not benefit from ECN?

•• ARED drops marked packets when average queue size isARED drops marked packets when average queue size isabove above maxmaxthth

•• This is done to deal with potentially non-responsive flowsThis is done to deal with potentially non-responsive flows•• We believe this policy is a premature optimizationWe believe this policy is a premature optimization

TimeTime











2626

DiscussionDiscussion3. Why does ECN improve REM more than PI?3. Why does ECN improve REM more than PI?

•• Without ECN REM dropsWithout ECN REM dropsmore packets than PImore packets than PI

•• REM causes more flows toREM causes more flows toexperience multiple lossesexperience multiple losseswithin a congestion windowwithin a congestion window

–– Loss recovered through timeoutLoss recovered through timeoutrather than fast recoveryrather than fast recovery

•• In general ECN allows more flows to avoid timeoutsIn general ECN allows more flows to avoid timeouts–– Thus ECN is ameliorating a design flaw in REMThus ECN is ameliorating a design flaw in REM

•• Future work: Differential congestion notificationFuture work: Differential congestion notification–– DonDon’’t signal short flows that cant signal short flows that can’’t adaptt adapt

REM Performance w/,w/o ECN at 90% Load

2727

The Effects of AQM on Web PerformanceThe Effects of AQM on Web PerformanceConclusionConclusion

•• No AQM is better than drop-tail FIFO for offered loadsNo AQM is better than drop-tail FIFO for offered loadsup to 80% of link capacityup to 80% of link capacity

•• For offered loads of 90% or greaterFor offered loads of 90% or greater……–– Without ECN, PI results in a modest performanceWithout ECN, PI results in a modest performance

improvement over drop-tail and other AQM schemesimprovement over drop-tail and other AQM schemes–– With ECN, both PI and REM provide significant performanceWith ECN, both PI and REM provide significant performance

improvementimprovement–– (But is ECN improving PI & REM by ameliorating design(But is ECN improving PI & REM by ameliorating design

limitations?)limitations?)

•• ARED consistently results in the poorest response timeARED consistently results in the poorest response timeperformanceperformance

–– Often worse than drop-tail FIFOOften worse than drop-tail FIFO2828

The Effects ofThe Effects ofActive Queue ManagementActive Queue Management

on Web Performanceon Web Performance

Long Le, Jay Aikat, Kevin Jeffay, and Don SmithLong Le, Jay Aikat, Kevin Jeffay, and Don Smith

TheThe UNIVERSITY UNIVERSITY ofof NORTH CAROLINA NORTH CAROLINA atat CHAPEL HILL CHAPEL HILL

http://www.cs.unc.edu/Research/dirt

SIGCOMMSIGCOMM 20032003


2525

DiscussionDiscussion2. Why does ARED not benefit from ECN?2. Why does ARED not benefit from ECN?

•• ARED drops marked packets when average queue size isARED drops marked packets when average queue size isabove above maxmaxthth

•• This is done to deal with potentially non-responsive flowsThis is done to deal with potentially non-responsive flows•• We believe this policy is a premature optimizationWe believe this policy is a premature optimization

TimeTime











2626

DiscussionDiscussion3. Why does ECN improve REM more than PI?3. Why does ECN improve REM more than PI?

•• Without ECN REM dropsWithout ECN REM dropsmore packets than PImore packets than PI

•• REM causes more flows toREM causes more flows toexperience multiple lossesexperience multiple losseswithin a congestion windowwithin a congestion window

–– Loss recovered through timeoutLoss recovered through timeoutrather than fast recoveryrather than fast recovery

•• In general ECN allows more flows to avoid timeoutsIn general ECN allows more flows to avoid timeouts–– Thus ECN is ameliorating a design flaw in REMThus ECN is ameliorating a design flaw in REM

•• Future work: Differential congestion notificationFuture work: Differential congestion notification–– DonDon’’t signal short flows that cant signal short flows that can’’t adaptt adapt

REM Performance w/,w/o ECN at 90% Load

2727

The Effects of AQM on Web PerformanceThe Effects of AQM on Web PerformanceConclusionConclusion

•• No AQM is better than drop-tail FIFO for offered loadsNo AQM is better than drop-tail FIFO for offered loadsup to 80% of link capacityup to 80% of link capacity

•• For offered loads of 90% or greaterFor offered loads of 90% or greater……–– Without ECN, PI results in a modest performanceWithout ECN, PI results in a modest performance

improvement over drop-tail and other AQM schemesimprovement over drop-tail and other AQM schemes–– With ECN, both PI and REM provide significant performanceWith ECN, both PI and REM provide significant performance

improvementimprovement–– (But is ECN improving PI & REM by ameliorating design(But is ECN improving PI & REM by ameliorating design

limitations?)limitations?)

•• ARED consistently results in the poorest response timeARED consistently results in the poorest response timeperformanceperformance

–– Often worse than drop-tail FIFOOften worse than drop-tail FIFO2828

The Effects ofThe Effects ofActive Queue ManagementActive Queue Management

on Web Performanceon Web Performance

Long Le, Jay Aikat, Kevin Jeffay, and Don SmithLong Le, Jay Aikat, Kevin Jeffay, and Don Smith

TheThe UNIVERSITY UNIVERSITY ofof NORTH CAROLINA NORTH CAROLINA atat CHAPEL HILL CHAPEL HILL

http://www.cs.unc.edu/Research/dirt

SIGCOMMSIGCOMM 20032003


!! Only WEB traffic was simulated !! Other TCP traffic may have different patterns and thus AQM would

have different impact on them

!! Presence of UDP traffic can have worse impact if AQM is present

!! Fairness wasn’t considered

!! Small incentives for ISPs to implement AQM !! All experiments show good response time for majority of connections

!! Almost nobody runs their networks at 90-98% load

!! Only the standard TCP implementation was considered. Probable cause of ARED unsatisfactory results !! More than 40 variants of pure Host-to-Host congestion control

algorithms exist

!! Standard TCP has imperfect performance in the simulated environments (100Mb/1Gb links)

!! Paper doesn’t talk about AQM performance issues

!! Only 3 AQM algorithms were examined

23 Problem – Solutions - Evaluations - Discussion 24

!! What is wrong with routing performance? "! Default FIFO queuing has a number of problems "! Active Queue Management (AQM) can help, but has its

costs

!! What are the AQM solutions? "! Random Early Dropping (RED, Adaptive RED) "! Proportional Integrator (PI) "! Random Exponential Marking (REM)

!! Does AQM really help? "! Some algorithms can improve standard TCP

performance in some environments. "! AQM can reduce channel utilization or increase latency

is something wrong with routing - university of california ... · red-pd fair red and malicious...

Documents