1

Core Stateless Fair Core Stateless Fair QueueingQueueing

Ion Stoica Ion Stoica Hui Zhang Hui Zhang Scott Scott ShenkerShenker

CMU CMU Xerox PARCCMU CMU Xerox PARC

2

Document OrganizationDocument Organization

Provide a brief description and Provide a brief description and overview of Coreless Stateless Fair overview of Coreless Stateless Fair QueingQueing

Describe the algorithms supporting Describe the algorithms supporting the modelthe model

Simulations and comparisons with Simulations and comparisons with differnet loadsdiffernet loads

Conclusion and resultsConclusion and results

3

IntroductionIntroduction

The purpose of this paper is to The purpose of this paper is to achieve congestion control by achieve congestion control by allocation of bandwidth in a fair allocation of bandwidth in a fair manner.manner.

The architecture proposed is a set of The architecture proposed is a set of algorithms which are simple and algorithms which are simple and inexpensive to implement on a inexpensive to implement on a contiguous portion of network with contiguous portion of network with well defined interior and edges.well defined interior and edges.

4

Mechanisms for Congestion Mechanisms for Congestion Control and FairnessControl and Fairness

FIFO FIFO Drop tail strategy.Drop tail strategy. RED RED Buffer management, probabilistically Buffer management, probabilistically

drop drop packets to keep the occupancy packets to keep the occupancy between between the max and min the max and min thresholdsthresholds

FREDFRED(Flow Random early drop) Extends (Flow Random early drop) Extends RED RED to achieve fairness. to achieve fairness.

DRRDRR(Deficit Round Robin) Employs (Deficit Round Robin) Employs sophisticated weighted fair sophisticated weighted fair

queuing.queuing.

5

Architecture for Core Stateless Fair Architecture for Core Stateless Fair QueuingQueuing

Edge RouterEdge Routera)a) Maintains Flow per Maintains Flow per

state.state.b)b) Label the packet Label the packet

with the rate with the rate estimateestimate

Core RouterCore Routera)a) Uses FIFO QueuingUses FIFO Queuingb)b) Probabilistic Probabilistic

droppingdroppingdepending on label, depending on label, fair rate of router fair rate of router depending on depending on aggregate trafficaggregate traffic

6

Functions of Edge and Core Functions of Edge and Core RoutersRouters

Classify the packets to a flowClassify the packets to a flow Label the packetLabel the packet

Update the fair share rate estimation for Update the fair share rate estimation for the outgoing linkthe outgoing link

Update the flow rate estimationUpdate the flow rate estimation Forward the packet depending on Forward the packet depending on

dropping probability max(0,1-dropping probability max(0,1-αα/r(i))/r(i)) Relabel the packet.Relabel the packet.

7

Fluid model and Packet Algorithm Fluid model and Packet Algorithm at the coreat the core

Suppose each flow arrival rate ri(t) is knownSuppose each flow arrival rate ri(t) is known If it’s fair share rate is a(t), then the service If it’s fair share rate is a(t), then the service

received by the flow at time t will be received by the flow at time t will be min(ri(t) ,a(t))min(ri(t) ,a(t))

Thus the total outgoing traffic from the Thus the total outgoing traffic from the router would be the ∑ (min(ri(t) ,a(t)) =Crouter would be the ∑ (min(ri(t) ,a(t)) =C

The fraction of packet dropped from each The fraction of packet dropped from each flow is calculated by ((ri(t) - a(t))/ ri(t).flow is calculated by ((ri(t) - a(t))/ ri(t).

8

Flow Arrival rate at the edge routerFlow Arrival rate at the edge router

Use a exponential averaging system.Use a exponential averaging system. The estimated rate of arrival of each The estimated rate of arrival of each

new packet is computed from the new packet is computed from the inter arrival time, the length of the inter arrival time, the length of the packet and the previous rate which is packet and the previous rate which is thus updatedthus updated

Rnew = (1-e-Ti/K)*Li/Ti + e-Ti/K*RoldRnew = (1-e-Ti/K)*Li/Ti + e-Ti/K*Roldwhere Li is the length of the packet at tiwhere Li is the length of the packet at ti

and T is the time between packet ti-1 and tiand T is the time between packet ti-1 and ti

9

Link Fair Rate EstimatorLink Fair Rate Estimator

The estimated rate is put into the label by the The estimated rate is put into the label by the edge router.edge router.

At the core router the fair rate is estimated from At the core router the fair rate is estimated from the algorithm of aggregated arrival rate and the algorithm of aggregated arrival rate and aggregated accepted rate which are updated aggregated accepted rate which are updated from each arrival of packets.from each arrival of packets.

10

Link congestedLink congested

F(α)=∑min(ri(t), α(t))F(α)=∑min(ri(t), α(t))α(t) is the fair share rateα(t) is the fair share rateF is the estimated fair F is the estimated fair packet accepance rate of packet accepance rate of all the traffic.all the traffic.

Agg(n )=(1-e-T/K)*L/T + e-Agg(n )=(1-e-T/K)*L/T + e-T/K*Agg(o)T/K*Agg(o)Agg(n)The estimated Agg(n)The estimated aggregate arrival rate aggregate arrival rate F is calculated similarlyF is calculated similarly

The estimate for fair share The estimate for fair share rate is updated likerate is updated like

αnew =αold *C/Fαnew =αold *C/F

Link not congestedLink not congestedαα is said to the largest rate is said to the largest rate of any active flow r(i) and of any active flow r(i) and aggregate received traffic aggregate received traffic

Agg(n) <C during an Agg(n) <C during an interval of length Kcinterval of length Kc

11

AlgorithmAlgorithm

The edge router estimates the rate usingThe edge router estimates the rate using

Rnew = (1-e-Ti/K)*Li/Ti + e-Ti/K*RoldRnew = (1-e-Ti/K)*Li/Ti + e-Ti/K*Rold

and puts in the labeland puts in the label

The core router finds the probability of dropping The core router finds the probability of dropping the packet using the fair rate estimator the packet using the fair rate estimator αα

Drop Prob=max(0,1-Drop Prob=max(0,1- αα /p.label) /p.label)

The rate estimator The rate estimator αα is updated with each packet is updated with each packet

12

Label rewritingLabel rewriting

If there is congestion at core router, If there is congestion at core router, the packet labels need to be updated the packet labels need to be updated at the core router which is done byat the core router which is done by

LLnewnew = min(L = min(Loldold, , αα))

Outgoing rate is the minimum of Outgoing rate is the minimum of incoming and the fair rateincoming and the fair rate

13

Weighed CSFQWeighed CSFQ

If priority is needed for certain specific If priority is needed for certain specific flows this algorithm can be updated to flows this algorithm can be updated to

include the weighs of the flows and include the weighs of the flows and thus the dropping probability of the thus the dropping probability of the packet would bepacket would beMax(0,1- Max(0,1- αα w wi/i/r(i))r(i))The sum of all weigted flow should be The sum of all weigted flow should be a solution for the link capacity.a solution for the link capacity.

∑ ∑wwii min( min(αα,r(i)/w,r(i)/wii) =C) =C

14

Performance bounds for weighed Performance bounds for weighed CSFQCSFQ

During a time interval a flow can During a time interval a flow can receive excessive service due to it’s receive excessive service due to it’s weight.weight.

The excessive service can be The excessive service can be restricted to this flow by restricted to this flow by implementing a performance bound implementing a performance bound such that it cannot exceed the fair such that it cannot exceed the fair share.share.

15

Choice of K for algorithmChoice of K for algorithm

e-T/Ke-T/K

A smaller K increases the system A smaller K increases the system responsiveness to rapid rate responsiveness to rapid rate fluctuationfluctuation

A larger K better filters the noise and A larger K better filters the noise and avoids potental system instability avoids potental system instability The K should be larger then delay The K should be larger then delay jitter of the flow and less then jitter of the flow and less then average duration of the flowaverage duration of the flow

16

SimulationsSimulations

Compare the CSFW performance Compare the CSFW performance against 4 other algorithmsagainst 4 other algorithms

FIFO and RED do not attempt to FIFO and RED do not attempt to achieve fair bandwidth allocationachieve fair bandwidth allocation

FRED and DRR Use other FRED and DRR Use other approaches to achieve fairnessapproaches to achieve fairness

17

A single 10 Mbps Link shared by N flowsA single 10 Mbps Link shared by N flows

Throughput for 32 UDP flows. The arrival rate for flow is (i+1) times larger then its fair share.

18

A single congested link with1 UDP flow and 31 TCP flowsA single congested link with1 UDP flow and 31 TCP flows

Throughput when the UDP flow is sending at 10 Mbps and the rest are TCP averaged over a 10 second interval

19

Normalized bandwidth of a TCP flow competing against N-1 UDP flows. Normalized bandwidth of a TCP flow competing against N-1 UDP flows. The UDP flows sending at twice their allocated rates as 2*10/NThe UDP flows sending at twice their allocated rates as 2*10/N

ResultsDRR performs well when there are less then 22 flows. CSFQ performs better with TCP burstiness as it allows packet buffering . CSFQ performs better then FRED throughout.

20

Multiple congested LinkMultiple congested Link

TCP Source

Udp1 Udp-10 UdpK1 UdpK10

TCP SinkGateway Gateway

21

UDP and TCP flow as a function of no of UDP and TCP flow as a function of no of congested links and Normalized throughputcongested links and Normalized throughput

a) The flow is UDP b) The flow is TCP

The UDP and TCP flow send at its fair share .909MbpsThe links are congested since all UDP send at a rate of 2MBps causing congestion which has the capacity of 10 Mbps

22

Coexistence of different adaptation schemesCoexistence of different adaptation schemes 3 Receiver layered Multicast flows along with 3 Receiver layered Multicast flows along with

a TCP of each having a fair share of 1 Mbps a TCP of each having a fair share of 1 Mbps

CSFQ

FRED in this does not provide fair bandwidth allocation as RLM and TCP use different end to end congestion control algorithms

DRR FRED FIFO

RED

23

On Off Flow with 19 competing TCP flowsOn Off Flow with 19 competing TCP flows

AlgorithAlgorithmm

DeliveredDelivered DroppedDropped

DRRDRR 601601 61576157

CSFQCSFQ 16801680 50785078

FREDFRED 17141714 50445044

REDRED 53225322 14361436

FIFOFIFO 54525452 13061306

The On Off perios is derived from exponential distribution of mean of 100 and 1900 ms. During the On-Off period the source sends at 10MbpsThe DRR and CSFQ show that the ON-Off source drops packets since it is limited to its fair share

24

Simulation of web traffic Simulation of web traffic Interarrival times are exponentially distributed with mean of Interarrival times are exponentially distributed with mean of

.05ms and length of each transfer is from a Pareto distribution..05ms and length of each transfer is from a Pareto distribution.

AlgorithAlgorithmm

Mean Mean timetime

Std. devStd. dev

DRRDRR 2525 9999

CSFQCSFQ 6262 142142

FREDFRED 4040 174174

REDRED 592592 12741274

FIFOFIFO 840840 16951695The mean transfer time and std deviation for 60 short TCP flows in the presence of UDP flow that sends at link capacity of 10 MbpsResult CSFQ worse then FRED since it has a larger queue size.

25

Large Latency modelLarge Latency model

AlgorithmAlgorithm MeanMean Std. devStd. dev

DRRDRR 60806080 6464

CSFQCSFQ 57615761 220220

FREDFRED 49744974 190190

REDRED 628628 8080

FIFOFIFO 378378 6969

The mean throughputs and std deviations for a 100 second interval for 19TCP and 1 UDp flow at link capacity of 10 Mbps. The propagation delay is 100 ms

Due to propagation delay we have to set the buffer size to be 256Kb and K,Kα,Kc to be 400 ms

26

Packet relabelingPacket relabelingFlow1 10 Mbps

Flow2 10 Mbps

Flow3 10 Mbps

Sink

Gateway

Gateway10Mbps Link10 Mbps Link

27

Throughput from CSFQ averaged over 10 Throughput from CSFQ averaged over 10 seconds for the 3 flows along link 2seconds for the 3 flows along link 2

TrafficTraffic Flow 1Flow 1 Flow 2Flow 2 Flow 3Flow 3

UDPUDP 3.363.36 3.323.32 3.283.28

TCPTCP 3.433.43 3.133.13 3.433.43

28

Identification of Friendly and Identification of Friendly and unfriendly flows by RED and CSFQunfriendly flows by RED and CSFQ

AlgorithAlgorithmm

UDPUDP TCP1TCP1 TCP2TCP2 TCP1TCP1 TCP2TCP2

REDRED .90.9066

.280.280 .278.278 .565.565 .891.891

CSFQCSFQ .55.5544

.468.468 .478.478 .729.729 .747.747

| Simulation1 | | Simulation2 |

The throughput of one UDP and 2 TCP flows along a 1.5 Mbps link under RED and CSFQ. In first simulation 1 Mbps UDP flow and 2 TCP flowIn the second simulation we have modified TCP which opens its congestion window 3 times faster

29

Achieving Fair bandwidth AllocationAchieving Fair bandwidth Allocation

Identifying FlowsIdentifying Flows

Use a fair detection algorithm to identify Use a fair detection algorithm to identify unfriendly flowsunfriendly flows

Isolating FlowsIsolating Flows

Router allocates resources separately for Router allocates resources separately for friendly and unfriendly flows.friendly and unfriendly flows.

30

SummarySummary

FRED requires per packet flow classification FRED requires per packet flow classification while CSFQ does not, so the fairness can be while CSFQ does not, so the fairness can be achieved in a scalable manner.achieved in a scalable manner.

The CSFQ would comprise of high speed The CSFQ would comprise of high speed backbones and lower speed edge routers backbones and lower speed edge routers where classification and per flow operation where classification and per flow operation is done. is done.

The edge routers must be distinguishable The edge routers must be distinguishable from the core by some configuration.from the core by some configuration.

31

AcknowledgementsAcknowledgements Professor Bob Kinicky WPI for valuable information and adviceProfessor Bob Kinicky WPI for valuable information and advice Ion Stoica CMU AuthorIon Stoica CMU Author Scott Shenker Xerox Author Scott Shenker Xerox Author Hui Zhang CMU Author Hui Zhang CMU Author Nagaraj Shirali For help with Jpeg graphsNagaraj Shirali For help with Jpeg graphs Choong-Soo Lee For help with Jpeg graphsChoong-Soo Lee For help with Jpeg graphs