advanced networking technologies - startseite tu ilmenau · advanced networking technologies...

1

Advanced NetworkingTechnologies

Chapter 3Routers and Switches –Size and Organization

of Router BuffersThis chapter is heavily based on material by Holger

Karl (Universität Paderborn), AQM slides are based on material by Manolis Sifalakis (IBM)

2

Routers as queue servers?

q Just suppose… q Traffic were Poissonian, routers were M/M/1

q At a buffer size of B, link load ½, packet loss probability drops exponentially in buffer size B

q i.e., small buffers should avoid packet loss

Advanced Networking (SS 17): 03 - Size of Router Buffers

from [1]

3

Question for this chapter

q How big should the buffers in a router be?

q Goal: Maximum efficiency = keep every outgoing link busy all the time

q Argument for large buffers: Can compensate for senders not sending all the time, smooth out variations in ingoing traffic

q Argument for small buffers: TCP will fill every bottleneck buffer, will lead to large delays


4

Overview

q Brief recap: TCP congestion control scheme q The single flow caseq Many flows: synchronized or not? q Tiny buffers q Optimizing Drop Behaviour


5

TCP ACK/self-clocking

q Suppose TCP has somehow determined a “correct” size of its congestion windowq Suppose also that the TCP source has injected this entire amount of data

into the network but still has more data to sendq When to send more data?

q Only acceptable when there is space in the network againq Space is available when packets leave the networkq Sender can learn about packets leaving the network by receiving an

acknowledgement!q Thus: ACK not only serves as a confirmation, but also as a permit to

inject a corresponding amount of data into the network

! ACK-clocking (self-clocking) behavior of TCP


6

AIMD – Sawtooth pattern of TCP’s offered load

q In summary: TCP uses an additive increase multiplicative decrease (AIMD) scheme

q Consequenceq A TCP connection perpetually probes the network to check for additional

bandwidthq Will repeatedly exceed it and fall back, owing to multiplicative decreaseq Sawtooth pattern of TCP’s congestion window/offered load

■ This is still simplified; we have to introduce one more mechanism!


60

20

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0

KB

Time (seconds)

70

304050

10

10.0Con

gest

ion

win

dow

7

Quickly initialize a connection: Slow start

q Additive increase nice and well when operating close to network capacity

q But takes a long time to converge to it for a new connectionq Starting at a congestion window of, say, 1 or 2

q Idea: Quickly ramp up the congestion window in such an initialization phaseq Goal: double congestion window each RTTq Implemented by: increase congestion window by

one packet per arriving ACKq Instead of just adding a single packet per RTT

Advanced Networking (SS 17): 03 - Size of Router Buffers 7

Source Destination

Note: Name “slow start” is historic – it was slow compared to some earlier, even more aggressive scheme

8SS 16, v.1.1 Advanced Networking (SS 17): 03 - Size of Router Buffers 8

TCP Tahoe congestion window dynamics (w/o anim)

Initial congest

ion window

size

Initial congest

ion thresh-

hold

Slow start

Additive increase,

congestion avoidance

New slow start to new threshold, then linear increase

20=40/2

Reset CW to 1, new

threshold = CW/2

9

TCP Reno congestion control dynamics


10

Summary: TCP Reno sender congestion control


CongWin and Threshold not changed

Increment duplicate ACK count for segment being acked

SS or CADuplicate ACK

Enter slow startThreshold = CongWin/2, CongWin = 1 MSS,Set state to “Slow Start”

SS or CATimeout

Fast recovery, implementing multiplicative decrease. CongWin will not drop below 1 MSS.

Threshold = CongWin/2, CongWin = Threshold,Set state to “Congestion Avoidance”

SS or CALoss event detected by triple duplicate ACK

Additive increase, resulting in increase of CongWin by 1 MSS every RTT

CongWin = CongWin+MSS * (MSS/CongWin)or:CongWin += 1/CongWin

CongestionAvoidance (CA)

ACK receipt for previously unacked data

Resulting in a doubling of CongWin every RTT

CongWin = CongWin + MSS, If (CongWin > Threshold)

set state to “Congestion Avoidance”

Slow Start (SS)

ACK receipt for previously unacked data

CommentsTCP Sender Action StateEvent

11

TCP Reno, relevant scenario

q Assumptions:q Sender is backlogged (always has packets to send) q No queuing delays for ACKsq Only one bottleneck, maximum output rate Cq Tp propagation delay, Tq queuing delay

q Behaviour: congestion window W is always fully utilized by sender, it has W outstanding packets, it can send new packet after RTT q Hence: Sending rate R = W/RTT = W/ (2Tp + Tq)

q We are interested in congestion avoidance mode, no timeouts, losses always detected via dup-acksq ACK arrives: W += 1/Wq Loss detected: W /= 2

■ … and has to wait for enough ACKs to arrive before it may continue sending!


from [1]

12

Homework to deepen understanding of Reno

q Derive the period of a single, TCP Reno flow’s sawtooth pattern (under the above assumptions)

q Hints q Find a differential equation linking

■ the change of the congestion window per RTT ■ the size of the congestion window both expressed as a function of the queue length at time t

q Solve the differential equation (think about square roots)q Assume router buffer is empty when the TCP flow starts q Assume you have a router buffer size B q Determine the sawtooth pattern period as a function of B and C


14

Overview



15

Router buffer size for the single TCP-Reno flow

q Assume there is only a single TCP flow in the network q … and the above assumptions as well

q Question: How large should the router buffer be at the bottleneck link so that this link is always sending packets?


from [1]

16

Congestion avoidance dynamics

q TCP increases congestion window, filling up the buffer eventually q Sender will increase window a bit more, causing the first packet loss q Sender detects loss (dup acks), cuts congestion window in half, and

pauses q Sender had W* packets in flight when loss detected (congestion window

always fully used!) ■ Note: W* measured in packets here, not bytes

q Now allowed to only have W*/2 packets in flight, must wait for W*/2 ACKs to arrive

q During this pause, bottleneck link sends packets from buffer q Buffer needs to be just big enough to hold enough packets to cover this

pause


17

ACK arrivals & buffer size

q So: how long does the sender pause?q Router sends packets from buffer at rate C packets/s q ACKs, hence, arrive at sender at rate C ACKs/s

q Sender has to wait for W*/2 ACKs, this takes W*/2/C seconds

q Hence: Router must be able to send packets for (at least) W*/2/C seconds from its bufferq At sending rate C, that takes C (W*/2/C) = W*/2 packets! q Example: I need to send 5 packets per second, for 10 seconds. That

requires 50 packets.

q Hence: Buffer B >= W*/2 !


18

At what congestion window does a loss happen?

q Or: What is W*? q Look at situation when the sender just resumes transmission

q Its current congestion window is W=W*/2, the router buffer is empty q We are in congestion avoidance!

q For the bottleneck link to be fully utilized, the sender should ideally send at packet rate C q But a TCP sender’s rate R is determined by its congestion window and

RTT; R = W/RTT q Hence we require R = C = W/RTT q RTT here is only 2Tp, since buffer is empty, no queuing delay

q Hence: C = W*/2 / 2Tp !


19

What can we control? Buffer size!

q So two equations now: q B >= W*/2 q C = W*/2 / 2Tp

q where C, Tp are given, B can be chosen

q Solve for buffer size B:

B >= 2TpC

q Famous “rule of thumb”: buffer size should be at least the bandwidth-delay product of any flow


Summary: • Just before packet drop: W = 2TpC + B• Just after packet drop: W/2 = 2TpC

Note: Why W*/2? Reno cuts in half! Actually: B = Wmax – Wmin!

20

Behaviour of a single TCP flow


from [1]

21

Overview



22

Many TCP flows

q So far: only one flow in the network q What happens if many flows multiplex a bottleneck link?

q Step 1: let all these flows be long-lived and synchronized in timeq Step 2: all long-lived, but non-synchronousq Step 3: mix long-lived and short-lived flows


23

Many long-lived, synchronized TCP flows

q Synchronized: q Flows are allowed to start at

arbitrary times q Under some circumstances:

they will quickly synchronize, i.e., their congestion window evolution will become parallel, observing packets losses at nearly the same points in time ■ Homework: Find out when

and why synchronization happens!

q Due to synchronization, same dynamics, same buffer requirements as a single flow


24

Many-long lived, desynchronized TCP flows

q What if flows do not synchronize? q Effectively: adding up sawtooth patterns, shifted with respect to each

other q Smooth each other out

q Recall: Buffer needed for difference from (aggregated) congestion window minimum to (aggregated) congestion window minimumq Smoothing: Difference max-min becomes smaller q Less buffer??


25

Long-lived, desynchronized – Ansatz

q Assume n flows pass through router, congestion windows not synchronized (independent offsets) q Each with window Wi, aggregate window W = åi Wi

q All senders backlogged again, i.e., all senders fully exploit their windows q Where are the packets?

q Either in flight, in buffer, or lost: W(t) = Q(t) + 2TpC + ²q Or looking at the queue: Q(t) = W(t) – 2TpC - ²

q W is a random variable, Tp, C, ² constants q We need random distribution of Wq Then we know distribution of Q q Then we can bound probability of Q being empty


26

Long-lived, desynchronized – Distribution of W

q Distribution of Wi : Uniform U(2/3 E[Wi], 4/3 E[Wi])q Hint: Think about the sawtooth pattern, its triangular shape, express E[Wi]

as function of W*q Hence: standard deviation ¾Wi

= 1/(3·30.5) E[Wi]

■ Property of uniform distribution q Distribution of W: normal, by law of large numbers q We want standard deviation of queue length


27

Buffer empty

q How likely is it that link is utilized? q I.e., how likely is it that buffer is empty? q Or: buffer smaller than given threshold ?

q Queue length is normal distributed, we know its standard deviation !we can bound deviations from mean

q Examples, with n=10.000 flows: q Utilization: probability of not seeing a queue shorter than some constant


Typical core router:>= 10.000 flows!

28

What about short flows?

q Slow flow: Never leaves slow start q Assume:

q Access links have infinite speed ■ Access slower than backhaul link: bursts are smoothed out anyway

q New flows arrive according to Poisson processq Each flow generates a sequence of packet bursts

■ With some distribution on how many bursts each flow generates■ Suppose flow wants to send l packets, it will send bursts

2, 4, 8, 16, … 2n-1, R where n= number of bursts, R = l mod (2n-1-1)

q Burst arrivals are also Poisson process! ■ Justification: Each flow small compared to overall load (!?)

q Hence: Router turns into an M/G/1 server


29

Queuing theory on M/G/1

q Poisson arrival, general service time for jobs, one server q Here “job”: a packet burst q Let X be random describing job length = burst size; we need first and

second moment ■ Small homework: assume l is uniformly distributed up to some max,

empirically derive these moments q Known: Average number of jobs in queue for M/G/1 for given E[X2]

and link load ½

q Note: independent of number of flows!


E{Tv} =1

2(1� ⇢)

E{X2}E{X}

30

Average queue length for short flows


Why peak at 14,not 16?

31

A first summary

q This analysis so far: q A simplistic model seems to require router buffers on the order of the

datarate-delay product to ensure high link utilization q Understanding the behaviour of many flows sharing a link (irrespective of

short- or long-lived), similar behaviour (link utilization!) can be achieved with substantially smaller routers ■ Datarate-delay product, divided by square root of number of

concurrent flows q In mixed traffic, long flows determine behaviour

q Open issues: q Does this hold up in practice? � a marketing problem! q Number of flows in a real router? � see later q Sacrifice 100% utilization goal?


32

Overview



33

Very small buffers?

q What happens if we make the buffers VERY small? q In the model considered so far, huge packet losses, low utilization q Losses come from TCP’s packet bursts

q So let’s reconsider our model – what if TCP would not burst packets into the network whenever it has the chance? q “Pacing” TCP – in protocol, or implicitly by slow access links


34

Model assumptions for very small buffers

q Model:q TCP sends packets via a Poisson process, with rate W/RTT q TCP uses some maximum window size Wmax

■ Hence, maximum rate is Wmax/RTT q We have N flows, all have same RTT q Many flows share a link of capacity Cq Shared link is overprovisioned by factor 1/½, i.e.,

C ¸ (1/½)· N · Wmax/RTT ■ Hence, highest possible throughput is ½ C

q Goal: We are willing to tolerate a utilization £ < 100%q £ = Achieved throughput / ½ C


35

Buffer for utilization £

q To obtain utilization of £ under Poissonian packet arrivals, we need a buffer size of:

q Why not surprising: q It depends on £ and ½, also on Wmax

q Why surprising: q It does NOT depend on link capacity! Nor on number of flows!

q Example: Wmax= 64 (Linux), ½ =0.5, £=0.9: B = 15!


36

Tiny buffers, realistic model

q Poissonian packet arrival is simply wrong q But gives interesting insight; although proof complicated

q In practice: use Paced TCP q TCP sends packets not

in a burst when window opens, but at fixed rate of W/RTT.

q Still works surprisingly well!


From [2]

37

Buffer Size


Thro

ughp

ut

log(W)

25,000~50

~ 90%100%

2,500,00010Gb/sWANNumberofpackets

20pkts

Integratedall-opticalbuffer[UCSB2008]

On-chipbuffersSmallerdesignLowerpower

From:McKeown, SIGCOMM Keynote, 2013

RTT ⇥ C

RTT ⇥ CpN

38

Overview



39

Active Queue Management

q As Queues build up, RTT increases, congestion becomes hard to remediate (notice this has the opposite effect from the original intend)q Takes longer for endpoints to react (TxWin adjustment)

q ACK spacing can get compressed increasing burstiness of transmissions

q The larger the capacity of the queue

q The more detached are the endpoints from the truth about congestion

q The more likely that the queues become persistent: “bufferbloat” (long standing delay is treated as the norm)

q � Active Queue Management

AdvancedNetworking(SS17):02- Input-bufferedswitches

40

RED Model

q Maintain Exp Moving Avg(EWMA) of queue lengthq Byte mode vs. Packet mode

depending if Tx delay is a function of packet length or not

q For each packet arrivalif (avgq < minth) do nothing

if (maxth ≤ avg) mark(packet)

if (minth ≤ avgq < maxth)

calculate probability Pmmark(Pm, packet)

q Marked packets are either dropped or ECN flagged


Min threshMax thresh

AvgQueue Length

minth maxth

Pmax

1.0

Avg queue length

Dro

ppr

obab

ility

Actual Queue Length

PktArrival

41

RED Parameters

q EWMA of Queue size computed at every packet arrival (not periodically!)

avgqnow = wq * qlennow + (1-wq) * avgqprev

Special condition if queue was idle, i.e. qlennow = 0q Same as if it had been 100% link utilization with 0 queue.q Approx. that m small packets could have been processed

m = (tnow – tlast_arrival )/ pkt_size nominal

avgqnow = (1-wq)m * avgqprev

q Packet marking probability is a function of % of the between thresholds utilitisation

Pm = Pmax * (avgqnow – minth) / (maxth – minth)


42

Issues with RED: configurability

q avgq is an EWMA � wq adjusts the lag and trend and window of averagingq Short window: fast sensing but vulnerable to transientsq Long window: slow adaptation

q minth � adjusts the “power of the network”q “Too close” to 0: the queue is likely to have idle periods (bandwidth not

used)q “Too far” from 0: increases path latency, delays feedback signal to

endpointsq maxth– minth� adjusts the frequency of marking

q “Too small”: the AQM becomes spasmodic in its reaction, forces flows to sync

q Must be larger that the typical avgq increase in an RTTq A visual analogy: which vessel size for which sea condition?

q Think traffic bursts like the swell of the sea (height, length)q Think of the AQM as a speed boat or a a tanker depending on swell


43

Issues with RED: configurability

q Average Queue size oscillationq Difficult to control congestion when many flows

q esp. unresponsive ones

q Many optimizations…


time

Qlenmax

0

Actual

Avg

32 flows

time

Qlenmax

0

Actual

Avg

8 flows

44

RED & variants – FRED

q Fair RED or Flow RED: fairness among flow typesq Flow type differentiation based on queue use

■ non-adaptive (UDP), fragile (sensitive to loss), robustq per-active-flow (present in the queue buffer) accounting and loss-

regulation■ All flows entitled to admit minq packets without loss■ Adjust minq based on avg. per flow occupancy (avgcq) of queue■ Set upper capacity per flow type to maxq and count violations (strikes).

Then for frequent violators lower maxq = avgcq (to punish them more often)


45

RED & variants – CHOKE

q CHOose and Kill unresponsive flows... or .. CHOose and Keep responsive flowsq Compare incoming packet with random selected packet in queueq Aggressive flows become more likely to select


Min threshMax thresh

Apply RED (variant)

PktArrival

Flow IDMatch

No

Yes

Select Randomly

46

RED & variants – ARED

q Adaptive RED: q reduces delay variance + introduces parameter auto-tuneq Adapt Pmax periodically, slowly, and with an AIMD policyq Fix maxth = 3 * minthq Fix wq = 1 - exp(1/Link_capacity)q Goal: To keep avgq constant at about (maxth+ minth)/2


47

RED & variants – SRED

q Stabilised RED – eliminate need of avgq (and wq)q Pm = f(inst. queue length, # of active flows, rank of flow)q Zombie list: history of K seen flows with Hit counters. On packet arrival

pick Zombie flow randomlyif flows match

hits++ else

replace Zombie with prob. Prq Statistical counting of flows based on hit frequency q Rank of flow = Hit counter on match


48

RED & variants – BLUE

q Putting past insights in new lightq Avoid parameter tuning nightmareq Avoid effects of avg queue fluctuation on AQM

q Adaptive marking probabilityq Pm = f (packet loss, link idle events)q [Pkt loss] if (tnow – tlast_arrival > freeze_period) Pm= Pm + d1q [Idle link] if (tnow – tlast_arrival > freeze_period) Pm= Pm - d2q d1 >> d2 : faster reaction to congestion up-rise than decreaseq freeze_period works as a sort-of A/D discretizer (step-hold)

■ Filters out high-freq transient oscillations■ Adjusts parameter at packet arrivals times mod a fixed quantum

q SFBlue uses ideas of SFQ and FRED to discriminate flows


49

CoDel – Controlled delay

q Time-based model of queue dynamics instead of a spatial one

q Objective: Monitor how long the Min queue length remains above a threshold (desired Min) q Less descriptive than Avg. Min queue length, q Yet sufficient, and simpler computationally

q Metric: Sojourn time as a measure of instantaneous queue lengthq How long packets stay in the queue: Time delta between a packet’s

departure and arrival timeq Works with a single queue or multiple queuesq Works for variable link rates (e.g. wireless links) – well statistically

speaking!q Simple to measure, easy to implement


50


On packet arrival: On packet departure: timestamp(packet)

1. dropping rate increases linearly with the measured RTT

2. if a last epoch of drops is not too long in the past, the last

good drop rate is also remembered!

sojourn = now – packet.tstampif (sojourn < Target)if (drp_mode == 1)drp_mode = 0exit_drp = nowif (now – exit_drp >= Interval)drp_count = 0

else // sojourn > Targetif (drp_mode==0 && now - exit_drp < Interval)drp_mode = 1if (now >= next_drp)drp(packet)drp_count++next_drp = now + Interval/sqrt(drp_count)

else if (drp_mode = 0) // start dropdrp_mode = 1drp(packet)drp_count = 1next_drp = now + Interval/sqrt(drp_count)else // already in drp_modeif (now >= next_drp)drp(packet)drp_count++next_drp = now + Interval/sqrt(drp_count)


51


On packet arrival: On packet departure: timestamp(packet)

Sojourn falls below Target

Only reset drop rate memory if Sojourn is below Target for Interval

Sojourn above Target again after temporary improvement, resume last drop rate

Sojourn above Target first time, after Interval, start dropping

Sojourn continues to remain above Target, continue dropping


sojourn = now – packet.tstampif (sojourn < Target)if (drp_mode == 1)drp_mode = 0exit_drp = nowif (now – exit_drp >= Interval)drp_count = 0

else // sojourn > Targetif (drp_mode==0 && now - exit_drp < Interval)drp_mode = 1if (now >= next_drp)drp(packet)drp_count++next_drp = now + Interval/sqrt(drp_count)

else if (drp_mode = 0) // start dropdrp_mode = 1drp(packet)drp_count = 1next_drp = now + Interval/sqrt(drp_count)else // already in drp_modeif (now >= next_drp)drp(packet)drp_count++next_drp = now + Interval/sqrt(drp_count)

52


q Significantly less configuration magic involvedq Interval: const (≥ 1RTT)q Target (delay): const

■ max{ equiv of 1-2 packets worth of queue, 5% of worst case RTT}q Drop/Mark rate: const acceleration in Interval

■ inverse-square-root progression à linear increase of drops per RTT■ dropping speed up is independent of queue accumulation speed !?!?

q fq_Codel combines CoDel with SFQq treats different traffic classes fairq gives starting flows a good head start

q Sojourn measurement does require blocking of the queue!q by contrast to queue length averaging

q Drop/Mark at the head of the queue not the tail


53

Conclusions

q Buffer size, sending rate and RTT interact closely for link utilization maximization

q Rule of thumb for single flows or synchronized flows

q Non-synchronized flows: required buffer only square-root of number of flows of rule-of-thumb size

q Sacrifice utilization, and even tiny, fixed-size buffers suffice!q Main relevance: makes it feasible to use optical buffers

q Large buffers should/can be controlled, but not simple!


54

References

1. G. Appenzeller, I. Keslassy, and N. McKeown, “Sizing router buffers”, in ACM SIGCOMM Computer Communication Review, 2004, vol. 34, no. 4, p. 281.

2. M. Enachescu, Y. Ganjali, A. Goel, N. McKeown, and T. Roughgarden, “Routers with very small buffers”, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications, 2006.

3. Nichols, Kathleen; Jacobson, Van, "Controlling Queue Delay". ACM Queue. August 2012.


advanced networking technologies - startseite tu ilmenau · advanced networking technologies...

Documents