outline introduction to tcp the single bottleneck case (deterministic) large network model...

Outline

• Introduction to TCP

• The single bottleneck case (Deterministic)

• Large network model (Stochastic)

• Matching TCP dynamics and MPEG video dynamics

• Input–output network models

The Dynamics of TCP

Stephan Bohacek with

Boris Rozovskii, Joao Hespanha, Katia Obraczka,

TCP/IP

• IP is the addressing system.• TCP is provides congestion control.• 95% of Internet data uses TCP• Web, email, FTP and telnet use TCP• Video and audio do not use TCP• In the future video is expected to use a larger portion

of the Internet bandwidth.

Why Study TCP?

• The stability of the Internet is vital for the economy. Thus, it is important that TCP is stable, reasonable fair and reasonable efficient.

• TCP Friendly

– Video and audio cannot use TCP.

– Video and audio must share network resources with TCP flows.

– Video/audio congestion control protocols must be compatible with TCP.

Network Congestion

Router 1

Router 2

Router 3

1Mbs

1Mbs

1Mbs

Networks provide “best effort” delivery. • If a router has a packet it will send it to its next hop. • If the outgoing link is occupied, then the router puts the packet in a queue. • If the queue is full it drops the packet. The router does not inform the sender of the packet that it has dropped the packet.Routers do not reroute packets over alternative paths.

queue

maximum sending rate

An Introduction to TCP• TCP is acknowledgment based. The sender sends

a packet of data. When the receiver receives the packet, it responds by sending an acknowledgment packet to the sender.

Sender receiver

An Introduction to TCP

• TCP is acknowledgment based. The sender sends a packet of data. When the receiver receives the packet, it responds by sending an acknowledgment packet to the sender.

• If the sender fails to receive an acknowledgment, it assumes that the packet has been dropped and decreases the sending rate.

• If the sender receives an acknowledgment, it assumes that the network is not congested and increases the sending rate.

• The congestion window, X, defines the maximum number unacknowledged packets.

• When the sender receives an acknowledgment it increases the congestion window by 1/[Xt].

• When a packet drop is detected, the sender divides the congestion window in half.

Some Details About TCP

Note: the congestion window is not the sending rate. sending rate = congestion window / roundtrip time.

Number of unacknowledgedpackets equals Cwnd. Send no more packets.

How the Congestion Window Increases

send pkt Asend pkt Bsend pkt Csend pkt D

X=4

ack A

ack B

ack C

ack Dsend pkt Esend pkt Fsend pkt Gsend pkt Hsend pkt F

X=4.25X=4.5X=4.75X=5

time

Packet arrives at receiver.Receiver sends an acknowledgment.

roundtriptime

sender receiver

TCPLinear increase when no drop is detectedMultiplicative decrease when a drop is detected

TCP:

cwnd(congestion window)

time

drop detected

The Single Bottleneck Case

router router

source 1 destination 1100Mbs 100Mbs

10Mb

Single bottleneck source 2

source n

destination 2

destination n

source i sends to destination i

The Single Bottleneck Case

•In the case of TCP on the single bottleneck topology with equal propagation delays, all flows synchronize exponentially fast. •Hence, in this setting, TCP is stable, fair. •Furthermore, the queue never empties, so the maximum throughput is achieved, thus TCP is efficient.

cong

est i

on w

indo

w

The Single Bottleneck CaseEvery time the queue fills, each flow gets exactly one drop.

2

3

Packets enter the queue exactly as fast as the leave.

1

Q Q

1

11 2 3

10ms

3 2 110ms

30ms

1

132

1

The Single Bottleneck CaseEvery roundtrip time, the congestion window increases by one.

1

Q Q

1

11 2 3

10ms

3 2 110ms

30ms

1

11

•But when the congestion window increases by one, two packets are send consecutively.•This second packet is causes the queue size to increase by one.•If the queue is full, when two packets are sent, then the second packet is dropped.

32

1

The Single Bottleneck Case

Let CWi(k) be the congestion window of the ith flow during the kth roundtrip time cycle.

drop a was thereif

2

drops no were thereif 11 kCW

kCWkCW i

i

i

drop a was thereif

2

drops no was thereif k111 ,

ji,

, kee

kCWkCWke jijiji

kke ji as 0,

General Topology(topology = network configuration)

Worst case • Each flow has a congestion window = 1• The network is totally utilized.• The next roundtrip time all flows increment there congestion window

to 2.• Half of the packets are dropped.• Congestion windows decrease back to 1.• No congestion collapse. 1/3 of all data gets through.

Typical case• Each flows congestion window is between 10 and 40• Each roundtrip time the utilization increases by between 1/10 and 1/40

Unequal propagation delays

• Flow S1D1 has a shorter roundtrip time, so it increases the its congestion window faster.

• When the queue in Q1 fills because S1 D1 is sending too fast S2 D2 may or may not get a drop.

• S2 D2 congestion window must be three times as large to send at the same rate as S1 D1.

S1

Q1 Q2S2

D1

D2

10ms

10ms

10ms

20ms

60ms

The Parking Lot Topology parking lot after a football game

exit

alternatingmerge

cars

The cars in the aisle further from the exit have to let the most other cars in front of them. This aisle moves at the exit rate 2-5

The Parking Lot Topology

router router router routersource destination

S1 D1 S2 Dn-1 DnSn

Competing TCP flows

Si Di have a small roundtrip time. These flows have fast dynamics and

have closely spaced drops.

source destination has large roundtrip time and takes a long time to

recover from a drop. Furthermore this experiences many drops.

Stochastic Models of TCP

cong

estio

n w

indo

wSingle bottleneck

case is deterministic

Time (seconds)

cong

estio

n w

indo

w

The multi-hop case can be

modeled probabilistically

50 100 150 200 250 300 350 400 450 5000

5

10

15

20

25

30

35

4045

50

Ingredients of our TCP Model

• Roundtrip time – a one dimensional mean reverting stochastic process.

• Packet loss – each packet is drop with probability g where g depends on roundtrip time and a hidden Markov process.

controls the rate at which the transition probability converges to the stationary distribution.

trrpr

trprr

trpt

,,2

,2

22

A Simple Stochastic Model of the Roundtrip Time

tttt dBRdtRdR

2

2

CIR Model (short-term interest rates)

0for on Distributi Stationary 1

ReRRp x

Note that the stationary distribution does not depend on .

forward equation

Define Rt to be the queuing delay. Roundtrip time – propagation delay

mean reverting mean = /

gamma distribution

Gamma Distribution

i - departure rate at ith queuei - arrival rate at ith queue

Heavy-traffic assumptionAccount for most of the delay

p (d) i exp(-i d) for i i i

qt1 qt

2 qtn-1 qt

nsource destination

queues

If i = j and i = j, then the roundtrip time is distributed according to the Gamma distribution with = n and = - .

Estimation of , , and (many approaches)

tttt dBRdtRdR

td

X

XdX

X

X

dP

dP T

t

tTt

t

t 0 2

2

0 2,0,0

,,

2

1

Tt

T

t

Tt

t

T

tT

T

t

Tt

t

dtXdtX

T

dXX

TdtX

X

dtX

dXX

T

002

00

0

0

1

11

1

1

likelihoodratio

maximum likelihoodestimates

note the change in representation

Estimation of , , and

tttt dBRdtRdR 2

2

tt

n

kkt

RR

Rpn

log1loglog

log 1

likelihood logn

1

1

Select a set of observations {Rtk} where tk << tk+1 Hence, {Rtk} are approximately i.i.d.

tt RR loglog'

log

tR

the maximum likelihood estimates

satisfy

mean of Rt

mean of log( Rt )

under the independence assumption

Estimating Diffusion

tttt dBRdtRdR

tttt dBRdtRdR 2

2

2

2

Estimation of , , and

qorder of kindfirst theoffunction Bessel modified

1/2: , ,: ,1/2:

2,,;|

2

2/12/

q

tt

ot

q

qvu

ot

I

qcRvecRuec

uvIu

vceRRp

The transition probability is known

Computationally difficult

n

kktkt

RRp0

1* ,,;|logminarg:

and are found as before and

0 2 4 6 80

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5Fitted Measured

0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25Fitted Measured

Roundtrip Time Quasi-stationary DistributionNight Midday

50 55 60 65 70 75 80 85 90 95 1000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

ns-2 simulation

RTT

P(R

TT

)

Observed RTT Dist Fitted RTT Dist

RTTRTT

P(R

TT

)

P(R

TT

)

Models for Packet Drop Probability

0 ,,,,, dRSKRSg tttt

ttt

tt

tt

ttt

ttt RScRbSaSaaRSg 112

210,,

Let St be the sending rate at time t.Let Rt be the roundtrip time experienced by a packet sent at time t.Let t be the congestion level at time t.Let g be the probability that a packet is dropped

general model

memoryless

2210, t

tt

tttt RaRaaRg

depends on roundtrip time only

Preliminary work indicates that, for reasonable sending rates, the drop probability mostly depends on roundtrip time.

Since drops are rare, it is difficult to collect data for slow sending rates.

* *

Determining the Conditional Drop Probability

1

111

||

|,||,

ktktktk

ktktktktkktktk

RRpRdP

RRpRRdPRRdp

ktktkt

ikt

n

ii

ktktktikt

n

ii

ktktktkt

ktktktktkktktktkktk

dRRRpRa

dRRRpRa

dRRRpRg

dRRRpRdPdRRRdpRdp

10

10

1

111

|

|

|

|||,|

system of linear equations

time.roundtrip for the iesprobabilitn transitio the- |

timeroundtrip a experiencepacket past given the

dropped ispacket next y that theprobabilit the- |

1

1

1

ktkt

kt

ktk

RRp

R

Rdp

observable

assume the congestion level, , is constant

We assume that given Rtk, dk is independent of Rtk-1

Histogram of Observed RTT Increments (real data)

Gaussian (RTT0=28)Queue empties slowly

Queue empties quickly

Agrees with queuing theory(Diffusion Approximation)

Modified Diffusion Approximation for a Single Queue

oo dtdDPdtdF ,|,,

otherwise 0

for ,,2

2

,tdd

t

mtdde

t

mtdddtdF o

o

md

o

om

t

mtdd

xo

o

dxet

mtdd

2

2

2

1

Observed Conditional Drop Probabilities

16 18 20 22 24 26 28 30-0.005

0

0.005

0.01

0.015

0.02

0.025

Roundtrip Time

observed conditional drop probability

night

12 noon

3pm6pm

P(D

rop

| Rou

ndtr

ip T

ime

)

9am

Drop – Event Probability

Drop Model Parameter Variation

1

5

10

15

x 10-3 Drop Model Parameter Variation

X 1

1

0

10

20x 10

-3

X 2

1-20

-10

0

x 10-3

X 3

1-10

-50

5

x 10-3

Monday Tuesday Wednesday Thursday Friday Saturday Sunday

X 4

g(Rt, t) = 0(t) + 1( ) T1(R) + 2( ) T2(R) + …

Basic Stochastic Model of TCP

tt

proptt dN

Xdt

TRdX

2

1

Nt is a Cox process with intensity n(Rt, Xt, t)

tttt dBRdtRdR 2

2

n(Rt, Xt, t) = Xt / Rt · g(Rt, Xt, t)

Sending Rate = Xt / Rt = Packets per RTT / Seconds per RTT = Packets / Second

Drop Probability = g(Rt, Xt, t)

Xt – congestion windowRt + Tprop – roundtrip time

drop probability

Stochastic Model of TCPwith fast recovery

tttt

tpropttWtWt

tt

propttWt

dBRdtRdR

dNTRdtdW

dNX

dtTR

dX

2

11

2

11

2

00

0

After a drop is detected, the congestion window, X, is halved and remains constant until the dropped packet is resent and acknowledged.

• A drop is detected at t=0, so W0=R0

• Wt > 0 for 0 < t < R0

• WRo = 0 and the congestion window

continues to increase until the next

drop occurs.

fast recovery

Stochastic Model of TCPwith fast recovery and time-out

tttt

ttpropttWtWt

tttt

propttWt

dBRdtRdR

dTdNTRdtdW

dTXdNX

dtTR

dX

2

11

12

11

2

0

0

When a time-out occurs, the congestion window is set to 1.

The Forward Equation

trxpxrntrwxpxrn

trwxpw

trwxpxTr

trwxrpr

trwxprr

trwxpt

propTrww

wwprop

,,0,22,21,,,,1

,,,1,,,11

,,,2

1,,,

2,,,

0

00

22

22

tdrdwdxPrwx

trwxp

CRBWAXPtCBAP ttt

,,,,,,Let

,,:,,,Let 2

No Time-outsStationary environment (model parameters are constant)

Assumptions:

Midday RTT and Drop Data

0 5 10 15 20 25 300

0.02

0.04

0.06

0.08

0.1

0.12Marginal Density p(X)

X - Congestion Window

p

15 20 25 300

5

10

15

20

25

30

X -

Co

ng

est

ion

Win

do

w

R - Roundtrip Time

Marginal Density p(X,R)

= 4, = 0.85, = 0.5

0 2 4 6 8 10 12 140

0.05

0.1

0.15

0.2

0.25

RTT

Pro

b(R

TT

)

Fitted Measured

0 2 4 6 8 10 12 140

0.005

0.01

0.015

0.02

0.025

0.03

RTT

drop

pro

b

Fitted Measured

Drop Probability

Roundtrip Time Stationary Distribution

ns-2 (network simulator)topology

Source 1 2 3 4 5 Destination

A B DC

S T U V

Competing TCP Sources

ns-2 Simulation

20 30 40 50 60 700

5

10

15

20

25

30

X -

Con

gest

ion

Win

dow

R - Roundtrip Time

Marginal Density p(X,R)

= 2, = 0.17, = 0.34

0 5 10 15 20 25 30 35 40 45 500

0.01

0.02

0.03

0.04

0.05

0.06

0.07

RTT

P(R

TT

)

Fitted Measured

50 60 70 80 90 100 110-0.005

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

RTT

drop

pro

b

Drop Probability

Roundtrip Time Stationary Distribution

0 10 20 30 40 500

0.01

0.02

0.03

0.04

0.05

0.06

0.07current Simulated

Marginal Density p(X)

The Dependence on Drop Probability

0 5 10 15 20 25 300

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

X - Congestion Window

p

Drop Prob Scale = 0.75Drop Prob Scale = 1.0 Drop Prob Scale = 1.25Drop Prob Scale = 1.5 Drop Prob Scale = 1.75Drop Prob Scale = 2.0

10-3

10-2

10-1

100

101

102

The TCP Friendly Rule

p

Mea

n C

onge

stio

n W

indo

w

Drops are modeled as a Cox process with intensity

n(Rt, Xt, t) = Xt / Rt · g(Rt, Xt, t) • Scale

The TCP Friendly Rule

E(X) = C / E(p)1/2

Dependence on

tttt dBRdtRdR 2

2

15 20 25 300

5

10

15

20

25

X -

Co

ng

est

ion

Win

do

w

R - Roundtrip Time

Marginal Density p(X,R)

= 0.07

Recall that controls the rate at which the transition probability converges to the stationary distribution.

0 10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

X - CWnd

P(X

)

CWnd density vs.

= 0.015 = 0.15 = 0.25 = 0.5 = 0.75 = 1.5

10-3

10-2

10-1

100

101

102

103

Loss Probability

The TCP Friendly Rule?

RT

T

Se

nd

ing

Ra

te =

Co

ng

es

tion

Win

do

w

When is it safe to send video?

Objective: We are want to transmit a movie over the Internet.

Problem: We don’t want to overly interfere with TCP file transfers.

Solution : Match the dynamics: send the video at the same rate that TCP would send data. (This is the current dogma. But it may not be the best approach.)

work in progress

MPEG 1 Model

MPEG has three types of frames• I Frame – No time correlation is used. The

compression is the same as image compression.• P Frame – Some time correlation is used.• B Frame – Extensive time correlation between

frames is used.

I B B P B B P B B P B B I B B P

GOP – Group of Pictures

MPEG 1 Model

Jk+1 = Jk with probability qJk+1 = Yk with probability 1 – q and Yk is i.i.d. lognormal.

The J component is constant during a scene constant.The scene changes with probability q

The size of the P and B frames are i.i.d. lognormal.

The size of I frames is decomposed into two: Ik = Jk + Lk

The L component is i.i.d. and is normally distributed.

MPEG 1 Modelsmoothing

In order to avoid bursts, the sending rate is “smoothed” in two ways•Instead of breaking the video into frame, it is broken into GOPs.•The sending rate is smoothed (low-pass filtered).

Vk+1 = Vk + (1- )(Jk + Lk + P1k + P2k + P3k

+ B1k + B2k + B3k + B4k + B5k + B6k + B7k + B8k)Jk+1 = Jk with probability qJk+1 = Yk with probability 1 – q and Yk is i.i.d. lognormal.

with 0 < 1.

new GOP framessmoother

Two Dynamical Systems

tt

proptt dN

Xdt

TRdX

2

1

tttt dBRdtRdR 2

2

Vk+1 = Vk + (1- )(Jk + Lk + P1k + P2k + P3k

+ B1k + B2k + B3k + B4k + B5k + B6k + B7k + B8k)Jk+1 = Jk with probability qJk+1 = Yk with probability 1 – q and Yk is i.i.d. lognormal

TCP

MPEG Video

0 0.2 0.4 0.6 0.8 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Percentile

(P(M

PE

G >

TC

P P

erce

ntile

)

95 100 105 1100

0.5

1

1.5

2

x 106PDF of both MPEG and TCP T

prop = 90 = 0.95

R - RTT(ms)

X -

CW

nd

Comparing TCP and MPEG distributions

The Wizard of OZ with = 0.95and

LA to San Jose during the day,but varying Tprop = 90ms.

P(MPEG > TCP) = 0.42

90 95 100 105 110 1150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

R

P(M

PE

G >

TC

P |

RT

T)

P( MPEG > TCP | RTT )

P(R)

The Wizard of OZ with = 0.95and

LA to San Jose during the day,but varying Tprop = 70ms.

75 80 85 900

0.5

1

1.5

2

2.5

3x 10

6PDF of both MPEG and TCP Tprop

= 70 = 0.95

R - RTT(ms)

X -

CW

nd

P(MPEG > TCP) = 0.24

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

TCP Percentile

P(M

PE

G >

TC

P P

erce

ntile

)

70 75 80 85 90 950

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

R

P(M

PE

G >

TC

P |

R)

P( MPEG > TCP | RTT )

P(R)

Comparing TCP and MPEG distributions

Exit Times for TCP

16 18 20 22 24 26 28 300

5

10

15

20

25

30

R - RTT(ms)

X -

CW

nd Q

50th Percentile

How long does a TCP flow send at a rate that is higher than 50% of other TCP flows?

QRXtRXT tt ,:inf:, 00

rxTErxf ,, where

Question:

Answer:

Exit Times for TCP

tt

proptt dN

Xdt

TRdX

2

1

tttt dBRdtRdR

2

2Basic model

xrnrxfrx

frxfxr

rxfr

rrxfr

rrxLf

,,,2

,1

,2

1,:,

2

22

,1Lf Qrxrxf ,for 0,

Define

Let

Then rxTErxf ,,

Define QRXtRXT tt ,:inf:, 00

Exit Times of a MPEG flow from the TCP’s 50% percentile sending rate

16 18 20 22 24 26 28 300

0.5

1

1.5

2x 107

R - RTT(ms)Sen

ding

Rat

e (b

its p

er s

econ

d)

Q

50th Percentile

There are two ways to leave Q1. R decreases2. Video sending rate

decreases.

Exit Times of a MPEG flow from the TCP’s 50% percentile

Qrivtrivf ,,for 0,,,

then .,,,,,, tRIVftRIVTE tttttt

srivfs

srivfLR ,,,,,,1

Qriv ,, 0,,,0,,,,,|, 11 dudlTrivfrlufiIvVlIuVP nnnnMPEG

R Triivvf ,0,,,: 22121 1r

Qriv ,,

trivft

trivfr

rtrivf

rrtrivfLR ,,,,,,

2,,,,,,

2

22

Ts ,0

where

Let

Exit Times of a MPEG flow from the TCP’s 50% percentile

40 45 50 55 60 65 70 75 800

200

400

600

800

1000

1200

1400

Tprop

E(T

)

MPEG =0.05MPEG =0.5 MPEG =0.95TCP

40 45 50 55 60 65 70 75 800

0.02

0.04

0.06

0.08

0.1

0.12

Tprop

P(M

PE

G >

TC

P 5

0th P

erc

en

tile

)

MPEG =0.05MPEG =0.5 MPEG =0.95

For Tprop between 60 and 80, there is a benefit to smoothing.

For Tprop > 80, the MPEG flow may interfere with TCP flows

Variation of loss probability with sending rate

Internet

sendingrate

lossprobability

Initial test: stimulate with a periodic square wave

20s

One packet every 10ms

One packet every 40ms

sendingrate

Variation of loss probability with sending rate

S D

1 2 3 4

5 6 7 8

ns configuration

Flow under test S D

Paths for competing flows:1 5, 1 6, 1 7, 1 82 6, 2 7, 2 83 7, 3 8, 4 8

For each path there was either one, three, or five TCP flows.

0 5 10 15 20 25 30 35 402

4

6

8

10

12x 10-3

1 TCP flow/connection

0 5 10 15 20 25 30 35 400.025

0.03

0.035

0.04

0.045

3 TCP flows/connection

Loss

Pro

babi

lity

0 5 10 15 20 25 30 35 400.05

0.06

0.07

0.08

5 TCP flows/connection

Time (sec)

Variation of loss probability with sending rate

ns simulations

The step response is similar to a step response of a linear system with the damping decreasing as the congestion increases.

Simulation time = 27days

Variation of loss probability with sending rate

0 5 10 15 20 25 30 35 400.028

0.0285

0.029

0.0295

0.03

0.0305

Time (seconds)

Los

s P

rob

ab

ility

LA (USC) LA (Culver City)

It appears that sending rate and loss probability are not related!

experiment duration = 1 week

Future Directions

•The Internet as an input/output system.So far we assumed that the roundtrip time and drop probability is independent of our sending rate. What is the effect of sending data too fast? If the dependence of the drop probability on sending rate is known, then the behavior of competing TCP flows can be determined and the effect of the sending rate can be determined.

•Time-varying model parametersThe work so far assumes that the parameters are independent of time. The effect of their variation should be accounted for.

the end