utility, fairness, tcp/ip steven low cs/ee netlab.caltech.edu feb 2004
TRANSCRIPT
Acknowledgments Caltech
Bunn, Choe, Doyle, Jin, Newman, Ravot, Singh, J. Wang, Wei
UCLA Paganini, Z. Wang
CERN Martin
SLAC Cottrell
Internet2 Almes, Shalunov
Cisco Aiken, Doraiswami, Yip
Level(3) Fernes
LANL Wu
Protocol Decomposition
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …Topology, power control Maximize capacity
Shortest-path routing Minimize path costs
Duality model (Kelly, Low et al)
Maximize aggregate utility
HOT (Doyle et al)
Minimize user response time
Heavy-tailed file sizes
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
TCP/AQM
Congestion control is a distributed asynchronous algorithm to share bandwidth
It has two components TCP: adapts sending rate (window) to congestion AQM: adjusts & feeds back congestion information
They form a distributed feedback control system Equilibrium & stability depends on both TCP and AQM And on delay, capacity, routing, #connections
pl(t)
xi(t)TCP: Reno Vegas
AQM: DropTail RED REM/PI AVQ
Network model
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
liRlif link uses source if 1
liR lib link uses source if 1
Network model
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
lieR lis
lif link uses source if
lieR lislib link uses source if E
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
Methodology
Protocol (Reno, Vegas, RED, REM/PI…)
Equilibrium Performance
Throughput, loss, delay
Fairness
Dynamics Local stability Global stability
))( ),(( )1(
))( ),(( )1(
txtpGtp
txtpFtx
Network model
F1
FN
G1
GL
R
RT
TCP Network AQM
x y
q p
))( ),(( )1(
))( ),(( )1(
tRxtpGtp
txtpRFtx T
Reno, Vegas
DT, RED, …
liRli link uses source if 1 IP routing
Duality model
))( ),(( )1(
))( ),(( )1(
txtpGtp
txtpFtx
Primal-dual algorithm:
)( )( max )( min
subject to )( max
00
0
:Dual
:Primal
ll
ll
sss
xp
sss
x
xcpxUpD
cRxxU
s
s
Duality Model of TCP
))( ),(( )1(
))( ),(( )1(
txtpGtp
txtpFtx
Primal-dual algorithm:
Reno, Vegas
DropTail, RED, REM
Source algorithm iterates on rates Link algorithm iterates on prices With different utility functions
Summary: duality model
cRx
xUs
ssxs
subject to
)( max0
Flow control problem (Kelly, Malloo, Tan 98)
TCP/AQM Maximize utility with different utility functions
Primal-dual algorithm
))( ),(( )1(
))( ),(( )1(
tRxtpGtp
txtpRFtx T
Reno,
VegasDropTail, RED, REM
Result (L 00): (x*,p*) primal-dual optimal iff 0 ifequality with ** lll pcy
Example utility functions
1 log
1 )1( :General
log : Vegas
32log
1 :2-Reno
3/2tan23
:1-Reno
11
1
i
i
ii
ii
ii
i
iii
x
x
x
Tx
Tx
T
TxT
/
FAST, STCP
(Mo, Walrand 00)
Methodology
Protocol (Reno, Vegas, RED, REM/PI…)
Equilibrium Performance
Throughput, loss, delay
Fairness
Dynamics Local stability Global stability
))( ),(( )1(
))( ),(( )1(
txtpGtp
txtpFtx
222
2
3
33
)1(4
)1 )(
2
-(Nc
N
c
Theorem (Low et al, Infocom’02) Reno/RED is locally stable if
Stability: Reno/RED
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
TCP: Small Small c Large N
RED: Small Large delay
Stability: scalable control
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
lll
l ctyc
tp )(1
)()(
)(tq
mii
iii
i
extx
Theorem (Paganini, Doyle, L, CDC’01) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology
Linear Stability: scalable control
Theorem (Paganini, Doyle, Low, CDC’01) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology
2
1
( )fy R s x ( )Tbq R s p
0i ii i
i
xx q
M
1l l
l
p yc
Globally stable in presence of delay?
Stability: Stabilized Vegas
)()(1)( tan)(
1 )()(1-
2tqtt
tTx iid
tqtxi ii
ii
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
lll
l ctyc
tp )(1
)(
Theorem (Choe & L, Infocom’03) Provided R is full rank, feedback loop is locally stable if
),( max aTx ii
Stability: Stabilized Vegas
)()(1)( tan)(
1 )()(1-
2tqtt
tTx iid
tqtxi ii
ii
F1
FN
G1
GL
Rf(s)
Rb’(s)
TCP Network AQM
x y
q p
lll
l ctyc
tp )(1
)(
Application Stabilized TCP with current routers Queueing delay as congestion measure has right scaling Incremental deployment with ECN
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
Reno TCP
Packet level Designed and implemented first
Flow level Understood afterwards
Flow level dynamics determines Equilibrium: performance, fairness Stability
Design flow level equilibrium & stability Implement flow level goals at packet level
Packet level
ACK: W W + 1/W
Loss: W W – 0.5W
Reno AIMD(1, 0.5)
ACK: W W + a(w)/W
Loss: W W – b(w)W
HSTCP AIMD(a(w), b(w))
ACK: W W + 0.01
Loss: W W – 0.125W
STCP MIMD(a, b)
RTT
baseRTT W W :RTT FAST
Flow level: Reno, HSTCP, STCP, FAST
Similar flow level equilibrium
= 1.225 (Reno), 0.120 (HSTCP), 0.075 (STCP)
pkts/sec
Flow level: Reno, HSTCP, STCP, FAST
Different gain and utility Ui
They determine equilibrium and stability
Different congestion measure pi Loss probability (Reno, HSTCP, STCP) Queueing delay (Vegas, FAST)
Common flow level dynamics
windowadjustment
controlgain
flow levelgoal=
Implementation strategy
Common flow level dynamics
windowadjustment
controlgain
flow levelgoal=
Small adjustment when close, large far away Need to estimate how far current state is wrt target Scalable
Window adjustment independent of pi Depends only on current window Difficult to scale
FAST TCP
Theorem (Jin, Wei, L ‘03) In absence of delay at a single link Mapping from w(t) to w(t+1) is contraction Global exponential convergence Full utilization after finite time Utility function: i log xi (proportional fairness)
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
FAST TCPutil: 95%
Linux TCPutil: 19%
1Gbps path; 180 ms RTT; 1 flowJin, Wei, Ravot, etc (Caltech, Nov 02)
Dynamic sharing: 3 flowsFAST Linux
Dynamic sharing on Dummynet capacity = 800Mbps delay=120ms 3 flows iperf throughput Linux 2.4.x (HSTCP: UCL)
FAST Linux
throughput
loss
queue
STCPHSTCP
Dynamic sharing on Dummynet capacity = 800Mbps delay=120ms 14 flows iperf throughput Linux 2.4.x (HSTCP: UCL)
30min
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
Network model
F1
FN
G1
GL
R
RT
TCP Network AQM
x y
q p
))( ),(( )1(
))( ),(( )1(
tRxtpGtp
txtpRFtx T
Reno, Vegas
DT, RED, …
liRli link uses source if 1 IP routing
Motivation
ll
li l
lliR
iiixp
iii
xR
cppRxxU
cRxxU
ii
max)( max min
subject to )( maxmax
00
0
:Dual
:Primal
Motivation
Can TCP/IP maximize utility?
ll
li l
lliR
iiixp
iii
xR
cppRxxU
cRxxU
ii
max)( max min
subject to )( maxmax
00
0
:Dual
:Primal
Shortest path routing!
TCP-AQM/IP
Theorem (Wang, et al 03)
Primal problem is NP-hard
Ai
iAi
i cc
Proof Reduce integer partition to primal problem
Given: integers {c1, …, cn}Find: set A s.t.
TCP-AQM/IP
Theorem (Wang, et al 03)
Primal problem is NP-hard
Achievable utility of TCP/IP?
Stability? Duality gap?
Conclusion: Inevitable tradeoff between
achievable utility routing stability
Ring networkdestination
r
Single destination Instant convergence of
TCP/IP Shortest path routing
Link cost = pl(t) + dl
price static
TCP/AQM
IPr(0)
pl(0)
r(1)
pl(1)
… r(t), r(t+1) , …
routing
Ring networkdestination
r
TCP/AQM
IPr(0)
pl(0)
r(1)
pl(1)
… r(t), r(t+1) , …
Stability: r ?
Utility: V ?r* : optimal routing
V* : max utility
Ring networkdestination
r
link cost = pl(t) + dl
0
0||*
*
VV
rr
Theorem (Infocom 2003)
Solve primal problem asymptoticallyas
Stability: r ?
Utility: V ?
Ring networkdestination
r
link cost = pl(t) + dl
Theorem (Infocom 2003)
large: globally unstable small: globally stable medium: depends on r(0)
Stability: r ?
Utility: V ?
General network
Conclusion: Inevitable tradeoff between
achievable utility routing stability
random graph20 nodes, 200 links Achievable
utility
Outline
Network model FAST TCP
Equilibrium Stability Implementation Experiments
TCP/IP interaction Fairness-efficiency
Applications
TCP/AQM
IP
Transmission
WWW, Email, Napster, FTP, …
Ethernet, ATM, POS, WDM, …
TCP/AQM: duality model Flow control problem (Kelly, Malloo, Tan 98)
TCP/AQM Maximize utility with different utility functions (L 00): (x*,p*) primal-dual optimal iff
Primal-dual algorithm
Reno, Vegas, FAST
DT, RED, REM/PI, AVQ
Fairness
maximum throughput
proportional fairness
min delay fairness infinity maxmin
fairness
(Mo, Walrand 00)
Example 1
Conjecture T() is nonincreasing
i.e. a fair allocation is always inefficient
1/(L+1)
L/(L+1)
1/2
1/2
maxmin proportional
Intuition
“The fundamental conflict between achieving flow fairness and maximizing overall system throughput….. The basic issue is thus the trade-off between these two conflicting criteria.”
Luo,etc.(2003), ACM MONET
Results
Theorem: Necessary & sufficient condition for general network
Corollary 1: true if N(R)=1
1/(L+1)
L/(L+1)
1/2
1/2
maxmin proportional
Results
Theorem: Necessary & sufficient condition for general network
Corollary 2: true if N(R)=2 2 long flows pass through same# links
Counter-example There exists a network such that
dT/d > 0 for almost all >0 Intuition
Large favors expensive flows Long flows may not be expensive
Maxmin may be more efficient than proportional fairness