tcp nice: a mechanism for background transfers

April 21, 2023 Department of Computer Sciences, UT Austin

TCP Nice: A Mechanism for Background Transfers

Arun Venkataramani, Ravi Kokku, Mike Dahlin

Laboratory of Advanced Systems Research(LASR)

Computer Sciences, University of Texas at Austin


What are background transfers?

Data that humans are not waiting for Non-deadline-critical Unlimited demand

Examples- Prefetched traffic on the Web

- File system backup

- Large-scale data distribution services

- Background software updates

- Media file sharing


Desired Properties

Utilization of spare network capacity

No interference with regular transfers- Self-interference

- applications hurt their own performance

- Cross-interference- applications hurt other applications’ performance


Goal: Self-tuning protocol

Many systems use magic numbers- Prefetch threshold [DuChamp99, Mogul96, …]

- Rate limiting [Tivoli01, Crovella98, …]

- Off-peak hours [Dykes01, many backup systems, …]

Limitations of manual tuning- Difficult to balance concerns

- Proper balance varies over time

Burdens application developers- Complicates application design

- Increases risk of deployment

- Reduces benefits of deployment


TCP Nice

Goal: abstraction of free infinite bandwidth Applications say what they want- OS manages resources and scheduling

Self tuning transport layer- Reduces risk of interference with FG traffic

- Significant utilization of spare capacity by BG

- Simplifies application design


Outline

Motivation

How Nice works

Evaluation

Case studies

Conclusions


Why change TCP?

TCP does network resource management- Need flow prioritization

Alternative: router prioritization+ More responsive, simple one bit priority

- Hard to deploy

Question: - Can end-to-end congestion control achieve non-

interference and utilization?


Traditional TCP Reno

Adjusts congestion window (cwnd) - Competes for “fair share” of BW (=cwnd/RTT)

Packet losses signal congestion - Additive Increase

- no losses Increase cwnd by one packet per RTT

- Multiplicative Decrease:- one packet loss (triple duplicate ack) halve cwnd- multi-packet loss set cwnd = 1

Problem: signal comes after damage done


TCP Nice

Proactively detects congestion

Uses increasing RTT as congestion signal- Congestion incr. queue lengths incr. RTT

Aggressive responsiveness to congestion

Only modifies sender-side congestion control- Receiver and network unchanged

- TCP friendly


TCP Vegas

Early congestion detection using RTTs

Additive decrease on early congestion

Restrict number of packets maintained by each flow at bottleneck router

Small queues, yet good utilization


TCP Nice

Basic algorithm- 1. Early Detection thresh. queue length incr. in RTT

- 2. Multiplicative decrease on early congestion

- 3. Maintain small queues like Vegas

- 4. Allow cwnd < 1.0

per-ack operation:

- if(curRTT > minRTT + threshold*(maxRTT – minRTT))

numCong++; per-round operation:

- if(numCong > f.W) W W/2 else { … Vegas congestion control }


Nice: the works

Non-interference getting out of the way in time Utilization maintaining a small queue

pkts

minRTT = maxRTT =

B

t.BAdd * Add + Mul +

Reno

Nice

Add * Add * Add *

Mul +

Mul +


Evaluation of Nice

Analysis

Simulation micro-benchmarks

WAN measurements

Case studies


Theoretical Analysis

Prove small bound on interference

Main result – interference decreases exponentially with bottleneck queue capacity, independent of the number of Nice flows

Guided algorithmic design- All 4 aspects of protocol essential

Unrealistic model- Fluid approximation

- Synchronous packet drop


TCP Nice Evaluation

Micro-benchmarks (ns simulation)- test interference under more realistic assumptions

- Near optimal interference- 10-100X less than standard TCP

- determine if Nice can get useful throughput- 50-80% of spare capacity

- test under stressful network conditions

- test under restricted workloads, topologies


TCP Nice Evaluation

Measure prototype on WAN+ test interference in real world

+ determine if throughput is available to Nice

- limited ability to control system

Results- Nice causes minimal interference

- standard TCP – significant interference

- Significant spare BW throughout the day


NS Simulations

Background traffic: - Long-lived flows

- Nice, rate limiting, Reno, Vegas, Vegas-0

- Ideal: Router prioritization

Foreground traffic: Squid proxy trace- Reno, Vegas, RED

- On/Off UDP

Single bottleneck topology Parameters- spare capacity, number of Nice flows, threshold

Metric- average document transfer latency


Network Conditions

0.1

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Spare Capacity

Reno

Vegas

V0

Nice

Router Prio

Nice causes low interference even when there isn’t much spare capacity.


Scalability

0.1

1

10

100

1e3

1 10 100

Do

cum

ent

Lat

ency

(se

c)

Num BG flows

Vegas

V0

Nice

Router Prio

Reno

W < 1 allows Nice to scale to any number of background flows


Utilization

0

2e4

4e4

6e4

8e4

1 10 100

BG

Th

rou

gh

pu

t (K

B)

Num BG flows

Router Prio

Vegas

V0

Reno

Nice

Nice utilizes 50-80% of spare capacity w/o stealing any bandwidth from FG


Case study 1 : Data Distribution

Tivoli Data Exchange system

Complexity- 1000s of clients

- different kinds of networks – phone line, cable modem, T1

Problem- tuning data sending rate to reduce interference and

maximize utilization

Current solution- manual setting for each set of clients


Case study 1 : Data Distribution

50

100

150

200

250

300

350

20 40 60 80 100 120 140 160 180

Pin

g la

ten

cy

(in

ms

)

Completion time(in seconds)

Manual tuning

Nice point

Austinto

London


Case study 2: Prefetch-Nice

Novel non-intrusive prefetching system [USITS’03]

Eliminates network interference using TCP Nice Eliminates server interference using simple monitor

Readily deployable- no modifications to HTTP or the network

- JavaScript based

Self-tuning architecture, end-to-end interference elimination


Case study 2: Prefetch-Nice

A self-tuning architecture gives optimal performance for any setting

Hand-tuned threshold

Aggressive threshold+

Self-tuning protocol

Aggressiveness

Res

po

nse

tim

e


Prefetching Case Study

Cable modem

0

20

40

60

80

100

120

None Cons Aggr

Prefetching mode

Res

pons

e ti

me

(sec

)

RenoNice


Prefetching case study

Phone line

0500

1000150020002500300035004000

None Cons Aggr

Prefetching mode

Res

pons

e ti

me

(sec

)

RenoNice


Conclusions

End-to-end strategy can implement simple priorities

Enough usable spare bandwidth out there that can be Nicely harnessed

Nice makes application design easy


TCP Nice Evaluation Key Results

Analytical- interference bounded by factor that falls exponentially with

bottleneck queue size independent of no. of Nice flows

- all 3 aspects of algorithm fundamentally important

Microbenchmarks (ns simulation)- near optimal interference

- interference up to 10x less than standard TCP

- TCP-Nice reaps substantial fraction of spare BW

Measure prototype on WAN- TCP Nice: Minimal interference

- standard TCP: Significant interference

- significant spare BW available throughout the day


Freedom from thresholds

Dependence on threshold is weak- small threshold )low interference

0

4

8

12

16

20

0 0.2 0.4 0.6 0.8 1

Do

cum

ent

Lat

ency

(se

c)

Threshold

I » O(e-B/m.(1-t))


Manual tuning

Prefetching – attractive, restraint necessary- prefetch bandwidth limit [DuChamp99, Mogul96]

- threshold probability of access [ Venk’01]

- pace packets by guessing user’s idle time [Cro’98]

- prefetching during off-peak hours [Dykes’01]

Data distribution / Mirroring- tune data sending rates on a per-user basis

- Windows XP – Background (BITS)- rate throttling approach- different rates different priority levels


Goal: Self-tuning protocol

Hand-tuning may not work- Difficult to balance concerns

- Proper balance varies over time

Burdens application developers- Complicates application design

- Increases risk of deployment

- Reduces benefits of deployment


Nice: the works

Nested congestion triggers: - Nice < Vegas < Reno

pkts

minRTT = maxRTT =

B

t.BAdd * Add + Mul +

Vegas

Nice

Add * Add + Add +

Reno Add * Add * Add *


TCP Vegas

E = W / minRTT

A = W /observedRTT

Diff = (E – A)/minRTT

W*

DiffActual

Expected

1/minRTT

Window

Th

rou

ghp

ut

if(Diff < / minRTT)

W Ã W + 1

else if(Diff > / minRTT)

W Ã W - 1

Additive decrease when packets queue


Nice features

Small , not good enough. Not even zero! Need to detect absolute queue length Aggressive backoff necessary Protocol must scale to large number of flows

Solution

- Detection ! ‘threshold’ queue build up

- Avoidance ! Multiplicative backoff- Scalability ! window is allowed to drop

below 1


Internet measurements

Nice vs Reno: cable modem

01020304050607080

FG BG2F

G

FG/BG

9FG

FG/8BG

Tra

nsfe

r ti

mes

(se

c)

BGFG

Long Nice and Reno flows Interference metric: flow completion time




Extensive measurements over different kinds of networks- Phone line- Cable modem - Transcontinental link (Austin to London)- University network (Abilene: UT Austin to Delaware)



Nice vs Reno: Austin to London

0

20

40

60

80

100

FG BG2F

G

FG/BG

9FG

FG/8BG

Tra

nsfe

r ti

mes

(se

c)

BGFG




Nice vs Reno: phone line

050

100150200250300350400

FG BG 2FG FG/BG

Tra

nsfe

r ti

mes

(se

c)

BGFG




Useful spare capacity throughout the day

0

5

10

15

20

25

May 10 May 11 May 12 May 13 May 14 May 15

Res

pon

se T

ime

(sec

)

Time of day

Austin to London

Reno

Nice



Useful spare capacity throughout the day

0

50

100

150

200

250

300

May 13 May 14 May 15 May 16

Res

pon

se T

ime

(sec

)

Time of day

Cable modem

Reno

Nice


NS Simulations - Red

0

10000

20000

30000

40000

50000

60000

70000

80000

1 10 100

BG T

hrou

ghpu

t (K

B)

Num BG flows

Router PrioNice-0.03

NiceVegasReno


NS Simulations - Red

0.1

1

10

100

1 10 100

Doc

umen

t Lat

ency

(sec

)

Spare Capacity

Router PrioNice-0.03

Nice

Reno

Vegas


Analytic Evaluation

All three aspects of protocol essential Main result – interference decreases

exponentially with bottleneck queue capacity irrespective of the number of Nice flows

- fluid approximation model

- long-lived Nice and Reno flows

- queue capacity >> number of foreground flows

tcp nice: a mechanism for background transfers

Documents

congestion controlreceiver

congestion signalcongestion

endtoend congestion

vegas congestion control

early detection

cwndmultipacket loss

bottleneck queue capacity

backup systems