The Internet Congestion Manager

Hari BalakrishnanMIT Laboratory for Computer

Sciencehttp://wind.lcs.mit.edu/

CM teamDave Andersen, Deepak Bansal, Dorothy

Curtis, Bodhi Priyantha, Srinivasan Seshan

Motivation

• End-systems implement many functions– Reliability– In-order delivery– Demultiplexing– Message boundaries– Connection abstraction– Congestion control

Of these, congestion control MUST be done for all communications

Of these, congestion control MUST be done for all communications

State of Congestion Control

“Multimedia Transmissions Drive Net Toward Gridlock”

Sara RobinsonNew York Times8/23/99

Two Issues

• Adaptation– Enable applications to learn about

network conditions and adapt

• Concurrent flows– Efficiently multiplex streams and share

paths

Congestion Manager (CM): A new end-system architecture for congestion management

Congestion Manager (CM): A new end-system architecture for congestion management

Adaptation

• Available bandwidth• Round-trip times• Loss information

Application

Transport

Network

Link

CM exports a simple adaptation APICM exports a simple adaptation API

• Little information transferred across layers to applications• More & more non-TCP applications

Physical networks

Concurrent Flows

• Overload causes congestion

CM abstracts all congestion-related information into one place

CM abstracts all congestion-related information into one place

• Concurrent streams compete, causing aggressive network behavior!

Additive-Increase &Multiplicative-Decrease (AIMD)

Win

dow

Time

“Slow start”

What Can We Do?

• Inside-the-network smarts– RED and variants– Fair queueing and variants– Differentiated services– Integrated services

• End-system architecture– Leverage a decade of experience– Interact well with router mechanisms

The Big Picture

IP

HTTP Video1

TCP1 TCP2 UDP

Audio Video2

All congestion management tasks performed in CMApps learn and adapt using API

All congestion management tasks performed in CMApps learn and adapt using API

CongestionManager

Per-”macroflow”statistics

(cwnd,rtt,…)API

The CM ArchitectureApplication

(TCP, http, video, etc.)

Prober

CongestionController

Scheduler

Responder

Congestion Detector

Sender Receiver

CM protocol

API

Application

API

Hints Dispatch

Problems

• How does CM control when and whose transmissions occur?– Keep application in control of what to

send

• How does CM discover network state?– What information is shared?– What is the granularity of sharing?

Key issues: API and information sharingKey issues: API and information sharing

Transmission API

• Buffered-send– Does not allow apps to “pull back”

dataApp

CM

IP

cm_send( ,dst)

Packet queueto dst

Rate change

Lesson: move buffering into the applicationLesson: move buffering into the application

Pull out and re-encode (hard!)

Transmission API (cont.)

• Callback-based send

cm_request()cmapp_send() based on

allowed rate

App

CM

IP

Enables apps to adapt “at the last instant”Enables apps to adapt “at the last instant”

Schedule requests,not packets

send( )

cm_notify(nsent)

Transmission API (cont.)• Request API works for asynchronous sources

wait for (some_events) {

get_data(); /* e.g., from a file, image capture, etc. */

send(); /* call cm_request() and send on callback */

}

• But what about synchronous sources (e.g., audio with constant sampling rate)?do_every_t_ms { /* timer loop */

get_data();

send(); /* oops, waiting for send callback wrecks timing */

}

• Solution: rate-change callback cmapp_update(rate, srtt)

Feedback about Network State

• Monitoring successes and losses– Application hints– Probing system

• Notification API (application hints)– Application calls cm_update(nsent,

nrecd, congestion indicator, rtt)

IP header

Probing System

• Receiver modifications necessary– Support for separate CM header– Uses sequence number to detect losses– Sender can request count of packets received

• Receiver modifications detected/negotiated via handshake– For incremental deployment

IP payload CM headerIP payload

Congestion Controller

• Responsible for deciding when to send a packet

• Window-based AIMD with traffic shaping (“TCP-friendly”)

• Exponential aging when feedback low– Halve window every RTT (use min RTT)

• Can plug in other algorithms– Selected on a “macroflow” granularity

CM Scheduler

• Responsible for deciding who should send a packet

• Currently hierarchical round robin• Hints from application or receiver

– Used to prioritize flows

• Plug in other algorithms– Selected on a “macroflow” granularity– Prioritization interface may be different

How will apps use the CM?

• TCP– Asynchronous callback API

• HTTP– Rate callbacks to decide content

• Congestion controlled UDP– Buffered API

• Conferencing applications– Synchronous API for real-time streams– Combination of others for other streams

TCP/CM

TCP(reliability,

conn. mgmt.)

CM

1. cm_open

2. cm_request

3. cmapp_send 6. cm_updateon ACK

7. cm_close

Steps 2-6 occur multiple times

IP

5. cm_notify

4. send

CM Web Performance

With CM

CM greatly improvespredictabilityand consistency

TCP Newreno

Time (s)

Seq

uen

ce n

um

ber

Layered Streaming Audio

Audio adapts to available bandwidthCombination of TCP & Audio compete equally with normal TCP

0

50

100

150

200

250

300

350

400

450

0 5 10 15 20 25

Time (s)

Seq

uen

ce n

um

ber

Competing TCP

TCP/CM

Audio/CM

Status & Future

• Implementation in progress (Linux)– TCP/CM + Web server– Layered audio server– Congestion-controlled UDP sockets– Modify clients to reflect user preferences

• Handling diffserv and traffic splitters• Aggregation: per-host, per-subnet,…• Feedback frequency analysis• Aging of congestion information

Deployment Issues

• In the long-term, change senders and receivers to use the CM

• In the short-term, change only senders easier deployment– Requires receiver app feedback (e.g., TCP

ACKs)

• CM protocol and headers negotiable incremental deployment

• IETF working group to be formed

Summary

• CM enables proper and stable congestion behavior

• Simple API to enable application to learn and adapt to network state

• Improves consistency and predictability of network transfers

• Provides benefit even when deployed at senders alone