ittc high-performance networkingjpgs/courses/hsnets/lecture-transport-hsn-display.pdf · – error...

© James P.G. SterbenzITTCHigh-Performance Networking

The University of Kansas EECS 881End-to-End Transport

© 2004–2010 James P.G. Sterbenz18 November 2010

James P.G. Sterbenz

Department of Electrical Engineering & Computer ScienceInformation Technology & Telecommunications Research Center

The University of Kansas

[email protected]

http://www.ittc.ku.edu/~jpgs/courses/hsnets

rev. 10.0

18 November 2010 KU EECS 881 – High-Speed Nets – End-to-End Transport HSN-TL-2

© James P.G. SterbenzITTC

End-to-End TransportOutline

TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations



End-to-End TransportOutline

TL.1. Functions & mechanismsTL.2. State managementTL.3. Framing and multiplexing

TL.4. Error controlTL.5. Transmission controlTL.6. TCP optimisations

network

application

session

transport

network

link

end system

network

link

node

network

link

nodenetwork

link

node

application

session

transport

network

link

end system



Transport LayerService and Interfaces1

• Transport layer (L4) service to application layer (L7)• Transfer PDU E2E (end-to-end)

– sender: encapsulate ADU into TPDU and transmit– receiver: receive TPDU and decapsulate into ADU

• Multiplexing to application on end system• Optional reliability:

– error checking/correction/retransmission– need depends on application– may be done application-to-application (A2A)

• Flow control between end systems– assistance for network congestion control



Transport LayerService and Interfaces2

• Transport layer uses service of network layer (L3)– network layer establishes a path through the network

• Transport mechanisms depend on L3 service– datagrams vs. connections– best effort vs. QOS– error characteristics– strength and symmetry of connectivity

What are the Internet assumptions & service model?



Transport LayerTypes of Transport Protocols

• Spectrum of transport protocol specialisation– general purpose– functionally partitioned– application-oriented– special purpose

• Software engineering and implementation choice



Transport LayerService and Interfaces

• Transport PDU encapsulates application data unit– ADU – application data unit– TPDU – transport protocol data unit

• TPDU = header + ADU + optional trailer (protocol dependent)

application layer

network layer

transport layer

application layer

network layer

transport layer

ADU

ADU TH

ADU

NPDU TH



Transport LayerFunctional Placement

• Transport layer functionality only in end system– host software or– network interface (NIC)

switch

TPDUs

end system

network interface

mem CPU

frames

end system

network interface

memCPU

framespacket



Transport LayerEnd-to-End vs. Hop-by-Hop

• Transport layer is E2E analog of HBH link layer– link layer (L2) transfers packets HBH– network layer (L3) determines path of concatenated links



Ideal End-to-End Network ModelBandwidth and Delay

• Zero end-to-end delay• Unlimited end-to-end bandwidth

M app

M app

D = 0

R = ∞

ES1

CPU

ES2

CPU



End-to-End TransportTL.1 Functions and Mechanisms

TL.1 Functions and mechanismsTL.1.1 End-to-end semanticsTL.1.2 End-to-end mechanisms

TL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations



E2E Functions and MechanismsEnd-to-End Semantics

• Hop-by-hop functions– link layer protocols– link compression and FEC– network forwarding– link / subnet error control– embedded protocols

(e.g. protocol boosters)

• End-to-end functions– transport protocols– source routing– end-to-end encryption– session protocols– application protocols

Hop-by-Hop

End-to-End



E2E Functions and Mechanisms End-to-End Argument

• Hop-by-hop function composition ≠ end-to-end• Examples

– HBH encryption has data in clear inside network nodes– HBH link error control doesn’t cover network layer errors

End-to-End Argument T-3

Functions required by communicating applications can be correctly and completely implemented only with the knowledge and help of the applications themselves. Providing these functions as features within the network itself is not possible.



E2E Functions and Mechanisms Hop-by-Hop Performance Enhancement

• E2E functions replicated HBH if performance benefit• Example

– per link error control in high bandwidth-×-delay networksreduce E2E control loop when error

Hop-by-Hop Performance Enhancement Corollary T-3A

It is beneficial to duplicate an end-to-end function hop-by-hop if the result is an overall (end-to-end) improvement in performance.



End-to-End vs. Hop-by-HopPerformance Enhancement Example

• E2E vs. HBH error control for reliable communication– E2E argument says error control must be done E2E

• e.g. E2E ARQ (error check code and retransmit if necessary– but should HBH error control also be done?

100 mwireless LANUniv. Kansas

100 mfiber LAN

Univ. Sydney

15 000 kmfiber WAN



End-to-End vs. Hop-by-HopPerformance Enhancement Example

• E2E vs. HBH error control for reliable communication– E2E argument says error control must be done E2E

• e.g. E2E ARQ (error check code and retransmit if necessary– but should HBH error control also be done?

• Effect of high loss rate on wireless link– ~250 ms RTT retransmission for every corrupted packet

• Error control on wireless link reduces to ~1μs– shorter control loop results in dramatically lower E2E delay

100 mwireless LANUniv. Kansas

100 mfiber LAN

Univ. Sydney

15 000 kmfiber WAN



E2E Functions and Mechanisms E2E Argument Misinterpretations

• E2E-only– do not replicate E2E services or features HBH– violated HBH performance enhancement corollary

• Everything E2E– implement as many services or feature E2E as possible– misstatement of Internet design philosophy:

simple stateless network for resilience and survivability



E2E Functions and MechanismsMeaning of “In the Network”

• Functional (general use in this tutorial)– layers 1 – 3

• physical• MAC• link• network

• Topological– colocated with network nodes

• Administrative– owned by network service provider



End-to-End TransportTL.1.2 End-to-End Mechanisms

TL.1 Functions and mechanismsTL.1.1 End-to-end semanticsTL.1.2 End-to-end mechanisms

TL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations



E2E Functions and MechanismsEnd-to-End Mechanisms

• Framing• Multiplexing• End-to-end state and connection management• Error control• Flow control• Congestion control



E2E Functions and MechanismsTransport Protocol Service Categories

• Transfer mode– connectionless datagram– connection-oriented– transaction– continuous media streaming

• Reliability– loss tolerance

• Delivery order– ordered– unordered

• Traffic characteristics



E2E Functions and MechanismsTypes of Transport Protocols

• Spectrum of transport protocol specialisation– general purpose– functionally partitioned– application-oriented– special purpose

• Software engineering and implementation choiceTransport Protocol Functionality T-IV

Transport protocols must be organised to deliver the set of end-to-end high-bandwidth, low latency services needed by applications. Options and service models should be modularly accessible, without unintended performance degradation and feature interactions.



E2E Functions and MechanismsExample7.2 End-to-End Internet Protocols

• Transport protocolsProtocol Name Function Status Ref

TCP transmission control protocol

reliable data transfer with congestion control

socket access to unreliable IP datagrams

file and document transfer(typically over UDP)

remote login

reliable data transfer with no congestion control

signallingproposed for wireless

RFC 793STD 007

UDP user datagram protocol

standard

standard

standards track

experimental

experimental

RFC 768STD 006

RTP real-time protocol RFC 1889

T/TCP TCP for transactions RFC 1644

RDP reliable data protocol RFC 908

proposed standardSCTP stream control

transmission protocol RFC 2960



E2E Functions and MechanismsExample7.2 End-to-End Internet Protocols

“application layer” only because they run over TCP or UDP

Protocol Name Function/use

HTTP hypertext transfer protocol Web browsing

SMTP simple mail transfer protocol email relay and delivery

FTP file transfer protocol file and document transfer

Telnet telnet remote login

NFS network file system remote access to files

RTSP real-time streaming protocol control of multimedia streaming

• “Application layer” protocols



E2E Functions and MechanismsControl of State

• Open loop– control parameters– no feedback

• Closed loop– initial parameters– feedback

• dynamic adaptation

• Closed loopwith HBH feedback

• Nested control loops

Open loop

parameters

data

Nested control loops

initial parameters

data

E2E feedback

receiversender

Closed loop with HBH feedback

HBH feedback

initial parameters

data

E2E feedback

Closed loop

initial parameters

data

E2E feedback

HBH loop

HBH loop




Open- vs. Closed-loop Control T-6D

Use open-loop control based on knowledge of the network path to reduce the delay in closed loop convergence. Use closed-loop control to react to dynamic network and application behaviour; intermediate per hop feedback can sometimes reduce the time to converge.

• Open-loop control– use when possible to eliminate control loop latency

• particularly for high bandwidth-×-delay product networks

• Closed-loop feedback control– use when necessary, e.g. reliability




Aggressiveness of Closed-Loop Control T-6Da

The response to closed-loop feedback must be rapid enough so the system converges quickly to a new stable state – but not so aggressive that oscillations occur due to overreaction to transients.

• Aggressiveness of closed-loop control– rapid enough to system converges– not too aggressive avoid instability and oscillations



End-to-End TransportTL.2 State Management




End-to-End ProtocolsState Management

7.1 Functions and mechanisms7.2 State management

7.2.1 Impact of high speed7.2.2 Transfer modes7.2.3 State establishment and maintenance7.2.4 Assumed initial conditions

7.3 Framing and multiplexing7.4 Error control7.5 Flow and congestion control7.6 Security and information assurance



State ManagementTransport and Network Connections

• L4 connections– required over

• connectionless L3• path with any

connectionlesssubnetwork

– may exploit• connection-oriented L3

SETUP CONNECT

network layer

end system

transport layer

connection-orientednetwork

network layer

end system

transport layer

setup-req

network layer connection

connectionless network

network layer

transport layer SETUP CONNECT

network layer

transport layer

transport layer connection

network layer

transport layer SETUP CONNECT

network layer

transport layer

connection-oriented subnetwork

connectionlesssubnetwork

heterogeneous subnetworks



E2E State ManagementImpact of High Speed

• Connection shortening– transmission delay decreases– signalling delay constant

• E2E signalling– significant fraction of time

slow high-speedfast

~1.5tRTT

~2.5tRTT ACK

RELEASEACK

RELEASE

tRTT

SETUP

RELEASE

ACK

tp

td

tp

td→0

SETUP SETUP



E2E State ManagementImpact of Long Latency

• Connection lengthening– transmission delay increases– signalling delay increases

• E2E signalling– open state longer– denial of resource to others

short links long links

SETUP

ACK

RELEASE

SETUP

ACK

RELEASE



Transfer ModesDatagram

• Independent packets– each has fully self-describing header– no network state needed to forward

tp+td



Transfer Modes Connection

• Explicit connection setup– 3-way handshake

SETUP / CONNECT / ACK• Data flow

– packets need connection or flow identifier• used by nodes to look up state

• Release of resources and state– explicit RELEASE– time out state

~tRTT

ACK

RELEASE

CONNECT

SETUP

~1.5tRTT

~tRTT

tr



Transfer Modes Connection

• ConnectionStateMachine– idle– establishing

• install state– initialising

• stateconvergence

– steady state– closing

• release state

initialising

idle

steady-state

closing closing

establishing

CONNECTfrom net

requested

receive SETUP from net

originate SETUP request

issue CONNECT

ACK from net

RELEASEfrom net

RELEASEfrom app

issue SETUP

issue RELEASE

issue RELEASE



Transfer Modes Stream

• Various mechanisms to start stream– explicit client request– server push– may or may not establish connection state

• Data flow – synchronisation and control

• embedded or• out-of-band

REQUEST

RELEASE

CONNECT

SETUP



Transfer Modes Stream

• Playback buffer to reorder and absorb jitter– adds delay

maximum playout point

1

application

2346

5 67

playout buffer

receiving end system

Continuous Media Streaming T-I.2

Timeliness of delivery should not be traded for reliable transfer in real-time continuous media streams. Use a playback buffer to reorder and absorb jitter.



Transfer Modes Transaction

• Transaction request– may or may not establish connection state– explicit release of connection state optional

• Data returned• Overlap to reduce latency

– request/response with control

REQUEST

RELEASE

tr

RESPONSE

Minimise Transaction Latency T-6A

Combine connection establishment with request and state transfer to reduce end-to-end latency for transactions.



E2E State ManagementHard vs. Soft

• Hard state– explicitly established and released– deterministic– delay to establish and memory to maintain

• Soft state– established and removed as needed and memory allows– robust to failures– if not explicitly established, delay to accumulate

Hard vs. Soft State T-5A

Balance the determinism and stability of hard state against the adaptability and robustness to failure of soft state.



E2E State ManagementState Aggregation and Sharing

• Sharing to reduce state– temporal: reuse past state– spatial: e.g. end system state

max link rate link characteristics

(sub)network characteristics

path MTU RTT estimate per E2E path

rate/window ACKs/NAKs

global (per interface)

per network

per connection/stream

appid1

appid2 per application

per application transfer

p e r t r a f f I c c l a s s



E2E State ManagementState Aggregation and Sharing

• Sharing of state– reduces granularity of individual entities– state shared is fate shared state

State Aggregation T-5B

Spatially aggregate state to reduce complexity, but balance against loss in granularity. Temporally aggregate to reduce or eliminate establishment and initialisation phase.State shared is fate shared.



E2E State ManagementAssumed Initial Conditions

• Rapid convergence to steady state– assume initial conditions

• normal or common case• past behaviour• current network state

Use Initial Assumptions to Converge Quickly T-5E

The time to converge to steady state depends on the accuracy of the initial conditions, which depends on the currency of state information and the dynamicity of the network. Use past information to make assumptions that will converge quickly.



End-to-End TransportTL.3 Framing and Multiplexing

TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexing

TL.3.1 Framing and fragmentationTL.3.2 Application layer framingTL.3.3 Multiplexing

TL.4 Error controlTL.5 Transmission controlTL.6 TCP optimisations



E2E Framing and MultiplexingPacket Size and Control Overhead

• Small packets– large overhead– short interarrival

• Grouped• Blocked / bursts• Large

– increased delay• Jumbogram

Balance Packet Size T-8A

Trade between control overhead and fine enough grained control. Choose size and aggregation methods that make sense end-to-end.

tpkt

tpkt

tgrp




• Large packets– impose jitter and delay on one another

interference

in1

in2

out delayed



E2E State ManagementMTU and per Hop Fragmentation

• Subnetworks have limits on packet size– MTU: maximum transfer unit

• If |TPDU| > min(MTU)– fragmentation occurs in the network if permitted

• significant impact on packet processing rate and delay

– most modern networks drop• fragmentation disabled to prevent overwhelming slow path

Avoid Hop-by-Hop Fragmentation T-5F

The transport layer should be aware of or explicitly negotiate the maximum transfer unit of the path to avoid the overhead of fragmentation and reassembly.




Application Layer Framing T-8A

To the degree possible, match ADU and TPDU structure. In cases where the application can better determine structure and react to lost and misordered ADUs, ALF should be employed.

transport layer fragmentation

ADU

MTU

TPDU

ADU ADU

TPDU TPDU TPDU

TPDU

ADU ADU

TPDU TPDU TPDU

ADU

application layer framing



E2E Framing and MultiplexingLayered Multiplexing

• Multiplexing– application

• interapplicationcoordination

– transport• applications

sharing network interface

– network• end systems

sharing switch– MAC and link

• interfaces sharing medium

a4 pb ADU a4 pc ADU

host4 TPDU TPDU

host3

a3 pa ADUTPDU

NPDUs

network muxing

a3 pa ADU a4 pc ADUa4 pb ADU

host2

pa ADUappx

pb ADUappy

a4 pb ADU

pb ADUappb

pc ADUappc

pc ADUappz

network demuxing

pa ADUappa

transport muxing

transport demuxing

a4 pc ADU

host1

a3 pa ADU TPDU TPDU

TPDU



E2E Framing and MultiplexingIntegrated Multiplexing

• Multiplexing– significant overhead

• processing limits• delay

– undesirable fate sharing• QOS

– perform all layers at once• network layer for wired• MAC layer for wireless

Limit Layered Multiplexing T-4A

Layered multiplexing should be minimised and performed in a single integrated manner for all layers at the same time.

network muxing

a3 pa ADU a4 pc ADUa4 pb ADU

host2

appx a3 pa ADU

appy a4 pb ADU

host1

appb

appc

host3 a3 pa ADU

a4 pb ADU

a4 pc ADUhost4

a4 pc ADUappx

network demuxingappa

NPDUs



End-to-End TransportTL.4 Error Control

TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error control

TL.4.1 Types and causes of errorsTL.4.2 Impact of high speedTL.4.3 Closed-loop retransmissionTL.4.4 Open-loop error control

TL.5 Transmission controlTL.6 TCP optimisations



E2E Error ControlTypes of Errors0

What are the types of errors?




• Bit errors• Packet errors

types?




• Bit errors• Packet errors

– packet loss– fragment loss– burst loss– packet misordering– packet insertion– packet duplication



E2E Error ControlCauses of Bit Errors0

• Bit errorscauses?



E2E Error ControlCauses of Bit Errors1

• Bit errors– flaky hardware– wireless channels

• noise and interference

– optical channels• long-haul fiber links engineered on margin with FEC• control of amplifier and regenerator cost



E2E Error ControlCauses of Packet Loss0

• Packet losscauses?




• Packet loss– network buffer overflow or intentional drop– transport protocol packet discard when bit error





• Packet fragment losscauses?impact?





• Packet fragment loss– loss of a piece of a fragmented packet– effect: error multiplication

• loss of a fragment in net generally results in entire packet loss



E2E Error ControlCauses of Packet Burst Loss0

• Packet burst losscauses?



E2E Error ControlCauses of Packet Burst Loss1

• Packet burst loss– network buffer overflow during congestion– wireless channel fades



E2E Error ControlCauses of Packet Misordering0

• Packet misorderingcauses?



E2E Error ControlCauses of Packet Misordering1

• Packet misordering– multiple paths through switch or network– non-deterministic paths through switch or network– path reconfiguration

• after failure• due to mobility• due to load balancing



E2E Error ControlCauses of Packet Insertion0

• Packet insertionmeaning?causes?



E2E Error ControlCauses of Packet Insertion1

• Packet duplication– undetectable header error

• packet appears that was destined elsewhere

– long packet life• packet with same destination and sequence number still in net



E2E Error ControlCauses of Packet Duplication0

• Packet duplicationcauses?



E2E Error ControlCauses of Packet Duplication1

• Packet duplication– retransmission but original still arrives



E2E Error ControlImpact of High Speed

• Bandwidth-×-delay product increases– fixed duration error event larger relative impact– closed-loop reaction occurs later in terms of # bits sent

• Sequence numbers– sequence numbers wrap if sequence space too small

• causes packet insertion



End-to-End TransportTL.4.3 Closed-Loop Retransmission






E2E Error ControlARQ Closed Loop Retransmission

• Automatic repeat request (ARQ)– PDUs are retransmitted for reliable transfer

• Acknowledgments are used to request retransmission– ACK: positive acknowledgement– NAK (or NACK): negative acknowledgement

which is better?



E2E Error ControlARQ Closed Loop Retransmission

• Automatic repeat request (ARQ)– PDUs are retransmitted for reliable transfer

• Acknowledgments are used to request retransmission– ACK: positive acknowledgement

• required for full reliability by E2E arguments

– NAK (or NACK): negative acknowledgement– tradeoff between error rate and predictability

• hybrid: NAKs supplemented by ACKs when necessary for E2E

Simplest ARQ mechanism?



0

E2E Error ControlClosed Loop Retransmission: Stop-and-Wait

• Each packet is acknowledged

1




• Each packet is acknowledged0

0

2




• Each packet is acknowledged– subsequent packet waits on previous ACK

disadvantages?

0

0

3

A0





• significant delay and underutilisation• 1 RTT per packet• serious penalty in long-delay environment

0

0

A0

1tack

4






– sender timer tack fires if ACK not receivedissues?

0

0

A0

1tack

5



1




– sender timer tack fires if ACK not received• too conservative: issues?• too aggressive: issues?

0

0

A0

1tack

6



1




– sender timer tack fires if ACK not received• too conservative: unnecessary delay• too aggressive: spurious retransmissions

0

0

A0

1tack

7

1






– sender timer tack fires if ACK not received• too conservative: unnecessary delay• too aggressive: spurious retransmissions

How can we do better?

0

0

A0

1

1

A1

tack

1

8



E2E Error ControlClosed Loop Retransmission: Go-Back-n

• Go-back-n : pipeline transmissions 0

1




• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight 0

0

2

1

A0



2


• Go-back-n : pipeline transmissions + multiple packets simultaneously in flight

• Packets are sequentially acknowledged

0

0

3

1

A0

1

A1

tack



3

2



• Packets are sequentially acknowledgedhow is packet loss detected?

0

0

4

1

A0

1

A1

tack



4

3

2



• Packets are sequentially acknowledgedo previous ACK retransmitted for

• subsequent packets after loss• out of sequence packet

0

0

5

1

A0

1

A1

tack

A1



5

4

3

2





o sender timer fires if ACK not receivedimplication?

0

0

6

1

A0

1

A1

tack

A1

A1



2

5

4

3

2





o sender timer fires if ACK not received• reset transmission beginning at lost packet

0

0

7

1

A0

1

A1

tack

A1

A1

A1



3

A2

2

5

4

3

2






0

0

8

1

A0

1

A1

tack

A1

A1

A1 2



4

3

A2

2

5

4

3

2






0

0

9

1

A0

1

A1

tack

A1

A1

A1 2

3

A3



5

4

3

A2

2

5

4

3

2






0

0

10

1

A0

1

A1

tack

A1

A1

A1 2

3

A3

A4

4



6

5

4

3

A2

2

5

4

3

2






0

0

11

1

A0

1

A1

tack

A1

A1

A1 2

3

A3

A4

4

5

A5



6

5

4

3

A2

2

5

4

3

2






disadvantages?

0

0

12

1

A0

1

A1

tack

A1

A1

A1 2

3

A3

A4

4

5

A56








– significant loss penalty for high bw-×-delay• go back and retransmit all since loss• many unneeded retransmissions• significant additional delay

A0

A1

A1

0

1

2

3

4

5

6

2

3

4

5

6

A1

A2

A3

A4

A5

A1

0

1

2

3

5

4

tack

13




• Optimisation possible for go-back-n?

A0

A1

A1

0

1

2

3

4

5

6

2

3

4

5

6

A1

A2

A3

A4

A5

A1

0

1

2

3

5

4

tack




• Optimisation go-back-ntimer the only way to detect loss?

A0

A1

A1

0

1

2

3

4

5

6

2

3

4

5

6

A1

A2

A3

A4

A5

A1

0

1

2

3

5

4

tack



E2E Error ControlClosed Loop Retransmission: Fast Retransmit

• Fast retransmit– optimisation for go-back-n

0

1




• Fast retransmit– optimisation for go-back-n 0

0

2

1

A0



2



0

3

1

A0

1

A1

tack



3

2



0

4

1

A0

1

A1

tack



4

3

2



• Assume several duplicate ACKs are loss

0

0

5

1

A0

1

A1

tack

A1



2

5

4

3

2



• Assume several duplicate ACKs are losso even if timer hasn’t yet fired

0

0

6

1

A0

1

A1

tack

A1

A1



3

A2

2

5

4

3

2



• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly

0

0

7

1

A0

1

A1

tack

A1

A1

A12



43

A2

2

5

4

3

2




0

0

8

1

A0

1

A1

tack

A1

A1

A123

A3



543

A2

2

5

4

3

2




0

0

9

1

A0

1

A1

tack

A1

A1

A123

A3A4

4



6

543

A2

2

5

4

3

2




0

0

10

1

A0

1

A1

tack

A1

A1

A123

A3A4

45

A5



6

543

A2

2

5

4

3

2




0

0

11

1

A0

1

A1

tack

A1

A1

A123

A3A4

45

A56





• Assume several duplicate ACKs are losso even if timer hasn’t yet fired+ recovers from loss more quickly– still suffers from go-back-n inefficiencies

12

6

543

A2

2

5

4

3

2

0

0

1

A0

1

A1

tack

A1

A1

A123

A3A4

45

A56



E2E Error ControlClosed Loop Retransmission: Delayed ACK

• Delayed ACKo optimisation for go-back-n

• Aggregate several ACKs+ reduces ACK traffic+ allows some receiver resequencing

• within ACK aggregation quantum




• Go-back-n̶ fast retransmit and delayed ACK help slightly̶ but still significant delay penalty for losses

Alternative?



E2E Error ControlClosed Loop Retransmission: Selective Repeat

• Go-back-n̶ fast retransmit and delayed ACK help slightly̶ significant delay penalty for losses

• Alternative− don’t go back: selective repeat




• Selective repeato all packets acknowledged

1

0




• Selective repeato all packets acknowledged 0

0

2

1

A0



2



0

3

1

A0

1

A1

tack



3

2



0

4

1

A0

1

A1

tack



4

3

2



– Missequenced packetso acknowledged and held at receiver

0

0

5

1

A0

1

A1

tack

A3

3



5

4

3

2



• Missequenced packetso packets held and reordered at receiver– increases receiver complexity

0

0

6

1

A0

1

A1

tack

A3

3

A4

43



2

5

4

3

2




• Lost packetso selectively retransmitted

0

0

7

1

A0

1

A1

tack

A3

3

A4

43

543

A5



62

5

4

3

2




• Lost packetso selectively retransmitted+ no go-back-n latency penalty– requires more receiver buffer space

0

0

8

1

A0

1

A1

tack

A3

3

A4

43

543

A5

A2

5432



762

5

4

3

2





0

0

9

1

A0

1

A1

tack

A3

3

A4

43

543

A5

A2

5432

A6

6



8762

5

4

3

2





0

0

10

1

A0

1

A1

tack

A3

3

A4

43

543

A5

A2

5432

A6

6

A7

7



9

8762

5

4

3

2





• Latency and Bandwidth reduced

0

0

11

1

A0

1

A1

tack

A3

3

A4

43

543

A5

A2

5432

A6

6

A7

87

A8



E2E Error ControlClosed Loop Retransmission: Periodic State

• Selective ACK blockso bit vectors can aggregate ACKs+ significant reduction in control packets+ simple receiver implementation possible

5

4

6

7

0

1

A01_3

0

1

2

3

2

8

9

7–2

8

9A4567

~1.5tRTT

3

43

543

6543

765543

A2



E2E Error ControlClosed Loop Retransmission: Comparison

0

0

A0

1

1

A1

tack

1

A0

A1

A1

0

1

2

3

4

5

6

2

3

4

5

6

A1

A2

A3

A4

A5

A1

0

1

2

3

5

4

tack

9

8762

5

4

3

2

0

0

1

A0

1

A1

tack

A3

3

A4

43

543

A5

A2A6

6

A7

87

A8

5

4

6

7

0

1

A01_3

0

1

2

3

2

8

9

7–2

8

9A4567

~1.5tRTT

3

43

543

6543

765543

A2



E2E Error ControlImpact of High Speed

• Bandwidth-×-delay product increases– fixed duration error event larger relative impact– reaction to errors decreases in terms of number of bits sent

• Sequence numbers– sequence numbers wrap if sequence space too small

• causes packet misinsertion

Packet Control Field Values T-8C

Optimise header control field values to trade efficiency againstexpected future requirements. Fields that are likely to be insufficient for future bandwidth-×-delay products should contain a scale factor.



E2E Error ControlDecoupling of Control Mechanisms

• Error control mechanisms– difficult to optimise for high performance

when entangled with flow/congestion control(more later )

Decouple Error, Flow, and Congestion Control T-6E

Exploit selective acknowledgements and open-loop rate control to decouple error, flow, and congestion control mechanisms.



E2E Error ControlClosed Loop Retransmission: Multicast

• ACK implosion: ACK suppression neededo NAKs (negative ACKs) with liveness pollingo localised hop-by-hop buffering and retransmission– significant reduction of traffic on relatively reliable links

R

6

ACK

ACK×2ACK×2

ACK×4

ACKACK×6

R

6

NAK

NAK

NAK

NAK×2

31 2 4 31 2 4

ACKACK

ACK

5ACK

NAK

5NAK



Error ControlExample:7.6 SNA Error Control

• SNA (IBM Systems Network Architecture)

• Not in agreement with end-to-end arguments, but…– engineered reliability of store-and-forward hosts

• no part of data path not covered by ECC: processor + memory

Scope Layer Name Reliability

hop-by-hop link data link control yes

end-to-end transport path control no



End-to-End TransportTL.4.4 Open-Loop Error Control






E2E Error ControlOpen Loop

• Open loop error controlo some applications do not need guaranteed reliability

• applications that are tolerant to some loss• real-time or interactive delay requirements

– eliminates control loop latency for recovery+ requires additional end-systems bandwidth & processing+ requires additional network bandwidth

• Forward error correction– block codes: per block recovery– convolutional codes: continuous over window



E2E Error ControlHybrid Open/Closed-Loop Control

• Hybrid open loop with closed loop when necessary– statistical error bound with FEC– retransmission only when FEC doesn’t allow correction– appropriate for

• high latency and bandwidth-×-delay paths• asymmetric bandwidth paths

Forward Error Correction T-6De

Use FEC for low-latency, open-loop flow control when bandwidth is available and statistical loss tolerance bounds areacceptable to the applications.



End-to-End TransportTL.5 Transmission Control

TL.1 Functions and mechanismsTL.2 State managementTL.3 Framing and multiplexingTL.4 Error controlTL.5 Transmission control

TL.5.1 Impact of high speedTL.5.2 Open-loop rate controlTL.5.3 Closed-loop flow controlTL.5.4 Closed-loop congestion controlTL.5.5 Hybrid flow and congestion control

TL.6 TCP optimisations



E2E Transmission ControlDefinitions

• Flow control– control transmission to avoid overwhelming receiver– end-to-end control





• Congestion control– control transmission to avoid overwhelming network paths





• Congestion control– control transmission to avoid overwhelming network paths– network control with end-to-end assistance (throttling)

• Types– explicit relies on congestion information from nodes

• allows decoupling of error, flow, and congestion mechanisms

– implicit assumes loss is congestion• bad assumption in wireless networks (see [Krishnan 2004])



E2E Transmission ControlImpact of High Speed

• Bandwidth-×-delay product increases– response to an event happens after more bits transferred– it may be too late to react

• all bits causing problem may already be transmitted• other cause of problem may have gone away



E2E Transmission ControlOpen-Loop Rate Control

• Initial negotiation– admission control limits traffic to available resources

admissioncontrol




• Initial negotiation– admission control limits traffic to available resources– receiver negotiates what it can accept (flow control)

admissioncontrol

receivernegotiation





• Steady state– policing enforces traffic contract from transmitter

admissioncontrol

receivernegotiation

ratescheduling

ratepolicing

conforming traffic





• Steady state– policing enforces traffic contract from transmitter

• excess traffic may be marked and passed if capacity available

admissioncontrol

receivernegotiation

ratescheduling

ratepolicing

conforming trafficexcess traffic




Use Knowledge of Network Paths for Open-Loop T-6Do

Control Exploit open-loop rate and congestion control based on a priori knowledge to the degree possible to reduce the need for feedback control.

• Requires connection establishment– overhead and delay for connection setup

• optimistic establishment or fast reservations ameliorate

+ QOS guarantees possible in steady state+ feedback control loops not needed during transmission+ network stability less dependent on congestion control

• unless resources significantly overbooked



E2E Transmission ControlOpen-Loop Rate Control: Parameters

• Main parameters– peak rate rp

– average rate ra

– burstiness (peak to average ratio or max burst size)

• Derived parameters– jitter

rp

ra bursty

uniform

time



E2E Transmission ControlClosed-Loop Flow and Congestion Control

Use Closed-Loop Congestion Control to Adjust to T-6Dc

Network Conditions Closed-loop feedback is needed to adjust to network conditions when there are no hard reservations.

networkcongestion control

packet scheduling

flow control



E2E Transmission ControlClosed-Loop Flow Control

• Feedback from receiver to limit rate• Window to limit amount of unacknowledged data

– static window– dynamic window

• combined with congestion control



Congestion ControlIdeal Network

offered load

ideal

carr

ied

load

offered load

ideal

1.0

dela

y

throughput delay

• Ideal network:– throughput: carried load = offered load (45° line)– zero delay (flat line)

Why isn’t this possible?



Congestion ControlReal Network

• Delay can’t be zero: speed-of-light– delay through network channels– delay along paths in network nodes

• Throughput can’t be infinite– channel capacity limits bandwidth– switching rate of components limits bit rate



Congestion ControlDesired Real Network Behaviour

knee

offered load

ideal1.0

1.0

carr

ied

load

offered load 1.0

dela

y

knee

throughput

dmin

delay

• Desired network:– carried load = offered load up to share of capacity– what happens to delay?



Congestion ControlCauses of Congestion

• Congestion when offered load → link capacity• Best effort

– occurs whenever load is high

• Probabilistic guarantees– occurs when load exceeds resource reservations

• Absolute guarantees– can’t happen (in theory)



Congestion ControlInfinite Buffers / No Retransmission

• Congestion when offered load → link capacity• Infinite buffers

– queue length builds to infinity– d → ∞

todo: figures



Congestion ControlFinite Buffers with Retransmission

• Finite buffers cause packet loss– retransmissions contribute to offered load

• assuming reliable protocol

– throughput curve levels off



Congestion ControlMultihop Network

• Multiple hops– downstream packet loss– upstream transmission wasted



Congestion ControlConsequences of Congestion

• Delay increases– due to packet queuing in network nodes– due to retransmissions when packets overflow buffers

• finite buffers must drop packets when full

• Throughput– levels off gradually (with real traffic)– then decreases

• due to retransmissions when packets dropped• particularly over multiple hops

– congestion collapse• “cliff” of the throughput curve



Congestion ControlLocal Action

• Local action at point of congestion• Drops packets: discard policy

– tail drop simplest: drop packets that overflow buffers– more intelligent policies possible

• need flow or connection state in switches• discriminate which flows are causing congestion and penalise



Congestion ControlEnd-to-End Action

• End-to-end action• Throttle source

– reduce transmission rates– prevent unnecessary retransmissions

• Types– explicit– implicit



Congestion ControlCongestion Avoidance and Control

knee cliff

offered load

ideal1.0

1.0

carr

ied

load

offered load

ideal

1.0

dela

y

knee

cliff

CA CC

throughput

dmin

delay

CA

CC

Congestion Avoidance T-II.4c

Congestion should be avoided before it happens. Keep queues from building and operate just to the left of the knee to maximise throughput and minimise latency.



E2E Transmission ControlClosed-Loop Congestion Control

• Feedback from network to limit rate• Window to limit amount of unacknowledged data

– dynamic window– conservation of packets in the network– self clocking

• Required unless open loop with hard reservations

Use Closed-Loop Congestion Control to Adjust to T-6Dc

Network Conditions Closed-loop feedback is needed to adjust to network conditions when there are no hard reservations.



Congestion ControlWindow Size

• Window size critical– big enough to fill pipe

A0

A1

0

1

A2

5

slow path fast path

0

1

2

3

A0A1A2A3

A4A5A6A7

4

5

6

A3

A4

A5

2

3

4

5

6

0123

4567

8 9 1011

4 5 6 7

0 1 2 3

891011

12

A8A9

A10A11



Congestion ControlSlow-Start Initialisation

A0

A1

A2

A3

A4

A5

A6

• Slow start initialisation– increase window until path loaded

• Critical parameters– initial window size– rate of increase

Tradeoffs?

0

3

4

5

6

0

1

2

1

2

3

4

5

6

7

8

7

8

9

10

9

10

11

12

11

12

13

14

13

14

15

A7

A8

A9

A10

A11

A12

13

14

15

16

17

18

19

20

21

20

21

18

19

10

11

12

13

14

15

16

17

A4

A5

A6

A0

A1

A2

A3

0

1

2

3

0

1

2

3

4

5

4

5

6

7

6

7

8

9

8

9

10

11

12

A7

A8

A9

A10

A11

A12

A10

A13

A14

A15

A16

A17

tRTT

tRTT

tRTT



Congestion ControlSlow-Start Initialisation

A0

A1

A2

A3

A4

A5

A6

• Slow start initialisation– increase window until path loaded

• Critical parameters– initial window size– rate of increase

• Tradeoffs– conservative on high bw-×-delay:

• multiple round trips• never get to full rate for transactions

– aggressive:• induced congestions

0

3

4

5

6

0

1

2

1

2

3

4

5

6

7

8

7

8

9

10

9

10

11

12

11

12

13

14

13

14

15

A7

A8

A9

A10

A11

A12

13

14

15

16

17

18

19

20

21

20

21

18

19

10

11

12

13

14

15

16

17

A4

A5

A6

A0

A1

A2

A3

0

1

2

3

0

1

2

3

4

5

4

5

6

7

6

7

8

9

8

9

10

11

12

A7

A8

A9

A10

A11

A12

A10

A13

A14

A15

A16

A17

tRTT

tRTT

tRTT



Congestion ControlAIMD Steady State

• AIMD steady state– additive-increase slowly increases rate

• increment window

– multiplicative-decrease quickly throttles with congestion• divide window

– RED attempts to reduce when congestion impending

additive increase

multiplicative decrease

time

rate

congestion detection



Congestion ControlAIMD Fairness and Stability

• AIMD– R = available bandwidth– initial bandwidth share I– AI: both increase

• until loss beyond R– MD: both decrease

• half way to origin• halve window size

– trajectory toward ideal• intersection of R and equal

[based on Kurose fig. 3.53]

equal bandwidth

shareR

Co n

nec t

ion

2 th

roug

hput

RConnection 2 throughput

I

ideal



Congestion ControlSlow Start + AIMD

• Initialisation phase: slow start• Steady-state phase: AIMD

AI

time

MD

send

ing

rate

slow start

steady state phase initialisation

congestion detection



Congestion ControlOptimisation for High-Speed

• Initialisation– start with bigger window– increase more rapidly than “slow start” doubling– danger: too aggressive can cause congestion collapse



Congestion ControlOptimisation for High-Speed

• Initialisation– start with bigger window– increase more rapidly than “slow start” doubling– danger: too aggressive can cause congestion collapse

• Steady state– better estimate capacity to avoid MD halving

• TCP BIC and CUBIC binary search for faster convergance



Congestion ControlImplicit Control

• Implicit control– missing ACK assumed to be congestion

good assumption?




• Implicit control– missing ACK assumed to be congestion– reasonable when all losses are due to congestion

• fiber optic channels connected by reliable switches






– performs poorly when significant losses in channelwhy?






– performs poorly when significant losses in channel• mobile wireless links• under-provisioned CDMA (including optical CDMA)






– performs poorly when significant losses in channel• mobile wireless links• under-provisioned CDMA (including optical CDMA)

Alternatives?



Congestion ControlExplicit Control

• Explicit control– explicit congestion notification (ECN)

• throttle with ECN message

– some decoupling of error and congestion control• throttle before packet loss• not sufficient for lossy wireless link



Congestion ControlExplicit Control

• Explicit control– explicit congestion notification (ECN)

• throttle with ECN message

– some decoupling of error and congestion control• throttle before packet loss• not sufficient for lossy wireless link

– not sufficient for discrimination of corrupted packets• ELN (explicit loss notification) necessary



Congestion ControlSteady State Comparison

• Implicit control– standard– with fast retransmit

• Explicit control– ECN

tack

implicit explicit fast retransmit

A7

A7

A7

A7

A7

13

14

15

16

8

9

10

11

12

11

12

9

10

8

A4

A5

A6

A0

A1

A2

A3

0

1

2

3

0

1

2

3

4

5

4

5

6

7

6

7

9

10

11

12

A7

A7

A7

A7

A8

A9

8

A71

A72

A7313

14

8

9

10

11

12

13

14

11

12

9

10

8

A4

A5

A6

A0

A1

A2

A3

0

1

2

3

0

1

2

3

4

5

4

5

6

7

6

7

9

10

11

12

A7

A7

A7

A8

A9

8

12

13

13

14

15

16

17

18

19

20

21

A4

A5

A6

A0

A1

A2

A3

0

1

2

3

0

1

2

3

4

5

4

5

6

7

6

7

9

10

11

12 11

12

9

10

8

13

14

ECN8

18

19

16

17

15

20

21

A7

A8

A9

A10

A11

A12

A13

A14

A15

A16

A17

A18

A10

A11



End-to-End TransportTL.6 TCP Optimisations


TL.6.1 TCP overviewTL.6.2 TCP optimisations for high performance



Transmission Control ProtocolOverview

• TCP: transmission control protocolRFC 0793 / STD 0007+ RFC 1122 implementation requirements+ RFC 1323 high performance extensions+ RFC 2018 SACK (selective acknowledgements)+ RFC 5681 congestion control+ RFC 3465 appropriate byte counting+ RFC 3168 ECN (explicit congestion notification)



TCP OverviewTransfer Mode and Characteristics

• Connection-oriented transfer mode– latency of connection setup– connection state must be maintained on each end system

• Point-to-point full-duplex– reliable byte stream: no message boundaries– pipelined transfer (not just stop-and-wait)

• Flow control– will not overwhelm receiver

• Congestion control– attempt to avoid congesting network– implicit and optional ECN



TCP OverviewSegment Format: Overview

• Relatively large header– fixed length & position fields– variable length options– header length4b in Bytes

• = 40 + |options| B

• Variable length payload• Options

– may not be present• Checksum on entire segment

– same algorithm as UDP

checksum urg data pointerreceive window

ack numbersequence number

dest port #source port #

[options (variable length)]

applicationpayload

(variable length)

flagsHL

32 bits

HL



TCP OverviewSegment Format: Header Flags

• Control flags (1 bit each)– CWR cong. win. reduced– ECE ECN echo– URG urgent data– ACK acknowledgement– PSH push data– RST reset connection– SYN connection setup– FIN connection teardown




options (variable length)

applicationpayload

(variable length)

HL

32 bits

CEUAPRSF

URG

ACK

PSH

RST

SYN

FIN

ECE

CWR



TCP OverviewConnection Management: Segment Format

• Flags define packet type– ACK segment

• ACK for SYN or FIN• SYNACK may be

piggybacked• ACK for data

– RST (reset) segment• e.g. bad port received

– SYN (synchronise) segment• connection setup

– FIN (finish) segment• connection release


ACK numbersequence number



applicationpayload

(variable length)

CEUAPRSFHL



data segments

data segments

TCP OverviewConnection Management: Setup

• Three way handshake– “client” and “server”1. SYN (init seq#) →

• randomised for security2. ← SYNACK (init seq#)

• randomised for security• after buffer allocation• SYN flood vulnerability

3. ACK →– buffer allocation– segment may include data– client may send more

4. server may return data

net

SYN (ISNs)

ACK

SYN(ISNc)ACK

FIN

ACK

FIN

ACK

bufferalloc

bufferalloc

estab.state

C S



data segments

data segments

TCP OverviewConnection Management: Teardown

• Two two-way handshakes– each is a half-close– may occur simultaneously

• full close– ACK can’t be piggybacked

• Client closes socket1. FIN →2. ← ACK– timed wait before close

• Server closes socket1. ← FIN2. ACK →

net

SYN

ACK

SYNACK

FIN

ACK

FIN

ACK

bufferfree

bufferfree

C S

twait

closed



TCP OverviewRFC 0793 State Machine

+---------+ ---------\ active OPEN | CLOSED | \ -----------+---------+<---------\ \ create TCB | ^ \ \ snd SYN

passive OPEN | | CLOSE \ \------------ | | ---------- \ \create TCB | | delete TCB \ \

V | \ \+---------+ CLOSE | \| LISTEN | ---------- | | +---------+ delete TCB | |

rcv SYN | | SEND | | ----------- | | ------- | V

+---------+ snd SYN,ACK / \ snd SYN +---------+| |<----------------- ------------------>| || SYN | rcv SYN | SYN || RCVD |<-----------------------------------------------| SENT || | snd ACK | || |------------------ -------------------| |+---------+ rcv ACK of SYN \ / rcv SYN,ACK +---------+| -------------- | | -----------| x | | snd ACK | V V | CLOSE +---------+ | ------- | ESTAB | | snd FIN +---------+ | CLOSE | | rcv FIN V ------- | | -------

+---------+ snd FIN / \ snd ACK +---------+| FIN |<----------------- ------------------>| CLOSE || WAIT-1 |------------------ | WAIT |+---------+ rcv FIN \ +---------+| rcv ACK of FIN ------- | CLOSE | | -------------- snd ACK | ------- | V x V snd FIN V

+---------+ +---------+ +---------+|FINWAIT-2| | CLOSING | | LAST-ACK|+---------+ +---------+ +---------+| rcv ACK of FIN | rcv ACK of FIN | | rcv FIN -------------- | Timeout=2MSL -------------- | | ------- x V ------------ x V \ snd ACK +---------+delete TCB +---------+------------------------>|TIME WAIT|------------------>| CLOSED |

+---------+ +---------+



TCP OverviewActual Connection State Machine

[Sewell] http://www.cl.cam.ac.uk/users/pes20/Netsem



TCP OverviewSegment Format: Sequence and Window

• Sequence #– forward data transfer– 1st byte # in segment

• ACK number– reverse acknowledgements– seq # of next byte expected– if ACK flag set– data segment may

piggyback ACK• Receive window

– # unACKed bytes





applicationpayload

(variable length)

HL CEUAPRSF



TCP OverviewUnidirectional Data Transfer

net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

1

• Data is sequence of bytes– number by byte #

(not segment #)



data segment (MSS bytes)SN=?


net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

2


(not segment #)– each segment ≤ MSS B

• Sequence numbers– byte # in byte stream

of 1st byte in segment



data segment (MSS bytes)SN=ISNs


net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=?

3





• ACK numbers– seq # of next byte expected





net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

data segment (MSS bytes)SN=?

4





• ACK numbers– seq # of next byte expected





net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

data segment (MSS bytes)SN=ISNs+MSS

ACK AN=?

5





• ACK numbers– seq # of next byte expected– cumulative ACKs





net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS


ACK AN=ISNs+2MSS

6






what next?



data segment (MSS bytes)SN=ISNs+2MSS



net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS


ACK AN=ISNs+2MSS

7






what next?



data segment (MSS bytes)SN=ISNs+2MSS



net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS


ACK AN=ISNs+2MSS

8ACK AN=ISNs+3MSS






• Out of order arrival– implementation specific




TCP OverviewBidirectional Data Transfer

net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

1

• Data both directions– ACKs returned both ways

what next?



data segment (MSS bytes)SN=ISNc



net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

2

• Data both directions– ACKs returned both ways– may be piggybacked

what next?






net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

3


ACK AN=ISNc+MSS

what next?

• Data both directions– ACKs returned both ways– may be piggybacked– delayed ACK waits briefly






net

SYN (ISNs)

ACK

SYN(ISNc)ACK

C S

ACK AN=ISNs+MSS

4


ACK AN=ISNc+MSS

• Data both directions– ACKs returned both ways– may be piggybacked– delayed ACK waits briefly

• Bidirectional stream– data segments– piggybacked ACKs



End-to-End TransportTL.6.2 TCP Optimisations for High Performance


TL.6.1 TCP overviewTL.6.2 TCP optimisations for high performance



TCP OptimisationsExample7.8 Congestion Control

• Fast recovery– 1/2 congestion window after 3dupACK rather than slow start

• Partial acknowledgement response [Hoe 1996]– 1/2 window reduction only once with partial retransmit ACK– rather than per packet

• ABC: appropriate byte counting for cwnd [Allman 2003]

• RED: random early detection (discard) [Floyd 1993]– discard packets when router queue threshold exceeded– throttle TCP source earlier before congestion occurs

• ECN: explicit congestion notification [Floyd 1994]– use IP ECN to trigger multiplicative decrease



TCP OptimisationsExample7.8 Explicit Congestion Notification

• Network nodes setcongestion indication– hist. ICMP source quench– ECN bits in IP header

• TCP reacts– uses flags for signalling





applicationpayload

(variable length)

CEUAPRSFHL



TCP OptimisationsExample7.8 Congestion Control Research1

• Research optimisations [jury still out]– Pacing [Visweswaraiah 1997]

• spread segment transmissions• rather than burst

– Rate control [Padhye 1998]• rate based on equations that describe TCP behaviour• can also make UDP transmission “TCP friendly”

– ELN: explicit loss notification [Samaraweera 1997]ETEN: explicit transport error notification [Krishnan 2004]

• signal loss due to channel bit errors• loss of ACK is not misconstrued as congestion



TCP OptimisationsExample7.8 Congestion Control Research2

• Research optimisations [jury still out]– Vegas [Brakmo 1995]

• RTT estimate rather than AIMD

– HSTCP (high-speed TCP) [RFC 3649 Floyd 2003]STCP (scalable TCP) [Kelly 2003]

• adaptive congestion window in response to LFP

– XCP (explicit control protocol) [Katabi 2003]• congestion header with cwnd, RTT estimate, feedback field

– and many others…



TCP OptimisationsExample7.8 TCP Implementations

• Tahoe– slow start and congestion avoidance algorithms– fast retransmit after triple duplicate ACKs

• Reno – widely implemented– Tahoe + fast recovery

• NewReno [Floyd 1999]– Reno + partial ACK recovery

• BIC (binary-increase TCP) [Xu 2004]– binary search of window size during steady state– CUBIC variant is now default in Linux kernel > 2.6.19

• CTCP (compound TCP) [Tan 2005]– maintains dual AIMD and Vegas-style delay windows– default in MS-Windows since Vista



TCP OptimisationsHybrid Control

• Dynamic rate control– open-loop rate control modified by network feedback

• example: ATM ABR

• Pacing to reduce burstiness– sender base rate control augments closed-loop control– window transmission spread over RTT– options

• pace initial window, allow ACK self clocking to take over• periodic repacing to compensate for ACK compression• continuous pacing



TCP OptimisationsExample7.9 Header Fields

• TCP/IP headers• bw-×-delay

– fields that limit• sequence space• timer related• window size

• Field predictability– use template for

• constant• predictable

– must compute• highly variable

payload

TCP header 40B

IP header 40B

payload

source port #

rcv window size

destination port sequence number

acknowledgement number

checksum urgent pointer

total length

destination address

frag offset

source address

identifier ver TOS hl

flags TTL header checksum

source port #

rcv window

destination port # sequence number

acknowledgement number

checksum urgent pointer hdr len

total length

destination address

frag offset

source address

identifier ver TOS hl

flags TTL header checksum

UAPRSF hdr len UAPRSF

protocol protocol

constant predictable bw-×-delay sensitive



TCP OptimisationsExample7.9 High-Bandwidth-×-Delay

• High bandwidth-×-delay paths (long fat pipes)problem?




• High bandwidth-×-delay paths (long fat pipes)– problem: large number of bits in flight

Implications to TCP?




• High bandwidth-×-delay paths (long fat pipes)– problem: large number of bits in flight

• Implications to TCP– max window size is 64KB problem? – packet loss has significant impact on goodput– only one RTT measurement per window– sequence numbers limited to 32 bit problem?




• Common optimisations [RFC 1323]– Window scale option [RFC 1323]

• SYN option power-of-2 multiplier to initial window

– RTTM (round-trip time measurement) [RFC 1323]• timestamp to allow RTT measurement

– PAWS (protection against wrapped seq. nos.) [RFC 1323]• 32-bit timestamp augments 32-bit sequence number

– SACK (selective acknowledgement) [RFC 2018]– byte range header options for 3 – 4 selective ACK ranges

– Fast retransmit [RFC 2581]• retransmit on triple duplicate ACKs without waiting for timer



TCP OptimisationsExample7.9 SACK

• TCP provides reliable byte stream– cumulative ACKs perform poorly if loss rate not very low– cumulative ACKs limit ability to reorder

• Selective ACKs [RFC2018]– selective repeat ARQ mechanism

• used in conjunction with cumulative ACK mechanisms– SACK byte range in TCP header option

• Retransmissions triggered by:– holes in SACK ranges (SACK is positive ACK)



TCP OptimisationsExample7.9 SACK Header

• SACK permitted negotiation– kind=4 in SYN

• SACK ranges– kind = 05

• after kind=01 NOP pads– length = 8n+2 B

for n SACK blocks• max of 3 SACK blocks

– assuming another option

– each block:• seq num of 1st byte• seq num of last byte +1

– ack semantics unchanged




applicationpayload

(variable length)

flagsHL

32 bits

seq num after last byteseq num of 1st byte

05 len0101



TCP OptimisationsExample7.9 SACK Operation

• Capability announced with SYN– SACK permitted option 5

• Cumulative ACKs as before– ACK byte in last segment – with no missing prior segments

• Selective ACKs– sent only when non-contiguous

segments received– indicate byte range received– holes between SACK blocks

indicate missing byte ranges



TCP OptimisationsExample7.9 Research Optimisations

• Experimental optimisations– larger initial window ≈ 4KB [RFC 3390]– discussion of initial cwnd of 10 segments on TCPM list– concern about aggressiveness

• Protocol Research– trailer checksum [Bridges 1994]

• hotly debated ID• dismissed by IETF community• question of performance gain vs. implementation pain

– pacing [Partridge 2001]• spreads window transmission within RTT• bandwidth estimation and RTT measurement



TCP OptimisationsExample7.9 Research Optimisations

• Implementation optimisations– header prediction [Jacobson 1990, Pink 1994]

• Implementation research– TCP/IP ILP [Clark 1990]

• complex interactions with cache



End-to-End TransportAdditional Sources

• Michael WelzlNetwork Congestion Control:Managing Internet TrafficWiley 2005http://www.welzl.at/congestion



End-to-End TransportAcknowledgements

Some material in these foils comes from the textbook supplementary materials:

• Kurose & RossComputer Networking:A Top-Down Approach Featuring the Internethttp://wps.aw.com/aw_kurose_network_3

• Sterbenz & TouchHigh-Speed Networking:A Systematic Approach toHigh-Bandwidth Low-Latency Communicationhttp://hsn-book.sterbenz.org

ittc high-performance networkingjpgs/courses/hsnets/lecture-transport-hsn-display.pdf · – error...

Documents