Abstract of thesis entitled

“Congestion Controlfor Transmission Control Protocol (TCP)

in Wireless Networks”

Submitted byChengdi LAI

for the degree of Master of Philosophy

at The University of Hong Kongin August 2011

Congestion control regulates the amount of data traffic injected by end

systems into communication networks, preventing persistent network over-

load. In the Internet, it is typically realized by Transmission Control Protocol

(TCP), which backs off transmission of end systems upon detecting packet

loss, with the assistance of active queue management (AQM) algorithms,

which probabilistically drop packets at the intermediate routers based on the

buffer occupancy.

The thesis adapts the present Internet congestion control to the wire-

less environment that is gradually becoming an indispensable component of

the Internet. Firstly, we propose a novel TCP variant, known as TCP for

non-congestive loss (TCP-NCL), to perform effective congestion control, se-

quencing control, and loss recovery over wireless networks where reordering,

error-prone channels are prevalent. Conventional TCP tends to misinterpret

packet reordering and wireless loss as congestive loss, and back off unnecessar-

ily. We develop a smart TCP sender (STS) model that offers more reliable

signals of packet loss and network congestion over reordering, error-prone

channels. TCP-NCL is devised as a practical approximation of STS. Our

simulation results show that TCP-NCL can accurately differentiate among

the occurrences of packet reordering, congestive loss, and non-congestive loss,

and can thus serve as the aforementioned unified solution. Deployment of

TCP-NCL requires sender-side modification only.

Secondly, we have performed a series of simulation experiments to inves-

tigate the performance of AQM under wireless losses. Our results report that

random early detection (RED), a classical AQM, fails to maintain a stable

backlog against time-varying wireless packet error rate (WPER), resulting

in poor delay performance and unfairness towards bursty traffic. We apply

the internal model principle to devise a family of AQM enhancements, which

can automatically adjust packet dropping rate of AQM to compensate for

wireless losses, and thus stabilize the backlog. We further devise the integral

controller (IC) as an embodiment of the principle. Our simulation stud-

ies show that IC is robust against time-varying WPER, even in challenging

scenarios like fat pipes and perturbing traffic flows.

In summary, our proposed TCP and AQM enhancements can perform

effective congestion control over the heterogeneous wired/wireless Internet,

and help the latter to operate in the optimal region of low delay with small

jitters and high throughput.

(372 words)

Congestion Control

for Transmission Control Protocol (TCP)

in Wireless Networks

by

Chengdi LAI

B.Eng. (EComE) H.K.

A Thesis Submitted in Partial Fulfilment of the Requirements

for the Degree of Master of Philosophy

at The University of Hong Kong

August 2011

Declaration

I declare that the thesis and the research work thereof represents my own

work, except where due acknowledgement is made, and that it has not been

previously included in a thesis, dissertation or report submitted to this Uni-

versity or to any other institution for a degree, diploma or other qualifica-

tions.

Signed .....................................................................

Chengdi LAI

i

Acknowledgements

The thesis marks a first milestone for my research journey, which started

three years ago when my current advisors, Prof. Victor On-Kwok Li and

Dr. Ka-Cheong Leung, introduced to me the prosperous realm of networking

research. Over the years, their pursuit of excellence, encouragement of inno-

vation, and emphasis on rigorousness have greatly influenced me. Prof. Li’s

great vision and innovative insights have kept me inspired in pursuit of re-

search work of profound impact. Dr. Leung has patiently led me to formulate

my research methodology and taste via the numerous email exchanges and

conversations. I would like to express my earnest gratitude to them.

I owe my special thanks to Prof. Steven H. Low and Prof. Ricky Y.K. Kwok

for serving as my thesis examiners. Their constructive comments assist me

to improve the quality of this thesis. I thank Dr. K.P. Chan for chairing my

thesis committee.

I have been very fortunate to encounter several wonderful mentors during

the past five years in the University of Hong Kong, spanning my B.Eng. and

M.Phil. programs. I appreciate Prof. Kwok, Prof. Lawrence K. Yeung, and

Dr. Vincent Tam for their insightful advice on career and life. I would also

like to take the opportunity to thank Fr. John Coghlan for providing me three

ii

years’ enjoyable residence in Ricci Hall during my undergraduate study.

The high level of professionalism of the support staff in the Department

of Electrical and Electronic Engineering, the University of Hong Kong, has

greatly facilitated my research work. In particular, I would like to thank

Ms. Danita Lee, Ms. Julie Hung, and Ms. Lily Lo for their assistance on ad-

ministrative issues, and Mr. C.L. Chan and Mr. Andy Fok for their technical

support.

I am obligated to my colleagues, Dr. Albert Y.S. Lam, Dr. Guanghua Yang,

Dr. Yanhui Geng, Mr. Ze Zhao, Mr. Yile Yang, Mr. Jin Xu, Mr. Xiaoguan Fang,

Dr. Qiong Sun, Dr. Jun Hong, Dr. Kuang Xu, and Dr. Jialing Xu, for their

support and assistance.

Last, but not least, I am deeply indebted to my parents and my girlfriend,

Miss Shuzhen Shen, for their love, and for always being there for me through

thick and thin.

iii

Contents

Declaration i

Acknowledgements ii

Contents iv

List of Figures vi

List of Tables viii

1 Introduction 11.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Internet Congestion Control . . . . . . . . . . . . . . . . . . . 2

1.2.1 TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 AQM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Wireless Channels . . . . . . . . . . . . . . . . . . . . . . . . . 101.4 Thesis Statement and Outline of Thesis . . . . . . . . . . . . . 121.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 14

2 Related Work 162.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Wireless TCP Enhancements . . . . . . . . . . . . . . . . . . 162.3 Design and Analysis of AQM . . . . . . . . . . . . . . . . . . 192.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 21

3 A Serialized-Timer Approach for Enhancing Wireless TCP 233.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 233.2 STS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2.1 RD Timer . . . . . . . . . . . . . . . . . . . . . . . . . 313.2.2 CD Timer . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3 TCP-NCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.3.1 NRU Process . . . . . . . . . . . . . . . . . . . . . . . 43

iv

3.3.2 RD and CD Timers . . . . . . . . . . . . . . . . . . . . 463.3.3 RTO Timer . . . . . . . . . . . . . . . . . . . . . . . . 503.3.4 Kernel Implementation . . . . . . . . . . . . . . . . . . 51

3.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . 543.4.1 Non-congestive loss . . . . . . . . . . . . . . . . . . . . 563.4.2 Packet Reordering . . . . . . . . . . . . . . . . . . . . 613.4.3 Congestive Loss . . . . . . . . . . . . . . . . . . . . . . 63

3.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 68

4 Enhancing AQM to Combat Wireless Losses 694.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 694.2 AQM under Wireless Losses . . . . . . . . . . . . . . . . . . . 724.3 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.4 Robust AQM based on Internal Model Principle . . . . . . . . 82

4.4.1 IC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.5 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . 94

4.5.1 One Wireless Bottleneck Link . . . . . . . . . . . . . . 944.5.2 Two Wireless Bottleneck Links . . . . . . . . . . . . . 103

4.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 105

5 Conclusions and Future Research 1075.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 1075.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1075.3 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . 111

5.3.1 A Theoretical Framework for Wireless TCP . . . . . . 1115.3.2 A Unified AQM Solution for Wireless Networks . . . . 113

5.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 115

Bibliography 116

Acronyms 125

v

List of Figures

1.1 Fast retransmit and fast recovery. . . . . . . . . . . . . . . . . 5

3.1 A flowchart for STS model. . . . . . . . . . . . . . . . . . . . 263.2 Evaluation of τ ∗

CDi. . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3 Software architecture for TCP. . . . . . . . . . . . . . . . . . . 523.4 The network topologies used for performance comparison . . . 533.5 Goodput performance against packet error rate over infrastructure-

based wireless network. . . . . . . . . . . . . . . . . . . . . . . 553.6 Goodput performance against delay of the wireless link over

infrastructure-based wireless network. . . . . . . . . . . . . . . 573.7 Goodput performance against bandwidth of the wireless link

over infrastructure-based wireless network. . . . . . . . . . . . 583.8 Goodput performance over multihop ad-hoc wireless network . 623.9 Jain’s fairness index (J) over wired network with a dumbbell

topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.10 GBR over wired network with a dumbbell topology. . . . . . . 65

4.1 A wireless bottleneck link. . . . . . . . . . . . . . . . . . . . . 724.2 Queue length dynamics under varying wireless losses in the

dumbbell topology with TCP NewReno/RED. . . . . . . . . . 744.3 Queue length dynamics under varying wireless losses in the

dumbbell topology with TCP-NCL/RED. . . . . . . . . . . . 754.4 Control block diagram of the linearized congestion control sys-

tem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.5 Two wireless bottleneck links. . . . . . . . . . . . . . . . . . . 924.6 Comparison of queue length dynamics with same configura-

tions as in Figure 4.2. . . . . . . . . . . . . . . . . . . . . . . . 974.7 Comparison of queue length dynamics under different wireless

losses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.8 Comparison of queue length dynamics with different bottle-

neck link parameters. . . . . . . . . . . . . . . . . . . . . . . . 100

vi

4.9 Comparison of queue length dynamics with the presence ofother traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.10 Comparison of queue length dynamics in the topology of twowireless bottleneck links. . . . . . . . . . . . . . . . . . . . . . 104

vii

List of Tables

3.1 General notations. . . . . . . . . . . . . . . . . . . . . . . . . 293.2 Notations for events. . . . . . . . . . . . . . . . . . . . . . . . 303.3 Network configurations for the infrastructure-based wireless

network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.1 Network configurations for the single wireless bottleneck net-work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

viii

Chapter 1

Introduction

1.1 Chapter Overview

The thesis adapts the present Internet congestion control to the wireless en-

vironment that is gradually becoming an indispensable component of the

Internet. In this chapter, we bring the background materials on Internet

congestion control and the challenges it encounters in the wireless environ-

ment. On this basis, we present the thesis statement and outline the thesis.

Section 1.2 succinctly describes the present Internet congestion control, as

fulfilled by Transmission Control Protocol (TCP) with the assistance of active

queue management (AQM) algorithms. Section 1.3 discusses the challenges

posed by the wireless environment on the proper functioning of congestion

control. Section 1.4 presents our thesis statement, and outlines the thesis.

Section 1.5 briefly summarizes the chapter.

1

1.2 Internet Congestion Control

The modern age has witnessed the great success of communication networks

in vastly facilitating information exchange and resource sharing among ge-

ographically distributed people. Packet-switched networks are among the

most common communication networks. In a packet-switched network, end

systems are inter-connected via a group of packet switches such as routers

and network hubs. An end system groups its transmitted data into separate

segments, or packets, before delivering it over the network. Upon receiving

a packet from end systems or other packet switches, a packet switch places

the packet in its backlog, a queue of packets waiting for forwarding. It for-

wards the packet to the next switch towards the destination or directly to

the destination when the outgoing link becomes free for transmission.

The Internet is a global system of packet-switched network. Using the

standardized Transmission Control Protocol/Internet Protocol (TCP/IP) pro-

tocol suite, the Internet inter-connects numerous heterogeneous networks and

is the largest network of networks.

Network congestion or network overload occurs when the amount of data

traffic injected into the network by end systems exceeds its transmission

capacity. At packet switches, packets persistently arrive faster than can be

forwarded. The backlog grows in length, elongating the time packets spent in

queueing. This queueing delay adds to the end-to-end delay experienced by

packets. When the backlog further grows, it can exceed the buffer size of the

packet switch, thereby discarding arrived and/or backlogged packets. Under

persistent network overload, the high levels of transmission delays and losses

2

experienced by packets result in poor data delivery services to end systems

and hence users.

Historically, severe network congestion, known as congestion collapse [56],

occurred to the Internet in the 1980s [30]. Due to poorly regulated transmis-

sion of end systems, packets were held in long backlogs at packet switches.

End systems deemed these packets as being lost and retransmitted them,

exacerbating network congestion and wasting network capacity for transmit-

ting duplicate data. Consequently, the network observed up to three orders

of magnitude decrease in the rate of successful data delivery. Moreover, the

end-to-end delay became excessively long.

Congestion control regulates the transmission rate of end systems, pre-

venting persistent network overload. It is therefore crucial to ensuring the

proper operation of the network. In the present Internet, the task of con-

gestion control is implemented by TCP running at end systems with the

assistance of AQM running at routers in a distributed manner. In the fol-

lowing two subsections, we review TCP and AQM, respectively.

1.2.1 TCP

Transmission Control Protocol (TCP) [4, 30, 60] is the de facto standard for

the Internet. Using the connection-less, unreliable data delivery services of

Internet Protocol (IP), it provides end-to-end, in-order, reliable data deliv-

ery services to various applications like File Transfer Protocol (FTP), Telnet,

and Hypertext Transfer Protocol (HTTP). Initially, TCP did not incorpo-

rate end-to-end congestion control [60]. The latter was first installed in TCP

3

around 1988 in an successful attempt to combat the occurrence of conges-

tion collapse, and has since become an essential element of TCP, central to

ensuring the stable, efficient operation of the present Internet.

TCP is connection-oriented. A TCP connection is set up between two

end systems when they agree on a few parameter settings and each reserve

resources like memory space for the transmission via TCP, after which the

transmission begins. The transmission ends when the connection is closed

and the resources reserved by end systems will be released.

The TCP connection is duplex. Both end systems can transmit and re-

ceive data. Nevertheless, we can conceptually focus on one-way transmission

over a TCP connection to avoid unnecessary complications. Moreover, in re-

ality the bulk of data transfer over a TCP connection takes place in one way,

say, from a web server to a web client. Correspondingly, the end systems

that transmits and receives data are referred to as TCP sender and TCP

receiver, respectively.

A TCP sender consecutively assigns byte numbers to its transmitted data

bytes, and a TCP receiver expects the number of received data bytes to be

consecutively ordered. When receiving a data packet, a TCP receiver noti-

fies the sender of successfully delivered data via an acknowledgment (ACK)

packet that contains acknowledgment number, namely the number of next

byte it expects. The time elapsed between when a TCP sender transmits a

data packet and when the packet is acknowledged is known as round-trip time

(RTT). An example is illustrated in Figure 1.1. The right and left arrows

denote data packets and ACK, respectively. The first data packet contains

bytes numbered 201 to 300, as indicated by the two numbers above its cor-

4

Sender Receiver

301:400

201:300

401:500

501:600

601:700

ACK 301

301:400

ACK 301

ACK 301

ACK 301

Figure 1.1: Fast retransmit and fast recovery.

responding arrow. Its arrival at the receiver is acknowledged to the sender

via the first ACK containing acknowledgment number 301. These in-order

data bytes will be delivered to the applications after some extra processing.

The next packet arrived contains bytes numbered 401 to 500 and is thus out

of order. In this case, the receiver immediately sends back an ACK with

acknowledgment number 301. This duplicate acknowledgement contains the

same acknowledgment number as the previous ACK. The out-of-order data

packet will be buffered by the receiver until the arrival of all packets before it,

when the consecutively ordered data will be further processed and delivered

to the application. In-order data delivery is thus fulfilled. We come back to

the example shortly.

A TCP sender applies a window-based control to regulate the amount

of outstanding data, namely the data that has been transmitted but not

yet acknowledged by the receiver. The outstanding data consists of data

5

buffered by the receiver, data packets in transit, and data packets whose cor-

responding ACKs are in transit. Its amount shall not exceed the minimum of

receiver’s advertised window (awnd) and sender’s congestion window (cwnd).

awnd is set by the receiver and fed back to the sender via ACK for indicating

the available spaces in receiver buffer. It prevents a fast sender from over-

whelming a slow receiver. cwnd is determined by the sender, representing

the latter’s estimation of the load affordable by the network.

The acknowledgment scheme and window-based transmission provides a

basis for realizing the important features of end-to-end congestion control

and reliable transmission. In fulfilling the former feature, cwnd has to be

dynamically adjusted throughout the TCP session in accordance with the

ever-changing available network capacity. Signals of network underload and

overload are needed for triggering the increase and decrease of cwnd, respec-

tively. In fulfilling the latter feature, signals of packet loss are needed for

triggering retransmission of lost data packets to ensure the eventual delivery

of every data byte.

A TCP sender uses successful, prompt data delivery as a signal that the

network can potentially carry more load than currently offered. The slow

start and congestion avoidance algorithms are devised on this premise, and

govern the increment of cwnd when it is smaller than and greater than slow

start threshold value (ssthresh), respectively. Slow start increases cwnd by

one data packet size upon receiving an non-duplicate ACK that acknowl-

edges the successfully delivery of new data. Congestion avoidance increases

cwnd by one data packet size per RTT if all data packets transmitted in the

previously RTT are acknowledged.

6

A TCP sender uses the same set of signals for indicating packet loss

and network overload. Two types of signals are used, namely retransmission

timeout (RTO) and triple duplicate ACKs. A retransmission timer is started

when a data packet is first injected into the network, and will timeout if the

packet is still not acknowledged when the timer expires. Upon the occurrence

of an RTO, all the outstanding packets will be retransmitted. At the same

time, the network is deemed severely congested and cwnd will be forced to

reopen from one packet size by employing the slow start algorithm.

At the receiver side, each out-of-order packet will trigger a duplicate

ACK. At the sender side, when the number of duplicate ACKs reaches a

certain threshold value, say, three, the packet expected by the receiver will

be deemed as lost due to congestion. Fast retransmit and fast recovery will

be activated, retransmitting the packet expected by the receiver and halving

cwnd. ssthresh is then set to cwnd so that cwnd will reopen by employing the

congestion avoidance algorithm. Therefore, triple duplicate acknowledgment,

a direct signal of out-of-order packet events, is further used as an indication

of congestive packet loss. In the example illustrated in Figure 1.1, the sec-

ond, third, and fourth data packets arrived at the receiver are all out of order

since they do not contain the byte numbered 301 expected by the receiver,

each triggering a duplicate ACK. The sender activates fact retransmit and

fast recovery upon receiving the third duplicate ACK.

7

1.2.2 AQM

A queue management algorithm manages the backlog at routers, determining

when and how packets should be dropped. Conventionally, a router does

not drop packets until the backlog exceeds the buffer size, forcing discard of

incoming data packets, namely drop tail, or packets in the backlog. The TCP

data flows will keep on increasing cwnd and generate more data packets in

transit until a router buffer reaches its limit. Consequently,

1. The end-to-end delay for data delivery is significantly increased.

2. Insufficient buffer space incurs a much higher packet loss rate for bursty

traffic that tends to fill up the buffer space.

On the other hand, when buffer overflow occurs, the situation is likely

to persist for a while due to the delay of TCP in detecting the congestion.

Packets belonging to different TCP flows will be dropped simultaneously. All

or most TCP flows will be triggered to back off, leaving the buffer largely

unoccupied. This TCP synchronization [80] in turn creates another two

problems:

1. The backlog fluctuates severely between the full buffer size immedi-

ately before a TCP synchronization and the low level immediately af-

terwards. This creates large jitters in the queueing delay and thus

end-to-end delay.

2. The backlog may become empty after a TCP synchronization, leaving

the outgoing link idle.

8

Active queue management (AQM) [11] was first proposed in the 1990s

to tackle the aforementioned problems with conventional queue management

algorithms. AQM probabilistically drops packets before buffer overflow based

on a dropping rate determined from the past and/or present queue lengths.

The backoff of different TCP flows are therefore desynchronized and spread

across time. With the coordination of an effective AQM design, the aggregate

transmission of the TCP flows can keep a router buffer occupied with a small,

stable backlog. This significantly reduces the end-to-end delay, smooths out

delay jitters, leaves sufficient buffer space to absorb bursty traffic, and keeps

the bottleneck link backlogged and thus fully utilized.

Random early detection (RED) [11, 22] is a most classical AQM algo-

rithm. It maintains an average queue size r, say, by computing the running

average of sampled backlog size b with queue weight α:

r ← (1− α)r + αb (1.1)

The packet dropping probability p is zero if r is less than a preset threshold

value minThresh, and one if r is greater than the other preset threshold value

maxThresh. When r is between minThresh and maxThresh, p is determined

as a linear function of r:

p← pMaxr −minThresh

maxThresh−minThresh(1.2)

RED performs probabilistic packet dropping based on p upon each packet

arrival.

9

1.3 Wireless Channels

The next generation network is expected to be a heterogeneous network of

networks, including both wired and wireless components. Wireless networks

may extend beyond access networks to backhaul networks [29], and even

backbone networks. The present Internet congestion control is designed

based on the premise of conventional wired network, where data are mostly

correctly, orderly delivered. It encounters significant challenges in the wire-

less environment due to the latter’s more unpredictable nature. In particular,

non-congestive packet losses and packet reordering, while being rare events

in conventional wired networks, are common in wireless networks [44].

As compared with the wired media, the wireless medium provides much

more lossy physical links for data transmissions. Signals propagating over

wireless channels suffer from degradation, interference, and noise. Packets

received may be damaged beyond the recovery capability of error control

codes, if any. These packets are thus discarded, leading to the occurrence of

non-congestive packet losses.

Packet reordering refers to the disruption of the packet order of a TCP

flow. Despite conventional beliefs that packet reordering is a transient or

pathological network behavior, it is in fact persistently observed over modern

networks and can be caused by a myriad of reasons [44, 45]. Due to the high

transmission error rates and, in some cases, mobility in wireless networks,

packet reordering is further increased substantially when the transmission

medium evolves from physical cables to wireless. Specifically, four causes of

packet reordering are commonly observed over wireless networks, including:

10

Link-layer retransmission (LLRTX): To combat high transmission error

rates of wireless channels, some link layer retransmission mechanisms [7, 28]

have been proposed to locally retransmit damaged packets at the link layer.

As a side effect of local retransmission, the packet order of a flow is altered.

Path change: Over a mobile ad-hoc networks (MANET), a TCP flow

traverses a number of wireless nodes. The transmission path of the flow may

be altered when some nodes move. The RTT of the connection may change

too. Consequently, it is possible that some packets transmitted after the path

change arrive at the receiving end before those packets transmitted prior to

the path change.

Hand-off: In an infrastructure-based network, the complete transmission

coverage of an entire area is achieved cooperatively by a set of base stations.

When a mobile client moves from the radio range of one base station to that

of another, a hand-off between these two base stations occurs, changing the

transmission path for the flow. The resulting variation in RTT may lead to

the occurrence of packet reordering.

Packet-level multi-path routing: In wireless mesh network (WMN) and

many other networks of rich connectivity, multi-path routing have been pro-

posed for load balancing. When it is carried out at the packet level, packets

may be reordered due to the difference in time delays of distinct paths.

The present Internet congestion control therefore encounters two-fold

challenges in the wireless environment. First, TCP sender triggers fast re-

transmit and fast recovery upon triple duplicate ACKs. While the latter is

dominantly caused by congestive loss over conventional wired networks, it is

often triggered by additionally packet reordering and non-congestive loss in

11

the wireless environment. Thus, TCP tends to falsely retransmit packets and

reduce cwnd from time to time, injecting duplicated bytes into the network

and keeping cwnd unnecessarily small. Consequently, the available network

resources are wasted and under-utilized.

Second, the extra source of packet losses, namely wireless losses, may

interfere with the normal operation of AQM, which communicates implicitly

with TCP via active packet drops. This casts doubt on whether AQM can

continue to fulfill the objective of maintaining a stable, small backlog under

wireless losses.

1.4 Thesis Statement and Outline of Thesis

Our thesis statement is to demonstrate that the present Internet conges-

tion control can be systematically adapted to the wireless environment at a

minimal disruption and deployment cost.

Presently, the Internet congestion control is generally loss-based. Packet

drops are used as a signal of network overload, based on which TCP acti-

vate congestion response and AQM probabilistically trigger the congestion

responses of different TCP connections. We devise our enhancements within

the scope of loss-based TCP and AQM. This brings the inherent advantage

of minimizing both the deployment cost and the potential disruptions to the

existing congestion control deployment.

On the TCP side, we have developed a smart TCP sender (STS) model,

that uses a serialized-timer structure to offer more reliable signals of packet

loss and network congestion over general (reordering), error-prone channels.

12

Timers are serialized in the sense that the second timer is started upon the

expiration of the first timer. We have devised a novel TCP variant, known

as TCP for non-congestive loss (TCP-NCL), as a practical approximation

of the STS. TCP-NCL can accurately differentiate among the occurrences

of packet reordering, congestive loss, and non-congestive loss, and can thus

serve as a unified solution for effective congestion control, sequencing control,

and loss recovery over wireless networks. Deployment of TCP-NCL requires

sender-side modification only.

On the AQM side, we have investigated the performance of AQM under

wireless losses using simulation experiments. Our results report that RED

fail to maintain a stable backlog against time-varying wireless packet error

rate (WPER). We have applied the internal model principle to devise a family

of AQM enhancements, which can adjust packet dropping rate to compen-

sate for wireless losses without requiring exact knowledge of WPER, and

thus stabilize the backlog. We further devise integral controller (IC) as an

embodiment of the principle, which performs robustly against time-varying

WPER in various challenging scenarios like fat pipes and perturbing traffic

flows.

In summary, our proposed TCP and AQM enhancements can perform

effective congestion control over the heterogeneous wired/wireless Internet,

and help the latter to operate in the optimal region of low delay with small

jitters and high throughput.

The rest of this thesis is organized as follows.

Chapter 2 presents a literature review of existing work on wireless TCP

enhancements and on the design and analysis of AQM. It puts our work in

13

perspective.

Chapter 3 develops the STS model. TCP-NCL is then devised as a prac-

tical approximation of the STS model. Extensive simulation experiments are

performed to examine the performance of TCP-NCL against packet reorder-

ing, non-congestive packet loss, and congestive loss.

Chapter 4 performs simulation-based investigation of AQM under wire-

less losses. A linear model of TCP/AQM under wireless losses is formulated.

On this basis, the internal model principle is applied to devise a family of

AQM enhancements that stabilize the backlog. IC is introduced as an em-

bodiment of the principle, with a set of design rules developed. Extensive

simulations are conducted to evaluate IC against time-varying wireless losses

under various network scenarios, like fat pipes and perturbing traffic flows.

Chapter 5 concludes the thesis. It summarizes our major contributions,

and presents an overview of future research directions.

The work presented in this thesis has in parts been published in [39, 38,

36, 37, 47].

1.5 Concluding Remarks

In this chapter, we bring the background materials on Internet congestion

control and the challenges it encounters in the wireless environment. Con-

gestion control is crucial to ensuring the stable, efficient operation of packet-

switched networks like the Internet. The present Internet congestion control,

as fulfilled by TCP and AQM, performs well for conventional wired networks.

However, it encounters significant challenges in wireless networks, in large

14

part due to the frequent occurrences of packet reordering and non-congestive

loss.

We suggest loss-based TCP and AQM enhancements to adapt the Internet

congestion control to the wireless environment at a minimal deployment cost

and disruption.

15

Chapter 2

Related Work

2.1 Chapter Overview

In this chapter, we conduct a literature survey of related work performed

to date. Section 2.2 studies existing wireless TCP enhancements that adapt

TCP to the general (reordering), error-prone channels. Section 2.3 studies

existing work on the design and analysis of AQM. Section 2.4 summarizes

limitations of these and puts our work in perspective. The work presented

in this chapter has in parts been published in [39, 37].

2.2 Wireless TCP Enhancements

In this section, we summarize existing work for adapting TCP to perform

congestion control and loss recovery over general error-prone transmission

channels in wired/wireless networks. We confine ourselves to the scope of

loss-based TCP variants that use congestive loss as signals of network con-

16

gestion. Some TCP variants propose inferring network loading condition

based on explicit network feedbacks [32, 75] or RTT [12, 71] for adapting

TCP to high-speed networks. Yet, the former requires universal support by

all the traversed intermediate routers, which can hardly be attained due to

the heterogeneity of the Internet. The latter, known as delay-based TCP

variants, may not work well over reordering networks, where RTT can be

varied by factors other than network loading. Besides, they may not share

the bandwidth fairly with loss-based TCP in widest current usage [55, 67].

AVG, DEL, EWMA, and INC in [9], RR-TCP [81], TCP-DCR [8], and

TCP-PR [10] propose to abandon the fixed triple duplicate ACKs as sig-

nals of congestive packet loss. Instead, they proactively postpone a packet

retransmission until a corresponding timer expires or when the number of du-

plicate ACKs received reaches an adaptively evolved threshold value. This

is considered a more reliable signal of packet loss over reordered channels.

However, these variants use the same set of signals for activating congestion

response, and are incapable of differentiating between congestive and non-

congestive losses. They thus have to rely on LLRTX for making the on-going

random packet losses over wireless links transparent to the transport layer.

DSACK TCP [23], TCP-DOOR [69], and TCP-Eifel [52] are designed

with the premise that triple duplicate ACKs may be unreliable indications of

congestive loss. Their approaches differ, however, in that they try to detect

spurious retransmission after activating fast retransmission and fast recovery

upon the arrival of triple duplicate ACKs1. Upon successful detection, cwnd

1The same design philosophy is also employed by F-RTO [62], which focuses on detect-ing spurious retransmission triggered by RTO. However, false RTO is not the major reasonfor performance deterioration over wireless networks. We thus do not include F-RTO as

17

will be recovered immediately or via the slow start process. TCP-DOOR

also disables congestion control for a time interval as it assumes out-of-order

events are often caused by path change. However, these variants are generally

unable to recover cwnd unnecessarily reduced in the event of non-congestive

packet losses. Moreover, as shown by our simulation results, TCP-DOOR

tends to excessively disable congestion control actions under persistent packet

reordering, resulting in substantial deterioration in performance.

JTCP [74], TCP-Veno [24], and TCP-Westwood (TCPW) [13] focus on

differentiating congestive and non-congestive losses by using the estimated

network load upon the arrival of triple duplicate ACKs, which is still treated

by them as a credible signal of packet loss for triggering retransmission.

One of their major merits is that the binary signal of packet loss no longer

dictates the activation of congestion response. Instead, the signal is combined

with the estimated network load to decide by how much cwnd should be

reduced. Yet, the inherent assumption of nearly in-order channel obviously

limit the applicability of these variants over wireless networks and many

other modern networks, where packet reordering are common. Particularly,

persistent reordering may significantly affect the adaptation of cwnd to the

available network bandwidth by frequently activating minor reductions of

cwnd and triggering false retransmissions.

TCP-Probing [35] applies a different approach for differentiating conges-

tive and non-congestive losses upon inferring packet loss from the arrival

of triple duplicate ACKs. After cwnd is halved in response to the inferred

packet loss, probing data packets are injected into the network. The inferred

a wireless TCP enhancement.

18

packet loss is categorized as non-congestive if the RTT of the first two ac-

knowledged probing packets are smaller than the best RTT, or the minimum

of the measured RTTs during the TCP session. Again, TCP-Probing will

trigger frequent false fast retransmissions due to persistent packet reorder-

ing. Furthermore, the stored best RTT may fail to capture network changes.

Specifically, when a path change or handoff occurs within a TCP session and

increases the minimum attainable RTT, the stored best RTT will not be up-

dated. Consequently, it may become impossible for the probe packets to be

acknowledged within the best RTT even under light network loading.

The existing unified solutions require information and/or modifications of

the network protocol stack beyond the transport layer. ATCP [48] introduces

an ATCP layer between TCP and IP. The new layer switches TCP among

various pre-defined states in accordance with the network condition, trying

to avoid spurious packet retransmission and congestion response.

In summary, there is a lack of a unified solution at the transport layer with

reliable signals of network overload and packet loss for performing congestion

control and loss recovery over general, error-prone transmission channels.

Such unified solution is highly desirable, as it adapts TCP to a wide range

of wireless and, in some cases, wired transmission channels at a minimal

deployment cost.

2.3 Design and Analysis of AQM

In this section, we summarize the related work on the design and analy-

sis of AQM, with special emphasis on those applying the control-theoretic

19

approaches and/or adapting AQM to wireless networks.

First proposed in 1993, RED [22] demonstrates the inherent advantage

of AQM in helping the network to operate in the optimal region of high

throughput and low delay. However, parameter tunings and new variants of

RED invariably adopted the trial-and-error approach due to the difficulties

in understanding the dynamics of TCP/AQM.

The fluid models of TCP/AQM, as presented in, say, [33, 50, 53], provide

a basis for systematic design and analysis of AQM. The optimization-based

approach [33, 49] interprets TCP/AQM as a distributed algorithm for solv-

ing a utility maximization problem, subject to capacity constraints. The

major focus is on the optimal solutions attained at the equilibria. However,

transient responses of AQM are mostly not captured.

The control-theoretic approach views Internet congestion control as a non-

linear control system. The system is first linearized around its equilibrium

so as to analyze the dynamics of TCP/AQM around the equilibrium. This

enables parameter tuning of RED and design of new AQM algorithms based

on frequency-domain analysis [26, 27, 51, 79], which is powerful in improving

the transient response of AQM and guaranteeing the stability of the linear

system. Interested readers can refer to [65] for a survey. The work in this

area closest to our study is [26], which designs a proportional-integral (PI)

controller for the general network topology. Generally, the IC in our work

can be considered a special case of the PI controller with the proportional

part being set to zero. However, the PI controller considered in [26] assumes

the proportional part to be non-zero, so that it cannot be reduced to the IC.

Moreover, we apply the IC to combat wireless losses, whereas [26] focuses on

20

designing a PI controller in wired networks.

The global stability and region of attraction of Internet congestion control

have been studied in the context of a non-linear system. Please refer to [20,

72, 78] and the references therein. In a nutshell, these are concerned with

whether the system will converge to the equilibrium starting from a feasible

initial status not necessarily close to the equilibrium [34]. Among them,

Fan et al. [20] examine the robustness of the network flow control against

disturbances. Yet, the AQM in [20] is static rate-based, which determines the

packet dropping rate as a function of the instantaneous incoming data rate.

Our work considers buffer-based AQM variants, which are widely used in the

Internet to determine the packet dropping rate from the queue length.

In adapting AQM to wireless networks, the inter-flow fairness in ad-hoc

networks can be improved [76] by virtually aggregating packets queued in

those interfering nodes to one queue, and applying AQM to manage the

queue.

2.4 Concluding Remarks

In this chapter, we have summarized existing work on wireless TCP enhance-

ments and on the design and analysis of AQM. We find that there is a lack of

transport layer unified solution for both packet reordering and non-congestive

loss. Such solution is highly desirable for adapting TCP to the reordering,

error-prone communication channel prevalent in wireless environment at a

minimal deployment cost.

The performance of AQM in wireless environment remains to be thor-

21

oughly investigated. In particular, it is largely unexplored in the literature

whether AQM can maintain a stable, small backlog under wireless losses. If

not, an AQM enhancement for stabilizing the backlog would be highly de-

sirable so as to improve the delay performance and fairness towards bursty

traffic in wireless networks..

22

Chapter 3

A Serialized-Timer Approach

for Enhancing Wireless TCP

3.1 Chapter Overview

In this chapter, we develop a unified solution for performing effective con-

gestion control, sequencing control, and loss recovery over communication

networks with general (reordering), error-prone transmission channels like

wireless networks. The proposed modifications involved are limited to sender-

side TCP only, thereby facilitating possible future wide deployment. As we

find in the literature review in Chapter 2, most existing TCP variants only

focus on tackling with the presence of either non-congestive packet loss or

packet reordering. A few unified solutions for both problems generally re-

quire information and modifications beyond the scope of the transport layer,

hindering possible future wide deployment.

We develop an ideal TCP sender model, known as smart TCP sender

23

(STS), for constructing more reliable signals of packet loss and network con-

gestion over a general error-prone channel for activating packet retransmis-

sion and congestion response, respectively. The two signals of packet loss

and network congestion are the expirations of two serialized timers. The

first timer is started when a packet is first injected into the network, while

the second timer is started upon the expiration of the first timer. The timer

expiration periods have been optimized for minimizing quantified risks asso-

ciated with spurious congestion response and excessively delayed congestion

response.

We further devise a new TCP variant, known as TCP for non-congestive

loss (TCP-NCL), as a practical approximation of STS. TCP-NCL realizes

the serialized-timer structure economically by using timestamps. Deploy-

ment of TCP-NCL requires maintenance of a few unused bits in the existing

data structure for TCP and a few extra variables only. The maintenance is

performed upon arrival of acknowledgement packet.

We have performed extensive simulation experiments to examine the per-

formance of TCP-NCL and compare it with several well-known TCP variants

in the literature using Network Simulator (ns) Version 2.29 [19]. Our results

report that TCP-NCL attains significant performance improvement over gen-

eral, error-prone communication networks, thereby demonstrating robustness

against packet reordering and non-congestive packet loss. At the same time,

multiple competing TCP connections, with some employing TCP-NCL and

some employing SACK TCP, the popular standardized TCP variant, are

capable of sharing bandwidth fairly over conventional wired networks. TCP-

NCL can thus serve as the aforementioned unified solution for performing

24

effective congestion control, sequencing control, and loss recovery over com-

munication networks with general, error-prone transmission channels.

Section 3.2 develops the STS model. Section 3.3 presents TCP-NCL

based on the analytical results of the STS model. Section 3.4 examines the

performance of TCP-NCL and compares it with some popular TCP variants.

Section 3.5 concludes and discusses some possible extensions of this work.

The work presented in this chapter has in parts been published in [39, 38, 36].

A U.S. patent has been filed for the work [47].

3.2 STS Model

In constructing an aforementioned unified solution, we start from the follow-

ing two initial observations:

Nature of signals: Over general communication channels, there are two

prevalent approaches for inferring the occurrence of packet loss based on: 1)

the number of consecutively received duplicate ACKs reaching an adaptively

evolved threshold value or, 2) a timer1 expiration. However, one major con-

cern with the former approach is that a TCP sender may not be able to

accumulate enough duplicate ACKs to perform loss recovery because of the

insufficient number of the outstanding packets. Special measures, like limited

retransmit [2], would have to be incorporated in order to alleviate the prob-

lem by sending extra data packets upon the arrival of duplicate ACKs. By

contrast, a more elegant and robust approach for estimating the probability

1The timer is different from the retransmission timer, which triggers a packet retrans-mission and activates the slow start algorithm when it expires. TCP-PR is an exceptionsince it uses a modified retransmission timer so that fast retransmit and fast recovery areactivated upon the timer expiration.

25

Update

Expires?i CD

PiSend Packet

Pi &Resend Packet CD

Expires?

Yes

Yes

No

No

No

Yes

No

Yes

Activate Congestion Control

iRD

(Added by TCP−NCL)

& Start Timer iRD

Start Timer i

ACKi Received?

ACKi Received?

RTT

Figure 3.1: A flowchart for STS model.

26

of packet loss is to measure the time elapsed since a packet transmission (or

some other similar instances) and refer to the past recent history of RTT.

Separation of signals: Over error-prone channels, signals of packet loss

alone can only serve as prior signals of network congestion. It is necessary

to take additional information into account to confirm whether the inferred

loss can be categorized as a congestive loss.

In place of the conventional approach of activating fast retransmission and

fast recovery simultaneously, we propose to delay the decision on whether to

activate a congestion response behind that of a packet retransmission for a

short time period. The motivation behind doing so is two-fold. First, infor-

mation inferred by the occurrences of the events which happen after a packet

retransmission can be incorporated to decide whether a congestive loss has

occurred. Such information is a real-time reflection of the network loading

condition so that this helps to make an appropriate decision. Specifically, if

an ACK for a packet lags behind a retransmission of that packet by a short

duration, it should be treated as a signal of no network congestion. The ACK

packet can either be for the originally transmitted packet or the retransmit-

ted packet. In the former case, false retransmission, which can hardly be

fully eliminated in reality, is detected, thereby rejecting the necessity of a

congestion response. In the latter case, a fairly short round-trip time and

thus a lightly loaded network can be inferred.

Second, it is more “affordable” for TCP to trigger a false fast retransmis-

sion than a false congestion response. In the event of a false fast retransmis-

sion, there is at most one full-sized packet being retransmitted spuriously.

However, there is a significant portion of the available bandwidth being left

27

unused in the event of a falsely activated congestion response. For TCP vari-

ants which activate fast retransmit and fast recovery simultaneously, they

may defer both measures to avoid the penalty of a false congestion response

until a congestive loss can be confirmed highly accurately. However, over

general error-prone reordering channels, such deferment on fast retransmit

may result in an expensive RTO. For example, TCP-DCR postpones fast re-

transmit by one RTT upon receiving the first duplicate ACK,so that a packet

is retransmitted, if needed, about two RTTs after its first transmission. By

advancing packet retransmission before the activation of congestion response,

the risk of triggering an RTO can be more effectively reduced while avoiding

the heavy penalty due to a false congestion response. Nevertheless, while a

comparatively higher level of false retransmission can be tolerated with our

proposal, it should still be maintained at a minimal level to avoid significant

waste of the limited bandwidth.

Based on the above observations, we design our STS model, as illustrated

in Figure 3.1. A new retransmission decision (RD) timer RDi is started

whenever a new packet Pi is injected into the network. If ACKi is received

by the TCP sender before RDi expires, RDi will be cancelled. Otherwise,

Pi will be retransmitted and a congestion response decision (CD) timer CDi

will be started.

CDi will be cancelled if ACKi arrives before it expires. Otherwise, the

congestion control mechanisms will be activated upon the expiration of CDi.

Therefore, the installation of CDi allows the TCP sender extra time after the

packet retransmission to decide whether congestion control shall be activated.

ACKi arriving before the expiration of CDi shall be treated as an indication

28

Table 3.1: General notations.

Notation Meaning

CS Number of packets TCP sender misses to send due toSC

CT Number of packets TCP sender misses to send dueto the extra reduction of TCP sender’s throughputintroduced by RTO

F (t) Cumulative distribution function of RTT for PacketPi

G Goodput of TCP sendermaxRi Maximum attainable RTT for Packet Pi

minRi Minimum attainable RTT for Packet Pi

P oi First transmitted Packet Pi

P ri Retransmitted Packet Pi

PC Congestive loss ratePL Loss ratePW Non-congestive loss rateT Throughput of TCP senderW Average cwnd over a TCP session

of no network congestion. Thus, this eliminates the need for activating any

congestion control measures.

The expiration periods of RDi and CDi, denoted as τRDiτCDi

, are eval-

uated in Sections 3.2.1 and 3.2.2, respectively. To facilitate the subsequent

discussion, we define the notations as shown in Tables 3.1 and 3.2.

The following assumptions are made to simplify the discussion:

1. Data packets are of equal size. Accordingly, we express the size of

congestion window and slow start threshold (ssthresh) in terms of the

number of packets.

2. P (PAi(t)|CLoi ) continuously monotonically increases with time t and

29

Table 3.2: Notations for events.

Notation Meaning

CLoi P o

i lost due to congestionCLr

i P ri lost due to congestion

NLoi P o

i experiencing non-congestive lostNLr

i P ri experiencing non-congestive lost

RTXi Fast retransmission of Pi

PAi(t) Either P oi or P r

i acknowledged by time t after RTXi

PAoi (t) P o

i acknowledged by time t after retransmissionPAr

i (t) P ri acknowledged by time t after retransmission

PUi(t) Both P oi and P r

i unacknowledged by time t afterRTXi

PUoi (t) P o

i unacknowledged by time t after retransmissionPU r

i (t) P ri unacknowledged by time t after retransmission

TO Retransmission timeout

limt→∞ P (PAi(t)|CLoi ) > 0. Given that PLo

i is lost due to congestion,

the chance of Packet Pi being acknowledged relies on P ri . By allowing

P ri more time (i.e., increasing t), the chance of Pi being acknowledged

does not drop. Furthermore, the chance should be non-zero, given that

P ri is allowed sufficient time (t→∞),

3. Non-congestive loss is assumed to be random. In other words, we do not

consider correlated or burst loss for simplicity. Nevertheless, simulation

results will show that our proposed TCP variant still offers competent

performance under burst loss. Thus, NLoi is mutually independent with

any events occurring with P ri and vice versa. In particular, it follows

that:

P (NLoi ∩ PU r

i (t)) = P (NLoi ) · P (PU r

i (t)) (3.1)

30

3.2.1 RD Timer

τRDidetermines how long we have to wait before activating packet retrans-

mission. The assignment of τRDirequires careful balancing between the ob-

jectives of prompt retransmission for preventing RTO and avoiding spurious

packet retransmission.

Ideally, it should be guaranteed that, when P oi is lost due to congestion,

P ri should be promptly transmitted so that the remaining time before RTO

is sufficient for it to be acknowledged. This ensures that cwnd will be halved

instead of being reset to one at the onset of network overload. In other words,

RTO will be incurred if and only if P ri is lost:

P (TO|CLoi ) = P (PLr

i ) = PL (3.2)

On the other hand, an excessively small τRDiwill result in spurious packet

retransmissions. Let F (τRDi) = 1− ζ , where 0 ≤ ζ ≤ 1. A fast retransmit of

packet Pi will be activated upon the expiration of RDi, given Pi is not yet

acknowledged. Thus,

P (RTXi) = P (PUoi (0))

= P (PUoi (0)|PLo

i )P (PLoi )

+P (PUoi (0)|PLo

i )P (PLoi )

= PL + ζ(1− PL) (3.3)

In addition to fast retransmit, packet retransmission can also be activated

by RTO. Yet, with fast retransmit promptly activated, the occurrences of

31

RTO will be maintained at a minimal level. Thus, the difference between

throughput and goodput, without considering the protocol overhead, should

mainly be attributable to fast retransmit. Accordingly, we have:

T ≃ [1 + P (RTXi)]G (3.4)

Hence,

G

T≃ [(1 + PL) + ζ(1− PL)]−1 (3.5)

To improve efficiency, we thus have:

ζ ≪ 1 (3.6)

Since the retransmission timer only serves as a coarse upper bound for

performing loss recovery and congestion control, it is possible for (3.2) and

(3.6) to be simultaneously satisfied with τRDiappropriately set. A closed-

form expression for the optimal value of τRDiis left as part of the future work.

Nevertheless, we assume that (3.2) and (3.6) are satisfied in the following

discussion.

3.2.2 CD Timer

The setting of τCDishould guarantee that, if ACKi arrives before the ex-

piration of CDi, P oi does not experience congestive loss. Analytically, this

32

is:

P (CLoi |PAi(t)) = 0 (3.7)

for all t ∈ [0, τCDi]. In this way, we ensure that the TCP sender will not

misinterpret congestive loss as non-congestive loss.

Denote τu , suptt|P (PAi(t)|CLoi ) = 0, which is well defined by As-

sumption 2. A sufficient and necessary condition for (3.7) is given by the

following lemma.

Lemma 3.1 P (CLoi |PAi(t)) = 0 if and only if t ≤ τu.

Proof Observe that:

P (CLoi |PAi(t))P (PAi(t)) = P (PAi(t)|CLo

i )P (CLoi ) (3.8)

and

P (PAi(t)) ≥ P (PAri (0)) = (1− PL)(1− ζ) > 0 (3.9)

Thus, P (CLoi |PAi(t)) = 0 if and only if P (PAi(t)|CLo

i ) = 0. On the

other hand, by construction, P (PAi(t)|CLoi ) = 0 if and only if t ≤ τu. The

result of the lemma therefore follows.

We do not know an exact expression for τu for the general case at the

moment. In the simple case of a single flow over a bottleneck link, when a

packet is dropped due to buffer overflow, the buffer will be kept full up to

when the lost packet is retransmitted. τu is therefore maxR in this case. Yet,

33

this do not apply the the case of multiple flows since upon the occurrence

of buffer overflow some flows may back off earlier than others. Nevertheless,

a lower bound for τu can be obtained for the general case. Noting that

P (PAi(minRi)|CLoi ) ≤

P (PAri (minRi))

P (CLoi )

and P (PAri (minRi)) = 0, we have

P (PAi(minRi)|CLoi ) = 0. Thus, a general lower bound for τu is minRi.

Now, we shall seek for an optimal solution for τCDiwithin [0, τu]. Con-

sider a time period t (0 ≤ t ≤ τu) after the retransmission of packet Pi, when

a TCP sender is facing the decision of whether or not to activate a congestion

response. The risk associated with a positive decision is that the network may

not be congested (i.e. P oi is not dropped due to congestion). Consequently,

the spuriously activated congestion response will reduce cwnd unnecessarily.

On the other hand, if the network is indeed overloaded (i.e. P oi is lost due to

congestion) and the activation of congestion response is excessively delayed,

the network congestion may be exacerbated and an expensive RTO may be

incurred.

To reach a rational decision regarding the optimal τCDi, we need to com-

pare the quantified risks of activating congestion response and delaying con-

gestion response. Before doing that, we first quantify the costs of falsely

activating congestion response and excessively delaying congestion response

when network is indeed overloaded, respectively, by the following two defini-

tions.

Definition 3.1 The cost of falsely activating congestion response, denoted

as CS, is the average number of packets a TCP sender misses to send due to

the activation of a congestion response,

34

Definition 3.2 The cost of excessively delaying congestion response, de-

noted as CT , is the average number of packets a TCP sender misses to send

due to the occurrence of RTO.

Consider the activation of congestion response, when both cwnd and

ssthresh are set to 0.5 W . Thus TCP sender is at the congestion avoidance

stage to begin. cwnd is incremented by one at every RTT cycle afterwards.

Let nCA be the number of cycles during which the TCP sender is in the

congestion avoidance stage with cwnd less than W . Accordingly, we have:

nCA = ⌊0.5 W⌋ (3.10)

Therefore, when a congestion response is falsely activated, the number of

packets the TCP sender misses to send is:

CS = W nCA −nCA∑

i=1

(⌈0.5 W⌉+ i− 1)

= 0.5 ⌊0.5 W ⌋ (⌊0.5W ⌋+ 1) (3.11)

Consider the occurrence of an RTO, when cwnd is reset to one and

ssthresh is set to 0.5 W . For simplicity, we would assume that ACK ar-

rives before the occurrence of another RTO. Thus, after being reset, the

TCP sender is in the slow start stage and W will double itself at every RTT

round as long as it is less than ssthresh. Once W reaches ⌈ssthresh⌉, the TCP

sender leaves the slow start stage and enter the same congestion avoidance

stage. Let nSS be the number of cycles during which the TCP sender is in

35

the slow start stage, we have:

nSS = ⌈log2 (0.5 W )⌉ (3.12)

Upon a genuine congestive loss, it is generally desirable for a TCP sender

to halve cwnd and directly enter the congestion avoidance stage. Therefore,

if an RTO occurs instead and forces cwnd to be reset to one, the TCP sender

throughput is penalized when the slow start phase is activated. The number

of packets the TCP sender misses to send is:

CT = W nSS −nSS∑

i=1

2i−1

= ⌈log2 (0.5 W )⌉ ·W − 2⌈log2 (0.5 W )⌉ + 1 (3.13)

We are now ready to quantify the risks associated with activating a con-

gestion response and delaying congestion response. The following two metrics

are introduced, respectively, for the purpose.

Definition 3.3 The expected cost of activating a congestion response, de-

noted as ECA(t), is the product of the conditional probability of P oi not being

lost due to congestion, given that Pi is unacknowledged, and the cost of falsely

activating congestion response. In other words,

ECA(t) = P (CLoi |PUi(t)) · CS (3.14)

Definition 3.4 The expected cost of delaying a congestion response, de-

noted as ECD(t), is the product of the conditional probability that a conges-

36

tive loss and an RTO have occurred, given that Pi is unacknowledged, and

the cost of excessively delaying congestion response. In other words,

ECD(t) = P (TO ∩ CLoi |PUi(t)) · CT (3.15)

The two conditional probabilities involved in Definitions 3.3 and 3.4 are

expressed in terms of loss probabilities, namely PC , PW , and PL, and distri-

bution function of RTT, F (t), by Lemmas 3.2 and 3.3, respectively.

Lemma 3.2 For 0 ≤ t ≤ τu,

P (CLoi |PUi(T ))

=PW [1− (1− PL)F (t)]

PC + PW [1− (1− PL)F (t)](3.16)

(3.17)

Proof We begin by noting that:

P (CLoi ) = P (PUi(t)) · P (CLo

i |PUi(t))

+P (PAi(t)) · P (CLoi |PAi(t)) (3.18)

For 0 ≤ t ≤ τu, since P (CLoi |PAi(t)) = 0 by Lemma 3.1, this reduces to:

P (CLoi ) = P (PUi(t)) · P (CLo

i |PUi(t)) (3.19)

It follows that:

P (CLoi |PUi(t)) = 1− P (CLo

i |PUi(t))

37

= 1− P (CLoi )

P (PUi(t))(3.20)

where 0 ≤ t ≤ τu.

We now derive P (PUi(t)), which will complete the proof.

P (PUi(t))

= P (PUoi (t) ∩ PU r

i (t))

= P (PUoi (t) ∩ PU r

i (t)|PLoi )P (PLo

i )

+P (PUoi (t) ∩ PU r

i (t)|PLoi )P (PLo

i ) (3.21)

Since:

P (PUoi (t) ∩ PU r

i (t)|PLoi ) ≤ P (PUo

i (t)|PLoi )

= 1− F (t + τRDi)

≤ ζ (3.22)

where ζ ≪ 1 by (3.6), we have:

P (PUi(t)) ≈ P (PUoi (t) ∩ PU r

i (t)|PLoi )P (PLo

i ) (3.23)

Noting that:

P (PUoi (t) ∩ PU r

i (t)|PLoi )

= P (PU ri (t)|PLo

i )− P (PU ri (t) ∩ PAo

i (t)|PLoi )

(3.24)

38

and

P (PAoi (t) ∩ PU r

i (t)|PLoi ) ≤ P (PAo

i (t)|PLoi ) = 0 (3.25)

we further obtain:

P (PUoi (t) ∩ PU r

i (t)|PLoi ) = P (PU r

i (t)|PLoi ) (3.26)

(3.23) thus simplifies to:

P (PUi(t)) ≈ P (PU ri (t) ∩ PLo

i )

= P (PU ri (t) ∩ CLo

i ) + P (PU ri (t) ∩NLo

i )

= P (CLoi )P (PU r

i (t)|CLoi )

+P (PU ri (t)) · P (NLo

i ) (3.27)

The last step above applies (3.1)

Recall that τu = suptt|P (PAi(t)|CLoi ) = 0. Thus, for 0 ≤ t ≤ τu,

P (PU ri (t)|CLo

i ) ≥ P (PUi(t)|CLoi )

= 1− P (PAi(t)|CLoi ) = 1 (3.28)

Substituting (3.28) into (3.27) yields:

P (PUi(t)) ≃ P (CLoi ) + P (PU r

i (t)) · P (NLoi )

(3.29)

39

where P (PU ri (t)) can be derived as:

P (PU ri (t)) = P (PU r

i (t)|PLri )P (PLr

i )

+P (PU ri (t)|PLr

i )P (PLri )

= PL + [1− F (t)](1− PL) (3.30)

Substituting (3.30) into (3.29), we obtain:

P (PUi(t)) = PC + [1− (1− PL)F (t)]PW (3.31)

where 0 ≤ t ≤ τu.

Lemma 3.3 For 0 ≤ t ≤ τu,

P (TO ∩ CLoi |PUi(t)) =

PC PL

PC + PW [1− (1− PL)F (t)](3.32)

Proof Following a similar process to (3.18) - (3.20), we can obtain:

P (TO ∩ CLoi |PUi(t)) =

P (TO|CLoi )P (CLo

i )

PUi(t)

(0 ≤ t ≤ τu) (3.33)

Equ. (3.32) follows by substituting (3.31) and (3.2) into (3.33).

Thus:

ECA(t) =PW [1− (1− PL)F (t)]

2PC + PW [1− (1− PL)F (t)]×⌊0.5 W ⌋(⌊0.5W ⌋+ 1) (3.34)

40

DEC (t)

EC (t)A

tτu

(a) Case I.

τ

DEC (t)

EC (t)A

tu

(b) Case II.

τTH

EC (t)A

EC (t)D

tτu

(c) Case III.

Figure 3.2: Evaluation of τ ∗CDi

.

ECD(t) =PC PL

PC + PW [1− (1− PL)F (t)]

×(⌈log2 0.5W ⌉W − 2⌈log2 0.5W ⌉ + 1) (3.35)

Now,

∂

∂tECA(t) ≤ 0,

∂

∂tECD(t) ≥ 0 (3.36)

Thus, when the activation of congestion response is postponed further, ECD(t),

which quantifies the risk associated with delaying a congestion response, in-

creases, while ECA(t), which quantifies the risk associated with activating

a congestion response, drops. When ECA(t) is greater than ECD(t), it is

advantageous to set τCDino less than t since the operation cost of activat-

ing congestion response is ECA(t), which is larger than that of deferring it,

ECD(t). The cost may drop when t increases. Similarly, when ECD(t) is

greater than ECA(t), it is advantageous to set τCDino greater than t since

the operation cost of deferring congestion response is ECD(t), which is larger

than that of activating it, ECA(t). The cost may drop when t decreases.

41

Therefore, the evaluation of the optimal solution of τCDi, τ ∗

CDi, subject

to the constraint 0 ≤ τCDi≤ τu, can be divided into three cases, as depicted

in Figure 3.2. Case I arises when ECD(t) exceeds ECA(t) for any t ≥ 0.

The gap between the two would keep on increasing when t increases. Thus,

the congestion response should be activated at t = 0, or set τ ∗CDi

to be zero.

Case II arises when ECD(t) fails to catch up with ECA(t) for all t such that

0 ≤ t ≤ τu. Since we cannot delay a congestion response further according to

the prior constraint, we have to set τ ∗CDi

to be τu. The final case arises when

ECD(t) catches up with ECA(t) for some t such that 0 ≤ t = τTH ≤ τu. τ ∗CDi

thus corresponds to τTH since ECA(t) is greater than ECD(t) prior to τTH

and less than ECD(t) after τTH .

The optimal value of τCDican thus be determined by the following theo-

rem.

Theorem 3.1 Suppose Assumptions 1-3 hold. The optimal value for τCDi

within [0, τu] in the sense of minimizing the associated expected cost, τ ∗CDi

, is

given by:

τ ∗CDi

=

0, PC > (PL · CT

CS+ 1)−1 PL

τu, PC < PL CT

[1−(1−PL)·F (τu)]·CS+ 1−1 · PL

τTH , otherwise.

(3.37)

where τTH = F−1(

11−PL

− CT PC

CS ·(PL−PC)· PL

1−PL

)

.

Proof The proof follows from the foregoing discussion.

42

When non-congestive loss is negligible, we fall into the first condition in

Theorem 3.1. Packet retransmission and congestion response should thus be

activated simultaneously. When non-congestive loss becomes non-negligible

(say, PW > PC), it generally holds that PC < (PLCT

CS+ 1)−1 PL. It follows

that τ ∗CDi∈ [minRi, τu].

3.3 TCP-NCL

The STS model is a limited, idealized TCP sender. First, it does not pro-

vide a closed-form expression for τRD i. Second, it assumes, among other

things, prior knowledge of the RTT distribution F (t), which is impractical.

Therefore, we propose TCP-NCL to closely approximate STS.

The serialized-timer structure of the STS model is supplemented by TCP-

NCL with the NCL-RTT-Update (NRU) process for maintaining statistics of

RTT samples so as to estimate F (t). Section 3.3.1 describes the NRU pro-

cess. Section 3.3.2 explains the implementation of RD and CD timers using

timestamps. The pseudocode for these are exhibited in Algorithms 3.3.1 and

3.3.1, respectively. Section 3.3.3 describes the minor modifications involved

in the retransmission timeout (RTO) timers. Section 3.3.3 describes kernel

implementation of TCP-NCL and the overhead involved.

3.3.1 NRU Process

The NRU process is invoked if an ACK, say ACK i, arrives before the ex-

piration of its corresponding CD timer, CD i. This can be further divided

into two cases: ACK i arrives before or after RD i expires. In the latter

43

case, there is an ambiguity regarding whether ACK i is for P oi or P r

i . Thus,

some additional measures would be needed to make sure that ACK i is for

P oi before recording the corresponding RTT sample. We identify the time

instance for the most recent transmission of Pi as txTime[i]. Upon the arrival

of ACKi, if the present time (identified by now) exceeds txTime[i] by less

than β τCD i(0 < β < 1), this infers that P r

i cannot be acknowledged within

such a short interval and thus the RTT sample (τRD i+ now − txTime[i]),

is inserted into rttRcd. Otherwise, the corresponding RTT sample will be

ignored.

By updating RTT based on ACKs received in this manner, the RTT

sampling process is more robust to changes in the network environment,

especially to abrupt increases in RTT caused by path change or handoff.

When the latter occurs, it is likely that ACK i will not be received before

RTi expires. Yet, the corresponding RTT sample can still be accurately

recorded as long as it does not exceed (τRD i+ β τCD i

).

The assignment of τRD iand τCD i

, as will be discussed in Section 3.3.2,

only relies on the maximum RTT sample, maxR, and minimum RTT sample,

minR. There is thus no need to keep all the recent maximum RTT record

length (MRRL) samples in storage. Instead, when a new RTT is greater

than maxR/less than minR, it is used to update maxR/minR.

However, merely doing so may prevent us to discard some outdated sam-

ples that are very large or very small. The distribution of RTT is time-

variant over wireless networks, where the network topology and/or loading

may change over a TCP session. The time-varying property of F (t) requires

prompt discard of outdated samples. In [10], TCP-PR needs to keep track

44

of updated maximum RTT sample as well. A srtt variable is maintained

for the purpose. Upon obtaining a new RTT sample, srtt is multiplied by

α(− 1

cwnd), where α is a system designed parameter in the interval (0, 1).

The new sample is then used to update srtt if greater than the current value

of srtt. The multiplication, which reduces srtt by a factor of α per RTT,

ensures that outdated peaks in RTT will eventually be replaced. However,

it is generally computationally expensive for the kernel.

We thus apply a computationally cheaper strategy for the similar purpose

of discarding outdated maximum and minimum RTT samples. We will main-

tain two new variables maxR2 and minR2 in supplement to the prescribed

updating process of maxR and minR.

A periodic downward counter, rcdAge, is initialized as MRRL and decre-

mented by one upon obtaining a new RTT sample. Similarly as maxR, an

RTT sample is used to update maxR2 if greater than it. Yet, when rcdAge

reaches zero, maxR will be set as maxR2 before maxR2 is set as the newly

obtained RTT sample without comparison. rcdAge is then set as MRRL

again.

Thus, every time upon completing the updates on maxR, maxR2, and

rcdAge, maxR2 always maintains the maximum value within the most recent

(MRRL− rcdAge + 1) RTT samples. Regarding maxR:

1. When rcdAge is MRRL, maxR has just been updated by the old value

of maxR2. In addition, it will be set to the new RTT sample if smaller

than it. Thus, it will keep the maximum value within the most recent

(MRRL + 1) RTT samples.

45

2. Upon obtaining the next n (1 ≤ n ≤ MRRL − 1) samples, rcdAge is

decremented by one each time and thus n = MRRL − rcdAge. Since

rcdAge is greater than zero, maxR is simply updated by the new RTT

if smaller than it. Thus, it maintains the maximum value within the

most recent ((MRRL + 1) + n) RTT samples.

3. Upon obtaining one more sample, rcdAge will be decremented to zero

and then set to MRRL. We thus go back to the situation described to

1).

Overall, maxR always maintains the maximum value within the most recent

(2MRRL - rcdAge + 1) RTT samples, where 1 ≤ rcdAge ≤ MRRL. This

enables discard of outdated samples while ensuring maxR is the maximum

among a sufficiently large number (≥ MRRL+1) of most recent RTT samples.

The maintenance and usage of minR2 are carried out in an analogous

manner. It is therefore ensured that minR maintains the minimum value

within the most recent (2MRRL - rcdAge + 1) RTT samples.

3.3.2 RD and CD Timers

τRD iis set to (maxR + minR). Intuitively, this assignment is an appro-

priate choice, keeping spurious retransmission at a minimal level without

excessively delaying fast retransmit. As will be shown by our simulation re-

sults, TCP-NCL demonstrates a very efficient usage of network bandwidth in

that it almost always attains the highest connection goodput among all our

compared TCP variants. This performance can hardly be achieved if either

spurious retransmission constitutes a significant wastage of total through-

46

Algorithm 1 Procedure NRU (Packet Pi)

1: rcdAge ← rcdAge − 12: if Pi is on rcPkts then3: rtt ← now − txTime[i]4: else5: if now − txTime[i] < β ∗ tauCD then6: rtt ← now − txTime[i] + tauRD7: else8: exit from the process9: end if

10: end if11: if rcdAge = 0 then12: minR ← minR213: maxR ← maxR214: minR ← rtt15: minR ← rtt16: rcdAge ← MRRL17: end if18: maxR2 ← max(maxR2 , rtt)19: maxR ← max(maxR, rtt)20: minR2 ← min(minR2 , rtt)21: minR ← min(minR, rtt)22: tauCD ←maxR + minR23: tauCD ← K ∗maxR + (1−K) ∗minR

47

Algorithm 2 Procedure update list (Packet ACK i)

1: for each Packet Pj on cdPkts do2: if tauCD < now − txTime[j] then3: remove Pj from cdPkts4: activate congestion response5: end if6: end for7: Retrieve the Packet Pi acked by ACK i

8: if Pi is in rdPkts or cdPkts then9: NRU (Pi)

10: remove Pi from rdPkts/cdPkts11: end if12: for each Packet Pj on rdPkts do13: if tauRD < now − txTime[j] then14: remove Pj from rdPkts15: add Pj to to-be-rtxed16: end if17: end for18: wnd ← min(cwnd , awnd)19: while number of outstanding packets ≤ wnd do20: if there is a Packet Pj on to-be-rtxed then21: add Pj to cdPkts22: retransmit Pj

23: txTime[j]← now24: else25: Transmit the next new packet Pk

26: Add Pk to rdPkts27: txTime[k]← now28: end if29: end while

48

put, or RTO frequently occurs and forces cwnd to be reset to one. Thus, the

choice of setting τRD ias (maxR + minR) seems to be appropriate.

By the discussion at the end of previous section, τ ∗

CD i

∈ [minRi, τu].

minRi can be approximated by minR, whereas τu is lower bounded by minR

and can be up to maxR. Thus, we assign τCDito be (K ∗maxR + (1−K) ∗

minR), where K ∈ [0, 1] is a system-designed parameter. It is worth noting

that, while STS model specifies τ ∗

CD i

dependant on PC , we remove such

dependence in TCP-NCL. This is because congestive loss are usually rate

events in the network, and it is very difficult to measure PC accurately from

end systems.

With our approximation above, at any time instance, the expiration pe-

riods of RD/CD timers for all the RD/CD pending packets would be the

same. We thus define two variables, namely, tauRD and tauCD, for keeping

the present values of the expiration periods of RD and CD timers, respec-

tively, instead of keeping a unique copy for each packet.

To fulfill the effects of the RD and CD timers without incurring the heavy

cost of maintaining one timer per outstanding packet, we define rdPkts and

cdPkts for keeping the packets whose RD and CD timers are supposedly

pending, respectively. The two lists are updated upon arrival of an acknowl-

edgment packet as follows:

1. Activate congestion response measure if there are packets that have

stayed on cdPkts for more than tauCD, and remove the timeout’ed

packets.

2. Remove the acknowledged packet if it is on rdPkts or cdPkts.

49

3. Retransmit packets that have stayed on rdPkts for more than tauRD,

and transfer them from rdPkts to cdPkts with timestamps recording

the retransmission time instance.

4. Transmit new packets and add them to rdPkts with timestamps record-

ing the transmission time instance.

When activating the congestion control measures, we adopt an approach

similar to that used in TCP-PR, that packet losses within the same burst are

considered as a single signal about the onset of network congestion and the

reduction of cwnd will be triggered only once. Thus, when cwnd is halved,

all the present packets in rdPkts and cdPkts are exempted from causing cwnd

to be halved again.

3.3.3 RTO Timer

With standardized TCP design, severe congestion is inferred and go-back-N

retransmission is performed by retransmitting all outstanding packets upon

the occurrence of RTO. The expiration period of RTO timer is set to the sum

of mean RTT and a multiple of RTT deviation. The RTO timer is reset upon

packet (re)transmission. Since fast retransmit performs retransmission upon

receiving three duplicate acknowledgments, the (fast) retransmitted packet

is allowed sufficient time to be acknowledged before the occurrence of RTO

and the associated the go-back-N retransmission.

We keep RTO timer in the design of TCP-NCL so as to perform worst-case

recovery. Moreover, this ensures that TCP-NCL can emulate the behavior of

standardized TCP design in conventional wired networks, thereby reducing

50

its possible disruptions to the latter. Yet, the occurrence of RTO needs to

be well coordinated with the serialized-timer structure that has postponed

packet retransmission and congestion response proactively. It is undesirable

for RTO timer to fire before the RD and CD timers fires. Moreover, a packet

retransmitted upon the expiration of RD timer should be allowed sufficient

time to be acknowledged before the go-back-N retransmission is considered

necessary and activated. Thus, TCP-NCL impose two modifications to the

RTO timer:

1. The expiration period of RTO timer is modified to be (tauRD + 2 ∗

tauCD).

2. A further condition for performing go-back-N retransmission is im-

posed. Upon the occurrence of RTO, it checks whether there are

retransmitted packets that are not acknowledged within tauRD, or

whether the congestion window is no greater than one. If either is

true, rdPkts and cdPkts will be cleared, and all the outstanding pack-

ets will be retransmitted at a data rate regulated by the congestion

window.

3.3.4 Kernel Implementation

The difficulty in modifying the kernel hinders the implementation of most

TCP variants proposed. To facilitate the implementation, we devise a generic

framework for developing TCP variants in the Linux/UNIX kernel. We ob-

serve that a majority of TCP variants differ in their congestion control and

loss recovery operations, which are realized cooperatively by three TCP ma-

51

jor functions2, namely, the TCP input function tcp input, the TCP output

function tcp output, and the RTO timer function tcp timer rexmt, as illus-

trated in Figure 3.3. Thus, a TCP variant can be deployed by: 1) defining its

data structures in the TCP control block, 2) maintaining the data structures

in the three aforementioned functions, and 3) modifying/replacing the fast

retransmit, fast recovery, and/or RTO algorithms.

socket receivebuffer

tcp_input

ipintr

tcp_output

ip_output

tcp_timer_rxmt

softclock

tcp_usrreq

system calls

Figure 3.3: Software architecture for TCP.

Following the framework, we have deployed a first version of TCP-NCL .

1) The sk buff structure in TCP control block records all the outstanding

packets with timestamps, and each packet has several associated flags that are

left unused [63]. The timestamps suffices for the purpose of txTime[i], and the

unused flag bits can be set to indicate the membership of a packet in rdPkts

and cdPkts. We further define a few extra variables, namely minR, maxR,

minR2, maxR2, rcdAge, tauRD, and tauCD in TCP control block. Since each

TCP session maintains one copy of TCP control block, the implementation of

TCP-NCL requires storing a small number of extra variables per connection

only.

2) In accordance with our algorithms described in Sections 3.3.1-3.3.3,

2The architecture follows that of BSD UNIX [73]. Despite the possible differences inthe naming conventions and function calling sequences, most Linux/UNIX kernels adoptsimilar structures in their TCP/IP protocol stacks.

52

SS DBS100 Mbps 1 Mbps

50 ms5 msS

(a) Infrastructure-based wireless network.

R3 DR1SSS1 Mbps

50 ms50 ms

1 Mbps

(b) Multi-hop wireless network.

1MbpsR2R15 ms

S1

S2

100 Mbps5 ms

SN100 Mbps

5 ms

50ms...

D2100 Mbps

5 ms

100 Mbps

...

D1100 Mbps5 ms

DN100 Mbps

5 ms

(c) Wired network with a dumbbell topology.

Figure 3.4: The network topologies used for performance comparison

these variables are updated in tcp input based on the incoming acknowl-

edgements, and properly reset in tcp timer rxmt. This requires a constant

number of extra maintenance steps per data packet. Again, we stress that

the RD and CD timers are approximated via timestamps, and implementing

TCP-NCL does not require maintenance of extra timers.

3) The fast retransmit and fast recovery algorithms are replaced in tcp input

by our serialized timer structure. The major change involved is to activate

fast recovery upon detecting CD timeout’ed packets (namely, packets that

have stayed in cdPkts for more than tauCD) instead of upon activating fast

retransmit. The extra overhead involved is minimal.

53

3.4 Performance Evaluation

In this section, we present our simulation results. We have performed exten-

sive simulation experiments to evaluate the performance of TCP-NCL using

Network Simulator (ns) 2.29 [19].

We have evaluated several instances of TCP-NCL with different settings

of K. Recall that K ∈ [0, 1] is the system-designed parameter involved in

setting the expiration period of the CD timer (tauCD = (1 − K) · minR +

K · maxR). A larger value of K generally corresponds to a more aggres-

sive instance of TCP-NCL. In the following, we will focus on two instances

of TCP-NCL, TCP-NCL (K=0) and TCP-NCL (K=0.5), with K set to 0

and 0.5, respectively. TCP-NCL (K=0) therefore corresponds to the most

conservatively configured TCP-NCL.

We have also included a TCP variant degenerated from TCP-NCL, TCP-

NCL-RD, for comparison. TCP-NCL-RD differs from TCP-NCL in that it

disables the CD timer by setting the expiration period of the CD timer to

zero. Thus, it proactively postpones packet retransmission until the expira-

tion of the RD timer, when it will assume the packet is lost due to congestion

and activates congestion response and packet retransmission simultaneously.

We aim to examine the effect of the serialized-timer structure by comparing

TCP-NCL with TCP-NCL-RD.

The performance curves of four TCP variants in the literature, namely

RR-TCP [81], TCP-DCR [8], TCP-PR [10], and TCPW [13], are replicated

from our ongoing study [43] for comparison. The former three variants attain

the best goodput performance against packet reordering according to our

54

0

0.2

0.4

0.6

0.8

1

0 0.02 0.04 0.06 0.08 0.1

Con

nect

ion

Goo

dput

(M

bps)

Packet Error Rate

TCP-NCL (K=0.5)TCP-NCL (K=0)

TCPWTCP-NCL-RD

RR-TCPTCP-PR

TCP-DCR

Figure 3.5: Goodput performance against packet error rate overinfrastructure-based wireless network.

simulation-based comparison in [77], while TCPW is well known as a solution

for wireless packet losses.

The simulated network topologies include the infrastructure-based wire-

less network, the multi-hop wireless network, and the wired network with a

dumbbell topology, as illustrated in Figure 3.4. Unless specified otherwise,

the queue management algorithm for each link buffer is drop tail. The packet

size is 1000 bytes. More detailed network configurations will be described in

the following.

Sections 3.4.1 and 3.4.2 examine and compare the performance of all

the aforementioned TCP variants against non-congestive loss and packet re-

ordering over the infrastructure-based wireless network and the multi-hop

wireless network ,respectively. Section 3.4.3 examines the responsiveness of

TCP-NCL against congestive loss over the wired network with a dumbbell

topology.

55

3.4.1 Non-congestive loss

The infrastructure-based wireless network is illustrated in Figure 3.4 (a). A

TCP sender and a TCP receiver are connected through a wired with band-

width 100 Mbps and propagation delay 5 ms, and a wireless link with band-

width bw Mbps and propagation delay d ms. Random packet errors with rate

pe are deliberately introduced into the wireless link. Packets experiencing

transmission errors are discarded at the receiving end.

In each test over the topology, a total of 20 runs, each lasting 2000 seconds

and using different seeds for generating the packet error, have been performed

to compute an average value and a 95% confidence interval of TCP goodput

in Mbps. In order to remove the effect of the transient states, only the

statistics in the last 1000 seconds in each run are collected for computing the

goodput.

Figures 3.5, 3.6, and 3.7 plot the connection goodput of the TCP variants

against pe, d, and bw, respectively. In each plot, the legends for TCP variants

are sorted descendingly according to their levels of goodput at the rightmost

data point. In most cases, the confidence intervals are too narrow to be

distinguishable. The configuration settings are summarized in Table 3.3. A

hyphen (-) in an entry indicates that the corresponding parameter is being

varied to generate the plot.

Figure 3.5 exhibits the connection goodputs of the TCP variants against

packet error rate pe. The two instances of TCP-NCL attains the best per-

formance. At pe = 1%, TCP-NCL (K=0.5) and TCP-NCL (K=0) attains

around 30% and 15% performance improvement over TCP-NCL-RD, respec-

56

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250 300 350 400

Con

nect

ion

Goo

dput

(M

bps)

Delay of the Wireless Link (ms)

TCP-NCL (K=0.5)TCPW

TCP-NCL (K=0)TCP-NCL-RD

TCP-DCRTCP-PRRR-TCP

(a) pe = 1%.

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250 300 350 400

Con

nect

ion

Goo

dput

(M

bps)

Delay of the Wireless Link (ms)

TCP-NCL (K=0.5)TCPW

TCP-NCL (K=0)TCP-NCL-RD

TCP-DCRTCP-PRRR-TCP

(b) pe = 5%.

Figure 3.6: Goodput performance against delay of the wireless link overinfrastructure-based wireless network.

tively. Their performance gains further increase as pe increases, and reach

around 200% and 100%, respectively, at pe = 10%. These verify the ef-

fectiveness of the installation of the CD timer in combating wireless losses.

TCPW attains the third best performance. It performs similarly as TCP-

NCL (K=0) at pe (up to 2%), but performs slightly worse as pe further

increases. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates

similar drastic decrease in goodput as pe increases. They exclude the pos-

57

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7

Con

nect

ion

Goo

dput

(M

bps)

Bandwidth of the Wireless Link (Mbps)

TCP-NCL (K=0.5)TCPW

TCP-NCL (K=0)TCP-PR

TCP-DCRTCP-NCL-RD

RR-TCP

(a) pe = 1%.

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7

Con

nect

ion

Goo

dput

(M

bps)

Bandwidth of the Wireless Link (Mbps)

TCP-NCL (K=0.5)TCPW

TCP-NCL (K=0)TCP-PR

TCP-DCRTCP-NCL-RD

RR-TCP

(b) pe = 5%.

Figure 3.7: Goodput performance against bandwidth of the wireless link overinfrastructure-based wireless network.

sibility of non-congestive packet loss, resulting in severe under-utilization of

network resources.

Figure 3.6 exhibits the connection goodputs of the TCP variants against

propagation delay of the wireless link d, for pe = 1%, 5%. All TCP variants

generally perform worse as d increases. TCP-NCL (K=0.5), TCPW, and

TCP-NCL (K=0) attain much higher connection goodput than other TCP

variants. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates

58

Table 3.3: Network configurations for the infrastructure-based wireless net-work

Figure bw d peFigure 3.5 1 50 -

Figure 3.6(a) 1 - 1%Figure 3.6(b) 1 - 5%Figure 3.7(a) - 50 1%Figure 3.7(b) - 50 5%

similar severe performance deterioration as d increases. To fully utilize the

wireless link capacity, the size of congestion window (cwnd) has to be at least

the bandwidth-delay product, namely the product of bw and the round-trip

propagation delay. Such delay increases linearly as d increases. With a larger

value of d, it takes a TCP connection longer to resume cwnd to be above the

product upon spurious reduction of cwnd due to non-congestive losses.

Figure 3.7 exhibits the connection goodputs of the TCP variants against

bandwidth of the wireless link bw, for pe = 1%, 5%. TCP-NCL (K=0.5),

TCPW and TCP-NCL (K=0) manages to maintain a linear increase in good-

put as bw increases up to 4 Mbps, thereby maintaining a constant bandwidth

utilization. All other TCP variants essentially maintain constant connec-

tion goodput as bw increases, corresponding to a decrease in bandwidth

utilization. Similar inferences made from Figure 3.6 can apply here. The

bandwidth-delay product increases linearly proportionally to bw. With a

larger value of d, it takes a TCP connection longer to recover from spurious

cwnd reduction due to non-congestive losses.

Up to now, we can observe that the two instances of TCP-NCL attain

significant performance gains over TCP-NCL-RD across different configura-

59

tions of bw, d, for non-zero pe, by merely the installation of the CD timers.

The serialized-timer structure thus proves to be very effective in shielding

the impact of non-congestive losses.

We now draw a closer comparison between TCP-NCL and TCPW. We

focus on comparing TCP-NCL (K=0), the most conservatively configured

TCP-NCL, with TCPW. Generally, increasing K helps TCP-NCL to avoid

spurious reduction of cwnd more effectively. Thus, all inferred advantages

of TCP-NCL (K=0) over TCPW in combatting non-congestive losses will

safely hold for TCP-NCL configured otherwise.

TCP-NCL and TCPW apply distinctly different strategies for perform-

ing effective congestion avoidance in the presence of non-congestive losses.

TCP-NCL tries to construct more reliable signals of congestive loss by differ-

entiating between congestive loss and non-congestive loss, and halves cwnd

when inferring congestive loss. TCPW does not perform the differentiation.

Rather, it reduces cwnd to its estimated available network capacity upon

each inferred loss. Therefore, we can make three inferences in comparing

TCP-NCL and TCPW:

1. In the presence of non-congestive losses, TCPW tends to trigger spu-

rious reduction of cwnd more frequently than TCP-NCL. This is ex-

emplified scenarios with low bandwidth-delay product and high packet

error rates. For example, TCP-NCL (K=0) outperforms TCPW for

pe ≥ 3%, d = 50, bw = 1 in Figure 3.5.

2. When both perform a spurious reduction of cwnd due to non-congestive

losses, TCP-NCL tends to reduce cwnd more abruptly than TCPW.

60

This is exemplified in scenarios with high bandwidth-delay product

and low packet error rates. For example, TCPW outperforms TCP-

NCL (K=0) for pe = 1%, d ≥ 100, bw = 1 in Figure 3.6(a), and for

pe = 1%, d = 50, bw ≥ 2 in Figure 3.7(a).

3. The aforementioned two inferences can counteract each other in scenar-

ios with both high bandwidth-delay produce and high packet error rate.

For example, TCP-NCL (K=0) performs similarly as TCPW for pe =

5%, d ≥ 100, bw = 1 in Figure 3.6(b) and for pe = 5%, d = 50, bw ≥ 2

in Figure 3.7(b).

3.4.2 Packet Reordering

In the multi-hop wireless network as illustrated in Figure 3.4 (b), a TCP

sender is connected to a TCP receiver via six wireless links. Random chan-

nel error is introduced into the wireless links with a random packet error

rate ranging from 0 to 10%. Packets experiencing transmission errors are

discarded at the receiving end of the wireless link. Yet, the setting differs

from the infrastructure-based wireless network in that the link layer packets

thus discarded are locally retransmitted at the link layer around 110 ms after

its most recent (re)transmission. The retransmission period of 110 ms is the

sum of two-way propagation delay, (50 ms× 2), one-way transmission delay,

(1000 bytes/1 Mbps), and a delay margin of 2 ms. The retransmission will

be performed for up to three times. This greatly enhances the chance of

successful packet delivery at each hop. Yet, packet reordering will be intro-

duced since the retransmitted packets will be intermixed with later packets.

61

0

0.2

0.4

0.6

0.8

1

0 0.02 0.04 0.06 0.08 0.1

Con

nect

ion

Goo

dput

(M

bps)

Packet Error Rate

TCP-NCL(K=0.5)TCP-PR

TCP-NCL(K=0)TCP-NCL-RD

TCPWTCP-DCR

RR-TCP

(a) Four hop.

0

0.2

0.4

0.6

0.8

1

0 0.02 0.04 0.06 0.08 0.1

Con

nect

ion

Goo

dput

(M

bps)

Packet Error Rate

TCP-NCL(K=0.5)TCP-PR

TCP-NCL(K=0)TCP-NCL-RD

TCPWRR-TCP

TCP-DCR

(b) Six hop.

Figure 3.8: Goodput performance over multihop ad-hoc wireless network

The intensity of packet reordering will increase as packet error rate increases.

Moreover, under high channel error rate, some packets may still fail to be

correctly delivered after three link-layer retransmissions. Consequently, TCP

will be confronted with both packet reordering and non-congestive packet

loss.

Figure 3.8 exhibits the connection goodput of the TCP variants against

packet error rate over four-hop and six-hop wireless networks. Over the

four-hop connection (Figure 3.8(a)), the two instances of TCP-NCL, TCP-

62

PR, and TCP-DCR outperform other variants for pe up to 7%, thereby

demonstrating robustness to persistent reordering. TCP-NCL-RD performs

slightly worse. It activates congestion response if a packet not on ePkt is not

acknowledged tauRD(= maxR +minR) after packet transmission. The time

lag is generally shorter than that of TCP-NCL, TCP-PR, and TCP-DCR,

leading to a higher chance of premature congestion responses.

When pe further increases beyond 7% and non-congestive losses are no

longer transparent to the transport layer, TCP-NCL (K=0.5) outperforms

TCP-PR while the performance of TCP-DCR is seriously deteriorated. In

such scenario, the installation of the CD timers with a sufficiently long expi-

ration period helps TCP-NCL (K=0.5) increase the reliability for signals of

congestive loss.

Similar inferences can be made for the six-hop connection (Figure 3.8(b)),

and TCP-NCL (K=0.5) demonstrates more significant advantages over the

other variants.

3.4.3 Congestive Loss

In the wired network with a dumbbell topology as illustrated in Figure 3.4(c),

N pairs of TCP senders and receivers are sharing a wired bottleneck link with

bandwidth 5 Mbps, propagation delay 50 ms, and buffer size 50 packets. For

i = 1, 2, ...N , one TCP connection is set up from sender Si to receiver Di.

We examine the responsiveness of TCP-NCL against congestive loss over the

topology. In general, if a TCP variant, say, TCP-X, is competently responsive

against the occurrence of congestive loss, it is expected to fulfill the following

63

0.75

0.8

0.85

0.9

0.95

1

1.05

1.1

0 10 20 30 40 50 60 70

Jain

’s F

airn

ess

Inde

x

Number of Connections

SACK TCPTCP-NCL (K=0)

TCP-NCL (K=0.5)

(a) Drop tail.

0.75

0.8

0.85

0.9

0.95

1

1.05

1.1

0 10 20 30 40 50 60 70

Jain

’s F

airn

ess

Inde

x

Number of Connections

SACK TCPTCP-NCL (K=0)

TCP-NCL (K=0.5)

(b) RED.

Figure 3.9: Jain’s fairness index (J) over wired network with a dumbbelltopology.

two objectives:

1. (TCP-friendliness) A connection driven by standardized TCP design

(e.g. SACK TCP) can maintain similar goodput when its competing

connections are driven by TCP-X as that when its competing connec-

tions are driven by standardized TCP design. This ensures that TCP-X

will not grasp bandwidth from standardized TCP in wide current usage.

64

0

0.2

0.4

0.6

0.8

1

1.2

0 10 20 30 40 50 60 70

Rat

io o

f Mea

n G

oodp

ut to

Fai

r B

andw

idth

Number of Connections

Without TCP-NCLWith TCP-NCL (K=0)

With TCP-NCL (K=0.5)

(a) Drop tail.

0

0.2

0.4

0.6

0.8

1

1.2

0 10 20 30 40 50 60 70

Rat

io o

f Mea

n G

oodp

ut to

Fai

r B

andw

idth

Number of Connections

Without TCP-NCLWith TCP-NCL (K=0)

With TCP-NCL (K=0.5)

(b) RED.

Figure 3.10: GBR over wired network with a dumbbell topology.

2. (Intra-protocol fairness) Multiple competing connections driven by TCP-

X should be able to share links fairly.

For evaluating the TCP-friendliness of TCP-NCL, we conduct three tests

with drop tail as the buffer management algorithm for the bottleneck link.

In the first test, all connections are driven by SACK TCP. In the second

and third tests, half of the connections are driven by TCP-NCL (K=0) and

TCP-NCL (K=0.5), respectively, with the other half driven by SACK TCP.

65

In each test, suppose connections 1, 2, ..., M are SACK TCP connections and

M + 1, M + 2, ..., N are non-SACK TCP connections (and thus M = N in

the first test and M = N/2 in the remaining three tests), we compute the

goodput to bandwidth ratio (GBR):

GBR =1M

∑M

i=1 xi

1N

C(3.38)

where C denotes the capacity of the bottleneck link and equals 5 Mbps. The

GBR is the ratio of average SACK TCP connections to the fair bandwidth for

each TCP connection (namely the total link capacity divided by the number

of TCP connections), and is comparable across the three tests. Moreover, in

the first test, the multiple SACK TCP connections should be able to share

the bandwidth efficiently to generate the goodput, thereby attaining a GBR

close to one. By examining the GBR obtained in the latter two tests and

comparing them with the benchmark GBR obtained in the first case, we

can evaluate how the presence of TCP-NCL (K=0)/TCP-NCL (K=0.5) will

affect standardized TCP and thus the TCP-friendliness of TCP-NCL.

We then repeat the three tests with RED as the buffer management al-

gorithm.

Figure 3.10 plots GBR against N when the buffer management algorithm

for the bottleneck link is drop tail and RED. We observe that when compet-

ing with TCP-NCL (K=0), the GBR of SACK TCP is generally not affected

in most cases. The conservatively configured instance of TCP-NCL demon-

strates good TCP-friendliness. On the other hand, when competing with

TCP-NCL (K=0.5), SACK TCP suffers up to 10% to 40% decrease in GBR.

66

With a longer expiration period of the CD timer due to large K, some retrans-

mitted packets may be acknowledged before CD timer expires even though

the originally packet is lost due to congestion. TCP-NCL (K=0.5) may thus

occasionally miss the occurrence of congestive losses, grasping bandwidth

yielded by the backoff of SACK TCP and undermining its TCP-friendliness.

For measuring intra-protocol fairness, we introduce the Jain’s fairness in-

dex [31], J , as the performance metric. For a set of N connections, 1,2,..., N ,

sharing a bottleneck link with connection goodputs x1, x2, ..., xN , respec-

tively, J is defined as:

J ,(∑N

i=1 xi)2

N∑N

i=1 x2i

(3.39)

By construction, J is bounded between zero and one, and attains one only

when all xi’s are equal. Furthermore, it equals k/N when only k out of

the N connections share the bandwidth equally and the remaining (N − k)

connections receive a zero bandwidth.

Using drop tail as the buffer management algorithm for the bottleneck

link, we perform three simulation tests with all connections driven by SACK

TCP, TCP-NCL (K=0), TCP-NCL (K=0.5), respectively. We then repeat

the three tests using RED as the buffer management algorithm.

Figure 3.9 plots J against N . We observe that all the three variants in

comparison attain J close to one. This demonstrates that TCP-NCL is ca-

pable of maintaining similar intra-protocol fairness as the standardized TCP

design SACK TCP. In particular, even though connections driven by TCP-

NCL (K=0.5) are not very friendly towards connections driven by SACK

67

TCP, they are able to divide the bandwidth fairly among themselves.

3.5 Concluding Remarks

In this chapter, we have proposed a novel TCP variant, known as TCP-

NCL, as a unified solution for performing loss recovery, sequencing control,

and congestion control over general error-prone channels. In particular, we

propose the use of two serialized timers for obtaining more reliable signals for

packet loss and network overload separately. The STS model has been con-

structed based on the concept of expected cost and closed-form expressions

are derived as references for setting the timer expiration periods. We note

that the timers are mostly determined intuitively in existing work. Our simu-

lation investigations reveal that TCP-NCL is capable of attaining significant

performance improvement over general, error-prone channels. It also demon-

strates good responsiveness against congestive loss by maintaining effective

congestion avoidance and competent TCP-friendliness.

There are several possible extensions to our work, some of which are listed

below:

1. develop a distribution model for RTT so as to make TCP-NCL a mem-

oryless algorithm, and;

2. implement and examine the performance of TCP-NCL on experimental

testbeds.

68

Chapter 4

Enhancing AQM to Combat

Wireless Losses

4.1 Chapter Overview

In this chapter, we answer the following basic question that is largely unex-

plored in the literature:

(BQ) Can active queue management (AQM) maintain a stable, small

backlog under wireless losses?

In wireless networks, TCP tends misinterpret wireless losses as conges-

tive loss and back off spuriously. Nevertheless, when many TCP flows are

sharing a bottleneck wireless link, it is unlikely that a large fraction of them

will experience wireless losses simultaneously (unless a link is broken). Thus,

spurious backoffs due to wireless losses would affect the aggregate transmis-

sion of all flows much less severely than when there are only a few flows.

The wireless link will still be kept backlogged, as shown by our simulation

69

results.1 In our simulation experiments, we find that RED fails to maintain

a stable backlog under wireless losses.

Apparently, the problem may be made trivial by applying one of the

following two approaches:

1. Design a TCP enhancement that can fully differentiate between con-

gestive and wireless losses. Unfortunately, this is hardly attainable

due to the limited network information available to the end systems.

Our simulation experiments demonstrate that the performance prob-

lem of RED persists when the end systems are running wireless TCP

enhancements like TCP-NCL. As shown by our results in Chapter 3,

TCP-NCL is already very successful in differentiating congestive and

wireless losses.

2. Use packet marking instead of packet dropping to signal network con-

gestion, such as [6]. However, we argue that a globally enabled marking

scheme is impractical due to heterogeneity of the Internet.2

We therefore opt to tackle the problem through the loss-based AQM en-

hancements, which make use of packet drops for signalling network conges-

tion. We model the dynamics of the congestion control system around its

equilibrium by a linear control system, and study the interaction of wireless

losses and AQM in this control-theoretic framework. By employing the queue

1The observation has also been reported in the literature and shaped the design prin-ciples of several enhancements for adapting TCP to high-speed and/or wireless networks.Interested readers can refer to [15, 70] and the references therein.

2For example, explicit congestion notification (ECN) [61] is the most widely adoptedmarking scheme, but is generally disabled by default in several versions of Microsoft Win-dows [17].

70

size as the system output, wireless losses are considered as disturbances to

the system.

Furthermore, we propose a family of solutions for stabilizing the queue

size against wireless losses based on internal model principle [14]. Internal

model principle rejects disturbances, or forces the part of the output due

to disturbances to asymptotically converge to zero, by including a model of

the disturbances, or poles of the disturbances, as part of the feedback control

loop. In the case of the congestion control system, it can be realized by simply

modifying the formula of the packet dropping rate in AQM. The application

of internal model principle in combating wireless losses offers the following

advantages:

1. It enables AQM to adjust the packet dropping rate to compensate for

the packet drops due to wireless losses without any specific knowledge

about the wireless packet error rate (WPER);

2. It is realized as an independent module in AQM, and thus offers good

extensibility to incorporate other AQM enhancements, such as [76], for

improving inter-flow fairness in ad-hoc networks.

By assuming the intensity of wireless losses to be slowly time-varying, we

devise the integral controller (IC) as an embodiment of internal model princi-

ple. We have developed design rules for the IC. Our simulation results show

that IC is capable of maintaining stable queue sizes and performing effective

congestion avoidance against wireless losses. Its performance is maintained

in the presence of perturbing traffic flows (e.g. HTTP and constant bit rate

(CBR) traffic).

71

Section 4.2 discusses the BQ using several simulated examples, and ex-

poses the performance problem of RED. Section 4.3 presents our system

model, a linear control system obtained by linearizing the non-linear control

system of TCP/AQM, as the basis for our analysis. Section 4.4 proposes a

family of solutions for enhancing AQM to combat wireless losses based on

internal model principle, including the IC. The design rules are developed

for the IC. Section 4.5 presents our simulation results. Section 4.6 concludes

the paper and discusses some possible extensions to our work. The work

presented in this chapter has in parts been published in [37].

Notation conventions: we use bold fonts to denote matrices and vectors,

as in “x(t)”, except for the diagonal matrix Λ. When the time index is not

specified, such as “x”, we refer to an equilibrium value. We use x(t) and x(t)

to represent (x(t)−x) and (x(t)−x), respectively. λ(.) and σ(.) denotes the

eigenvalue and singular value of a matrix, respectively.

4.2 AQM under Wireless Losses

In this section, based on a simulation study, we discuss the BQ raised in

Section 4.1: Can AQM maintain a stable, small backlog under wireless losses?

R2R15 ms

S1

S2

100 Mbps5 ms

SN100 Mbps

5 ms

...D2

100 Mbps100 Mbps

...

D1100 Mbps

DN100 Mbps

1 ms

1 ms

1 ms

Figure 4.1: A wireless bottleneck link.

72

We use the simulation topology as exhibited in Figure 4.1, similar to [1,

15]. It models the scenario that two wired LANs are connected via a wireless

link. A total of N pairs of source and destination are connected via a wireless

bottleneck link with capacity 15 Mbps, and propagation delay 50 ms. Within

each pair (Si, Di), M long-lived TCP flows are set up from Si to Ni. The size

of a link buffer is 800 packets. We introduce vary the WPER of the bottleneck

link. Time is divided into periods, each of exponential distribution, with a

mean of 100 seconds. At the beginning of each period, the WPER of the

bottleneck link is determined from a uniform distribution over [0, 4%].3 The

error rate is then kept constant during that period, whose duration follows

an exponential distribution, and is changed again at the beginning of the

next period.

The AQM algorithm simulated is RED. The parameter settings of RED

follow the result of frequency domain analysis in [27]. The queue weight and

pMax are set to 2.66 · 10−6 and 0.1, respectively. minThresh and maxThresh

of RED are set to 150 and 700 packets, respectively.

We first consider the following question: Is a wireless link backlogged un-

der wireless losses? Figure 4.2 plots the queueing dynamics when the end sys-

tems are running TCP NewReno [21], a standard TCP variant. When there

are only 20 flows traversing the bottleneck link as shown in Figure 4.2(a),

spurious backoffs of some flows due to wireless losses affect the aggregate

transmission of the flows considerably. The aggregate connection goodput is

measured to be around 12 Mbps, or 80% of the link capacity, and the back-

3We present simulation results with more exhaustive parameter settings in Section 4.5.Nevertheless, the discussion in the following applies generally.

73

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(a) N = 20, M = 1.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(b) N = 100, M = 1.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(c) N = 20, M = 5.

Figure 4.2: Queue length dynamics under varying wireless losses in the dumb-bell topology with TCP NewReno/RED.

74

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(a) N = 20, M = 1.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(b) N = 100, M = 1.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

RED

(c) N = 20, M = 5.

Figure 4.3: Queue length dynamics under varying wireless losses in the dumb-bell topology with TCP-NCL/RED.

75

log is zero most of the time. In this case, the major concern is to enhance

wireless TCP to improve the link utilization efficiency.

When the number of flows increases to 100 by increasing either the num-

ber of source-destination pairs, or the number of flows per pair, the aggregate

transmission of the flows is able to keep the bottleneck link backlogged and

fully utilized, despite the presence of wireless losses. This is clearly exhibited

in Figure 4.2(b),(c). A large N naturally corresponds to a large wired LAN.

A large M can correspond to multiple simultaneous file transfers, or TCP

enhancements that use parallel TCP sockets for better performance over

wireless networks, such as [15]. In this case, our major concern shifts from

link utilization to the backlog dynamics under wireless losses. We note that

the backlog fluctuates severely across time due to the time-varying wireless

losses. Thus, the answer to the BQ for RED is negative.

Now, we explore some feasible directions for reducing the observed back-

log fluctuations. Can TCP be enhanced so as to reduce the backlog fluctua-

tions? Figure 4.3 exhibits the queueing dynamics when the end systems are

running TCP-NCL. When there are only 20 flows as shown in Figure 4.3(a),

we note that TCP-NCL has improved the link utilization over TCP NewReno.

The link is backlogged from the 800th second to the 1300th second, and the

aggregate connection goodput is measured to be around 14 Mbps, or 93% of

the link capacity.

When the number of flows increases to 100 as shown in Figure 4.3(b),(c),

the link is always backlogged. Again, we note that the backlog fluctuates

severely, despite the increased accuracy of end systems in differentiating be-

tween congestive and wireless loss. This is because it is nearly impossible

76

to fully differentiate between congestive and wireless losses in TCP due to

the limited network information available to end systems. Thus, the backlog

fluctuation cannot be reduced significantly simply by enhancing TCP only.

The problem should be tackled by enhancing AQM directly.

We now take a closer look at the cause of the backlog fluctuations under

time-varying wireless losses. In another set of conducted simulations under

time-invariant wireless losses, we have observed that different WPER essen-

tially cause the backlog to converge to different levels. Thus, time-varying

wireless losses effectively force the backlog to switch between different equi-

librium levels from time to time. Moreover, the sluggish transient response

of RED in attaining such switching induces further backlog fluctuations.

Therefore, the key to maintaining a small, stable backlog under wireless

losses is for AQM to direct the backlog to track a fixed reference level de-

spite the variations in the WPER. We now present a systematic approach to

achieve this goal.

4.3 System Model

In this section, we present our system model. We start by constructing a

non-linear system of TCP/AQM. We then linearize the system around its

equilibrium and develop its frequency domain representations. The latter

will serve as the basis for further analysis in later sections.

We begin by considering a generic communication network with a set of

TCP flows F and a set of bottleneck links L.4 We confine our discussion to

4A non-bottleneck link delays packets for almost a constant amount of time and canthus be modelled as part of the propagation delay experienced by a traffic flow traversing

77

C(s) P(s)++

−

wp

bp ~~+r=0

Figure 4.4: Control block diagram of the linearized congestion control system.

the case of unique static routing that a Flow f traverses one identified path

Lf ⊆ L.

The non-linear system of TCP/AQM is a time-delay system. There are

time lags between when a packet is dropped over Link l, and when the source

of Flow f detects the loss, and between when the source of Flow f injects data

into the network, and when Link l receives the data. We define the former

as backward delay,←−dfl(t), and the latter as forward delay,

−→dfl(t). Denoting

the round-trip time of Flow F as df(t), it can be established that [50]:

−→dfl(t) +

←−dfl(t) = df(t) (4.1)

We define the routing matrix incorporated with equilibrium forward de-

lays as:

[RF (s)]lf ,

e−s−→dfl Flow f traverses Link l

0 otherwise(4.2)

and further denote R , RF (0).

For each TCP Flow f , the dynamics of its congestion window Wf(t)

evolves in accordance with the aggregate packet dropping probability qf (t).

that link.

78

Denote the transmission rate of Flow f as xf(t). By [50], we have:

Wf(t) = xf (t− df)(1− qf(t))1

Wf (t)

−xf (t− df)qf(t)Wf (t)

2(4.3)

In wireless networks, qf (t) is determined as:

qf (t) ≈∑

l∈L

Rlf(pl(t−←−dfl) + pwl(t−

←−dfl)) (4.4)

where pl(t) and pwl(t) are the packet dropping probabilities over Link l due

to AQM and wireless losses, respectively. We note that a packet needs to be

stored in the buffer managed by AQM, before being transmitted over the link

subject to wireless losses. Thus, the delay between packet transmission and

packet drop due to wireless losses is slightly greater than the delay between

packet transmission and packet drop due to AQM. Nevertheless, we assume

the difference to be negligible for simplicity.

For each link l, the instantaneous queue size bl(t) is determined by the

aggregate incoming data rate yl(t) and the link capacity cl, following the

system dynamics:

bl(t) = yl(t)− cl (4.5)

yl(t) is the sum of the time-delayed source rates of all flows traversing Link

79

l:

yl(t) =∑

f∈F

Rlfxf (t−−→dfl) ≈

∑

f∈F

Rlf

Wf(t−−→dfl)

df(t)(4.6)

The approximation above is made based on the assumption that df(t), which

is determined by the collected transmission of many flows, varies in a larger

time scale than xf (t).

Besides data transmission and packet dropping, the dynamics of flows

and links are also coupled via the RTTs of the flows. RTT is the sum of

queueing delays at links and round-trip propagation delay, τf :

df(t) = τf +∑

l

Rlf

bl(t)

cl

(4.7)

Denote the Laplace transforms of (pl(t) : l ∈ L), (pwl(t) : l ∈ L), and

(bl(t) : l ∈ L) as p(s), pw(s), and b(s), respectively. Following a similar

procedure as [27], we linearize the non-linear system specified in (4.3), (4.4),

(4.5), and (4.6) around its equilibrium attained with pw(t) ≡ 0, and apply

the Laplace transform to the resulting linear system. Define a plant transfer

function matrix, P(s), as the transfer function from (p(s) + pw(s)) to b(s),

we obtain:

P(s) = −(

sI + R diag(Wf

d2f

)RT diag(1

cl

)

)−1

·RF (s) diag

e−sdf

(s +qfWf

df)qfd2

f

RF (−s)T

(4.8)

80

P(s) represents the interaction of the TCP dynamics, queueing dynamics,

and routing. The stability of this open-loop system can be established by

the following lemma.

Lemma 4.1 The system specified in (4.8) is stable if R is of full row rank.

Proof It suffices to show that all eigenvalues of Ω , R diag(Wf

d2f

)RTdiag( 1cl)

are negative.

Since eigenvalues are invariant under matrix commutation,

λ(Ω) = λ(diag 1

cl

R diag(Wf

d2f

)RT ) (4.9)

By definition, there exists a unit vector v such that:

λ(Ω)v = diag 1

cl

R diag(Wf

d2f

)RTv (4.10)

or

λ(Ω) =vTR diag(

Wf

d2f

)RTv

vT diagclv

=‖diag(

√Wf

df)RTv‖22

‖diag√clv‖22≥ 0 (4.11)

The equality holds only when RTv = 0, which cannot be attained since RT

is full column rank and v 6= 0.

It is time to include AQM into the model. In the time domain, AQM

determines p(t) based on b(t). In the frequency domain, it can be represented

by a transfer function matrix of dimension |L| × |L|, C(s), from the queue

81

size deviation b(s) to the deviation of packet dropping rate by AQM p(s).

The control loop is therefore closed, as shown in Figure 4.4.

The stability of this closed-loop system can be established by the following

lemma.

Lemma 4.2 The system in Figure 4.4 is stable if det(I+P(s)C(s)) contains

zeros on the open left half plane (OLHP) only.

Proof The closed-loop transfer function from the reference signal, r, to b is

given by:

Grb(s) = (I + P(s)C(s))−1P(s)C(s) (4.12)

This is stable if the set of poles for Grb(s), which is a subset of the zeros

of det(I + P(s)C(s)), are on OLHP.

Finally, the role of wireless losses is represented by pw in Figure 4.4. To-

gether with p(s), pw(s) determines the system output b(s) via P(s). How-

ever, while p(s) is in turn determined by b(s) via C(s) and thus part of

the feedback control loop, pw(s) is not in the loop, but serves as an external

disturbance to the system.

4.4 Robust AQM based on Internal Model

Principle

In this section, we develop an approach to track the backlog to a reference

level despite the presence of wireless losses, based on the frequency domain

82

representation of the TCP/AQM system under wireless losses as shown in

Figure 4.4. Per our discussion in Section 4.2, the objective is to maintain a

stable, small backlog under wireless losses.

In the time domain, the prescribed objective is met if b(t), the backlog

deviation from the equilibrium level, converges asymptotically to zero. This

means that b(s) does not contain any unstable poles in the frequency domain.

We begin by noting the following relationship between b and pw:

b(s) = (I + P(s)C(s))−1P(s)pw(s) (4.13)

On the other hand, without loss of generality, we can express the distur-

bance of wireless losses, pw, as:

pw(s) =1

φ(s)k(s) (4.14)

where φ(s) ∈ F contains all the unstable poles of pw and k(s) , φ(s)pw(s).

F denotes the set of the rational functions in s. Obviously, all entries in

k(s) will not contain any unstable poles. The unstable poles determine the

asymptotic behavior of pw(s). Thus, 1φ(s)

is often referred to as a model of

the disturbance.

It follows that:

b(s) =1

φ(s)(I + P(s)C(s))−1P(s)k(s) (4.15)

When the system in Figure 4.4 is stable, the unstable poles of b(s), if any,

come from the unstable poles contained in φ(s). To attain disturbance rejec-

83

tion of pw, or to prevent the unstable poles contained in φ(s) from becoming

poles of b(s), the internal model principle proposes to insert the model of

disturbance 1φ(s)

inside the feedback loop [14].

The design of AQM determines C(s), whereas the plant transfer function

P(s) is invariant across different AQM algorithms. Therefore, we put 1φ(s)

inside the feedback loop by selecting C(s) as:

C(s) =1

φ(s)ΛN(s)ΛD(s)−1 (4.16)

where ΛN(s), ΛD(s) ∈ F|L|·|L| are coprime diagonal matrices. det(ΛN(s)) does

not share roots with φ(s), and det(ΛD(s)) does not have any roots on closed

right half plane. We constrain C(s) to be diagonal so that each intermediate

router can execute AQM algorithm distributedly without incurring the need

of exchanging the instantaneous queue sizes with each other.

Theorem 4.1 Consider the system in Figure 4.4 with P(s) and C(s) as

specified in (4.8) and (4.16), respectively. Suppose the conditions of Lem-

mas 4.1 and 4.2 hold so that both open-loop and closed-loop stabilities are

established. Let P(s) = D−1(s)N(s) be a left coprime factorization. The

system attains disturbance rejection of (4.14) if det(N(s)) does not contain

zeros at the roots of φ(s).

Proof Substituting (4.16) into (4.15) gives:

b(s) = ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s) (4.17)

Disturbance rejection is attained if none of the zeros of det(φ(s)D(s)ΛD(s)+

84

N(s)ΛN(s)), which include the poles of b(s) as a subset, are on closed right

half plane (CRHP). For the roots of φ(s) (which are unstable poles of pw(s)

and thus on CRHP), the determinant reduces to:

det(N(s)ΛN(s)) = det(N(s)) det(ΛN(s)) (4.18)

This is non-zero since neither det(ΛN(s)) nor det(ΛN(s)) contains zeros at

the roots of φ(s) by assumption.

For s ∈ s|ℜ(s) ≥ 0 and φ(s) 6= 0:

det(φ(s)D(s)ΛD(s) + N(s)ΛN(s))

= det(D(s)) det(ΛD(s))

· det(φ(s)I + P(s)ΛN(s)ΛD(s)−1)

= φ(s)|L| det(D(s)) det(ΛD(s))

· det(I + P(s)C(s)) (4.19)

We now consider the three determinants in the final expression above one

by one. As R is of full row rank, P(s) does not have any unstable poles and

thus det(D(s)) does not contain any zeros on CRHP. Neither det(ΛD(s)) nor

det(I + P(s)C(s)) contain zeros on CRHP by assumption.

Since CRHP is a subset of s|φ(s) = 0⋃s|ℜ(s) ≥ 0 and φ(s) 6= 0,

we conclude that det(φ(s)I + P(s)ΛN(s)ΛD(s)−1) does not contain zeros on

CRHP.

Remark 4.1 When det(D(s)) does contain zeros at the roots of φ(s), there

will be some cancellations of unstable zero-pole pairs between C(s) and P(s).

85

This violates the condition of total stability, which means that every possible

input-output pair of the system is bounded-input-bounded-output (BIBO) sta-

ble. It is one of the physical constraints that a practical system should meet.

Interested readers can refer to [14] for details.

Remark 4.2 The attained disturbance rejection is robust against pertur-

bance to P(s), ΛN(s), and ΛD(s), as long as the mild assumptions of Theo-

rem 4.1 continue to hold after the perturbance. In practice, TCP AIMD pa-

rameters, routing topologies, and network capacities, may change over time,

leading to such perturbance. The AQM design specified in (4.16) can nonethe-

less maintain robustness against wireless losses.

Remark 4.3 Indeed, the attained disturbance rejection is not robust against

errors in implementing φ(s). Consider when, due to such errors, C(s) be-

comes

C(s) =1

φ(s)ΛN(s)ΛD(s)−1 (4.20)

where φ(s) is an approximation of φ(s). It follows that:

b(s) =

b1(s) ,︷ ︸︸ ︷

ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)

+

b2(s) ,︷ ︸︸ ︷

ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)φ(s)− φ(s)

φ(s)

(4.21)

Provided that the assumptions of Theorem 4.1 hold for C(s) specified in

86

(4.20), b1(s) contains stable poles only. b2(s) contains the unstable poles

of φ(s), and represents the steady state errors in tracking the backlog to a

reference size due to the inexact cancellation between φ(s) and φ(s).

Nevertheless, it is still very meaningful to apply the internal model princi-

ple with inexact implementation of φ(s). This reduces the steady state errors

than when the principle is not applied. For example, consider the simple case

that φ(s) = s− a and φ(s) = s− a + ǫ, where a and ǫ represent an unstable

pole and the error involved in implementing φ(s), respectively. b2(s) then

reduces to:

b2(s) = ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)ǫ

s− a(4.22)

which is proportional to ǫ. Therefore, we can keep the steady state error

minimal with a good approximate implementation of φ(s).

Generally, Theorem 4.1 can be applied in two directions. On the one

hand, when φ(s) is known5 , we construct an appropriate C(s) to yield a

desired disturbance rejection. For example, wireless losses may vary periodi-

cally at a frequency, say, 1T, resulting in φ(s) = 1−e−Ts. An AQM with C(s)

containing (1 − e−Ts) as the denominator can potentially help to maintain

a stable, small backlog under wireless losses with a periodic time-varying

packet loss rate. This deserves further study in the future.

On the other hand, without any specific knowledge of φ(s), φ(s) can be

designed to serve as a good approximation for the general scenarios, thereby

5We note that φ(s) corresponds to the “pattern” in the variation of wireless losses.Knowledge about φ(s) is a much weaker assumption than knowledge about the exactWPER.

87

attaining good performance in disturbance rejection.

We are going to investigate the design of such a controller in the next

subsection.

4.4.1 IC

Here, we approximate the disturbance pw as a step input function. This

approximation is valid as long as the linear system in consideration converges

much faster than the variations in the WPER. It follows that φ(s) = s. From

(4.16), we can choose the diagonal element of C(s) as:

Cll(s) =gl

s, 1 ≤ l ≤ |L| (4.23)

which is essentially an integral controller (IC). It converts its input into the

integral of that input in the time domain. The integral controller will be part

of any other C(s) satisfying (4.23) and φ(s) = s, and is the essential element

fulfilling the internal model principle. Denote:

RF (·) , diagcl−1RF (·)diagWf

df

(4.24)

ωg , 0.1 minl∈L,f∈F

(

2

dfWf

,1

df

,σ2

min(RF (0)) cl

Wf

)

(4.25)

The following proposition gives the design rules for the integral controller.

Proposition 4.1 Consider the system in Figure 4.4 with P(s) and C(s)

as specified in (4.8) and (4.23), respectively. Suppose that R is of full row

88

rank. Let gl be chosen as g

cl. The system is closed-loop stable and attains

disturbance rejection of pw as specified in (4.14) with φ(s) = s if:

g ≤ ωg ·4 σ2

min(RF (0))

‖RF (0)‖11

(maxf∈F df Wf ) maxf∈F Wf

(4.26)

In the case of homogeneous flows sharing a bottleneck link, if maxf Wf ≥ 2,

a simplified condition can be obtained as:

g ≤ 0.8(N−)3

d5+c3

(4.27)

where N−, d+, and c correspond to the lower bound for the number of flows,

the upper bound for RTT, and the link capacity, respectively.

Proof By the generalized Nyquist criteria [18], it suffices to prove6 that the

eigenloci of the return ratio (−P (jω)C(jω)) does not encircle (−1, 0) on the

complex plane for ω 6= 0. By (4.24), We can rewrite (−P(jω)C(jω)) as:

−P(jω)C(jω)

=1

ω(jωdiag(

1

cl

) + RF (0)diag(1

Wf

)RF (0)T )−1RF (jω)

·diag

exp(j(−π

2− ωdf))

2(jω +qfWf

df)

RF (−jω)T diag(Ilcl)

(4.28)

If λ(jω) is an eigenvalue of P(jw)C(jw), there exists a normalized vector

6A complete proof should also cover the eigenloci of −P (δe−jω)C(δe−jω) for all ω ∈[−0.5π, 0.5π] as δ → 0. Yet, the proof is trivial.

89

v such that:

λ(jω)

=

v∗RF (jω)diag

(

j(−π2−ωdf ))

2(jω+qf Wf

df)

)

RF (−jω)Tv

ω

g(jωv∗diag(

1

cl

)v + v∗RF (0)diag(1

Wf

)RF (0)Tv)

(4.29)

Note that:

σ2max(RF (−jω)) ≤ ‖RF (jω)RF (−jω)T‖1

≤ ‖RF (jω)‖1‖RF (−jω)T‖1

= ‖RF (0)‖1 (4.30)

For ω ∈ [0, ωg], we obtain:

λ(jω)

≈v∗RF (jω)diag

(−jdf

2qfWf

)

RF (−jω)Tv

ω

gv∗RF (0)diag(

1

Wf

)RF (0)Tv

(4.31)

With the crossover frequency bounded below by ωg and |λ(jω)| decreasing

with ω, it suffices to have |λ(jωg)| < 1, which follows from (4.26).

In the case of N homogeneous flows sharing a bottleneck link of capacity

90

c, RF (0)RF (0)T = 1N

. Thus,

ωg = 0.1 minl∈L,f∈F

(2

dfWf

,1

df

,c

NWf

)

=0.2

max dfWf

(4.32)

and (4.26) reduces to

g ≤ ωg ·4

(maxf∈F df Wf ) maxf∈F Wf

=0.8

(maxf∈F df Wf )2 maxf∈F Wf

=0.8N3

c3d5f

(4.33)

The final line above follows since all the flows are homogeneous.

The RHS of (4.33) is lower bounded by 0.8(N−

)3

d5+

c3and therefore (4.27) serves

as a more stringent lower bound.

Remark 4.4 In [26], the proportional-integral (PI) controller is designed in

the form of kl(s+zl)s

, which however cannot reduce to the integral controller as

specified in (4.23).

After discretization via the bilinear transform for an IC, the packet drop-

ping rate for each link l can computed as:

pl[k] = pl[k − 1] + Gl(bl[k]− bl) (4.34)

where Gl = gl

fl. fl is the sampling frequency, i.e., the frequency at which

the queue size is sampled and (4.34) is used to update the packet dropping

rate. pl[k] and bl[k] are the step-wise packet dropping rate and queue size,

91

R3

... ...

R1

...

S3K

S32

S31

50msR2

...

S11 S12 S1N S21 S22

D11 D1ND12 D21 D22

50ms

...

...

D31

D32

D3K

S2H

D2H

12.8 Mbps 9.6 Mbps

Figure 4.5: Two wireless bottleneck links.

respectively. The computation can be realized as an independent module in

AQM, taking the queue length of a buffer as input and computing the packet

dropping rate due to the IC as the output.

Example 4.1 Consider the scenario illustrated in Figure 4.1, with the wire-

less link of capacity 15 Mbps and propagation delay 50 ms. A data packet

is 1000 bytes long, and thus cl = 1875 packets/second. A total of N = 100

homogeneous TCP flows share the link. Suppose that the equilibrium queue

size is bl = 220 packets. It follows that df is around 0.2 seconds.

Applying (4.27) using d+ = 0.2 seconds, N− = N = 100, c = cl =

1875 Packets,

g ≤ 0.8 · 1003

0.25 · 18753≈ 0.3792

Example 4.2 Consider another network scenario illustrated in Figure 4.5,

which can arise when three wired LANs are inter-connected to create a WAN

via wireless links. Three sets of flows F1, F2, and F3 share a network. There

are N1 = 60, N2 = 50, and N3 = 20 flows in F1, F2, and F3, respectively.

Flows within the same set are homogeneous. Each of those flows in Fi (i =

1, 2, 3) traverses from Sij to Dij where 1 ≤ j ≤ 60 for i = 1, 1 ≤ j ≤ 50 for

92

i = 2, and 1 ≤ j ≤ 20 for i = 3. We thus have two wireless bottleneck links,

whose dynamics are coupled via flows in F3. Suppose that the equilibrium

queue sizes of both wireless links are b1 = b2 = 200 packets. Let di and Wi be

the equilibrium RTT and window size for each of the flows in Fi, respectively.

It follows that:

c = (1600, 1200) Packets/Second

d = (0.18, 0.22, 0.35)T Seconds

W = (4.2, 4.5, 3.2)T Packets

and thus ‖RF (0)‖1 = 0.017.

Now, the matrix RF (0)RF (0)T becomes:

RF (0)RF (0)T ≈

0.014 0.00065

0.00065 0.015

(4.35)

which gives σ2min(RF (0)) = 0.0136.

By (4.25) and (4.26),

ωg ≈ 0.1 min(2

0.35 · 3.2 ,1

0.35,0.0136 · 1200

4.5) = 0.1786

g ≤ 0.1786 · 4 · 0.0136

0.017· 1

(0.35 · 3.2) · 4.5 ≈ 0.1134

93

4.5 Performance Evaluation

In this section, we evaluate the performance of IC and compare it with RED

using ns-2 simulation. This is necessitated by that the robustness and stabil-

ity of the IC have been established in the context of the linearized congestion

control system, which is nonlinear in reality. Sections 4.5.1 and 4.5.2 present

results obtained in simulation topologies illustrated in Figures 4.1 and 4.5,

respectively.

For IC, the packet dropping rate is updated at a frequency of 10 HZ. Nev-

ertheless, we have repeated the experiments with a different implementation

of IC, which updates packet dropping rate at every packet arrival as RED

does, with similar results reported.

The parameters of IC and RED follow Proposition 4.1 and references [26,

27], respectively. For RED, pMax and the queue weight are set to 0.1 and

2.6 × 10−6, respectively. The parameters minThresh and maxThresh of

RED, and the gain of IC, Gl, vary and will be specified in the following.

4.5.1 One Wireless Bottleneck Link

In the topology illustrated in Figure 4.1, a total of N pairs of source and

destination are connected via a wireless bottleneck link with capacity c Mbps,

and propagation delay τ ms. Within each pair (Si, Di), M long-lived TCP

Reno flows are sent from Si to Di. The size of the link buffer and the reference

queue size of IC are set to 800 Packets and 220 Packets, respectively. We

introduce variable WPER into the bottleneck link. Time is divided into

periods, each of exponential distribution, with a mean of lossP seconds. At

94

the beginning of each period, the WPER of the bottleneck is determined

randomly from a uniform distribution over [0, mL].

Figures 4.6-4.9 plot the traces of queue sizes under various configurations

summarized in Table 4.1. The ”HTTP” and ”CBR” columns in Table 4.1

indicates whether there are HTTP and CBR traffic flows coexisting with the

long-lived TCP flows, respectively. The final three columns are parameter

settings of IC and RED and computed from the network settings.

95

Table 4.1: Network configurations for the single wireless bottleneck network.

Figure N M c τ mL lossP HTTP CBR Gl minThresh maxThreshFigure 4.6(a) 100 1 15 50 4% 100 No No 2 · 10−5 150 700Figure 4.6(b) 20 5 15 50 4% 100 No No 2 · 10−5 150 700Figure 4.7(a) 20 5 15 50 8% 100 No No 2 · 10−5 150 700Figure 4.7(b) 20 5 15 50 4% 20 No No 2 · 10−5 150 700Figure 4.8(a) 20 5 15 200 4% 100 No No 2 · 10−7 150 400Figure 4.8(b) 20 10 15 200 4% 100 No No 1.6 · 10−6 150 400Figure 4.8(c) 20 10 30 50 4% 100 No No 1 · 10−5 150 400Figure 4.9(a) 20 5 15 50 4% 100 Yes No 2 · 10−5 150 700Figure 4.9(b) 20 5 15 50 4% 100 No Yes 2 · 10−5 150 700Figure 4.9(c) 20 5 15 50 4% 100 Yes Yes 2 · 10−5 150 700

96

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(a) N = 100, M = 1.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(b) N = 20, M = 5.

Figure 4.6: Comparison of queue length dynamics with same configurationsas in Figure 4.2.

Figure 4.6(a) and Figure 4.6(b) essentially retain the same configurations

as Figure 4.2 and Figure 4.2(b), respectively. We observe that IC helps main-

tain the queue size stable throughout the simulation period. Despite the vari-

ations in WPER, the equilibrium queue size of IC tracks the reference level

(220 Packets) very well. We do observe occasional “spikes” in the queueing

dynamics with IC. This is because IC needs to converge to a new equilibrium

packet dropping rate to maintain the same equilibrium queue size when the

97

WPER changes. Nevertheless, the transition is much smoother than that of

RED since the equilibrium queue size is unchanged and IC has better tran-

sient response than RED. Moreover, while the difference between the two

equilibrium packet dropping rate should be approximately the change in the

WPER, IC is self-adaptive and does not require any knowledge about the

WPER to complete the transition. We note that this is an essential prop-

erty of wireless AQM enhancements developed based on the internal model

principle.

In addition, Figure 4.6(a) and Figure 4.6(b) exhibit similar queueing dy-

namics. Generally, in all our conducted simulation experiments, the queueing

dynamics are quite similar across different values of M for 1 ≤ M ≤ 10, as

long as the total number of flows, NM , is kept constant.

Figure 4.7(a) and Figure 4.7(b) exhibit the queueing dynamics against

wireless losses with larger variations and more frequent variations in WPER,

respectively. Correspondingly, RED exhibit more severe and more frequent

fluctuations, respectively. IC basically maintains a stable queue size around

the reference level in both cases, except that occasional spikes due to transi-

tion becomes larger in Figure 4.7(a). This is as expected, since more abrupt

“jumps” in the WPER requires IC to undergo a greater change in the equi-

librium packet dropping rate.

Figure 4.8(a)-(c) exhibit the queueing dynamics with “fatter” pipes, i.e.,

a bottleneck link of larger propagation delay or higher bandwidth. Fig-

ure 4.8(a) increases the propagation delay to 200 ms, approximately the

two-way propagation delay with a geostationary satellite [3], and maintains

the total number of flows to be 100. We observe that queue dynamics tend

98

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(a) Larger variations in WPER.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(b) More frequent variations in WPER.

Figure 4.7: Comparison of queue length dynamics under different wirelesslosses.

99

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(a) Fatter pipe due to increased link delay.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(b) Fatter pipe due to increased link delay is sharedby more TCP flows.

0

100

200

300

400

500

600

700

800

0 200 400 600 800 1000 1200 1400

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(c) Fatter pipe due to increased bandwidth is sharedby more TCP flows.

Figure 4.8: Comparison of queue length dynamics with different bottlenecklink parameters.

100

to fluctuate very severely with both IC and RED. In fact, the link is not

backlogged for significant portion of time.

The congestion window of TCP flows regulate the number of packets

in transit. It grows larger with fatter pipes, since more packets in transit

are needed to keep the pipe full. When a packet is lost due to wireless

losses, the corresponding flow will spuriously reduce its window by half. With

fatter pipes, the window reduction in terms of packets becomes larger. This

leads to more severe fluctuations in the number of packets in transit, which

will possibly fall below the size of the pipe (≈ 2cτ) and leave the link not

backlogged. We recall that, when a link is not backlogged, it is outside the

operational region of wireless AQM enhancements.

To mitigate the effect of fatter pipes, we can increase the number of

TCP flows so that the congestion window of a single flow becomes smaller.

Figure 4.8(b) plots the queueing dynamics when the total number of flows

is increased to 200. In this case, we observe that the link is backlogged

throughout the simulation, and IC can effectively maintain a stable queue

size.

Similarly, Figure 4.8(c) plots the queueing dynamics when 200 flows are

sharing a bottleneck link of doubled bandwidth, in which IC still demonstrate

a robust tracking of the reference queue size. The backlog fluctuation with

RED is not very significant in this case due to the reduction in maxThresh as

computed. maxThresh is the maximum level of queue size allowed by RED.

However, such reduction is generally not recommended in other scenarios due

to the stability requirement of RED [27].

Figure 4.9(a)-(c) exhibit the queueing dynamics with the presence of other

101

0

100

200

300

400

500

600

700

800

0 500 1000 1500 2000 2500 3000

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(a) HTTP.

0

100

200

300

400

500

600

700

800

0 500 1000 1500 2000 2500 3000

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(b) CBR.

0

100

200

300

400

500

600

700

800

0 500 1000 1500 2000 2500 3000

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(c) HTTP and CBR.

Figure 4.9: Comparison of queue length dynamics with the presence of othertraffic.

102

traffic. Figure 4.9(a) plots the queueing dynamics when we introduce the

HTTP traffic into the dumbbell topology. In doing so, we attach 120 web

client nodes to R1 and ten web server nodes to R2. We can see that the

traces of the queue size are more “noisy”, since the introduction of the HTTP

traffic leads to a higher fluctuation on the network load. Nevertheless, IC

demonstrates an effective and robust tracking of the reference queue level

against the disruptions from both perturbing traffic and wireless losses.

Figure 4.9(b) plots the queueing dynamics when we introduce CBR traf-

fics into the dumbbell topology. In doing so, we add attach 30 UDP sources

to R1 to and their corresponding destinations to R2, with each UDP flow

driven by long-lived CBR at the constant rate of 0.1 Mbps. The UDP flows

represent unresponsive flows that do not back off upon network congestion.

With RED, the backlog generally shifts up due to the increased long-lived

traffic loading. On the other hand, the robust tracking by IC of the reference

queue level maintains despite the presence of unresponsive flows.

Figure 4.9(c) plots the queueing dynamics when both HTTP and CBR

traffic are present. We can identify the aforementioned impact of introducing

HTTP and CBR traffic. With RED, the backlog trace becomes more “noisy”

and shifts up as well. With IC, the backlog becomes more “noisy” but still

tracks the reference level.

4.5.2 Two Wireless Bottleneck Links

In the topology illustrated in Figure 4.5, network configuration follow those

in Example 4.2. The size of the two link buffers are both 600 packets. For

103

0

100

200

300

400

500

600

700

800

0 500 1000 1500 2000 2500 3000

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(a) Link 1.

0

100

200

300

400

500

600

700

800

0 500 1000 1500 2000 2500 3000

Que

ue le

ngth

(nu

mbe

r of

pac

kets

)

Time (second)

REDIC

(b) Link 2.

Figure 4.10: Comparison of queue length dynamics in the topology of twowireless bottleneck links.

104

both links, minThresh and maxThresh of RED are set to 150 and 500 packets,

respectively. Gl’s are set to be 7 · 10−6 and 9.5 · 10−6 for Link 1 and Link

2, respectively. We introduce time-varying losses into both links, with an

average loss period of 100 s and a maximum WPER of 3 %. The simulation

results are plotted in Figure 4.10. Again, severe fluctuations can be observed

in the queue sizes with RED over both Link 1 and Link 2. On the other

hand, the IC is successful in maintaining stable queue sizes throughout the

simulation.

4.6 Concluding Remarks

In this chapter, centered around the BQ “Can AQM maintain a small, stable

queue under wireless losses?”, we carry out a series of study. We find that

RED, one of the most representative AQM, fail to maintain a stable queue

size against time-varying wireless losses. The key to solving the problem is for

the AQM to force the queue size to track a reference queue level despite the

presence of wireless losses. Based on the frequency domain representation of

TCP/AQM, we apply the internal model principle to reject the disturbance

of wireless losses, thereby attaining such tracking. We devise the integral

controller (IC) as an embodiment of the internal model principle, and develop

a set of design rules for IC. We have conducted a series of experiments to

examine the performance of IC and compare it with RED. In particular, we

have evaluated the impact of severe variations of wireless losses, fat pipes,

and HTTP traffics on the two AQM algorithms. In general, IC demonstrates

very robust performance in maintaining a small, stable backlog under wireless

105

losses.

There are several possible extensions to our work, including:

1. developing a global stability analysis for the system of TCP/buffer-

based AQM perturbed by wireless losses;

2. applying Theorem 4.1 to design an AQM that attains clean disturbance

rejection of wireless losses with known φ(s), as discussed in Section 4.4,

and;

3. developing a unified AQM solution for resolving both fairness and sta-

bility issues in ad-hoc wireless networks.

106

Chapter 5

Conclusions and Future

Research

5.1 Chapter Overview

In this chapter, we conclude the thesis and discusses some future research

directions. Section 5.2 discusses main results acquired and summarizes the

major contributions of the thesis. Section 5.3 discusses some topics worth

further investigation. Section 5.4 concludes the chapter with a discussion.

5.2 Contributions

In Chapter 2, we conduct a literature survey of the work performed to date on

wireless TCP enhancements and on design and analysis of AQM. We find that

there is a lack of transport layer unified solution for enhancing TCP against

packet reordering and non-congestive loss. We identify that the performance

107

of AQM under wireless losses is largely unexplored in the literature.

In Chapter 3, we have developed a ideal TCP sender model, known as

smart TCP sender (STS), for constructing more reliable signals of packet loss

and network congestion over general, error-prone channels. The two signals

employed are the expirations of two serialized timers, where the second timer

is started upon the expiration of the first timer. The timer expiration periods

have been optimized for minimizing quantified risks associated with spurious

congestion response and excessively delayed congestion response.

We have devised a novel TCP variant, known as TCP for non-congestive

loss (TCP-NCL), as a practical approximation of the STS model. TCP-NCL

can thus serve as a unified solution for effective congestion control, sequenc-

ing control, and loss recovery over wireless networks. The deployment of

TCP-NCL requires modification to sender-side TCP only, thereby facilitat-

ing possible future wide deployment. The deployment of TCP-NCL requires

maintenance of a few unused bits in the existing data structure for TCP and

a few extra variables only. The maintenance is performed upon arrival of

acknowledgement packet.

We have performed extensive simulation experiments to examine the per-

formance of TCP-NCL and compare it with several well-known TCP variants

in the literature using Network Simulation (ns) 2.29. Our results report that

TCP-NCL attains significant performance improvement over general, error-

prone communication networks, thereby demonstrating robustness against

packet reordering and non-congestive packet loss. At the same time, multiple

competing TCP connections, with some employing TCP-NCL and the others

employing standardized TCP design, are capable of sharing bandwidth fairly

108

over conventional wired networks. TCP-NCL is therefore responsive against

congestive loss, and backward compatible with the present TCP Internet.

In Chapter 4, we raise the basic question: Can AQM maintain a sta-

ble, small backlog under wireless losses? Our simulation results report that,

while many TCP flows can keep a wireless link backlogged despite spurious

backoffs, RED fails to maintain a stable backlog under time-varying wireless

losses. Due to the fundamentally limited information on network loading

available to end systems, the backlog stabilization should be best attained

via AQM enhancements. We further identify that the key to resolving the

problem is to robustly track the backlog to a preset reference level.

Based on a linear model of TCP/AQM, we apply the internal model

principle to realize such tracking. The application of internal model principle

in combating wireless losses enables AQM to adjust the packet dropping rate

to compensate for the packet drops due to wireless losses without any specific

knowledge about the wireless packet error rate (WPER). The principle can be

realized as an independent module in AQM, and thus offers good extensibility

to incorporate other AQM enhancements.

By assuming the intensity of wireless losses to be slowly time-varying,

we devise the integral controller (IC) as an embodiment of internal model

principle. We have developed design rules for IC. Our simulation results show

that IC is capable of maintaining stable queue sizes and performing effective

congestion avoidance against wireless losses. Its performance is maintained

in the presence of perturbing traffic flows (e.g. HTTP traffic).

The major contributions of the thesis are recapped as follows:

1. A TCP sender model for constructing reliable signals of packet loss and

109

network congestion over general, error-prone communication networks

Based on the novel serialized-timer structure, the STS model constructs

reliable signals of packet loss and network congestion. It attains an

optimal tradeoff between signal reliability and responsiveness against

congestion by minimizing quantified risks associated with spurious con-

gestion response and excessively delayed congestion response.

2. A unified solution for effective congestion control, sequencing control,

and loss recovery over wireless networks

As a transport layer solution, TCP-NCL enables accurate differentia-

tion among the occurrences of packet reordering, congestive loss, and

non-congestive loss. It significantly enhances robustness against packet

reordering and non-congestive loss while maintaining responsiveness

against congestive loss.

3. A simulation-based investigation of the performance of AQM under

wireless losses

The series of simulation experiments on the performance of RED under

time-varying losses reveal the irrobustness of RED. It helps to clarify

the different operational regions of TCP and AQM enhancements. Due

to the fundamentally limited information on network loading available

to end systems, the backlog stabilization should be best attained via

AQM enhancements. The results further indicate that the key to main-

taining a stable backlog is to robustly track the backlog to a preset

reference level.

110

4. A systematic AQM enhancement for backlog stabilization under wireless

losses

The application of internal model principle enables AQM to adjust the

packet dropping rate to compensate for wireless losses without requiring

exact knowledge of the WPER, thereby maintaining a stable backlog

under time-varying wireless losses. This significantly improves end-to-

end delay performance and fairness towards bursty traffic.

5.3 Future Research

Our major results, as summarized in Section 5.2, demonstrate that the In-

ternet congestion control fulfilled by TCP and AQM can be systematically

adapted to the wireless environment at a minimal deployment cost and dis-

ruptions, preserving the loss-based nature. However, there remain several

research directions worth thorough investigation, such as those enlisted at

the end of Chapters 3 and 4. We discuss two topics in greater depth in the

following.

5.3.1 A Theoretical Framework for Wireless TCP

The literature on wireless TCP enhancements is vast [44]. Various propos-

als generally adapt TCP to the dynamic wireless environment by exploit-

ing readily available or self-constructed signals of the occurrences of various

events. For example, the network loading conditions can be inferred from

RTT, packet losses, or explicit network feedbacks.

111

However, a theoretical framework is lacking in at least two aspects. First,

it would be highly desirable to compare the various signals in terms of quan-

tified metrics and operational regions. For example, RTT provides richer

information about network loading than packet losses, thereby enabling finer

tuning of end system transmission. The merit is particularly useful in alle-

viating the fluctuation of TCP in high-speed networks [71]. On the other

hand, RTT can be varied by non-congestive factors in many scenarios such

as reordering networks, thereby undermining its reliability as compared with

packet loss. A systematic categorization and quantified characterization of

these signals would vastly facilitate understanding about the strength and

limitations of existing TCP variants, and development of new TCP variants

that push towards fundamental limits.

Second, wireless TCP enhancements generally modify TCP at a finer

granularity, or in a more general manner, than have been modeled in the

literature. For example, many TCP variants opt to proactively postpone

packet retransmission and congestion response until congestive loss is prob-

able. In the popular control-theoretic formulation [33], the delay would not

change the converged equilibrium as long as the stability condition is ob-

served. Yet, the simulation findings reveal that it requires extra effort to

maintain TCP-friendliness of such TCP variants [10]. The discrepancy is

partly because the sophisticated impact of RTO remains to be taken into

account. Consequently, the TCP-friendliness and fairness of wireless TCP

enhancements have to be examined via extensive simulation experiments,

which cannot exhaust all possible network scenarios in reality. As another

example, some recent results on a network with heterogeneous congestion

112

control [67, 68], namely a network with multiple congestion control schemes

based on different signals, assume each congestion control scheme utilizes

one type of signal only. However, TCP variants, such as TCP-NCL proposed

in this thesis, tends to exploit information conveyed by the mix of multiple

signals to attain better performance. Therefore, it remains to develop theo-

retical understanding regarding the dynamics and equilibrium of a network

shared by TCP variants. This helps to clarify the set of constraints a TCP

proposal needs to meet to ensure inter-flow fairness.

The aforementioned two aspects are expected to be complementary. The

systematic study of signals available to end systems essentially views the

network as a black box and focuses on optimizing a single TCP flow. This

hopefully will give rise to a generalized expression of TCP dynamics, and

provides a basis for studying the network of TCP variants. A better un-

derstanding of the equilibrium and dynamics of such networks, on the other

hand, provides further performance metrics for our study of the signals.

5.3.2 A Unified AQM Solution for Wireless Networks

Wireless networks bring unique challenges to AQM that are not covered by

the vast literature on the design and analysis of AQM in wired networks.

First, as reported in Chapter 4, different WPERs can cause the backlog to

converge to different levels, and time-varying WPER can induce fluctuations

in backlog. To the best of our knowledge, our work presented in Chapter 4

is the first investigation on whether and how AQM can maintain a stable

backlog under wireless losses.

113

Second, in wireless ad-hoc networks, transmission at adjacent nodes may

interfere with each other. The implication on congestion control is two-

fold. On the one hand, TCP flows compete for “space”, namely the chance

for transmission within an interfering range, in addition to competing for

wireless bandwidth. Thus, backlogs at interfering nodes need to be aggre-

gately managed by a queue management algorithm to facilitate fair allocation

among competing flows [76]. On the other hand, packet losses occur due to

channel contention, in addition to router buffer overflow, and both should be

interpreted as signals of network congestion. However, unlike losses due to

buffer overflow, contention losses may saturate when the offered network load

exceeds a threshold value [25]. Consequently, the TCP flows tend to push the

network beyond the optimal loaded point, leading to degraded throughput.

AQM should thus be in place to deliberately tune up the packet dropping

rate to force earlier backoff of the TCP flows.

A unified AQM solution for resolving issues induced by wireless losses,

channel contention and other special characteristics of wireless networks is

much more than a trivial combination of the solutions proposed for each issue.

In particular, to stabilize the backlog under wireless losses we need to tune

down packet dropping rate to various levels to compensate for the variation

in wireless packet error rate, whereas to force TCP flows to operate at the

optimal loading point in ad-hoc networks we need to tune up packet dropping

rate to force earlier backoffs in TCP. A possible approach to attain both

objectives is to make use of some occurrences of wireless losses to deliberately

induce backoffs in TCP to restore the optimal loading point, and at the same

time to adjust the packet dropping rate of AQM to compensate for variations

114

of wireless losses. Again, the exact knowledge of wireless packet error rate

should not be assumed, or the adaptiveness to the ever-changing wireless

environment would be sacrificed. The approach can thus be best fulfilled on

the basis of a valid model of TCP/AQM under wireless losses and channel

contentions, which itself is a challenging task to be accomplished.

5.4 Concluding Remarks

In this thesis, we aim to systematically adapt the present Internet conges-

tion control to the wireless environment at a minimal deployment cost and

disruption. Our adaptations include TCP and AQM enhancements, which

jointly help the heterogeneous wired/wireless Internet to operate in the op-

timal region of low delay and high throughput.

On the TCP side, we have developed the STS model that uses a serialized-

timer structure to offer more reliable signals of packet loss and network con-

gestion over general, error-prone channels. We have devised TCP-NCL as a

practical approximation of STS, which can serve as a unified solution for ef-

fective congestion control, sequencing control, and loss recovery over wireless

networks. The deployment of TCP-NCL requires sender-side modification

only.

On the AQM side, we have investigated the performance of AQM under

wireless losses using simulation experiments. Our results report that RED fail

to maintain a stable backlog against time-varying WPER. We have applied

the internal model principle to stabilize the backlog. We further devise IC

as an embodiment of the principle, which performs robustly against time-

115

varying WPER in various simulated challenging scenarios.

There are several research directions in wireless Internet congestion con-

trol worth future thorough investigation, for which the work completed in

this thesis provide a solid basis. These include a theoretical framework for

the design and analysis of wireless TCP enhancements, and a unified AQM

solution for wireless networks.

116

Bibliography

[1] I.F. Akyidiz, C. Morabito, and S. Palazzo. TCP-Peach: A New Conges-tion Control Scheme for Satellite IP Networks. IEEE/ACM Transactionson Networking, Vol. 9, No. 3, Jun. 2001.

[2] M. Allman, H. Balakrishnan, and S. Floyd. Enhancing TCP’s Loss Re-covery using Limited Transmit. IETF RFC 3042, Jan. 2001.

[3] M. Allman, D. Glover, and L. Sanchez. Enhancing TCP Over SatelliteChannels using Standard Mechanisms IETF RFC 2488, Jan. 1999.

[4] M. Allman, V. Paxson, and E. Blanton. TCP Congestion Control. IETFRFC 5681, Sep. 2009.

[5] M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. IETFRFC 2581, Apr. 1999.

[6] S. Athuraliya, S.H. Low, V.H. Li, and Q. Yin. REM: Active QueueManagement. IEEE Network, Vol. 15, No. 3, pp. 48-53, May 2001.

[7] H. Balakrishnan, V.N. Padmanabhan, S. Seshan, and R.H. Katz. AComparison of Mechanisms for Improving TCP Performance over Wire-less Links. IEEE/ACM Transactions on Networking, Vol. 5, No. 6,pp. 756-769, Dec. 1997.

[8] S. Bhandarkar and A.L.N. Reddy. TCP-DCR: Making TCP Robust toNon-Congestion Events. Lecture Notes in Computer Science, Vol. 3042,pp. 712-724, May 2004.

[9] E. Blanton and M. Allman. On Making TCP More Robust to Packet Re-ordering. ACM SIGCOMM Computer Communication Review, Vol. 32,Issue 1, pp. 20-30, Jan. 2002.

[10] S. Bohacek, J.P. Hespanha, J. Lee, C. Lim, and K. Obraczka. A NewTCP for Persistent Packet Reordering. IEEE/ACM Transactions onNetworking, Vol. 14, No. 2, pp. 369-382, Apr. 2006.

117

[11] B. Braden, D. Clark, J. Crowsoft, B. Davie, S. Deering, D. Estrin,S. Floyd, V. Jacobson, G. Minshall, C. Patridge, L. Peterson, K. Ra-makrishnan, S. Shenker, J. Wroclawski, and L. Zhang. Recommenda-tions on Queue Management and Congestion Avoidance in the Internet.IETF RFC 2309, Apr. 1998.

[12] L. Brakmo and L. Peterson. TCP Vegas: End to End Congestion Avoid-ance on a Global Internet. IEEE Journal on Selected Areas in Commu-nication, Vol 13, No. 8, pp. 1465-1480, Oct. 1995.

[13] C. Casetti, M. Gerla, S. Mascolo, M.Y. Sanadidi, and R. Wang. TCPWestwood: End-to-End Congestion Control for Wired/Wireless Net-works. Wireless Networks, Vol. 8, No. 5, pp. 467-479, 2002.

[14] C.-T. Chen. Linear System Theory and Design. New York: Oxford Uni-versity Press, 1999.

[15] M. Chen and A. Zakhor. Flow Control over Wireless Network and Appli-cation Layer Implementation. Proc. of IEEE INFOCOM, pp. 103-113,Mar. 2006.

[16] D.M. Chiu and R. Jain. Analysis of the increase and decrease algorithmsfor congestion avoidance in computer networks. Computer Networks andISDN Systems, Vol. 17, No. 1, pp. 1-14, 1989.

[17] J. Davis. New Networking Features in Windows Server2008 and Windows Vista. http://technet.microsoft.com/en-us/library/bb726965.aspx. Feb. 2008.

[18] C.A. Desoer and Y.T. Wang. On the Generalized Nyquist Stability Cri-terion. IEEE Transactions on Automatic Control, Vol. 25, No. 2, pp. 187-196, Apr. 1980.

[19] K. Fall and K. Varadhan. The ns Manual (formerly ns Notes and Doc-umentation). The VINT Project, 6 January 2009.

[20] X. Fan, M. Arcak, and J.T. Wen. Robustness of Network Flow Controlagainst Disturbances and Time-Delay. System & Control Letters, Vol.53, No. 11, pp. 1329, Sep. 2004.

[21] S. Floyd, T. Henderson, and A. Gurtov. The NewReno Modification toTCP’s Fast Recovery Algorithm. IETF RFC 3782, Apr. 2004.

118

[22] S. Floyd and V. Jacobson. Random Early Detection Gateways for Con-gestion Avoidance. IEEE Transactions on Networking, Vol. 1, No. 4,Aug. 1993.

[23] S. Floyd, J. Mahdavi, M. Mathis, and M. Podolsky. An Extension to theSelective Acknowledgment (SACK) Option for TCP IETF RFC 2883,July 2000.

[24] C.P. Fu and S.C. Liew. TCP Veno: TCP Enhancement for Transmis-sion over Wireless Access Networks. IEEE Journal on Selected Areas inCommunication, Vol. 21, No. 2, pp. 216-228, Feb. 2003.

[25] Z. Fu, P. Zerfos, H. Luo, S. Lu, L. Zhang, and M. Gerla. The Impactof Multihop Wireless Channel on TCP Throughput and Loss. Proc. ofIEEE INFOCOM, Mar. 2003.

[26] H. Han, C.V. Hollot, Y. Chait, and V. Misra. TCP Networks Stabilizedby Buffer-Based AQMs. Proc. of IEEE INOFOCOM, Mar. 2004.

[27] C.V. Hollot, V. Misra, D. Towsley, and W.-B. Gong. Analysis and Designof Controllers for AQM Routers Supporting TCP Flows. IEEE Trans-actions on Automatic Control, Vol. 47, No. 6, June 2002.

[28] F. Hu and N. Sharma. Enhancing Wireless Internet Performance.IEEE Communication Surveys and Tutorials, Vol. 4, No. 1, pp. 2-15,Dec. 2002.

[29] Institute of Electrical and Electronics Engineers. IEEE Standard forLocal and Metropolitan Area Networks Part 16: Air Interface for FixedBroadband Wireless Access Systems. New York: Institute of Electricaland Electronics Engineers, 2009.

[30] V. Jacobson Congestion Avoidance and Control. ACM SIGCOMM Com-pututer Communication Review, Vol. 25, No. 1, pp. 157-187, Jan. 1995.

[31] R. Jain, D.M. Chiu, and W. Hawe. A Quantitative Measurement ofFairness and Discrimination for Resource Allocation in Shared Systems.Technical Report DEC-TR-301, Degital Equipment Corporation, 1984.

[32] D. Katabi, M. Handley, and C. Rohrs. Congestion Control for HighBandwidth-delay Product Networks. ACM SIGCOMM Computer Com-munication Review, Vol. 32, No. 4, Oct. 2002.

119

[33] F. Kelly, A. Maulloo, and D. Tan. Rate Control in Communication Net-works: Shadow Prices, Proportional Fairness and Stability. Journal ofthe Operational Research Society, Vol. 49, No. 3, pp. 237-252, Mar. 1998.

[34] H.K. Khalil. Nonlinear Systems. Upper Saddle River, New Jersey:Prentice-Hall, 2002.

[35] A. Lahanas and V. Tsaoussidis. Improving TCP Performance overNetworks with Wireless Components using ’Probing Devices’. Interna-tional Journal of Communication Systems, Vol. 15, No. 6, pp. 495-511,July/Aug. 2002.

[36] C. Lai, K.-C. Leung, and V.O.K. Li. Demo Abstract: A Testbed for TCPin Heterogeneous Wired/Wireless Networks. Proc. of IEEE INFOCOM2011, Apr. 2011.

[37] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing AQM to Combat Wire-less Losses. Submitted.

[38] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing Wireless TCP: ASerialized-Timer Approach. Proc. of IEEE INFOCOM 2010, Mar. 2010.

[39] C. Lai, K.-C. Leung, and V.O.K. Li. TCP-NCL: A Unified Solution forTCP Packet Reordering and Random Loss. Proc. of IEEE PIMRC 2009,Sep. 2009.

[40] T.V. Lakshman and U. Madhow. The The Performance of TCP/IPfor Networks with High Bandwidth-Delay Products and Random Loss.IEEE/ACM Transactions on on Networking, Vol. 5, No. 3, June 1997.

[41] D. Lapsley and S.H. Low. Random Early Marking for Internet Conges-tion Control. Proc. of IEEE Globecom, pp. 1747-1752, Dec. 1999.

[42] G.S. Lee, L.L.H. Andrew, A. Tang, and S.H. Low. WAN-in-Lab: Mo-tivation, Deployment and Experiments. Proc. of PFLDnet, pp. 85-90,Feb. 2007.

[43] K.-C. Leung, C. Lai, V.O.K. Li, and D. Yang. A Packet ReorderingSolution to Wireless Losses in Transmission Control Protocol (TCP).Submitted.

[44] K.-C. Leung and V.O.K. Li. Transmission Control Protocol (TCP) inWireless Networks: Issues, Approaches, and Challenges. IEEE Commu-nication Surveys and Tutorials, Vol. 8, No. 4, pp. 64-79, Fourth Quar-ter 2006.

120

[45] K.-C. Leung, V.O.K. Li, and D. Yang. An Overview of Packet Reorder-ing in Transmission Control Protocol (TCP): Problems, Solutions, andChallenges. IEEE Transactions on on Parallel and Distributed Systems,Vol. 18, No. 4, pp. 522-535, Apr. 2007

[46] V.O.K. Li, L. Cui, Q. Liu, G.-H. Yang, Z. Zhao, and K.-C. Leung. AHeterogeneous Peer-to-Peer Network Testbed. Proc. of ICUFN, pp. 46-51, Jun. 2009.

[47] V.O.K. Li, C. Lai, and K.-C. Leung. Differentiation among Occurrencesof Packet Reordering, Congestive Packet Loss, and Random PacketLoss in Communication Networks. United States Regular ApplicationNo. 12/879,880, Sep. 2010.

[48] J. Liu and S. Singh. ATCP: TCP for Mobile Ad Hoc Networks. IEEEJournal on Selected Areas in Communication, Vol. 19, No. 7, pp. 1300-1315, July 2001.

[49] S.H. Low. A Duality Model of TCP and Queue Management Algorithms.IEEE/ACM Transactions on Networking, Vol. 11, No. 4, pp. 525-536,Aug. 2003.

[50] S.H. Low, F. Paganini, and J.C. Doyle. Internet Congestion Control.IEEE Control Systems Magazine, Vol. 22, No. 1, pp. 28-43, Feb. 2002.

[51] S.H. Low, F. Paganini, J. Wang, and J.C. Doyle. Linear Stability ofTCP/RED and a Scalable Control. Computer Networks Journal, Vol. 43,No. 5, pp. 633-647, Dec. 2003.

[52] R. Ludwig and R.H. Katz. The Eifel Algorithm: Making TCP RobustAgainst Spurious Retransmissions. ACM SIGCOMM Computer Com-munication Review, Vol. 30, Issue 1, pp. 30-36, Jan. 2000.

[53] V. Misra, W.-B. Gong, and D. Towsley. Fluid-based Analysis of a Net-work of AQM Routers Supporting TCP Flows with an Applicationto RED. ACM SIGCOMM Computer Communication Review, Vol. 30,No. 4, pp. 151-160, Oct. 2000.

[54] D. Mitzel. Overview of 2000 IAB Wireless Internetworking Workshop.RFC 3002, Network Working Group, Dec. 2000.

[55] J. Mo, R.J. La, V. Anantharam, and J. Walrand. Analysis and Com-parison of TCP Reno and Vegas. Proc. of IEEE INFOCOM, Mar. 1999.

121

[56] J. Nagle. Congestion Control in IP/TCP Internetworks. RFC 896, Net-work Working Group, Jan. 1984.

[57] R. Pan, B. Prabhakar, and K. Psounis. CHOKe: A Stateless AQMScheme for Approximating Fair Bandwidth Allocation. Proc. of IEEEINFOCOM, Mar. 2000.

[58] C. Parsa and J. Garcia-Luna-Aceves. Improving TCP Congestion Con-trol over Internets with Heterogeneous Transmission Media. Proc. ofIEEE ICNP 1999, pp. 213-221, Oct./Nov. 1999.

[59] L. Peterson, T. Anderson, D. Culler, and T. Roscoe. A blueprint forintroducing disruptive technology into the Internet. ACM SIGCOMMComputer Communication Review, Vol. 33, No. 1, pp. 59-64, Jan. 2003.

[60] J. Postel. Transmission Control Protocol. RFC 793, Protocol Specifica-tion, DARPA Internet Program, Sep. 1981.

[61] K.K. Ramakrishnan, S. Floyd, and D. Black. The Addition of ExplicitCongestion. Notification (ECN) to IP. IETF RFC 3168, Sep. 2001.

[62] P. Sarolahti and M. Kojo. Forward RTO-Recovery (F-RTO): An Algo-rithm for Detecting Spurious Retransmission Timeouts with TCP andthe Stream Control Transmission Protocol (SCTP). IETF RFC 4138,Aug. 2005.

[63] S. Seth and M.A. Venkatesulu. TCP/IP Architecture, Design and Im-plementation in Linux. Hoboken, New Jersey: Wiley, 2008.

[64] S. Shenker. Fundamental Design Issues for the Future Internet. IEEEJournal on Selected Areas in Communications, Vol. 13, No. 7, pp. 1176-1188, Sep. 1995.

[65] R. Srikant. The Mathematics of Internet Congestion Control. Cam-bridge, MA:Birkhauser, 2004.

[66] W. Stevens. TCP Slow Start, Congestion Avoidance, Fast Retransmit,and Fast Recovery Algorithms. IETF RFC 2001, Jan. 1997.

[67] A. Tang, J. Wang, S.H. Low, and M. Chiang. Equilibrium of Hetero-geneous Congestion Control: Existence and Uniqueness. IEEE/ACMTransactions on Networking, Vol. 15, No. 4, pp. 1422-1435, Aug. 2007.

122

[68] A. Tang, D. Wei, S.H. Low and M. Chiang. Equilibrium of Heteroge-neous Congestion Control: Optimality and Stability. IEEE/ACM Tran-sanctions on Networking, Vol. 18, No. 3, pp. 844-857, Jun. 2010.

[69] F. Wang and Y. Zhang. Improving TCP Performance over Mobile Ad-Hoc Networks with Out-Of-Order Detection and Response. Proc. ofACM MOBIHOC 2002, pp. 217-225, June 2002.

[70] J. Wang, J. Wen, J. Zhang, and Y. Han. TCP-FIT: An Improved TCPCongestion Control Algorithm and its Performance. Proc. of IEEE IN-FOCOM, Apr. 2011.

[71] D. Wei, C. Jin, S. Low, and S. Hegde. FAST TCP: Motivation, Architec-ture, Algorithms, Performance. IEEE/ACM Transactions on Network-ing, Vol. 14, No. 6, pp.1246-1259, Dec. 2006.

[72] J.T. Wen and M. Arcak. A Unifying Passivity Framework for NetworkFlow Control. IEEE Transactions on Automatic Control, Vol. 49, No. 2,pp. 162-174, Feb. 2004.

[73] G. Wright and W. Stevens. TCP/IP Illustrated, Volume 2: The Imple-mentation. Addison-Weasley, 1995.

[74] E. H.-K. Wu and M.-Z. Chen. JTCP: Jitter-Based TCP for Heteroge-neous Wireless Network. IEEE Journal on Selected Areas in Communi-cation, Vol. 22, No. 4, pp. 757-766, May 2004.

[75] Y. Xia, L. Subramanian, I. Stoica, and S. Kalyannaramana. One MoreBit is Enough. IEEE/ACM Transactions on Networking, Vol. 16, No. 6,Mar. 2008.

[76] K. Xu, M. Gerla, L. Qi, and Y. Shu. Enhancing TCP Fairness in AdHoc Wireless Networks Using Neighborhood RED. Proc. of ACM MO-BICOM, Sep. 2003.

[77] D. Yang, K.-C. Leung, and V.O.K. Li. Simulation-based Comparisonsof Solutions for TCP Packet Reordering in Wireless Networks. Proc. ofIEEE WCNC, pp. 3240-3245, March 2007.

[78] L. Ying, G.E. Dullerud, and R. Srikant. Global Stability of InternetCongestion Controllers with Heterogeneous Delays. IEEE/ACM Trans-actions on Networking, Vol. 14, No. 3, pp. 579-591, Jun. 2006.

123

[79] H. Zhang, B. Liu, and W. Dou. Design of a Robust Active QueueManagement Algorithm Based on Feedback Compensation. ACM SIG-COMM Computer Communication Review, Vol. 33, No. 4, pp. 277-285,Oct. 2003.

[80] L. Zhang and D. Clark. Oscillating Behavior of Network Traffic: A CaseStudy Simulation. Internetworking: Research and Experience, Vol. 1,No. 2, pp. 101-112, 1990.

[81] M. Zhang, B. Karp, S. Floyd, and L. Peterson. RR-TCP: A Reordering-Robust TCP with DSACK. Proc. of IEEE ICNP 2003, pp. 95-106,Nov. 2003.

[82] Z. Zhao, Q. Liu, G.-H. Yang, V.O.K. Li, and L. Cui. A Multi-radioWireless P2P Network Testbed. Proc. of ACM WiNTECH, pp. 93-94,Sep. 2009.

124

Acronyms

ACK acknowledgment. 4, 17, 18, 25, 27, 28, 35, 43, 44

AQM active queue management. 1, 3, 9, 12–16, 19–22, 69–73, 77–79, 81–84,86, 90, 97, 100, 104–110, 112–115

awnd advertised window. 6

CBR constant bit rate. 71, 93, 102

CD congestion response decision. 28, 43, 49, 51, 54, 57, 59, 63, 66

CRHP closed right half plane. 85

cwnd congestion window. 6–8, 12, 17, 18, 29, 31, 34–36, 59, 60

ECN explicit congestion notification. 70

FTP File Transfer Protocol. 3

GBR goodput to bandwidth ratio. 65, 66

HTTP Hypertext Transfer Protocol. 3, 93, 102, 104, 108

IC integral controller. 13, 14, 20, 71, 72, 87, 90, 92–94, 96, 97, 100, 102,104, 108, 114

IP Internet Protocol. 3, 19

LLRTX link-layer retransmission. 11, 17

MANET mobile ad-hoc networks. 11

MRRL maximum RTT record length. 44–46

125

NRU NCL-RTT-Update. 43

OLHP open left half plane. 82

PI proportional-integral. 20, 21, 90

RD retransmission decision. 28, 43, 49, 51, 54

RED random early detection. 9, 13, 20, 66, 67, 70, 72, 73, 76, 77, 92–94,97, 100, 102, 104, 108, 109, 114

RTO retransmission timeout. 7, 17, 31, 43, 49–52, 111

RTT round-trip time. 4, 11, 17, 19, 27–29, 35, 37, 43–46, 50, 68, 80, 88, 91,110, 111

ssthresh slow start threshold value. 6, 7, 29, 35

STS smart TCP sender. 12, 14, 23–25, 28, 43, 49, 68, 107, 109, 114

TCP Transmission Control Protocol. 1, 3, 8–21, 23–25, 27–30, 33–36, 43,44, 46, 50–52, 54–56, 58, 59, 61–63, 65–67, 69, 70, 72, 73, 76–78, 81,83, 91, 93, 100, 104, 106–115

TCP-NCL TCP for non-congestive loss. 13, 14, 24, 25, 43, 46, 49, 51–56,59, 60, 62–64, 66–68, 70, 76, 107–109, 112, 114

TCP/IP Transmission Control Protocol/Internet Protocol. 2, 53

WMN wireless mesh network. 11

WPER wireless packet error rate. 13, 71, 73, 77, 86, 87, 93, 96, 97, 102,108, 110, 112, 114, 115

126