stochastic network optimization and the theory of network throughput, energy, and delay michael j....

Stochastic Network Optimization and the Theory of Network Throughput, Energy, and Delay

Michael J. NeelyUniversity of Southern Californiahttp://www-rcf.usc.edu/~mjneely/

*Sponsored in part by the DARPA IT-MANET Program, NSF OCE-0520324, NSF Career CCF-0747525

General mobile network

http://www-rcf.usc.edu/~mjneely/

Outline: 1.Analogy between Info Theory and Network Theory

• “Capacity” Definitions• Canonical Models

2.*Overview talk on Stochastic Network Optimization: • History• Landmark Results• Application to general multi-hop stochastic networks

3.*Focus talk on 1-hop multi-user wireless downlink• Fundamental energy-delay tradeoff• Low complexity achievability

*Details can be found in the following (available on my webpage):•L. Georgiadis, M. J. Neely, L. Tassiulas, “Resource Allocation and Cross-Layer Control in Wireless Networks,” Foundations & Trends in Networking, vol. 1, no. 1, pp. 1-144, 2006.

•M. J. Neely, “Optimal Energy and Delay Tradeoffs for Multi-User Wireless Downlinks,” IEEE Transactions on Information Theory, vol. 53, no. 9, pp. 3095-3113, Sept. 2007.

Part 1: Analogy between info theory and network theory

Capacity Region Λ = Set of all end-to-end rate vectors (or matrices) achievable over a network.

Information Theory View of Capacity• Optimizes over all maps of symbols into codewords• Results known for point-to-point links• Results known for small 1-hop systems (broadcast/MAC)

Network/Queueing Theory View of Capacity• Sometimes called “transport capacity” [Gupta/Kumar]• Optimizes over all routing/scheduling/resource allocation• Typically “link based” (with some extensions…)• Simplified PHY layer (SINR, Interference Sets, etc.)• Results hold for arbitrarily large networks, with mobility

1 Wireless Link = AWGN Channel(symbol-by-symbol transmission)

1 Wireless Link = ON/OFF Channel(slot-by-slot packet transmission)

“info theory” “queueing theory”

+Symbols

Noise

C = log(1 + SNR)

Capacity maximizes time avg. bit rate. [Optimizes over all coding strategies.]

[very deep math for 1 link]

Packet Arrivals Pr[ON]=p

C = p packets/slot

Capacity = time avg packettransmission rate. [nothing tooptimization here]

[Basic Queue Stability theory]

Capacity: Capacity:

Mathematical Models for a Wireless System (two meaningful perspectives)


1 Wireless Link = AWGN Channel(symbol-by-symbol transmission)

1 Wireless Link = ON/OFF Channel(slot-by-slot packet transmission)


+Symbols

Noise

C = log(1 + SNR)

Achievability: Random CodingConverse: Sphere-Packet, Fano

[very deep math for 1 link]

Packet Arrivals Pr[ON]=p

C = p packets/slot

Achievability: ObviousConverse: Obvious

[Basic Queue Stability theory]

Capacity: Capacity:



N-User Gauss. Broadcast Downlink(symbol-by-symbol transmission)

N-User Downlink (Fading Channels)(opportunistic packet transmission)


bitsbitsbits

ON/OFFON/OFF

ON/OFF

•Capacity REGION is set of all supportable long term bit rate vectors •Optimizes over all Coding Strategies

•Capacity REGION is set of all supportable packet arrival rate vectors•Optimizes over all scheduling strategies•Example: Observe Channel states, then decide which queue to serve



N-User Gauss. Broadcast Downlink(symbol-by-symbol transmission)

N-User Downlink (Fading Channels)(opportunistic packet transmission)


bitsbitsbits

ON/OFFON/OFF

ON/OFF



Capacity Region: all (1,…, N) s.t.

for all subsets K of users.

[Tassiulas & Ephremides ‘93]

Capacity Region: all (1,…, N) s.t.

(degraded Gauss. BC)

N-Node Static Multi-Hop Network(multiple sources and destinations)



•Infinite Traffic•Symbol-by-Symbol Transmissions•Interference Channels•Optimize the coding

Capacity = ???

• Pr[Channel k = ON] = pk

• Random Packet Arrivals• Optimize Scheduling/Routing

p1

p2

p3

p4

p5

p6

Capacity = Known Exactly (Multi-Commodity Flow Subject to “Graph Family” Link Constraints)






•Coding Tools (inner bounds): NetCoding, Cooperative Trans., etc. •Converse Tools (outer bounds):cut-sets, multi-terminal info theory

Capacity = ???

• Scheduling Tools: Max Weight Matching (MWM), Backpressure• Converse Tools: Queue Stability,Flow Conservation, Optimization

p1

p2

p3

p4

p5

p6

Capacity = Known Exactly



N-Node MANET

Scheduling Tools: Max Weight Matching (MWM), BackpressureRoutingConverse Tools: Queue Stability, Flow Conservation, Optimization

Capacity = ???

N-Node MANET

Capacity = Known Exactly




T/RT/R

T/RT/R

T/R

T/R

T/R

T/R

T/RT/R

T/R

T/RT/R

T/R

T/R

T/R

N-Node MANET

Capacity = ???

N-Node MANET




T/RT/R

T/RT/R

T/R

T/R

T/R

T/R

T/RT/R

T/R

T/RT/R

T/R

T/R

T/R

Stochastic Network Theory extends to:•Bursty Traffic•Arbitrary Mobility•Performance Optimization [Neely thesis 2003, Infocom 2005][Neely, Modiano, Rohrs JSAC 2005][Georgiadis, Neely, Tassiulas F&T 2006]

Part 2: Overview of Stochastic Network OptimizationWhat Problems Can be Solved Today? •Slotted time system t = {0, 1, 2, …}•Q(t) = Queue State Vector (possibly multi-hop)•S(t) = “Topology State” (random, chosen by environment)•I(t) = “Control Action” or “Tranmission mode” , I(t) in I•General “Link Transmission Rate Vector Function: Rate vector(t) = C(I(t), S(t))•General “Penalty/Reward vector”: Penalty vector(t) = x(I(t), S(t))•Queue Evolution:

Q(t) Q(t+1)I(t), S(t)Q(t+1) = max[Q(t) – out(t), 0] + in(t)

f() convexhi(x) convex

Part 2: Overview of Stochastic Network Optimization

How is this solved? We have a general and extensive theory: •Lyapunov Drift and Stability for Networks [Tassiulas & Ephremides TAC 1992, IT 1993]•Drift for Joint Network Stability and Performance Optimization [Neely thesis 2003, Infocom 2005], [Georgiadis, Neely, Tassiulas F&T 2006]•Virtual Queues [Neely Infocom 2005, IT 2006], [Georgiadis, Neely, Tassiulas F&T 2006]•Auxiliary Variables and “Flow State” Queues [Neely, Modiano, Li Infocom 2005], [Georgiadis, Neely, Tassiulas F&T 06]

Alternative Approaches: •Downlink, Linear Utilities [Tsibonis, Georgiadis, Tassiulas Infocom 03]•Flow Based, Static Channels [Cruz & Santhanam, Infocom 03], [Lin, Shroff CDC 04, Infocom 05]•Fluid Model Analysis for Multi-Hop and General Utilities [Stolyar, Queueing Systems 05] --- “Primal-Dual” Alg.•Infinite Backlog Assumption, 1-hop downlink [“Prop. Fair” Alg] [Agrawal, Subramanian, Allerton 02], [Kushner, Whiting Allert. 02] [Eryilmaz, Srikant Infocom 2005] , [Liu, Chong, Shroff 03]

Part 2: Overview of Stochastic Network Optimization

How is this solved? We have a general and extensive theory: •Lyapunov Drift and Stability for Networks [Tassiulas & Ephremides TAC 1992, IT 1993]•Drift for Joint Network Stability and Performance Optimization [Neely thesis 2003, Infocom 2005], [Georgiadis, Neely, Tassiulas F&T 2006]•Virtual Queues [Neely Infocom 2005, IT 2006], [Georgiadis, Neely, Tassiulas F&T 2006]•Auxiliary Variables and “Flow State” Queues [Neely, Modiano, Li Infocom 2005], [Georgiadis, Neely, Tassiulas F&T 06]

Note: Our work is unique in that it: -Solves full problem on the actual queueing network of interest-Links very nicely to the previous Tassiulas drift framework-Gets Strongest Results, and Explicit performance-delay Tradeoffs

[O(1/V) ; O(V)] peformance-delay for any network, any utility

Question: Is this the optimal tradeoff?

The Basic Stability Theory for Networks:

Tassiulas & Ephremides [Trans. Autom. Control 1992]•Multi-hop network with Random Packet Arrival Processes•Link Scheduling according to “Feasible Activation Sets”•Lyapunov drift for stability•“Backpressure” Routing and Max-Weight Scheduling•Gives Stability for any rate vector inside capacity region•Does not require knowledge of traffic rates

Tassiulas & Ephremides [Trans. Inform. Theory 1993]•Single-Hop Dynamic Channels•“Opportunistic” scheduling (ie, “channel-aware”)•Lyapunov drift for stability•Max-Weight Algorithm does not require channel statistics or traffic arrival rates, gets stability whenever possible

Quick Description of “Backpressure Routing” and “Max-Weight Scheduling”:

n n

[Red data is destined for red node, Yellow data is destined for yellow node]

• No pre-defined routes! • Optimal Link Activation Set determined by a max-weight rule•“Which packet to send over a link” is determined by a differential backlog index (backpressure)• The max-weight link activation can be NP hard for networks with interference, but is trivial (and distributed) for orthogonal links

An Abbreviated History of Lyapunov Drift for Network Stability: (for various computer networks and switching systems)•Tassiulas & Ephremides (Backpressure, Max-Weight) [1992, 1993]•Kumar & Meyn [1995]•McKeown, Anantharam, Walrand [1996, 1999]•Kahale & Wright [1997]•Andrews, Kumaran, Ramanan, Stolyar, Whiting [2001]•Leonardi, Mellia, Neri, Marsan [2001]•Neely, Modiano, Rohrs [2003, 2005] Extends to MANETs

Performance Optimization (time varying channels) but withno queueing or stability constraints (infinite backlog assumption):•R. Agrawal, V. Subramanian [2002] (“Proportionally Fair Alg”)•Kushner, Whiting [2002] (“Proportionally Fair Alg”)•Liu, Chong, Shroff [2003]

Corresponding Results for Static Networks (non-stochastic):These use static convex optimization theory to maximizenetwork utility. Lagrange multipliers are “shadow prices.” There is eitherno queueing analysis, or approximate queueing analysis.

Wireline Networks, Fixed Route Selection, Flow BasedKelly [1997]Kelly, Maullou, Tan [1998]Low, Lapsley [1999]

Wireless Networks, Fixed Route Selection, Flow Based•Xiao, Johansson, Boyd [2001]•Lee, Mazumdar, Shroff [2002]•Julian, Chiang, O’Neill Boyd [2002]•Chiang [2004, 2005]

Scheduling for Utility Optimization (static networks)•Cruz & Santhanam [2003] (scheduling decisions are chosen over time)•Lin & Shroff [2004, 2005] (scheduling decisions are chosen over time)

Work on Utility Optimization for Stochastic Networks:

Wireless Downlink, Time Varying Channels, Infinite Data, No Queueing•R. Agrawal, V. Subramanian [2002] (“Proportionally Fair Alg”)•Kushner, Whiting [2002] (“Proportionally Fair Alg”)•Liu, Chong, Shroff [2003]

Joint Stability and Performance Optimization (Time Varying Channels): •Tsibonis, Georgiadis, Tassiulas [2003]

Solves downlink with special structure, and with linear utilities•Neely [2003, 2005] “Dual Method”

Solves the general problem! (multi-hop, stochastic, concave utilities)Obtains explicit [O(1/V), O(V)] utility-delay tradeoff!

•Stolyar [2005] “Primal-Dual Method”A different approach to the general problem, solves on a fluid model

•Eryilmaz & Srikant [2005] “Dual Method” Downlink, infinite backlog, fluid model analysis

•Lee, Mazumdar, Shroff [2006] Stochastic gradients, flow based, infinite backlog

λ1

λ2

λ1

λ2

Input Rate Output Rate

1

2

S1(t) {ON, OFF}

S2(t) {ON, OFF}

“Proportionally Fair” algorithm (designed for infinite backlog)

[Example from Neely, Modiano, Li Infocom 2005, TON 2008]

λ1

λ2

λ1

λ2


1

2

S1(t) {ON, OFF}

S2(t) {ON, OFF}

“Max-Weight” algorithm (designed only to achieve stability, Tass. & Eph. 93)


λ1

λ2

λ1

λ2


1

2

S1(t) {ON, OFF}

S2(t) {ON, OFF}

“An Ideal” algorithm [Neely 2003, 2005]


Work on Utility Optimization for Stochastic Networks:

Wireless Downlink, Time Varying Channels, Infinite Data, No Queueing•R. Agrawal, V. Subramanian [2002] (“Proportionally Fair Alg”)•Kushner, Whiting [2002] (“Proportionally Fair Alg”)•Liu, Chong, Shroff [2003]

Joint Stability and Performance Optimization (Time Varying Channels): •Tsibonis, Georgiadis, Tassiulas [2003]

Solves downlink with special structure, and with linear utilities•Neely [2003, 2005] “Dual Method”

Solves the general problem! (multi-hop, stochastic, concave utilities)Obtains explicit [O(1/V), O(V)] utility-delay tradeoff!

•Stolyar [2005] “Primal-Dual Method”A different approach to the general problem, solves on a fluid model

•Eryilmaz & Srikant [2005] “Dual Method” Downlink, infinite backlog, fluid model analysis

•Lee, Mazumdar, Shroff [2006] Stochastic gradients, flow based, infinite backlog

General Theory in F&T book: (Georgiadis, Neely, Tassiulas F&T 2006)

Unifies Lyapunov drift and Performance Optimization with a very simple modification: Every slot, observe queues and channels, and take a greedy action to minimize “Drift + Penalty”:

Δ(Q(t)) + VE{Φ(t)|Q(t)}


Drift

Penalty

General Theory in F&T book: (Georgiadis, Neely, Tassiulas F&T 2006)

Unifies Lyapunov drift and Performance Optimization with a very simple modification: Every slot, observe queues and channels, and take a greedy action to minimize “Drift + Penalty”:

Δ(Q(t)) + VE{Φ(t)|Q(t)}


Theorem (Neely 2003, 2005): |E{Φ} - E{Φ*}| < O(1/V) E{Queue Size} < O(V)

Can Solve Problems of the Type [F&T 2006]:

where x is a vector of general “network penalties/rewards” (throughput, energy, reliability, etc., and penalties are arbitrary functions of control decisions) and f() and h() are convexfunctions.

•Joint Lyap. Drift and Performance Opt.•Virtual Queues•Auxiliary Variables

Important and New Techniques of Stochastic Network Optimization [F&T 2006]:

Flow control reservoir

raw data

-flow state queues-aux. vars.

Structure of the Algorithms:•Flow Control (separable at each source)•Modified Max-Weight/Backpressure Scheduling and Routing

Flow Control

Application to Mobile Networks with Unreliable Channels

= Stationary Node

= Locally Mobile Node

= Fully Mobile Node

= Sink

= Stationary Node

= Locally Mobile Node

= Fully Mobile Node

= Sink

Application to Mobile Networks with Unreliable Channels

Avg. Power Avg. Delay

Performance-Delay Tradeoff: [O(1/V), O(V)]

Average Power Versus Delay(Fix a set of transmission rates for each node)[Neely, Urgaonkar 2006, 2009]

Delay Theory For Networks:

1. Network Calculus for Deterministic Networks [Rene Cruz, Trans. Inf. Theory 1991]

(no performance-delay tradeoff for deterministic networks)

2. Achievable [O(1/V); O(V)] performance-delay tradeoff for stochastic networks (random traffic and/or channels) [Neely 2003, 2005][Georgiadis, Tassiulas, Neely F&T 2006]

3. Energy-Delay square root Tradeoff for a single queue [Berry-Gallager Trans. Information Theory 2002]

4. For general network, optimal tradeoff is either a square root law or a logarithm law! [Neely IEEE JSAC 2006, IEEE Transactions on Information Thoery 2007]

Optimal Energy and Delay Tradeoffs forMulti-User Wireless Downlinks

Michael J. NeelyUniversity of Southern California

http://www-rcf.usc.edu/~mjneely/Infocom 2006, Barcelona, Spain

*Sponsored by NSF OCE Grant 0520324

1 2 N

Avg. Delay

Avg

. Pow

er

http://www-rcf.usc.edu/~mjneely/

Assumptions: 1) Random Arrivals A(t) i.i.d. over slots. Rate vector (bits/slot)2) Random Channel states S(t) i.i.d. over slots.3) Transmission Rate Function P(t) --- Power allocation during slot t (P(t) ) S(t) --- Channel state during slot t

t0 1 2 3 …

Time slotted system (t {0, 1 , 2, …})

rate

i

power P

(P(t), S(t))

Good

Med

Bad

1 2 N

rate

i

power P

Good

Med

Bad

1 2 N

Control: Allocate Power (P(t) ) in Reaction to Current Channel State And Current Queue Backlogs.

Goal: Stabilize with Minimum Average Power while also Maintaining Low Average Delay.

rate

i

power P

Good

Med

Bad

1 2 N

Control: Allocate Power (P(t) ) in Reaction to Current Channel State And Current Queue Backlogs.

Goal: Stabilize with Minimum Average Power while also Maintaining Low Average Delay.

[ Avg. Power and Avg. Delay are Competing Objectives! ] What is the Fundamental Energy-Delay Tradeoff?

1 2 N

Our Previous Work on Minimum Power Scheduling with Delay Tradeoffs [Neely Infocom 2005]:

UN(t)U1(t) U2(t)

V

P

V

Av.Delay

O(1/V)

O(V)

P*

( P* = Min Av. Power for Stability )

1 2 N


UN(t)U1(t) U2(t)

( P* = Min Av. Power for Stability )

V

P

V

Av.Delay

O(1/V)

O(V)

P*

1 2 N


UN(t)U1(t) U2(t)

Analysis: Use theory of Performance Optimal Lyapunov Scheduling: -Neely, Modiano 2003, 2005-Georgiadis, Neely, Tassiulas [F&T 2006, NOW Publishers]

Achieves: [O(1/V), O(V)] energy-delay tradeoff

V

P

V

Av.Delay

O(1/V)

O(V)

P*

1 2 N


V

P

V

Av.Delay

O(1/V)

O(V)

UN(t)U1(t) U2(t)

Analysis: Use theory of Performance Optimal Lyapunov Scheduling: -Neely, Modiano 2003, 2005-Georgiadis, Neely, Tassiulas [F&T 2006, NOW Publishers]

Achieves: [O(1/V), O(V)] energy-delay tradeoff

P*

V

P

V

Av.Delay

O(1/V)

( V )

P*

The Fundamental Berry-Gallager Bound for Energy-DelayTradeoffs in a Single Wireless Downlink:

A(t)

(t) = (P(t), S(t))

Av. Delay >= ( V )

[Berry, Gallager 2002]

Approach Achievability via Technique of Buffer Partitioning.

Precedents for Energy and Delay Optimizationfor Single Wireless Links:

-Berry and Gallager [2002] ( Fundamental Square Root Law )

-Uysal-Biyikoglu, Prabhakar, El Gamal [2002] -Khojastepour and Sabharwal [2004]( “Lazy Scheduling” and Filter Theory for Static Links )

-Fu, Modiano, Tsitsiklis [2003]-Goyal, Kumar, Sharma [2003]-Zafer and Modiano [2005]( Dynamic Programming, Markov Decision Theory )

Challenging to extend optimal delay results for stochastic systems beyond a single queue because…

1) Parameter Explosion: (cannot practically measure) Number of channel state vectors S grows geometrically with number of links N. Markov Decision Theory and Dynamic Programming requires knowledge of:

S = Pr[ S(t) = S] (for each channel state S ).

2) State Space Explosion: (cannot practically implement) Number of Queueing State Vectors U grows geometrically.

Idea: Combine Techniques of Buffer Partitioning and Performance Optimal Lyapunov Scheduling.

1 2 N

V

P

V

Av.Delay

P*

Goals: 1) Establish the fundamental energy-delay curve for multi-user systems (extend Berry-Gallager to this case).

2) Design a dynamic algorithm to achieve optimal energy-delay tradeoffs. (Must overcome the complexity explosion problem).

Specifically: Define a general power cost metric h(P ):

1 2 N

Define average power cost:

Define: h* = Min. avg. power cost for network stability

(Push h arbitrarily close to h*, with optimal delay tradeoff…)

Theorem 1: (Characterize h*) Assume . The min average power cost h* is given as the solution to:

Define ( ) = min. avg. power cost h* above.

Corollary: For each , there is a stationary randomized alg. such that:

The Fundamental Energy-Delay Tradeoff:

) ))mild admissibility assumptions

Theorem 2 (Multi-User Berry-Gallager Bound): If

Then if avg. cost satisfies:

We necessarily have:

1 2 N

V

h

V

Av.Delay

h*( V )

O(1/V)

Achieving Optimal Tradeoffs via Buffer Partitioning…

Recall the Berry-Gallager threshold algorithm for single queues:

(t) = (P(t), S(t))

U(t)

max

QUQmax

drift

L R

[Requires full knowledge of channel probs S]



(t) = (P(t), S(t))

U(t)

max

QUQmax

drift

L R



(t) = (P(t), S(t))

U(t)max

QUQmax

drift

L R

Let’s Consider Multi-Dimensional Buffer Partitioning:

Q

U1

U2Case N=2

1 2 N

Q


Q

U1

U2Case N=2

Q


Q

U1

U2Case N=2

(not implementable)

Q

Analysis of the Threshold Algorithm:(exchanging sums over the 2N regions yields…)

iL(t) = Pr[Ui(t) <Q]

iR(t) = 1 - i

L(t)

An Online Algorithm for Optimal Energy-Delay Tradeoffs:

1 2 N

Define the bi-modal Lyapunov Function:

UiQ

Designing “gravity”into the system:

An Online Algorithm for Optimal Energy-Delay Tradeoffs:

1 2 N

Define the bi-modal Lyapunov Function:

UiQ

Designing “gravity”into the system:“Usually” creates proper drift direction…

1 2 N*Key inequality that holds with equality for the stationary threshold algorithm.

Need to strengthen the drift guarantees…Want to also ensure for all i {1, 2, …, N}


Use Virtual Queue Concept from [Neely Infocom 2005]:

Xi(t)Ai(t) + 1i

R(t) i(t) + 1iL(t)

indicator functions

Xi(t+1) = max[Xi(t) - i(t) - 1iL(t), 0] + Ai(t) + 1i

R(t)


Use Virtual Queue Concept from [Neely Infocom 2005]:

Xi(t)Ai(t) + 1i

R(t) i(t) + 1iL(t)

indicator functions

Xi(t) Stable i + 1iL > i + 1i

R

To Stabilize Virtual Queues Xi(t) and Actual Queues Ui(t):

The Tradeoff Optimal Control Algorithm (TOCA):

1) Every slot t, observe channel state S(t) and queue backlogs U(t), X(t). Allocate power P(t) = P, where P solves:

2) Transmit with rate i(t) = i(P(t), S(t)). 3) Update the Virtual Queues Xi(t):

Xi(t+1) = max[Xi(t) - i(t) - 1iL(t), 0] + Ai(t) + 1i

R(t)

Theorem 3 (TOCA Performance): For suitable , Q:

1 2 N

V

h Av.Delay

()

V

O(1/V)( V )

Comparing TOCA (statistics unaware) to Berry-Gallager (statistics aware) for a single queue

Beyond the Berry-Gallager Bound: Logarithmic delay!

If the Minimum Energy function () is peicewise linear(not strictly concave), then under suitable, Q, TOCA yields:

() (shown in 1 dimension)

Further, logarithmic delay in this scenario is optimal! Simple One Queue Example: P(t) ={0, 1} Watt. Two Equally Likely Channel States (GOOD, BAD):

U(t) (t)=(P(t),S(t))

() Can show thatlogarithmic delay is necessaryand achievable!

Conclusions:1 2 N

V

h Av.Delay

()

V

O(1/V)( V )

-Extend Berry-Gallager Square Root Law to Multi-User Systems.-Novel Lyapunov Technique for Achieving Optimal Energy-Delay Tradeoffs.-Overcome the Complexity Explosion Problem.-Channel Statistics, Traffic Rates not Required.-Superior Tradeoff via a Logarithmic Delay Law in exceptional (piecewise linear) cases.

stochastic network optimization and the theory of network throughput, energy, and delay michael j....

Documents