orsis conference, jerusalem mountains, israel may 13, 2007 yoni nazarathy gideon weiss university of...

ORSIS Conference,Jerusalem Mountains, Israel

May 13, 2007

ORSIS Conference,Jerusalem Mountains, Israel

May 13, 2007

Yoni NazarathyGideon Weiss

University of Haifa

Yoni NazarathyGideon Weiss

University of Haifa

Transient Fluid Solutions and

Queueing Networks withInfinite Virtual Queues

Transient Fluid Solutions and

Queueing Networks withInfinite Virtual Queues

Yoni Nazarathy, Gideon Weiss, University of Haifa, 2007 2

Multi-Class Queueing Networks (Harrison 1988, Dai 1995,…)Multi-Class Queueing Networks (Harrison 1988, Dai 1995,…)

1 2

6

5 4

3

{1,..., }

{ ( ), 0}k

K

Q t t

Queues/Classes

6K

Routing Processes

(0)kQ k Initial Queue Levels

' ( ) , 'kk n k k

Resources

( )kS t k

Processing Durations

{1,..., }

{ } {0,1}I K ik ik

I

A A A

Resource Allocation (Scheduling)

( )

(0) 0 ( ) ( )

( ) ( ) 0

k

k ik k kk

k k

T t

T A T t T s t s s t

T t only when Q t

Network Dynamics

' ' ''

( ) (0) ( ( )) ( ( ( )))k k k k k k k kk

Q t Q S T t S T t

4I


INTRODUCING: Infinite Virtual QueuesINTRODUCING: Infinite Virtual Queues

( ) (0) ( ( ))R t R S T t t

5 10 15 20 25 30

-1

-0.5

0.5

1

1.5

2

2.5

Regular Queue

( ) : {0,1,2,...}kQ t

Infinite Virtual Queue

( )kQ t t

Example Realization

( )R t

NominalProduction

Rate

Relative Queue Length


IVQ’s Make Controlled Queueing Network even more interesting…IVQ’s Make Controlled Queueing Network even more interesting…

Some Resource

The Network PUSH

PULL

To Push Or To Pull? That is the question…

( ) (0) ( ( ))R t R S T t t

High Utilizatio

n

of ResourcesHigh and Balanced

Throughput

Stable and Low

Queue Sizes

Low variance of the

departure process

What does a “good” control achieve?

Fluid oriented Approach:Choose a “good” nominal production rate (α)…


AN EXAMPLE: A Push-Pull Queueing System (Weiss, Kopzon 2002,2006)AN EXAMPLE: A Push-Pull Queueing System (Weiss, Kopzon 2002,2006)

1 1

22

Server 2Server 1

PUSH

PULL

PULL

PUSH

1 1 2 2, 1 1 2 2, “Inherently Stable” “Inherently Unstable”

0,0 1,0 2,0 3,0 4,0 5,0

0,1

0,2

0,3

0,4

n1

n2

1 1 1 1 1 1

1 1 1 1 1 1

2

2

2

2

2 2

2

2

2

2

0,0

1,3

2,0 3,0 4,0 5,0

0,1

0,2

0,3

0,4

n1

n2

11 1 1 1 1

1 1

1 1 1 1

2

2

2

2

2

2,1

2,2

2,3

2

2

22

2

2

3,1

3,2

3,3

2

2

2

2

2

2

4,1

4,2

4,3

2

2

2

2

2

2

5,1

5,2

5,3

2

2

22

2

2

1 1

1,0

1,4

1

1 1

2,4

0,5

2

1,5

1

1 1

2,5

2

2

2

1

1 1

4,5

2For Both Cases,

Positive Recurrent Policies Exist

1 1 2 1 2 2

1 2 1 2

( )

2 2 1 2 1 1

1 2 1 2

( )

1 2 1 Require:

Low variance of the departure process?

PROBABLY

NOT WITH THESE

POLICIES?


OUR MODEL:MCQN+IVQ

OUR MODEL:MCQN+IVQ


Extend the MCQN to MCQN + IVQExtend the MCQN to MCQN + IVQ

0

( )

(0) 0 ( ) ( )

( ) ( ) 0

k

k ik k kk

k k

T t

T A T t T s t s s t

T t only when Q t for k

{1,..., }

{ ( ), 0}k

K

Q t t

1 2

6

5 4

3

Queues/Classes

Routing Processes

(0)kQ k

Initial Queue Levels

' 0( ) 'kk n k k

( )kS t k

Processing Durations

Resource Allocation (Scheduling)

Network Dynamics

0

0

{1,..., }

{ ( ), 0}

{ ( ), 0}k

k

K

Q t t k

R t t k

0(0)kQ k

' ' ' 0'

( ) (0) ( ( )) ( ( ( ))) 0( )

( ) (0) ( ( ))

k k k k k k k kk

k

k k k k k

Q t Q S T t S T t k KZ t

R t R S T t t k K

NominalProductio

nRates

0 {1,2,3,5}

{4,6}

K

K

Resources

{1,..., }

{ } {0,1}I K ik ik

I

A A A


Rates Assumptions of the Primitive SequencesRates Assumptions of the Primitive Sequences

' 0'

1

1

( )lim

'( )lim

0 '

1lim ( )

kk

t

kkkk

n

n

kn

l

S t

tP kn

kn

X l Cn

1

( ) max{ : ( ) }n

k kl

S t n X l t

Primitive Sequences:

' ' ' ' 0

{ ( ), 0} (0) 0 ( )

{ ( ), 0,1,2...} (0) 0 ( ) ( ) , 'k k k

kk kk kk kk

S t t S S t k K

n n n n n k K k K

May also define:

rates assumptions:

{ ( ), 1,2,..}kX l l k K


Static Fluid FormulationStatic Fluid Formulation


The input-output matrix (Harrison)The input-output matrix (Harrison)

( ) ( )TR I P diag

' 0

' ' ' 0

'

'

0

k

kk k k k

k k k K

R P k k k K

k K

Given, the rates assumptions , a fluid view of the outcome of one unit of work on class k’:

is the average depletion of queue k per one unit of work on class k’.

'kkR

( , )P

The input-output matrix:


0

, , 0

max 1

0i I i

k

x

for k K

The Static Equations

Rx

A

A feasible static allocation is the triplet , such that:( , , )x

1

1

1

,K K I K

K

I

K

R A

x

- MCQN model

- Nominal Production rates for IVQs

- Resource Utilization

- Resource Allocation

Similar to ideas from Harrison 2002 (and much older ideas).


Rate Stable Controls

Rate Stable Controls


•Lyapunov function:•Find allocation that reduces it as fast as possible:

Maximum Pressure Policies (Tassiulas, Stolyar, Dai & Lin)Maximum Pressure Policies (Tassiulas, Stolyar, Dai & Lin)

•Reminder: is the average depletion of queue k per one unit of work on class k’. •Treating Z and T as fluid and assuming continuity:

•Reminder: is the average depletion of queue k per one unit of work on class k’. •Treating Z and T as fluid and assuming continuity:

( ) ( ) ( )f t Z t Z t

( ) 2 ( ) ( ) 2 ( ) ( )d

f t Z t Z t Z t RT tdt

( ) ( )Z t RT t

'kkR

( )arg max ( )T

a A tZ t R a

•An allocation at time t: a feasible selection of values of •At any time t, A(t) is the set of available allocations.• , so there is always some allocation.0 ( )A t

Intuitive Meaning of the Policy

( )T t

“Energy” Minimization

The Resulting Policy

Choose:

Feasible Allocations


Given a MCQN with IVQs defined with nominal production rates that are given by a feasible static allocation. The non-processor splitting, no-preemption Maximum Pressure Policy is stable for any primitive sequences that satisfy the rates assumptions in the following two senses:

Given a MCQN with IVQs defined with nominal production rates that are given by a feasible static allocation. The non-processor splitting, no-preemption Maximum Pressure Policy is stable for any primitive sequences that satisfy the rates assumptions in the following two senses:

Rate Stability TheoremRate Stability Theorem

( )lim 0t

Z t

t

( )lim 0

(0, )

N

N

Z t

tuniformly on t T

( ) ( )

(0) ( )

Nk k

N

S t S N t

Z o N

(1) – Rate Stability for infinite time horizon:

(2) – Given a sequence for Finite time horizon, T:

Where satisfies:( )NZ t

Proof is an adaptation of Dai and Lin’s 2005, Theorem 2.

( )NZ t


Work in progress….Work in progress….

How fast does the queue (virtual queue) sizes grow?


How do simpler policies (randomized), that follow the static fluid equations compare?

How do simpler policies (randomized), that follow the static fluid equations compare?

General Applications…General Applications…


Possible Applications of the Theorem

Steady State Systems

Systems with Time Varying Parameters

Tracking Transient Fluid Solutions

of a MCQN


Transient Fluid Solutions

Transient Fluid Solutions


Sta

cked

Que

ue L

evel

s

time T

Q1

Q2Q3

Trajectory of a single job

Finished Jobs

Example NetworkExample Network

Stacked Queue level representation:

Stacked Queue level representation:

Server 1Server 2

1

23

3

10

( )T

kk

Q t dt

Attempt to minimize:


10

( )T K

kk

Q t dt

Sta

cked

Que

ue L

evel

sT

J12

J11

J10

J9

J8

J7

J6

J5

J4

J3

J2

J1

Sta

cked

Que

ue L

evel

s

T

Sta

cked

Que

ue L

evel

s

T

J12

J11

J10

J9

J8

J7

J6

J5

J4

J3

J2

J1

Minimizing inventory costs.

Minimizing the total job waiting time.(truncated to time horizon).

Maximizing the total time from job completion to the time horizon. (maximizing “useful life”)

Corresponds to:

Minimization of


Fluid formulationFluid formulation

1 2 3

0

1 1 1 1

0

2 2 1 1 2 2

0 0

3 3 2 2 3 3

0 0

1 3

2

min ( ) ( ) ( )

( ) (0) ( )

( ) (0) ( ) ( )

( ) (0) ( ) ( )

( ) ( ) 1

( ) 1

( ), ( ) 0

T

t

t t

t t

q t q t q t dt

q t q u s ds

q t q u s ds u s ds

q t q u s ds u s ds

u t u t

u t

u t q t

(0, )t T

s.t.

This is a Separated Continuous Linear Program (SCLP)

Server 1Server 2

1

23


Fluid solutionFluid solution

•SCLP – Bellman, Anderson, Pullan, Weiss.•Piecewise linear solution. •Simplex based algorithm, finds the optimal solution in a finite number of steps (Weiss).

The Optimal Solution:

•SCLP – Bellman, Anderson, Pullan, Weiss.•Piecewise linear solution. •Simplex based algorithm, finds the optimal solution in a finite number of steps (Weiss).

The Optimal Solution:

0 10 20 30 40

0

5

10

15

20

3 3

2 2

1 1

1 3

2

(0) (0) 15

(0) (0) 1

(0) (0) 8

1.0

0.25

40

Q q

Q q

Q q

T

3( )q t

2 ( )q t

1( )q t


0 10 20 30 40

0

5

10

15

20

0 10 20 30 40

0

5

10

15

20

Structure of the optimal solution – comparison to LBFSStructure of the optimal solution – comparison to LBFS

Last Buffer First Server (LBFS):Last Buffer First Server (LBFS):

Improve: Don’t wait with the emptying of buffer 1 until 3 is empty…

The optimal solution:

352Obj

376Obj


Fluid TrackingPolicy

Fluid TrackingPolicy


Near Optimal Control over a Finite Time HorizonNear Optimal Control over a Finite Time Horizon

Approximation Approach:1) Approximate the problem using a fluid system.2) Solve the fluid system (SCLP).3) Track the fluid solution on-line (Using MCQN+IVQs).4) Under proper scaling, the approach is asymptotically optimal.

Approximation Approach:1) Approximate the problem using a fluid system.2) Solve the fluid system (SCLP).3) Track the fluid solution on-line (Using MCQN+IVQs).4) Under proper scaling, the approach is asymptotically optimal.

Solution is intractable

10

( )T K

kk

Min Q t dt

Finite Horizon Control of MCQN


4 Time Intervals4 Time Intervals

For each time interval, set a MCQN with Infinite Virtual Queues.

3

1

2

3

1

2

3

1

2

3

1

2

0 10 20 30 40

5

10

15

20

25

30

0 {} {} {2} {2,3}nK

31 1 10 0 1 0 14 4 4 4

{1,2,3} {1,2,3} {1,3} {1}nK

0 { | ( ) 0, }nk nk q t t

{ | ( ) 0, }nk nk q t t


Let be an objective value for any general policy then:

- Scaling: speeding up processing rates by N and setting initial conditions:

Asymptotic Optimality TheoremAsymptotic Optimality Theorem

( )Q t

( )NQ t( ) (0)NQ t NQ

*fV

*1liminf N

fN

V VN

1lim ( ) ( ) 0N

NQ t q t uniformly on t T

N

*1lim N

fN

V VN

- Queue length process of finite horizon MCQN

- Value of optimal fluid solution.

NV

Using our maximum pressure based fluid tracking policy:


Example realizations, N={1,10,100}Example realizations, N={1,10,100}

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

1N

10N

100N

seed 1 seed 2 seed 3 seed 4 seed 1 seed 2 seed 3 seed 4


Work in progress…. Continued….Work in progress…. Continued….



How fast is convergence stated in the asymptotic optimality theorem???

How fast is convergence stated in the asymptotic optimality theorem???

*1lim N

fNV V

N


Empirical Asymptotics N = 1 to 106Empirical Asymptotics N = 1 to 106

0 200000 400000 600000 800000 1106

-2

0

2

4

6

8

10

0 200000 400000 600000 800000 1106

-2

0

2

4

6

8

10

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

-2

0

2

4

6

8

10

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

0

1000

2000

3000

4000

5000

0 200000 400000 600000 800000 1106

-2

0

2

4

6

8

10

10

1

{}

{1,2,3}

(0, 0, 1)

K

K

u

10

1

{}

{1,2,3}

(0, 1, 1)

K

K

u

10

1

{2}

{1,3}

(0.25, 1, 0.75)

K

K

u

10

1

{2,3}

{1}

(0.25, 0.25, 0.25)

K

K

u

3 3( ) ( )Nn nQ Nq

2 2( ) ( )Nn nQ Nq

1 1( ) ( )Nn nQ Nq

1 Queue 1 Queue


0 10 20 30 40

5

10

15

20

25

30


ThankYou

ThankYou

orsis conference, jerusalem mountains, israel may 13, 2007 yoni nazarathy gideon weiss university of...

Documents

time t

total time

queue virtual queue

infinite time horizon

fluid view

static fluid equations

feasible static allocation

fluid oriented approach