a numerical algorithm for the solution of product-form models with

A numerical algorithm for the solution ofproduct-form models with infinite state spaces

Simonetta Balsamo, Gian-Luca Dei Rossi, Andrea Marin

Universita Ca’ Foscari di VeneziaDipartimento di Informatica

Italy

September, 2010

IntroductionThe INAP+ algorithm

ExampleConclusion

Presentation outline

1 IntroductionSketch of the resultsRCAT by example

2 The INAP+ algorithm

3 Example

4 Conclusion

Simonetta Balsamo, Gian-Luca Dei Rossi, Andrea Marin A numerical algorithm for the solution of product-form models with infinite state spaces


ExampleConclusion

Sketch of the resultsRCAT by example

Product-form models

Consider model S consisting of sub-models S1, . . . ,SN

Let m = (m1, . . . ,mN) be a state of model S and π(m) itssteady state probability

S is in product-form with respect to S1, . . . ,SN if:

π(m) ∝ g1(m1) · g2(m2) · · · gN(mN)

where gi (mi ) is the steady state probability distribution of Siappropriately parametrised

The cardinality of the state space of S is proportional to theproduct of the state space cardinalities of its sub-models ⇒product-form models can be studied more efficiently!



ExampleConclusion


Some problems. . .

How to decide if a model yields a product-form solution?

list of instances: BCMP theorem, Coleman/HendersonStochastic Petri Nets, G-networks. . .general criteria: Markov implies Markov property, RCAT (andextensions),. . .

How to find the correct parametrisation for the sub-models?

solving the linear system of traffic equations forBCMP/Jackson queueing networkssolving the non-linear traffic equations for G-networkssolving the system of equations of RCAT



ExampleConclusion


Sketch of the results

We propose an algorithm that. . .

decides if a model S has a product-form solution with respectto a set of sub-models S1, . . . , SN

derive the correct parametrisation for the sub-models

is able to deal with a class of sub-models with infinite statespaces

compute the unnormalised steady-state solution

The algorithm is based on the Reversed Compound AgentTheorem (RCAT) [Harrison, 2003] and its extensions.



ExampleConclusion


Modelling the cooperation

Cooperation between sub-models is pairwise

The transition in one sub-model (active) forces a transition inanother one (passive)

Synchronised transitions may occur only jointly

The transition rate for the passive sub-model is that of thecooperating active transition

A sub-model may be passive with respect to a cooperationand active with respect to another

Cooperations are identified by labels



ExampleConclusion


Pairwise interacting agents

1 2

3

1 2

3 4

ACTIVE

PASSIVEa

b

ǫ

PEPA-like cooperation with an active and a passive agentActive transitions have a ratePassive transitions have a unspecified rateActive/Passive transitions occur only simultaneously with therate of the active ones



ExampleConclusion


RCAT conditions

1 2

3

1 2

3 4

ACTIVE

PASSIVEa

b

ǫ

Passive transitions outgoing from every state

Active transitions incoming into every state

Same reversed rate for all active transitions



ExampleConclusion


RCAT parametrisation and solution

1 2

3

1 2

3 4

ACTIVE

PASSIVEa

b

ǫ

Parametrisation: replace all the passive transitions with thereversed rate of the corresponding active transitionsSolution: let g1 and g2 be the solution of the parametrisedagents, then the solution π of the cooperating agent is inproduct-form:

π ∝ g1 · g2Simonetta Balsamo, Gian-Luca Dei Rossi, Andrea Marin A numerical algorithm for the solution of product-form models with infinite state spaces


ExampleConclusion

The Iterative Numerical Algorithm for product-forms(INAP+)

Algorithm introduced in [Marin et al. ’09] for models withfinite state spaces (INAP)

The improvements in INAP+ are:

possibility to deal with infinite state space models thanks to adynamic truncationfaster convergence time (experimental results)possibility to express the iterations in Matrix-form (simplifiedimplementation)



ExampleConclusion

The problem of truncation

INFINITE STATESPACE MODELWITH PASSIVETRANSITIONS

FINITE STATESPACE MODEL

VALUES OF THEREVERSED RATES

TRUNCATIONOPERATOR

CLOSURE OFTHE MODEL

INFINITE STATESPACE MODELWITH ONLY ACTIVETRANSITIONS

SiRτ

i(Si)

Ka, a ∈ Pi

Rτ

i

Rτi (Si ) is a finite model (whose number of states belongs to

interval [MINi ,MAXi ]) but none of the states with stationaryprobability higher that τ has been truncated

Easy to use for geometric distributions

But. . . we don’t know the reversed rates for the passivetransitions labelled by a ∈ Pi



ExampleConclusion

Algorithm definition: notation

N: number of agents

Sk , 1 ≤ k ≤ N: state space of agent k

gk , 1 ≤ k ≤ N: hypothetical stationary distribution of agent k

n: number of the iteration being performed

gprevk : stationary distribution computed at step n − 1

M: maximum number of iterations

ε: tolerance



ExampleConclusion

Non-optimized algorithm

Initialize randomly gk for a truncation of S ′k for all k = 1, . . . ,Nn = 0repeat

Compute the reversed rates K(n)a for the active labels

Use K(n)a to close and truncate Sk for all k = 1, . . . ,N

Update gk for all k = 1, . . . ,Nn← n + 1

until n > M or ∀k = 1, . . . ,N. |gk − gprevk | < ε ;

if the reversed rates are not constant thenfail: RCAT product-form not identified

return {gk}k=1,...,N



ExampleConclusion

Computing new gk and convergence

Computing gk and product-forms

gk are computed using the global balance equation system

If gk at step n are identical to gk at step n − 1

Constant reversed rates for active transitions ⇒ product-formsolution foundOtherwise ⇒ no product-form found

Convergence

Convergence has been proved for specific cases

A maximum number of iterations is used to avoid infiniteloops



ExampleConclusion

Computing the reversed rates /1

Even if the RCAT product-form exists during the algorithmiteration the reversed rates of the active transitions may benon-constant

How to compute rates K(n)a for each synchronising label a at

iteration n?

The reversed rate of an active transition labelled by a isRτ

i (Si ) is:

q(n)i (α

a−→ β) =g

(n)i (α)

g(n)i (β)

qi (αa−→ β)



ExampleConclusion

Computing the reversed rates /2

K(n)a is defined as the weighted mean of the reversed rates of

all the active transitions labelled by a

The weight of transition αa−→ β is gi (β)(n)

Hence:

K(n)a =

∑α∈Si

g(n)i (α)

g(n)i (β)

qi (αa−→ β)g

(n)i (β) (1)

h∑

α∈Rτi (Si )

g(n)i (α)qi (α

a−→ β) (2)



ExampleConclusion

Queue with negative customers

0 1 2 3

(a,⊤)

(a,⊤)(a,⊤)(a,⊤)(a,⊤)

(b,⊤)(b,⊤)(b,⊤)(b,⊤)

(c, µ)(c, µ)(c, µ)(c, µ)

Transitions a cause a customer deletion, b a customer arrival

Exponential distribution π(n) ∝ ρn

ρ =Kb

µ+ Ka



ExampleConclusion

Queue with catastrophes

0 1 2 3

(a,⊤)

(a,⊤)

(a,⊤)(a,⊤)

(a,⊤)

(b,⊤)(b,⊤)(b,⊤)(b,⊤)

(c, µ)(c, µ)(c, µ)(c, µ)

Transitions a cause a catastrophe, b a customer arrival

Exponential distribution π(n) ∝ ρn

ρ =Kb + µ+ Ka −

√K 2b + µ2 + K 2

a + 2KbKa + 2µKa − 2Kbµ

2µ



ExampleConclusion

Mix of negative customer and catastrophes triggers

λ1

2

3

4

5

6

7

8

9

10−−

−

−

−

−−

−

C

C

C

C

J

J

J

G

G

G

J: Jackson queue, C: Queue with catastrophes, G: G-queuewith negative customers



ExampleConclusion

Final remarks on INAP+

Compute the un-normalised steady-state distribution forproduct-form models

Based on RCAT

Can be applied to a class of infinite state space models thanksto a dynamic truncation strategy

Fast convergence (experimental result):

6 iterations for the considered example, with a tolerance of10−5

Convergence proved only for special cases

Can solve heterogeneous models (consisting of Petri netblocks, and various types of queues)

Complexity of K models with N states each: O(N3 + K 2N)for each iteration



ExampleConclusion

Future works

Proof of convergence for more general cases

Extension of both the formalism and the algorithm to includenon-pairwise cooperations (e.g. G-networks with positive andnegative triggers)

How to easily express the operator of truncation?

Development of a tool with a simple GUI to allow the user toanalyse heterogeneous models

Approximated product-form analysis based on[Marin et al. ’10a, Marin et al. ’10b]



ExampleConclusion

Further reading

P.G. Harrison: Turning back time in Markovian process algebra,Theoretical Computer Science, vol. 290, n. 3, pp. 1947–1986, 2003

A. Marin and S. R. Bulo: A general algorithm to compute thesteady-state solution of product-form cooperating Markov chains,in Proc. of MASCOTS 2009, pp. 515–524, 2009

A. Marin, M.G. Vigliotti: A general result for deriving product-formsolutions of Markovian models,Proc. of WOSP/SIPEW ’10 Int. Conf. on Perf. Eval.

A. Marin, M.G. Vigliotti: On product-form approximations ofcooperating stochastic models,Proc. of ISCIS ’10 Int. Conf.


a numerical algorithm for the solution of product-form models with

Documents