volume 6, number 2, pages 113{127 - university of albertazhiyongz/read/binder1.pdf · 2010. 5....

INTERNATIONAL JOURNAL OF c⃝ 2010 Institute for ScientificINFORMATON AND SYSTEMS SCIENCES Computing and InformationVolume 6, Number 2, Pages 113–127

CONVERGENCE OF A MONTE CARLO METHOD FOR FULLY

NONLINEAR PARABOLIC LOCAL AND NON-LOCAL PDES

ARASH FAHIM

Abstract. This article is about the proof of convergence of the probabilistic

numerical scheme introduced in [11] and [10] for fully nonlinear parabolic local

and non-local PDEs. Some applications will also be introduced to mention the

importance of the method.

Key Words. Viscosity Solutions, Monotone Schemes, Monte Carlo Approx-

imation, Non-local PDE.

1. Introduction

The probabilistic numerical schemes are now very demanding for problems whereclassical numerical schemes fail to work because of the problem of dimensionality.For approximation of the solution of PDE, Feynman-Kac formula was a startingpoint. The fact that we can not generalize Feynman-Kac formula to non-linearPDEs, turnes attentions to BSDE (Backward SDE1). BSDEs propose a numericalmethod for semi-linear PDEs. See for instance Bally and Pages [2], El Karouiand Hamadene and Matoussi [9] and lecture note of Pardoux [17]. For quasi-linear PDEs, we deal with weakly and fully FBSDEs (Forward Backward SDE).For instance see Antonelli [1], Delarue and Menozzi [8] and Pardoux and Sow [15].

Therefore, approximation of solutions of PDEs needs to approximate the solutionof BSDEs. The discretization involves the computation of conditional expectations,which truns the algorithm into an implementable scheme. For an complete asymp-totic analysis of the approximation, including the non-parametric regression forestimation of conditional expectation see [2], [6] and [12].

The generalization of probabilistic numerical methods to fully non-linear PDEswas first introduced in [7] by the help of 2BSDEs (2nd order BSDE). In the lightof the scheme proposed be [7], [11] and [10] introduce a monotone scheme for fullynon-linear PDEs naturally without appealing to the notion of backward stochasticdifferential equation. Instead, they divide the equation into linear and non-linearpart and then approximate the solution. The linear part gives us a diffusion processwhich we choose in order to stablish the scheme. Then the scheme consists ofconditional expectations which should be approximated using Monte Carlo technics.

The result of this paper establish the convergence of this approximation towardsthe unique viscosity solution of the non-local fully-nonlinear PDE. Here, we do notrequire the fully nonlinear PDE to have a smooth solution, and we only assumethat it satisfies a comparison result in the sense of viscosity solutions. The otherassumptions we need for this result are a uniform ellipticity condition and the

Received by the editors January 1, 2004 and, in revised form, March 22, 2004.

2000 Mathematics Subject Classification. 65C05, 49L25.The author want to thank Mohammad Reza Razvan for his fruitful suggestions.1Stochastic differential equations

113

114 A. FAHIM

diffusion coefficient is needed to dominate the partial gradient of the remainingnonlinearity with respect to its Hessian component.

Our proofs rely on the monotonic scheme method developed by Barles andSouganidis [3] in the theory of viscosity solutions.

The paper is organized as follows. In Section 2, we provide a natural presentationof the scheme without appealing to the theory of backward stochastic differentialequations. Section 3 is dedicated to proof of convergence result. Finally, Section 4contains some application in the world of non-financial and financial mathematics.

Notations For scalars a, b ∈ R, we write a ∧ b := min{a, b}, a ∨ b := max{a, b},and a+ := max{a, 0}. By M(n, d), we denote the collection of all n × d matriceswith real entries. The collection of all symmetric matrices of size d is denoted Sd,and its subset of nonnegative symmetric matrices is denoted by S+d . For a matrixA ∈ M(n, d), we denote by AT its transpose. For A,B ∈ M(n, d), we denoteA · B := Tr[ATB]. In particular, for d = 1, A and B are vectors of Rn and A · Breduces to the Euclidean scalar product.

For a function u from [0, T ]× Rd to R, we say that u has q−polynomial growth(resp. α−exponential growth) if

supt≤T, x∈Rd

|u(t, x)|1 + |x|q

<∞, (resp. supt≤T, x∈Rd

e−α|x||u(t, x)| <∞).

For a suitably smooth function φ on QT := (0, T ]× Rd, we define

|φ|∞ := sup(t,x)∈QT

|φ(t, x)| and |φ|1 := |φ|∞ + supQT×QT

|φ(t, x)− φ(t′, x′)||x− x′|+ |t− t′| 12

.

2. Disceretization

Suppose the following non-local Cauchy problem:

−LXv(t, x)− F(t, x, v(t, x), Dv(t, x), D2v(t, x), v(t, ·)

)= 0, on [0, T )× Rd,(2.1)

v(T, ·) = g, on ∈ Rd.(2.2)

where F : R+ × Rd × R× Rd × Sd × Cd → R and LX given by:

LXφ(t, x):=(∂φ∂t

+ b ·Dφ+1

2a ·D2φ

)(t, x)

+

∫Rd

∗

(φ(t, x+ η(t, x, z))− φ(t, x)− 11{|z|≤1}Dφ(t, x)η(t, x, z)

)dν(z).

Here Sd and Cd are respectively the spaces of symetric d × d real matrices andbounded real functions on Rd, Rd

∗ := Rd \ {0}, a = σTσ where σ : Rd → S+d and

b : Rd → Rd. LX is the infinitesimal generator of a jump-diffusion SDE Xt withIto-Levy decomposition:

dXt = b(t,Xt)dt+ σ(t,Xt)dWt +

∫{|z|>1}η(t,Xt−, z)J(dt, dz) +

∫{|z|≤1}η(t,Xt−, z)J(dt, dz),

where J and J are respectively a Poisson jump measure and its compensation.Our purpose is to introduce a Monte Carlo method which approximates the

solution of problem (2.1)-(2.2). Suppose that h = TN and ti = ih. We define the

MONTE CARLO METHOD FOR FULLY NONLINEAR PARABOLIC IPDES 115

Euler discretization of Levy process X together with truncation of Levy measureby:

Xt,x,κh = x+ b(t, x)h+ σ(t, x)Wh +

∫{|z|>κ}

η(t, x, z)J([0, h], dz),(2.3)

Xx,κti+1

= Xti,X

x,κti

,κ

h and Xx,κ0 = x,(2.4)

where b(t, x) = b(t, x)+∫{|z|>1} η(t, x, z)ν(dz) and κ is the truncation bound. Sup-

pose that Nκt is the poisson process derived from jump measure J by fixing the size

of all jumps of size greater than κ to 1.

Nκt =

∫{|z|>κ}

J([0, T ], dz).(2.5)

Notice that Nκt has intensity equal to λκ :=

∫{|z|≥κ} ν(dz). Let N

κt be the compen-

satoin of Nκt e.g.

Nκt =

∫{|z|>κ}

J([0, T ], dz),(2.6)

where J is the componsatoin of J . By simple calculations, we can write Xt,x,κh as

a compound poisson process. So,

Xt,x,κh = x+ bκ(t, x)h+ σ(t, x)Wh +

Nκh∑

i=1

η(t, x, Yi),

where bκ(t, x) = b(t, x)−∫{κ<|z|≤1} η(t, x, z)ν(dz), Yis are i.i.d. R

d∗−valued random

variables, independent of W distributed as 1λκν(dz).

Assuming that the PDE (2.1) has a classical solution, it follows from Ito’s formulathat

Eti,x

[v(ti+1, Xti+1

)]= v (ti, x) + Eti,x

[∫ ti+1

ti

LXv(t,Xt)dt

],

where we ignored the difficulties related to local martingale part, and Eti,x :=E[·|Xti = x] denotes the expectation operator conditional on {Xti = x}. Since vsolves the PDE (2.1), this provides

Eti,x

[v(ti+1, Xti+1

)]= v(ti, x)− Eti,x

[∫ ti+1

ti

F (·, v,Dv,D2v, v(t, ·))(t,Xt)dt

].

By approximating the Riemann integral, and replacing the process X by its Eulerdiscretization, this suggest the following approximation of the value function v

vh(T, .) = g and vh(ti, x) = Th[vh](ti, x),(2.7)

where for every function ψ : R+ × Rd −→ R with exponential growth:

Th[ψ](t, x) := E[ψ(t+ h, Xt,x

h

)]+ hF (t, x,Dhψ,ψ(t+ h, ·)) ,(2.8)

Dhψ :=(D0

hψ,D1hψ,D2

hψ),(2.9)

Dkhψ(t, x) := E[Dkψ(t+ h, Xt,x

h )], k = 0, 1, 2 ,(2.10)

and Dk is the k−th order partial differential operator with respect to the spacevariable x.

The differentiations in the above scheme are to be understood in the sense ofdistributions. This algorithm is well-defined whenever g has exponential growthand F is a Lipschitz map. To see this, observe that any function with exponential

116 A. FAHIM

growth has weak gradient and Hessian because the Gaussian kernel is a Schwartzfunction, and the exponential growth is inherited at each time step from the Lips-chitz property of F .

At this stage, the above backward algorithm presents the serious drawback ofinvolving the gradient Dvh(ti+1, .) and the Hessian D2vh(ti+1, .) in order to com-pute vh(ti, .). The following result avoids this difficulty by an easy integration byparts argument.

Lemma 2.1. For every function φ : QT → R with exponential growth, we have

Dhφ(ti, x) = E[φ(ti+1, Xti,xh )Hh(ti, x)],

where Hh = (Hh0 ,H

h1 ,H

h2 ) and

Hh0 = 1, Hh

1 =(σT)−1 Wh

h, Hh

2 =(σT)−1 WhW

Th − hIdh2

σ−1.(2.11)

Remark 2.1. This Lemma is used in [14] for sensibility analysis purpose.

Proof. The main ingredient is the following easy observation. Let G be a onedimensional Gaussian random variable with unit variance. Then, for any functionf : R −→ R with exponential growth, we have

E[f(G)Hk(G)] = E[f (k)(G)],(2.12)

where f (k) is the k−th order derivative of f in the sense of distributions, and Hk

is the one-dimensional Hermite polynomial of degree k.1 Now, let φ : Rd −→ R be a function with exponential growth. Then, by directconditioning, it follows from (2.12) that

E[φ(Xt,x

h )W ih

]= h

d∑j=1

E[∂φ

∂xj(Xt,x

h )σji(t, x)

],

and therefore:

E[φ(Xt,x

h )Hh1 (t, x)

]= σ(t, x)TE

[∇φ(Xt,x

h )].

2 For i = j, it follows from (2.12) that

E[φ(Xt,x

h )W ihW

jh

]= h

d∑k=1

E[∂φ

∂xk(Xt,x

h )W jhσki(t, x)

]

= h2d∑

k,l=1

E[

∂2φ

∂xk∂xl(Xt,x

h )σlj(t, x)σki(t, x)

],

and for j = i

E[φ(Xt,x

h )((W ih)

2 − h)]

= h2d∑

k,l=1

E[

∂2φ

∂xk∂xl(Xt,x

h )σli(t, x)σki(t, x)

].

This provides

E[φ(Xt,x

h )Hh2 (t, x)

]= σ(t, x)TE

[∇2φ(Xt,x

h )σ(t, x)].

�

Observe that the choice of the drift and the diffusion coefficients b and σ in thenonlinear PDE (2.1) is arbitrary. So far, it has been only used in order to define the


underlying diffusion X. Our convergence result will however place some restrictionson the choice of the diffusion coefficient, see Assumption F.

3. Convergence

We need to impose the following assumptions.Assumption F The nonlinearity F is Lipschitz with respect to (r, p, γ, ψ)

uniformly in (t, x) and sup(t,x)∈[0,T ]×Rd |F (t, x, 0, 0, 0)| < ∞. Moreover, F is in-creasing with respect to ψ, uniformly elliptic and dominated by the diffusion of thelinear operator LX , i.e.

F (r, p, γ, ψ) ≤ F (r, p, γ, φ) when φ ≤ ψ(3.1)

εId ≤ ∇γF ≤ a on Rd × R× Rd × Sd for some ε > 0.(3.2)

and is increasing with respect to ψ. Our first main result is:

Theorem 3.1 (Convergence). Let Assumption F hold true, µ and σ be Lipschitzon x and 1/2−Holder continuous on t and assume that the fully nonlinear nonlocalPDE (2.1) has comparison for bounded functions. Then for every bounded functiong, there exists a bounded function v so that

vh −→ v locally uniformly.

In addition, v is the unique bounded viscosity solutionof the relaxed boundary prob-lem (2.1)-(2.2).

Remark 3.1. Assume that the coefficients b and σ are bounded. Then, Therestriction to bounded terminal data g in the above Theorem 3.1 can be relaxed byan immediate change of variable. Let g be a function with α−exponential growthfor some α > 0. Fix someM > 0, and let ρ be an arbitrary smooth positive functionwith

ρ(x) = eα|x| for |x| ≥M,

so that both ρ(x)−1∇ρ(x) and ρ(x)−1∇2ρ(x) are bounded. Let

u(t, x) := ρ(x)−1v(t, x) for (t, x) ∈ [0, T ]× Rd.

Then, the nonlinear PDE problem (2.1)-(2.2) satisfied by v converts into the fol-lowing nonlinear PDE for u:

−LXu− F(·, u,Du,D2u

)= 0, on [0, T )× Rd

and(3.3)

v(T, ·) = g := ρ−1g on ∈ Rd,

where

F (t, x, r, p, γ) := rb(x) · ρ−1∇ρ+ 1

2Tr[a(x)

(rρ−1∇2ρ+ 2pρ−1∇ρT

)]+ρ−1F

(t, x, rρ, r∇ρ+ pρ, r∇2ρ+ 2p∇ρT + ργ

).

Recall that the coefficients b and σ are assumed to be bounded. Then, it is easy tosee that F satisfies the same conditions as F . Since g is bounded, the convergenceTheorem 3.1 applies to the nonlinear PDE (3.3). �

118 A. FAHIM

3.1. Convergence. As, in [11], our first main convergence results follow the gen-eral methodology of Barles and Souganidis [3], and requires that the nonlinearnonlocal PDE (2.1) satisfies a comparison result in the sense of viscosity solutions.

Definition 3.1. The viscosity sub(super) solution of (2.1)-(2.2) is a LSC(USC)function v such that for any (t0, x0) and any smooth function φ with

0 = min(max){v − φ} = (v − φ)(t0, x0)

we have

0 ≥ (≤) −LXφ(t0, x0)− F(·, φ,Dφ,D2φ,φ(·)

)(t0, x0),

v(T, ·) ≥ (≤) g(·).

Definition 3.2. We say that (2.1) has comparison for bounded functions if forany bounded upper semicontinuous viscosity supersolution v and any bounded lowersemicontinuous subsolution v on [0, T ) × Rd, satisfying v(T, ·) ≥ v(T, ·), we havev ≥ v.

Remark 3.2. We could have imposed some technical assumption on F and g whichensure that the above comparison assumption is satisfied. For more discussion, seeRemark 4.8(2) in [7].

By the result of [3], under the comparison principle, a consistent, monotone andstable scheme converges to the unique viscosity solution of problem (2.1)-(2.2). Theproofs of the following Lemmas follows the same lines of argument as the similarLemmas in [11].

Lemma 3.1. Let φ be a bounded smooth function with the bounded derivatives.Then for all (t, x) ∈ [0, T ]× Rd:

lim(t′, x′) → (t, x)(h, c) → (0, 0)t′ + h ≤ T

φ(t′, x′)−Th[c+ φ](t′, x′)

h= −

(LXφ+ F (·, φ,Dφ,D2φ)

)(t, x).

Proof. The proof is straight forward and relies on the use of Lebesq gue dominatedconvergence Theorem. �

Lemma 3.2. Let φ and ψ be two bounded functions from [0, T ]× Rd to R. Then

φ ≤ ψ=⇒Th[φ](t, x) ≤ Th[ψ](t, x) + Ch(E[(ψ − φ)(t+ h, Xt,x

h )])

for some C > 0.

Proof. By Lemma 2.1 the operator Th can be written as:

Th[ψ](t, x) = E[ψ(Xt,x

h )]+ hF

(t, x,E[ψ(Xt,x

h )Hh(t, x)], ψ(t, ·)).

Let f := ψ − φ ≥ 0 where φ and ψ are as in the statement of the lemma. Let Fτ

and Fζ denote the partial gradient with respect to τ = (r, p, γ) and ζ, respectively.By the mean value Theorem

Th[ψ](t, x)−Th[φ](t, x) = E[f(Xt,x

h )]+ hFτ (θ) · Dhf(X

t,xh ) + ⟨Fζ |f(t+ h, ·)⟩.

Because F is increasing with respect to ζ, Fζ is a positive operator. So,

Th[ψ](t, x)−Th[φ](t, x) ≤ E[f(Xt,x

h ) (1 + hFτ (θ) ·Hh(t, x))],


for some θ = (t, x, r, p, γ, ζ). By the definition of Hh(t, x):

Th[ψ]−Th[φ] = E[f(Xt,x

h )

(1 + hFr + Fp.σ

T−1Wh

+h−1Fγ · σT−1(WhW

Th − hI)σ−1

)],

where the dependence on θ and x has been omitted for notational simplicity. SinceFγ ≤ a by (3.1) of Assumption F, we have 1− a−1 · Fγ ≥ 0 and therefore

Th[ψ]−Th[φ] ≥ E[f(Xt,x

h )(hFr + Fp.σ

T−1Wh + h−1Fγ · σT−1

WhWTh σ

−1)]

= E[f(Xt,x

h )

(hFr + |Ah|2 −

h

4FTp (Fγ)

−1Fp

)],

where

Ah :=1√hF 1/2γ σT−1

Wh +

√h

2F−1/2γ Fp.(3.4)

Hence

Th[ψ]−Th[φ] ≥ E[f(Xt,x

h )

(hFr −

h

4FTp F

−1γ Fp

)]≥ −ChE

[f(Xt,x

h )]

for some constant C > 0, where the last inequality follows from the Lipschitzproperty of F and the fact that |F−1

γ |∞ <∞ by (3.1). �

Remark 3.3. The monotonicity result of the previous Lemma 3.2 is slightly dif-ferent from that required in [3]. However, as it is observed in Remark 2.1 in [3],their convergence theorem holds under this approximate monotonicity. From theprevious proof, we observe that if the function F satisfies the condition

Fr −1

4FTp F

−1γ Fp ≥ 0,(3.5)

then the standard monotonicity condition

φ ≤ ψ =⇒ Th[φ](t, x) ≤ Th[ψ](t, x)(3.6)

holds. As Remark 3.8 in [11], we can manipulate the equation to a new one withnonlinearity F so that the corresponding scheme; Th; satisfies (3.6).

The following Lemma shows the stability of the scheme.

Lemma 3.3. Let φ,ψ : [0, T ] × Rd −→ R be two L∞−bounded functions. Thenthere exists a constant C > 0 such that

|Th[φ]−Th[ψ]|∞ ≤ |φ− ψ|∞(1 + Ch).

In particular, if g is L∞−bounded, the family (vh)h defined in (2.7) is L∞−bounded,uniformly in h.

Proof. Let f := φ− ψ. Then, arguing as in the previous proof,

Th[φ]−Th[ψ] = E[f(Xh)

(1− a−1 · Fγ + |Ah|2 + hFr −

h

4FTp F

−1γ Fp

)]+h⟨Fζ |f(t+ h, ·)⟩,

120 A. FAHIM

where Ah is given by (3.4). Since 1− Tr[a−1Fγ ] ≥ 0 and |F−1γ |∞ < ∞ by (3.1) of

Assumption F, it follows from the Lipschitz property of F that

|Th[φ]−Th[ψ]|∞ ≤ |f |∞(1− a−1 · Fγ + E[|Ah|2] + Ch

)= |f |∞

(1 +

h

4FTp F

−1γ Fp + Ch

)≤ |f |∞(1 + Ch).

To prove that the family (vh)h is bounded, we proceed by backward induction. Bythe assumption of the lemma vh(T, .) = g is L∞−bounded. We next fix some i < nand we assume that |vh(tj , .)|∞ ≤ Cj for every i+ 1 ≤ j ≤ n− 1. Proceeding as inthe proof of Lemma 3.2 with φ ≡ vh(ti+1, .) and ψ ≡ 0, we see that∣∣vh(ti, .)∣∣∞ ≤ h |F (t, x, 0, 0, 0)|+ Ci+1(1 + Ch).

Since F (t, x, 0, 0, 0) is bounded by Assumption F, it follows from the discrete Gron-wall inequality that |vh(ti, .)|∞ ≤ CeCT for some constant C independent of h.�

The following Lemmas complete the proof of convergence together with corollary(3.1).

Lemma 3.4. Let b and σ be Lipschitz in x. Then under Assumption F, the functionvh is Lipschitz in x, uniformly in h.

Proof. We report the following calculation in the one-dimensional case d = 1 inorder to simplify the presentation. For fixed t ∈ [0, T −h], we argue as in the proofof Lemma 3.2 to see that for x, x′ ∈ Rd with x = x′:

|vh(t, x)− vh(t, x′)| ≤∣∣∣E[(vh(t+ h, Xt,x

h )− vh(t+ h, Xt,x′

h ))

(1− σ−2(x)Fγ + h−1Fγσ

−2(x)W 2h + Ch

)+FγDv

h(t+ h, Xt,x′

h )(σ−1(x)− σ−1(x′)

)Wh

]∣∣∣,for some positive constant C. Assuming that vh(t+ h, .) is Lipschitz with constantLt+h, this shows that

lim sup|x−x′|→0

|vh(t, x)− vh(t, x′)||x− x′|

≤∣∣∣E[Dvh(t+ h, Xt,x

h ){(1 + ah+ b

√hN)

(1− α+ αN2 + Ch)− βb√hN}]∣∣∣,

where a := b′(x), b := σ′(x), and β := σ−2(x)Fγ are all bounded and determin-

istic, and N is a standard normal distribution. Let P be the probability measureequivalent to P with density Z := (1− α+ αN2 + Ch)/(1 + Ch). Then


|vh(t, x)− vh(t, x′)||x− x′|

≤Lt+hEP[∣∣∣(1 + ah+ b

√hN)(1 + Ch)− βb

√hNZ−1

∣∣∣]≤Lt+h

√EP[∣∣∣(1 + ah+ b

√hN)(1 + Ch)− βb

√hNZ−1

∣∣∣2]

=Lt+h

√E[Z∣∣∣(1 + ah+ b

√hN)(1 + Ch)− βb

√hNZ−1

∣∣∣2].


We next develop the calculation of the square term inside the expectation, andcollect the terms which are multiplied by the same power of h. The crucial obser-vation is that the coefficient

√h is multiplied by (1 + Ch)bNZ − 2βbN which has

zero mean. Hence


|vh(t, x)− vh(t, x′)||x− x′|

≤ Lt+h

√1 + C ′h

for some constant C ′. Since the scheme consists of T/h steps, the latter estimateshows that vh(t, .) is Lipschitz with constant bounded by

LT

√1 + C ′h

T/h∼ LT e

TC′/2.

�

Lemma 3.5. Let b and σ be 1/2−Holder-continuous in t. Then under AssumptionF, the function vh is 1/2−Holder-continuous in t, uniformly in h.

Proof. The 1/2−Holder continuity follows the same line of argument as the proofthat vh is Lipschitz in x if we could prove

|vh(t, x)− g(x)| ≤ C(T − t)12 .

As in the proof of x-Lipschitz regularity of vh;

vh(t, x) = E[vh(ti+1, X

t,xti+1

)(1− αi+1 + αi+1N

2i + Ch

)],

and for any j > i,

vh(tj−1, Xt,xtj−1

) = Ej−1

[vh(tj , X

t,xtj )

(1− αj + αjN

2j + Ch

)],

whereNj =Wtj

−Wtj−1√h

has a standard Gaussian distribution, αj is a Fj−1−adapted

random variable and C is a constant independent of h. Now we combine the aboveformula for j from i to N(last step),

vh(t, x) = E

g(Xt,xT )

N−1∏j=i

(1− αj + αjN

2j + Ch

) .For simplicity we denote

∏N−1j=i

(1− αj + αjN

2j + Ch

)by Pi. Now, we can arrive

at the most technical part of the proof. Let first approximate g by a smooth functiongε using a family of molifiers; gε = g ∗ ρε so that

|gε − g| ≤ Cε.

Therefore;

vh(t, x)− g(x) = E[(g(Xt,x

T )− g(x))Pi

]+ g(x)E[Pi]− g(x).

But from the footnote comment E [Pi] ≤ eC(T−t). Therefore,

|vh(t, x)− g(x)| ≤∣∣∣E [(g(Xt,x

T )− g(x))Pi

]∣∣∣+ C|g|∞(T − t).(3.7)

Let’s go to the part E[(g(Xt,x

T )− g(x))Pi

]in (3.7). First we add and subtract as

follows:

g(Xt,xT )− g(x) = g(Xt,x

T )− gε(Xt,xT ) + gε(X

t,xT )− gε(x) + gε(x)− g(x).

122 A. FAHIM

Replacing the above in E[(g(Xt,x

T )− g(x))Pi

]and by approximation of g by gε,∣∣∣E [(g(Xt,x

T )− g(x))Pi

]∣∣∣ ≤ E[Pi

∣∣∣g(Xt,xT )− gε(X

t,xT )∣∣∣]

+∣∣∣E [(gε(Xt,x

T )− gε(x))Pi

]∣∣∣+ |gε − g|∞E [Pi] ,

the first and third term in the right hand side of the above inequality are less thanCε for some constant C. So,∣∣∣E [(g(Xt,x

T )− g(x))Pi

]∣∣∣ ≤ Cε+∣∣∣E [(gε(Xt,x

T )− gε(x))Pi

]∣∣∣ .(3.8)

Now, the last job is to estimate the term E[(gε(X

t,xT )− g(x)

)Pi

]. By applying

Ito formula to gε, we have;

gε(Xt,xT )− gε(x) =

∫ T

t

(Dgε(X

t,xs )b(s, Xt,x

T ) +1

2Tr[D2gε(X

t,xT )a(s, Xt,x

T )])

ds

+

∫ T

t

Dgε(Xt,xs )σ(s, Xt,x

T )dWs,

where b(s, x) = b(tj , x) and σ(s, x) = σ(tj , x) for tj ≤ s < tj+1 and a = σT σ.Consider that

E

[Pi

∫ T

t


T )dWs

]=

N−1∑j=i

E

[Pi

∫ tj+1

tj

Dgε(Xt,xs )σ(s)dWs

].

Each term in the above right hand summation can be written by the means ofconditional expectation

E

[Pi

∫ tj+1

tj


](3.9)

= E

[Etj+1

[Pi

∫ tj+1

tj


s )dWs

]]

= E

[PiP

−1j+1

∫ tj+1

tj

Dgε(Xt,xs )σ(s)dWsEtj+1 [Pj+1]

].

Because PiP−1j+1 is Fj+1−measurable. So,

E

[Pi

∫ tj+1

tj


]

= E

[PiP

−1j+1

∫ tj+1

tj


](1 + Ch)

T−tjh .

For the expectation on the right hand side we have

E

[PiP

−1j+1

∫ tj+1

tj


]

= E

[PiP

−1j σ(s)Etj

[(1− αj + αjN

2j )

∫ tj+1

tj

Dgε(Xt,xs )dWs

]].


Notice that σ(s) is Ftj−measurable for s ∈ [tj , tj+1). Then

Etj

[(1− αj + αjN

2j )

∫ tj+1

tj

Dgε(Xt,xs )dWs

]

= h−1Etj

[((Wtj+1 −Wtj )

2 − h) ∫ tj+1

tj

Dgε(Xt,xs )dWs

].

By Ito isometry

Etj

[((Wtj+1 −Wtj )

2 − h) ∫ tj+1

tj

Dgε(Xt,xs )dWs

]

= 2Etj

[∫ tj+1

tj

WsDgε(Xt,xs )ds

].

Therefore,

Etj

[(1− αj + αjN

2j )

∫ tj+1

tj

Dgε(Xt,xs )dWs

]

= 2h−1Etj

[∫ tj+1

tj

WsDgε(Xt,xs )ds

]

= 2h−1

∫ tj+1

tj

sEtj

[Ws

sDgε(X

t,xs )

]ds

= 2h−1

∫ tj+1

tj

sσ(tj)Etj

[D2gε(X

t,xs )]ds.

So,∣∣∣∣∣Etj

[(1− αj + αjN

2j )

∫ tj+1

tj

Dgε(Xt,xs )dWs

]∣∣∣∣∣ ≤ 2Ch−1ε−1

∫ tj+1

tj

sds = Chε−1.

Therefore, (3.9) could be bounded by∣∣∣∣∣E[Pi

∫ tj+1

tj


]∣∣∣∣∣ ≤ C(1 + Ch)T−tj

h hE[PiP

−1j

]≤ C(1 + Ch)

T−th hε−1.

So, ∣∣∣∣∣E[Pi

∫ T

t


]∣∣∣∣∣ ≤ CeT−tε−1N−1∑j=i

h = C(T − t)ε−1.

Now we try to bound the integrand in the above integral;

Dgε(Xt,xs )b(s, Xt,x

T ) +1

2Tr[D2gε(X

t,xT )a(s, Xt,x

T )].

By Lipschitz continuity of g and boundedness of b and σ, the first term above isbounded and |D2gε|∞ ≤ Cε−1 (Because D2gε = Dρε ∗Dg.). So there exist someconstant C such that,∣∣∣∣Dgε(Xt,x

s )b(s, Xt,xT ) +

1

2Tr[D2gε(X

t,xT )a(s, Xt,x

T )]∣∣∣∣ ≤ C + Cε−1.

124 A. FAHIM

Therefore; ∣∣∣E [(gε(Xt,xT )− gε(x)

)Pi

]∣∣∣ ≤ C(T − t) + C(T − t)ε−1.(3.10)

By (3.10) and (3.8), we conclude that∣∣∣E [(g(Xt,xT )− g(x)

)Pi

]∣∣∣ ≤ Cε+ C(T − t)ε−1 + C(T − t).

By (3.7),

|vh(t, x)− g(x)| ≤ Cε+ C(T − t)ε−1 + C(T − t).

It just remains to choose ε =√T − t. �

Remark 3.4. Notice that the proof of the above Theorem propose a simpler prooffor the stability Theorem.

|v(t, x)| = |g|∞E

N−1∏j=i

(1− αj + αjN

2j + Ch

) .But, Ej−1

[(1− αj + αjN

2j + Ch

)]= 1− αj + αj + Ch = 1 + Ch. So,

E

N−1∏j=i

(1− αj + αjN

2j + Ch

) = (1 + Ch)(T−t)/h

which is not greater than eC(T−t).

Corollary 3.1. Define:

v∗(t, x) := lim inf(t′, x′) → (t, x)

h → 0

vh(t′, x′) and v∗(t, x) := lim sup(t′, x′) → (t, x)

h → 0

vh(t′, x′).

Then v∗ and v∗ satisfy the final condition (2.2).

4. Applications

In this chapter, we give a few applications of fully non-linear parabolic equationswhere we can use (2.7). The applications are both financial and non-financial.

4.1. Mean curvature flow problem. The mean curvature flow equation de-scribes the motion of a surface where each point moves along the inward normaldirection with speed proportional to the mean curvature at that point. This geomet-ric problem can be characterized as the zero-level set S(t) := {x ∈ Rd : v(t, x) = 0}of a function v(t, x) depending on time and space satisfying the geometric partialdifferential equation:

vt −∆v +Dv ·D2vDv

|Dv|2= 0 and v(0, x) = g(x)(4.1)

and g : Rd −→ R is a bounded Lipschitz-continuous function. We refer to [16]for more details on the mean curvature problem and the corresponding stochasticrepresentation.

For instans, to model the motion of a sphere in Rd with radius 2R > 0, we takeg(x) := 2R − |x|2 so that g is positive inside the sphere and negative outside. Wefirst solve the sphere problem in dimension 3. In this case, it is well-known that


the surface S(t) is a sphere with a radius R(t) equal to 2√R2 − t for t ∈ (0, R2).

Reversing time, we rewrite (4.1) for t ∈ (0, T ) with T = R2:

−vt −1

2σ2∆v + F (x,Dv,D2v) = 0 and v(T, x) = g(x),(4.2)

where

F (x, z, γ) := γ

(1

2σ2 − 1

)+z · γz|z|2

.

The algorithm described above is designed to handle any type of geometry.One advantage of this method is the total parallelization that can be performed

to solve the problem for different points on the surface.

4.2. Portfolio optimization. For the sake of simplicity, we just consider simpleBlack-Scholes model in one dimension. The generalization to more complicatedmodels in higher dimensoins is straight forward. The PDEs arise here have rarelyclosed form solutions. For some few examples see [13], [5] and [4] Consider aportfolio of one risky asset St and a bank account B where the dynamics of St is

dSt

St−= bdt+ σdWt +

∫R∗

η(z)J(dt, dz)

where J(dt, dz) = J(dt, dz)−dt×dν(z), J and ν are a Poisson random measure andits assosciated Levy measure, respectively. For the sake of simplicity we supposethat J has finite activity and moreover is supported in {|z| ≥ 1}. Then the dynamicsof discounted wealth process after change of numeraire is

dXx,θt = θ

((b− r)dt+ σdWt +

∫R∗

η(z)J(dt, dz)

).

Our goal is to maximize

E[U(Xθ

T )|X0 = x]

for some utility function U . For this purpose, let’s define

v(t, x) := supθ·

E[U(Xθ

T )|Xt = x].

It is known that v is a viscosity solution of

−∂v∂t

− supθ

{θ(b− r)

∂v

∂x+

1

2θ2σ2 ∂

2v

∂x2+(4.3) ∫

{|z|≥1}

(v(t, x+ θη(z))− v(t, x)) dν(z)

}= 0,

v(T, ·) = U(·).(4.4)

In the absence of jump process the above final value problem will change to

−∂v∂t

+(b− r)2

(∂v∂x

)22σ2 ∂2v

∂x2

= 0,

v(T, x) = U(x).

Here the nonlinearity F could be easyly derived by calculating the supremum.Therefore,

F (r, γ) :=(b− r)2p2

2σ2γ.

126 A. FAHIM

But in the case of jump-diffusion, (4.3) has encounter the problem of having θ insidethe argument of the solutions. In this case, there is no easy representation for thenonlinearity and the supremum should be approximated through some optimizationtechnics. However, in [5] and [4], there are such equations with closed form solutionsin one dimension but still without explicite form for nonlinearity. This makes thescheme unimplementable if we could not find a good approximation of F in areasonable time.

Remark 4.1. The same argument yields the HJB PDE for more complicated mod-els such as Heston model and CEV2 models when one wants to deal with stochasticvolatility .

Acknowledgments

This article is derived from my thesis under the cotutelle program between EcolePolytechnique in France and Sharif University of Technology in Iran under thesupervision of Nizar Touzi and Bijan Z. Zangeneh. I need to thank from both ofthe directors for their supports and patience.

References

[1] F. Antonelli, “Backward-Forward Stochastic Differential Equations” Journal of Computa-tional Finance, Vol.9,1, 1-40, 2005

[2] V. Bally, “An Approximation Schemes For BSDEs and Applications to Control and NonlinearPDEs”. Prepublication 95-15 du Laboratoire de Statistique et Processus de l’Universite duMaine, (1995).

[3] G. Barles, P. E. Souganidis, “Convergence of Approximation Schemes for Fully Non-linearSecond Order Equation”. Asymptotic Anal., 4, pp. 271–283, 1991.

[4] F.E. Benth, K.H. Karlsen, K. Reikvam “Optimal portfolio management rules in a non-Gaussian market with durability and intertemporal substitution”. Finance Stochast. 5, 447-

467 (2001).[5] F.E. Benth, K.H. Karlsen, K. Reikvam “Optimal portfolio selection with consumption and

nonlinear integro-differential equations with gradient constraint: A viscosity solution ap-proach”. Finance Stochast. 5, 275-303 (2001).

[6] B. Bouchard, N. Touzi, “Discrete-time approximation and Monte Carlo simulation of back-ward stochastic differential equations”. Stochastic Processes and their Applications 111, 175–206 (2004).

[7] P. Cheridito, H. M. Soner, N. Touzi, N. Victoir, “Second Order Backward Stochastic Dif-

ferential Equations and Fully Non-Linear Parabolic PDEs”. Communications on Pure andApplied Mathematics, Volume 60, Issue 7, Date: July 2007, Pages: 1081-1110.

[8] F. Delarue and S. Menozzi, “A Forward-Backward Stochastic Algorithm for Quasi-Linear

PDEs”. Annals of Applied Probability, Vol. 16, No. 1, 140-184, 2006.[9] N. El Karoui, S. Hamadene, A. Matoussi, “Backward Stochastic Differential Equations and

Applications”. Indeference Pricing: Theory and Applications, edited by Rene Carmona,Princeton Series in Financial Engineering, 267-320, (2009).

[10] A. Fahim, “A Probabilistic Numerical Method for Fully Nonlinear Nonlocal Parabolic PDEs”.Preprint (2009).

[11] A. Fahim, N. Touzi and X. Warin, “A Probabilistic Numerical Method for Fully NonlinearParabolic PDEs”. submitted to Ann. Appl. Probab. (2009).

[12] E. Gobet, J. P. Lemor, X. Warin “A regression-based Monte-Carlo method to solve backwardstochastic differential equations”. Annals of Applied Probability, Vol.15(3), pp.2172-2002 -2005.

[13] S. L. Heston, “A Closed-Form Solution for Options with Stochastic Volatility with Applica-

tions to Bond and Currency Options”. The Review of Financial Studies, Vol.6(2), 327-343,1993.

2Constant Elasticity of Variance


[14] P. L. Lions, H. Regnier, Calcul du prix et des “Sensibilites d’une option americaine par unemethode de Monte Carlo”, (2001), Preprint.

[15] E. Pardoux, A. B. Sow, “Probabilistic interpretation of a system of quasi-linear parabolicPDEs” Stochastics Stochastics Rep. 76, No. 5, 429-477 (2004).

[16] H. M. Soner and N. Touzi, “A stochastic representation for mean curvature type geometricflows”. The Annals of Probability, Vol 31(3),1145-1165 (2003).

[17] E. Pardoux, “Backward Stochastic Differential Equations and Viscosity Solutions of Systemof Semi-linear Parabolic and Elliptic PDEs of Second Order” Lecture Note.

CMAP, Ecole Polytechnique Paris & Sharif University of Technology, Tehran

E-mail : [email protected]

INTERNATIONAL JOURNAL OF c© 2010 Institute for ScientificINFORMATION AND SYSTEMS SCIENCES Computing and InformationVolume 6, Number 2, Pages 128–141

CONTACT PROBLEM AND CONTROLLABILITY

FOR SINGULAR SYSTEMS IN

BIOMEDICAL ROBOTICS

IVAN M. BUZUROVIC1 AND DRAGUTIN LJ. DEBELJKOVIC2

Abstract. Contact problem for a robotic system in interaction with the en-

vironment is challenging especially in the bioengineering applications. In this

paper we presented general method for the system modeling. Analyzing system

dynamics in the working regime it has been concluded that mathematical model

has a singular characteristic in the contact region. Therefore, the system was

described as a singular system of differential equations. During the working

regime contact surface was treated as a constraint to the system. Geometric

conditions for transformation to the state space model have been developed.

Controllability analysis for the robotic system was performed using geomet-

ric approach. State feedback was introduced to stabilize the system. Both

the mathematical conditions for the eigenvalues assignment and the feedback

matrix structure were presented

Key Words. robotic system, controllability, geometric approach, singular

system.

1. Introduction

The application of the robotic systems has been widespread over the range ofdifferent fields. Biomedical robotics has attracted many researchers who developedand significantly improved different treatments and medical procedures. Varietiesof the robotic applications and solutions in medicine have been presented overthe past several years. In this paper we have summarized different techniquesfor adequate modeling and control for the medical robots in the contact with anenvironment. Contact surface could be either a patient or a device. Here, we haveintroduced a general theoretic overview and suggested a specific approach to thesolution of the insertion problem. For many applications it is enough to considercontact force as a disturbance to the system. Sometimes stochastic character andunexpected range of the contact forces could significantly change or damage contactsurface which could be unacceptable for some medical treatments. Furthermore,contact force which has an unknown value and characteristics could produce acompromised medical outcome. Due to the reasons given here, there is a neednot only to measure the force, but to control it and to obtain adequate controlalgorithms which can keep the force within acceptable limits. Contact force, whichcan occur due to physical constraints to the robotic systems, is able to significantlyinfluence system dynamics. The mathematical model of the system described hereis usually a singular system of differential equations. Force control for singularsystems is a challenging task. Approach to the force control demands specificmathematical preparations before force feedback is applied. In the robot working

Received by the editors August 19, 2009 and, in revised form, November 27, 2009.128

CONTACT PROBLEM AND CONTROLLABILITY FOR SINGULAR SYSTEMS 129

regime there is a problem to obtain qualitative control signal when contact forceacts upon the system. The control algorithm is different in the second phase ofthe working regime, because contact force does not act upon the system. For thatphase it is possible to apply the classic control theory. To model the robotic systemwith the tasks described above, the entire system can be decomposed to the roboticsubsystem and the environment subsystem. Modeling of the system by the methodmentioned has been proved to be suitable when the force appears as a result ofthe interaction of the two subsystems. The mathematical model of the systemhas a singular characteristic. The singular system theory could be applied to thecase described. For the second phase in which there is no interaction, the dynamicbehavior can be analyzed by the classic theory.

2. Chronological literature review

McClamroch [16] was one of the first authors who used the singular system ofdifferential equations for mathematical modeling of the robotic system with thecontact problems.The author suggested several classes of the robotic systems inwhich the singular system approach was suitable to be applied. Hogan [6] usedthe previous technique to investigate the stability of the robots with contact tasks.That article has brought unique mathematical formulations of the external and in-ternal boundedness that led us to solve singular differential equations. Huang andMcClamroch [8] defined optimal control for robots tracking predefined contours.This model included algebraic equations and together with differential equationsformed the singular system of differential equations. Contact force was regulatedby the optimal control algorithm. Hui and Goldenberg [9] presented the singularmathematical model of the manipulator in contact with rigid environments. Theydefined conditions for the feedback design. McClamroch and Wang, [18] solved sta-bilization and tracking problems for constrained robotic systems. They developedfeedback by introducing a generalized error and generalized velocity. Furthermore,they investigated stability conditions for such a system. By using the describedmethod it was possible to determine the effects of the contact force to the systemdynamics. After publishing an article [19] on the similar topic, Mills joined thegroup of the authors who had done respectable work in the field of constrainedrobotics. After that he published several related articles with significant results. Inthe article [19] he described the dynamics of constrained robots as well as the hy-brid control. Carelli and Kelly [2] introduced the adaptive controller for constrainedrobots described by the singular system of differential equations. The adaptive con-trol law was based on actuator torque calculations. McClamroch [15] applied thewell-known singular perturbation technique to the constrained mechanical system.Using appropriate assumptions, he concluded that dynamical behavior of the sys-tem depends on the small system parameters, such as system dumping coefficientor system stiffness. Mills and Goldenberg [27] continued the work on force andposition control for manipulators with contact tasks. In that article the authorsdescribed a unique method for feedback parameter calculations. The solution wasapplied to the tracking problem. Their system was decomposed to the slow andfast subsystems. Furthermore, influence on the singularities to the new perturbedsystem was investigated. One year later, Krishnan and McClamroch [12] presenteda new approach to the contact force regulation. The method was applied to themathematical model which was linearized in the surroundings of the contact point.Mills [21] investigated the robotic system which transited from the nominal regimeto the contact regime. For that specific working cycle the system stability was

130 I.M. BUZUROVIC AND D.LJ. DEBELJKOVIC

analyzed with special emphasis on the transition moments. In the article [26] Millsintroduced the investigation on the singular perturbed system which was extendedto the force feedback development. A force sensor was installed to detect the torqueat the joint on which contact force reacts. The dynamical behavior was investigatedand it was analytically shown that high frequency dynamic could be neglected dur-ing the system control. In the article [24] by Mills and Lokhorst the series ofexperimental results with different control laws was presented. Their experimentswere undertaken using two-degree-of-freedom robot. Mills and Lui [26] continuedthe investigation of robotic systems being in contact with environments. They ap-plied dumping control for generalized contact forces and positions. The influenceof the friction was neglected. Singular perturbation method was applied during themathematical modeling. Goddard et.al [4] solved sliding and rolling problems of theend-effector. The rigid contact surface was fixed. This method was applied to in-dustrial robots for insertion and parts assembly. Mills [23] investigated the stabilityfor robotic systems with elastic joint in the constrained environment. He introducedconditions for the asymptotic stability. Krishnan and McClamroch in the articles[11] and [10] presented a nonlinear mathematical model of the constrained robots.The mathematical model was singularly perturbed. Consequently, the system wastransformed to slow and fast subsystems. Control design was performed using acomposite approach. Mills and Lokhorst [25] proposed a control methodology thataddresses the problem of the control of robotic manipulators during a general classof tasks that requires the manipulator to make a transition from noncontact motionto contact motion and contact motion to noncontact motion. During noncontactmotion, a control suitable for the noncontact phase of motion is applied; duringcontact, another control, suitable for contact motion, is applied. These differentcontrol schemes are applied to the manipulator in such a way that the overall con-trol is discontinuous in nature, [25]. In the article [34] Wapenhas et.al presenteda complete design procedure for determining an optimal force control for assemblytasks while assuring stability. Based on a constrained motion model of the robot,including elasticity of joints and force sensors, a custom design scheme was appliedfor individual types of mating tasks. Hu and Davison in the article [7] studied theproblem of position tracking and contact force control for a constrained manipula-tor, the end-effector of which is required to move along a preassigned trajectory witha specified contact force. The basic procedure for controller design was divided intotwo steps: first, the nonlinear descriptor system is linearized into a linear system;then various control schemes for different cases are proposed based on the resultantlinear model. Mills [22] analyzed the problem of control of generalized contact forceswith a manipulator controller that has traditionally been regarded as a noncontacttask trajectory controller. Experimental results of a two-degree-of-freedom directdrive manipulator during contact with a one-degree-of-freedom linear mechanicalimpedance illustrated the usefulness of the proposed method. Chan et.al [3] de-veloped the impedance control scheme for robot manipulators performing assemblytasks which required interactions with the environment. Lui and Goldenberg in thearticle [14] presented a new robust control approach for robot manipulators based ona decomposition of model uncertainty. Parameterized uncertainty is distinguishedfrom unparameterized uncertainty. Prokop and Pfeiffer [29] suggested a methodfor planning of robotic assembly by numerical optimization of position and jointcontroller coefficients. Together with constraints ensuring practical applicability anonlinear vector optimization problem is stated for a peg-inhole insertion task andthat problem was solved. Stokic and Vukobratovic [30] solved the problem of the


Figure 1. Model of the constrained robotic system: a) fixed base,b) manipulator c) contact surface, T - contact point, f - contactforce.

practical stabilization of robots being in contact with a dynamic environment. Thegoal of another article [31] of Vukobratovic et.al was to shed light on the controlproblem of constrained robot motion from the aspect of the dynamical nature ofthe environment with which the robot was in contact. In the [32] Vukobratovic et.alpresented the current state of the art in the adaptive control of single rigid roboticmanipulators in the constrained motion tasks. A complete mathematical model ofa single rigid robotic manipulator in contact with dynamic environment was pre-sented. The same author in the [33] investigated the problem of impendence controlusing a unified approach to contact tasks control in robotics, where the interpre-tation of the contact between the robot and its known environment were based ona sufficiently correct treating of interactions between the environment and robotdynamics. McClamroch et.al [17] analyzed control problems for a specific mechan-ical system consisting of a rigid base body with an unactuated internal degree offreedom. The key assumptions were that the translational and rotational motionsof the base body can be completely controlled by external forces and moments,while the internal degree of freedom is unactuated. The presented idea can ap-ply to the system in the contact task. Park and Kim [28] presented a differentialgeometric analysis of manipulability for holonomic multiple robot systems contain-ing active and passive joints. The system was described using singular differentialequations. The authors used a geometric approach. Kuzmina in the [13] workedon the analysis methods in complex system dynamics. The generalized approach,based on Lyapunov’s methods, is considered unified for mechanical systems. Theprevious two references presented general approaches to elaborate the technologyof modeling in mechanics using singular systems theory. Ho et.al [5] presented themodel for the constrained robot dynamics, incorporating constraint uncertainties.The rigid constraining surfaces were represented by a set of the algebraic functions.

3. Singular systems and mathematical modeling

As suggested above, the most accurate mathematical model for constrainedrobots should include dynamics of the system due to interaction between the robotand the surface. General guidelines for mathematical modeling together with thebasic equations are presented. The model of the manipulator with its constraintsis shown in Figure 1. Generally speaking, open kinematic chain with n joints isanalyzed.


The generalized coordinates vector is denoted by q, q∈<n , the contact forcevector is denoted by f. Force f ∈<nappears when end-effector touches constrainsurface c. The differential equation which describes the influence on the contactforce to the system is

(1) M(q)q +G(q, q) = τ + JT (q)f.

M (q)∈<nxn denotes inertia matrix function and G(q)∈<n is vector functionwhich describes Coriolis, centrifugal and gravitational effects. τ is torque vectorof the joints, τ∈<n . J (q) ∈ <nxn is defined as Jacobian matrix function. If theposition vector of the contact point in fixed coordinate system is denoted by p∈<m ,than the algebraic equation for contact surface is written as

(2) φ(p) = 0,

in which φ: <m→<1 is scalar function. For further mathematical modeling it isnecessary to introduce three assumptions.Assumption 1 : Friction between end-effector and surface at point T could be ne-glected.Assumption 2 : First derivative of the function φ(p) is a smooth function.Assumption 3 : Matrix M is symmetric and positive defined.Having in mind Assumptions 1 -3, contact force vector can be presented as [16]

(3) f = DT (p)λ,

where λ is scalar multiplier for constrained function, and D(p) is constrained func-tion gradient. Expression for D(p) is

(4) D(p) =∂φ(p)

∂p.

Establishing direct kinematic relationship between end-effector and point T on thecontact surface, using generalized coordinates, one can write p=H (q), where H (q)is vector function. In that case, Jacobian is of the following form

(5) J(q) =∂H(p)

∂q.

As a result of combination of the equations (3)-(5) with (1) general dynamic equa-tions for the robotic system in contact with environment is obtained [16],

(6)

[M(q) 0

0 0

] [q

λ

]=

[−G(q, q) + τ + JT (q)DT (H(q))λ

φ(H(q))

].

Equation (6) consisted of the n differential equations and one algebraic equationwith n+1 unknown values, n generalized coordinates and scalar multiplier λ.Based on the structure of equation (6) the system is represented as a singularsystem of differential equations. The physical explanation is the following: themathematical model includes contact surface due to its influence on the systemdynamics


4. Linearization procedure and matrix representation

Using transformation given at [16] it is possible to transform equation (6) intothe form shown below:

(7) I 0 00 M(q) 00 0 0

qq

λ

=

0 I 00 0 JTDT

0 0 0

qqλ

+

0−H(q, q)−G(q)

φ(p)

+

0I0

τIn the following discussion we assume that force acts upon the constrained surfaceat a certain point where the position, velocity and acceleration amplitudes do notchange significantly in the surroundings of that point. The point is called nominaland it is denoted by

(8) (q0, q0, λ0)T ,

where q0 is a generalized coordinate in nominal system configuration. First deriva-tion of the q0 is nominal joint velocity, and λ0 is a nominal value of the multiplier.For this case, linearization problem is to represent equation (7) around nominalstate (8). At the nominal point (8) the robotic system is at rest, so both nominalvelocity and nominal acceleration are equal to zero. Expanding equation (7) in thesurroundings of the nominal point, using Taylor series it can be described with the[27]

(9) δτ = M(q0)δq + ∂/∂q(G− JTDTλ)|0δq − JTDT |0δλ,In which variances are defined as

(10) δq = q − q0, δq = q − q0, δq = q − q0, δτ = τ − τ0, δλ = λ− λ0.

Combining equations (7)-(10) the linearized equation for the system, described by(1), can be written as

(11) I 0 00 M(q0) 00 0 0

δqδq

δλ

=

0 I 0∂∂q (G− JTDTλ)|0 0 JTDT |0

DJ |0 0 0

δqδqδλ

+

0−H(q, q)−G(q)

φ(p)

+

0I0

δτ.

All coefficients of the system (11) have constant values and they are not timedependent. To calculate nominal values of the generalized torques τ0, it is necessaryto calculate τ from equation (7) for nominal point (8). Result is as follows inequation (12).

(12) τ0 = G(q0)− JT (q0)DT (L(q0))λ0

Analyzing characteristics of the contact force, it can be concluded that end-effectoracts upon the surface and surface reacts at the point T. Former force is called activeforce and latter is denoted as the passive force. During working cycle it is importantto control passive force because that can damage the surface. For surgery robots,


reactive force could be responsible for tissue deformation as well as unnecessarydamage of the healthy organs. Passive force could be expressed as

(13) DT (L(q0))λ0 = f.

Combining equations (12) and (13) nominal value for generalized torque is

(14) τ0 = G(q0)− JT (q0)f.

Multiplier λ0 could be calculated from equation (12), using (14)

(15) λ0 = (DDT )−1Df,

Linearized equation (11) together with (12)-(15) represents equation of motion forthe robotic system in contact with working environment. Described equations arecomplements of the differential and algebraic equations and consequently, theyrepresent a singular mathematical model of the system described. The obtainedresults could be summarized with the following theorem.Theorem 1 : The special class of the nonlinear singular system described by dif-ferential equation (7) is equivalent to its own transformed system (11), linearizedin the surroundings of the nominal point (8), if and only if there are equations (13)and (14) and multiplier λ0 defined by equation (15), for which (11) is fulfilled onconstrained domain ∆.Proof : Having equation (9) and the condition (16) in mind

(16) (∂/∂q)H(q, q)|0 = 0,

together with equations for contact surface (2) and geometric characteristics of thesurface

(17)∂φ(p)

∂p

∂p

∂q|0δq = 0,

expressions (13) and (14) can be calculated from (12). Calculating the value formultiplier λ0 at a nominal point (8), it can be concluded that (11) represents thelinearized equation of the system (7). From that fact it can be concluded that bothequations (7) and (11) are equivalent, q.e.d.♦To determine the appropriate control law for the special class of the system de-scribed, it is of the primary interest to represent the system (11) in the state spaceform.Analyzing equation (14) it can be assumed that nominal torques could change theirvalues in the surroundings of the nominal point (8)

(18) τ0 = τ0 + ∆τ.

Dynamic torque ∆τ represents gravitational influence on the system during theworking regime. The value ∆τ shows changes of the other dynamic parameters andconsequently, in the system dynamics. State space vector can be adopted as

(19) x =

δqδqδλ

.


Using (18) and (19), equation (11) can be transformed as follows

(20)

I 0 00 M(q0) 00 0 0

δqδq

δλ

= 0 I 0∂∂q (G− JTDTλ)|0 0 JTDT |0

DJ |0 0 0

δqδqδλ

+

0I0

δτ +

0∆τ0

.

Now it is possible to represent the robotic system (7) which is in contact withworking environment in its state space form (21) with vector d as a disturbance

(21) Ex(t) = Ax(t) +Bu(t) + d,

where corresponding matrices are defined as

(22)

E =

I 0 00 M(q0) 00 0 0

, A =

0 I 0∂∂q (G− JTDTλ)|0 0 JTDT |0

DJ |0 0 0

B =

0I0

u = δτ d =

0∆τ0

.

4.1. Geometric conditions for system transformation. Equation (22) canbe transformed to the nonsingular and singular parts. The following equation is aresult of the described transformation

(23)x(t) = A′x(t) +B′u(t) =

[0 I

−M−1K −M−1D

]x+

[0

M−1L

]u

C ′x(t) =[P 0

]x(t) ≥ d′

,

where Py ≥ d ’ is the surface equation in the state space, K, L and P are corre-sponding matrices. The compound equivalent equation is of the following form

(24)x = A′x+B′u0 ≤ C ′x+ d

.

To analyze geometric conditions for system transformation it is necessary to connectequations (24) and (21). Result is

(25) E =

[I 00 0

], A =

[A′ 0C ′ 0

], B =

[B′ 00 0

], D =

[0d

].

For system subspace analysis we use a geometric approach to show whether thedescribed transformation exists.Theorem 2 : System (1), with equation of the constrained surface (2) is trans-formable to system (21) if and only if conditions (26) are fulfilled

(26) C ′B′ = 0, ℵ(B′) ⊆ <(C ′),

where ℵ(.) and <(.) denote null space and range of the corresponding matrices,respectively.


Proof : To prove conditions (26) it is sufficient to prove equivalency of each condi-tion itself. From the structure of the matrices C ’ and B ’ it can be concluded thattheir product is equal to zero. From the geometric characteristics of the orthogonalinvariant subspaces it can be stated that condition ℵ(B ’) ⊆ <(C ’) is fulfilled. Toconfirm the statement, null space for the matrix B ’ is calculated and the resultis ℵ(B ’) = {0}. The range of the matrix C ’ contains zero-vector, <(C ’) ⊃ {0}.Consequently, one can conclude that geometric condition (26) for transformationof the robotic system (1), with equation of the contained surface (2) is fulfilled ifand only if transformation is possible, q.e.d.♦.

5. Geometric structure and controllability

Here we introduce state space feedback described by

(27) u = Kx(t),

where K is the matrix. Applying feedback (27) to the system characterized bymatrices (25), controlled system is obtained as follows

(28)Ex = (A+BK)xy = (C +DK)x

.

Definition 1: System (28) is controllable if the matrix pencil

(29) C(s) =[sE −A B

]does not have definite or indefinite zeros.Theorem 3 : Systems characterized by matrices (25) are controllable if and only ifnone of the matrix eigenvalues C (s) is equal to zero. That condition is representedas

(30) C(s) =

sI −I 0∂∂q (G− JTDTλ)|0 sM(q0) −JTDT |0

−DJ |0 0 0

0I0

.Proof : Equation (22) represents structure of the robotic system. Combining equa-tions (22), (25) and (29) it can be concluded that condition (30) is fulfilled, q.e.d.♦.Definition 2: Systems characterized by matrices (25) are reachable if (29) isfulfilled and if

(31) rang[E B

]= n.

Theorem 3 : Systems characterized by matrices (25) are reachable if and only ifcondition (32) is fulfilled:

(32) rang

I 0 0 00 M(q0) 0 I0 0 0 0

= n

Proof : The procedure is similar to the proof of the Theorem 3, using the conditionof Definition 2, ♦.Corollary 1 : Analyzing condition (32), it can be concluded that controllability ofthe linearized robotic system (21) in contact with environments depends on inertiamatrix in the surroundings of the contact point.


This conclusion is significant because the design of the system with contact taskscan influence the controllability and consequently the system stability. Condition(32) could be used for potential stability checking during the system design.Definition 3: Systems characterized by matrices (25) are infinite controllable if(29) does not have indefinite zeros.In the following part, eigenstructure and eigenvalues assignments for the roboticsystem are analyzed. State space feedback was used and geometrical approach wasapplied to solve controllability problem.Theorem 4 : Assume that system (21) is controllable by eigenvalues assignmentprocedure. Let the {σi} (i∈1, 2. . . h, h = rang E ) be a symmetric set of the definitecomplex numbers. Assume that subspace = = span {vi}, i∈1, 2. . . h exists, so thefollowing conditions are true

(1) vi ∈ ℘ if σi ∈ <n and vi= vi* is complex conjugate if and only if σi =σi*.(2) vectors {vi} are linearly independent and vi ∈℘=(σiE – A)-1B= [ 0 0

–(1/JTDT )In|0]T

(3) = ∩ I 3 = 0.

If the conditions (i)-(iii) are fulfilled, real matrix F exists, so the matrix pair(sE -A-BF ) is regular, (A+BF ) vi = σi E vi.Proof : Matrix Pσ = [σE -A B ] can be associated with complex number σ . Qσis a compatible matrix which columns span null space of the Pσ , ℵ( Pσ ). Qσ isdefined as

(33) Qσ =

[NσRσ

].

From the structure of the matrices (22) it can be concluded that rang B = m, whichimplies that columns of the Nσ are linearly independent.From the statement vi ∈ ℘i = ℵ( Nσ ), it can be stated that vi = NσiKi, for someKidefined as

(34) (σiE −A)Nσ iKi +BRλ iKi = 0.

F0 is defined as the transformation of subspaces, F0 : = → Ψ as it is in (35)

(35) F0vi = −RσiKi, i ∈ 1, 2, ..., n.

In order to achieve regularity of the matrix pencil (sE - A- BF ) it is necessaryto define the extension of F to F0. Dimension of the subspace = is dim = = h,therefore dim ℵ(E ) = n - h and = ∩ ℵ(E ) = 0. Consequently,

(36) =⊕ ℵ(E) = ℘.

It can also be concluded that equation (37) is fulfilled,

(37) E= = E(=⊕ ℵ(E)) = E℘ = <(E).

Assume that x∈℘ and condition (36) is fulfilled. Subspace transformation F ′:℘→Ψis defined as the extension on the F 0. Ex and (A+BF ′) is represented by decom-position (38)

(38) E=⊕ ℘∗ = ℘,


where ℘* is any subspace of the dimension n-h which supplements <(E ) to the ℘.In the decompositions (36) and (38) matrices E and (A+BF ′) can be representedas

(39) E =

[I 00 0

], A+BF ′ =

[A11 A12

0 A22

].

Statement (39) can be confirmed with equation (22), because dim I = h and eigen-values of the A11 are {σi}, i∈1, 2. . . h. Matrix pencil (sE - A- BF ′) is regular ifand only if A22 is a nonsingular matrix.Assume that Qi is projection on the ℘* along ℵ(E ). Nonsingular character of thematrix A22 is equivalent to the nonsingular value of the expression (40)

(40) Qi(A+BF ′)|ℵ(E).

Furthermore, it can be written

(41) Qi(A+BF ′)|ℵ(E) = QiAi|ℵ(E) +QiBF i,

where F1= F |ℵ(E ). Matrix F1: ℵ(E ) → Ψ exists, therefore expression (40) isnonsingular if and only if all eigenvalues of the Qi Ai |ℵ(E ) are controllable withQi B,

(42) Qi(A|ℵ(E) +B) = ℘∗.

The system (21) is controllable; therefore, it is possible to define F1. By doing so,expression (40) becomes nonsingular. Now, F : ℘ → Ψ is defined as

(43) F |= = F0, F |ℵ(E) = F1.

Defining real transformation represented by matrix F, it is posible to adjust eigen-values of the system (21). In that case, system (21) became controllable, q.e.d.♦.Comment : Theorem 4 gives possibility for derivation of the opposite statement. Ifthe {σi}, i∈1, 2. . . h is the set of the finite eigenvalues, it is not necessary for (sE- A- BF ) to be regular. In that case

(44) (A+BF )vi = σiEvi,

for some vectors vi, i∈1, 2. . . l. It can be proved that vectors vi, i∈1, 2. . . l fulfillconditions (i)-(iii) for robotic system described by equation (21). Also, it can beproved that subspace = from the statement in Theorem 4 exists for any symmetricset of the complex numbers {σi}, i∈1, 2. . . h. Due to that fact, system (21) iscontrollable in both finite and infinite eigenvalues. In that case, matrix F exists,so the (sE - A- BF ) is a regular expression. Furthermore, infinite eigenvalues ofthe (sE - A- BF ) are {σi}, i∈1, 2. . . h. That implies existence of the vi ∈℘, i∈1,2. . . h, so it can be concluded that

(45) span{vi}⋂ℵ(E) = 0.

The possible method to define subspace = is to define linearly independent set ofthe vectors vi ∈=, but vi /∈ ℵ(E ). In that case = = span {vi} and =∩ℵ(E )=0. From


the practical point of view, this means that it is possible to adjust eigenvalues andeigenvectors to the system (21) using feedback defined by matrix F.

6. Conclusion

In this article general method for mathematical modeling of constrained robotshas been presented. The rigid constrained surface is represented by a set of thealgebraic functions. Therefore, the mathematical model of the robotic system is thesingular system of differential equations. Nonlinear system is transformed to thestate space system. Linearization was performed in the surroundings of the contactpoint. A geometric approach was used for both the controllability analysis andthe eigenstructure assignment. System transformation conditions are presented.It can be concluded that system transformation is possible only if the range ofthe transformed matrix C contains the null space of the transformed matrix B.Robotic system was controllable only if the matrix C eigenvalues were not zeros.Controllability of the linearized robotic system depends on inertia matrix in thesurroundings of the contact point. Structure of the space feedback matrix wasanalyzed. Conditions for system stabilization using space feedback were derived.

7. Appendix

In Appendix, algorithm for the null space and range calculation related to thecorresponding matrices are presented. Algorithm is used in the proof of Theorem 4.More detailed calculations of the different matrix subspaces and algorithms appliedin the geometric theory can be found in [1].

–Direct sumfunction A = dirsum(A,varargin)for k=1:length(varargin)(n,m) = size(A);(o,p) = size(varargink);A = [A zeros(n,p); zeros(o,m) varargink];end

–Null subspacefunction Q = ker(A)comment : Q=ker(A) is an orthonormal basis for kerA.Q = ortco(A’);

function Q = ortco(A)comment : ORTCO Complementary orthogonalization.comment : Q=ortco(A) is an orthonormal basis for the orthogonal complement ofimA.(ma,na) = size(A);if norm(A,’fro’)¡0.0001, Q = eye(ma); return, end(ma,na) = size(ima(A,1));RR = ima([A/norm(A,’fro’),eye(ma)],0);Q = RR(:,na+1:ma);if isempty(Q)Q = zeros(ma,1);end


–Sum of the subspacesfunction Q = sums(A,B)comment : Q = sums(A,B) is an orthonormal basis for subspace im[A B] = imA +imB.Q = ima([A B],0);

References

[1] Basile G., Marro G., (1992) ’Controlled and Conditioned Invariants in Linear System Theory’,

Prentice Hall.Available: http://www3.deis.unibo.it/Staff/FullProf/GiovanniMarro/geometric.htm

[2] Carelli R., Kelly R., (1989) ’Adaptive Control of Constrained Robots Modeled by SingularSystems’, Proc. of the 28th IEEE Conf. on Decision and Control, Tampa, Florida, USA, pp.

2635-2640.

[3] Chan S. P. and Liaw H. C., (1995) ’Impedance Control Strategy for Robotic Assembly Tasks’,Proc. of the 1995 IEEE IECON 21st International Conf. on Industrial Electronics, Control

and Instrumentation, Vol.2, pp. 1372-1377.

[4] Goddard R.E., Zheng Y.F., Hamami H., (1992) ’Dynamic Hybrid Velocity/Force Control ofRobot Compliant Motion over Globally Unknown Objects’, IEEE Transactions on Robotics

and Automation, Vol. 8, No. 1, pp. 132-138.

[5] Ho Y.K., Wang D., Soh Y.C., (2000) ’Modeling of Constrained Robot System with ConstraintUncertainties’, Journal of Robotic Systems, 17(1), by John Wiley and Sons Inc., pp. 53-61.

[6] Hogan N., (1988) ’On the Stability of Manipulators Performing Contact Tasks’, IEEE Journal

of Robotics and Automation, Vol.4, No. 6, pp. 677-686.[7] Hu Y.Z. and Davison E.J., (1994) ’Position Tracking and Force Control of Robotic Manipu-

lators with a Descriptor System Model - Part II: The Exact Control and a Control Strategybased on LQ Method’, Proc. of the 33rd IEEE Conf. on Decision and Control, Buena Vista,

Florida, USA, pp. 2394-2399.

[8] Huang H.P., McClamroch N.H., (1987) ’Time-Optimal Control for a Robotic Contour Fol-lowing Problem’, IEEE Journal of Robotics and Automation, Vol.4, No. 2, pp. 1140-1145.

[9] Hui R. C., Goldenberg A.A., (1988) ’Modeling the Manipulation of Rigid Objects as a Singular

Systems’, Proc. of the 1988 IEEE SMC International Conf. on Systems, Man and Cybernetics,Vol.1, pp. 240-244.

[10] Krishnan H. and McClamroch N.H., (1994) ’On the Connection Between Nonlinear

Differential-Algebraic Equations and Singularly Perturbed Control Systems in NonstandardForm’, IEEE Transaction on Automatic Control, Vol. 39, No. 5, pp. 1079-1084.

[11] Krishnan H. and McClamroch N.H., (1993) ’On the Connection Between Nonlinear Differen-

tial Algebraic Equations and Singularly Perturbed Control Systems in Nonstandard Form’,Proc. of the 32nd IEEE Conf. on Decision and Control, San Antonio, Texas, USA, pp. 889-

894.[12] Krishnan H. and McClamroch N.H., (1990) ’A New Approach to Position and Contact Force

Regulation in Constrained Robot Systems’, Proc. of the 1990 IEEE International Conf. on

Robotics and Automation, Cincinnati, OH, USA, pp. 1344-1349.[13] Kuzmina L.K., (1999) ’Stability Theory Methods and Mechanics Singular Systems’, Proc. of

the 1999 IEEE SMC International Conf. on Systems, Man and Cybernetics, Vol.1, pp. 1-62- 1-67.

[14] Liu G. and Goldenberg A.A., (1996) ’Uncertainty Decomposition-Based Control of Robot

Manipulators’, IEEE Transactions on Control Systems Technology, Vol. 4, No. 4, pp. 384-

393.[15] McClamroch N.H., (1989) ’A Singular Perturbation Approach to Modeling and Control of

Manipulators Constrained by a Stiff Environment’, Proc. of the 28th IEEE Conf. on Decisionand Control, Tampa, Florida, USA, pp. 2407-2411.

[16] McClamroch N.H., (1986) ’Singular Systems of Differential Equations as Dynamic Models for

Constrained Robot Systems’, Proc. of the 1986 IEEE International Conf. on Robotics and

Automations, Vol. 3, pp. 21-28.[17] McClamroch N.H., Rui C., Kolmanovsky I., Cho S., (1998) ’Control Problems for Planar

Motion of a Rigid Body with an Unactuated Internal Degree of Freedom’, Proc. of the IEEEAmerican Control Conference, Philadelphia, Pennsylvania, USA, pp. 229-234.


[18] McClamroch N.H., Wang D., (1988) ’Feedback Stabilization and Tracking of Constrained

Robots’, IEEE Transaction on Automatic Control, Vol. 33, No. 5, pp. 419-426.

[19] Mills J.K., (1988) ’Constrained Manipulator Dynamic Models and Hybrid Control’, Proc. ofthe IEEE International Symposium on Intelligent Control, Vol. I, Arlington, VA, USA, pp.

424-429.

[20] Mills J.K., (1991) ’Dynamic of Robotic Manipulator with Wrist-Mounted Force-Torque Sen-sor: A Singular Perturbation Approach’, IEEE Transaction on Robotics and Automation,

Vol. 7, No. 6, pp. 870-876.[21] Mills J.K., (1990) ’Manipulator Transition to and from Contact Task: A Discontinuous Con-

trol Approach’, Proc. of the 1990 IEEE International Conf. on Robotics and Automations,

Vol. I, pp. 440-446.[22] Mills J.K., (1994) ’Manipulator Control of Generalized Contact Force and Position’, IEEE

Transaction on Systems, Man. and Cybernetics, Vol. 24, No. 3, pp. 523-530.

[23] Mills J.K., (1992) ’Stability and Control of Elastic-Joint Robotic Manipulators duringConstrained-Motion Task’, IEEE Transaction on Robotics and Automation, Vol. 8, No. 1,

pp. 119-126.

[24] Mills J.K. and Lokhorst D.M., (1991) ’Experimental Results in Manipulator Contact TaskControl’, Proc. of the 1991 IEEE International Conference on Robotics and Automation,

Sacramento, California, USA, pp. 1645-1651.

[25] Mills J.K. and Lokhorst D.M., (1993) ’Stability and Control of Robotic Manipulators duringContact/Noncontact Task Transition’, IEEE Transaction on Robotics and Automation, Vol.

9, No. 3, pp. 335-345.[26] Mills J.K. and Lui G.L., (1991) ’Robotic Manipulator Impedance Control of Generalized

Contact Force and Position’, IEEE/RSJ International Workshop on Intelligent Robots and

Systems IROS ’91, Osaka, Japan, pp. 1103-1108.[27] Mills J.K., Goldenberg A.A., (1989) ’Force and Position Control of Manipulators during

Constrained Motion Task’, IEEE Transaction on Robotics and Automation, Vol. 5, No. 1,

pp. 30-36.[28] Park F.C. and Kim J.W., (1998) ’Manipulability and Singularity Analysis of Multiple Robot

Systems: A Geometric Approach’, Proc. of the 1998 IEEE International Conf. on Robotics

and Automation, Leuven, Belgium, pp. 1032-1037.[29] Prokop G. and Pfeiffer F., (1996) ’Improved Robotic Assembly by Position and Controller Op-

timization’, Proceedings of the 1996 IEEE International Conf. on Robotics and Automation,

Minneapolis, Minnesota, USA, pp. 2182-2187.[30] Stokic D., Vukobratovic M., (1997) ’An Efficient Method for Analysis of Practical Stability of

Robots Interacting with Dynamic Environment’, Proc. of the 1997 IEEE/RSJ InternationalConf. Intelligent Robots and Systems IROS, Vol. 1, pp. 175-180.

[31] Vukobratovic M., Rodic A. Ekalo Y., (1997) ’Impedance Control as a Particular Case of the

Unified Approach to the Control of Robots Interacting with a Dynamic Known Environment’,Journal of Intelligent and Robotic Systems Vol. 18, pp. 191-204.

[32] Vukobratovic M. Tuneski A., (1996) ’Adaptive Control of Single Rigid Robotic Manipulators

Interacting with Dynamic Environment- An Overview’, Journal of Intelligent and RoboticSystems Vol. 17, pp. 1-30.

[33] Vukobratovic M., Wang H.O., (1997) ’How to Control Robots Interacting with DynamicEnvironment’, Journal of Intelligent and Robotic Systems Vol. 19, pp. 119-152.

[34] Wapenhans H., Seyfferth W., Pfeiffer F., (1993 ) ’Robotic Force Control for Flexible Assem-bly’, Proc. of the 1993 IEEE International Conf. on Robotics and Automations, Vol. I, pp.

225-231.

1Department of Radiation Oncology, (NCI-designated), Division of Medical Physics, Thomas

Jefferson University, Philadelphia, PA 19107, USAE-mail : [email protected]

2Department of Automatic Control, School of Mechanical Engineering, University of Belgrade,

Belgrade, SerbiaE-mail : [email protected]

INTERNATIONAL JOURNAL OF c⃝ 2010 Institute for ScientificINFORMATON AND SYSTEMS SCIENCES Computing and InformationVolume 6, Number 2, Pages 142–154

A DOUBLE MOVER-STAYER MODEL FOR CREDIT RATINGS

ERIC S. FUNG AND TAK KUEN SIU

Abstract. Modeling the dynamics of credit ratings plays an important role in

credit risk management and portfolio risk management. Both institutional and

individual investors make use of credit ratings produced by some well-known

international credit ratings agencies for analyzing the financial wealth of firms

and other jurisdictions. In this paper, we develop a model for stochastic move-

ments of ratings of a population of credit entities based on a double mover-

stayer (DMS) model with three independent discrete-time and discrete-state

models. This provides a more flexible and general paradigm to incorporate the

heterogeneity of rating behavior and to discriminate and classify a population

of credit entities according to their ratings behavior. A two-stage procedure is

adopted to estimate the DMS model. We first employ the maximum likelihood

approach to uncover the dynamics of transitions of each credit rating sequence

and to identify the relationship among ratings sequences in a population of

credit entities. We then develop an efficient adaptive estimation method to

estimate the model parameters. Numerical experiments are conducted to illus-

trate the practical implementation of the model.

Key Words. Double Mover-Stayer Model; Multiple Credit Ratings; Popu-

lation Heterogeneity; Two-State Estimation; Maximum Likelihood; Adaptive

Method.

1. Introduction

Modeling the dynamics of credit ratings is an important step for credit riskmanagement and portfolio risk management. In practice, credit ratings may beproduced by some ratings agencies. Some major international ratings agenciesinclude Standard & Poors, Moodys and Fitch. Some of these ratings are publiclyavailable. Credit ratings may also be produced based on some internal assessmentof the firms. These ratings may not be observed by investors outside the firms.Credit ratings may also refer the “true” underlying rating of a firm, which areunobservable and may be estimated from some observable ratings data, such asratings data produced by agencies. Information about the evolution of credit ratings1 over time provides market practitioners with an important piece of informationfor assessing financial health of firms or other jurisdictions. Indeed, the payoffs ofa number of securities depend on the credit ratings of issuers of these securities, forexample, the coupon payments of corporate bonds may be increased if the ratingsof the firms issuing the bonds or debtors are downgraded to certain threshold levels.

Received by the editors October 7, 2008 and in revised form, October 21, 2008.

2000 Mathematics Subject Classification. 35R35, 49J40, 60G40.This research was supported by BU FRG/07-08/II-43.1In a broad sense, these ratings could be publicly available ratings, internal ratings or the

“true” underlying ratings.

142

A DOUBLE MOVER-STAYER MODEL FOR CREDIT RATINGS 143

So, knowledge about credit ratings may be useful for predicting future payoffs ofsecurities, and hence, it may lead to better investment decisions and outcomes.

Markov chains have emerged as a natural and prominent tool for modeling theevolution of marginal credit ratings over time. They have widely been used byboth academic researchers and market practitioners for modeling the stochasticevolution of credit ratings over time. Markov chains are also the main paradigm formodeling credit ratings data from rating agencies. Transition credit matrices, whichoriginally come from transition probabilities matrices of Markov chain, become theworkhorse for analyzing credit ratings in finance and banking industries. Duffieand Singleton (2003) (see Section 4.5 therein) provide an excellent account of theuse of Markov chains for credit ratings. Credit Metrics (Gupton et al. (1997))provides a very comprehensive account of practical implementation of transitioncredit matrices.

Two key assumptions underlying most of the Markov chain models for credit rat-ings are the homogeneity of rating behavior and the stationarity of rating behavior.However, some evidence show that in practice, various sources of heterogeneityin the rating behavior exist. One important form of heterogeneity is known asthe ageing effect of bond ratings, which basically refers to the phenomenon thatthe likelihood that corporate bonds change ratings or default is lower during theearly years after their issuance than it is for seasoned bonds. Some empirical ev-idence for the ageing effect of bond ratings include Altman (1998), Asquith et al.(1989), Keenan et al. (1999), and others. However, the ageing effect of bond rat-ings cannot be incorporated in most of the credit ratings models, which are basedon time-homogeneous Markov chain. This motivates the quest for an appropriatemodel which is able to incorporate the ageing effect in ratings behavior.

A possible way to incorporate the ageing effect in ratings behavior is to con-sider the discrete-time mover-stayer model pioneered by Blumen et al. (1955).The discrete-time mover-stayer model can be regarded as an extension of a timenonhomogeneous Markov chain. Basically, a mover-stayer model is defined as amixture of two simple independent Markov chains, namely, a stayer and a mover.Movers evolve over time according to a discrete-time Markov chain while stayersremain in their initial state. In the context of credit ratings analysis, a populationof credit entities can be divided into two groups, movers and stayers. Movers arecredit entities which have transitions in their ratings, for example, seasoned bonds,while stayers refer to credit entities which are immobile in their ratings, for exam-ple, newly issued bonds. A mixture of movers and stayers is more flexible thanthe “vanilla” Markov chain model to describe ratings behavior. Frydman (1984)proposes the maximum likelihood method for estimating the discrete-time mover-stayer model. His work is subsequently extended in Frydman et al. (1985) fortesting the adequacy of the discrete-time mover-stayer model for describing creditbehavior. Altman and Kao (1991) adopt the method in Frydman et al. (1985) forinvestigating behavior of ratings migration. Frydman and Kadam (2002) consider acontinuous-time mover-stayer model with a mixture to incorporate the ageing effectin modeling bond ratings migration. A key common feature of these works is thata population of credit entities is separated into two (extreme) groups, stayers andmovers. However, in practice, among those credit entities, some of them are moremobile than the others in terms of rating transitions. For example, entities withdifferent initial rating categories may have different levels of mobility or entitiesfrom different industrial sectors may have different modes of mobility. With thisintuition and practical consideration in mind, it would be beneficial to have a model

144 ERIC S. FUNG AND TAK KUEN SIU

which can provide flexibility to discriminate and to incorporate this difference inthe degree of mobility in rating transitions.

In this paper, we articulate this problem and develop a discrete-time doublemover-stayer (DMS) model for modeling credit ratings of a population of creditentities, say corporate bonds. It is our view that the discrete-time mover-stayer ismore convenient than its continuous-time counterpart from an econometric perspec-tive. So we consider here a discrete-time model. The basic idea of the discrete-timeDMS model is to describe the heterogeneity in rating behavior by a mixture ofone stayer and two independent Markov chains. Stayers represent those immobilecredit entities while the two independent Markov chains or movers are two groupsof mobile credit entities with different levels of mobility, say a high level of mobilityand a low level of mobility. The difference in the level of mobility may be caused bydifferent industrial sectors or some external macro-economic factors. So, the DMSmodel allows the discrimination or classification of a population of credit entitiesin three groups, say immobility, low mobility and high mobility. It is more gen-eral and flexible than the discrete-time mover-stayer model, which is dichromaticin nature. The additional Markov chain in the DMS model can serve as an inter-mediate step to bridge the gap between the two extremes, namely, immobility andhigh mobility. In general, one may consider a finite number, an infinite numberor a continuum 2 of independent Markov chains in the movers’ part. However, tokeep things simple and to illustrate the main idea, we consider the case that wehave two Markov chains in the movers’ part. In practice, the DMS model proposedhere can be applied to discriminate or separate a population of credit entities orbonds according to the past and current ratings into three groups, immobility, lowmobility and high mobility. The DMS model can also incorporate the heterogeneityof credit entities, such as the ageing effect of bonds. Hanson et al. (2007) point outthe importance of incorporating the heterogeneity in rating behavior in evaluatingrisk measures and risk capitals. In particular, they show that the heterogeneityin rating behavior has significant impact on the loss distribution of a risky profileor portfolio, which is used for evaluating risk measures and capitals. So, we canhave a more complete evaluation of risk measures and capitals if the heterogeneityin rating behavior is incorporated. This has important implications for regulatorypolicy making given the priority of many central banks and monetary policies inputting forward the implementation of the three-pillar risk management practice ofBasel II (see, also, Kadam and Lenk (2008), for this viewpoint). We employ a two-stage procedure to estimate the DMS model. At the first stage of the procedure, weadopt the maximum likelihood approach to uncover the dynamics of transitions ofeach rating sequence and to identify the relationship among ratings in a populationof credit entities. We then develop an efficient adaptive method to estimate themodel parameters. Numerical experiments are conducted to illustrate the practicalimplementation of the model.

The rest of the paper is organized as follows. In Section 2, we introduce thediscrete-time double mover-stayer model for credit ratings. Section 3 presents themaximum likelihood method together with adaptive approach for estimating themodel. In Section 4, numerical experiments are conducted to illustrate the practicalimplementation of the model and to identify key features of and insights into the

2The case of a continuum of independent Markov chains provides a theoretical framework to

model the gradual change from immobility to high mobility in an analogy sense, instead of in adigital sense, like the standard mover-stayer model. This may represent an interesting area ofindependent research.


heterogeneity in rating behavior that can be obtained from the model. Finally,concluding remarks are given in Section 5.

2. The Double Mover-Stayer Model for Credit Ratings

In this section, we first provide the background of the mover-stayer model andthen introduce a discrete-time double mover-stayer (DMS) model for ratings ofa population of credit entities. The mover-stayer model is a mixture of two in-dependent discrete-time homogeneous Markov chains. The first Markov chain isdegenerate and stationary. This Markov chain is called a stayer, which, in the con-text of credit ratings, describes the situation that the rating of a credit entity alwaysremains in its initial rating. The second Markov chain is a first-order discrete-timeMarkov chain, which is called a mover. In the context of credit ratings, a moverdescribes the dynamics of the rating of a credit entity, which is mobile in ratingtransitions. The key idea of the mover-stayer model is to provide flexibility in mod-eling the heterogeneity of state transitions, or rating transitions in the context ofcredit risk modeling, of a Markov chain. Essentially, two extreme types of statetransitions, namely, “no move” and “mover”, are incorporated in the mover-stayermodel. So, the nature of the mover-stayer model is rather dichromatic. In thesequel, we first present the mathematical set up of the mover-stayer model. Toadapt to the current context of the paper, we present the mover-stayer model inthe context of credit rating modeling.

Let w denote the number of credit rating categories taken by each entity in apopulation of credit entities or bonds. For each i = 1, 2, . . . , w, let si denote theconditional probability that a credit entity in a population is a stayer given that itsinitial state is i; 1−si is the conditional probability that the credit entity is a movergiven that its initial state is i. To describe the transition behavior of the stayer, wedegenerate a transition probability matrix of the stayer by setting it as the diagonalmatrix S := diag(s1, s2, . . . , sw). This diagonal matrix is called a stayer transitionmatrix in the context of mover-stayer models. In the context of credit risk analysis,the diagonal matrix describes the rating behavior of a stayer or an immobile creditentity. Let M := {mij}i,j=1,2,...,w denote the transition probability matrix of thefirst-order homogeneous Markov chain representing the mover. Suppose {Z(j)|j ≥0} denotes the dynamics of the rating of a credit entity and it follows a mover-stayermodel. Then, the transition probabilities of {Z(j)|j ≥ 0} are given by:

P (Z(j) = i|Z(0) = k) =

{(1− si)m

(j)ik if i = k

si + (1− si)m(j)ii if i = k ,

(1)

where i, k = 1, 2, . . . , w and m(j)ik represents the j step transition probability from

state k to state i.Since the mover-stayer model consists of only one single mover, it may not be

flexible enough to accommodate different levels mobility in rating transitions. How-ever, in practice, credit entities or bonds exhibit different degrees of mobility, whichmay be attributed to the variability of credit quality or industrial/sector effect. So,it is imperative to develop a model which is able to incorporate the heterogeneityof the mobility of movers. The discrete-time DMS model is introduced here for thispurpose.

The DMS model is a mixture of three independent discrete-time homogeneousMarkov chains, one stayer and two movers. We provide the intuition behind theDMS model in the context of credit risk analysis. Firstly, we consider ratings from apopulation of credit entities. The population may contain entities which are stable


with respect to rating transitions, those having low mobility of rating transitionsand those having high mobility of rating transitions. Credit entities with stableratings are regarded as stayers, which remain in their initial ratings. Entities withlow and high mobility in their ratings are viewed as the two movers. Transitions ofratings of these movers are modeled as two independent first-order Markov chains.Here we assume that there is no inter-transition relationship between transitionsof ratings of two entities. Our main focus here is to model the heterogeneity ofa population of credit entities and to classify them in different categories, stayers,low mobility movers and high mobility movers. We do not focus on the modelingof the dependency of credit ratings, which has been studied in some detail in Siuet al. (2005). Indeed, the dynamics of ratings of different entities are independentwith each other and the evolution of ratings of an entity follows either a stayer orone of the two independent movers. The distinctive feature of the proposed DMSmodel is that it depends on a mixture of two independent Markov chains, and thisis different from the standard mover-stayer model in which only a single Markovchain is considered. The mathematical formulation in the DMS model is similarto the mover-stayer model, except that there is two independent movers instead ofone.

Let M := {mij}i,j=1,2,...,w and H := {hij}i,j=1,2,...,w denote the transition prob-ability matrices of two independent first-order Markov chains representing moverswith low mobility and high mobility in rating transitions, respectively.

Now, suppose we have a population of n credit ratings. For each l = 1, 2, . . . , n,let Zl := {Zl(j)|j = 0, 1, . . . , J} denote a sequence of credit ratings of the lth creditentity, where J + 1 is the length of the rating sequence. Under the discrete-timeDMS model, the transition probabilities of {Zl(j)}j=0,1,...,J,l=1,2,...,n are given by:

Prob(Zl(j) = k|Zl(0) = i) =

{ηi(si + (1− si)(vim

(j)ii + (1− vi)h

(j)ii ) if i = k

ηi(1− si)(qzm(j)ki + (1− qz)h

(j)ki ) if i = k

for 1 ≤ i, k ≤ w, where

(i) vi := I{(mii=max{mii,hii})} and IA is the indicator of an event A;(ii) qz ∈ {0, 1};(iii) m

(j)ki represents the j-step transition probability of the mover with transition

probability matrix M from state i to state k;

(iv) h(j)ki represents the j-step transition probability of the mover with transition

probability matrix H from state i to state k.

Under the discrete-time DMS model, the dynamics of rating of an individual creditentity or bond follow one of the three simple Markov chain models in the mixturemodel. It is not unreasonable to assume that the rating of a credit entity, whichremains in its initial rating i, follow either a stayer or a mover having low mobility.This assumption is imposed in the DMS formulation and the value of vi is taken tobe I{(mii=max{mii,hii})}. Note that qz = 1 if the rating sequence follows the Markovchain with transition probability matrix M, which is the mover with low mobility.Whereas, qz = 0 if the rating sequence follows the Markov chain with transitionprobability matrix H.

In the next section, we shall discuss the estimation of the discrete-time DMSmodel.


3. Estimation Methods

The estimation of the discrete-time DMS model is itself an interesting and chal-lenging problem. The hard work here lies on the estimation of matrices S, M and Hwith partial information for the stayer and the two movers only. Indeed, the stayerand the two movers are not directly observable. For example, if an observable rat-ing sequence persistently or systematically remains in its initial rating category, wecannot tell directly and exactly whether this sequence is a stayer or one of the twomovers since a mover has positive probability of staying in its initial state. Wearticulate the estimation problem by proposing a two-stage iterative procedure toestimate the model parameters. The first stage of the procedure is to classify creditentities according to the heterogeneity of their rating behaviors, say different levelsof mobility in rating transitions. Here we require that the rating sequence shouldcontain at least one state, which is different from its initial state. At the secondstage of the procedure, we estimate the model parameters by maximizing likelihoodestimation based on the classification result of ratings obtained from the first stage.The rating sequence can then be re-classified according to the maximum likelihoodestimates of the model parameters. The process is then repeated iteratively untilthe clusters of the rating sequences remain unchanged or the estimates of modelparameters converge.

In the discrete-time DMS model presented in the last section, the number ofmodel parameters is of the order O(w2), where w is the number of rating categories.To begin with the two-stage iterative procedure, we first initialize the class of theheterogeneity rating sequences. Then the transition frequency from state j to statei, denoted by xij , is obtained from the classification result, and the transitionfrequency is used to estimate the transition probability. In the sequel, we describein some detail in the estimation method. First, we need to introduce some notation.The notation was employed in Frydman (1984). Suppose

(1) ni0,...,ij := the number of credit entities with a rating sequence of (i0, . . . , ij);(2) πi0...,iJ := the probability of obtaining a rating sequence (i0, . . . , ij);(3) nik := the total number of transitions from the rating category k to the

rating category i;(4) ηi := the initial distribution at the rating category i;(5) ni(j) := the number of credit entities in the rating category i at time j;(6) ni := the number of credit ratings with a rating sequence of (i0, . . . , ij)

such that i0 = i1 = . . . = ij = i;(7) xik := the total number of transitions from the rating category k to the

rating category i for those rating sequences of credit entities belonging tothe mover with high mobility in rating transitions;

(8) w := the number of possible rating categories(9) J + 1 := the length of historical rating sequences.

Note that under the discrete-time DMS model, the dynamics of the rating se-quences are independent with others and are governed by either one of the inde-pendent Markov processes. Using the notation defined above, we then have

w∑i=1

ηi = 1, and n =w∑i=1

ni(j) for j = 1, 2. . . . ,

where n is the number of credit entities in a population.


The likelihood function L := L(η,S,M,H) is then given by:

L =∏

i0,...,iJ

(πi0...,iJ )ni0,...,ij

=

(w∏i=1

ηni(0)i

)(w∏i=1

(si + (1− si)(vimJii + (1− vi)h

Jii)

ni

)∏i,j

hxik

ik

(

w∏i=1

(1− si)ni(0)−ni

)∏i =k

mnik−xik

ik

w∏i=1

m(nii−Jni−xii)ii

subject to

w∑i=1

ηi = 1,

w∑i=1

mik = 1,

w∑i=1

hik = 1, for k = 1, . . . , w.

where vi = I{(mii=max{mii,hii})}; mik is the transition probability of the Markovchain with low mobility in rating transitions from state k to state i; hik is thetransition probability of the Markov chain with high mobility in rating transitionsfrom state k to state i.

To maximize the likelihood function L, it is more convenient to consider anequivalent problem to maximize the log-likelihood function, namely, log(L), givenas below:

log(L)

=∑

i0,...,iJ

ni0,...,iJ logπi0...,iJ

=

w∑i=1

ni(0)logηi +

w∑i=1

nilog(si + (1− si)(vimJii + (1− vi)h

Jii) +

∑i,k

xikloghik

+w∑i=1

(ni(0)− ni)log(1− si) +∑i =k

(nik − xik)logmik +w∑i=1

(nii − Jni − xii)logmii

By considering the log-likelihood function together with the constraints, we maxi-mize the following Lagrangian function:

L

=∑

i0,...,iJ

ni0,...,iJ logπi0...,iJ

=w∑i=1

ni(0)logηi +w∑i=1

nilog(si + (1− si)(vimJii + (1− vi)h

Jii) +

∑i,k

xikloghik

+

w∑i=1

(ni(0)− ni)log(1− si) +∑i =k

(nik − xik)logmik +

w∑i=1


+λ

(1−

w∑i=1

ηi

)+

w∑k=1

µk

(1−

w∑i=1

mik

)+

w∑k=1

ρk

(1−

w∑i=1

hik

)(2)

where λ, µk and ρk are the Lagrange multipliers.


To begin with, we estimate the initial distribution ηi by evaluating the partialderivatives of L with respect to ηi, since

∂2L

∂(ηi)2= −ni(0)

(ηi)2< 0,

so, the Lagrangian function L is concave in ηi, and there exists a unique minimumpoint of L as a function of ηi.

So, the first-order condition

∂L

∂(ηi)= 0 ,

implies that

ni(0)

ηi− λ = 0 or ληi = ni(0) .

Therefore,w∑i=1

(ληi) = λ =w∑i=1

ni(0), and, hence, the maximum likelihood estimate

ηi of ηi is given by:

ηi =ni(0)∑wi=1 ni(0)

.

We then derive the maximum likelihood estimate si of si in a similar fashion.Firstly, we note that

∂2L

∂(si)2= − ni(0)(1− ei)

2

((1− ei)si + ei)2− ni(0)− ni

(1− si)2< 0 ,

where ei := (vimJii + (1− vi)h

Jii).

Then, the first-order condition

∂L

∂si= 0

implies that

si =ni(0)ei − nini(0)(ei − 1)

.


Substituting this si = si into Equation (2) gives

L

=w∑i=1

ni(0)logηi +w∑i=1

nilog

(ei + (1− ei)

ni(0)ei − nini(0)(ei − 1)

)

+

w∑i=1

(ni(0)− ni)log

(1− ni(0)ei − ni

ni(0)(ei − 1)

)

+∑i,k

xikloghik +∑i =k

(nik − xik)logmik +

w∑i=1


+λ

(1−

w∑i=1

ηi

)+

w∑k=1

µk

(1−

w∑i=1

mik

)+

w∑k=1

ρk

(1−

w∑i=1

hik

)

=

w∑i=1

ni(0)logηi +

w∑i=1

nilog

(nini(0)

)+

w∑i=1

(ni(0)− ni)log

(ni − ni(0)

ni(0)(ei − 1)

)

+∑i,k

xikloghik +∑i =k

(nik − xik)logmik +w∑i=1


+λ

(1−

w∑i=1

ηi

)+∑k=1

µk

(1−

w∑i=1

mik

)+∑k=1

ρk

(1−

w∑i=1

hik

)(3)

where ei = (vimJii + (1− vi)h

Jii).

Based on the likelihood function, L, we can estimate mik for i = k. Since

∂2L

∂(mik)2= − (nik − xik)

(mik)2< 0

the function L is concave in mik. By setting∂L

∂mik= 0, we get

mikµk = (nik − xik) or µk =

∑i=k(nik − xik)

1−mkk

So, the maximum likelihood estimate mik of mik is given by:

mik =(1−mkk)(nik − xik)∑

i =k(nik − xik)for i = k

For simplicity, let a1 = J(ni(0)−ni), a2 = (nii−Jni−xii) and a3 =∑

k =i(nki−xki).We observe that ai ≥ 0 for i = 1, 2, 3. Then, we can substitute the expression formik into Equation (3) and obtain

L =

{a2logmii + a3log(1−mii) + C0 if vi = 0

(ni − ni(0))log(1−mJii) + a2logmii + a3log(1−mii) + C1 if vi = 1

where C1 and C2 are independent of mii.Then, we can estimate mii and find that

∂L

∂mii=

a2mii

− a3(1−mii)

if vi = 0

a1mJ−1ii

1−mJii

+a2mii

− a3(1−mii)

if vi = 1


It is easy to verify that when vi = 0,

∂2L

∂(mii)2= − a2

(mii)2− a3

(1−mii)2< 0 .

By setting ∂L∂(mii)

= 0, an explicit form for the maximum likelihood estimate mii of

mii when vi = 0 is obtained as follows:

mii =(nii − Jni − xii)∑wj=1(nji − xji)− Jni

.

In the case that vi = 1, we may consider ∂L∂(mii)

= 0. This is equivalent to consid-

ering the root of the function, denoted by f(mii), as follows:

f(mii) := (1−mii)(a1m

Jii + a2(1−mJ

ii))− a3mii(1−mJ

ii)

= (a3 + a2 − a1)mJ+1ii + (a1 − a2)m

Jii − (a2 + a3)mii + a2

Since f(0) = a2 > 0, f(1) = 0 and

limx→1−

f ′(x) = Ja3 − a1 > 0 ,(4)

f(mii) has a root in the interval (0,1).If, further, J → ∞, it is not difficult to check that f(mii) has a unique root in

the interval (0,1).

Although there is no explicit form of a solution to ∂L∂(mii)

= 0, we can solve the

equation by employing some numerical methods, say Newton’s method.

Similarly, we can estimate hik for i = k. Since

∂2L

∂(hik)2= − xik

(hik)2< 0 ,

the Lagrangian function L is concave in hik. The first-order condition

∂L

∂hik= 0 ,

then implies that

hik =xik(1− hkk)∑

i =k xik, for i = k .

Substituting the expression for hik into Equation (3) gives:

L =

(ni − ni(0))log(1− hJii) +

∑k =i

xkilog(1− hii) + xiiloghii +D0 if vi = 0∑k =i

xkilog(1− hii) + xiiloghii +D1 if vi = 1

where D0 and D1 are independent of hii.Then, we can estimate hii and find that

∂L

∂hii=

a1h

J−1ii

(1− hJii)−∑

j =i xji

1− hii+xiihii

if vi = 0

−∑

j =i xji

1− hii+xiihii

if vi = 1


It is easy to verify that when vi = 1,

∂2L

∂(hii)2= −

∑j =i xji

(1− hii)2− xii

(hii)2< 0 .

By setting ∂L∂(hii)

= 0, an explicit form of the maximum likelihood estimate hii of

hii when vi = 1 is obtained as follows:

hii =xii∑wj=1 xji

.

In the case when vi = 0, we may consider ∂L∂(hii)

= 0. This is equivalent to consid-

ering the root of the function g(hii) defined by putting

g(hii) := (1− hii)(b1h

Jii + xii(1− hJii)

)− b2hii(1− hJii)

= (xii + b2 − b1)hJ+1ii + (b1 − xii)h

Jii − (xii + b2)hii + xii ,

where b1 = (ni(0)− ni)J and b2 =∑

i =j xji.

Since g(0) = xii > 0, g(1) = 0 and

limx→1−

g′(x) = Jb2 − b1 > 0 .

Hence, g(hii) has a root in the interval (0,1). If J → ∞, it is not difficult to checkthat g(hii) has a unique root in the interval (0,1). Similarly, we can solve the aboveequation by employing some numerical methods, such as Newton’s method.

Based on the above estimation method, we can estimate the model parametersby the following iterative method:

(1) Set k = 1 and initialize the mover’s class of rating sequences which have atleast one state different from the initial state.

(2) Evaluate ni(0), ni and x(k)ij for i, j = {1, 2, . . . , w}.

(3) Estimate η(k)i by setting ηi =

ni(0)∑wi=1 ni(0)

for i = 1, 2, . . . , w.

(4) For each i, set vi = 1 and evaluate mii and hii using the maximum likelihood

method. If mii ≥ hii, the presumption is correct and set m(k)ii = mii and

h(k)ii = hii; otherwise, vi should be set to 0 and evaluate mii and hii in this

case, and set m(k)ii = mii and h

(k)ii = hii, where it is easy to verify that

mii < hii.

(5) Based on the result in (4), we can estimatem(k)ij and h

(k)ij for i, j = 1, 2, . . . , w.

(6) Evaluate the value of s(k)i for i = 1, 2, . . . , w.

(7) Let e =∑w

i=1(|η(k)i − η

(k−1)i | + |s(k)i − s

(k−1)i |) +

∑i,j(|m

(k)ij − m

(k−1)ij | +

|h(k)ij − h(k−1)ij |), if e <threshold, stop, otherwise go to step (8).

(8) Based on the transition matrix of M (k) = {m(k)ij } and H(k) = {h(k)ij } and

using the maximum likelihood estimates, re-classify the mover’s class ofthe rating sequences which have at least one state different from the initialstate. Set k = k + 1 and go to step 2.

4. Numerical Results

We present a numerical experiment to illustrate the practical implementationof the discrete-time DMS model for ratings of a population of credit entities. Inthis experiment, we suppose that there are 30 years historical rating data of creditentities or bonds. Each bond at a fixed time period can take one of five possible


rating categories. To evaluate the performance and effectiveness of the discrete-time double mover-stayer model, let Xl denote the class of the rating of the lth

entity. Xl = 0 if it is a stayer; Xl = 1 if it is a mover with low mobility in ratingtransitions; Xl = 2 if it is a mover with high mobility in rating transitions. If Xl

represents the estimated class of the rating of the lth entity, the classification resultof a rating sequence is then measured by the classification accuracy r defined as:

r :=1

n

n∑j=1

I{Xj=Xj} .

where n is the number of credit entities in a population.Now, we specify the specimen values of the model parameters in the DMS model.

In the numerical experiment here, we treat these hypothetical values of model pa-rameters as if they were the “true” values of the model parameters. The hypothet-ical DMS model with these parameters are then regarded as if it were the “true”underlying model. This model is used for simulating the realizations of ratings ofthe credit entities. These simulated data are supposed to be the observed ratings.They are used to estimate the model parameters.

Since there is no prior information for the level of mobility in rating transitions,we initialize the levels of mobility for each rating sequence and simulate 100 scenar-ios. We have simulated various numbers of ratings of entities, n, and the averageresults are reported in Table 1.

n||M −MTrue||2

||MTrue||2||H −HTrue||2

||HTrue||2||diag(S)− diag(STrue)||2

||diag(STrue)||2r

100 0.126 0.131 0.378 0.90300 0.078 0.081 0.228 0.91500 0.067 0.070 0.180 0.90

Table 1: Various relative error results with J=30

¿From the table, we observe that even in the limit length of the rating sequence(=30), the estimated rating migration matrices M and H are close to their corre-sponding true rating migration matrices. The relative errors of the estimation ofthe two transition probability matrices are less than 14%, 9% and 7% in the caseof n = 100, n = 300 and n = 500, respectively. We also observe that the relativeerror of the estimated stationary rating matrix is high in the case of n = 100. Themain reason is that the stationary probability of each initial state i is relatively low,and we have only 100 observable credit entities with five different states. So, theoccurrence of the observed stationary rating sequences may not reflect correctly thetrue value of the stationary probability. The relative error of the estimated station-ary rating matrix drops significantly when the the number of rating sequences, n,increase. In the view of the average classification result, the average classificationaccuracy is about 90% in each case, and this accuracy is robust to the number ofrating sequences.

5. Conclusion

We proposed a mixture model, namely, the discrete-time double mover-stayermodel to describe rating transitions of a population of credit entities. The proposedmodel can be applied to separate the stayer and mover models and to classifythe credit entities according to their levels of mobility in rating transitions. Wepresented a two-stage iterative method to classify credit entities and to estimatethe model parameters. We illustrated the practical implementation of the model


by conducting a numerical experiment. We found that the proposed model canclassify over 90% of the rating sequences and the estimation result seems robust.

References

[1] Altman, E.I., The Importance and Subtlety of Credit Rating Migration, Journal of Banking

and Finance, 22 (1998), pp. 1231-1247.[2] Altman, E.I. and Kao, D.L., Examining and Modeling Corporate Bond Rating Drift, New

York University Salomon Center Working Paper Series, (1991), s-91-39.[3] Asquith, P., Mullins, D.W., andWolff, E.D., Original Issue High Yield Bonds: Aging Analyses

of Defaults, Exchanges, and Calls, The Journal of Finance. 44 (1989), pp. 923-952.[4] Blumen, I., Kogan, M., and Mccarthy, P.J., The Industrial Mobility of Labor as a Probability

Process, Cornell Studies of Industrial and Labor Relations (Vol. 6), Ithaca, N.Y.: CornellUniversity Press, (1955).

[5] Duffie, D., and K. Singleton, Credit Risk, Princeton, Princeton University Press, (2003).[6] Friedman, Avner, Partial Differential Equations of Parabolic Type, Robert E. Krieger Pub-

lishing Company, Malabar, Florida, (1983).

[7] Frydman, H., Maximum Likelihood Estimation in the Mover-Stayer Model., Journal of theAmerican Statistical Association, 79 (1984), pp. 632-637.

[8] Frydman, H., Kallberg, J.G., and Kao, D.L., Testing the adequacy of Markov chains andmover-stayer models as representations of credit behavior., Operations Research, 33 (1985),

pp. 1203-1214.[9] Frydman, H., Estimation in the Mixture of Markov Chains Moving at Different Speeds (2002).

Manuscript available from the author.[10] Glowinski, R., Lions, J. L. and Tremolieres, R., Numerical Analysis of Variational Inequalities,

Norh-Holland Publishing Company, (1981).[11] Gupton, G. M., Finger, C. C., Bhatia, M., CreditMetricsTM , J.P. Morgan, New York,

(1997).[12] Hanson, S. G. M., Pesaran, H., and Schuermann, T., Firm heterogeneity and credit risk

diversification, Working Paper, Wharton Financial Institutions Center, (2007).[13] Kadam A., and Lenk P., Bayesian Inference for Issuer Heterogeneity in Credit Ratings Mi-

gration, Journal of Banking and Finance, Forthcoming.[14] Keenan, S. C., Sobehart, J. and Hamilton, D.T., Predicting Default Rates: A Forecasting

Model for Moodys Issuer-Based Default Rates, Moodys Special Comment, August (1999).[15] Siu T, Ching W, Fung E., and Ng M., On a Multivariate Markov Chain Model for Credit

Risk Measurement, Quantitative Finance, 5 (2005), pp. 543-556.

Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

E-mail : [email protected]: http://www.math.hkbu.edu.hk/people/slfung.html

Department of Mathematics and Statistics, Curtin University of Technology, Perth, Australia.

E-mail : [email protected]: http://www.maths.curtin.edu.au/staff/ksiu/index.cfm

INTERNATIONAL JOURNAL OF c© 2010 Institute for ScientificINFORMATON AND SYSTEMS SCIENCES Computing and InformationVolume 6, Number 2, Pages 155–168

UNIFIED METHODOLOGY FOR CLOSED-LOOP OPTIMAL

CONTROL PROBLEM

YOSHIKO IMURA AND D. SUBBARAM NAIDU

Abstract. Based on the earlier works of the authors [17, 8], this paper presents

the development of a unified methodology for closed-loop optimal control sys-

tems leading to matrix differential or difference Riccati equation. The method

has been developed based on Hamiltonian formalism. The unified method-

ology results are simultaneously applicable to both shift (q)-operator-based,

discrete-time systems and the derivative (d/dt)-operator-based, continuous-

time systems. It is shown that the closed-loop optimal control results that

are now obtained separately for continuous-time and discrete-time systems can

be easily obtained from a single unified methodology in terms of both problem

formulation and its solution. An example is given to illustrate the proposed

methodology.

Key Words. Unified Approach; Closed-Loop Optimal Control; Matrix Dif-

ferential (or Difference) Riccati Equation, Delta Approach; Continuous-Time

Systems; Discrete-Time Systems

1. Introduction

It is well known that the discrete-time systems, characterized by using the for-

ward shift operator (q) and the z-transform domain with unit circle as the stabilityboundary, suffer from the “over crowding” of all the stable roots inside the unit cir-cle and the associated numerical difficulties [10]. The pioneering work by Middletonand Goodwin [12, 2, 4] aroused interest in delta (δ) operators leading to an unifiedmethodology to alleviate some of the difficulties associated with the shift operator-based, discrete-time systems. The unified results are applicable simultaneously forboth continuous-time and discrete-time cases and thus avoid the development andstudy of the results for closed-loop optimal control separately for continuous-timeand discrete-time systems. The unified methodology has recently been developedfor optimal control systems [23, 16, 17], optimal control systems with state con-straints [13, 14] and for singularly perturbed systems [20, 21, 22, 18, 19].

In this paper, a unified methodology is developed for closed-loop optimal controlproblem based on the previous results by the authors [5, 16, 17, 6, 8, 7, 9]. Al-though the unified methodology to closed-loop optimal control was first developedin [12], the present methodology is developed using Hamiltonian formalism [3] andthe methodology using Lagrangian formalism [3] is given in [9]. Further, in ourworks, the unified form is viewed as one encompassing delta form (see Figure 1)and it is clearly shown how the shift (q)-operator-based, discrete-time results and

Received by the editors August 1, 2009 and, in revised form, xxxxx.2000 Mathematics Subject Classification. 35R35, 49J40, 60G40.

155

156 Y. IMURA AND D.S. NAIDU

the derivative (d/dt)-operator-based, continuous-time results are obtained from theresults of the unified methodology in terms of both problem formulation and itssolution. We first provide some brief background material for unified formulation.Next, we summarize the results that are presently available for closed-loop opti-mal control for continuous-time and discrete-time systems. We then present thedevelopment of the unified methodology leading to the matrix differential or differ-ence Riccati equation. An illustrative example is given to demonstrate the resultsdeveloped in this paper.2. Unified methodology

In general, the continuous-time (CT) and discrete-time (DT) approaches, inspite of some similarities, are treated separately in terms of developing variousmethodologies for optimal control systems [1, 11, 15]. The open-loop formulationleads to the solution of a two-point, boundary value problem (TPBVP) in termsof state and costate equations. On the other hand, the closed-loop formulationfor linear quadratic regulator or tracking problems leads to matrix differential,difference or algebraic Riccati equation. The unified methodology provides a newformulation and methodology that is simultaneously applicable to both continuous-time and discrete-time systems. This section introduces, in brief, the theoreticalbackground for unified methodology and develops the basic relationship amongdifferent operators and notations [12].

2.1. Basic Relations. By definition, the forward shift (q) and the delta (δ) oper-ators are given by

qx(kT ) = x((k + 1)T ); δx(kT ) =x((k + 1)T )− x(kT )

T,(1)

where kT is the sampling instant and T is the sampling interval. Using (1), we get

δx(kT ) =qx(kT )− x(kT )

T−→ δ =

q − 1

T.(2)

Next, consider a linear, time-invariant (LTI) continuous-time system

x(t) = Ax(t) +Bu(t), y(t) = Cx(t),(3)

where x(t) is n×1 state vector, y(t) is m×1 output vector, u(t) is r×1 control vectorand A, B and C are matrices of appropriate dimensions. In shift-operator (q) no-tation, the corresponding sampled-data system with zero-order hold and samplinginterval T for the LTI system (3) becomes

x((k + 1)T ) = eATx(kT ) +

(

∫ ∆

0

eA(T−σ)Bdσ

)

u(kT ),

= Aqx(kT ) +Bqu(kT ),(4)

y(kT ) = Cqx(kT ),(5)

Aq = eAT , Bq =

∫ ∆

0

eA(T−σ)Bdσ, Cq = C.(6)

Using the δ-operator relation (2), the discrete-time system (4) and (5) becomes

δx(kT ) =

[

Aq − I

T

]

x(kT ) +

[

Bq

T

]

u(kT ) = Aδx(kT ) +Bδu(kT ),

y(kT ) = Cδx(kT ),(7)

UNIFIED METHODOLOGY FOR CLOSED-LOOP OPTIMAL CONTROL PROBLEM 157

Aδ =Aq − I

T, Bδ =

Bq

T, Cδ = Cq.(8)

With the above notation, the delta system (7) is written in the unified notation as

ρx(τ) = Aρx(τ) +Bρu(τ), y(τ) = Cρx(τ).(9)

Further, we introduce

τf

Sτof(τ)dτ =

{

∫ tft0f(t)dt continuous form,

T∑kf−1

k=k0f(kT ) delta form,

(10)

where t0, k0 refer to initial conditions and tf , kf refer to final conditions.Note that, for the delta system (7), as T → 0

Aδ|T→0 = A; Bδ|T→0 = B; Cδ = Cq = C.(11)

The relationships between various operators, transforms formulations and systemsfor the unified, delta, discrete (shift), and continuous cases are shown in Figure 1.Here, the transforms s and z under Figure 1(d) refer to Laplace- and Z- transforms,respectively.

3. Closed-Loop Optimal Control for CT and DT Systems

In this section, we briefly summarize the well-known results in closed-loop op-timal control for continuous-time (CT) and discrete-time (DT) systems [1, 11, 15]in order to compare the results of the unified methodology to those of delta, shiftand continuous forms. In particular, we provide the results for the case of bound-ary conditions leading to a fixed-final-time and free-final-state problem, which isdiscussed in the present case for unified methodology.

3.1. Continuous-Time System. For the plant, the cost function and the bound-ary conditions, respectively, as

dx(t)

dt= x(t) = f(x(t),u(t), t) = A(t)x(t) +B(t)u(t),(12)

J [u(t)] = M(x(tf ), tf ) +

∫ tf

t0

V (x(t),u(t), t)dt,

=1

2x′(tf )F(tf )x(tf ) +

1

2

∫ tf

t0

[

x′(t)Q(t)x(t) + u′(t)R(t)u(t)]

dt(13)

x(t0) = x0 is given, x(tf ) is free, tf is fixed,(14)

we need to find the closed-loop optimal control. Here, F(t) is the terminal costweighted matrix and is n×n positive semidefinite, Q(t) is the error weighted matrixand is n × n positive semidefinite, and R(t) is the control weighted matrix and isr × r positive definite. The matrix differential Riccati equation (DRE) becomes

P(t) = −P(t)A(t)−A′(t)P(t) −Q(t) +P(t)B(t)R−1(t)B′(t)P(t) ,

P(tf ) = F(tf ).(15)

The optimal state and control relations are

x∗(t) = A(t)x∗(t)−B(t)R−1(t)B′(t)P(t)x∗(t) ,(16)

u∗(t) = − R−1(t)B′(t)P(t)x∗(t) ,(17)

where P(t) is the n× n symmetric, positive definite Riccati coefficient matrix.


tkT

kT

t

T® 0

d = (q-1)/T

(a) Independent Variables

® 0

d = (q-1)/T

® 0

d = (q-1)/T

® 0

T

T T

r

d

q

S

åT

å

ò

b

g

z

s

(b) Operators

( c ) Dynamic Operations (d) Transforms

Unified Delta Discrete Continuous

d/dt

= (z-1)/Tg

T

Figure 1. Notations for Unified methodology

3.2. Discrete-Time System. Here, we review the well-known results on closed-loop optimal control with Hamiltonian method for shift form [1, 11, 15]. For thesystem, the performance index and the boundary conditions, respectively, as

x(k + 1) = qx(k) = fq(x(k),u(k), k) = Aq(k)x(k) +Bq(k)u(k),(18)

Jq[u(k)] = Mq(x(kf ), kf ) +

kf−1∑

k=k0

Vq(x(k),u(k), k),

=1

2x′(kf )Fq(kf )x(kf ) +

1

2

kf−1∑

k=k0

[

x′(k)Qq(k)x(k) + u′(k)Rq(k)u(k)]

,(19)

x(k0) = x0 is given, x(kf ) is free, kf is fixed,(20)


we need to find the closed-loop optimal control. The matrix difference Riccatiequation (DRE) becomes

P(k) = A′q(k)

[

P−1(k + 1) +Bq(k)R−1q (k)B′

q(k)]−1

Aq(k) +Qq(k).(21)

The optimal state and control equations are

x∗(k + 1) =

{

Aq(k)−Bq(k)R−1q (k)B′

q(k)A−Tq (k)

[

P(k)−Qq(k)]

}

x∗(k),

u∗(k) = − R−1q (k)B′

q(k)A−Tq (k)

[

P(k)−Qq(k)]

x∗(k),(22)

where P(k) is an n×n positive definite symmetric solution of the matrix differenceRiccati equation (DRE) and the notationA−T

q indicates the inverse of the transposeof Aq.

4. Unified methodology for Closed-Loop Optimal Control Problem

In this section, we address the unified methodology to the problem of the closed-loop optimal control with Hamiltonian formalism. Given the optimization problemdescribed by the plant, the cost function and the boundary conditions, respectively,as

ρx(τ) = fρ(x(τ),u(τ), τ) = Aρ(τ)x(τ) +Bρ(τ)u(τ),(23)

Jρ[u(τ)] = Mρ(x(τf ), τf ) +τf

Sτo

Vρ(x(τ),u(τ), τ)dτ,

=1

2x′(τf )Fρ(τf )x(τf ) +

1

2

τf

Sτo

[

x′(τ)Qρ(τ)x(τ) + u′(τ)Rρ(τ)u(τ)]

dτ,(24)

x(τ0) = x0 is given, x(τf ) is free, τf is fixed,(25)

we need to find the closed-loop optimal control. Here, Fρ(τ) is the terminal costweighted matrix which is n × n positive semidefinite, Qρ(τ) is the error weightedmatrix which is n×n positive semidefinite, andRρ(τ) is the control weighted matrixwhich is r × r positive definite. The result optimal control under Hamiltonianformulation [8, 9] is rewritten here as [9]

τf

Sτo

[

{

∂H∗−ρ [τ ]

∂x∗(τ)+

∂ ρM∗ρ [τ ]

∂x∗(τ)

}′

µx(τ)

+

{

∂ ρM∗ρ [τ ]

∂ρx∗(τ)− λρ(τ)

}′

µρx(τ) +

(

∂H∗−ρ [τ ]

∂u∗(τ)

)′

µu(τ)

]

dτ = 0.(26)

H∗−ρ [τ ] = Hρ(x

∗(τ),u∗(τ),λρ(τ), τ) =

Vρ(x∗(τ),u∗(τ), τ) + λ

′ρ(τ)fρ(x

∗(τ),u∗(τ), τ).(27)

where superscript ∗− is used here instead of ∗ because λρ is not yet optimum. Thenby introducing the Riccati coefficient matrix, the corresponding matrix Riccatiequation in delta form will be derived.


4.1. Delta Form. Here, we show how to obtain delta form from the unified formfor closed-loop optimal control problem using the following transformations:

τ → kTτ0 → k0Tτf → kfTρ → δ

τf

Sτodτ → T

∑kf−1k=k0

,

fρ[τ ] → fδ[kT ]Mρ[τ ] → Mδ[kT ]Vρ[τ ] → Vδ[kT ]

Jρ[u(τ)] → Jδ[u(kT )]λρ(τ) → λδ(kT )Pρ(τ) → Pδ(kT ) ,

Aρ(τ) → Aδ(kT )Bρ(τ) → Bδ(kT )Qρ(τ) → Qδ(kT )Rρ(τ) → Rδ(kT )Fρ(τ) → Fδ(kT ) .

(28)

Using the previous unified to delta form transformations (28), the closed loop op-timal control problem in unified form (23 - 25) is transformed into delta form asfollows:

4.2. Unified to Delta Form. For the optimal control problem stated in termsof the plant, performance index, and the boundary conditions, as

δx(kT ) = fδ(x(kT ),u(kT ), kT ) = Aδ(kT )x(kT ) +Bδ(kT )u(kT ),(29)

Jδ[u(kT )] = Mδ(x(kfT ), kfT ) + T

kf−1∑

k=k0

Vδ(x(kT ),u(kT ), kT ),

=1

2x′(kfT )Fδ(kfT )x(kfT )

+1

2T

kf−1∑

k=k0

[

x′(kT )Qδ(kT )x(kT ) + u′(kT )Rδ(kT )u(kT )]

(30)

x(k0T ) = x0 is given, x(kfT ) is free, kfT is fixed,(31)

find the closed-loop optimal control. In terms of the Hamiltonian, the relations forthe optimal conditions become [15, 9] become

∂H∗δ [kT ]

∂u∗(kT )= 0;

∂H∗δ [kT ]

∂x∗(kT )= −δλ∗

δ((k − 1)T ),∂H∗

δ [kT ]

∂λ∗δ(kT )

= δx∗(kT )

[

∂M∗δ [kT ]

∂x∗(kT )− λ

∗δ((k − 1)T )

]∣

∣

∣

∣

∣

k=kf

= 0,(32)

H∗δ [kT ] = Hδ(x

∗(kT ),u∗(kT ),λ∗δ(kT ), kT ),

= Vδ(x∗(kT ),u∗(kT ), kT ) + λ

∗′

δ (kT )fδ(x∗(kT ),u∗(kT ), kT ).

=1

2x∗′

(kT )Qδ(kT )x∗(kT ) +

1

2u∗′

(kT )Rδ(kT )u∗(kT )

+ λ∗′

δ (kT )[

Aδ(kT )x∗(kT ) +Bδ(kT )u

∗(kT )]

(33)

First, we obtain the optimal control using (33) in (32),

Rδ(kT )u∗(kT ) +B′

δ(kT )λ∗δ(kT ) = 0 −→

u∗(kT ) = − R−1δ (kT )B′

δ(kT )λ∗δ(kT ),(34)

Next, we obtain state and costate equations using (33) in (32),

δx∗(kT ) = Aδ(kT )x∗(kT ) +Bδ(kT )u

∗(kT ).(35)


Using the definition of δ, the state equation becomes

x∗((k + 1)T ) =[

TAδ(kT ) + I]

x∗(kT ) + TBδ(kT )u∗(kT ).(36)

Now, from (32) and (33), the costate equation becomes

− δλ∗δ((k − 1)T ) = Qδ(kT )x

∗(kT ) +A′δ(kT )λ

∗δ(kT ).(37)

Again, using the definition of δ, the costate equation becomes

λ∗δ((k − 1)T ) = TQδ(kT )x

∗(kT ) +[

TAδ(kT ) + I]′

λ∗δ(kT ).(38)

Next, to obtain closed-loop optimal control in terms of the states, we introduceRiccati coefficient matrix in delta form, Pδ, as

λ∗δ(kT ) = Pδ((k + 1)T )x∗((k + 1)T ), or

λ∗δ((k − 1)T ) = Pδ(kT )x

∗(kT ).(39)

Now, as in any closed-loop optimal control problem [15, 11] (and hence the detailsgiven in [9] of the algebra are omitted), we eliminate costates λ∗

δ(kT ) and λ∗δ((k −

1)T ) from the coupled control and state equations (34) and (38). The final relationsfor the Riccati equation, optimal state and costate are

Pδ(kT )− TQδ −[

TAδ + I]′

Pδ((k + 1)T )[

TAδ + I]

+ T[

TAδ + I]′

Pδ((k + 1)T )Bδ(kT )[

TB′δPδ((k + 1)T )Bδ +Rδ

]−1

B′δPδ((k + 1)T )

[

TAδ + I]

= 0, Fδ(kfT ) = Pδ(kfT ).

u∗(kT ) = −R−1δ B′

δ

[

TAδ + I]−T [

Pδ(kT )− TQδ

]

x∗(kT ).

x∗((k + 1)T ) ={

TAδ + I− TBδR−1δ B′

δ

[

TAδ + I]−T [

Pδ(kT )− TQδ

]

}

x∗(kT )(40)

4.3. Delta to Continuous-Time Form. Here, the closed-loop optimal controlresult in delta form will be transformed into continuous-time form using the trans-formations shown below, as T → 0,

kT → t(k + 1)T → t(k − 1)T → t

k0T → t0kfT → tfδ → d

dt

T∑kf−1

k=k0→

∫ tft0

dt.

fδ[kT ] → f [t]Mδ[kT ] → M [t]Vδ[kT ] → V [t]

Jδ[u(kT )] → J [u(t)]λδ(kT ) → λ(t)Pδ(kT ) → P(t)

Pδ((k + 1)T ) → P(t)

Aδ(kT ) → A(t)Bδ(kT ) → B(t)Qδ(kT ) → Q(t)Rδ(kT ) → R(t)Fδ(kT ) → F(t)

(41)

Using (41) in (29) - (31), the problem statement is seen to be transformed to therelations describing the plant, performance index and boundary conditions givenby (12) to (13).

We now show that the continuous-time result can be obtained from the deltaresult. To do so, we apply the transformation (41) to the delta result (40). Note


that in order to simplify the notations, (t) will be omitted from A(t), B(t), Q(t),and R(t). First, rewriting the optimal state equation in delta form, (40),

x∗((k + 1)T ) ={

TAδ + I− TBδR−1δ B′

δ

[

TAδ + I]−T [

Pδ(kT )− TQδ

]

}

x∗(kT )(42)

Modifying the previous equation,

x∗((k + 1)T )− x∗(kT ) ={

TAδ − TBδR−1δ B′

δ

[

TAδ + I]−T [

Pδ(kT )− TQδ

]

}

x∗(kT )(43)

δx∗(kT ) =x∗((k + 1)T )− x∗(kT )

T

=

{

Aδ −BδR−1δ B′

δ

[

TAδ + I]−T [

Pδ(kT )− TQδ

]

}

x∗(kT )(44)

In the previous equation, when T → 0, using (41),

x∗(t) =[

A−BR−1B′P(t)]

x∗(t).(45)

Next, from (40), when T → 0, the optimal control equation becomes

u∗(t) = −R−1B′P(t) x∗(t).(46)

From (40), when T → 0, the boundary condition becomes F(tf ) = P(tf ). Finally,rewriting (40),

δPδ(kT ) =Pδ((k + 1)T )−Pδ(kT )

T= − TA′

δPδ((k + 1)T )Aδ −A′δPδ((k + 1)T )−Pδ((k + 1)T )Aδ − Qδ

+[

TAδ + I]′

Pδ((k + 1)T )Bδ

[

TB′δPδ((k + 1)T )Bδ +Rδ

]−1

B′δ Pδ((k + 1)T )

[

TAδ + I]

.(47)

Now, when T → 0, using (41) in the previous relation, the matrix differentialRiccati equation in continuous-time form becomes as

P(t) = − Q−A′P(t)−P(t)A + P(t)BR−1B′P(t).(48)

4.4. Delta to Shift Form. In this section, it will be shown how the closed-loop optimal control result in delta form will be transformed into shift form. Thetransformations are shown here.

kT → kTk0T → k0TkfT → kfT

δ →q−1T

T∑kf−1

k=k0→ T

∑kf−1

k=k0

fδ [kT ] → 1T

{

fq [kT ] − x(kT )}

Mδ [kT ] → Mq [kT ]

Vδ [kT ] →Vq [kT ]

TJδ [u(kT )] → Jq [u(kT )]λδ(kT ) → λq((k + 1)T )Pδ(kT ) → Pq(kT )

Pδ((k + 1)T ) → Pq((k + 1)T )

Aδ(kT ) →Aq(kT )−I

T

Bδ(kT ) →Bq(kT )

T

Qδ(kT ) →Qq(kT )

T

Rδ(kT ) →Rq(kT )

TFδ(kT ) → Fq(kT )

(49)


Let us transform the delta problem into shift (discrete-time) problem,

q − 1

Tx∗(kT ) =

x∗((k + 1)T )− x∗(kT )

T

=Aq(kT )− I

Tx∗(kT ) +

Bq(kT )

Tu∗(kT )

or x∗((k + 1)T ) = Aq(kT )x∗(kT ) +Bq(kT )u

∗(kT ).(50)

Next, using (49) in (30), the performance index becomes

J∗q [u(kT )] =

1

2x

′∗(kfT )Fq(kfT )x∗(kfT )

+1

2

kf−1∑

k=k0

[

x′∗(kT )Qq(kT )x

∗(kT ) + u′∗(kT )Rq(kT )u

∗(kT )]

(51)

Finally, using (49) in (31),

x(k0T ) = x0, x(kfT ) is free, kfT is fixed.(52)

4.5. Delta to Shift Form. Now, we are going to show that the shift form resultcan be obtained from the delta result. To do so, we apply the delta form to shiftform transformation (49) to the delta result (40). Here, in order to simplify thenotations, (kT ) will be omitted from Aq(kT ), Bq(kT ), Qq(kT ), and Rq(kT ).First, using (49) in (40),

Qq +A′qPq((k + 1)T )Aq −Pq(kT )

− T A′qPq((k + 1)T )

Bq

T

[

TB′

q

TPq((k + 1)T )

Bq

T+

Rq

T

]−1

B′q

TPq((k + 1)T )Aq = 0.(53)

Modifying the previous equation (see [9]),

Pq(kT ) = Qq +A′q

{

P−1q ((k + 1)T ) +BqR

−1q B′

q

}−1

Aq.(54)

The previous relation is the matrix differential Riccati equation in shift form. Next,using (49) in (40), the optimal control equation becomes

u∗(kT ) = −R−1q B′

qA′q

[

Pq(kT )−Qq

]

x∗(kT ) .(55)

Then, from (40), the boundary conditions become Fδ(kfT ) = Pδ(kfT ). Finally,using (49) in (40),

x∗((k + 1)T ) =

{

Aq − TBq

TT R−1

q

Bq

T

′

A−Tq

[

Pq(kT )− TQq

T

]

}

x∗(kT ).

=

{

Aq −BR−1q B′

qA−Tq

[

Pq(kT )−Qq

]

}

x∗(kT ).(56)

Thus, the previous result turned out to be the same as the shift results.

5. Illustrative Example

Let us consider an example to illustrate the method developed in this paper.


5.1. Continuous-Time Form. Given a second order system, the performanceindex and the boundary conditions as

x1(t) = x2(t); x2(t) = −2x1(t) + x2(t) + u(t),(57)

J =1

2

[

x21(10) + x1(10)x2(10) + 2x2

2(10)]

+1

2

∫ 10

0

[

2x21(t) + 6x1(t)x2(t) + 5x2

2(t) + 0.25u2(t)]

dt,(58)

x1(0) = 2, x2(0) = −3, x1(10) = free, x2(10) = free,(59)

we need to find the optimal control [15]. First, the differential Riccati equation(DRE) becomes

p11 = −4p12 − 4p212 + 2,

p12 = p11 + p12 − 2p22 − 4p12 p22 + 3,

p22 = 2p12 + 2p22 − 4p222 + 5,

p11(10) = 1, p12(10) = 0.5, p22(10) = 2.(60)

Next, the optimal state relations become

x∗1(t) = x∗

2(t),

x∗2(t) = (−4p12 − 2)x∗

1(t) + (−4p22 + 1)x∗2(t)

where x∗1(0) = 2, x∗

2(0) = −3.(61)

Finally, the optimal control becomes

u∗(t) = −[

4p12 x∗1(t) + 4p22 x

∗2(t)

]

.(62)

5.2. Shift Form. For the second order system with cost function and boundaryconditions as

qx1(k) = x1(k) + x2(k),

qx2(k) = −2x1(k) + 2x2(k) + u(k),(63)

J =1

2

[

x21(10) + x1(10)x2(10) + 2x2

2(10)]

+1

2

9∑

k=0

[

2x21(k) + 6x1(k)x2(k) + 5x2

2(k) + 0.25u2(k)]

,(64)

x1(0) = 2, x2(0) = −3, x1(10) = free, x2(10) = free,(65)

one needs to find the optimal control. First, the difference Riccati equation (DRE)becomes

p11 =1

4p+22 + 1

(

p+11 − 4p+12 + 4p+22 + 4p+11p+22 − 4p212

)

+ 2,

p12 =1

4p+22 + 1

(

p+11 − 4p+22 + 4p+11p+22 − 4p212

)

+ 3,

p22 =1

4p+22 + 1

(

p+11 + 4p+12 + 4p+22 + 4p+11p+22 − 4p212

)

+ 5,(66)


P(k) =

[

p11 p12p12 p22

]

, P(k + 1) =

[

p+11 p+12p+12 p+22

]

,

p+11(10) = 1, p+12(10) = 0.5, p+22(10) = 2.(67)

Next, the optimal state becomes

x∗1(k + 1) = x∗

1(k) + x∗2(k)

x∗2(k + 1) =

(

p11 − p12 − 1)

x∗1(k) +

(

p12 − p22 + 4)

x∗2(k)

x∗1(0) = 2, x∗

2(0) = −3.(68)


u∗(k) = −[

(

− p11 + p12 − 1)

x∗1(k) +

(

− p12 + p22 − 2)

x∗2(k)

]

(69)

5.3. Delta Form. Given the second order system in terms of performance indexand boundary conditions as

δx1(kT ) = x2(kT ),

δx2(kT ) = −2x1(kT ) + x2(kT ) + u(kT ),(70)

J =1

2

[

x21(10) + x1(10)x2(10) + 2x2

2(10)]

+1

2T

9∑

k=0

[

2x21(kT ) + 6x1(kT )x2(kT ) +5x2

2(kT ) + 0.25u2(kT )]

,(71)

x1(0) = 2, x2(0) = −3, x1(10) = free, x2(10) = free,(72)

the optimal control is to be found. First, the DRE becomes

p11T

=1

T (4Tp+22 + 1)

(

p+11 − 4Tp+12 + 4T 2p+22 + 4Tp+11p+22 −

4Tp+212

)

+ 2,

p12T

=1

T (4Tp+22 + 1)

[

Tp+11 + (−2T 2 + T + 1)p+12 − 2T (T + 1)p+22 +

4T 2p+11p+22 − 4T 2p+2

12

]

+ 3,

p22T

=1

T (4Tp+22 + 1)

[

T 2p+11 + 2T (T + 1)p+12 + (T + 1)2p+22 + 4T 3p+11p+22

−4T 3p+212

]

+ 5,

p+11(10) = 1, p+12(10) = 0.5, p+22(10) = 2.(73)

Next, the optimal states become

δx∗1(kT ) = x∗

2(kT ),

δx∗2(kT ) =

[

4

2T 2 + T + 1

(

Tp11 − p12 − 2T 2 + 3T)

− 2

]

x∗1(kT )

+

[

4

2T 2 + T + 1

(

Tp12 − p22 − 3T 2 + 5T)

+ 1

]

x∗2(kT ),

x∗1(0) = 2, x∗

2(0) = −3.(74)



u∗(kT ) =−4

2T 2 + T + 1

[

(

− Tp11 + p12 + 2T 2 − 3T)

x∗1(kT )

+(

− Tp12 + p22 + 3T 2 − 5T)

x∗2(kT )

]

(75)

5.4. Delta to Continuous-Time Form. Let us see if the continuous-time formcan be obtained from the delta form by making T → 0.

−p+11 − p11

T= −δp11 =

1

4Tp+22 + 1

(

− 4p+12 + 4Tp+22 − 4p+212

)

+ 2,

making T → 0, −p11(t) = −4p12 − 4p212 + 2.(76)

Similarly p12(t) = p11 + p12 − 2p22 − 4p11p22 + 3,

p22(t) = 2p12 + 2p22 − 4p222 + 5,

p11(10) = 1, p12(10) = 0.5, p22(10) = 2.(77)

For the optimal state, making T → 0

x∗1(t) = x∗

2(t), x∗2(t) =

(

− 4p12 − 2)

x∗1(t) +

(

− 4p22 + 1)

x∗2(t),

x∗1(0) = 2, x∗

2(0) = −3.(78)

Finally, for the optimal control, making T → 0 in (75),

u∗(t) = −[

4p12x∗1(t) + 4p22x

∗2(t)

]

(79)

Thus we showed that continuous-time results can be obtained from those of thedelta form.

5.5. Delta to Shift Form. Let us see if the shift form can be obtained from thedelta form. First, in the matrix DRE, substituting T = 1,

p11 =1

4p+22 + 1

(

p+11 − 4p+12 + 4p+22 + 4p+11p+22 − 4p+2

12

)

+ 2,

p12 =1

4p+22 + 1

(

p+11 − 4p+22 + 4p+11p+22 − 4p+2

12

)

+ 3,

p22 =1

4p+22 + 1

(

p+11 + 4p+12 + 4p+22 + 4p+11p+22 − 4p+2

12

)

+ 5

p+11(10) = 1, p+12(10) = 0.5, p+22(10) = 2.(80)

For the optimal state, substituting T = 1,

x∗1(k + 1) = x∗

1(k) + x∗2(k),

x∗2(k + 1) =

(

p11 − p12 − 1)

x∗1(k) +

(

− p12 + p22 − 2)

x∗2(k)

x∗1(0) = 2, x∗

2(0) = −3.(81)

Finally, for the optimal control, substituting T = 1

u∗(k) = −[

(

− p11 + p12 − 1)

x∗1(k) +

(

− p12 + p22 − 2)

x∗2(k)

]

(82)

Thus we showed that shift form results can be obtained from delta form results.


6. Conclusions

In this paper, a unified methodology was developed for the closed-loop optimalcontrol problem based on the previous results by the authors [5, 16, 17, 6, 8, 7,9]. The present methodology is developed using the Hamiltonian formalism [3]leading to the matrix differential or difference Riccati equations. . Further, in ourworks, it is clearly shown how the shift (q)-operator-based, discrete-time results andthe derivative (d/dt)-operator-based, continuous-time results are obtained from thedelta results under the unified methodology. An illustrative example was given todemonstrate the results developed in this paper. The unified results are applicablesimultaneously for both continuous-time and discrete-time cases and thus avoid thedevelopment and study of the results of open-loop optimal control separately forcontinuous-time and discrete-time systems.

References

[1] B.D.O. Anderson and J.B. Moore. Optimal Control: Linear Quadratic Methods. PrenticeHall, Englewood Cliffs, NJ, 1990.

[2] A. Feuer and G.C. Goodwin. Sampling in Digital Signal Processing and Control. Birkhauser,Boston, MA, 1996.

[3] M. Giaquinta and S. Hildebrandt. Calculus of Variations: Volume II: The HamiltonianFormalism. Springer-Verlag, New York, NY, 1996.

[4] G.C. Goodwin, S.F. Graebe, and M.E. Salgado. Control System Design. Prentice Hall, UpperSaddle River, NJ, 2001.

[5] Y. Imura and D.S. Naidu. Unified approach to open-loop optimal control. Technical report,Measurement and Control Engineering Research Center: Idaho State University, Pocatello,ID, 2003.

[6] Y. Imura and D.S. Naidu. Unified approach for open-loop optimal control with applications toaerospace systems. In Proceedings of the 16th International Federation of Automatic Control(IFAC) Symposium on Automatic Control in Aerospace, St. Petersburg, Russia, June 14-182004.

[7] Y. Imura and D.S. Naidu. Unified approach for closed-loop optimal control. In Proceedings ofthe 2007 IEEE International Conference on Control and Automation (ICCA), Guangzhou,China, pages 2322–2327, May 30-June 1 2007.

[8] Y. Imura and D.S. Naidu. Unified approach for open-loop optimal control. Optimal Control:Applications & Methods, 28(2):59–75, March-April 2007.

[9] Yoshiko Imura. Unified Approach to Optimal Control Systems. PhD thesis, Measurement andControl Engineering, Idaho State University, Pocatello, ID, July 2007. Thesis Supervisor: D.S.Naidu.

[10] B.C. Kuo. Digital Control Systems, Second Edition. Holt, Rinehart, and Winston, New York,NY, 1980.

[11] F.L. Lewis and V.L. Syrmos. Optimal Control, Second Edition. John Wiley & Sons, NewYork, NY, 1995.

[12] R.H. Middleton and G.C. Goodwin. Digital Control and Estimation: A Unified Approach.Prentice Hall, Englewood Cliffs, NJ, 1990.

[13] M.J. Murillo. Unified Approach to Optimal Control Systems with State Constraints. PhDthesis, Measurement and Control Engineering, Idaho State University, Pocatello, ID, August2002. Thesis Supervisor: D.S. Naidu.

[14] M.J. Murillo and D.S. Naidu. Discrete-time optimal control systems with state constraints.In AIAA Guidance, Control, and Navigation (GN&C) Conference and Exhibit, Monterey,CA, August 5-8 2002.

[15] D.S. Naidu. Optimal Control Systems. CRC Press, Boca Raton, FL, 2003.[16] D.S. Naidu and Y. Imura. Unified approach for Euler-Lagrange equation arising in calculus

of variations. In Proceedings of the Automatic Control Conference (ACC), pages 3263–3268,Denver, CO, June 4-6 2003.

[17] D.S. Naidu and Y. Imura. Unified approach for Euler-Lagrange equation arising in calculusof variations. Optimal Control: Applications & Methods, 25:279–293, 2004.


[18] K.-H. Shim and M.E. Sawan. Linear-quadratic regulator design for singularly perturbedsystems by unified approach using δ operators. International Journal of Systems Science,32(9):1119–1135, 2001.

[19] K.-H. Shim and M.E. Sawan. Near-optimal state feedback design for singularly perturbedsystems by unified approach. International Journal of Systems Science, 33(3):197–212, 2002.

[20] H. Singh. Unified Approach for Singularly Perturbed Control Systems. PhD thesis, MarquetteUniversity, Milwaukee, WI, May 2001.

[21] H.S. Singh, R.H. Brown, and D.S. Naidu. Unified approach to linear quadratic regulator withtime-scale property. Optimal Control: Applications & Methods, 22(1):1–16, 2001.

[22] H.S. Singh, R.H. Brown, D.S. Naidu, and J.A. Heinen. Robust stability of singularly per-turbed state feedback systems using unified approach. IEE Proceedings: Control Theory andApplications, 148:391–396, November 2001.

[23] T. Song, E.G. Collins Jr., and R.H. Istepanian. Improved closed-loop stability for fixed-point controller realizations using the delta operator. International Journal of Robust andNonlinear Control, 11(1):41–57, January 2001.

Department of Electrical Engineering and, Measurement and Control Engineering ResearchCenter, Idaho State University, Pocatello, Idaho, 83209-8060, U.S.A.,

E-mail : [email protected] and [email protected]


THE RESEARCH ON REDUNDANCY DETECTION

ALGORITHM FOR WIRELESS SENSOR NETWORKS

WEI QU, JINKUAN WANG, AND ZHIGANG LIU

(Communicated by Jinkuan Wang)

Abstract. Due to the imbalance of the node’s energy consumption the sens-

ing radius of node is different between each other, and in view of this complex

application environment, a redundancy detection algorithm is proposed. The

corresponding redundancy detection criteria are proposed under Boolean sens-

ing model and probability sensing model and the reasonableness of the criteria

are analyzed. Simulation results show that, in the complex environment when

the nodes have the same sensing radius under Boolean sensing model, the al-

gorithm activates less node in work and the redundancy detection is thorough,

and when the nodes have different sensing radius, the algorithm detects the

redundant node adequately and efficiently still; the algorithm under probabil-

ity sensing model detects redundant nodes thorough and at the same time it is

easier and more accurate compared with grid-based method.

Key Words. Wireless sensor networks(WSN), coverage model, redundancy

detection, coverage probability.

1. Introduction

How to use the limited energy efficiently and extend the lifetime of the networkas much as possible is a challenge for WSN [1-3]. Usually, the nodes are deployeddensely in WSN, and some sensing fields of the nodes are overlapped, if all the nodeswork together, there will be a lot of redundant data that consume a lot of energy[4]. Therefore to study the node’s redundancy detection and put the redundantnodes into sleep status while maintaining the coverage quality of original networkcan reduce energy consumption and extend the lifetime of the network effectively.

Ref [5] proposed the redundancy detection algorithm of sponsored coverage calcu-lation (SCC), which worked by the help of local neighbors’ geometric location. Thealgorithm did not consider the coverage contribution of the node Rs away from thenode and the approximation of fan-shaped area didn’t equal to the coverage area,some coverage overlaps were ignored, so it got more nodes in work. On the basis of[5], ref [6] took the coverage contribution of the node 2Rs away from it and designeda density control algorithm (DCA) according to boundary coverage. The algorithmdidn’t consider the boundary effect, which damaged the performance. The studybefore generally based on the condition of the same sensing radius, while in com-plex application environment with imbalanced energy consumption, the residualenergy of the node is different. In order to balance the energy consumption of the

Received by the editors October 15, 2009 and, in revised form, * *, 2009.2000 Mathematics Subject Classification. 35R35, 49J40, 60G40.This research was supported by National Natural Science Foundation of China, under Grant

no.60874108.

169

170 Q.WEI,J.K.WANG AND Z.G.LIU

node, sink node adjusts and broadcasts sensing radius of each node periodicallyaccording to the energy property of each node received at sink node and the actualapplication need [7], under this condition the performance of general algorithmswere damaged. Furthermore the general algorithms took Boolean sensing modelto study redundancy detection, while in practical application, probability sensingmodel describes the coverage quantity of the network finer. Therefore a redundancydetection algorithm fits for complex application environment with imbalanced en-ergy consumption is proposed. Basing on the consideration of boundary effect andthe function of effective neighbor, the corresponding redundancy detection criteriaunder Boolean sensing model and probability sensing model are proposed and thereasonableness of the criteria are analyzed. Under the condition of maintaining thecoverage quality of original network, the algorithm can detect the redundant nodemore adequately.

2. Assumptions and definitions

2.1. Assumptions. Assume the location of the node is known; the sensing radiusand communication radius of the node are Rsi and Rc, they satisfy that Rc = 2Rsi0

(Rsi0 is the largest sensing radius of nodes), so the issues on the basis of networkcoverage contains connectivity; the sensing area of si is C(si);the distance betweensi and the point p satisfies dsi→p < Rsi , then p is covered by si.

2.2. Definitions. Definition1: The number of the node who covers the point p isthe coverage degree of p, denoted by deg(p).

Definition2: If the distance between si and sj satisfies 0 < dsi→sj < Rsi + Rsj ,then sj is the coverage neighbor of si.

Definition3: The node that covers the detected node’s sensing field (C(si)) ir-replaceably is the effective coverage neighbor of the detected node, else it is theineffective coverage neighbor.

Definition4: sj is the coverage neighbor of si, when the distance between themsatiates dsi→sj ≤ Rsj −Rsi , sj is the first category neighbor of si, we call Nfec forshort; When the point between the sensing cycles of si’s coverage neighbors and thepoint between the boundary of the network and the sensing cycles of si’s coverageneighbors exist within C(si), the neighbor is the second category neighbor of si, wecall Nsec for short; the other coverage neighbors are the third category neighbors,we call Ntec for short.

Definition5: The point between the sensing cycles of si’s Nsec, the point betweenthe boundary of the network and the sensing cycles of si’s Nsec, and the pointbetween the sensing cycles of si’s Ntec and si are the key points.

Definition6: Under probability sensing model, the coverage probability of point pis P (p), the coverage probability of the node is P (si), and the coverage probabilityof the network is P (N):

(1) P (p) = 1− [1− P (si, p)]

N∏

l=1

[1− P (nl, p)]

(2)

{

P (si, p) =1

(1+αdsi→p)βif dsi→p ≤ Rsi

P (si, p) = 0 if dsi→p > Rsi

(3) P (si) = min[P (p)] p ∈ C(si)

(4) P (N) = min[P (si)] si ∈ A

SAMPLE FOR HOW TO USE IJNAM.CLS 171

Where P (si, p) is the coverage probability of point p when it is within the sensingarea of si with the assumption that it’s covered only by si; P (nl, p) is the coverageprobability of point p when it is within the sensing area of nl with the assumptionthat it’s only covered by nl. nl is the coverage neighbor of si, N is the numberof the coverage neighbor of si; A is the network the nodes in; α and β are theparameters of sensor type related to physical characteristics, α=0.1, β=3.

3. Redundancy detection algorithm

Step1: The node gets the relative information distributed, which include thelocation and the radius of the node;

Step2: The node takes the selection of effective neighbor in accordance with theinformation it gets and recognize effective neighbor under Boolean sensing model;while under probability sensing model, sink node divides the network into grids andfigure out the coverage probability of the network in accordance with (1-4), andbroadcasts the coverage probability in the network;

Step3: The node gets its status in accordance with the rule of redundancy de-tection.

3.1. The selection of effective neighbor. Usually, the nodes are deployeddensely in WSN, but not all the nodes have irreplaceably contribution to the cov-erage of area, in order to reduce the time carrying redundancy computing, theselection of effective neighbor is designed: When the start and ending points of thearc formed by the cover of the coverage neighbor of si on the sensing cycle of siare within the arc formed by the cover of the other coverage neighbor of si on thesensing cycle of si.

i) Rs1 6= Rs2 and there is no intersection within C(si) between the two neighbors’sensing cycles corresponding to the two arcs;

ii) Rs1 = Rs2

If the condition satisfies any condition of i or ii, then the coverage neighborcorresponding to the covered arc is ineffective. si get the effective neighbor afterrecognized the ineffective ones, and put the effective neighbors into cache.

3.2. The rule of redundancy detection.

• Boolean sensing model

i) The number of Nfec in si’s effective neighbors is at least k;ii) The coverage degree of the key points within C(si) in the network formed by

si and si’s effective neighbors are at least k (the coverage degree here excludes thecover of si);

iii) The number ofNfec in si’s effective neighbors is k0, furthermore, the coveragedegree of the key points within C(si) in the network formed by si and si’s effectiveneighbors are at least k0′ excludes the cover of si andNfec in si’s effective neighbors,k0′ satisfies k0 + k0′ ≥ k. If the condition satisfies any condition of i, ii or iii, si isredundant and the coverage degree of C(si) is k excludes the cover of si.

• Probability sensing model

Assume si is off-duty, if the coverage probability of the key points within C(si)in the network formed by si and si’s coverage neighbors are at least kP (N)’,whereP (N)’ is the threshold of the coverage probability of the network and it sets accord-ing to the requirements of actual network, then si is redundant and the coverageprobability of C(si)is kP (N)’ excludes the cover of si.

• Proof


i) Under Boolean sensing model(1) For the first sufficient conditionBecause dsi→sj ≤ Rsj −Rsi , therefore C(si) ⊆ C(sj),then si is redundant.And because there are at least k nodes satisfy dsi→sj ≤ Rsj − Rsi ,therefore si

is redundant and the coverage degree of C(si) is k excludes the cover of si.(2)For the second sufficient conditionFor it is to detect the redundancy status of si, so the coverage degree below

means the degree excludes the cover of si. We know from the condition that thereis no key point within C(si) in the network with the coverage degree lower thank. Prove by contradiction, assume that p is the point with the smallest coveragedegree within C(si) in the network, the coverage degree of p is k0, and k0 < k; thearea within C(si) is divided into small coverage area sets by the sensing cycles ofnodes, each small coverage area is surrounded by the arc of sensing cycles or thearc of sensing cycle and the boundary of the network, and the points within thesame coverage area have the same coverage degree; p is the point within the smallcoverage area S. At first, to prove the boundary of S can’t be composed by theinternal of the sensing cycle’s arc of the other node besides si.

Assume there is a node u whose internal of the sensing cycle’s arc is the part ofthe boundary of S, and then crossing the arc would reach the area with the coveragedegree smaller than the coverage degree of p, which is contradict to the assumptionthat p is the point with smallest coverage degree within S, so the assumption ofthe node u doesn’t exist , so we should consider the following cases only: S issurrounded by the external of the arc formed by the sensing cycles of the effectiveneighbor of si, as in Fig.1(a); S is surrounded by the external of the arc formedby the sensing cycles of the effective neighbor of si and the internal of the arc ofthe sensing cycle of si, as in Fig.1(b); S is surrounded by the external of the arcformed by the sensing cycles of the effective neighbor of si and the boundary ofthe network within C(si), as in Fig.1(c); S is surrounded by the external of thearc formed by the sensing cycles of the effective neighbor of si , the boundary ofthe network within C(si), and the internal of the arc of the sensing cycle of si, asin Fig.1(d) . Then prove for the above cases. As in Fig.1 (a), p is within S, Sis surrounded only by the external of the arc formed by the sensing cycles of theeffective neighbor of si, and the sensing cycle is outside the sensing area of the node.So the boundary of S and the intersect points between the sensing cycles on theboundary have the same coverage degree as the coverage degree of p, and k0 < k,that’s contradict to the assumption that there is no key point within C(si) in thenetwork with the coverage degree lower than k. So the assumption that p is thepoint with the smallest coverage degree within C(si) in the network, the coveragedegree of p is k0, and k0 < k is wrong, that’s to say there is no point within C(si)in the network with the coverage degree smaller than k. The proofs of the otherthree cases are similar to Fig.1 (a). So si is redundant and the coverage degree ofC(si) is k.

(3) For the third sufficient conditionThe number of Nfec in si’s effective neighbors is k0,therefore from (1) we know

that si is redundant and the coverage degree of C(si) is k0.And because the coveragedegree of the key points formed by si and si’s effective neighbors is at least k0’excludes the cover of si and Nfec in si’s effective neighbors, therefore from (2) weknow that si is redundant and the coverage degree of C(si)is k0’ excludes the coverof si and Nfec in si’s effective neighbors,furthermore, k0+k′0 ≥ k,so si is redundantand the coverage degree of C(si) is k.


ii) Under probability sensing modelFrom the proof of (2) we know that the key points under Boolean sensing model

are the boundary points of the small coverage area, and Boolean sensing model isthe ideal condition of probability sensing model, therefore, the key points above arealso the boundary points under probability sensing model. So assume si is off-duty,if the coverage probability of the key points within C(si) in the network formed bysi and si’s coverage neighbors are at least k times of the threshold of the coverageprobability of the network , then si is redundant.

Integrated i and ii, the proposition is established.

Figure 1. The small coverage area (the dotted cycle is the sensingcycle of si, the shaded area is S)

3.3. Simulation and performance evaluation. In order to investigate the per-formance of our algorithm, take MATLAB7.0 to verify, the nodes deployed inthe rectangular network randomly, the size of the network is 50m x 50m, andP (N)’=0.4.

3.3.1. The performance under Boolean sensing mode. Fig.2 shows the re-lationship between on-duty node number and the deployed node number in thenetwork, when Rsi=10m. We can see that the number of the node activated inwork in our algorithm is the smallest, it is about 30%-40% of SSC, which decreasesabout 60%-70%; and it is about 50%-60% of DCA, which decreases about 40%-50%. And the number of the node activated in work in our algorithm doesn’tchange along with the number of node deployed in the network. That’s mainlybecause our algorithm considered all the coverage contribution of the node within2Rsi away from the node, it also considered the boundary effect and designed theredundancy detection works well for boundary, so the number of the node activated


in work in our algorithm doesn’t change along with the number of node deployedin the network and the redundancy detection is thorough, which activate less nodein work, the unnecessary energy consumption of the network is less, that’s benefitfor extending the lifetime of network.

Fig.3 shows the relationship between the node on-duty, the sensing radius andthe deployed node number in the network (k is the coverage degree), when thesensing radius taken randomly from the interval of 3m to 8m, the interval of 5m to10m, and the interval of 7m to 12m.We can see that our algorithm demonstratessuperior performance under all conditions, the redundancy detection is thoroughand fully.

100 150 200 250 30010

20

30

40

50

60

70

80

90

100

110

120

Deployed node number

On

−d

uty

no

de

nu

mb

er

OurDCASSC

Figure 2. On-duty node number vs deployed node number

100150

200250

300

3−85−10

7−12

10

20

30

40

50

60

70

80

90

100

Deployed node numberSensing radius

On−

duty

nod

e nu

mbe

r

k=2

k=1

Figure 3. The relationship between the node on-duty and thesensing radius and the deployed node number

3.3.2. The performance under probability sensing model. Divide the net-work into grids of 1m x 1m, the grid-based method takes the coverage probabilityof the center in each sub-grid as the coverage probability of the sub-grid, so it needsto compute the coverage probability of all sub-grids, while in our algorithm underprobability sensing model, it just needs to compute the coverage probability of thekey points in the network, it is easier than grid-based method. Simulation resultsto verify the correctness and performance of our algorithm are given below.


Fig.4 and Fig.5 show the performance of coverage probability of the node underprobability sensing model and grid-based method, the sensing radius of the nodesare taken randomly from 10m to 15m. As in Fig.4, we can see that the coverageprobability of the node is always smaller under probability sensing model than thatin grid-based method, when the node density varies. The difference is about 0.04.That’s mainly because that grid-based method takes the coverage probability of thecenter in each sub-grid as the coverage probability of the sub-grid, while the smallestcoverage probability maybe at other area, therefore the coverage probability of thenode under probability sensing model is more accurate.

Fig.5 shows the coverage probability of network under probability sensing modeland grid-based method. We can see that the coverage probability of network in-creases along with the increases of node density, that’s because the larger the densityof node the better covering for the network, and the coverage probability of networkis larger; at the same time, the coverage probability of network under probabilitysensing model are smaller than that in grid-based methodwhen the node density isthe same. So the coverage probability of network under probability sensing modelis closer to the true value, it’s more accurate compared with grid-based method.

50 100 150 2000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Number of node

Cov

erag

e pr

obab

ility

diff

eren

ce

Figure 4. Coverage probability difference

50 100 150 2000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of node

Cov

erag

e pr

obab

ility

of n

etw

ork

Grid Probability sensing model

Figure 5. The coverage probability of network

Fig.6 and Fig.7 show the performance of the algorithm under probability sensingmodel when the sensing radius taken randomly from the interval of 3m to 8m, the


interval of 5m to 10m, and the interval of 7m to 12m.Fig.6 shows the relationshipbetween the node on-duty, the sensing radius and the deployed node number in thenetwork. We can see that the number of the node on-duty in our algorithm underprobability sensing model doesn’t change along with the increase of the number ofnode deployed in the network, the redundancy detection is thorough and fully. Theexperiment of Fig.6 combined with the experiment of Fig.3 further validated thesuperiority of our algorithm.

Fig.7 shows the relationship between the coverage probability of the network,the sensing radius and the deployed node number in the network. We can see thatthe coverage probability of the network increases along with the increases of nodedensity when the sensing radius of node takes in the same sensing radius interval;the coverage probability of the network increases along with the increases of thesensing radius of node when the number of node deployed are the same; the coverageprobability of the network is more accurate under probability sensing model thangrid-based method.

100

150

200

3−85−10

7−12

35

40

45

50

55

60

65

Deployed node numberSensing radius

On−

duty

nod

e nu

mbe

r

Figure 6. The relationship between the node on-duty and thesensing radius and the deployed node number under probabilitysensing model

100

150

2003−8

5−10

7−12

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Sensing radiusDeployed node number

Cov

erag

e pr

obab

ility

of n

etw

ork

Grid

probability sensing model

Figure 7. The relationship between the coverage probability ofnetwork and the sensing radius and the deployed node numberunder probability sensing model


4. Conclusion

A redundancy detection algorithm fits for complex application environment withimbalanced energy consumption is proposed in this paper. The corresponding re-dundancy detection criteria under Boolean sensing model and probability sensingmodel are proposed and the reasonableness of the criteria are analyzed. Simulationresults show that, in the complex environment when the nodes have the same sens-ing radius under Boolean sensing model, the algorithm activates less node in workand the redundancy detection is thorough, and when the nodes have different sens-ing radius, the algorithm detects the redundant node adequately and efficiently still,which saves the energy consumption and benefits for the extending of lifetime of thenetwork; the algorithm under probability sensing model detects redundant nodesthorough and it is easier and more accurate compared with grid-based method.

Acknowledgments

The work is supported by National Natural Science Foundation of China, underGrant no.60874108.

References

[1] Elson J., Estrin D., ”Sensor Networks: A bridge to the physical world,” Wireless sensornetworks,pp.3-20,2004.

[2] Li L.,Wen X.M., ”Energy efficient optimization of clustering algorithm in wireless sensornetwork,” Journal of Electronics & Information Technology,Vol. 30, pp.966-969, 2008.

[3] Yu H.B.,Li B.X.,Zeng P., ”Multi-path routing protocol of wireless sensor networks for datamonitoring,” Control and Decision, Vol. 23, pp. 674-679, 2008.

[4] Wang S.,Wang X.,Bi D.W., ”Dynamic sensor selection optimization strategy for wirelesssensor networks,” Journal of Computer Research and Development, Vol. 45, pp. 188-195,2008.

[5] Tian D.,Georganas N.D., ”A coverage-preserving node scheduling scheme for large wirelesssensor networks,” ACM International Workshop on Wireless Sensor Networks and Applica-tions, pp. 32-41,2002.

[6] Jiang J., Dou W.H., ”A coverage-preserving density control algorithm for wireless sensornetworks,” International Conference on AD-HOC Networks & Wireless (ADHOC-NOW),pp.42-55, 2004.

[7] Jia J., Chen J., Chang G.R., ”Optimal Sensor Deployment Based on Secure Connection inMobile Sensor Network,” Journal of Software, Vol. 20, pp. 1038-1047,2009.

Wei Qu She is currently a PhD Candidate of Northeastern University in China. Her researchinterests are wireless ad hoc and sensor networks.E-mail: [email protected] Wang received the PhD degree from the University of Electro-Communications, Japan,in 1993. He is currently a professor in the School of Information Science and Engineering atNortheastern University, China, since 1998. His main interests are in the area of intelligentcontrol ,adaptive array and wireless sensor networks.E-mail: [email protected]

School of Information Science and Engineering, Northeastern University, Shenyang, Liaoning110004, China


A MODIFIED MAP CHANNEL ESTIMATION ALGORITHM FOR

MIMO-OFDM SYSTEMS

PENG XU1, JINKUAN WANG1 AND FENG QI2

Abstract. Maximum a posteriori probability (MAP) channel estimation algo-

rithm induces large matrix inversion and product operation when it is applied to

multiple-input multiple-output with orthogonal frequency division multiplexing

(MIMO-OFDM) systems. The data transmission efficiency will be reduced with

the increasing number of transmit antennas. According to these problems, a

modified MAP algorithm for MIMO-OFDM is proposed. This algorithm which

uses the characteristic of expectation maximum (EM) algorithm decreases the

high complexity of MAP algorithm. Besides, to improve the data transmission

efficiency and the mean square error (MSE) performance of channel estimation,

joint estimation is carried out over multiple OFDM symbols. Simulation results

indicate the proposed algorithm obtains better the performance of estimation

and higher data transmission efficiency than MAP algorithm.

Key Words. mimo, ofdm, channel estimation, em, map.

1. Introduction

The increasing growth of wireless communication requires high data-rate capabletechnologies. Novel techniques such as OFDM and MIMO stand as promisingchoices for future high data-rate systems [1]-[3]. MIMO-OFDM can be implementedto achieve a low error rate and high data rate by flexibly exploiting the diversitygain and the spatial multiplexing gain [4]-[6]. Realizing these gains requires thechannel state information (CSI) at the receiver, which is often obtained throughchannel estimation.

To estimate the CSI and correct the received signal, the pilot-based approachesinserted in frequency domain are widely used. Therefore, to recover transmitteddata through received data, the accuracy of CSI greatly influences the overall sys-tem performance. The channel estimation in MIMO systems becomes more com-plicated in comparison with single-input single-output systems due to simultaneoustransmission of signals from different antennas that cause cochannel interference.To estimate the MIMO channel independently, pilot sequences on the differenttransmit antennas must be orthogonal. In [7], time orthogonal scheme which isjust suitable for WLAN is investigated. In [8], phase shifted orthogonal (PSO)scheme is proposed by minimizing channel estimation mean square error (MSE).To solve channel estimation in limited scattering situations, angle-domain LS es-timator is proposed based on the assumption that channel coefficients in differentangle-domain bins can be assumed to be approximately spatially uncorrelated [9].

For single transmit antenna systems the MAP channel estimation algorithm canbe obtained by applying a data-independent wiener filter to the maximum likelihood(ML) estimate. When applied to MIMO-OFDM systems, however, its complexity

178

MODIFIED MAP CHANNEL ESTIMATION ALGORITHM 179

can be prohibitively high for most application. Expectation maximum (EM) al-gorithm which converts multi-input channel estimation problems into a series ofsingle-input channel estimation problems is becoming a powerful tool for iterativeparameter estimation in data communication. In most cases, EM algorithm isapplied as a practical method to find the maximum likelihood (ML) estimates ofparameters of interest. Two EM algorithms, the classical EM and space alternatinggeneralized EM (SAGE) algorithms, are compared in terms of their mean squareerror (MSE) and convergence performance [10]. In MIMO system, an optimal MAPestimator is proposed and its high complexity is decreased by iteration [11]. As forthe optimal MAP estimator, [12] makes use of low-rank approximation to deal withthe high complexity.

In this paper, we propose a modified MAP algorithm. Firstly, we develop MAPchannel estimation relying on EM algorithm which reduces high complexity of MAPalgorithm. Then, we enhance the data transmission efficiency and MSE perfor-mance with multiple OFDM symbols. The proposed algorithm has lower complex-ity, higher data transmission efficiency and better MSE performance. Simulationresults show the effectiveness of the proposed algorithm.

2. Problem formulation

2.1. MIMO-OFDM system model. Consider a typical MIMO-OFDM systemwith Nt transmit and Nr receive antennas. The high rate symbols to be transmit-ted are first grouped into K subcarriers and an inverse discrete Fourier transform(IDFT) is applied to each OFDM symbol at each transmitter. The IDFT at thetransmitter and the discrete Fourier transform (DFT) at the receiver serve to mod-ulate and demodulate the data on the orthogonal subcarriers, respectively. Thechannel has an exponentially decaying multipath power delay profile which de-termines the power distribution among the taps, and the maximum tap delay isassumed to be equal to the OFDM guard interval. hr,t(l) represents the tap gainof path l for the subchannel from transmit antenna t to receive antenna r. Thereceived nth OFDM symbol in the time domain is given by

(1) Yr(k) =

Nt∑

t=1

dt(k)Hr,t(k) + Zr(k)

where dt(k) is the transmitted symbol at the kth subcarrier from the tth antennas,

Hr,t(k) =∑L−1

l=1 hr,t(l)e−j2πkl/K is the kth element of the Fhr,t, F is K×L matrix

with F [k, l] = e−j2πkl/K , 0 ≤ k ≤ K − 1, 0 ≤ l ≤ L − 1, hr,t = [hr,t(0), hr,t(1), ...,

hr,t(L − 1)]T, Zr(k) =

1√K

∑K−1n=0 zr(n)e

−j2πnk/K , zr(n) is the zero-mean additive

white Gaussian noise (AWGN) with variance σ2.

2.2. Expectation Maximization (EM) channel estimation algorithm. TheEM algorithm, developed in [10], is a general method for solving maximum likeli-hood (ML) estimation problems given incomplete data. Let Yr denote the observedor “incomplete” data, Yr is the received data from the rth receive antenna. Andthe “complete” data Yr,t is defined as

(2) Yr,t = DtHr,t + Zr,t 1 ≤ t ≤ Nt

where Yr =∑Nt

t=1 Yr,t, Zr =∑Nt

t=1 Zr,t. Yr,t is the component of the received signaltransmitted by the antenna through the channel with impulse response Hr,t. It iseasy to show that the EM algorithm for the above particular choice of completedata takes the following form.

180 P. XU, J.K. WANG, F. QI

• E step: For i = 1, 2, ...Nt, compute

(3) Y(i)r,t = Dt(p)Fph

(i)r,t

(4) Ψ(i)r,t = Y

(i)r,t + βt[Yr −

Nt∑

t=1

Y(i)r,t ]

where superscript i denotes the ith iteration, and the βt are chosen such that∑Nt

t βt = 1, typically, βt are chosen as β1 = β2 = ... = βNt= 1/Nt. Fp is a P × L

matrix with corresponding jth row of DFT matrix, j = 1, ps, 2ps, ..., (P − 1)ps,P and ps is the number of pilot subcarriers and pilot subcarriers spacing in oneOFDM symbol. Dt(p) = diag[Dt(0), Dt(ps), ..., Dt((P − 1)ps)] is the transmitted

pilot matrix. h(i)r,t is channel coefficient of kth iteration.

• M step: For i = 1, 2, ...Nt, compute

(5) h(i+1)r,t = argmin

hr,t

{‖Ψ(i)r,t −Dt(p)Fph

(i)r,t‖}

Solving (5), we obtain

(6) h(i+1)r,t = FH

p D−1t(p)Ψ

(i)r,t

Since Dt(p) is a diagonal matrix, D−1t(p) can be obtained via division only. A proper

selection of the initial value of h(0)r,t is very important for the convergence speed of

EM algorithm. The channel estimates may be initially set as h(0)r,t = 1L, 1 ≤ r ≤

Nr, 1 ≤ t ≤ Nt, where 1L is a L× 1 vector whose elements are all 1’s.

3. Simplified MAP (SMAP) channel estimation algorithm

Although a great many channel estimation algorithms start from LS algorithm,its performance can not meet fast-fading channels. Through MAP algorithm, wecould obtain ideal MSE performance. The received signal vector on all subcarriersat antenna r can be expressed as

(7) Yr = DHr + Zr

where Yr = [Yr(0), Yr(1), ..., Yr(K−1)],D = [D1, ..., DNt], Dt = diag[dt(0), dt(1), ...

, dt(K − 1)], Zr = [Zr(0), Zr(1), ..., Zr(K − 1)], Hr = [HTr,1, H

Tr,2, ..., H

Tr,Nt

]T . Theobvious drawback of optimal MAP algorithm is its high computational complexity.We need to invertKNt×KNt data-dependent correlation matrix andK×KNt datamatrix during estimating each OFDM symbol. MAP algorithm will be invalid whenKNt is large [11]. According to [10], EM channel estimation algorithm can transferMIMO channel estimation problem into Nt independent SISO channels estimationproblems [12]. After simplifying, we just need a K ×K diagonal matrix inversion,a K×K data-dependent correlation matrix inversion and the multiplication of twoK ×K matrices. (7) is rewritten as [12]

(8) Yr =

Nt∑

t=1

DtHr,t +

Nt∑

t=1

Zr,t

Because of the ability of transferring high-dimensional issue to a series of low-dimensional issues, we design MAP estimator based on (8) accordingly. MAP


estimator maximizes the probability density function (pdf) of Hr,t conditional onthe received signal and the transmitted data matrix as [11]

Hr,t = argmaxHr,t

f(Hr,t|Yr,t, Dt)

= argmaxHr,t

f(Yr,t|Hr,t, Dt)f(Hr,t|Dt)(9)

where

f(Yr,t|Hr,t, Dt) = π−1|RZ |−1exp((DtHr,t − Yr,t)

HRZ(Yr,t −DtHr,t))(10)

f(Hr,t|Dt) = π−1|RH |−1exp(−HHr,tR

−1H Hr,t)(11)

It is shown in [11] that the MAP estimate of Hr,t can be expressed as

Hr,t = µ+RHDHt (DtRHDH

t +Rz)−1(Yr,t −Dtµ)(12)

where Rz is zero-mean noise vector, and µ and RH denote, respectively, the meanand correlation matrix of Hr,t . In a quasi-static channel, Rz is expressed as Rz =σ2Iz , where Iz is identity matrix and σ2 = σ2

AWGN/Nt . Considering channelcoefficients have a mean zero, (12) can be rewritten as

Hr,t = RH(RH + σ2(DHt Dt)

−1)−1D−1t Yr,t(13)

where (m,n)th element of RH is expressed as

[RH ]m,n = ε

L−1∑

l=0

e−l/Le−2π(m−n)l/K(14)

where ε = (1−e−l/L)/(1−e−(L+1)/L) is normalization factor to ensure ε∑

l e−l/L =

1. Clearly, the inversion of the N ×N data-dependent correlation matrix and themultiplication of two N×N matrices need be done for all sub-channels during eachOFDM symbol interval.

To estimate the coefficients for channel vectors from all transmit antennas,SMAP algorithm can be implemented as

• E step: For i = 1, 2, ...Nt, compute

(15) Y(i)r,t = Dt(RH + σ2(DH

t Dt)−1)R−1

H F h(i)r,t

(16) Ψ(i)r,t = Y

(i)r,t + βt[Yr −

Nt∑

t=1

Y(i)r,t ]


(17) h(i+1)r,t = argmin

hr,t

{‖Ψ(i)r,t −Dt(RH + σ2(DH

t Dt)−1)R−1

H F h(i)r,t‖}


(18) h(i+1)r,t = FHRH(RH + σ2(DH

t Dt)−1)−1D−1

t Ψ(i)r,t


4. Pilot-reduced SMAP channel estimation algorithm

Pilot-based channel estimation algorithms has better tracking and convergenceproperties than blind algorithm at the cost of data transmission efficiency. Espe-cially for MIMO-OFDM systems, more and more pilot subcarriers are required ineach OFDM symbol with the increasing number of transmit antennas. So, we pro-pose a pilot-reduced SMAP channel estimation algorithm which employs multipleOFDM symbol to estimate channel coefficients jointly. In [8], to consider a mini-mum number of pilot tones or a maximum spacing, the number of pilot subcarriersP in each OFDM symbol should be a power of 2 and need meet P ≥ 2⌈log2(LNt)⌉

and, clearly, the number of P will be unacceptable when Nt is continuously increas-ing.

Here, we propose the design and placement of pilot subcarriers in each OFDMsymbol. The estimation over multiple OFDM symbols can not only further MSEperformance but also improve data transmission utilization. To estimate the MIMOchannel, it is important that the subcannels from the different transmit antennasto every receive antenna can be uniquely identified. To achieve that, the pilotsequences on different transmit antennas should be orthogonal. At the same time,we consider training over G continuous OFDM symbols for estimating the wirelessfading channel. First, pilot sequences in each OFDM symbol from different transmitantennas need satisfy phase shifted orthogonal with condition on G = 1, 2, 4. Afterthat, the placement of pilot subcarriers of gth OFDM must be satisfied

(19) Tp(g) = 1 + (g − 1)×K

P+G× (p− 1)×

K

P

where g = 1, ..., G, p = 1, 2, ..., P/G, P = 2⌈log2(LNt)⌉. Through (19), we can obtainthe number of pilot required in each OFDM symbol as follow:

Table 1. Number of pilots with different G

Number of transmit antennas

G 2 3 4 5 6 7 81 16 32 32 64 64 64 642 8 16 16 32 32 32 324 4 8 8 16 16 16 16

Seen from the table above, the amount of pilots required for each OFDM symbolwill be reduced obviously along with the increasing G. SMAP algorithm can bewritten as

For g = 1, 2, ...G,For i = 1, 2, ...Nt, compute• E step:

(20) Y(i)r,t(g) = Dt(g)(RH(g) + σ2(DH

t(g)Dt(g))−1)R−1

H(g)F h(i)r,t(g)

(21) Ψ(i)r,t(g) = Y

(i)r,t(g) + βt[Yr(g) −

Nt∑

t=1

Y(i)r,t(g)]


(22) h(i+1)r,t(g) = arg min

hr,t(g)

{‖Ψ(i)r,t(g)−Dt(g)(RH(g)+σ2(DH

t(g)Dt(g))−1)R−1

H(g)F h(i)r,t(g)‖}


(23) h(i+1)r,t(g) = FHRH(g)(RH(g) + σ2(DH

t(g)Dt(g))−1)−1D−1

t(g)Ψ(i)r,t(g)


Through above process of iteration, we obtain channel estimation of every OFDMsymbol, and then joint estimation can be derived by

(24) h(i+1)r,t =

1

G

G∑

g=1

h(i+1)r,t(g)

5. Performance analysis

5.1. Complexity performance comparison between MAP and SMAP al-

gorithm. Seen from [11] and (7), the complexity of MAP algorithm is as fol-lows. For each OFDM symbol, the pseudo-inverse of a data matrix of size K ×

KNt: O(K3); inversion of data-dependent correlation matrix of size KNt ×KNt:

O((KN)3); multiplication of two KNt ×KNt matrices: O((KN)

3). However, for

the proposed SMAP algorithm, for each OFDM symbol, during each process EM ofiteration, the inverse of a diagonal matrix of size K ×K: O(K); inversion of data-dependent correlation matrix of size K ×K: O(K3); multiplication of two K ×Kmatrices: O(K3). Besides, RH(g) does not change for a large number of OFDMsymbols since it is a function of channel power delay profile (PDP). Therefore, fora given large number of OFDM symbols, the term RH(g) can be assumed to becontant and calculated in advance. Although the exact complexity of these steps

are difficult to quantify, it is far lower than O((KN)3) since all matrices involved

are either fixed (e.g., F and RH(g), which do not need to be updated on a perOFDM symbol basis) or diagonal (e.g., Dt(g)).

5.2. Performance comparison of SMAP algorithm over different G. Ac-cording to the pilot-reduced SMAP channel algorithm, we want to make sure howthe MSE performances of SMAP algorithm changes based on different G. The MSEcriterion is defined as

MSE(G) ,1

NrNtL

Nr∑

r=1

Nt∑

t=1

‖hr,t(G)− hr,t‖2(25)

where hr,t(G) = 1G

∑Gg=1 hr,t(g), G = 1, 2, 4. Actually, depending on (25), we

obtain the MSE performance with the goal to know whose performance will begood. In order to compare MSE(1), MSE(2) and MSE(4), we just need compute

‖hr,t(G) − hr,t‖2 instead of the whole (25). So, simplified expression of MSE(G)

can be given by

MSE(G) ,

‖hr,t − hr,t‖2 : G = 1

‖ 12

∑2g=1 hr,t(g) − hr,t‖

2 : G = 2

‖ 14

∑4g=1 hr,t(g) − hr,t‖

2 : G = 4

(26)

Here, we assume wireless channel is a quasi-static channel that is time-invariantwithin the transmission of a few block. So, the power of estimated channels be-tween the adjacent several OFDM symbols may be considered as constant, for a

great many OFDM symbols, that is < hr,t(m), hr,t(m) >=< hr,t(n), hr,t(n) >=<

hr,t, hr,t >,m 6= n, 1 ≤ m,n ≤ G. Besides, we employ an important and basicrestriction that channels in different OFDM symbols is statistical independence.

Thus, we get < hr,t(m), hr,t(n) >= 0. Then, we apply basic property of norm todeal with the comparison of MSE performance of different G. On the basis of norm


axiom, the MSE under three conditions of G are formulated.

MSE(1) = ‖hr,t − hr,t‖2

= < hr,t, hr,t > −2 < hr,t, hr,t > + < hr,t, hr,t >(27)

MSE(2) = ‖1

2

2∑

g=1

hr,t(g) − hr,t‖2

= < hr,t, hr,t > −2 < hr,t, hr,t > +1

2< hr,t, hr,t >(28)

MSE(4) = ‖1

4

4∑

g=1

hr,t(g) − hr,t‖2

= < hr,t, hr,t > −2 < hr,t, hr,t > +1

4< hr,t, hr,t >(29)

Obviously, we can obtain the MSE information between different G, MSE(4) <

MSE(2) < MSE(1), that is, MSE(4) < MSE(2) < MSE(1). All the analy-sis shows that the proposed modified MAP channel estimation has a relative lowcomplexity, more low demand for pilots and better MSE performance.

6. Simulation results

In the previous sections, we presented the step and mathematical analysis of theproposed modified MAP algorithm. Here, we evaluate the SMAP performance overone or multiple OFDM symbols in different simulations. We assume that the chan-nels of different transmit-receive antenna pair are independent, and the hr,t(l)

′s areuncorrelated zero-mean Gaussian random variables with exponential power delayprofile E[|hr,t(l)|

2] = 1, l = 0, 1, ..., L− 1, L = 8. In our simulation, we consider auncoded MIMO-OFDM system occupying a bandwidth of 20MHz. We consider anOFDM system with K = 128 subcarriers, 15KHZ subcarriers spacing, CP=8. Thesubcarriers are modulated by QPSK symbols. The entire simulations are conductedin the equivalent baseband, and we assume both symbol synchronization and carriersynchronization are perfect. Throughout the simulation, the parameter signal-to-noise (SNR) is defined as a ratio of received bit energy to the power spectral densityof noise. Some statistical information such as doppler frequency, auto-covarianceof channels, and noise power is assumed to be known for this paper.

Fig. 1 to Fig. 4 achieve the channel estimation over one OFDM symbol andexperiments in Fig. 1 and Fig. 2 operate for 2 × 4 MIMO system. Fig. 1 showsthe MSE as a function of different iterations at three SNR based on the channelestimates of SMAP estimators. Relying on the fast convergence of EM, SMAPwhich is based on EM also has good convergence rate as expected. Seen from thepicture, SMAP algorithm will achieve convergence at about 5 to 10 times. Fig.2 provides the MSE performance based on the channel estimates of the LS, EMand SMAP estimators at different SNR. As the picture shows, SMAP algorithmoutperforms LS and EM algorithms because of the use of subcarriers relevance.Although SMAP algorithm is convergent at high SNR as the reason of EM, itsgood MSE performance at low SNR makes it practical.

Fig. 3 and Fig. 4 show that how the MSE performances of LS, EM and SMAPalgorithms are changing based on the number of transmit and receive antennas atSNR = 0. Fig. 3 indicates that EM algorithm will obtain good MSE performanceonly if Nt = Nr = 2 or 4. However, SMAP algorithm ouperforms other two


0 5 10 15 20 2510

−4

10−3

10−2

10−1

Number of iterations

MS

E

SMAP snr=0dBSMAP snr=5dBSMAP snr=10dB

Figure 1. MSE versus number of iterations

0 5 10 15 20 2510

−4

10−3

10−2

10−1

100

SNR(dB)

MS

E

LSEMSMAP

Figure 2. MSE versus SNR

alogrihtms regardless Nt = Nr or Nt < Nr. Fig. 5 shows the MSE performanceachieved by the LS and SMAP channel estimators based on different G for 2 × 4MIMO system. Seen from table 1., when Nt = 2 and G = 1, 2 and 4, for SMAPalgorithm, pilot subcarrers needed in one OFDM symbol is 16, 8 and 4, respectively.And MSE of SMAP algorithm will obtain gradual enhancement in pace with thedecreasing number of pilots. However, the MSE performance of LS algorithm willbe reduced obviously when the number of pilots in each OFDM symbol can notmeet P ≥ 2⌈log2(LNt)⌉.


2 3 4 5 6 7 810

−3

10−2

10−1

100

Number of receive antennas

MS

E

LSEMSMAP

Figure 3. MSE versus number of receive antennas (Nt =Nr, SNR = 0)

2 3 4 5 6 7 810

−3

10−2

10−1

100

Number of receive antennas

MS

E

LSEMSMAP

Figure 4. MSE versus number of receive antennas (Nt = 2 ≤

Nr, SNR = 0)

7. Conclusions

In this paper, we have proposed a MAP channel estimation algorithm for MIMO-OFDM systems in order to solve the problems induced by optimal MAP algorithm.The first problem is the high complexity brought about by optimal MAP algorithmwhich requires large matrix operation. And the second is the low data transmissionefficiency due to the defect of pilot-based MAP algorithm. According to theseshortcomings, we employ the characteristic of EM algorithm to simplify the optimalMAP algorithm. Besides, to improve the data transmission efficiency and the MSEperformance of channel estimation, joint estimation is carried out over multiple


0 5 10 15 20 2510

−4

10−3

10−2

10−1

100

101

SNR(dB)

MS

E

LS G=1LS G=2LS G=4SMAP G=1SMAP G=2SMAP G=4

Figure 5. MSE versus different G

OFDM symbols. Simulation results indicate the proposed algorithm obtains betterperformance of estimation and higher data transmission efficiency than optimalMAP algorithm.

Acknowledgment

The work is supported by the National Natural Science Foundation, under Grantno. 60874108

References

[1] Foschini G. J., “On limits of wireless communications in a fading environment when usingmultiple antennas,” IEEE Trans. Wireless Pers. Commun., Vol. 6, pp. 311–335, 1998.

[2] Cimini L. J., “Analysis and simulation of a digital mobile channel using orthogonal frequencydivision multiplexing,” IEEE Trans. Commun., Vol. 33, pp. 665–675, 1985.

[3] Cimini L. J., Daneshrad B. and Sollenberger N. R., “Clustered OFDM with transmitterdiversity and coding,” in Proc. 1996 IEEE Global Telecommunications Conf., London, U.K., pp. 703–707, 1996.

[4] Shui D., Foschini M. J., Sazontov A.G. and Gans M. J., “Fading correlation and its effecton the capacity of multi-element antenna systems,” IEEE Trans. Commun., Vol. 48, pp.502–513, 2000.

[5] Akhtar J. and Gesbert D., “A closed-form precoder for spatial multiplexing over correlatedMIMO channel,” Proc. IEEE, Vol. 4, pp. 1847–1851, 2003.

[6] Alamouti S. M., “A simple transmit diversity technique for wireless communications,” IEEETrans. Inform. Theory, Vol. 16, pp. 1451–1458, 1998.

[7] Van Zelst A. and Schenk T.C.W., “Implementation of MIMO OFDM-based wireless LANsystems,” IEEE Trans. Signal Process., Vol. 52, pp. 483–494, 2004.

[8] Barhumi I., Leus G. and Moonen M., “Optimal training design for MIMO OFDM systemsin mobile wireless channels,” IEEE Trans. Signal Process., Vol. 51, pp. 1615–1624, 2003.

[9] LI H., Chin K. H., Bergmans J.W.M. and Willems F.M.J., “Pilot-based angle-domain channelestimation techniques for MIMO-OFDM systems,” IEEE Trans. Veh. Technol., Vol. 57, pp.906–920, 2008.

[10] Xie Y. and Georghiades C.N., “Two EM-type channel estimation algorithm for OFDM withtransmitter diversity,” IEEE Trans. Commun., Vol. 51, pp. 106–115, 2003.

[11] Deng X., Haimovich A. M. and Garcia-Frias J., “Decision directed iterative channel estimationfor MIMO systems ,” Proc. IEEE, Vol. 4, pp. 2326–2329, 2003.


[12] Gao J. and Liu H.P., “Low-Complexity MAP channel estimation for mobile MIMO-OFDMsystems ,” IEEE Trans. Commun., Vol. 7, pp. 774–780, 2008.

Peng Xu was born in Shenyang, China, in 1981. He got his M.S Degree in Communication andInformation Systems from Northeastern University in China in 2008. Now he is working for hisPhD at Northeastern University. His research interest includes Next-generation mobile communi-cation technology and OFDM channel Identification.E-mail: [email protected]

Jinkuan Wang received the PhD degree from the University of Electro-Communications, Japan,in 1993. He is currently a professor in the School of Information Science and Engineering at North-eastern University, China, since 1998. His main interests are in the area of intelligent control andadaptive array.E-mail: [email protected]

Feng Qi got his masters degree in the Department of Electrical Engineering (ESAT), KatholiekeUniversiteit Leuven, Belgium. Now he is doing his PhD research in ESAT-TELEMIC in the sameuniversity. His research interest includes quasi-optical imaging system, antenna and antenna arraydesign in the region of microwave and millimeter wave.E-mail: [email protected]

1Engineering Optimization and Smart Antenna Institute, Northeastern University at Qin-huangdao, 066004, China

URL: http://www.neuq.edu.cn/sasp/index.htm

2ESAT-TELEMIC, Katholieke Universiteit Leuven at Leuven, 3001, BelgiumURL: http://www.esat.kuleuven.be/telemic/


ON STABILITY CONDITIONS FOR HIGH-SPEED VARIANTS

OF TCP AND RED BASED AQM

AUN HAIDER, HARSHA SIRISENA, VICTOR SREERAM, AND RICHARD HARRIS

Abstract. This paper investigates the stability of Active Queue Management

(AQM) system formed by high-speed variants of Transmission Control Protocol

(TCP) and Random Early Detection (RED) routers. Dynamical modeling is

used to analyze the stability of the closed loop system at the operating point

in terms of the number of high-speed TCP connections. It has been shown

that period doubling bifurcations can occur by varying the system parame-

ters. Two conditions for designing a stable high-speed TCP-RED system for a

high capacity Gigabit network are also derived. Finally, Lyapunov Exponents

and Bifurcation diagrams have been employed to investigate the dynamical

behaviour of AQM systems.

Key Words. High-speed variants of standard TCP, RED, AQM, and Stability.

1. Introduction

The current best-effort Internet has been ubiquitously used around the world. Itssuccessful operation mainly depends upon the use of TCP for data transfer betweenend hosts, i.e. data sender and receiver. It has predominantly deployed two versionsof TCP, Reno and Sack, which have been evolving in the last two decades. TCP overInternet Protocol (IP) provides necessary stability and robustness to the currentInternet. The operation of TCP depends on four intertwined algorithms: slow start,congestion avoidance, fast retransmit and fast recovery, [1]. Whereas, amount ofdata pumped into network is mainly controlled by congestion avoidance algorithm,[1] and [2].

A number of intermediate routers are usually involved in establishing and main-taining TCP connection(s) between sender(s) and receiver(s) of data. The simplestpossible type of a router is Droptail, which starts to drops the incoming packetsafter its receive buffer has become completely full. However, a significant prob-lem associated with Droptail is flow synchronization [3]. Also, it cannot supportQuality-of-Service applications such as in DiffServ architectures, see [4] and its ref-erences. Therefore, in order to overcome weaknesses of Droptail and efficientlyutilize the network resources, Random Early Detection RED was proposed in [3].A RED router while working in conjunction with TCP, can form a closed loopfeedback system, that is widely known as Active Queue Management (AQM). Atthe onset of congestion, an AQM control loop can perform dropping or marking ofpackets as congestion signals to data sender(s). Whereof receiving the proactivelymarked acknowledgement packet the sender will decrease the congestion window (aTCP variable that controls the the number of data packets that can be send intothe network for transmission).

Received by the editors on November 10, 2009.An initial version of this work has been published in ICMA 2007, [9].

189

190 A. HAIDER, H. SIRISENA, V. SREERAM, AND R. HARRIS

Recently, due to major advancements in router hardware and fiber optical baseddata transmission link technologies, there has been a wide spread trend of upgrad-ing of existing as well as deployment of new high-speed networks, e.g. [5], [6], [7]and [8]. These high-speeds networks are being used for various data intensive appli-cations, such as High Energy/Nuclear Physics, Bio-Informatics and Telemedicine,[10]. These networks need to transfer a huge amount of data over Gigabit linksfor which the performance of ordinary standard TCP is not satisfactory. For in-stance, a standard TCP connection will take 1 2

3 hours to fully utilize a 10 Gb/s link,[11]; which is highly time consuming and thus undesirable. Therefore, in order toefficiently utilize the available bandwidth, new data transfer protocols are required.

Towards this end, in about the last seven years, several variants of TCP for high-speed networks have been proposed in literature. These proposals can be classifiedinto following three categories: (i) pure packet loss based, such as HighSpeed TCP[11], Scalable TCP [12] and Binary Increase Congestion Control (BIC) [13] (ii) delaybased approaches such as Fast TCP [14] (iii) combination of loss and delay, suchas Compound TCP [15]. This paper only considers loss and loss-delay combinationapproaches for high-speed TCP, as pure delay based congestion control has severeproblems in fair sharing of bandwidth with loss based algorithms, [15]. It extendsour previous work by generalizing the modelling and analysis of stability conditionsderived for Scalable TCP-RED based AQM, [9]. The basic notation and modelused in this paper was originally proposed in [16] in the context of standard TCP,which has also been used in [17] for the bifurcation analysis of AQM formed bystandard TCP and RED router.

The rest of the paper is organised as follows: dynamics of congestion control forstandard and high-speed variants of TCP are presented in Section 2. A discretetime model of high-speed TCP-RED is developed in Section 3. The conditions forstability are derived in Section 4. Case studies are provided in Section 5. Finally,conclusions and future work are presented in Section 6.

2. Dynamics of congestion control

2.1. Standard TCP. The operation of congestion avoidance algorithm in stan-dard TCP can be summarized in the following two steps [1]:

• For each acknowledgement packet, update the congestion window Wr, asWr ←Wr +

ar

Wr;

• For packet drop/mark, update Wr as Wr ←Wr − br ·Wr and for acknowl-edgements during slow start Wr ← Wr + cr ;

where ar = 1, br = 0.5 and cr = 1; with all quantities expressed in maximumsegment size (packets). It is also known as additive-increase multiplicative-decreaseof the congestion window. A typical congestion window plot for a standard TCPconnection, operating in the congestion avoidance phase, is shown in Fig. 1. Byusing a sawtooth series, as shown in Fig. 1, the congestion window, Wr, andthroughput (amount of data transferred successfully), Tr (Packets/s), for packetdrop/mark probability of p and round trip time R, has been derived in [18] as:

(1) Wr =

√

2arbr(2 − br)

·1

p,

(2) Tr =

√

ar(2 − br)

R√2brp

≡

√1.5

R√p.

ON STABILITY CONDITIONS FOR HIGH-SPEED VARIANTS OF TCP AND RED BASED AQM191

W

W(1−b)

W(1−b)+a

Congestion Window

Time

W(1−b)+2a

1R

2R

Figure 1. Standard TCP in congestion avoidance phase.

2.2. High-speed variants of TCP. In order to obtain a generalized expressionfor stability of high-speed variants of TCP, synchronised packet loss model hasbeen employed [13]. In this model all high-speed TCP connections, competingfor bottleneck link bandwidth, simultaneously experience the loss events. Thus,each high-speed TCP connection with different round trip time will experience adifferent loss rate. It is in contrast to uniformly distributed packet loss model,where all connections regardless to their round trip times will obtain same packetloss rate. In general, throughput of high-speed variants of standard TCP, withsynchronized loss model, can be elegantly expressed as [13]:

(3) Th =1

R·Kh

pd,

where Kh and d are constants whose value depend upon the type of high-speedTCP, as has been summarized in Table 1. Both (3) and (2) give throughput in

Type of high-speed TCP Kh d

Standard TCP [1] 1.22 0.5Scalable TCP [12] 0.08 1HighSpeed TCP [11] 0.15 0.82BIC TCP [13] - [0.5, 1]Compound TCP [15] - 0.75

Table 1. Values of Kh and d for TCP variants for high-speed networks.

Packets/s, which can be converted into bits/s by multiplying them with maximumsegment (Packet) size M (bits). A snapshot of variations in congestion window anda plot of response function, equation (3), for major types of high-speed TCPs havebeen shown in Fig. 2 and 3 respectively.


0

1000

2000

3000

4000

5000

6000

0 5 10 15 20 25 30

Con

gest

ion

Win

dow

(pa

cket

s)

Time (s)

HighSpeed TCP Scalable TCP

BIC TCP Compound TCP

Figure 2. A snapshot of congestion avoidance phase in HighSpeed[11], BIC [13] and Compound TCP [15].

0.1

1

10

100

1000

10000

100000

1e+06

1e+07

1e+08

1e+09

1e-10 1e-09 1e-08 1e-07 1e-06 1e-05 0.0001 0.001 0.01 0.1

Sen

ding

rat

e (P

acke

ts/R

TT

)

Loss rate p

Standard TCPHighSpeed TCP

Scalable TCPCompound TCP

Figure 3. Response function, plotted on logarithmic scale, forhigh-speed TCPs.

3. Discrete time Model for High-Speed TCP-RED System

Let us consider the feedback control loop formed by high-speed TCP and a REDrouter, as shown in Fig. 4. In order to describe the behavior of this closed loop sys-tem, we consider discrete time intervals, tk+1 = tk +R, where R consists of a prop-agation delay and queuing delay. The packet mark/drop probability in any giventime slot, tk, is pk. On receiving the congestion signal (through marked/droppedpackets), the sender reduces its congestion window. Thus, variations of the queueat the RED router are governed by qtk+1

≡ qk+1 = G(pk). RED computes a new es-timate of exponential weighted moving average, q e,k+1, of the instantaneous queuesize qk+1, i.e. q e,k+1 = A(q e,k, qk+1). After computing q e,k+1, RED computes a


Data receiverRED

Router

Data Packets Routed Packets

Acknowledgement Packets

1

qmin

qmax

q

Round Trip Time, R

pmax

−

Data Sender

High−speed TCP

Figure 4. Feedback closed loop formed by high-speed TCP andRED router.

new value of pk+1 as pk+1 = H(q e,k+1). Therefore, we have the following set ofequations to describe the closed loop shown in Fig. 4:

qk+1 = G(pk),

q e,k+1 = A(q e,k, qk+1),

pk+1 = H(q e,k+1).(4)

For a link capacity of C (bits/s), maximum queue size (buffer capacity) of B (pack-ets), fixed number of connections N and packet size M , the following idealizedqueue law for high-speed TCP, modified from that proposed in [16] for standardTCP, is as follows:

(5) qk+1 = G(pk) ≡

{

max(

B, CM

(

MKhCN

·pdk

−R0

))

, forpk ≤ p0;

0, otherwise;

where, p0 is maximum value of probability for which link capacity is fully utilizedand R0 is the corresponding round trip time. The physical interpretation of thisequation is that, assuming the buffer capacity B is not exceeded, the number ofpackets in the buffer is equal to the window size less the number of packets in-flight,i.e. the product of the networks capacity in packets/s times the round-trip time.

In a real system (with time varying N , M and R0), for the maximum valuesof N , M and minimum value of R0, (5) is called “maximum queue law”; whereasfor the minimum values of N , M and maximum value of R0 it is called “minimumqueue law”, [16]. However, the actual queue law can be in between the minimumand maximum values. The behavior of a RED router, i.e. pk+1 = H(q e,k+1), canbe described by the following:

(6) pk+1 =

0, if q e,k+1 < qmin;

1, if q e,k+1 ≥ qmax;q e,k+1−qmin

qmax−qmin· pmax, if q e,k+1 ∈ [qmin, qmax],

where average of queue size variations, q e,k+1, is computed by following recursion:

(7) q e,k+1 = A(q e,k, qk+1) ≡ (1 − w)q e,k + w · qk+1,


where w is queue averaging weight, whose value can be set as, e.g., in [3] and [21].Also, we have 0 < pmax < 1 and the following heuristic, [3]:

(8) qmax − qmin = 2qmin.

Thus, the system formulated in (4) has been completely described by (5), (6) and(7). For large buffer B, we can write (5) as:

(9) qk+1 =C

M

{(

MKh

CN· pdk

)

− R0

}

.

Substituting (9) in (7) we get:

(10) q e,k+1 = (1 − w)q e,k + w ·

{(

NKh

pdk

)

−R0C

M

}

.

Next, plugging (6) in (10) results in:

(11) q e,k+1 = (1− w)q e,k + w

[

NKh ·

{

qmax − qmin

pmax(q e,k − qmin)

}d

−R0C

M

]

.

Simplifying (11) we have:

(12) q e,k+1 = q e,k + w

[

NKh ·

{

qmax − qmin

pmax(q e,k − qmin)

}d

−

(

R0C

M+ q e,k

)

]

.

The “equilibrium value” or “fixed point” in (12) occurs when q e,k+1 = q e,k ≡ q ∗,[17], giving the following condition:

(13) NKh ·

(

qmax − qmin

pmax

)d

=

(

q ∗ +R0C

M

)

(q ∗ − qmin)d,

where 0.5 ≤ d ≤ 1. In the case d = 0.5, i.e. standard TCP, we will have thefollowing expression:

(14) (q ∗ − qmin)

(

q ∗ +R0C

M

)2

=(NK)2

pmax

(qmax − qmin)

For d = 1, i.e. Scalable TCP case, we can have:

(15) q ∗2 + q ∗

(

R0C

M− qmin

)

−

{

R0Cqmin

M+

NKs

pmax

· (qmax − qmin)

}

,

which is in contrast to the cubic equation derived in [17], for the standard TCP-RED case. After substituting (8) in (15), an expression for q ∗ can be obtainedas:

q ∗ = −1

2

(

R0C

M− qmin

)

+1

2

√

(

R0C

M− qmin

)2

− 4

(

R0Cqmin

M+

2NKsqmin

pmax

)

.

(16)

where the positive square root is selected on the RHS of (16) because q∗ > 0 1.

1For further insight, expressions for (q ∗− qmin)

2 can be obtained through (16); i.e. assuming

k1 = R0C

Mand k2 = 2Ks

pmaxwe can have:

(17) (q ∗− qmin) = −

1

2(k1 + qmin)±

1

2

√

(k1 − qmin)2− 4qmin (k1 +Nk2),

which can be further manipulated, by using using binomial expansion, as:

(18) (q ∗− qmin)

2 =

{

k21 − qmin(k1 +Nk2)(1 +K1 + qmin), for positive root in (16)

q2min − qmin(k1 +Nk2)(1 + k1 + qmin), for negative root in (16);


4. Stability Analysis

Stability of high-speed TCP-RED system, (4), can be analyzed by linearizing(11) at the fixed point q ∗ and determining the eigenvalue, [17]:

(19)∂q e,k+1

∂q e,k

∣

∣

∣

qe,k=q ∗

= 1− w −

(

qmax − qmin

pmax

)dwNdKh

(q e,k − qmin)d+1.

which after substitution of (8) can be written as:

(20)∂q e,k+1

∂q e,k

∣

∣

∣

qe,k=q ∗

= 1− w −

(

2qmin

pmax

)dwNdKh


The eigenvalue given by (20) depends upon the fixed point q ∗ as defined by (15).In order that (4) be stable, the eigenvalue must lie inside the unit circle, giving thefollowing condition for stability:

(21)

∣

∣

∣

∣

∣

1− w −

(

2qmin

pmax

)dwNdKh

(q e,k − qmin)d+1

∣

∣

∣

∣

∣

< 1.

There will be a period doubling bifurcation with oscillations when the eigenvalue,(19), becomes equal to -1, [17]. The value of the queue weight, w, for whicheigenvalue equals -1 is given by:

(22) wcrit =2

1 +(

2qmin

pmax

)d

· NKhd(q ∗−qmin)d+1

.

To analyze the nature of bifurcations we consider a function S as defined in [22]:

(23) S =1

2

(

∂2q e,k+1

∂q e,k2

)2

+1

3

(

∂3q e,k+1

∂q e,k3

)

.

Taking second and third order derivatives in (11) we get:

(24)∂2q e,k+1

∂q e,k2

∣

∣

∣

qe,k=q ∗

=

(

2qmin

pmax

)dwNd(d+ 1)Kh


(25)∂3q e,k+1

∂q e,k3

∣

∣

∣

qe,k=q ∗

= −

(

2qmin

pmax

)dwNd(d+ 1)(d+ 2)Kh


Substituting (24) and (25) in (23) we obtain:

(26) S =1

2

(

∂2q e,k+1

∂q e,k2

)2

+1

3

(

∂3q e,k+1

∂q e,k3

)

.

A positive value of S indicates that the bifurcation is supercritical, whereas anegative value indicates subcritical region which represents discontinuity and is notdesired. The first term in (26) will be always positive. Thus, for S > 0 we canhave:

(27)

{

2NKswqmin

(q ∗ − qmin)2pmax

− 1

}

> 0,

which gives the following condition:

(28) (q e,k − qmin) <

(

3

2

qmin

pmax

wNd(d+ 1)Kh

d+ 2

)1

d+1

that can be substituted in (22) for d = 1, to obtain wcrit independent of q ∗.


In a real network scenario with non-adaptive RED [3], each of Ks, w, qmin and pmax

is a constant, whereas the number of high-speed TCP connections vary with time,i.e. N = N(tk) ≡ Nk. Hence, we can write (28) as:

(29) N >2

3

pmax

qmin

(q e,k − qmin)d+1

wd(d+ 1)Kh

where q ∗ is an implicit function of filter weight w via (15). To summarize, in orderthat AQM comprising high-speed TCP and RED be stable, both (21) and (29)must be satisfied.

5. Case Studies

In order to further investigate the behavior of dynamical system defined by(10), we employ Lyapunov Exponents (LEs), [22] pp. 283. They determine theexponential rates at which the nearby trajectories deviate from each other withtime. Consider two initial conditions x0 and x0 + δx0 for a dynamical systemdefined by f(x), which map to x1 and x1 + δx1; i.e. in general xn−1 + δxn−1

mapping to xn + δxn, respectively. Thus, we can have

(30) δxn = f ′(xn−1)δxn−1,

which can be further written as

(31)∣

∣

∣

δxn

δx0

∣

∣

∣=

n−1∏

i=0

|f ′(xi)|.

Assuming that (31) varies exponentially at for large values of n, we have:

(32)∣

∣

∣

δxn

δx0

∣

∣

∣= eλLn,

where λL is called Lyapunov Exponent, which is given by the following expression:

(33) λL = limn→∞

1

n

n∑

i=1

ln |f ′(xi)|.

λL < 0 indicates stable fixed point or stable periodic orbit for a non-conservativesystem having asymptotic stability. The superstable fixed points and periodic or-bits have λL = −∞. λL = 0 is a neutral fixed point for a conservative systemhaving Lyapunov stability. λL > 0 indicates instability and chaos, see [22] andhttp://hypertextbook.com/chaos/43.shtml

5.1. High-Speed Variants of TCP. We consider a system deploying high-speedvariants of TCP and RED router, as depicted in Fig. 5, and analyze its stabilitycharacteristics by using LEs and Bifurcation diagrams. In this setup we have chosenqmin = 500, qmax = 1500 packets, pmax = 0.1, bottleneck link capacity C = 2.4Gbps, round trip time of 0.1 s, number of connections N = 10 and maximum packetsize of 4000 bytes, [20].

Figure 5. Dumbbell Network Topology.


LEs for HighSpeed, Scalable, and Compound TCP-RED have been computed bysolving (33) with Mathematica package. For instance, the LEs for Compound TCPfor different values of R0C/M have been shown in Fig. 6. It has been observed thatfor each type of high-speed TCP-RED system the LEs have almost similar shapecharacteristics and these plots are always positive for 0 < w < 1; thus correspondingto system that has a stable fixed point or stable periodic orbits.

0.2 0.4 0.6 0.8 1.0w

-4

-3

-2

-1

Λ

Figure 6. LEs for Compund TCP [15].

It also shows that high-speed TCP-RED based system is more stable for highervalues of w. It corroborate the results presented for TCP-RED in [23], whichsuggests that 0.5 < w < 1 for smoother queue variations and reduced jitter (latencyin packet arrival) for the buffered packets. A bifurcation diagram is also often usedto investigate the behaviour of a dynamical system. It indicates the qualitativechanges in the nature and number of fixed points of a system whose parameters arebeing varied quasistatically. The horizontal axis shows the parameter being variedand vertical axis indicates the measure of steady states(equilibria/fixed points). Thebifurcation diagrams, for various values of round trip time, for different high-speedTCPs have been plotted in Fig. 7. It has been found that high-speed TCP-REDbased AQM start to show the bifurcation behaviour for R ∈ [0.01ms, 1ms] andthe plot tends to move toward the vertical axis for smaller values of R. Further,for comparison and completion purpose, we have also studied the case of standardTCP-RED based AQM.

5.2. TCP Reno/SACK. Consider the case of ordinary TCP employing REDin a dumbbell topology, as in Fig. 5, with qmin = 250, qmax = 750 packets,pmax = 0.1, bottleneck link capacity C = 75 Mbps, round trip time of 0.1 s, numberof connections N = 250 and maximum packet size of 4000 bytes, [17]. Using (33),the LEs are computed and plotted in Fig. 8(a). Next, in order to compare with thehigh-speed TCP-RED system, we set qmin = 500, qmax = 1500 packets and N = 10(with other parameters same) and plot LEs in Fig. 8(b). These plots show that LEsfor ordinary TCP-RED based system become positive for certain range of values of


0.04 0.05 0.06 0.07

520

540

560

580

600

(a)

0.035 0.040 0.045 0.050 0.055 0.060

520

540

560

580

600

(b)

0.015 0.020 0.025

510

520

530

540

550

560

(c)

0.08 0.10 0.12 0.14w

450

500

550

600

650

700

Average Queue Size

(d)

0.06 0.08 0.10 0.12w

450

500

550

600

650

700

Average Queue Size

(e)

0.01 0.02 0.03 0.04 0.05 0.06w

500

550

600

Average Queue Size

(f)

0.015 0.020 0.025

550

600

650

(g)

0.012 0.014 0.016 0.018 0.020 0.022

550

600

650

(h)

0.0105 0.0110 0.0115 0.0120

600

700

800

900

1000

1100

(i)

Figure 7. Bifurcation plots for various high-speed TCP-REDbased AQMs for R0=1, 0.1 and 0.01 ms: (a), (b) and (c) for High-Speed TCP [11]; (d), (e) and (f) for Scalable TCP [12]; (g), (h)and (i) for Compound TCP [15], respectively.


0.1 0.120.140.160.18 0.2 0.220.24-1.5

-1

-0.5

0

0.5

w

ΛL

(a)

0 0.05 0.1 0.15 0.2 0.25 0.3

-6

-5

-4

-3

-2

-1

0 w

ΛL

(b)

0.18 0.20 0.22 0.24w

600

800

1000

Average Queue Size

(c)

Figure 8. LEs and Bifurcation diagrams for standard TCP-REDbased AQM [1]; whereas, for case (a): qmin = 250, qmax = 750packets and for cases (b) and (c): qmin = 500, qmax = 1500 pack-ets.

queue weight w, thus causing instability. However, a high-speed TCP-RED basedsystem avoids such behaviour. Also, in contrast to high-speed TCP, bifurcationbehaviour can be observed for higher values of round trip time.

6. Conclusions and Future work

The dynamical behaviour of AQM formed by high-speed TCP-RED has beenanalyzed by developing a discrete time nonlinear model. A generalized equationdescribing the behaviour of the fixed point has been developed. Special case forScalable TCP has been also analyzed. Further, two conditions for designing sta-ble high-speed TCP-RED based AQM have been derived. The stability of ordi-nary TCP and high-speed TCPs with RED routers has been investigated by usingLyapunov Exponents and Bifurcation diagrams. It has been shown that ordinaryTCP-RED based systems are less stable than high-speed TCP-RED. Also, it hasbeen found that high-speed TCP-RED systems are more stable for higher values ofqueue weight, w ; a fact which has also been pointed out for ordinary TCP-REDsystems in [23].

Further, we plan to carry out packet level simulations on high-speed TCP-REDsystem to further investigate the stability conditions and also corroborate the infer-ences drawn from Lyapunov Exponents and bifurcation plots as presented in thispaper.

References

[1] M. Allman, V. Paxon and W. Steven, TCP Congestion Control, Request for comments, RFC2581, Category: Standards track, April 1999, http://www.ietf.org/rfc/rfc2581.txt

[2] G. Vinnicombe, On the Stability of Networks Operating TCP-Like Congestion Control, 2002,http://www-control.eng.cam.ac.uk/gv/internet/index.html

[3] Sally Floyd and Van Jacobson, Random Early Detection Gateways for Congestion Avoidance,IEEE/ACM Transactions on Networking, Vol. 1, No. 4, pp. 397-413, August, 1993.

[4] Ikjun Yeom, A. L. Narasimha Reddy, Modeling TCP Behavior in a Differentiated ServicesNetwork, IEEE/ACM Transactions on Networking, Vol. 9, No. 1, pp. 31-46, February 2001.

[5] Internet 2, http://www.internet2.edu/


[6] CERN DataTag, http://datatag.web.cern.ch/datatag/index.html[7] Japan Gigabit Network, http://www.jgn.nict.go.jp/english/index.html[8] Kiwi Advanced Research and Education Network (KAREN), http://www.karen.net.nz/[9] A. Haider, H. Sirisena, V. Sreeram, R. Harris, Stability Conditions For Scalable TCP-RED

Based AQM, International Conference on Mechatronics and Automation, pp. 576-581,5-8Aug. 2007, DOI: 10.1109/ICMA.2007.4303607.

[10] Maxine D. Brown, Blueprint for the Future of High Performance Networking, Communica-tions of the ACM, Vol. 46, No. 11, pp. 30-33, November 2003.

[11] S. Floyd, HighSpeed TCP for Large Congestion Windows, RFC 3649, December 2003, Cat-egory: Experimental, http://www.ietf.org/rfc/rfc3649.txt

[12] T. Kelly, Scalable TCP: Improving Performance in HighSpeed Wide Area Networks, ACMSIGCOMM Computer Communication Review, Vol. 33, No. 2, pp. 83-91, April 2003,http://www-lce.eng.cam.uk/ ctk21/scalable/#publications

[13] L. Xu, K. Harfoush and I. Rhee, Binary Increase Congestion Control (BIC) for Fast Long-Distance Networks, Proc. of IEEE INFOCOM, Vol. 4, pp. 2514-2524, 2004.

[14] Cheng Jin, D. X. Wei and S. H. Low FAST TCP: Motivation, Architecture, Algorithms,Performance Proc. of IEEE INFOCOM, Vol. 4, pp. 2490-2501, 2004.

[15] K. Tan, J. Song, Q. Zhang and M. Sridharan, Compound TCP: A Scalable and TCP-friendly Congestion Control for High-speed Networks, Proc. of PFLDNet, 2006, Nara, Japan,http://www.hpcc.jp/pfldnet2006/technical.html

[16] Victor Firoiu and Marty Borden, A Study of Active Queue Management for CongestionControl, IEEE INFOCOM’00, Vol. 3, pp. 1435-1444, March 2000.

[17] Priya Ranjan, Eyad H. Abed, and Richard J. La, Nonlinear Instabilities in TCP-RED,IEEE/ACM Transactions on Networking, pp. 1079-1092, 2004.

[18] Sally Floyd, Mark Handley and Jitendra Padhye, A Comparison of Equation-Based and AIMD Congestion Control, Preliminary version, May 2001, Available athttp://www.icir.org/tfrc/aimd.pdf

[19] Yee-Ting Li and Douglas Leith and Robert N. Shorten, Experimental Evaluation of TCPProtocols for High-Speed Networks IEEE/ACM Transactions on Networking, Vol. 15, No. 5,pp. 1109-1122, 2007.

[20] Harsha Sirisena, Aun Haider and Victor Sreeeram, Control-Theoretic Design of RED AQMfor Scalable TCP in High Speed Networks, IEEE ICC, Vol. 1, pp. 177-182, June 2006.

[21] Harsha Sirisena, Aun Haider and Krzysztof Pawlikowski, Auto-Tuning RED for AccurateQueue Control, Proceedings of IEEE Globecom ’02, Vol. 2, pp. 2010-2015, 2002.

[22] J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcationsof Vector Fields, Springer Verlag, Theorem 3.5.1 pp. 158, 1983.

[23] Archan Misra, Teunis Ott and John Baras, Effect of Exponential Averaging on the Variabilityof a RED Queue, Proc. of IEEE ICC 2001, Vol. 6, pp. 1817 - 1823.

School of Engineering and Advanced Technology, Massey University, Palmerston North, NewZealand


ECE, University of Canterbury Christchurch, New ZealandE-mail : [email protected]

EECE, University of Western Australia Perth, AustraliaE-mail : [email protected]

School of Engineering and Advanced Technology, Massey University, Palmerston North, NewZealand


Received by the editor, September 12, 2008, and in revised form on April 14, 2010 201

INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCES Volume 6, Number 2 , Pages 201-219

©2010 Institute for Scientific Computing and Information

DESIGN OF OPTIMAL SELF-TUNING REGULATORS

FOR LARGE-SCALE STOCHASTIC SYSTEMS

SAMIRA KAMOUN

Abstract This paper deals with the optimal self-tuning regulators of large-scale stochastic systems with unknown and time-varying parameters, which is composed of several interconnected systems. We consider the class of interconnected systems, which are described by linear, single-input single-output and stochastic input-output mathematical models with unknown and time-varying parameters. Three self-tuning regulators for optimal control of large-scale stochastic systems are developed on the basis upon the generalized minimum variance approach with explicit and implicit schemes. An explicit self-tuning regulator for a variance constrained optimal control of large-scale systems is proposed. A numerical example is treated which demonstrates the effectiveness of developed theoretical results. Key Words: Large-scale stochastic systems; Input-output mathematical models; Self-tuning regulators; Generalized minimum variance regulation; Constrained optimal control.

1. Introduction

There exists an extensive and ever-increasing literature concerning analysis and control of large-scale systems with known parameters (see, e.g., [12]). However, a few results have been developed dealing with some schemes for self-tuning (or adaptive) control of large-scale systems with unknown and/or time-varying parameters on the basis, as often as not, upon decomposition or decentralization techniques (see, [5], [6]).

The self-tuning (or adaptive) control of stochastic large-scale systems with unknown but constant or time-varying parameters is a problem of theoretical and practical importance. To date, some approaches have been suggested under different assumptions on mathematical models, interconnections, disturbances, etc. of large-scale systems. In this case, the developed results can be classified principally in three groups according to the assumption on the description of the large-scale systems by mathematical models, i.e., continuous state-space mathematical models, discrete state-space mathematical models or input-output mathematical models. Notice therefore that the number of published results concerning large-scale systems, which are described by input-output mathematical models, is small enough.

Since the first introduction of the self-tuning regulator by Åström and Wittenmark [2], which automatically adjust regulator parameters on-line in response to change in the process or the environment, many theoretical and practical results (in particularly for single-input single-output systems) are presented in the literature. A good analysis of the properties of the self-tuning control can be obtained from Clarke [3]. More recent results are discussed in [9] and [11].

202 S. KAMOUN

In the optimal stochastic control approach, it is often well motivated to let the criterion for control be to minimize the variance of the system output. Thus, we obtain the so-called minimum variance regulator [1], which can be applied only for minimum phase systems. Indeed, this regulator causes an unstable control law for non-minimum phase systems. To overcome this problem, we can use the generalized minimum variance approach, which was developed initially by Clarke, and Gawthrop [4] for single-input single-output systems by minimizing the system output with control costing. Some results in this area are developed in [9].

Design methods for self-tuning regulators have been based usually on the Recursive Extended Least Squares (RELS) method for determining the system parameters (explicit control scheme) or the regulator parameters (implicit control scheme). Parametric estimation of a certain class of large-scale systems using discrete-time schemes was discussed [7]. They considered deterministic and stochastic systems which are connected between them with outputs signals and described by input-output mathematical models, and proposed recursive parametric estimation algorithms based upon the recursive least squares method. In the same way, Kamoun and Titli [8] have been considered this class of large-scale systems and developed and implicit self-tuning controller.

This paper develops a self-tuning regulation of a winder class of large-scale stochastic systems, which is constituted by several interconnected systems, where the interconnection function of each interconnected system is defined by input and output signals of other systems. The interconnected stochastic systems are described by linear input output mathematical models with unknown and time-varying parameters. In this case, we extend the recursive extended least squares method and the generalized minimum variance approach for single-input single-output systems to cover this class of interconnected systems.

The rest of the paper is organized into 4 sections. Section 2 deals with a description of the large-scale by input-output mathematical models, which can be used in the digital control schemes, and some of their properties. A recursive parametric estimation algorithm of large-scale systems is described by using the extended least squares method. Section 3, which is the principal part of the paper, develops recursive algorithms for self-tuning regulation of large-scale systems by using the generalized minimum variance approach. The minimum variance control problem with input energy constraints is treated. A numerical example is given in Section 4. Conclusions are drawn in Section 5. 2. Problem formulation and parametric estimation

The practical importance of recursive algorithm for the design of self-tuning systems is essentially due to the introduction and diffusion of digital computers with ever increasing computational power and less costly-implemented parametric estimation or control schemes. For this, the exposition of this paper is limited to large-scale systems, which are described by discrete-time mathematical models (particularly input-output mathematical models).

To illustrate the conceptual ideas of our approach to optimal self-tuning regulation, we will consider a large-scale system S , which is constituted of N interconnected stochastic time-varying systems NSS ,,1 . Each interconnected system iS , Ni <<1 ,

OPTIMAL SELF-TUNING REGULATORS FOR STOCHASTIC SYSTEMS 203

can be described by the following Interconnected Autoregressive Moving Average eXogenous (IARMAX) mathematical model:

(1) ∑+∑+

+=

≠=

−−

≠=

−−

−−−−

N

ijjjij

tN

ijjjij

d

iiiid

ii

kykqAqkukqBq

keqCkukqBqkykqA

ijij

i

,1

1

,1

1

111

)(),()(),(

)()()(),()(),(

where +∈ Zk denotes the discrete-time variable, )(kui , )(kyi and )(kei represent the input, the output and the noise (defined more precisely later on) of the interconnected system iS , respectively, )(ku j is the inputs and )(ky j is the outputs of the other systems jS , Nj ,,1= , ij ≠ , 1−q is the backward shift operator (i.e.,

)1()(1 −=− kukuq ii ), id is the dead-time (in an integral number of sample intervals), ijd and ijt represent the interaction time-delay corresponding respectively to inputs and

outputs of other systems jS , ),( 1 kqAi− , ),( 1 kqBi

− , ),( 1 kqBij− and ),( 1 kqAij

− are unknown and time-varying polynomials, and )( 1−qCi is unknown polynomial, which are defined by:

(2) ii

nAnAiii qkaqkakqA −−− +++= )()(1),( ,

11,

1

(3) ii

nBnBiii qkbqkbkqB −−− ++= )()(),( ,

11,

1

(4) ijij

nBnBijijij qkbqkbkqB −−− ++= )()(),( ,

11,

1

(5) ifif

nAnAifififij qkaqkakakqA −−− +++= )()()(),( ,

11,0,

1

and

(6) ii

nCnCiii qcqcqC −−− +++= ,

11,

1 1)(

with 1)(0, =kaif , if if = and 0)(0, =kaif , if if ≠ ; Nfi ,,1, = . Here, the systems jS , Nj ,,1= , are connected via the interconnection polynomials

),( 1 kqBij− and ),( 1 kqAij

− . It is assumed that the noise )(kei can be modelled in terms of a stationary process with a rational spectral density, and without loss of generality, the polynomials ),( 1 kqAi

− , ),( 1 kqBi− , )( 1−qCi , ),( 1 kqBij

− and ),( 1 kqAij− are assumed

to have the same degree in , i.e. iijijiii nnAnBnCnBnA ===== . This assumption is not essential, and is made for the purpose of simplicity of representation.

In this representation, the composite system S can be described as:

(7) )()()(),()(),( 111 keqCkukqBkykqA −−− +=

where ),( 1 kqA − , ),( 1 kqB − and )( 1−qC are polynomial matrices, which are given by:

−−

−

−

−−

=

−−−

−−−

−−

−

−−−

−−−−−

−−

),(),(),(),(

),(),(),(),(),(

),(

111

11

11

12

121

11

112

11

1

111

1

21

112

kqAkqAqkqAqkqAq

kqAkqAqkqAqkqAqkqA

kqA

NNNt

Nt

NNt

tN

tt

N

NN

N

(8)

204 S. KAMOUN

(9)

=

−−−−

−−−

−−

−

−−−−

−−−−−−

−−

),(),(),(),(

),(),(),(),(),(

),(

111

11

11

12

121

11

112

11

1

111

1

221

1121

kqBqkqBqkqBqkqBq

kqBqkqBqkqBqkqBqkqBq

kqB

Nd

NNd

Nd

NNd

ddN

ddd

NN

NN

N

(10) )](,),([diag)( 111

1 −−− = qCqCqC N

For the present, let us make the following assumptions for each interconnected system iS , Ni ≤≤1 :

A1. the dimension in and the delays id , ijd and ijt are known exactly, and the parameters of polynomials ),( 1 kqAi

− , ),( 1 kqBi− , )( 1−qCi , ),( 1 kqBij

− and ),( 1 kqAij

− are unknown and time-varying; A2. the representation is chosen so that there is no common factor to all the polynomials

),( 1 kqAi− , ),( 1 kqBi

− , )( 1−qCi , ),( 1 kqBij− and ),( 1 kqAij

− ; A3. )}({ kei is a white Gaussian noise, which is constituted by an uncorrelated sequence

of random variables with zero mean and variance 2iσ ;

A4. the roots of )( 1−qCi are assumed to be strictly within the unit circle. This assumption is a mild restriction, cf. the representation and spectral factorisation theorems of Åström [1]. However, no corresponding assumption needs to be made concerning the roots of polynomials ),( 1 kqAi

− , ),( 1 kqBi− , )( 1−qCi , ),( 1 kqBij

− and ),( 1 kqAij

− . Therefore, the interconnected system iS may be open-loop unstable (roots of ),( 1 kqAi

− ) outside the unit circle) or non-minimum phase (roots of ),( 1 kqBi

− outside the unit circle); A5. the interconnected system iS is assumed to be stabilizable.

The ith interconnected system output variable in equation (1) may be expressed as:

(11)

)()()1()(

)()()1()(

)()1()(

)()()1()(

)()()1()()(

,1,

,1,

,1,

,1,

,1,

iijjnijijjij

iijjnijijjij

iiniiii

iiiniiii

iiniiii

ntkykatkyka

ndkukbdkukb

nkeckecke

ndkukbdkukb

nkykakykaky

i

i

i

i

i

−−−−−−+

−−++−−+

−++−++

−−++−−+

−−−−−=

with Nj ,,1= , ij ≠ . Our objective is to design recursive algorithms for solving optimal self-tuning

regulation of each interconnected stochastic time-varying system iS , Ni ≤≤1 , as described by (11).

The output )(kyi of the interconnected stochastic time-varying system iS can be expressed in the following compact form:

(12) )()()()( kekkky iiT

ii +ψθ=

where )(kiθ is a vector consisting of the parameters to be estimated and )(kiψ is a vector of delayed input, output and interaction. They are given by:


(13) )]()()()(

)()()()([)(

,1,,1,

,1,,1,,1,

kakakbkb

cckbkbkakak

ii

iii

nijijnijij

niiniiniiT

i

=θ

and

(14)

)]()1(

)()1( )()1( )()1( )()1([)(

iijjijj

iijjijjiii

iiiiiiiiT

i

ntkytky

ndkudkunkekendkudkunkykyk

−−−−

−−−−−−

−−−−−−−−−=ψ

with Nj ,,1= , ij ≠ . The posed problem consists in finding an estimate )(ˆ kiθ of the parameter vector

)(kiθ at each discrete-time k from the observation vector )(kiψ . However, this vector )(kiψ is not totally available to observe, since he is corrupted by the sequence noise

},,1);({ ii nggke =− , which is not measurable. In order to overcome this difficulty, we can use an approximation )(ˆ kiψ of )(kiψ such that:

(15)

)]()1(

)()1( )()1( )( )1( )()1([)(ˆ

iijjijj

iijjijjiii

iiiiiiiiT

i

ntkytky

ndkudkunkkndkudkunkykyk

−−−−

−−−−−ε−ε

−−−−−−−−−=ψ

with Nj ,,1= , ij ≠ and where )( gki −ε , ing ,,1= denotes the a priori prediction error. The a priori prediction error )(kiε can be defined as:

(16) )(ˆ)1(ˆ)()( kkkyk iT

iii ψ−θ−=ε

with )1(ˆ −θ ki the estimated parameters at discrete-time 1−k . It is to be noted that in observation vector )(kiψ the unmeasurable sequence noise },,1);({ ii nggke =− is replaced by its estimate },,1);({ ii nggk =−ε .

The problem is that the system parameters )(kiθ are unknown and time-varying. In spite of this, the goal is to seek )(ˆ kiθ based on experimental data, while assuming that: 1. the available information at each discrete-time k , iMk ,,1= , is defined as:

},,,1);(),(),(),({)( ijNjkykukykukI jjiii ≠== ; 2. the input signal )(kui is bounded and persistently exciting; 3. the sequence of normal random variables )}({ kei and the observation vector )(kiψ

are independent processes; 4. the block size iM is chosen to be greater ( ii nM >> ).

The parametric estimation problem must formulate on the basis upon the minimization of a performance index depending on the prediction error )(kiε , and by using the extended least squares method.

The problem of parametric estimation of the interconnected systems iS , Ni ≤≤1 , with unknown and time-varying parameters can be formulated by minimizing the following quadratic criterion:

(17) ∑ ελ==

−i iM

ki

kMiii kkMJ

1

2 )()(21)(

206 S. KAMOUN

where )(kiλ is an exponential forgetting factor, which may be computed, on-line as:

(18) )1()1()( iiiii kk λ−λ+−λλ=λ ∗

with 10 <λ< i , 10 <λ< ∗

i and 1)0(0 <λ< i . It is easy to see that the forgetting factor )(kiλ is such that: ∗λ<λ< ii k)(0 , for iMk ,,1= . Notice that the forgetting factor is

used to make it possible to follow time-varying parameters. Thus, it is clear that with 1)( <λ ki more recent data are weighted more than old data. It is noted that the

introduction of the forgetting factor )(kiλ in the parametric estimation algorithm is recommended for interconnected system iS , Ni ≤≤1 , with time-varying parameters. Thus, for interconnected system with unknown but constant parameters we may take

1)( =λ ki , i.e., all experimental data have the same weight. The Recursive Extended Least Squares (RELS) parametric estimation algorithm for

parametric estimation of interconnected stochastic system iS , Ni ≤≤1 , with time-varying parameters is described by:

(19)

)(ˆ)1(ˆ)()(

)(ˆ)1()(ˆ)(

)1()(ˆ)(ˆ)1()1(

)(1)(

)()(ˆ)()1(ˆ)(ˆ

kkkyk

kkPkk

kPkkkPkP

kkP

kkkPkk

iT

iii

iiT

ii

iT

iiii

ii

iiiii

ψ−θ−=ε

ψ−ψ+λ

−ψψ−−−

λ=

εψ+−θ=θ

Convergence of stochastic parametric estimation algorithm is a fundamental objective in the design of recursive estimation schemes. Indeed, the major problem due to the use of the RELS parametric estimation algorithm (19) for estimate the parameter vector

)(kiθ of the interconnected stochastic system (12) is the question of convergence properties, i.e. the existence of necessary condition such that the estimated parameter vector )(ˆ kiθ converges towards )(kiθ with probability 1 as ∞→k .

The convergence analysis of recursive parametric estimation algorithms for large-scale systems composed of interconnected stochastic systems with constant parameters has been studied in [7] via the ordinary differential equation. In this way, it can be shown that a necessary condition to obtain strongly consistent estimates for the RELS parametric estimation algorithm is that the polynomial )( 1−qCi satisfies the following condition:

(20) 021

)(1

1 >−−qCi

Notice that the above convergence condition (20) will hold if the considered assumptions are respected. 3. Development of optimal self-tuning regulators

In this section, we develop three optimal self-tuning regulators for interconnected stochastic systems iS , Ni ≤≤1 , with time-varying parameters. An attempt is made to combine the problems of the recursive estimation and regulation of each interconnected system iS , Ni ≤≤1 , defined by (11).


Remarkable progress in the use of self-tuning regulator schemes in many industrial applications was achieved by introducing minimum variance approach. The reduction of random variations can, for example, lead to great energy savings. Suitable minimum variance self-tuning regulators can be developed by minimizing a criterion, penalizing system outputs and inputs with weighting factors.

In order to develop an optimal self-tuning regulator for each interconnected stochastic system iS , Ni ≤≤1 , we shall assume the following: 1. the delays ijd and ijt of interaction are such that: iij dd > and iij dt ≥ . This statement

is necessary to compute the control law )(kui ; 2. the coefficient )(1, kbi of the polynomial ),( 1 kqBi

− is assumed to be non-zero; 3. the sequence noise )}({ kei is independent of the sequence data

},,,1);(),(),(),({ ijNjkykukyku jjii ≠= . We now want to show that the subsystem output may be rewritten as:

(21)

∑ ++−+

∑ ++−+

++++=++

≠=

−−

≠=

−−

−−

−

−

−

N

ijjiijjij

i

N

ijjiijjij

i

iiiii

ii

i

iii

dtkykqAkqA

ddkukqBkqA

dkekqFkekqAkqG

kukqAkqqB

dky

,1

11

,1

11

11

1

1

1

)1(),(),(

1

)1(),(),(

1

)1(),()(),(),(

)(),(),(

)1(

where ),( 1 kqFi− and ),( 1 kqGi

− are polynomials obtained from the following diophantine equation:

(22) ),()(),(),( 11111 kqGqqCkqFkqA id

iiii −−−−−− −=

The polynomials ),( 1 kqFi− and ),( 1 kqGi

− are given by, respectively:

(23) ii

ddiii qkfqkfkqF −−− +++= )()(1),( ,

11,

1

and

(24) 11,

11,0,

1 )()()(),( +−−

−− +++= ii

nniiii qkgqkgkgkqG

3.1. Generalized minimum variance self-tuning regulation

Our objective is to find an admissible control sequence },,1);({ ii Mkku = such that the following performance index is minimized:

(25) )]()1([)1( 22 kudkydkJ iiiiii α+++=++ Ε

where Ε denotes the expectation and iα is a weighting factor ( 0>αi ). Let us now obtain an equivalent expression for )1( ++ dkyi in equation (21) by

explicating the parameter )(1, kbi . It is straightforward to show that:

(26) )1()()()()()1( 11, ++++=++ −

iiiiiii dkeqFkTkukbdky

where )(kTi is given by:

208 S. KAMOUN

(27)

∑ ++−+

∑ ++−+

+−=

≠=

−−

≠=

−−

−

−

−

−

N

ijjiijjij

i

N

ijjiijjij

i

ii

ii

i

ii

dtkykqAkqA

ddkukqBkqA

kekqAkqG

kukqAkqL

kT

,1

11

,1

11

1

1

1

1

)1(),(),(

1

)1(),(),(

1

)(),(),(

)1(),(),(

)(

with the polynomial ),( 1 kqLi− described as:

(28) 1

1,,2

1,1,,

11,2,3,1,1,2,

1

)()())()()((

))()()(()()()(),(+−+−

−

−−

−−+

+−+−=

ii

iii

nini

ninini

iiiiiii

qkbkaqkbkakb

qkbkakbkbkakbkqL

We will use the notation

(29) )()()()( 1, kukbkTkT iiii +=

The criterion to be minimized becomes:

(30)

α++++=++ − )()]1(),()([)1( 221 kudkekqFkTdkJ iiiiiiii

Ε

From this, it is easy to see that

(31) 22,

21,

22)](...)(1[)()()1( idiiiiiii kfkfkukTdkJ

iσ++++α+=++

Finally, from (29) and (31) we obtain

(32) 0)()()(

)1(1, =α+=

∂++∂

kukTbkudkJ

iiiii

ii

Then, the optimal control law )(kui which minimizes the criterion (25) is defined by:

(33)

∑ ++−−

∑ ++−−−=

≠=

−−

≠=

−−−

−

N

ijjiijjij

i

N

ijjiijjij

ii

i

ii

dtkykqWkqZ

ddkukqVkqZ

kykqZkqG

ku

,1

11

,1

111

1

)1(),(),(

1

)1(),(),(

1)(),(),(

)(

where the polynomials ),( 1 kqZi− , ),( 1 kqVij

− and ),( 1 kqWij− are given respectively

by:

(34) 1

,1

2,1,

11,

111

)()()(

)())(/(),(),(),(+−−

+−

−−−−

+++=

α+=

iiii

dndniii

iiiiii

qkzqkzkz

qCkbkqFkqqBkqZ

(35) ii

iidn

dnijij

iijij

qkvqkv

kqFkqBkqV−−

+−

−−−

++=

=

)()(

),(),(),(

,1

1,

111

and


(36) ii

iidn

dnijij

iijij

qkwqkw

kqFkqAkqW−−

+−

−−−

++=

=

)()(

),(),(),(

,1

1,

111

with ))(/()()( 1,1,1, kbkbkz iiii α+= , )()( 1,1, kbkv ijij = and )()( 1,1, kakw ijij = . It is to be noted that the first term in equation (33) results from the dynamic behaviour of the isolated system iS , Ni ≤≤1 , and the second and the third terms correspond to the interaction.

The above control law (33) gives the optimal generalized minimum variance regulator even if the interconnected system to be regulated has an instable inverse; i.e., a non-minimum phase interconnected system. Therefore, in order to assure stability of the closed-loop interconnected system iS , Ni ≤≤1 , it is necessary to prove that:

∞<)(kui , for ∞→k ; in other words, it will be shown under what conditions the control signal )(kui is bounded. Thus, by introducing equation (1) in equation (33) it is easy to see that stability of the developed control scheme is assured if all the roots of the following polynomial ),( 1 kqKi

− are inside the unit circle:

(37)

iiiii

n

i

niin

i

niini

i

iii

i

ii

iiiii

qkb

kaq

kb

kakb

qkbka

kbkb

kb

kqAkbkqqBkqK

−+−−

−

−−−

α+

α++

+α

++α

+=

α+=

)(

)()

)(

)()((

))()(

)(())(

)((

),())(/(),(),(

1,

,1

1,

1,,

1

1,

1,2,

1,1,

11,

11

Based on the modified Routh stability criterion of this polynomial ),( 1 kqKi− , it is

possible to obtain necessary conditions for the global stability of the controlled interconnected system iS , Ni ≤≤1 . It is important to recall that the zeros of the noise filter )( 1−qCi must lie within the unit circle.

Finally, the generalized minimum variance regulator which must generate an input signal )(kui such that the errors caused by the noise sequence )}({ kei are minimized according to criterion (25) is defined by equations (22) and (33). Unfortunately, this regulator cannot be computed since in the above control law (33) the coefficients of the polynomials ),( 1 kqGi

− , ),( 1 kqZi− , ),( 1 kqVij

− and ),( 1 kqWij− depend on the

unknown and time-varying interconnected system parameters. Thus, it is a natural to replace the parameters )(kiθ in equation by adjustable parameters )(ˆ kiθ which will be updated by the adaptation mechanism of the RELS parametric estimation scheme.

The posed optimal self-tuning regulation problem can be solved using explicit or implicit control schemes. Notice that in the explicit control scheme, the self-tuning regulator can be described by three steps. The first step is to estimate the interconnected system parameters )(kiθ . The second step is to determine the polynomials ),( 1 kqFi

− and ),( 1 kqGi

− from (22). In the third step, a design method is used to compute the parameters of the regulator using the obtained results in the first and second steps. However, in the implicit scheme the parameters of the regulator are estimated directly on the basis upon a reparameterization of the mathematical model of the interconnected system, i.e., only two steps are required for implementing the self-tuning regulator. We will discuss first the explicit self-tuning regulation problem.

210 S. KAMOUN

Explicit self-tuning regulation To use the explicit self-tuning regulation approach, the parameters of the

interconnected system iS , Ni ≤≤1 , have to be estimated. The procedure of applying the explicit self-tuning regulator to the interconnected system iS , Ni ≤≤1 , consists of the following three steps: Step 1: estimate the interconnected system parameters )(kiθ , defined by (13), by using

the RELS parametric estimation algorithm (19); Step 2: solve the diophantine equation (22) for polynomials ),( 1 kqFi

− and ),( 1 kqGi− ;

Step 3: calculate the optimal control law )(kui as:

(38)

)(z

)1()(ˆ

)(z

)1()(ˆ

)(z

)()(ˆ)1()(ˆ)(

i,1

,1 1,

i,1

,1 1,

i,1

2

1

0,,

k

sdtkykw

k

mddkukv

k

tkykgrkukzku

N

ijj

dn

siijjsij

N

ijj

dn

miijjmij

dn

r

n

titiiri

i

ii

ii

ii i

∑ ∑ +−+−−

∑ ∑ +−+−−

∑ ∑ −++−−=

≠=

+

=

≠=

+

=

+

=

−

=

Steps 1, 2 and 3 are repeated at each sampling instant k . It is important to note that the boundedness of the control signal )(kui depends on the estimated parameter

))(ˆ/()(ˆ)(ˆ 1,1,1, kbkbkz iiii α+= . Indeed, if the estimated parameter )(ˆ 1, kzi equals zero during computations, then the control law )(kui , described by (38), may diverge. To solve this problem, a test of )(ˆ 1, kzi is recommended (particularly in the initial transient period). For example, if at discrete-time k we obtain by parametric estimation (step 1)

0)(ˆ 1, =kzi , then we can compute the control law )( kui (step 3) with 01.0)(ˆ 1, =kzi . In certain applications, it is possible to calculate the control law )(kui with a fixed value of the estimated parameter ∗= 1,1, )(ˆ ii zkz chosen by the designer. Sometimes, and in order to obtain a good estimation, it can be advantageous to add an exciting bounded auxiliary signal to speed up the convergence of the RELS parametric estimation algorithm (19). To summarise, it is necessary to stress that it is important to have as much a priori knowledge about the interconnected system as possible. This knowledge may be used to choose best solution and specifications.

Notice that the developed self-tuning regulator uses on-line parametric estimation algorithm to estimate the parameters of the interconnected system iS , Ni ≤≤1 , given by (11). In addition, to compute the control law )(kui the diophantine equation (22) must be resolved in each sampling interval (step 2). It is of relevance to following to examine how the parameters of the polynomials ),( 1 kqFi

− and ),( 1 kqGi− can be

computed recursively, and consequently the diophantine equation (22) does have to be calculated. Thus, the problem of computing the above self-tuning regulator is simplified if it is observed by using a reparameterization of mathematical model of the interconnected system the step 2 is eliminated.


Implicit self-tuning regulation The implicit self-tuning regulator is designed to minimize the variance of the auxiliary

output )1( ++ ii dks defined as: (39) )())(/()1()1( 1, kukbdkydks iiiiiii α+++=++

The criterion to be minimized is given by:

(40) )]1([)1( 2 ++=++ iiii dksdkJ Ε

It is easy to show that the auxiliary output (39) can be rewritten as:

(41)

)1()1(),(

)1(),(

)1()](1[)(),()(),()1(

,1

1

,1

1

111

++υ+∑ ++−+

∑ ++−+

++−++=++

≠=

−

≠=

−

∗−−−

iiN

ijjiijjij

N

ijjiijjij

iiiiiiiii

dkdtkykqW

ddkukqV

dksqCkykqGkukqZdks

where )1(* ++ ii dks is the 1+id step ahead prediction of the auxiliary interconnected system output and )1(),()1( 1 ++=++υ −

iiiii dkekqFdk represents the prediction error (connected with control problem). Equation (41) can by defined at discrete-time k in the following compact form:

(42) )()()()( kkkks iiT

ii υ+ψθ=

where )(kiθ and )(kiψ represent respectively the regulator parameters and the observation vector. They are given by:

(43) )]( )()( )(

)( )()( )([)(

,1,,1,

,1,1,0,,1,

kwkwkvkv

cckgkgkzkzk

iiii

iiii

dnijijdnijij

niiniidniiT

i

++

−+=θ

and

(44)

)]()1(

)()1( )()1(

)( )1( )2()1([)(

iiijjijj

iiijjijjiii

iiiiiiiiiiT

i

ndtkytky

nddkudkunksks

ndkydkyndkudkuk

−−−−−

−−−−−−−−−−

−−−−−−−−=ψ∗∗

Let us note by )1(ˆ ++∗ii dks the 1+id step ahead adjustable prediction of the

auxiliary interconnected system output. Thus, we can write:

(45) )1(ˆ)(ˆ)1(ˆ ++ψθ=++∗ii

Tiii dkkdks

where )(ˆ kiθ is a parameter vector which contains all estimated parameters of the control law )(kui and )1(ˆ ++ψ ii dk is an approximation of )1( ++ψ ii dk , such that:

(46)

)]1()(

)1()( )1(ˆ)(ˆ

)1( )( )1()([)1(ˆ

+−−+−

+−−+−

+−+−−+−

+−+−−=++ψ∗∗

iijjiijj

iijjiijj

iiiii

iiiiiiiiT

i

ntkydtky

ndkuddkundksdks

nkykyndkukudk

212 S. KAMOUN

It is known that the self-tuning regulator minimizes the criterion (40) by setting the adjustable prediction )1(ˆ ++∗

ii dks equal to zero at each sampling instant k . Thus, the practical implementation of the implicit self-tuning regulator will be realized as follows: Step 1: estimate the interconnected system parameters )(kiθ , defined by (43), by using

the RELS parametric estimation algorithm (19); Step 2: compute the optimal control law )(kui from:

(47)

)(z

)1()(ˆ

)(z

)1()(ˆ

)(ˆ

)1(ˆ)(ˆ

)(z

)()(ˆ)1()(ˆ)(

i,1

,1 1,

i,1

,1 1,

1,

1,

i,1

2

1

0,,

k

sdtkykw

k

mddkukv

kz

hdkskc

k

tkykgrkukzku

N

ijj

dn

siijjsij

N

ijj

dn

miijjmij

i

n

hiihi

dn

r

n

titiiri

i

ii

ii

i

ii i

∑ ∑ +−+−−

∑ ∑ +−+−−

∑ +−++

∑ ∑ −++−−=

≠=

+

=

≠=

+

=

=

∗

+

=

−

=

Steps 1and 2 are repeated at each sampling interval. The main advantage of the presented implicit self-tuning regulator is the control signal

)(kui is not computed via the resolution of the diophantine equation. Moreover, this control signal is computed by using the parameters of the adjustable prediction. It is observed from (43) that only the unknown parameters of the polynomial )( 1−qCi of the interconnected system iS , Ni ≤≤1 , have to be estimated.

The number of parameters for estimation can be reduced if a simplified version of the above implicit self-tuning regulator is used. In this case, the implicit self-tuning regulator can be interpreted as choosing the control law )(kui such that the predicted value

)1( ++∗ii dks will be equal to zero. Based on this approach, we can consider a

simplified value )(~ ksi of the auxiliary output )(ksi defined by (42), where the parameters gic , and the corresponding signals )( gksi −∗ , ing ,,1= , are eliminated. Let:

(48) )()()()(~ kkkks iiT

ii υ+ψθ=

where the vectors )(kiθ and )(kiψ are given respectively by:

(49) )]( )()( )(

)( )()( )([)(

,1,,1,

1,0,,1,

kwkwkvkv

kgkgkzkzk

iiii

iii

dnijijdnijij

niidniiT

i

++

−+=θ

and


(50)

)]()1(

)()1( )( )1()2()1([)(

iiijjijj

iiijjijj

iiiiiiiiiiT

i

ndtkytky

nddkudkundkydkyndkudkuk

−−−−−

−−−−−

−−−−−−−−=ψ

From (50), it is clear that the observation vector )(kiψ is totally available at each discrete-time k , and let us assume that the prediction error )(kiυ in (48) is a normal distributed white noise with zero mean and finite variance. Then, in order to determine the parameters )(kiθ we may use the Recursive Least Squares (RLS) parametric estimation algorithm, which is simpler than RELS parametric estimation algorithm (19). Note that the RLS parametric estimation algorithm can be computed as (19) where the vector )(ˆ kiψ is replaced by )(kiψ , given by (50).

The simplified implicit algorithm for self-tuning regulation of the interconnected system iS , Ni ≤≤1 , consists of the two following steps: Step 1: estimate the interconnected system parameters )(kiθ , defined by (49), using

RLS parametric estimation algorithm; Step 2: compute the optimal control law )(kui from:

(51) 0)1()(ˆ =++ψθ iiT

i dkk

This implies that:

(52)

)(z

)1()(ˆ

)(z

)1()(ˆ

)(z

)()(ˆ)1()(ˆ)(

i,1

,1 1,

i,1

,1 1,

i,1

2

1

0,,

k

sdtkykw

k

mddkukv

k

tkykgrkukzku

N

ijj

dn

siijjsij

N

ijj

dn

miijjmij

dn

r

n

titiiri

i

ii

ii

ii i

∑ ∑ +−+−−

∑ ∑ +−+−−

∑ ∑ −++−−=

≠=

+

=

≠=

+

=

+

=

−

=

Steps 1and 2 are repeated at each discrete-time k . It is to be noted that the computation of this control law )(kui uses only parameters of

the simplified prediction )1(~ ++∗ii dks . Thus, it is not necessary to estimate the

unknown parameters of the interconnected system iS , Ni ≤≤1 , and consequently the implementation of the above regulation algorithm is relatively simple. 3.2. Variance constrained self-tuning regulation

In general, the minimum variance regulator often requires excessive control effort, however in many practical applications the control input is limited by constraints. For example, in the servo-damper system studied by Kang and Lee [10], the allowable displacement, or the variance of displacement in the stochastic sense, of the damper mass is limited. Thus, in the case when the minimum variance control requires too control signal a restriction on the input variance may be imposed.

214 S. KAMOUN

In this section, we treat the control problem when a restriction on the input variance is introduced. This problem can be formulated by minimising the stationary variance of the output of each interconnected system iS , N,,11 = , subject to the restriction that the input variance is less than or equal to a constraint given by the designer. The basic idea of the minimum variance control is to adjust the input signal )(kui such that the following criterion is minimum:

(53) )]1([)1( 2 ++=++ iiii dkydkJ Ε

In the variance constrained regulation problem, the criterion (53) may be minimized subject to the following inequality constraint:

(54) 22 )]([ ii ku β≤Ε

where 2iβ represents the maximum constrained input variance. The Lagrangien which

corresponds to the posed constrained regulation problem is described by:

(55) )]()()1([)1( 22 kukdkydkJ iiiiii γ+++=++ Ε

It is important to note that the criterion (55) has the same form as (25). However, in (55), the weight )(kiγ can be interpreted as a Lagrangien multiplier, and consequently the minimisation of (55) must be made with respect the condition of Kuhn-Tucker, which is necessary to obtain an optimal control law )(kui . This condition is defined as:

(56) 0)]([)( 22 =

β−γ iii kuk Ε

In order to obtain an optimal control law )(kui for solving the minimum variance control problem with input energy constraint, we propose the following explicit self-tuning regulator, which is composed of four steps: Step 1: estimate the interconnected system parameters )(kiθ , defined by (13), by using

the RELS parametric estimation algorithm (19); Step 2: determine the parameters of the polynomials ),( 1 kqFi

− and ),( 1 kqGi− via the

resolution of the diophantine equation (22); Step 3: compute the optimal control law )(kui from:

(57)

∑ ∑ +−+−+

∑ ∑ +−+−+

∑ ∑ −++−

γ+=

≠=

+

=

≠=

+

=

+

=

−

=

N

ijj

dn

siijjsij

N

ijj

dn

miijjmij

dn

r

n

titiiri

ii

ii

ii

ii

ii i

sdtkykw

mddkukv

tkykgrkukzkkz

kzku

,1 1,

,1 1,

2

1

0,,2

1,

1,

)1()(ˆ

)1()(ˆ

)()(ˆ)1()(ˆ)()(ˆ

)(ˆ)(

Step 4: adjust the input weight )(kiγ according to the Robbins-Moro scheme, let:

(58) ])1([)1()(

)1()( 222 ii

i

iiii ku

kkkk β−−

β

−γµ+−γ=γ


where )(kiµ is a positive time-varying parameter which may be chosen as: 1)(0 <µ< ki .

The choice of the parameter )(kiµ intervening in equation (58) depends on the nature of the interconnected system parameters. In our study, the parameters of the interconnected system iS , Ni ≤≤1 , are time-varying, then the parameter )(kiµ may be chosen as a small positive constant (i.e., ii k µ=µ )( ). 4. Illustrative example

In this section, we provide example to illustrate the presented theoretical results. Let us consider a large-scale stochastic system with unknown and time-varying parameters, which is composed of two interconnected systems 1S and 2S . The interconnected systems 1S and 2S are described respectively by the following IARMAX mathematical models:

(59) )3()()3()(

)1()()2()()1()()(

21,1221,12

11,1111,111,11

−+−+

−++−+−−=

kykakukb

keckekukbkykaky

and

(60) )3()()3()(

)1()()2()()1()()(

11,2111,21

21,2221,221,22

−+−+

−++−+−−=

kykakukb

keckekukbkykaky

In the IARMAX mathematical models (59) and (60), the sequences noises )}({ 1 ke and )}({ 2 ke are independent and generated as Gaussian distributed with zero means and

variances 05.021 =σ and 03.02

2 =σ , respectively. The developed explicit self-tuning regulator with the RELS parametric estimation

algorithm (19) is used to regulate the interconnected systems 1S and 2S , which are described by the IARMAX mathematical models (59) and (60), respectively. The initial values of the estimated parameters )0(ˆ

iθ and the gain matrix )0(iP , and the forgetting factor iλ , 2,1=i , are chosen as: 0)0(ˆ =θi and IPi 1000)0( = , where I denotes the identity matrix, 99.0=λi , and the number of data was 200 (i.e., 20021 == MM ,

200,,1=k ). The parameters of the IARMAX mathematical model (59) are chosen as: )4.0sin(04.085.0)(

1,1kka +−= , )3.0cos(03.028.0)(

1,1kkb += , 14.01,1 =c ,

)2.0sin(04.018.0)(1,12

kkb += and )3.0sin(05.026.0)(1,12

kka += . The parameters of the IARMAX mathematical model (59) are chosen as: )5.0cos(04.084.0)(

1,2kka +−= ,

)4.0cos(04.036.0)(1,2

kkb += , 20.01,2 =c , )2.0sin(03.016.0)(1,21

kkb += and )3.0cos(04.025.0)(

1,21kka += . The control laws )(1 ku and )(2 ku are computed on the

basis upon the minimization of the following criteria, respectively:

(61) )](05.0)2([)2( 21

211 kukykJ ++=+ Ε

and

(62) )](10.0)2([)2( 22

222 kukykJ ++=+ Ε

In order to evaluate the global quality of the RELS parametric estimation algorithm (19), we consider a parametric distance )(kdi defined by:

216 S. KAMOUN

(63) 5.0

,1 1

2

,

,,

,1 1

2

,

,,

1

2

,

,,

1

2

,

,,

1

2

,

,,

])(

)(ˆ)([]

)(

)(ˆ)([

])(ˆ

[])(

)(ˆ)([]

)()(ˆ)(

[)(

∑ ∑

−∑ ∑ +

−+

∑ +−

∑ +−

∑ +

−=

≠= =≠= =

===

N

ijj

n

r rij

rijrijN

ijj

n

r rij

rijrij

n

r ri

ririn

r ri

ririn

r ri

ririi

ii

iii

kbkbkb

kakaka

ckcc

kbkbkb

kakaka

kd

The elements of this parametric distance )(kdi , which correspond to the considered numerical example, are given as: 1=in , 2=N , 2,1, =ji , ij ≠ and 200,,1=k .

We can calculate the statistical average ium of the control law )(kui , the variance

2iuσ of the control law )(kui , the statistical average

iym of the output )(kyi and the variance 2

iyσ of the output )(kyi of the interconnected systems iS , ,2,1=i 200,,101=k , by using the following expressions:

(64) 100

)(m

200

101∑

= =ki

u

ku

i

(65) 100

]m)([200

101

2

2∑ −

=σ =kui

ui

i

ku

(66) 100

)(m

200

101∑

= =ki

y

ky

i

and

(67) 100

]m)([200

101

2

2∑ −

=σ =kyi

yi

i

ky

The obtained statistical average ium of the control law )(kui , the variance 2

iuσ of the control law )(kui , the statistical average

iym of the output )(kyi and the variance 2

iyσ of the interconnected systems iS , ,2,1=i are as follows: 08.0m1=u ,

24.021

=σu , 06.0m1

−=y , 12.021

=σ y , 01.0m2=u , 14.02

2=σu , 02.0m

2−=y and

07.022

=σ y . The curves of the output )(1 ky , the parametric distance )(1 kd , the variance )(2

1kuσ

of the control law )(1 ku and the variance )(21

kyσ of the output )(1 ky of the output )(1 ky of the interconnected systems 1S are given in Fig. 1. The curves of the output )(2 ky , the parametric distance )(2 kd , the variance )(2

2kuσ of the control law )(2 ku

and the variance )(22

kyσ of the output )(2 ky of the interconnected systems 2S are given in Fig. 2.


0 50 100 150 200-2

-1

0

1

2

0 50 100 150 2000

2.5

5

7.5

10

0 50 100 150 2000

1

2

3

4

0 50 100 150 2000

0.1

0.2

0.3

0.4

0 50 100 150 200-2

-1

0

1

2

0 50 100 150 2000

2.5

5

7.5

10

0 50 100 150 2000

1

2

3

4

0 50 100 150 2000

0.1

0.2

0.3

0.4

)(1 ky )(1 kd

)(21

kuσ )(21

kyσ

k

k

k

k

Fig. 1. Curves of the output )(1 ky , the parametric distance )(1 kd , the variance )(2

1kuσ of the control law )(1 ku and the variance )(2

1kyσ of the output )(1 ky .

)(2 ky )(2 kd

)(22

kuσ )(22

kyσ

k

k

k

k

Fig. 2. Curves of the output )(2 ky , the parametric distance )(2 kd , the variance )(2

2kuσ of the control law )(2 ku and the variance )(2

2kyσ of the output )(2 ky .

218 S. KAMOUN

The interpretation of the evolution curves of the different elements represented in Figs. 1 and 2 shows well the good quality of the regulation of the two considered interconnected systems 1S and 2S , which is obtained by the proposed generalized minimum variance self-tuning regulator. Thus, the evolution curves of the variances

)(21

kuσ , )(21

kyσ , )(22

kuσ and )(22

kyσ converge towards a constant minimum values. The evolution curves of the parametric distances )(1 kd and )(2 kd decrease (on average statistical) towards a low values; this proves the good quality of the estimate, which is obtained by the RELS parametric estimation algorithm (19). It is clear, however, that during initial phase the performance of the proposed self-tuning regulator is poor; this is due to the mediocre quality of the estimated parameters of the interconnected systems 1S and 2S . We notice that the quality of the regulation is based on the quality of the estimation. Globally, the quality of the estimation, which is obtained during the initial phase of the implementation of the RELS parametric estimation algorithm(19), depends on the choice of the values of the initial conditions )0(ˆ

iθ and )0(iP , 2,1=i . Indeed, if the initial condition of the estimated parameter vector )0(ˆ

iθ is chosen near to the true parameter vector )(kiθ and the value of the initial condition of the adaptation gain matrix is small (e.g., IPi =)0( ), then the estimated parameter vector

)(ˆ kiθ converges rapidly to the true parameter vector )(kiθ . 5. Conclusion

The optimal estimation and self-tuning control problem of a winder class of large-scale system composed of several interconnected stochastic systems were been considered in this paper. The interconnected systems are described by linear input-output mathematical models with unknown and time-varying parameters.

The problem of parametric estimation of large-scale stochastic systems by using the recursive extended least squares method. Recursive algorithms for self-tuning regulation of interconnected stochastic systems have been developed on the basis upon the generalized minimum variance approach with explicit and implicit schemes. Also, we have proposed an explicit self-tuning regulator, which can be applied when a restriction on the input variance is introduced.

An example of numerical simulation is treated in order to validate the effectiveness and the performance of the proposed generalized minimum variance self-tuning regulator. The obtained numerical results indicate that this self-tuning regulator works quite well.

The proposed self-tuning regulators could work fairly well for different types of interconnected systems, e.g., stable and unstable systems, minimum and non-minimum phase large-scale systems having dead time. Notice that the presented results concerning estimation and self-tuning regulation correspond to monovariable interconnected systems, but the idea can be applied for multivariable interconnected systems. References [1] Åström K.J. (1970), Introduction to Stochastic Control, New York: Academic Press.

[2] Åström K.J. and B. Wittenmark (1973), On self-tuning regulators, Automatica, Vol. 9, pp. 185−199.


[3] Clarke D.W. (1984), Self-tuning control of nonminimum-phase systems, Automatica, Vol. 20, pp. 501−517.

[4] Clarke D.W. and P.J. Gawthrop (1979), Self-tuning control, Proceedings of IEE, Vol. 126, pp. 633−640.

[5] Kamoun M. (1995), Design of robust adaptive regulators for large-scale systems, International Journal of Systems Sciences, Vol. 26, pp. 47−63.

[6] Kamoun M. (1997), Décentralisation de schémas de commande auto-ajustable de processus de grande dimension, Journal Européen des Systèmes Automatisés (RAIRO−AII−JESA), Vol. 31, pp. 289−306.

[7] Kamoun M. and A. Titli (1988), Parametric identification of large scale discrete time systems, Information and Decision Technologies, Vol. 14, pp. 289−306.

[8] Kamoun M. and A. Titli (1989), Implicit self-tuning control for a class of large-scale systems, International Journal of Control, Vol. 49, pp. 713−722.

[9] Kamoun S. (2003), Contribution to identification and adaptive control of complex systems (in French), PhD Thesis in Automatic Control and Computer Engineering, University of Sfax.

[10] Kang M.-S. and C.-W. Lee (1988), Weighted minimum variance control of servo-damper system with input energy constraint, Journal of Dynamic Systems, Measurement and Control, Vol. 110, pp. 70−77.

[11] Li Z. and R.J. Evans (2002), Generalized minimum variance control of linear time-varying systems, IEE Proc. Control Theory Appl., Vol. 149, pp. 11−116.

[12] Šiljak D.D. (1990), Decentralized Control of Complex Systems, New York: Academic Press.

Samira KAMOUN, Associate Professor of Automatic Control at the Department of Electrical Engineering of the National School of Engineers of Sfax (ENIS), B.P. 1173, 3038 Sfax, Tunisia. E-mail: [email protected]. She is member of the Automatic Control Unit (UCA) of Sfax. She obtained the doctorate thesis (PhD) on December 2003 in the field of Automatic Control and Computer Engineering from the National School of Engineers of Sfax. She research’s focus on identification and adaptive control of complex systems (large-scale systems, nonlinear systems, time-varying systems, stochastic systems), with applications to electrical engineering (electrical machines, etc.).

mailto:[email protected]�

Received by the editors June 30, 2009 220

DOUBLE LINEAR DISCRIMINATE ANALYSIS FOR FACE RECOGNITION

Billy, Li1, Wanquan Liu1, and Chong Lu2

1Curtin University of Technology 6102, Perth, WA, Australia

and 2Yili Normal College

835000, Yining , Xinjiang, P. R. China

Abstract In this paper we propose a new algorithm for face recognition named as Double Linear Discriminate Analysis (DLDA) in which we combine two linear discriminate methods: 2D-LDA and Fisher-face sequentially. Experimental results on benchmark face datasets show that DLDA not only achieves the best performance when the number of training samples is appropriate, but also its speed is the fastest among three of them. Though the proposed method can operate on both gray and colour images, the performance is significantly better while operating on colour face datasets in comparison to 2D-LDA, Fisher-face and the Discriminate Colour Space method proposed specifically for colour face recognition recently.

Key Words: Double Linear Discriminate Analysis (DLDA), Two-Dimensional Linear Discriminate Analysis (2D-LDA), Fisher-face, Face Recognition.

1. Introduction

Face recognition is a popular research topic in computer vision and its aim is to automatically recognize unknown human faces based on the training from a set of known faces. Currently most of the research is focused on gray face images, for example, Xiao-Yuan Jing [8] proposed "An improved LDA Approach" , P.N. Belhumeur [1] proposed “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection," and Chong Lu [9] proposed “An Innovative Weighted 2DLDA Approach for Face Recognition.” and so on. All these approaches are for gray images, which are represented as one matrix with elements as pixel values. As it is known that colour face images are represented by three matrixes and it is expected that more information in color face images can improve the performance of face recognition. Therefore researchers start to design classifiers for colour face images recently [10] [11].

One of the most important tasks in face recognition is feature extraction and one typical feature extraction method is the Eigen-faces method [4]; however, it is proven that Fisher-face (or called 1D-LDA) method [1] is superior to the Eigen-faces method and 1DLDA is considered to be a typical result in the field. Although different variants for 1DLDA are proposed, their feature extractions are similar since they are all working on vectors, which require converting the original matrix image to a vector. The


INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCE Volume 1, Number 1, Pages 1-22

INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCES Volume 6, Number 2, Pages 220-233

DOUBLE LINEAR DISCRIMINATE ANALYSIS FOR FACE RECOGNITION 221

advantages of 1DLDA can be identified as following: 1) it has good robustness with explicit statistical interpretations [12]; 2) it has nice engineering intuition [13]; 3) it has an explicit solution even in a regression form [7]. However, there are two main disadvantage of 1DLDA: One is its poor performance in the case of small number of training samples and the other is its large computational costs due to matrix-to-vector conversion.

In order to overcome those shortcomings in 1DLDA, Two-Dimensional Linear Discriminate Analysis (2D-LDA) method [2] is proposed, which operates on matrix directly instead of vectors. 2DLDA can work apparently faster since it manipulates matrix operations directly and can achieve better performance under small number of training samples. However, its robustness is still not well explored in statistical framework and also its ridge regression solution is hard to find as pointed out in [6].

These methods mentioned above are designed mainly for gray images only. By simply concatenating the three matrixes of colour images, one can apply these methods on colour images as did on gray images and in fact, it is found that with colour images, 1DLDA can achieve better accuracy at the cost of more computation resource usage [3]; however, this approach does not make use of the colour information in a smart way. Recently, the discriminate colour space (DCS) method [5] is proposed, which makes use of the colour information by linearly combining the three matrixes and then apply 2DLDA on the combined matrix.

Notice that 2D-LDA can overcome the large computational cost issue with better performance in the case of small number of training samples. With large number of training samples, 2DLDA and 1DLDA can achieve similar performance. However, we notice that 1DLDA has nice robustness and explicit ridge regression form and 2DLDA does not have such properties. Also the large computational cost in 1DLDA is mainly due to matrix-to-vector conversion when the size of face image is large. Based on these observations, we propose a new approach for face recognition as following. First we do 2DLDA on the face images and reduce the dimension of the face images and then apply the 1DLDA on the reduced size images for face recognition. We called this approach double Linear discriminate analysis (DLDA). In this case, we will utilize the advantages of 2DLDA and 1DLDA and avoid their shortcomings.

The details of DLDA method is presented in next section. Section 3 is the database detail for the experiment. Section 4 is the presentation of the experimental results and analysis. Finally, conclusions are given in Section 5.

2 Double Linear Discriminate Analysis 2.1 Data Representation for Colour Image

A gray image with a resolution of can be represented by one matrix A of the same size. Similarly, colour image with a resolution of in RGB space can be represented by three matrixes [R], [G] and [B]. To get a simple matrix in the case of gray images, we can reshape it to by concatenating [R], [G] and [B] together.

(1) Since the output from 2D-LDA method is 2D matrix but the input for 1DLDA is vector.

In order to combine these two methods, we need to convert a matrix to vector. Given a matrix J, we can convert it to a 1 column vector K:

222 B. LI, W. LIU AND C. LU

(2) Given N column-vector K, we can combine them to form a matrix C:

(3) Also we define the following symbols used in LDA.

(4)

2.2 2D-LDA

Let A be a 2D-matrix containing matrix representation of all training face images.

,

where Ai refers to image matrices in A that belong to Ti. The between class scatter matrix is defined as:

(5)


And the within class scatter matrix is defined as:

(6) The 2DLDA criterion is:

(7) where

The optimal projection is solved by:

(9) This is equivalent to solving the generalized Eigen value problem:

(10) By solving (10) and finding the d eigenvectors corresponding to the d greatest

Eigen-values, we can obtain the projection matrix X. Then we can obtain the feature matrix Bj by projecting Aj on the X:

(11)

2.3 1D-LDA 1DLDA is proposed in [1], in which one first uses PCA to reduce the dimension of the

data and then perform LDA to extract the features. Let C be a matrix containing vector representation of all training face images

converted from its matrix representation in (2) then arranged in (3). Let

Cj refers to vectors in C that belongs to Ti. The mean-corrected matrix is defined as:

(8)


(12) The PCA projection matrix y is obtained by choosing the d Eigen-vectors

corresponding to d largest Eigen-values for the Eigen problem defined as:

(13) Since the matrix is always too large in dimension size to handle, we usually

calculate y by using the surrogate z as:

(14) After we obtain z, we can obtain y by multiplying z with:

(15) We can obtain the dimension-reduced matrix Aj by PCA projection:

(16) Then the between class scatter matrix is defined as:

(17) The within class scatter matrix is defined as:

(18) Finally, we can obtain the projection matrix X similarly as in 2D case and then the Fisher feature vector Bj will be as following by projecting Aj on X.

(19) 2.4 DLDA Training


Given N training images for L different people, there will be Ti classes where each class Ti have Ni images. If the images are gray, we can

represent them as matrices directly; if they are color images, we use (1) to concatenate the matrices. Let these matrices be . First we perform 2D-LDA for feature extraction on each Aj. We calculate the global mean by:

(20) Similarly, we calculate the class mean by:

(21) Next we calculate the between-class and with-in class scatter matrices using equation

(5) and (6) for 2DLDA. Then we solve equation (9) and solve matrix with dimension d2dlda, which are the d2dlda, eigenvectors corresponding to the largest eigen values. We call this projection matrix . We then project all the images using (11) and obtain N lower dimensional feature matrices Bj.

Next we convert these matrixes Bj to vectors Kj as (2) and concatenate them to form a matrix C as (3), such that C is a matrix which each column is a face-vector. Then we implement 1DLDA as below.

We calculate C similarly using equation (20). Then we obtain Cmc using equation (12). Next we obtain z by solving equation (14) and we only keep the dpca largest corresponding Eigen-vectors in z. Then we obtain y using equation (15). y is the PCA projection matrix in the form as (10) and we call it xpca. Next we project Cmc using (16) and obtain a smaller matrix D.

Next we calculate and similarly using equation (20) and (21); Sb and Sw using

(17) and (18). Then we solve (9) and obtain projection matrix xlda in the form as (10) with dlda being the number of the corresponding largest Eigen-vectors.

Finally, we project Dj to Ej using (19) and projection matrix xlda, in turn obtaining

as the projected training samples.

Testing We first project the query image to the same dimension space as training samples E

and then use the nearest neighbour algorithm for classification. Given a query image in matrix representation Q, we project it to R by (11) using x2dlda

as the projection matrix that we have obtained during training. Then we convert R from matrix to vector using (2). In sequence, we calculate Rmc using (12) and we obtain U by projecting Rmc using (16) with xpca as the projection matrix. Next we project U to F using


(19) and projection matrix xlda which we have obtained during training. F will be the query image column vector in the same dimension space as the training samples E which we have obtained after training.

Next calculate the Euclidean distance from vector F to each of the training samples Ej. We can classify F to the class of the training sample which has the shortest distance to F.

3 Data Representation for Experiments 3.1 Databases

In order to show the effectiveness of the proposed algorithm DLDA, we have adopted the following four databases:

The Aberdeen database from Psychological Image Collection at Stirling (PICS)1

Georgia Tech face database (GTDB)

.

2

FERRET

.

3

The ORL Database of Faces (ORL)

.

4

1http://pics.psych.stir.ac.uk/ 2http://www.anefian.com/research/face_reco.htm 3http://face.nist.gov/colorferet/colorferet.html 4http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

. In order to show the results nicely, we need to address these databases, which come

with different lighting conditions. Pre-processing including selecting subjects, scaling and face cropping is performed in our experiments. We will detail these changes for each database below. For PICS database (Fig 1), there are 116 people and each people has different number of images varying from 1 to 18. In order to perform the experiments with enough samples, we choose 29 persons, each with 9 images. Also the size is cropped from 432x528 to 60x50 using a Matlab program. The chosen images represent different lighting and expression variations.


Figure 1 PICS

For GTDB database (Fig 2), there are 50 people and each people has 15 images. Also

the size is cropped from 150x150 to 47x33 and we downloaded the cropped images directly from website. The chosen images represent different variations in time, expression, lighting and pose. This is the largest dataset in our experiments.

Figure 2 GTDB For Ferret database (Fig 3), there are 1199 people and each people has different

number of images. We choose 25 persons each having 8 images. Also the size is cropped from 128x192 to 66x60 and we cropped images manually using a Matlab program. The chosen images represent different variations in time, expression, lighting and pose.

Figure 3 Ferret For ORL databases (Fig 4), there are 40 people and each people has 10 images. The


size is cropped from 92x112 to 67x55 and we cropped images manually using Matlab program. The chosen images represent different variations in time, expression, lighting and pose.

Figure 4 ORL

For PICS, GTDB and FERRET databases, the corresponding gray images are created using Matlab’s “rgb2gray” function for the conversion.

3.2 Experimental Methodology

The experiment will be carried out for each of the following algorithms: 1DLDA, 2D-LDA, DCS and DLDA. For each algorithm, the experiment repeated with 2, 4 and 6 training samples for each person except for GTDB in which 8, 10 and 12 training samples for each person will be used. For each number of training samples, the experiment will be repeated 10 times using different combination of training and testing samples choosing randomly, however the same combination will be used over different methods for comparison.

For setting up the experiment nicely, a set of indexes are generated corresponding to the training and testing pairs. For each database, 10 matrices with index indicating the training images are generated and stored. For example, for index (2 3 7 9) in one particular repetition on the PICS database, image number 2, 3, 7 and 9 of each subject are used as training and the rest (1 4 5 6 8) for testing. The purpose of storing these index matrices is to reuse the same combination when a different algorithm is used, so the result is comparable. For example, for the index above, in that particular repetition, different algorithms are all using (2 3 7 9) for training and (1 4 5 6 8) for testing on that database. Totally 10 index matrices are generated and thus 10 combination will be used.

For each repetition, the number of dimension from 1 to 35 is also being tested and the highest one for accuracy will be chosen. The accuracy is calculated as follow:

Finally, the accuracy of each repetition will be recorded and the average accuracy will be compared and analyzed in next section.


4 Experimental Results & Analysis

For the tables shown in this section, we label our proposed method to “DLDA” and the fisher-face method to “1D”, 2D-LDA method to “2D” for convenience. The result is presented from the least number of training images first.

4.1 Gray images

Performance For gray images, four databases are used, the gray-scaled version of PICS, GTDB,

FERET and the original ORL database.

Table 1 The performance on FERET database

Table 2 The performance on PICS database

DLDA 1D 2D

2 48.97% 55.67% 49.95%

4 79.24% 84.69% 75.93%

6 85.52% 89.31% 80.81%

Table 3 The performance on ORL database

DLDA 1D 2D 2 66.13% 63.66% 85.63% 4 94.58% 94.33% 94.17% 6 99.38% 97.88% 96.38%

Table 4 The performance on GTDB database

DLDA 1D 2D 2 36.72% 44.18% 53.28% 4 71.98% 73.98% 70.16% 6 80.80% 81.07% 76.18% 8 84.54% 83.57% 78.92% 10 85.80% 85.04% 80.60%

DLDA 1D 2D

2 23.00% 34.67% 34.13%

4 63.20% 69.60% 62.00%

6 80.00% 82.40% 74.00%


12 90.13% 90.13% 83.93% For gray images, one can observe from these tables that the proposed method performs

worse than 2DLDA when the number of training sample is small. Its performance is approaching 1DLDA and outperforms 2DLDA when the number of training samples increases. This coincides with our motivation for the proposed algorithm. It is worthy to note the performance on the GTDB where it contains the largest number of classes and number of images per subject. When the number of training sample increases to 12 per person, the proposed method performs similarly as 1DLDA but outperforms 2D-LDA. Next we will compare the efficiency of these algorithms. Speed

For these three algorithms, though there is no significant difference in speed as shown in the following table, the DLDA is slightly faster than 1DLDA and 2DLDA. The time in the table 5 below is the average time used (seconds) of all single experiments done on the same database including different number of training samples:

Table 5 The executing speed on different database

DLDA 1D 2D

PICS 3.02 3.30 3.34

FERET 2.36 2.73 2.66

ORL 7.75 8.09 8.59

GTDB 15.94 16.67 20.02

For gray images, on the aspect of speed, DLDA has a minor speed advantage on

average compared to other methods and this advantage will become significant when the number of images in the database increase as well as for colour datasets shown in next section. 4.2 Colour images Performance

For experiments with colour images, the following three databases are used: PICS, FERET and GTDB. Also we compared our result with the recently proposed DCS approach specifically for colour images.

Table 6 The performance on FERET database

DLDA 1D 2D DCS


2 30.13% 36.00% 46.27% 46.60%

4 73.10% 70.80% 72.20% 82.70%

6 91.60% 83.60% 89.80% 93.80%

Table 7 The performance on PICS database

DLDA 1D 2D DCS

2 48.52% 59.46% 57.78% 68.47%

4 86.76% 85.18% 83.66% 92.14%

6 93.33% 90.58% 89.66% 96.32%

Table 8 The performance on GTDB database

DLDA 1D 2D DCS

2 58.56% 50.65% 74.95% 67.26%

4 89.51% 79.22% 88.86% 88.26%

6 94.96% 86.44% 93.11% 93.09% 8 96.74% 89.26% 94.14% 94.97% 10 97.16% 90.60% 95.44% 95.72% 12 98.47% 93.47% 96.13% 97.40%

It is clear to see that DCS method outperform other methods significantly when

number of training sample is low. However, the performance of our proposed method approaches DCS as the number of training samples increases and outperforms all other methods including DCS when tested on GTDB with 8 or more training samples. Speed

There is a significant different in speed between the proposed method and the DCS method. Although 1D is faster, the difference with the proposed method is minor. Table 9 below shows the average time used (seconds) of all single experiments done on the same database including different number of training samples:

Table 9 The average time cost

DLDA 1D 2D DCS PICS 6.91 6.85 6.90 12.01 Feret 5.77 5.64 5.56 9.86 GTdb 32.80 29.13 35.68 52.57


We can see that DCS uses around two third more of the time. For the FERET database,

since it contains the least number of subjects: 25 and the least number of images per person: 8, so the speed difference is minor. Such difference becomes larger on PICS database where the subjects increase to 29 and number of images per subject increased to 9. The different becomes significant on GTDB where there are 50 subjects with 15 images each. The speed of DCS is very slow relatively (20 seconds slower than our proposed method).

5 Conclusions

In this paper, we proposed a new face recognition approach DLDA and this algorithm utilizes the advantages of 1DLDA and 2DLDA while overcoming their disadvantages. Although DLDA can be used over colour images to achieve fast speed and acceptable accuracy, and it just treats the colour image as a gray image directly with more pixels rather than using the colour information nicely like DCS.

DLDA combines 2D-LDA and 1DLDA algorithms sequentially to overcome the high dimension issue in 1DLDA and utilize the advantage for small number of 2DLDA. We found that its advantage is more significant when operate on colour images rather than gray images. Although it performs poorly when the number of training samples is low due to operation of 1DLDA in the last step, its performance is still compatible with the recently proposed method (DCS) and even outperform it when the training sample increases. Also, its speed is significantly faster than the DCS method.

In large-scaled colour database where available training samples is plenty and involved many subjects, DLDA will be a better alternative for the DCS method since it has high accuracy and faster speed.

REFERENCES [1] N. P. Belhumeur, J. P. Hespanha and D. J. Kriegman, Eigenfaces vs. fisherfaces: Recognition

using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (7) (1977) 711-720.

[2] M. Li and B. Yuan, 2D-LDA: A statistical linear discriminant analysis for image matrix, Pattern Recognition Letters, 26 (5) (2005) 527-532.

[3] M. Thomas, C. Kambhamettu and S. Kumar, (2008), Face recognition using a color subspace LDA approach, Proceedings - International Conference on Tools with Artificial Intelligence.

[4] M. Turk and A. Pentland, Eigenfaces for recognition. Journal of Cognitive,` Neuroscience, 3 (1) (1991) 71-86.

[5] J. Yang and C. Liu. A discriminant color space method for face representation and verification on a large-scale database, 19th International Conference on Pattern Recognition, ICPR (2008).

[6] Nam Thanh Nguyen, Wanquan Liu, Svetha Venkatesh, Ridge Regression for Two Dimensional


Local Preserving Projection, ICPR (2008). [7] Senjian An, Wanquan Liu and Svetha Venkatesh, Face Recognition Using Kernel Ridge

Regression", Proceedings of CVPR (2007). [8] Xiao-Yuan Jing, David Zhang, and Yuan-Yan Tang, An improved LDA Approach", IEEE

Transaction on Systems, Man, And Cybernetics|Part B: Cybernetics, 34(5) (2004)1942-1951. [9] Chong Lu, Wanquan Liu, Xiaodong Liu, Senjian An, An Innovative Weighted 2DLDA

Approach for Face Recognition, PCM (2009). [10] Chengzhang Wang, Baocai Yin, Xiaoming Bai, Yanfeng Sun, Color Face Recognition Based

on 2DPCA, ICPR (2008). [11] A. Yip and P. Sinha, (2001), Role of color in face recognition”, MIT AI Memo, Vol 35, December 2001. [12] Yanwei Pang, Lei Zhang, Mingjing Li, Zhengkai Liu, and Weiying Ma, A Novel Gabor-LDA

Based Face Recognition Method”. PCM, LNCS 3331 (2004) 352–358 [13] Matthew J. Hague, Tool for the Management of Concurrent Conceptual Engineering Design”,

Concurrent Engineering, Vol. 6, No. 2, (1998) 111-129 Billy Li, Curtin University of Technology, 6102, Perth, Western Australia. E-mail:

[email protected]. He received the BSc degree in Computer Science from

Curtin University of Technology in 2008 and he is pursuing his Honors degree

currently. His research area is pattern recognition.

Wanquan Liu, Curtin University of Technology, 6102, Perth, Western Australia. E-mail: [email protected]. He received the BSc degree in Applied Mathematics from Qufu Normal University, PR China, in 1985, the MSc degree in Control Theory and Operation Research from Chinese Academy of Science in 1988, and the PhD degree in Electrical Engineering from Shanghai Jiaotong University, in 1993. He once holds the ARC Fellowship and JSPS Fellowship and attracted research funds from different

resources. He is currently an Associate Professor in the Department of Computing at Curtin University of Technology. His research interests include large-scale pattern recognition, control systems, signal processing, machine learning, and intelligent systems.

Chong Lu, Yili Normal College, China. E-mail: [email protected]. He is a PhD student in School of Electronic and Information Engineering Dalian University of technology. He received the BS degree in Applied Mathematics from YiLi Normal College in 1990 and obtained the certificate of MS in Computer Application in 2002. His research interests include image processing, computational learning theory and pattern recognition. He became Professor at YiLi Normal College in 2007.




Received by the editors September 15, 2008 and, in revised form, November 15, 2008.

234

CLASSIFICATION OF MOTIONS IN A PLANE

XIN WANG, CORNELIS HOEDE AND XIAODONG LIU

Abstract. Applying a technique from coding theory we develop a classification of motions of a point in a plane. Not only the common shifts but also both viberations and turns in the movements are consided in our classification. Using vectors with four components to express the code words for each movement. Then we also standardize each componets into 0 or 1 which leading to 16 standard states in order to simplify the analysis. Based on the standard code words we can describe the whole movement in the form of the sequence of the standard states. Further we give one method to measure the similarity of two movements based on our tecknique of classification, which can be used to decide the type of any given movement and it proved to distinquish the different movements quite well. Then these encodings are tested on data in the form of measurements of body motions obtained from video recordings. First the encodings from manual and automatic measurements are compared, and the result shows us the reliability of the automatic measuremetns. The recordings of certain types of movements during different activities are investigated to see whether they can be used for classifying the activities. Because of the difficulty of decide when an activity begins and stops during a movement, finally a measure called agitation, which can be seen as an indicator for a change in activity, is introduced. Key Words. Classification, Cding, Curves, Movement, Agitation.

1. Introduction

In the field of human-media interaction one of the problems is to classify the motion

of people perceived on television. What are people doing? The answer to this question presupposes various things.

First there is the set of movements that are distinguished, like nodding, writing, reaching for something etc. Suppose there are movements distinguished and we have video recordings of people, how can a computer classify the movements made?

In [1] people sitting at a meeting are considered. Their position is characterized by the interrelation in the plane of the screen of the images of certain body parts. A possible characterization is as in Figure 1.

Point 1 represents the image of the top of the head, point 2 represents the image of the nose, points 3 and 4 represent the shoulder tips, points 5 and 7 the elbows and points 6 and 8 the hands. Point 9 represents the throat. Suppose now that the observed person displays a certain movement, then the 9 points will move across the screen and the type of movement will have to be recognizable from the curves described by the 9 characteristic points.

At the basis of combined movements of 9 points then is the movement of a single


INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCES Volume 6, Number 2, Pages 233−250

CLASSIFICATION OF MOTIONS IN A PLANE 235

point, e. g. that of point 6, representing the right hand or of point 2, representing the nose. We will focus on a single point mainly.

In [2] Hoede and Wang classified activities during meetings by considering the states of the meeting during some time period and determining which of 4 given types of activities came closest to the perceived activity. Basically we use the same method here. A point in the plane is supposed to describe a curve in the plane, this is the analogue of an activity.

The curve, from time to time, is in different “states”, like displacement or rotation, etc. This distinction in states in which the moving 8 points can be forms the basis of the classification of movements, analogous to the mentioned classification of activities in [2]. 2. Encoding a curve

Let us consider the curve describing a moving point in Figure 2. At time t = 0 the

movement starts and after 7 time units the point has arrived in the position labeled with 7. Let x and y denote the horizontal and vertical coordinates respectively. (xB, yB) are the coordinate of the point at t = 0, (xE, yE) are the coordinates of the point in position 7.

A very rude encoding would be (xE − xB, yE − yB), just mentioning the changes in coordinates, say (+12, −8). All curves starting in 0 and ending in 7 would have this encoding. More information can be encoded by the subdivision of the curve from measurements of the coordinates after 1, 2, 3, 4, 5, 6 and 7 time units, and giving the changes. In our example we get a vector with 14 components, 7 for horizontal changes and 7 for vertical changes.

v = (+1, −3, +1, +3, 0, +7, +3, | +2, −1, −4, +1, −4, +2, −4). The number of possible curves is still infinite, yet the curve reconstructed from this

vector by straight lines between consecutive positions already resembles the given curve. As we consider time units the distances between consecutive positions dimensionally

are velocities. The curve will be said to be in 7 consecutive states, say “moving right” or “moving left” and “moving up” or “moving down”.

One might also consider the changes with respect to the former state. For the first state we can compare with an auxiliary state “at rest” with changes 0 in both directions. In this way we obtain a vector with elements that are dimensionally accelerations.

a = (+1, −4, +4, +2, −3, +7, −4, | +2, −3, −3, +5, −5, +6, −6).

1×

×

9

3 ×

5 ×

×2

× 4

× 7

6 × × 8

Figure 1. Characteristic points of a posture

7

4

6

2

0

1

3

5

Figure 2. Example of a curve

236 X. WANG, C. HOEDE AND X. LIU

As this vector a immediate follows from v we do not yet take it into account in connection with our classification problem. So far the state of the movement during one time unit is just a pair of numbers indicating the coordinate changes, so shifts. We would like to have a second simple means of encoding. For this we consider, next to shifts, returns. Returns are very typical for vibrations. We assume that the presence, not necessarily the location, of a return in horizontal respectively vertical direction can be measured. Four types of turns were distinguished: h+, h−, v+, v−, see Figure 3. Note that the standard orientation of the coordinate axes has been assumed and that the direction of the movement is irrelevant. For example, h+ is a movement where the x coordinate first diminishes and then gets larger again.

In state 1 of the movement we have a shift (+1, +2) and we see one return in

horizontal direction, that we encode by h−, as the movement changes from right to left. A change from leftward going to rightward going would be encoded by h+. In state 1 there are no vertical returns. There is one in state 2, from up to down so v−. State 3 has a horizontal return h+, state 4 a vertical return v+. State 6 has a vertical return v+ and a vertical return v−.

The interesting states are 5 and 7. State 5 is that of a vibrational movement on top of a vertical shift. There is only one vertical return v−, but there are two horizontal returns h+ and two horizontal returns h−. State 7 appears as a rotational movement, there are two horizontal returns, h− and h+, and two vertical returns, v+ and v−. It is well known that a rotation can be seen as a superposition of a horizontal and a vertical vibration.

We face the problem of encoding returns, that can occur many times, but also in different orderings. Figure 4 shows two movements with the same numbers of returns of the same types.

Both (a) and (b) have 3 returns +1 and 3 returns −1 in both directions. The difference

is in the orderings. These orderings are (h−, h+, h−, h+, h−, h+, v+, v−, v+, v−, v+, v−) for (a) and (h−, v+, h+, v−, h−, v+, h+, v−, h−,v+, h+, v−) for (b). Given the shift and such an ordering of returns the movement during one time unit, what we call the “state” the overall movement is in, already has a rather specific form.

h+ h− v+ v−

Figure 3. Four types of “turns”

Figure 4. Two movement states with identical numbers of returns

(b) (a)


For practical encoding the returns should be determined and the location measurements at consecutive moments in time preferably should not yield the location of a return. However, the chance of this happening is small.

Let us now go back to our original example and give the encoding of the 7 states, by a vector having two components for the shift and a sufficiently large number of components for the returns in order of occurrence. When movements of people are encoded, then the number of returns, say per second, will not be very large. The number of all returns occurring in the overall movement is clearly an upper bound for one movement. When comparing movements, our final goal, the maximum number of occurring returns suffices. In Figure 2 there are 15 returns in all and state 5 has the maximum number of returns, namely 5. We therefore use vectors with 7 components.

For the 7 states these are: State 1: (+1, +2, h−,−, −, −, −), State 2: (−3, −1, v−,−, −, −, −), State 3: (+1,−4, h+,−, −, −, −), State 4: (+3, +1, v+,−, −, −, −), State 5: (0, −4, v−, h−, h+, h−, h+), State 6: (+7, +2, v+, v−, −, −, −), State 7: (+3, −4, h−, v+, h+, v−, −). We herewith have a preliminary encoding of the states of the movement described in

Figure 1. 3. Using ideas from coding theory

One of the aspects of the encoding that is somewhat annoying is the difference

between shift-encodings and return-encodings. We can give up some information about the movement, without actually losing all information about returns, In fact a return h+ can be followed by a return h−, which reveals a vibrational pattern or by a return v−, revealing a rotational pattern clockwise, or by a return v+, revealing a rotational pattern anti-clockwise, see Figure 5.

If we drop the distinction between clockwise and anti-clockwise there are basically

only two types of pairs of consecutive returns. One type has returns of the same type, so h → h or v → v, and the other type has returns of different type, so h → v or v → h. We will call a consecutive pair of vibrational type respectively of rotational type.

We can now count how many pairs of each type there are. For the two movements in Figure 3 we find for (a) 10 vibrational pairs and 1 rotational pair and for (b) 11 rotational pairs. If we now choose the format(horizontal shift, vertical shift, vibrational pairs of returns, rotational pairs of returns) the 7 states get the encodings:

State 1: (+1, +2, 0, 0), State 2: (−3, −1, 0, 0), State 3: (+1,−4, 0, 0), State 4: (+3, +1, 0, 0), State 5: (0, −4, 3, 1), State 6: (+7, +2, 1, 0),

h−

Figure 5. Three possible returns after a return h+

(a)

h+

v+

h+

(b)

h+

v−

(c)


State 7: (+3, −4, 0, 3). Before transforming these vectors further we remark that e.g. states 1 and 4 are rather

similar; two positive components followed by two zeroes. Looking at Figure 2 this is as should be. The states of the overall movements are indeed similar, right and up, no consecutive turns. States 4 and 6 are similar too; both right and up and states 6 showing only one vibrational pair. States 5 and 7 do not differ much on the shift aspect, but show essential differences in the pairs of consecutive returns.

One further simplification is to reduce the possible values for the components. Even with only two possible values per component, say 1 and 0, there are still 16 different states. For the first two components we want to use the acceleration vector, if greater than or equal to zero we encode by 1, if smaller than zero we encode by 0. For the third and fourth component we distinguish high (1) and low (0) values. e. g. by comparing with half the maximal occurring number. We then get

State 1 : (1, 1, 0, 0) ≡ S12, State 2 : (0, 0, 0, 0) ≡ S0, State 3 : (1, 0, 0, 0) ≡ S8, State 4 : (1, 1, 0, 0) ≡ S12, State 5 : (0, 0, 1, 0) ≡ S2, State 6 : (1, 1, 0, 0) ≡ S12, State 7 : (0, 0, 0, 1) ≡ S1. Now states 1 and 4 have the same encoding. The format of these “code words” is

conform that in coding theory. When the Galoisfield GF (2) is used, a natural “distance” between two code words is the number of components in which they differ. This is called the Hamming distance. States 1 and 4 have distance 0, also 1 and 6, 4 and 6. On distance 1 are states 1 and 3, 2 and 3, 2 and 5, 2 and 7, 3 and 4, 3 and 6, 4 and 6. On distance 2 are states 1 and 2, 2 and 4, 2 and 6, 3 and 5, 3 and 7, 5 and 7, 6 and 7.On distance 3 are states 1 and 5, 1 and 7, 4 and 5, 4 and 7, 5 and 6, 6 and 7. There are no states on maximum possible distance 4.

The 16 possible code words determine 16 states and, read as binary numbers, number these states from 0 = 0000 to 15 = 1111, note that the numbering we gave before was according to the consecutive parts of the movements, see the listing we gave before.

Before testing the encoding on some experimental data we will discuss the 16 “states” S0, S1,…, S15 in which a movement can be.

Using the format of the 4-vectors is ( horizontal shift, vertical shift, number of vibrational consecutive turns, number of rotational consecutive turns). The components were reduced to 0 or 1 for shifts corresponding to negative shifts respectively non-negative shifts. For turns half the maximum value occurring was used to give 0 or 1, but one might also make the distinction no pair of consecutive turns versus at least one pair of consecutive turns.To get an idea of the movements that the code vectors stand for we will now describe some standard example movements. The shifts all have the same non-zero value, where as only one pair of turns is given in case the third or/and fourth component is 1.

We start with the vectors S0, S4, S8 and S12. These vectors only have shifts, no turns. Figure 6 gives the corresponding standard movements.

The effect of nonzero values in the third or/and fourth component will only be discussed for the case that both first and second component are 0. That means that we get superpositions on the standard movement encoded by S0.

We first consider S1 = (0, 0, 0, 1). There is, at least, one rotational pair, so an hv-pair or a vh-pair. The four standard movements are given in Figure 7. Note that a turn can be positive or negative.

Now consider S2 = (0, 0, 1, 0). There is, at least, one vibrational pair, so an hh-pair or a vv-pair. Due to the fact that the (+) and (−)-sign can have different order there are again


4 standard movements, given in Figure 8. Finally consider S3 = (0, 0, 1, 1). One vibrational pair and one rotational pair is

encountered in case there are three consecutive turns, with two consecutive turns of the same type; hhv, vhh, vvh or hvv. There are two sign orders for the vibrational pair and two signs for the third turn. So in all there are 16 standard motions for S3. These are

h+h−v+, h−h+v+, h+h−v−, h−h+v−, v+h+h−, v−h+h−, v+h−h+, v−h−h+, v+v−h+, v−v+h+, v+v−h−, v−v+h−, h+v+v−, h−v+v−, h+v−v+, h−v−v+.

4. Describing the whole movement

We can now describe the whole movement as a sequence of states. The movement in

Figure 2 is the sequence S12 → S0 → S8 → S12 → S2 → S12 → S1. Here with we have achieved complete similarity with the description of a sequence of activities during a meeting, each activity being a sequence of states. Suppose that we have video recordings and a person carries out a sequences of movements, for which specific names exist. We mentioned nodding, writing etc. When we analyze the waving of a hand, so point 6 or point 8 of Figure 1 and find that there is a sequence of states S12 → S0 → S8, then we can inversely conclude that the movement of Figure 1 starts with waving. The observed sequence of states of the whole movement may not have partial sequences that precisely match the sequence characteristic for some specific movement. Then we face the problem to determine which of the specific movements comes closest to a chosen sequence of observed states. In [2] the answer to the question: “what happens at that time?” was answered by considering the last states occurring before the chosen time. In

h+ v+ h− v+ v−h+ v−h−

Figure 7. Four movements encoded by S1

h+h− h−h+ v+v− v−v+

Figure 8. Four movements encoded by S2

S0 =(0,0,0,0) S4 =(0,1,0,0) S8 =(1,0,0,0) S12 =(1,1,0,0)

Figure 6. Four standard movements


our approach this means that the states during the last three time periods are determined. In Figure 2 at t = 5 the partial sequence S8 → S12 → S2 is found. From this the most likely from the specific movements is to be determined.

Supposed there are 20 specific movements then there are 20 specific sequences or states, any sequence having S8 → S12 → S2 as partial sequence gives a specific movement that may be going on.

In [1], Broekhuijsen, Poppe and Poel give Figure 1 as describing 2D joint locations on the human body.

Now any movement will involve all points to a certain extent, but in many specific movements certain points are involved in particular. Suppose we consider nodding by the head. In case of consent the nose, point 2, will describe a vertical vibration. In case of doubt a slow horizontal vibration might be observed, whereas a more rapid horizontal vibration indicates lack of consent.

Movement of point 6 to the left, to the left in the next time unit, to the left again and then a return to the original position may be observed and recognized as writing. Nodding “yes” is a sequence of identical states in which there is no shift, so the first two components of the code word are 0 and 1 say and there might be consecutive returns, so that the code word would be (0, 1, 1, 0) for this specific movement of point 2, representing the nose. (0, 1, 1, 0) is state S6 in our encoding of states. So nodding “yes” is recognizable from the sequence of states S6→S6→S6 … of point 2. Writing with the right hand, point 6, may lead to some states in which a rest position, state S0 = (0, 0, 0, 0) is alternating with a “shift state” S8 = (1, 0, 0, 0) followed by a “shift + double return” state S10 = (1, 0, 1, 0), leading to a sequence S0→S8→S0→S8→S0→S8 →S10→S0 →S8→S0→S→… of point 6 or 8. In similar ways specific movements lead to specific state sequences.

Dependent on the number of characteristic points, 9 in Figure 1, the encoding should be given simultaneously for all these points. The natural way is to consider a matrix containing a code vector for each characteristic point as a row of the matrix. So in our example we have 9 rows of 7 elements. The whole movement is described by a 9×7 matrix.

5. Similarity of two movements

In order to classify an observed movement we have to calculate the similarity of that

movement with the standard movements. Let us therefore consider two encoded movements and compare them. We assume that

there are c common states and that there are d states in each code vector not occurring as state in the other code vector. Both code vectors have length c + d.

Now we consider the following three aspects: (i) S1: the similarity in elements, (ii) S2: the similarity in ordering of the common elements, (iii) S3: the similarity in position of common elements with respect to the

non-common elements. Given two sets A and B, the similarity measure often used is

S = | A ∩ B | / | A ∪ B |. However, a movement consisting of a number of consecutive states, may contain a

specific state more than once. For that reason we do not consider sets, but so-called bags, in which elements are not necessarily all distinct. As an example we consider A = {a, b, b}


and B = {b, b, b, c, c}. We define the union ∪* of A and B as A ∪* B = {a, b, b, b, c, c} and the intersection ∩* of A and B as A ∩* B = {b, b}, i.e. we consider the maximum number of occurrence for elements in the union and the minimum number of occurrence for elements in the intersection of bags.

For (i), we now have the following formula: S1 = | A ∩* B| / | A ∪* B|,

here |A| denotes the number of the elements of the bag A. For the code vectors that we consider this measure gives S1 = c / (c + 2d).

As to (ii), we consider two code vectors with totally the same bag of elements but having different orders. Then we define d2 as the smallest number of transpositions needed to transform from one code vector to the other. Let us look at the following example.

M1 = [2, 3, 9, 6, 1, 5, 7], M2 = [2, 3, 1, 5, 7, 6, 9]. Then, if we want to change M2 into M1, we can do it like this: M2→ [2, 3, 1, 5, 7, 9, 6]→ [2, 3, 1, 5, 9, 7, 6]→ [2, 3, 1, 9, 5, 7, 6]→[2, 3, 9, 1, 5, 7, 6] → [2, 3, 9, 1, 5, 6, 7]→[2, 3, 9, 1, 6, 5, 7]→[2, 3, 9, 6,1, 5, 7]. It is easily seen that this is done with the smallest number of steps needed, so here d2=7.

If d2max represents the maximum value of d2, then we define D2 = d2 / d2max ,

where d2max = (1/2) ⋅ m ⋅ (m −1), and m is the number of elements of either of the two code vectors. Then we can define S2 as

S2 = 1 − D2. (iii) We use an example to show how to calculate S3. Let M3 and M4 be two code

vectors, where M3 = [x, 1, x, 2, 8, 9, 3, x, 4, 5, 6, 7, x], M4 = [y, y, 2, 8, 9, 1, 5, y, y, 4, 7, 3, 6,]. First, we consider the common elements {1, 2, 3, 4, 5, 6, 7, 8, 9}. Replacing them by the letter c we change M3 and M4 into the following form:

M3′ = [x, c, x, c, c, c, c, x, c, c, c, c, x], M4′ = [y, y, c, c, c, c, c, y, y, c, c, c, c]. We now define d3 as the sum of the position differences of all the common elements.

The positions are, for M3′, (2, 4, 5, 6, 7, 9, 10, 11, 12) and, for M4′, (3, 4, 5, 6, 7, 10, 11, 12, 13). So in this example d3 = 1 + 0 + 0 + 0 + 0 + 1 + 1 + 1 + 1 = 5.

Similar to the procedure in (ii), we have D3 = d3 / d3max and S3 = 1 − D3 .

Note that d3max occurs if all four x’s are at the beginning and all four y’s are at the end. The common elements are on positions [5, 6, 7, 8, 9, 10, 11, 12, 13] respectively [1, 2, 3, 4, 5, 6, 7, 8, 9], so d3max = 9×4 = 36.

At last, we define the total similarity ST as ST = S1 ⋅ S2 ⋅ S3.

Apparently, 0 ≤ ST ≤ 1 because S1, S2, S3 are all between 0 and 1.

6. Using the encoding on experimental data The use of the encoding will be illustrated on the data presented in Appendix A. Video

recordings were used to measure the positions of body parts, of which two are chosen: toph(ead) and righth(and). This was done both manually (M) and automatically (A) at 61 moments in time, from frames chosen from 7500 frames, at times with consecutive equal differences. Both x−and y−position were measured for two persons p1 and p2.

The turns in horizontal (x−) direction and vertical (y−) direction were determined by finding consecutive identical values preceeded and followed by higher values (+turns) or by lower values(−turns) and choosing the middle of the identical values as time at which


the turn took place. For the chosen time interval we find the sequence v+ h+ v− v+ of turns. There are two

rotational pairs, v+ h+ and h+ v−, and one vibrational pair v− v+. in case a horizontal turn and vertical turn occur at the same time we choose their order in such a way that with the aforegoing turn a vibrational pair is formed. This choice is motivated by the fact that vibrational movements of body parts seem more likely.

We have no interpretation of the movements at our disposal here. The whole sequence of 60 position shifts may belong to one type of activity. We can not do more, in first instance, than illustrate the encoding and compare the manual data with the automatic data by comparing their encodings. Completely arbitrarily we considered sequences of 6 time intervals as “activities”, leading to 10 larger time intervals, that could now be encoded. In case a turn happened to take place at the boundary of two such intervals it was decided to let it be part of the second one, as it only manifests itself due to the second interval.

We found the following pairs of state sequences, starting from frame number 1. Here P1 and P2 are person 1 and person 2 respectlively.

Mtoph(p1): S4→S11→S15→S0→S12→S0→S15→S2→S10→S5 Ahoph(p1): S12→S0→S9→S2→S14→S9→S15→S1→S15→S7 Mrighth(p1): S8→S6→S5→S11→S4→S8→S10→S0→S3→S12 Arighth(p1): S11→S7→S7→S9→S4→S10→S7→S8→S14→S3 Mtoph(p2): S8→S0→S15→S1→S5→S8→S12→S2→S9→S4 Atoph(p2): S9→S2→S15→S1→S14→S10→S6→S2→S8→S4 Mtighth(p2): S12→S0→S6→S1→S0→S4→S8→S0→S4→S12 Atighth(p2): S4→S8→S4→S9→S13→S3→S12→S0→S9→S15. These encodings should be quite similar as both manual and automatic measurements

are carried out on the same data. However, the encodings are quite different. We can apply the similarity measure presented in Section 5 and then this is confirmed.

The similarity measure does not take into account that some movement types are similar. This comes forward in the Hamming distances between pairs of code vectors, so in the number of components that are different. S0 = (0, 0, 0, 0) and S15 = (1, 1, 1, 1) have the maximum distance 4. Indeed, the movements “negative shifts without vibrational or rotational pairs of turns” and “positive shifts with both vibrational and rotational pairs of turns” really are of different type.

We may therefore represent the sequences by (0, 1)-vectors with 40 components. For the first pair of sequences this yields

Mtoph(p1): (0 1 0 0 | 1 0 1 1 | 1 1 1 1 | 0 0 0 0| 1 1 0 0 | 0 0 0 0 |1 1 1 1 | 0 0 1 0 |1 0 1 0|0 1 0 1) Atoph(p1): (1 1 0 0 | 0 0 0 0 | 1 0 0 1 | 0 0 1 0| 1 1 1 0 | 1 0 0 1 |1 1 1 1 | 0 0 0 1|1 1 1 1|0 1 1 1). The Hamming distance between these two 40-vectors is 1+3+2+1+1+2+0+2+2+1=15.

So even when taking similarity of types into account the manual and automatic encodings of the same movement give rather different encodings with differences in 15 components, out of 40. The encodings of the shifts only show differences in 5 components, out of 20. This seems somewhat better, but still is too large a difference to make the automatic encoding trustworthy.

An even more important feature is the following. Considering just one, manual, series of measurements gives an encoding. These measurements start with frame number 1. However, if the encoding is based on measurements starting with frame number 125, 250,


375, 500 or 625, with the same interval length, the resulting encodings of the same movement show considerable dissimilarity.

As conclusions of using the encoding technique we have that manual and automatic measurements may lead to considerably different encodings and that the encodings are rather sensitive to the choice of the starting point of the measurements. 7. Testing the encoding of experimental data

The goal of classification of motions is to have a possibility to recognize “what is

going on”, preferably in an automatic way. As we do not have such measurements of a number of motions of the same type, we

went ahead as follows. The data came from measurements of a 5 minute video recording of two persons. Four persons were asked to classify, recognize, activities during these 5 minutes, that yielded the 7500 frames, 25 per second. The four persons, quite consistently, distinguished four activities, these were:

Listening 1: from frame number 1 to frame number 1375. Writing : from frame number 1375 to frame number 2000. Listening 2: from frame number 2000 to frame number 6125. Talking : from frame number 6125 to frame number 7000. We now encoded these four activities by encoding the motions of tophead, righthand

and lefthand, as measured manually, for both persons in the mentioned time intervals. It turned out that one person was left handed! The results were:

person 1: tophead: S1→S4→S3→S12 person 2: tophead: S0→S13→S3→S12 person 1: rightHand: S11→S3→S11→S7 person 2: righthand: S13→S13→S3→S13 person 1: lefthand: S12→S0→S11→S4 person 2: lefthand: S0→S0→S15→S7 .

To illustrate the encoding of combined movements, there three movements, we give the matrix encodings which the rows are corresponding to tophead, righthand, left hand:

During listening 1: p1:

001111011000

, p2:

000010110000

,

During writing: p1:

000011000010

, p2:

000010111011

,

During listening 2: p1:

110111011100

, p2:

111111001100

,

During talking: p1:

001011100011

, p2:

111010110011

.

Comparing the encodings during the two activities “listening” the encodings for


person 1 differ on four elements of the two matrices, 3 of which for the left hand. For person 2 the matrices differ on 9 elements, 4 of which for the left hand. Comparing, for person 1, the encoding for listening 1, with that for writing we find 5 different elements and comparing that for listening 1 with that for talking we find 6 different elements. Comparing writing with talking gives 3 different elements.

Remark: if we take into account all the parts of the body, for which measurements exist, we expect better distinction of the typical movements during the different activities. That is, under the assumption that such typical movements exist!

8. Agitation

One of the difficult problems is to find out when one activity ends and another starts

on the basis of the measured positions. Any indicator for such a transition should be given by changes. This idea, and the fact that our personal observations revealed rather erratic movements during the whole recording, led to the idea that the agitation of a movement might be an essential part of the movement.

Our data are sequences of positions, values, stemming from continuous movements of body parts. Let f(t) be a continuous function giving the position of a point in one dimension. Let f(t) have consecutive extremal values f0, f1, …, fn, on the time interval [0, T].

Definition 1: The agitation A of f(t) on the time interval [0, T] is

A= ∑−

=+ −

1

01 ||1 n

kkk ff

T

In Appendix B we give the differences in value for consecutive times of measurements for person 1. From these tables the total agitation, i.e. the sum of the agitations of movements of all body parts can be calculated. For one time period, this sum is proportional to the sum of the absolute values in a row. Total agitation is clearly high for frame numbers 375, 750, 1375, 2125, 2500, 4875, 5500 and 6625, see Figure 10. The conclusion is that agitation can indeed be seen as an indicator for a change in activity. Note, however, that frame numbers 375 and 4875 are indicated, although the human observers did not consider these moments as moments of change of activity.

9. Discussion

Because movements during activities are erratic, it is hard to classify activities as seen

on video recordings by classifying the movements of body parts during the activities. But the classification is still useful. A vibrational movement of the nose can be

classified as such. Whether such a movement is due to the person nodding “yes” or “no” is not certain, but rather likely.

It seems that the classification will work well there where movements are defined well. An example might be figure ice skating. Between the simple skating periods certain jumps are carried out, like triple rittberger or double salchow. Movements like these should lead to classifications by encodings that have a high correctness rate in recognizing which specific jump was carried out.

The example also confirms the usefulness of the agitation measure that, when having a high value, indicates when the, complex, jumps are carried out.


0

50

100

150

200

250

300

350

400

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 7000 7500 Frame Number

Sum of absolute change

REFERENCES [1] Jeroen Broekhuijsen, Ronald Poppe and Mannes Poel, (2006) ‘Estimating 2D Upper Body

Poses from Monocular Images’, Human Media Interaction Group, Department of Computer Science, University of Twente, The Netherlands, internal report.

[2] Cornelis Hoede and Xin Wang, (2007) ‘Classification of meetings and their participants’, Memorandum No. 1826, Department of Applied Mathematics, University of Twente, The Netherlands.

Appendix A:

Table 1 The positions of the movement of person 1

Frame number

Mtop head.x

Mtop head.y

Mright hand.x

Mright hand.y

Atop head.x

Atop head.y

Aright hand.x

Aright hand.y

MLeft hand.x

Mleft hand.y

1 220 330 215 135 225 335 220 140 160 110 25 220 330 215 135 225 340 215 135 160 110 250 220 330 215 135 225 340 220 135 160 110 375 190 335 200 150 195 345 215 140 170 230 500 190 335 200 150 195 345 245 135 170 230 625 190 335 200 150 195 345 235 140 170 230 750 190 335 280 85 195 340 275 80 170 230 875 190 335 280 85 195 340 275 80 170 230

1000 220 345 280 85 220 350 275 80 170 230 1125 220 345 280 85 225 350 270 80 190 230 1250 220 345 280 85 220 345 275 75 190 230 1375 210 325 245 125 215 330 235 130 190 120

Figure 10. Total agitation per time interval


1500 195 325 280 90 195 325 275 85 190 100 1625 190 325 280 90 195 325 280 90 185 100 1750 190 325 270 100 185 320 265 95 175 95 1875 220 340 270 100 225 340 260 90 175 95 2000 215 360 255 110 220 360 255 115 175 105 2125 210 350 220 220 210 350 310 140 190 225 2250 215 345 225 225 210 345 260 150 190 215 2375 200 330 220 220 205 330 275 170 190 215 2500 200 330 275 100 190 330 260 95 170 220 2625 190 330 275 100 195 330 260 95 170 220 2750 190 330 265 95 190 335 250 90 170 220 2875 190 330 265 95 190 335 255 85 170 220 3000 190 330 265 95 190 335 260 90 170 220 3125 190 330 265 95 190 340 265 95 170 220 3250 190 330 265 95 185 340 265 90 170 220 3375 190 330 265 95 185 340 260 90 170 220 3500 190 335 260 95 195 340 245 90 185 230 3625 190 335 260 95 185 335 250 95 185 230 3750 190 335 260 95 180 340 255 100 185 230 3875 190 335 275 85 200 340 270 85 170 230 4000 180 330 275 85 185 330 275 85 170 230 4125 180 330 275 85 185 330 275 85 170 230 4250 180 330 275 85 180 340 300 80 170 230 4375 180 330 305 85 180 340 300 85 170 230 4500 180 330 305 85 180 340 300 75 170 230 4625 180 330 305 85 180 340 300 90 170 230 4750 190 330 250 95 190 330 255 95 170 230 4875 235 345 275 95 240 345 275 90 200 100 5000 200 330 275 95 195 335 270 85 190 85 5125 210 350 275 95 210 350 270 90 190 85 5250 210 350 275 95 210 355 275 90 190 85 5375 210 350 275 95 215 345 275 90 190 85 5500 190 330 275 95 190 335 280 95 190 85 5625 190 330 275 95 190 335 275 95 190 85 5750 190 330 275 95 190 340 275 100 190 85 5875 200 330 275 95 205 340 275 100 190 85 6000 205 345 275 95 205 345 270 95 190 85 6125 210 350 275 95 210 350 270 90 190 85 6250 200 340 275 95 195 335 275 90 190 85 6375 200 340 275 95 205 345 270 90 190 85 6500 210 340 275 95 210 345 275 100 190 85 6625 210 340 235 220 205 340 260 150 190 85 6750 215 345 260 80 220 350 280 110 190 85 6875 200 345 260 80 205 350 255 85 190 85 7000 190 345 270 80 195 345 270 80 190 85 7125 190 345 270 80 200 350 265 90 190 85


7250 205 340 270 80 210 340 275 100 190 85 7375 220 350 270 80 220 350 265 85 190 85 7500 220 350 265 85 220 355 260 85 190 85

Table 2 The positions of the movement of person 2

Frame number

Mtop head.x

Mhop head.y

Mright hand.x

Mright hand.y

Atop head.x

Atop head.y

Aright hand.x

Aright hand.y

MLeft hand.x

MLeft hand.y

1 480 370 530 110 480 380 535 110 460 110 125 480 370 530 110 470 390 545 115 460 110 250 480 370 460 175 475 395 545 180 535 185 375 480 370 445 165 480 375 535 175 535 175 500 480 370 445 155 490 375 535 175 535 170 625 490 365 445 155 490 370 535 175 535 170 750 490 365 445 155 480 375 530 175 535 170 875 490 365 445 155 485 375 530 175 535 170

1000 490 365 445 155 500 370 535 175 535 170 1125 490 365 445 155 490 375 535 175 535 170 1250 480 385 535 115 480 390 530 105 495 135 1375 460 360 535 115 460 360 525 115 445 105 1500 445 320 520 105 445 320 555 140 440 95 1625 470 360 510 105 470 365 550 140 440 95 1750 445 330 500 105 445 335 545 135 440 95 1875 450 345 500 105 450 350 545 140 440 95 2000 465 380 560 155 470 395 560 155 430 95 2125 445 400 540 155 445 400 535 165 410 80 2250 445 400 540 155 445 405 530 160 420 80 2375 445 400 540 155 450 405 535 165 420 90 2500 445 400 540 155 440 405 530 165 430 90 2625 440 400 550 145 435 400 540 175 430 90 2750 470 405 550 145 465 405 500 155 470 280 2875 410 390 545 155 415 390 525 155 420 280 3000 410 390 545 155 410 390 525 155 420 280 3125 410 390 545 155 415 400 535 170 420 280 3250 410 390 545 155 415 395 525 155 420 280 3375 420 395 545 155 420 400 535 175 470 120 3500 420 395 545 155 420 400 535 170 470 90 3625 420 395 545 155 420 400 530 165 470 90 3750 420 395 545 155 420 400 530 165 470 90 3875 420 395 545 155 425 405 540 175 470 90 4000 430 395 520 160 425 400 530 175 470 90 4125 430 395 520 160 440 405 520 165 470 90 4250 430 395 520 160 425 405 525 170 470 90 4375 430 395 520 160 430 400 525 170 470 90 4500 430 395 520 160 430 400 515 165 470 90 4625 430 395 520 160 430 405 525 170 470 90 4750 430 395 520 160 425 400 525 175 470 90


4875 430 395 520 160 430 405 530 180 470 90 5000 440 400 520 160 435 405 530 170 470 90 5125 440 400 520 160 435 405 530 170 470 90 5250 440 400 520 160 430 410 525 170 470 90 5375 430 400 520 160 430 405 525 170 460 100 5500 430 400 520 160 430 405 525 175 460 100 5625 430 400 520 160 425 405 530 175 460 100 5750 420 400 520 160 420 400 530 170 460 100 5875 430 400 520 160 435 405 520 165 460 100 6000 425 400 520 160 425 405 535 170 460 100 6125 450 370 525 115 455 375 550 135 450 95 6250 470 370 525 115 475 370 540 135 450 95 6375 480 370 520 105 475 370 530 115 450 95 6500 480 365 520 105 480 365 525 105 455 90 6625 500 375 520 105 485 375 535 115 455 100 6750 475 370 520 105 480 370 550 145 445 100 6875 475 365 520 105 475 365 540 130 445 100 7000 475 365 520 105 475 370 545 140 445 100 7125 475 365 520 105 475 375 525 110 445 90 7250 475 365 520 105 480 370 550 135 445 90 7375 490 365 520 105 490 370 550 125 450 100 7500 480 365 570 130 480 365 575 140 420 100

Appendix B: The differences in value for consecutive times of measurements for p1

Framenumb

er

tophead.

x

tophead.

y

bottomhead.x

bottomhead.y

leftshoulder.x

leftshoulder.y

rightshoulder.

x

rightshoulder.

y

lefthand.

x

lefthand.

y

righthand

.x

righthand

.y

leftelbow.

x

leftelbow.

y

rightelbow.x

rightelbow.y

leftuparmwidt

h

leftlowerarmwidt

h

rightuparmwidth

rightlowarmwidth

125 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 375 -30 5 -10 10 -10 -15 -10 5 10 120 -15 15 20 -30 -20 -15 0 5 0 0 500 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 625 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 750 0 0 0 0 0 0 5 -15 0 0 80 -65 0 0 20 20 0 0 0 5 875 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

1000 30 10 10 20 10 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 1125 0 0 0 -10 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 1250 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1375 -10 -20 -5 -20 0 0 0 0 0 -110 -35 40 -15 20 5 -5 0 0 0 0 1500 -15 0 0 0 -10 0 -10 10 0 -20 35 -35 0 0 0 0 0 0 0 0 1625 -5 0 -5 -20 0 0 0 0 -5 0 0 0 0 0 -5 0 0 0 0 0 1750 0 0 0 0 0 0 0 0 -10 -5 -10 10 0 0 0 0 0 0 0 0 1875 30 15 15 15 10 5 10 -10 0 0 0 0 0 0 0 0 0 0 0 0


2000 -5 20 0 30 0 10 0 10 0 10 -15 10 0 0 0 0 0 0 0 0 2125 -5 -10 0 0 0 0 0 0 15 120 -35 110 35 -5 -30 0 0 0 0 0 2250 5 -5 10 0 0 0 0 0 0 -10 5 5 -5 0 0 0 0 0 0 0 2375 -15 -15 -5 -10 0 0 0 0 0 0 -5 -5 -5 -5 0 -5 0 0 0 0 2500 0 0 0 -10 -5 -20 0 -10 -20 5 55 -120 0 0 10 10 0 0 0 0 2625 -10 0 -10 -5 0 0 -5 5 0 0 0 0 0 0 0 0 0 0 0 0 2750 0 0 0 0 0 0 0 0 0 0 -10 -5 0 0 0 0 0 0 0 0 2875 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3125 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3375 0 0 -10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3500 0 5 0 0 -5 0 0 0 15 10 -5 0 0 0 0 0 0 0 0 0 3625 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3750 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3875 0 0 -10 -15 0 0 0 0 -15 0 15 -10 0 0 0 0 0 0 0 0 4000 -10 -5 10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4125 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4375 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 4500 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4625 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4750 10 0 10 0 0 0 0 0 0 0 -55 10 -5 0 0 0 0 0 0 0 4875 45 15 10 0 0 10 10 0 30 -130 25 0 -5 0 5 -10 0 0 0 -5 5000 -35 -15 -5 0 0 0 0 0 -10 -15 0 0 -10 -10 0 0 0 0 0 5 5125 10 20 -5 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5375 0 0 0 0 -10 0 -10 0 0 0 0 0 0 0 0 0 0 0 0 0 5500 -20 -20 0 -15 10 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 5625 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5750 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5875 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6000 5 15 5 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6125 5 5 -10 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6250 -10 -10 5 -20 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 6375 0 0 -5 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6500 10 0 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6625 0 0 15 0 0 0 -5 0 0 0 -40 125 0 0 -30 -10 0 0 0 -5 6750 5 5 -5 0 0 0 0 0 0 0 25 -140 0 0 20 10 0 0 0 0 6875 -15 0 -10 5 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 7000 -10 0 -5 0 0 0 0 0 0 0 10 0 0 0 5 0 0 0 0 5 7125 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7250 15 -5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7375 15 10 15 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7500 0 0 0 0 0 0 5 0 0 0 -5 5 10 10 0 0 0 0 0 0


Xin Wang, Research Center for Information and Control, Dalian University of Technology, Dalian 116026, PR China. Email: [email protected]. Wang is a lecturer at Department of Mathematics, Dalian Maritime University. She received her BSc degree in Mathematics from Liaoning Normal University in 2000, MSc degree in Mathematics from Dalian Maritime University, China in 2003. Her research interests are in the areas of machine learning, fuzzy theory and methods.

Cornelis Hoede, Department of Applied Mathematics, University of Twente, P.O.Box 217, 7500 AE, Enschede, The Netherlands. Email: [email protected]。Dr. Hoede is an emeritus professor of the University of Twente.He received his MSc. degree in Theoretical Physics from the University of Amsterdam in 1962. In 1965 he obtained his PhD in 1968 on the Ising problem, a subject in statistical mechanics.His research interests are very diverse, but mainly center around applications of Discrete Mathematics. He has

written about 150 papers on subjects like diffusion, statistical mechanics, block designs, graph theory, combinatorics, cooperative game theory, knowledge representation, mathematical sociology and linguistics.

Xiaodong Liu, Research Center of Information and Control, Dalian University of Technology, Dalian 116024, PR China. Email: [email protected]. He received the B.S. and M.S. degrees in mathematics from Northeastern Normal University, Shenyang, China, and Jilin University, Jilin, China, in 1986 and 1989, respectively, and the Ph.D. degree in control theory and control engineering from Northeastern University, Shenyang, China, in 2003. Dr. Liu is a professor of Dalian University of Technology. He has been a Reviewer for the American Mathematical Review since 1993. His research interests include algebra rings, combinatorics, topology molecular lattices, axiomatics

fuzzy sets (AFS) theory and its applications, knowledge discovery and representations, data mining, pattern recognition and hitch diagnoses, and analysis and design of intelligent control systems.





251

IMPROVE THE RELIABILITY OF EPIDEMIC DATA DISSEMINATION IN COMPLEX NETWORKS WITH LITTLE WASTE

YUTAN ZOU ZHIPING WANG

Abstract: We study the dynamics of epidemic spreading processes aimed at spontaneous

dissemination of information updates in populations with complex connectivity patterns. From Yamir

Moreno’s paper [1], the existence of hubs in scale–free networks results in a lower reliability but a

better efficiency. In this paper, we introduce a way to exploit the advantage of the hub’s large degrees

but with a lower cost. By analyzing the behavior of several parameters, such as infect-rate,

increase-waste (the number of the new useless connects), we get the results of value of the parameter

α of hubs’.

Key words: reliability, efficiency, scale–free networks, epidemic spreading processes, infect-rate,

increase-waste.

.

1. Introduction

Modern society increasingly relies on large-scale computer and communication networks, such as the Internet. A major challenge in these networks is the development of reliable algorithms for the dissemination of information from a given source to thousands, or even millions of users, such as for news and stock exchange updates, mass file transfers and Internet broadcasts [2, 3]. In epidemic-inspired communication this is achieved by exploring a mechanism analogous to the spreading of infectious diseases in populations [4]. Indeed, epidemic data dissemination in computer and communication networks shows interesting parallels with both disease propagation in populations and the spread of rumor in social networks. The information spreads like a benign epidemic through local interaction between nodes which forward the message they receive to a random selection of their peers in the network, until the whole system becomes “infected” with information. The great advantages of epidemic-style communication is that dissemination proceeds on a local basis, without any co-ordination from a central organizing body [4]. These protocols are also highly resilient to sudden failure of communication links and nodes.

A relevant result in the mathematical theory of epidemics is that the spreading of



252 Y. ZOU AND Z. WANG

infection in a population is strongly affected by the patterns of connectivity in the underlying contact networks. In particular in scale-free topologies, characterized by degree distributions with power law behavior p(k)～k-r [5, 6, 7], the statistical relevance of hubs makes the network highly permeable to attacks [8, 9, 10] and the spreading of infections [11] and highlights the need of special immunization strategies. This result suggests that the topology of the underlying computer and communication network might heavily affect the performance of epidemic-style data dissemination protocols. Surprisingly, however, the impact of network topology on such protocols has not been thoroughly explored, although the results could have an important technological value. Indeed, these protocols potentially find a large spectrum of application such as mobile communication networks and, more recently, resource discovery in the so-called peer-to-peer systems built on top of the Internet [12], and finally in grid computing [13].

2 .Brief introduction of a simple epidemic data dissemination model

In this paper we use a simple epidemic data dissemination model defined in Yamir Moreno’s paper [1] and perform a detailed numerical study of the dynamics of the information propagation in networks with diverse topological properties. Our basic model is a slightly modified version of the Daley and Kendall (DK) model [14, 15, 16] and it can be considered as the simplest epidemic algorithm for the updating of distributed databases [17]. We study the main relevant features of the model such as the infectant rate of a node which has k degrees. The results obtained point out that the more degree a node has, the less infectant rate it has.

The model [1] to be considered is defined in the following way. Each of the N elements of the network can be in three possible states. We call a node holding an update and willing to transmit the update a spreader. Nodes that are unaware of the update will be called ignorants while those that already know the update but are not willing to spread it anymore are called stiflers. We denote the density of ignorants, spreaders and stiflers at time t as ψ(t), Φ(t) and s(t) respectively such that for all t, ψ(t)+Φ(t)+s(t)=1. The spreading process takes place along the links between spreaders and ignorants. Each time step spreaders contact one (or more) neighboring nodes. When the spreader contacts an ignorant, the latter one turns into a new spreader at a rate λ. On the other hand, the spreader becomes a stifler with rate 1/α if a contact with another spreader or a stifler takes place. The parameter α can be considered as the average number of contacts with spreader/stifler nodes before the spreader turns itself into a stifler. This dynamics mimics the attempt of diffusing an update or rumor by nodes which have been recently updated. At the same time, if a node attempts too many times to communicate the update to nodes which have already received it, it will stop the

THE RELIBILITY OF EPIDEMIC STATE TRANSIMISION 253

process and turn itself into a stifler. In other words, the node realizes that the update has lost its novelty and becomes uninterested in diffusing it. The present dynamics thus introduces a trade-off in maximizing the number of updated nodes and minimizing the number of attempted contacts. Obviously, the efficiency of the spreading process will depend on the rate at which individuals lose interest in further spreading the rumor and the topology of the underlying network[1].

3. A way to exploit the advantage of the hub’s large degree

In this model, the waste (the useless connect) comes from two aspects. First, connecting to an ignorant while it has not become a spreader. This is determined by the parameter λ. Second, connecting to a spreader or a stifler. This is determined by the parameter α. In order to reduce the waste, we should increase λ and 1/α. But we can’t increase 1/α too much, because it will also reduce the reliability. If α is increased, reliability will improve but waste will increase too. This paper only discusses the parameter α only. Here the word “waste” only means the second aspect. In fact, this waste is a necessary cost for the dissemination of information. We can use the hubs’ bigger potential not only to increase more reliability but also to reduce waste.

In order to pay little waste for increasing reliability, we will consider how to improve the epidemic data dissemination model. We will increase hubs’ α only. (The other nodes will hold the old α). From Yamir Moreno’s paper the hubs die so early that many of their neighbors are still ignorants. That is to say that hubs’ transmission capacity is not completely used. Hubs have more potentials than the other nodes.

In order to characterize the propagation process, we first focus on the infect nodes’ number Ninfect of the networks which have gotten the update when the process dies out. The experiment is done on a BA network whose size is 1000 nodes. The more α is increased, the later nodes will die, and the more waste will be produced.

TABLE 1. The hubs’ life is prolonged only. The infect nodes’ number Ninfect, defined as nodes that have gotten the update when the process dies out. In the BA networks (Its size is 1000 nodes), we prolong the life of hubs whose degrees are more than 80, for different value of the parameter increase-waste. Increase-waste 100 300 500 700 900 1100 1300 1500 1700 1900 Ninfect (Increase

hubs’ α only)

531 556 569 586 591 606 618 626 630 639


TABLE 2. The infect nodes’ number Ninfect, defined as nodes that have gotten the update when the process dies out. In the BA networks, we prolong all nodes life, for different value of the parameter increase-waste. Increase-waste 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

ectN

inf (increase

all nodes’α )

553 579 620 644 659 699 708 721 732 745

Comparing the two tables, we can see the result of the first table is better than the second one. In other words, the same increase-waste can cause more Ninfect in the first table than in the second one. (From another point of view you can pay much less increase-waste in the first method than the second one to get the same Ninfect). Maybe we should not consider the connections which to the spreader or stifler are a waste but are actually a necessary cost. In order to increase the reliability we must pay more. But we should try our best to pull in our spending. The result is: we should prolong the hubs’ life more than the other. But we should not spend too much on it. How much increase-waste should be spent on the hubs? A mathematical analysis must be made.

We have performed large-scale numerical simulations by applying repeatedly the rules stated above on star-network (a network configuration in which there is only one path between a central or controlling node and each endpoint node.). We choose star-network, because it must exclude disturbs of the other nodes. Initially, we start from a single spreader (the central node) who is willing to spread the update through the network. We assume it has k degrees. All its neighbors are ignorants. At the first step, the average number of the nodes who are ignorants will become k(1-λ),and the average numbers of the infect nodes will become k-k(1-λ) (we do not consider the central node). At the second step, the average number of the nodes who are ignorants will become k(1-λ)2,and the average numbers of the infect nodes will become k- k(1-λ)2. With the mathematical induction we can get the conclusion that at the mth step, the average number of the nodes who are ignorants will become k(1-λ)m, and the average number of the infect nodes will become

mkk )1( λ−− (1)

And we know that the average number of contacts with spreader/stifler nodes before the

spreader turning itself into a stifler is α. If the central node becomes a stifler at the mth

step, the average number of m satisfies the equation


αλ =−−∑=

])1([1

m

n

nkk (2)

From (2) we can get another equation

kmm λαλλλ +−=+− + )1()1( 1 (3)

We assume the function f(m)=(1-λ)m+1+mλ, and it is easy to get the result that f(m+1) = f(m) + λ(m+1) – λ(1-λ)m+1. With the real meaning of λ, (0<λ≤1), We can prove the result of f(m)< f(m+1); with (3), We can prove if k increases, m will decrease. In order to characterize the propagation process, we first focus on the infect rate w of a k-degrees

node rumor propagation defined as mm

kkk )1(1)1( λλ

−−=−− .

It is obvious, if k increases, m will decrease and w will decrease too. There is an interesting result in Yumir Moreno’s paper [1]. In that paper, the first focus is on the reliability R of the rumor propagation defined as the final density s(t) of nodes that have got the update when the process dies out. There is an important result in that paper: the existence of hubs in scale–free networks makes the network appear less reliable. The reason, to us, is that hubs have large degree and as a result their infect rate is small. Also, their ignorant neighbors are hard for other nodes to reach. So they could be isolated and never get the update which result accords with our result in this paper. In this case, our result is proved from another point of view. There is another useful result we can get. We know that the existence of hubs in the networks can improve the efficiency, but results in the loss of reliability. In fact, less w means more potential. We should pay more to prolong the life of the nodes who have more potential. But when we pay some increase-waste to it, its w will increase and its potential will become smaller. If its infect rate is not still smaller, we should not pay to it. Instead, we should choose the smaller one. From (2) and (3), get a result that

λλλλα /)]1()1[( 1 kmm −−+−= + (4)

m in the equation is

wm )1(1log λ−−= (5)

From (4) and (5), we can know how much the α we should pay by k, λ and w.


FIG.1. Infect rate of node that has k degrees as a function of their connectivity k.

5. Summary

Existence of hub in the network results in smaller reliability. It is dead so early that it holds back the reliability. In order to improve the reliability, a simple solution is to increase the value of α, which however may cause much useless-connect (connect to the stifler or spreader). This paper claims that prolonging the hubs’ life is wiser for economizing cost as the hub is more important for reliability in the networks. It will cost smaller waste and increase more reliability. Because this method based on the hubs’ infect rate w is smaller, it has more potential. But if its α is increased too much, its infect rate w will be more than the normal nodes, which as a result, will cause more waste. And we also reach a useful equation (5). Form this equation we can know how much the α we should pay by k, λ and w.

REFERENCES [1] Y. Moreno, Maziar Nekovee and Alessandro Vespignani4 “E_ciency and reliability of epidemic data

dissemination in complex networks”, Phy. Rev. E69, 055101(R)(2004)

[2] D. Kosiur, “IP Multicasting: The Complete Guide to Interactive Corporate Networks”, Wiley Computer

Publishing, John Wiley & Sons, Inc, New York (1998).


[3] S. Deering, “Multicast routing in internetworks and extended LANs”, in Proc. ACM Sigcom ’88, page 55-64,

Stanford, CA, USA (1988).

[4] W.Vogels, R V. Renesse,. and K. Birman,. ”The Power of Epidemics: Robust Communication for

Large-Scale Distributed Systems”, in the Proceedings of HotNets-I, Princeton, NJ (2002).

[5] A.L. Barabási, and R. Albert, Science 286, 509 (1999); A.L. Barabási, R. Albert, and H. Jeong, Physica A

272, 173, (1999).

[6] M. Faloutsos, P. Faloutsos, and C. Faloutsos, Comp. Com. Rev. 29, 251 (2000).

[7] R. Pastor-Satorras, A. Vázquez, and A. Vespignani, Phys. Rev. Lett. 87, 258701 (2001); A. Vázquez, R.

Pastor-Satorras, and A. Vespignani, Phys. Rev. E 65, 066130 (2002).

[8] D. S. Callaway, M. E. J. Newman, S. H. Strogatz, and D. J. Watts, Phys. Rev. Lett. 85, 5468 (2000).

[9] R. Cohen, K. Erez, D. ben Avraham, and S. Havlin, Phys. Rev. Lett. 85, 4626 (2000).

[10] R. Albert, H. Jeong, and A.-L. Barabási, Nature 406, 378 (2000).

[11] R. Pastor-Satorras, and A. Vespignani, Phys. Rev. Lett. 86, 3200 (2001).

[12] Peer-to-Peer: Harnessing the Power of Disruptive Technologies, ed. A. Oram ( O’Reilly & Associates, Inc.,

Sebastopol, CA, 2001).

[13] I. Foster and C. Kesselman, eds., The Grid: Blueprint for a Future Computing Infrastructure, Morgan

Kaufman, San Francisco (1999).

[14] D. H. Zanette, Phys. Rev. E 64, 050901(R) (2001).

[15] Z. Liu, Y.C. Lai, and N. Ye, Phys. Rev. E 67, 031911 (2003).

[16] D. J. Daley and J. Gani, Epidemic Modeling (Cambridge University Press, Cambridge UK, 2000).

[17] A. J. Demers, D. H. Greene, C. Hauser, W. Irish, and J. Larson, Epidemic Algorithms for Replicated

Database Maintenance. In Proc. of the Sixth Annual ACM Symposium on Principles of Distributed

Computing, Vancouver, Canada, 1987.

Yutang Zou Transportation and Logistics Engineering School, Dalian Maritime University,

Dalian, China, 116026. E-mail: [email protected]. Received M. S. Degrees in Vehicle

Operation Engineering, Dalian Maritime University Dalian in 1993. He joined the

Transportation and Logistics Engineering School, Dalian Maritime University, Dalian since

1988. His current research interest is in computer aided design, computer virtual technology,

complex, supernetwork etc. Zhiping Wang Department of Mathematics, Dalian Maritime University, Dalian, China,

116026. E-mail: [email protected]. Received M. S. Degrees in department of

mathematics, Dalian Maritime University, Dalian in 1988. Received PhD degree in college

of marine engineering, Dalian Maritime University, Dalian in 2003. He joined the

department of mathematics, Dalian Maritime University, Dalian since 1988. His current

research interest is in graph theory, complex, supernetwork and chain supply


Received by the editors August 15, 2009 258

AN EVOLUTIONARY VARIABLE NEIGHBORHOOD SEARCH FOR SELECTING COMBINATIONAL GENE SIGNATURES IN PREDICTING CHEMO-RESPONSE OF OSTEOSARCOMA

1KIT Y. CHAN, 2HAILONG ZHU, 3MEHMET E. AYDIN and 4CHING C. LAU

Abstract In genomic studies of cancers, identification of genetic biomarkers from analyzing microarray chip that interrogate thousands of genes is important for diagnosis and therapeutics. However, the commonly used statistical significance analysis can only provide information of each single gene, thus neglecting the intrinsic interactions among genes. Therefore, methods aiming at combinational gene signatures are highly valuable. Supervised classification is an effective way to assess the function of a gene combination in differentiating various groups of samples. In this paper, an evolutionary variable neighborhood search (EVNS) that integrated the approaches of evolutionary algorithm and variable neighborhood search (VNS) is introduced. It consists of a population of solutions that evolution is performed by a variable neighborhood search operator, instead of the more usual reproduction operators, crossover and mutation used in evolutionary algorithms. It is an efficient search algorithm especially suitable for tremendous solution space. The proposed EVNS can simultaneously optimize the feature subset and the classifier through a common solution coding mechanism. This method was applied in searching the combinational gene signatures for predicting histologic response of chemotherapy on osteosarcoma patients, which is the most common malignant bone tumor in children. Cross-validation results show that EVNS outperforms the other existing approaches in classifying initial biopsy samples.

Key Words: Variable neighborhood search, evolutionary algorithm, cancer gene, histologic response, osteosarcoma.

1. Introduction

Osteosarcoma is the most common malignant bone tumor in children and accounts for 60 percent of malignant bone tumors diagnosed in the first two decades of life [19]. It is possible that resistant tumor cells have additional time to either metastasize to the lungs or evolve further during the period when ineffective preoperative chemotherapy is given. Therefore initial diagnosis, which aims at identifying whether the patients are likely to have a poor response to standard preoperative therapy, is necessary.

In cancer research, microarray chip can simultaneously interrogate thousands of genes, which provides an extremely powerful tool for genomic studies of cancer. A few key genes (typically involving oncogenes and tumor suppressor genes), when mutated, will cause dysregulation of the transcription and translation of other genes through complicated signaling pathways to initiate oncogenesis, and ultimately leading to derangement of the cellular phenotype and the clinical manifestations of cancer [6, 12]. Significance based methods (e.g. T-test, Confidence intervals, etc.) [7], which aim at


INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCE Volume 1, Number 1, Pages 1-22

INTERNATIONAL JOURNAL OF INFORMATION AND SYSTEMS SCIENCES Volume 6, Number 2, Pages 258-274

PREDICTING CHEMO-RESPONSE OF OSTEOSARCOMA 259

finding statistically significant genes in differentiating various patient groups, have been extensively utilized. However, the philosophy of these methods is to evaluate each single gene one by one, thus neglecting the intrinsic interactions among genes. Therefore, methods to assess the function of gene combinations in regulating tumor patterns are highly desired. Supervised classification is the most effective machine learning method to map the input space (with multiple predictor genes) and the output space (with labeled conditions).

Commonly used learning algorithms include neural network [2, 16], k-nearest neighbor [18], decision tree, multi-layer perceptron [17], self-organizing maps [12], hierarchical clustering [8], graph theoretic approaches [14], and support vector machine (SVM) [11, 25, 34], have been employed to identify gene signatures. Among all of them, SVM has been proven to have the best capability in controlling the tradeoff between empirical risk and model complexity to achieve good prediction [1, 18, 29, 30]. It has many appealing properties for classification of microarray data in osteosarcoma [20], including measures to prevent overfitting and local minima that are associated with other classification algorithms.

In our recent study, an integrated approach of support vector machine (SVM) with a variable neighborhood search (VNS) algorithm, that can effectively solve the problem of simultaneously optimizing gene subset and the classification of osteosarcoma, is introduced [4]. The rationale behind the use of VNS is its high efficiency in searching a tremendous solution space that can reach better solutions than classical local search algorithms and faster convergence speed than stochastic algorithms like evolutionary algorithms [27]. VNS achieves this with a systematic change of neighborhood whilst searching through solution space so as to avoid local minima traps, which are the hardest drawbacks with metaheuristics. The main limitation of VNS implementations arises with its inbuilt neighbourhood functions, which restrain the search with spinning in some particular regions of the space. After searching in a long time, jumping to some other regions of the search space becomes almost impossible.

The most effective healing option appears to be hybridised VNS with other heuristics such as the evolutionary algorithms. The main aim of the resulting algorithm, namely evolutionary variable neighbourhood search EVNS, is to avoid local minima traps and/or to have faster convergence. This idea behind EVNS is to run many VNSs distributively in a parallel way. In EVNS, many VNSs work independently on different individuals of the population as evolutionary algorithms do. Evolutionary algorithms use genetic operators, crossover and mutation, to explore the search space, while EVNS uses VNS to explore the search space. This method is applied in finding the combinational gene signatures and building models for predicting chemo-response of osteosarcoma. To evaluate the performance and robustness, the results of the proposed method were compared with the existing methods, VNS [5] and evolutionary algorithm [4] in which the same mircoarray dataset [20] was used.

2. Problem Formulation and Solution Representation

2.1 Problem Formulation Let a gene microarray dataset D be l

iii y 1)},{( =x , where mi ℜ∈x is the gene

expression level of the i-th patient, }1,1{−∈iy is the condition label for binary classification problem, and m is number of gene features.

260 K. CHAN, H. ZHU, M. AYDIN AND C. LAU

The dataset after performing gene selection is defined as ( ) ( ) DDx ⊂== l

iii y 1)},{( with ( ) 'mi ℜ∈x , where function selects 'm

( m≤ ) gene features among all the m gene features from the gene expression data set D. For a new sample x , the decision function of a SVM classifier with

radial-basis-function (RBF) kernel can then be defined based on the selected gene subset: (1) ( ) )))(,(sgn(),,,,(

vectorssupport ∑= xxDx iii KayCf σ

where σ is the width parameter of the RBF kernel and C is the regularization parameter, ia is solved by optimizing a quadratic function

(2)

⋅−= ∑∑

==

l

jijijiji

l

ii KyyaaaW

1,1

)](),(([21min )( xxa

subject to Cai ≤≤0 . The support vectors are only corresponding to those items with

0>ia . To develop a robust SVM model based on the training set, the leave-one-out

cross-validation (LOOCV) was applied to optimize the model parameters (σ and C). In LOOCV, one sample is leaved out as testing sample, and the remained 1l − samples are used as training data. Let kD represent the training set

( ){ }, , 1, 1, 1, ,i iy i k k l= − +x �� , then the accuracy for a validation is calculated by:

(3) ( ) kkkk

k yCfy

CJ −= ),,,,(1,,, σσ DxD

Thus the overall accuracy is ∑=

l

kk lJ

1

. Now the problems of gene feature

selection and SVM parameter optimization are integrated to optimizing the above objective function (3).

2.2 Solution representation

Solutions of the above problem are represented in combination of both binary and real codes where binary coded representation is for the selection of gene features with , and real coded representation is for the SVM parameters σ and C. This scheme of representation is illustrated in Figure 1.


Figure 1 Solution representation

As illustrated in the left hand side of Figure 1, binary coded representation [3, 16] is composed of a fixed-length binary string to determine the usage of gene features by their corresponding genes. It has the form of the binary string with m bits such that m’ of entries are 1 and the rest are 0. A bit with 1-element means that the corresponding gene feature is selected in the subset of gene features while a bit with 0-element indicates that the corresponding gene feature is not selected. For instance, a solution of [0,1,0,1,0,0] with 2'=m , i.e. the number of 1-elements of the solution, and m=6, i.e. the number of bits of the solution, represents the 2nd and 4th gene features are selected. As illustrated in the right hand side of Figure 1, real code is adopted for representing the two SVM parameters, the kernel width parameter σ and the regularization parameter C.

The number of bits m is equivalent to the total number of genes, and the number of 1-elements 'm is the number of selected gene signatures. Thus the number of possible gene subsets cn can be calculated as the following:

(4)

=

'mm

nc

In general, the number of the genes contained in microarray data is very large. This will make the whole solution space extremely large, thus impair the efficiency and effectiveness of the algorithm. Therefore, utilizing a pre-screening procedure to filter out those noisy genes will remarkablely improve the performance of this algorithm. 3. Variable neighborhood search VNS Variable neighborhood search (VNS) [22, 27] could be used to solve the integrated gene feature selection and SVM classification problem defined in (3) due to its ease of use with remarkable success in solving hard combinatorial optimization problems [9, 13]. It has been proposed to solve the gene signature selection problem [4] as formulated in (3). Basically, it carries out exploration within a limited region of the whole search space.

m’

Binary coded representation Real coded representation

σ C

Solution representation

1 0 0 1 0 1 1 0 1 0

Kernel width parameter σ and the regularization parameter C

Selection of m’ gene features among the m genes


That facilitates a provision of finding better solutions without going further investigation. It is shown to be a simple and effective search procedure that explores the solution space with systematic change of neighbourhood. It searches in which a local search intensifies the exploration within a preferred neighbourhood until a certain level of satisfaction. Once a local search was finished with a neighbourhood, then another neighbourhood is systematically moved to. That refreshes the search and let the algorithm converge faster. Its main components, neighborhood functions (NFs), and its detailed procedures are discussed in Section 3.1 and 3.2 respectively. 3.1. Neighborhood functions (NFs) In VNS, the neighborhood functions (NFs) are the methods in which the neighboring solutions are determined through. Therefore, they are the key elements of VNS in success of metaheuristics with exploration through search spaces. Following two types of NFs are used for exploring the solution space of the integrated gene feature selection and SVM classification problem as defined in (4):

- ‘MutationBin’ is a neighborhood function used to explore solutions of the binary representation by exchanging the entries of a 0- and 1- elements. For instance, suppose that the 2nd bit with entry 1- element and 5th bit with entry 0- element of the solution [0,1,0,1,0,0] are selected to be exchanged. Thus the 2nd gene is selected as the gene signature, and the 5th gene is not. After applying MutationBin, the new solution will be [0,0,0,1,1,0]. Obviously, the elements of the 2nd and 5th bits were exchanged from 1 to 0 and from 0 to 1 respectively. Thus after the performing the operation MutationBin, the 5th gene is selected as the gene signature, and the 2nd gene is not.

- ‘MutationReal’ is a neighborhood function that implies small shake on a randomly choice of SVM classifier parameters in the real coded representation of the solution. The MutationReal function is defined as the following shake function:

(5) ω+= ppshake )( where p represents the randomly chosen parameter, and ω is randomly generated

within the range ( )minmax1.0 pp −× , representing 0.1 times scale of the parameter

space of the SVM classifier.

3.2. VNS VNS starts with a randomly selected initial solution, SxC ∈=],,[ σ , where S is the whole search space, and manipulates the solutions via steps (a) and (b), where two main functions, Shake Function and Local Search Function LSF, for intensification and exploration in search.

The pseudo-code of the variable neighborhood search (VNS) is illustrated follows:

Repeat the following Step (a) to (c) until the stopping condition is met: Step (a) Perform Shake Function: x’=MutationReal(x) Step (b) Perform Local Search Function: x’’=LSF(x’’) Step (c) Improve or not: if x’’ is better than x, do x’’’→ x

In Step (a), Shake Function generates and/or modifies the solutions regardless of the quality of solution so as to initializes a fresh search in a local neighborhood or to switch to another neighborhood. Then Step (b) carries out the major intensive search by Local Search Function (LSF), which a simple hill-climbing algorithm based on both


aforementioned NSs detailed in the appendix is used. It explores for an improved solution within the local neighborhood chosen. After that the outcome of local search function is evaluated whether or not to adopt it as the solution for further search. Shake Function and LSF need to be chosen so as to achieve an efficient VNS. The NF discussed in Section 3.1 are used for Shake Function and LSF to obtain neighborhood changes and local intensification in VNS. Since the purpose of Shake Function is to diversify the exploration, it is designed to switch to another region of the search space so as to carry out a new local search over there. In this study, Shake Function is not applied to the binary coded representation part of solutions, but is designed to conduct a random move within the real coded part. Thus, the given solution *x operated with the Shake Function to obtain x’ uses MutationReal(x*). That is reiterated until the termination condition is met. 4. Evolutionary Variable Neighbourhood Search (EVNS) VNS is able to converge to the optimum value, but it could be very expensive to obtain a desired solution in terms of computational time. It can be found from the literature that VNS has been either hybridized with other methods such as genetic algorithms or parallelized [10, 23, 31]. In this paper, the evolutionary variable neighbourhood search (ENVS) algorithm was developed to overcome the long computational time for solving the gene signature problem as formulated in (3). It offers an evolutionary process in which a VNS algorithm substitutes for the genetic operators to evolve a population of solutions. The pre-defined number of iterations in VNS algorithm is kept short and sufficiently compact so that it can be easily used in any evolutionary process as an operator. This makes the EVNS implementable in various environments, working alongside other methods. We embedded a shortened VNS into an evolutionary algorithm, which adopts the VNS as the only operator and does not contain any other reproduction operators (crossover, mutation). The EVNS algorithm for solving (3) is sketched below: Begin Initialise the population (X), Set the number of evaluations (N) Repeat: Select an individual (xn) Operate by the NVS and generate the new individual (xn’) Evaluate the new individual xn’ for replacement Until n>=N End.

After initialization and parameter setting, the algorithm repeats the following steps: (i) selects one individual xn subject to the running selection rule; (ii) generate a new individual x n’ by the VNS operator; and (iii) evaluates whether or not to put it back into the population through a particular replacement rule. The VNS operator is basically a metropolis algorithm, which is the original inspirational idea, where inner repetitions are kept optional.


Implementations of NVS differ depending on the setting of inner repetitions, which are set to stabilize the solution before the NVS stops exploring the solution space. This identifies the total number of evaluations per run of the NVS operator. Obviously, the only operator running alongside the selection is the NVS. Since the NVS operator re-operates on particular solutions several times, the whole method works as if it is explored the solution space every particular number of iterations. If we assume that there is a single solution operated by this NVS, it will become a multi-start (not multi-run) algorithm that reruns repeatedly. Thus, the novelty of ENVS can be viewed from two points of view: one is its multi-start property, and the other is its evolutionary approach. The multi-start property provides ENVS with a more uniform distribution of random moves along the whole procedure and that helps to diversify the solutions. In fact, typical NVS works in such a way that the search space is explored by distributed random moves, where each random move starts a new hill climbing process to reach the global minimum. Since it almost behaves like a hill climber in the later stages of the process, it becomes harder to escape from local minima then, especially, when it is applied to difficult optimisation problems, which have harder local minima. The idea is to distribute the random moves more uniformly than exponentially across the whole process. Suppose that the landscape of the formulated problem (3) is l, and E0 is one of the very strict local minima. Furthermore, suppose we run a NVS algorithm that sticks in E0 under some initial conditions. Most of the time, getting stuck in such local minima happens in the later stages of runs, therefore the probability of moving to a rescuable neighbour is very low. In order to avoid sticking in E0, it is required to relax the restricted conditions to let the algorithm proceed by jumping to a solution state that avoids E0. A multi-start NVS algorithm is more useful to relax these conditions rather than a single run NVS since the random moves are more uniformly distributed in the multi-start one and the chance to commence new hill climbing cycles in the later stages is higher. Thus, a compact NVS algorithm that constantly picks the same solution and manipulates it along a number of iterations for several times can easily avoid the local minima, as it adopts a set of short Markov chains instead of a single and long one. This allows changing the direction of solution path towards a much more useful destination. The other property of EVNS is to tackle a population of individuals rather than a single individual. This decreases the effects of initial solutions on the optimization process. Many works on solving hard optimization problems by heuristics focused on the effects of initial solutions. When an initial solution has been chosen, there arise limited possible paths to proceed under the certain circumstances since the optimization process behaves as a Markov chain and each chain offers limited paths to the destination, as widely shown in the literature [31, 28]. Looking at the initial conditions, one can estimate the


probability of getting an optimal or useful near optimal solution with a particular initial solution. In fact, it is hard to ensure that all initial conditions can avoid the local optima in searching for reasonable time. Therefore, a diverse population of initial solutions can give higher probability than a single initial solution to catch the optimum or a useful near optimum within a reasonable time. Moreover, if useful selection and replacement strategies can be utilized, it will definitely help the process to improve the quality of solutions. So, for that reason, the ENVS algorithm is run on a population of solutions rather than an individual.

5. Data Description The osteosarcoma microarray data were collected through institutional review board-approved protocols at four centers (Texas Children’s Hospital/Baylor College of Medicine, Cook Children’s Medical Center, Pediatric Branch of the National Cancer Institute, and University of Oklahoma Health Science Center) after informed consents were signed [20]. A total of 20 samples, which are definitive surgery specimens, were employed to be used in this study. The definitive surgery samples were collected after the completion of preoperative chemotherapy. The drug responses are centrally reviewed by one pathologist after definitive surgery. Good response is defined as more than 90 percent necrosis in tumor, and poor response with less than 90 percent necrosis.

This amount of patient samples are considered to be valuable and satisfied in cancer researches in which were collected through many years of observation of diagnosis, treatment and surgery of the patients [20]. Also osteosarcoma is not that common, but long-term and strong chemotherapy needs to take to turn recovery. Our objective is to make use of this amount of patient samples to solve the integrated gene feature selection and SVM classification problem formulated in (3).

Raw quantification output of all array experiments were preprocessed and filtered by removing spots with low signal intensity and low sample variance (P > 0.01) as well as those that were missing in >50% of the experiments. A total of 1,934 genes remained after pre-processing and filtering. Intensities were then normalized by intensity dependent local weighted regression method. After normalization, intensity ratios were log transformed before further analysis. There were some missing data after filtering. Since most of the learning machines including SVM require complete data matrix, simply ignoring those genes with missing values may possibly miss some significant genes. In this study, we simply replaced those missing data by the mean value of the existing data sets. This approach ensures that the testing data are entirely independent to the training process to exclude any possibility of overestimation.

6 Results and Discussion

A case study of classification of osteosarcoma is proposed to be solved by EVNS. The effectiveness and robustness of the proposed EVNS is performed by comparing with the other two existing methods, genetic algorithm [5] and variable neighbourhood search [4]


which have been proposed to solve this classification problem. The 20 definitive surgery samples were employed to perform the LOOCV discussed in Section 2, the classifier was firstly trained by 19 out of the 20 definitive surgery samples, optimized and validated on 1 out of the 20 definitive surgery samples to classify good responders and poor responders. To reduce the computational cost for optimization, two-sample t-test is first performed to pre-screening those noisy genes among the 1934 genes in which the test values of all genes are illustrated in Figure 2.

0 200 400 600 800 1000 1200 1400 1600 1800 20000

0.5

1

1.5

2

2.5

3

3.5

4

4.5T-values of the gene features

sorted gene features

t-val

ues

Figure 2 t-values of the sorted gene features

192 most significant genes, which their t-value are higher than 2.151

The t-test is then used to evaluate how significance the EVNS better than the other algorithms is, and the t-values are shown in Table 2. It shows that all t-values in Table 2 are higher than 2.15. Based on the normal distribution table, if the t-value is

, are kept from the total 1934 genes. Then the algorithms used to train the SVM classifier with 5 genes out of the 192 genes. Since all algorithms, ENVS, GA and NVS are the stochastic algorithms, different solutions are obtained with runs. The better the algorithm is, the smaller mean and variance of solutions in all runs can be obtained. Therefore 30 test runs, which are detailed on Table 1, were performed. The means and variances of the three algorithms are also shown, and the numbers of times that the algorithms reached 100% accuracy are recorded on the table. It can be found from Table 1 that EVNS achieves the best mean accuracy among all the algorithms. In fact, EVNS obtains the highest mean accuracy. Also the variance of accuracy of EVNS is the smallest comparing with the other algorithms. The smaller the variance means the closer the values cluster around the mean. Since all the variance of accuracy of EVNS is the smallest, it demonstrates that the EVNS is capable to approach and keep searching around the optimal mean closer. Therefore EVNS can produce better and more stable solution quality than the other two algorithms. Also Table 1 shows that the numbers of times that the VNS, GA and EVNS can reach 100% accuracy are 3, 21 and 29 respectively. Therefore the capability of EVNS to reach 100% accuracy is higher than the other two algorithms.

1 Based on the normal distribution table, if the t-value is higher than 2.15, the significance is with 98% confident level.


higher than 2.15, the significance is with 98% confident level. Therefore the performance of EVNS is significantly better than the other two methods with 98% confident in classification of osteosarcoma. The results indicate that the proposed EVNS can achieve more robust and higher quality solution on searching feature subset and parameters of SVM classifier on osteosarcoma.


Table 1 Classification accuracies of the 30 runs, mean of accuracies, variance of accuracies, and number of times reached 100% classification accuracy

Accuracy of i-th run VNS GA EVNS

1 90 100 100

2 95 100 100

3 100 100 100

4 95 100 100

5 95 95 100

6 90 95 95

7 95 100 100

8 80 100 100

9 95 100 100

10 80 100 100

11 95 100 100

12 100 100 100

13 90 100 100

14 95 100 100

15 85 100 100

16 95 90 100

17 95 100 100

18 90 95 100

19 100 90 100

20 90 100 100

21 90 100 100

22 90 100 100

23 95 95 100

24 100 100 100

25 100 90 100

26 90 95 100

27 95 100 100

28 95 90 100

29 85 100 100

30 95 100 100

Mean 92.83 96.67 99.83

Variance 28.76 13.24 0.83

Times reached 100% 5 21 29


Table 2 The t-tests between VNS and EVNS, and between GA and EVNS VNS-EVNS GA-EVNS t-values 7.05 4.62

Among the total 30 runs, four subsets of gene signatures with 100 percent cross-validation accuracy are selected, and are shown in Table 3. In this table, the gene Enah/Vasp-like (EVL, also known as RNB6) appears in all the subsets. It was reported that RNB6 has been identified as a commonly down-regulated gene biomarker in various types of cancers [15, 24]. Another gene Cell division cycle 23, yeast, homolog (known as CDC23), when overexpressed, will leads to abnormal levels of anaphase-promoting complex (APC/C) targets, which is a large multisubunit ubiquitin-protein ligase required for the ubiquitinations and degradation of G1 and mitotic checkpoints regulators [32]. Some other genes, such as Early growth response 1 and C1q and tumor necrosis factor related protein 2, etc., also have relationship with oncogenesis or tumor development. So far no available information can be found to explain the cooperative relationship among genes in each subset. Therefore we cannot verify the validity of the selected genes as genuine biomarkers. Nevertheless, the results can be used as a hypothesis for further investigations. Performing real-time RT-PCR can validate the relevance of these genes as biomarkers. More molecular studies should be pursued to investigate the biological mechanism of these genes in determining drug response and chemoresistance.

Table 3 Subset of combinational gene signatures found by EVNS

1st subset

ESTs Highly similar to hypothetical protein

EVL Enah/Vasp-like

Acetyl-Coenzyme A transporter

Extra spindle poles like 1

Major histocompatibility complex, class II, DO beta

2nd subset

Cell division cycle 23, yeast, homolog

EVL Enah/Vasp-like

Extra spindle poles like 1 (S. cerevisiae)

Early growth response 1

Major histocompati-bility complex, class II, DO beta

3rd subset

SRY-box 9 (sex determining region Y)-box 9

EVL Enah/Vasp-like

ESTs, Highly similar to hypothetical protein

C1q and tumor necrosis factor related protein 2

Homo sapiens mRNA from chromosome 5q21-22, clone:357Ex

4th subset

Cell division cycle 23, yeast, homolog

EVL Enah/Vasp-like

Protein associated with PRK1

Hypothetical protein MGC19556

Ubiquitin specific protease 9, Y chromosome (fat facets-like Drosophila)

To further evaluate the credibility of the gene subsets, comparison of correlations


between gene signatures on the 5 gene subsets found by ENVS are carried out. Table 4 shows the correlations between the gene signatures on the 5 gene subsets found by EVNS. Also the correlation of the gene subset, which consists of the 5 genes (shown in Table 5) with the highest t-values among all the genes, is shown in Table 4. The mean of correlations, maximum correlation and minimum correlation in each gene subset are all shown in Table 4. It can be found from Table 4 that the mean correlation of 5 gene signatures with the highest t-values is larger than the five subsets of gene signatures found by EVNS. Also the minimum and maximum correlations found by the 5 gene signatures with the highest t-values are larger than the ones found by EVNS. If the correlations between gene signatures are close, then similar information is contained in the gene signatures. The smaller correlation found, the more information is contained on the gene subset. Therefore the results suggest that the gene subsets found by EVNS can explore more information than the one found by the 5 gene signatures with the highest t-values. Table 4 Correlation between genes in gene subsets found by EVNS The i-th to

the j-th gene pair

Gene subset (with

highest t-values)

1-st gene subset

(found by EVNS)

2-nd gene subset

(found by EVNS)

3-rd gene subset

(found by EVNS)

4-th gene subset

(found by EVNS)

1-2 0.16276 0.19654 0.45141 0.093325 0.3816 1-3 0.12967 0.18814 0.55831 0.11165 0.62298 1-4 0.25279 0.34208 0.56237 0.036027 0.17151 1-5 0.26563 0.51407 0.035131 0.082609 0.35725 2-3 0.74731 0.28718 0.27413 0.12928 0.24057 2-4 0.096354 0.091131 0.091691 0.068829 0.035131 2-5 0.52592 0.04635 0.17565 0.32967 0.21738 3-4 0.15023 0.43749 0.43749 0.046298 0.020122 3-5 0.35928 0.068829 0.068829 0.18343 0.054302 4-5 0.24182 0.19441 0.19441 0.056498 0.23068

Mean 0.74731 0.1893 0.22795 0.091009 0.18652 Min 0.096354 0.04635 0.035131 0.036027 0.020122 Max 0.74731 0.51407 0.56237 0.32967 0.62298

Table 5 Subset of combinational gene signatures with the highest t-values

ATPase, H+ transporting, lysosomal 56/58kD, V1 subunit B, isoform 1 (Renal tubular acidosis with deafness)

microfibrillar- associated protein 2

Mitogen-activated protein kinase kinase kinase kinase 1

Protein phosphatase 6, catalytic subunit

selectin L (lymphocyte adhesion molecule 1)


7. Conclusion In this paper, we have proposed an evolutionary variable neighborhood search algorithm EVNS, which is an integrated approach of variable neighborhood search VNS and evolutionary algorithm, aiming at selecting a compact gene subset and simultaneously optimizing SVM classifier parameters. As discussed in the literature, VNS algorithms may guarantee the optimum or a useful near optimum result. However, it may not reach the reasonable solutions in an affordable time. For this reason, VNS is hybridized with another heuristic algorithm, evolutionary algorithm. The resulting algorithm EVNS is identical to EA except the reproduction operations, crossover and mutation are replaced with VNS. It can work as a proper evolutionary algorithm and is more likely to avoid the local minima traps.

Applying EVNS on osteosarcoma microarray data resulted in 99.83 percent of cross-validation accuracy on the training dataset with 20 definitive surgery samples outperforming the other proposed algorithms, VNS [4] and evolutionary algorithm [5]. Apart from higher solution quality, more robust solutions can be produced by EVNS than the other proposed algorithms. In the mean time, four subsets of combinational gene signatures were discovered. Some of them are reported to have close relationship with oncogenesis and tumor development. Further laboratory test will be pursued to investigate the cooperative mechanism among each gene subset. This suggests that the results of EVNS can be used to generate hypothesis for the identification and validation of genetic biomarkers for diagnostic and therapeutic purposes. In the future, we will employ the proposed algorithm in solving other similar classification problems with large amount of gene data sets like the nasopharyngeal carcinoma or the lung cancer. REFERENCES 1. B. Boser, I. Guyon and V. Vapnik, An training algorithm for optimal marigin

classifiers, Proceedings of the Fifth Annual Workshop on Computational Learning Theory, (1992) 144-152.

2. F.Z. Brill, D.E. Brown and W.N. Martin, Fast genetic selection of features for neural network classifiers, IEEE Transactions on Neural Networks, 3(2) (1992) 324-328.

3. R.E. Caballero and P.A. Estevez, A niching genetic algorithm for selecting features for neural network classifiers, Proceedings of the 8th International Conference of Artificial Neural Network, 1 (1998) 311-316.

4. K.Y. Chan, H.L. Zhu, M.E. Aydin, C.C. Lau and H.Q. Wang, An integrated approach of support vector machine and variable neighborhood search for selecting combinational gene signatures in predicting chemo-response of osteosarcoma, Proceedings on International MultiConference of Engineers and Computer Scientists, 2008.

5. K.Y. Chan, H.L. Zhu, C.C. Lau, S.H. Ling and H.H.C. Ip, Gene signature selection for cancer prediction using an integrated approach of genetic algorithm and support vector machine, Proceedings on the IEEE International Conference on Evolutionary Computation, 2008.

6. M. Daly and R. Ozol, The search for predictive patterns in ovarian cancer: proteomics


meets bioinformatics, Cancer Cell, (2002) 111-112. 7. S. Dudoit, J. Fridlyand, and T. P. Speed, Comparison of discrimination methods for the

classification of tumors using gene expression data, Journal of the American Statistical Association, 97(457) (2002) 77-87.

8. M.B. Eisen, P.T. Spellman, P.O. Brown, D. Bostein, Cluster analysis and display of genome-wide expression patterns, Proceedings of the National Academy of Science, 95(14) (1998) 863–14.

9. K. Fleszar and K. S. Hindi, New heuristics for one-dimensional bin-packing, Computer. Operations Research, 29 (2002) 821-839.

10. Fogel, D.B., An introduction to simulated evolutionary optimization. IEEE Trans. Neural Networks, 5 (1994) 3–14.

11. T.S. Furer, N. Cristianini, N. Duffy, D.W. Bednarski, M. Schummer and D. Haussler, Support vector machine classification and validation of cancer tissue samples using microarray expression data, Bioinformatics, 16(10) (2000) 906-914,.

12. T.R. Golub, D.K. Slonim, P. Tamayo, C. Hurd, M. GassenBeek, J.P. Mesirov, H. Coller, M.L. Loh, J.R. Downing, M.A. Caligiuri, C.D. Blomfield, E.S. Lander, Molecular classification of cancer: class discovery and class prediction by gene-expression monitoring, Science, 286 (1999) 531-537.

13. P. Handsen, N. Mladenovic and U. Dragan Variable neighborhood search for the maximum clique, Discrete Applied Mathematics, 145 (2004) 117–125.

14. E. Hartuv, A. Schmitt, J. Lange, S. Meier-Ewert, H. Lehrach and R. Shamir, An algorithm for clustering cDNA fingerprints, Genomics, 66(3) (2000) 249-256.

15. S. Hasegawa et al., Genome-wide analysis of gene expression in Intestinal-Type Gastric cancers using a complementary DNA microarray representing 23,040 genes, Cancer Research, 62 (2002) 7012-7017.

16. J.H. Hong and S.B. Cho, Efficient huge-scale feature selection with speciated genetic algorithm, Pattern Recognition Letters, 27 (2006) 143-150.

17. J. Khan, J.S. Wei, M. Ringner, L.H. Saal, M. Ladanyi, F. Westermann, F. Berthold, M. Schwab, C.R. Antonescu, C. Peterson, P.S. Meltzer, Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks, Nature Medicine, 7(6) (2001) 673-679.

18. L. Li, C.R. Weinberg, T.A. Darden and L.G. Pedersen, Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method, Bioinformatics, 17(12) (2001) 1131–1142.

19. M.P. Link, M.C. Gebhardt and P.A. Meyers, Principles and Practice of Pediatric Oncology, (2002) 1051-1089.

20. T.K. Man, M. Chintagumpala, J. Visvanathan, J. Shen, L. Perlaky, J. Hicks, M. Johnson, N. Davino, J. Murray, L. Helman, W. Meyer, T. Triche, K.K. Wong and C.C. Lau, Expression profiles of osteosarcoma that can predict response to chemotherapy, Cancer Research, 65(18) (2005) 8142-8150.

21. Ma, J., Tian, P. and Zhang, D., Global optimization by Darwin and Boltzmann mixed strategy. Comput. Oper. Res., 27 (2000) 143–159.

22. N. Mladenovic and P. Hansen, Variable neighborhood search. Computer Operations Research, 24 (1997) 1097–1100.

23. Moilanen, A., Simulated evolutionary optimization and local search: introduction and application to tree search. Cladistics, (17) 2001 12–25.

24. J. Okutsu et al. Prediction of Chemosensitivity for patients with Acute Myeloid Leukemia, according to expression levels of 28 genes selected by genome-wide


complementary DNA microarray analysis, Molecular Cancer Therapeutics, 1 (2002) 1035-1042.

25. S. Peng, Q. Xu, X.B. Ling. X. Peng, W. Du and L. Chen, Molecular classification of cancer types from microarray data using the combination of genetic algorithms and support vector machines, FEBS Letters, 555 (2003) 358-362.

26. Reeves, C., Modern Heuristic Techniques for Combinatorial Problems, 1993 (Wiley: New York).

27. M. Sevkli and M.E. Aydin, Parallel variable neighbourhood search algorithms for job shop scheduling problems, IMA Journal of Management Mathematics, 2008 (in print).

28. Steinhofel, K., A. Albrecht, and C.K. Wong, Two Simulated Annealing-Based Heuristics for the Job Shop Scheduling Problem, European Journal of Operational Research 118 (1999) 524–548.

29. V.N. Vapnik, Statistical Learning Theory, Wiley Interscience, 1998. 30. V. Vapnik and O. Chapelle, Bounds on error expectation for support vector machines,

Neural Computation, 12 (2000) 2013-2036. 31. Van Laarhoven, P.J.M., E.H. Aarts, and J.K. Lenstra. (1992). “Job Shop Scheduling by

Simulated Annealing.” Operations Research 40(1), 113–125. 32. Wang Q, Moyret-Lalle C, Couzon F, 2003, Alterrations of anaphase-promoting

complex genes in human colon cancer cells, Oncogene, Vol. 22, No. 10, pp. 1486-1490, 2003.

33. Wong, S.Y.W., Hybrid simulated annealing/genetic algorithm approach to short term hydro-thermal scheduling with multiple thermal plants. Electric. Power Energy Syst., 2001, 23, 565–575.

34. Y.D. Zhao, C. Pinilla, D. Valmori, R. Martin and R. Simon, Application of support vector machines for T-cell epitopes prediction, Bioinformatics, Vol. 19, No. 15, pp. 1978-1984, 2003.

APPENDIX Local Search Function (LSF) is developed as a simple hill-climbing algorithm based on both aforementioned NFs as discussed in Section 3.1. As indicated in the following pseudo-code, the NFs are used complementary to each other in the way that the NFs keep iterating as long as better moves are resulted. It switches to the other move once the result produced is not better and the algorithm stops if the number of moves, n, meets a predefined number, nmax. The change of NF is organized with a binary integer variable, γ ∈ (0, 1), in which the value of γ is changed by using an absolute function denoted by |·| norm at the second part of step (b) of the pseudo-code. The procedures of LSF are as follows:

Algorithm LSF: x=LSF(x) 1. Set 0←n and 1←γ 2. While maxnn < do (a) if ( )1=γ then x’ ← MutationBin(x); else if ( )0=γ then x’

← MutationReal(x) (b) Set if ( ) ( )'xJxJ < then 'xx ← ; else 1−← lγ (c) 1+← nn


where ( )xJ is defined by (3) in Section 2.1.

Kit Yan Chan is currently a Senior Research Fellow in the Institute Digital Ecosystems and Business Intelligence Institue, Curtin University of Technology. Dr. Chan received his MPhil degree in Electronic Engineering from City University of Hong Kong, Hong Kong and his PhD degree in Computing from London South Bank University, United Kingdom. His research interests include computational intelligence and its applications in product design, signal processing, power systems and operation researches.

Mehmet Emin Aydin received his B.Sc. from İstanbul Technical University, MA from İstanbul University, and PhD from Sakarya University, Turkey. He is currently a Lecturer in the Department of Computer Science and Technology of the University of Bedfordshire, UK. His research interests include grid-enabled/parallel and distributed metaheuristics, network planning and optimization, evolutionary computation and intelligent agents and multi-agent systems. Currently, he is a member of The OR Society UK, ACM and IEEE Computer Society.

Hailong Zhu received his B.Sc (1996) and Ph.D (2003) in Mechanical Engineering of Xi'an Jiaotong University. His doctoral research was about statistical learning theory and biometrics recognition. He has been a student fellow in Microsoft Research Asia from 1999 to 2000. After graduation with Ph.D, he joined GE Global Research Center as a research engineer. In 2006, he joined the Research Institute of Innovative Products and Technologies of the Hong Kong Polytechnic University as an Assistant Professor. His current research interests include clinical decision support system, machine learning, bioinformatics and biometrics, etc.

Ching Lau M.D., Ph.D., is an Associate Professor of Pediatrics, and Co-Leader of the Pediatrics Program of the BCM Cancer Center. His research interests include the molecular biology of pediatric brain and bone tumors and the clinical applications of genomic technologies. Dr. Lau is the Director of Research of the Pediatric Neuro-oncology Program and the Director of the Cancer Genomics Program. He is also a member of the Bioinformatics Steering Committee and Biopathology and Translational Research Committee of COG. He is a member of several journal editorial boards including being named recently the founding Editor-in-Chief of the International

Journal of High Throughput Screening. 1 Digital Ecosystems and Business Intelligence Institue, Curtin University of Technology, Perth, Australia, 2 Rsearch Institute of Innovative Products and Technologies, The Hong Kong Polytechnic Unviersity, Hung Hom, Hong Kong 3 Department of Computing and Information Systems, University of Bedfordshire, Luton, United Kingdom 4 Departments of Pediatrics, Texas Children’s Cancer Center, Houston, Texas, USA


275

PERFORMANCE ANALYSIS OF THE LP NORM AND EFFECTS OF DIMENSIONALITY REDUCTION FOR MLP NEUTRAL

NETWORK BASED CALSSIFIER

SURESH S. SALANKAR AND BALASAHEB M. PATRE

Abstract This paper describes a sensitivity analysis for dimensionality reduction and the

effect of Lp error norm on the performance of the Multi-layer Perceptron Neural Network

(MLP NN). If the effect that each of the network inputs has on the network output after

training a neural network is known, then some of the network inputs can be removed from

the network. Consequently, the dimensionality of the network, and hence, the connectivity

and the training time can be reduced. Sensitivity analysis, which extracts the cause and

effect relationship between the inputs and outputs of the network, has been proposed as a

method to achieve this and is investigated for sonar and radar ionosphere databases.

Simulations demonstrate the validity of the method used.

Key Words, Classifier, MLP neural network, error norms, cross-entropy, sensitivity, ROC,

decision boundary

1. Introduction

The designer of a classification system does not usually know a priori which features will yield acceptable classification performance and, in theory, the classification performance is a nondecreasing function of the number of features. Hence, the designer might choose to use all available features. However, using a large number of features can be wasteful of both computational and memory resources. In addition, due to practical problems associated with training; a classifier with a finite amount of data, using a large number of features can actually cause degradation of classification performance [1]. The size of the training set directly influences the performance of any classifier trained nonparametrically (e.g., as neural network). Neural Networks require a lot of data for appropriate training, because there are no a priori assumptions about the data. It is important to know how the requirement on the size of the training set scales as a function of the size of the neural network for a given precision in the mapping. The number of training patterns (N) required to classify test examples with an error of δ is



276 S. SALANKAR AND B. PATRE

approximately given by WNδ

> , where W is the number of weights in the network

[2]. This equation shows that the number of required training patterns increases linearly with the number of free parameters of the MLP, which is excellent compared with other classification methods. A rule of thumb is 10N W≈ , i.e., the training set size should be 10 times larger than the number of network weights to accurately classify test data with about 90% accuracy [3]. In this rule of thumb, it is assumed that the training data is representative of all the conditions encountered in the test set. The main focus when creating the training set should be to collect data that cover the full known operating conditions of the problem we want to model. If the training set does not contain data from some areas of pattern space, the machine classification in those areas will be based on extrapolation. This may or may not correspond to the true classification boundary (desired output), so one should always choose samples for the training set that covers as much of the input space as possible. The practical limiting quantity is the number of training patterns; most of the times we need to compromise the size of the network to achieve appropriate training for the learning machine. A reasonable approach is to use feature extraction (i.e., a pre-processor) that decreases the input space dimensionality, thus reducing the number of weights in the network. In this research, feature extraction using sensitivity analysis has been proposed as an alternative to statistical methods for MLP NN based classifier on sonar and ionosphere signal databases. Each channel of the input vector is randomly perturbed around its mean value, while keeping the other inputs at their mean values and then the change in the output is measured. The change in the input is done by adding a random value of variance 0.1 to each sample and computing the output. Inputs that have large sensitivities have more importance in the mapping. The inputs with small sensitivities can be discarded. This helps the training (because it decreases the size of the network), decreases the cost of data collection, and when done right has negligible impact on performance. This research, demonstrate the effects of reducing the input space dimensions and error norms on the performance of MLP NN based classifier on the sonar and radar ionosphere databases. Rest of the paper is organized as follows: Section 2 describes the various error norms. The performance of MLP NN based classifier with different error norm is explained in Section 3. Effect of dimensionality reduction and different error norm on the performance MLP NN based classifier is explained in Section 4. Lastly, conclusions are given in Section 5.

2. Different Error Norms Criterion Feed-forward neural network are the most common neural architecture used in real-life application. They are trained with the supervised learning class of algorithms,

THE FERFORMANCE ANALYSIS OF DIMENSIONALITY REDUCTION 277

with the backpropagation algorithm being the most commonly used [4, 5]. Since the original algorithm was published, many modifications have been presented in an effort to improve the training performance [6-9]. The general idea behind this class of algorithms is the minimization of an error function, by iteratively updating an estimation of the network parameters (weights). The mean square error (MSE) is the most commonly used error function, although it has been suggested that it is not the best function to use, especially in classification problems [10-12]. A number of alternative error functions have been presented in the literature [13, 14]. The maximum likelihood (cross entropy) function was notably reported as a more appropriate function for classification problems.

2.1 pL Error Norms criterion

Normally 2L error norm is used as the distance metric in the performance cost

function. However, it does not always produce optimal solution. Different norms provide different solutions to a learning problem, because the weights are modified with information that depends on the choice of the norm, so the positioning of the discriminant functions and formation of decision boundary are affected by the norms for

the same training data set. Each norm weights the errors differently. Different pL error

norms ( p = 1, 3, 4 and 5) are employed along with information theoretic cross entropy

criterion in order to study their effects on the performance of the MLP classifier. In supervised learning the difference between the desired response and the actual learning system output is used to modify the system so that the minimum error of the performance surface is achieved. The issue is how to define the performance, also called the error criterion or the cost. In normal operation the learning machine provides an output for each input pattern, so the total cost J is computed as a sum of individual

costs, ,n kJ obtained from each input pattern presentation, i.e.

(1)

nk

k nJ J=∑∑

where k is an index over the system outputs and n is an index over the input patterns.

nkJ is the individual cost, defined as

(2) ( )nk nk nkJ f d y= −

( )nkf ε=


The only issue is then how to compute the individual cost as a function of nkε , which is

called the instantaneous error. The mean square error (MSE) criterion defines the individual cost as the square of the instantaneous error between the desired response and the system output, that is,

(3) ( )2nk nk nkJ d y= −

With the MSE the instantaneous cost is the square of the magnitude of the instantaneous error. This means that when the learning machine minimizes the error power, it weights the large errors more (quadratically). If the weight update performed by gradient descent is recalled as,

(4) ij j iw xηδ∆ =

One realizes that the weight values are updated proportionally to the size of the error, so the weights are sensitive to the larger errors. This is reasonable if the data is clean without many large deviations, but in practice the data sets may have outliers. Outliers thus may have an inordinate effect on the optimal parameter values of the learning machine. Learning machines with saturating nonlinearities control this aspect better than linear processing element (PE) machines, but they still are more sensitive to large errors than small errors. Since the values of the weights set the orientation and position of the discriminant function, we can deduce that outliers will "bias" the position of the discriminant function. This argument shows that if one wants to modify how the instantaneous error influences the weights, one can define the instantaneous cost more generally as

(5) p

nk nk nkJ d y= −

where p is an integer, which is normally called the p norm of the instantaneous error

nkε . When p = 2 we obtain the 2L norm that leads to the MSE criterion. When p = 1

we obtain the 1L norm, which is also called the Manhattan metric. Notice that the

1L norm weights the differences proportionally to their magnitude, so it is far less

sensitive to outliers than the 2L norm. For this reason it is called a more robust norm. In

general the pL norm for p > 2 weights large deviations even more. Different norms


provide different solutions to a learning problem; because the weights are modified with information that depends on the choice of the norm, so the positioning of the discriminant functions is affected by the norm (for the same training data set). For positive finite integers p , the derivative of the norm can be computed quite easily as

(6) 1 sgn( )pnk

nk nk nk nknk

J d y d yy

−∂= − −

∂

but pL norms do not cover all the cases of interest. For errors larger than 1, the

instantaneous cost nkJ for the pL norms always increases at the same rate or faster than

the instantaneous error which may not be our goal. There are cases of practical relevance

that do not have an analytic solution such as the L∞ norm (all errors are zero except the

largest). Another possible criterion is to simply use the sign of the deviation ( p = 0).

2.2 Cross-Entropy criterion Information theory was invented by Shannon [15] in the late 1940s to explain the content of messages and how they are corrupted through communication channels. The key concept in information theory is that of information. Information is a measure of randomness of a message. If the message is totally predictable, it contains no information. On the other hand, something much unexpected has high information content, so the information is inversely associated with the probability of an event. We can define the

amount of information in a random event kx with probability ( )kp x as

(7) 1( ) log( )k

k

I xp x

=

Entropy then becomes the mean value of ( )I x over the complete range of discrete

messages (2 1)k + with probabilities ( )k kp p x=

(8) ( ) ( )K

k kk K

H x p I x=−

= ∑

Entropy is a measure of the average amount of information contained in the event. Now

assume that we have two probability mass functions { }kp and{ }kq .


The relative entropy of the probability distribution P (function of some event r)

with respect to the second distribution Q is given by

(9) ( )( || ) ( ) log( )x

p xD p q p xq x

=

∑

This concept is called relative entropy and was introduced by Kullback [16]. It is commonly called Kullback-Leibler distance. Since the learning system output is approximating the desired response in a statistical

sense, it is reasonable to utilize the K-L criterion as our cost. In this case ( )p x

becomes the target density constructed by +1 and -1, and ( )q x the learning system

output density. This is particularly appropriate for classification problems where the

2L assumption is weak because the distribution of the targets is far from Gaussian. For c

classes the K-L information criterion becomes

(10) , log nkn k

n k nk

yJ dd

=

∑∑

where n is the index over the input patterns and k over the classes. Since this criterion works on probabilities, the output PEs should be softmax PEs. It can be shown (Hertz) that the instantaneous error backpropagated through the network (the partial derivative of

J with respect to y ) with the softmax activation function is

(11) k kk

J y dnet∂

= −∂

(where &k ky d are system output and desired output over index k)

This is an interesting result, since it says that the cross-entropy criterion can be implemented by the MSE criterion. However, the network uses output PEs that implements the softmax function. This can be easily accommodated if one associates the softmax PE with a linear PE in the backplane as its dual as shown in Figure 1.


Fig 1. Implementation of the cross-entropy criterion

In the two-class classification problem (single output PE) the softmax PE becomes a logistic PE. By not subjecting the error to the attenuation produced by the derivative of the output PE nonlinearity, the network converges faster. It is also possible to show that

the cross entropy is similar to the 1L norm, which means that this criterion weights small

errors more heavily than the 2L norm. The other important aspect of training the MLP

with cross-entropy is that the interpretation of the output as the a posteriori probability of the class, also hold in this case.

3 Performance of MLP NN Classifier with different error norms 3.1 Experimental results for sonar signal The sonar data is obtained by Terry Sejnowski, now at the Salk Institute and the University of California at San Deigo [17]. This data set was developed in collaboration with R. Paul Gorman of Allied-Signal Aerospace Technology Center. The data set, "sonar.data", is in the standard CMU Neural Network Benchmark format [18]. The following table 1 highlights the data partition schemes employed in order to design a classifier. The sonar database constitutes 208 instances with 60 continuous-valued inputs and one output denoting the class of the instance. The first 104 samples (1:104) are used for training; the next 104 samples (105:208) for testing and classifier comparison purpose. The task is to train a neural network classifier to discriminate between sonar signals bounced off a metal cylinder and those bounced off a roughly cylindrical rock. As there are 60 numeric inputs and one symbolic output (translated into two numeric-valued outputs, where “Metal Cylinder” is 0 1 and “Rock” is 1 0) for the given system, the number of input and output processing elements is chosen as sixty and two,

Table 1. Data partition scheme for sonar dataset Data Partition Training / Estimation instances Validation /Testing Instances

Set 1 (Normal 1:104 (104 samples) 105:208 (104 samples)


tagging)

“Rock” samples= 55

“Metal Cylinder” samples= 49

“Rock” samples= 42

“Metal Cylinder” samples= 62

Set 2 (Reverse

tagging)

105:208 (104 samples)

“Rock”samples=42

“Metal cylinder” samples= 62

1:104 (104 samples)

“Rock” samples= 55 “Metal

cylinder” samples= 49

respectively. An optimal designed of MLP based classifier is carried out as 60-8-2 [19]. The total number of free parameters (connection weights) for designed model (60–8-2) is (60x8+8x2+8+2) = 506. The learning and generalization ability of the estimated NN based classifier is assessed on the basis of certain performance measures such as average classification accuracy and area under the receiver operating characteristics (ROC) curve. In this research, different Lp error norms (p=1,2,3,4 and 5) are employed along with cross entropy criterion in order to study their effects on the performance of the MLP classifier on sonar data set. Table 2, displays the performance of MLP NN based classifier with different error norms.

Table 2. Performance of MLP NN classifier for sonar data set (before dimensionality

reduction) Error Norms Classification performance measures on sonar signal data set

Avg. Classification

accuracy in %

Area under ROC

curve

Area under convex

hulls of ROC curve

L1 87.17357911 0.927803 0.930492

L2 89.55453149 0.924155 0.937404

L3 87.25038402 0.937404 0.948925

L4 87.55760369 0.939708 0.948925

L5 89.97695853 0.954301 0.964862

Cross-entropy 87.17357911 0.941628 0.949885

3.2. Experimental results for radar ionosphere signal The radar data is obtained from Johns Hopkins University Ionosphere database [20, 21]. This radar data was collected by a system in Goose Bay, Labrador. This system consists of a phased array of 16 high-frequency antennas with a total transmitted power on the order of 6.4 kilowatts. The targets were free electrons in the ionosphere. "Good" radar returns are those showing evidence of some type of structure in the ionosphere. "Bad" returns are those that do not; their signals pass through the ionosphere. Received signals were processed using an autocorrelation function whose arguments are the time


of a pulse and the pulse number. There were 17 pulse numbers for the Goose Bay system. Instances in this database are described by 2 attributes per pulse number, corresponding to the complex values returned by the function resulting from the complex electromagnetic signal. Thus, there are 34 continuous valued attributes with respect to inputs and one additional attribute denoting class which is either "good" or "bad" according to the definition summarized above. The following table 3 highlights the data partition schemes employed in order to design a classifier. There are total 351 instances

Table 3. Data partition scheme for radar ionosphere data set Data Partition Training instances Testing instances

Set 1 (Normal

tagging)

1:200 (200 samples)

“Bad” samples = 99 (49.5%)

“Good” samples = 101 (50.5%)

201:351 (151 samples)

“Bad” samples = 27 (17.88%)

“Good” samples = 124 (82.119%)

Set 2 (Reverse

tagging)

152:351 (200 samples)

“Bad” samples = 51 (25.5%)

“Good” samples = 149 (74.5%)

1:151 (151 samples)

“Bad” samples = 75 (49.668%)

“Good” samples = 76 (50.33%)

with 34 continuous valued attributes with respect to inputs and one additional attribute denoting class of the instance. Two different data partitions are used with different tagging orders. In the first case, the first 200 samples (1:200) are used for training; the next 151 samples (201:351) for testing and classifier comparison purpose. In the second case, the last 200 samples (152:351) are used for training and the first 151 samples (1:151) for testing of classifier. As there are 34 numeric inputs and one symbolic output (translated into two numeric-valued outputs, where “good” is 0 1 and “bad” is 1 0) for radar ionosphere data set, the number of input and output processing elements is chosen as thirty four and two, respectively. An optimal designed of MLP based classifier is carried out as 34-15-12-2 [22]. The total number of free parameters (connection weights) for designed model (34–15–12–2) is (510+180+24+15+12+2) = 743. The learning and generalization ability of the estimated NN based classifier is assessed on the basis of certain performance measures such as average classification accuracy and area under the ROC curve. In this research, different Lp error norms (p=1,2,3,4,5 and ∞) are employed along with cross entropy criterion in order to study their effects on the performance of the MLP classifier on radar ionosphere data set. Table 4, displays the performance of MLP NN based classifier with different error norms.

Table 4. Performance MLP NN classifier for radar data set (without dimensionality

reduction)


Error Norms Classification performance measures on radar ionosphere signal data set

Avg. Classification

accuracy in %

Area under ROC

curve

Area under convex

hulls of ROC curve

L1 88.888 0.973417 0.977748

L2 92.83 0.939964 0.943996

L3 94.0412 0.943399 0.956691

L4 95.4898 0.957736 0.976105

L5 94.4444 0.915323 0.94444

L∞ 91.7861 0.953405 0.969385

Cross-entropy 94.04121 0.940711 0.957139

4. Performance of MLP NN Classifier with dimensionality reduction Input sensitivity analysis is performed on the two benchmark high dimensionality databases of sonar and radar ionosphere signals. The way we can perform input-sensitivity analysis is to train the network as we normally do and then fix the weights. The next step is to randomly perturb, one at a time, each channel of the input vector around its mean value, while keeping the other inputs at their mean values, and then measure the change in the output. The change in the input is normally done by adding a random value of a known variance to each sample and computing the output [23]. The sensitivity for input k is expressed as

(12)

2

1 12

P o

ip ipp i

kk

y yS

σ

−

= =

− =

∑∑

where ipy is the ith output obtained with the fixed weights for the pth pattern, p is the

number of patterns, o is the number of network outputs, and 2kσ is the variance of the

input perturbation. This is very easy to compute in the trained network and effectively measures how much a change in a given input affects the output across the training data set. Inputs that have large sensitivities have more importance in the mapping and therefore are the ones we should keep. The inputs with small sensitivities can be discarded. This helps the training (because it decreases the size of the network), decreases the cost of data collection, and when done right has negligible impact on performance. Sensitivity about the mean provides a measure of the relative importance among the inputs of the neural model and illustrates how the model output varies in response to variation of an input. The first input is varied between its mean +/- a


user-defined number of standard deviations while all other inputs are fixed at their respective means. Experiment results of sensitivity analysis on MLP NN were applied to sonar and radar ionosphere databases has been explained in the following section. 4.1. Experimental results for sonar signal Each channel of the input vector is randomly perturbed around its mean while keeping the other inputs at their mean values and then the change in the output is measured. The change in the input is done by adding a random value of variance 0.1 to each sample and computing the output. Sensitivity effectively measures how much a change in a given input affects the output across the training and testing data set. The network output is computed for a user-defined number of steps above and below the mean. This process is repeated for each input [24]. Figure 2 shows sensitivity analysis results (test on test data set), where R & M represents rock and metal samples respectively. Inputs with sensitivity less than a threshold value of 0.287 may be ignored due to their low contribution. As shown in table 5, seventeen inputs were found to have small sensitivities less than 0.287. Hence these inputs are ignored, so that input space dimensions are reduced to only 43 in comparison to original features as 60.

0

0.05

0.1

0.15

0.2

0.25

0.3

Sens

itivit

y

Input3Input4

Input12

Input15

Input18

Input19

Input20

Input23

Input24

Input27

Input32

Input33

Input38

Input40

Input42

Input43

Input44

Input Name

Sensitivity About the Mean

(R)

(M)

Fig 2. Sensitivity analysis for insignificant inputs of sonar data (Test on test data set)

Table 5. Sensitivity of insignificant inputs of sonar data

Insignificant Inputs

Sensitivity Rock Sensitivity Metal

Input3 0.106076669 0.077786992

Input4 0.084521959 0.039548851

Input12 0.211416651 0.220643601


Input15 0.125203893 0.154996297

Input18 0.200709918 0.212186692

Input19 0.224817482 0.211868315

Input20 0.182854964 0.174086536

Input23 0.011081462 0.025340356

Input24 0.088981912 0.083380475

Input27 0.227583144 0.208706072

Input32 0.095378025 0.080697865

Input33 0.287028327 0.283997544

Input38 0.27595991 0.284047612

Input40 0.248070699 0.223376551

Input42 0.105407691 0.082674617

Input43 0.180558998 0.146482655

Input44 0.06472767 0.078668842

The total numbers of free parameters (connection weights) for this model (43–8-2) are about 370. This leads to 26.877 per cent reduction in the free parameters (connection weights of the neural network including biases). MLP NN classifier is run at least three times with different initialization of connection weights. After reduction of input dimension, the optimal parameter settings for MLP NN based classifier are as follows. The number of input is chosen as 43 and the number of output PEs are 2, PEs in Hidden layer = 8 Transfer function of PEs in hidden and output layers are tanh Learning rule in hidden and output layers = step (with step size of 0.1) Classification performance measures of MLP NN classifier on sonar signal are listed in table 6.

Table 6. Performance of MLP NN classifier for sonar data set (with dimensionality reduction)

Error Norms Classification performance measures on sonar signal data set

Avg. Classification

accuracy in per cent

Area under ROC

curve

Area under convex

hulls of ROC curve

L1 85.17665131 0.93222 0.9399

L2 87.17357911 0.926651 0.936444


L3 88.40245776 0.93318 0.945661

L4 87.17357911 0.93606 0.943356

L5 86.78955453 0.940284 0.949501

Cross-entropy 88.78648233 0.936828 0.952573

4.2. Experimental results for radar ionosphere signal For radar ionosphere, after sensitivity analysis was applied, the network output is computed for a user-defined number of steps above and below the mean. This process is repeated for each input. Figure 3 shows sensitivity analysis results (test on test data set).

Insignificant Inputs

00.0050.01

0.0150.02

Input2Input4Input6Input10Input12Input16Input26Input32Input34

Inputs for Radar signal

Sens

itivity

BadGood

Fig 3. Sensitivity analysis for insignificant inputs of radar data (Test on test data set)

Nine inputs were found to have small sensitivity less than 0.02 shown in table 7. Hence these inputs are ignored, so that input space dimensions are reduced from 34 to only 25. As there are 25 numeric inputs and one symbolic (translated into 2 numeric) output for the given system, the optimal parameter settings for MLP NN based classifier for small data set (compressed radar signal data set) are as follows. The number of input is chosen as 25 and the number of output PEs are 2, PEs in Hidden layer H1 = 15 and PEs in Hidden layer H2 = 12, Learning rule = conjugate gradient Transfer function of processing elements in hidden layer and output layer are tanh Total no. of free parameters (connection weights) for this model (25–15–12–2) =608. This leads to 18.169 percent reduction in the free parameters (connection weights of the neural network including biases). MLP NN classifier is run at least three times with different initialization of connection weights.


Table 7. Sensitivity of insignificant inputs of radar data Insignificant Inputs Sensitivity bad return Sensitivity good return

Input2 0 0

Input4 0.01987877 0.020124368

Input6 0.015887188 0.015975945

Input10 0.01861355 0.019667568

Input12 0.018306895 0.01844427

Input16 0.017761322 0.017939855

Input26 0.016982039 0.017035733

Input32 0.01891216 0.018805528

Input34 0.014295743 0.014295307

Classification performance measures of MLP NN classifier on radar ionosphere signal are listed in table 8.

Table 8. Performance of MLP NN classifier for radar data set (with dimensionality reduction)

Error Norms Classification performance measures on radar ionosphere signal data set

Avg. Classification

accuracy in %

Area under ROC

curve

Area under convex

hulls of ROC curve

L1 92.189 0.993877 0.995669

L2 94.444 0.966395 0.976697

L3 93.63799 0.948925 0.962366

L4 94.0412 0.988501 0.991487

L5 93.63799 0.982079 0.987157

L∞ 89.9343 0.942652 0.950866

Cross-entropy 93.63799 0.960573 0.97267

5. Conclusion The comparative results obtained before and after sensitivity analysis method were given in terms of classification accuracy and area under the ROC curve for sonar and radar ionosphere databases. The result shows that removing inputs with low sensitivity values from a large network does not greatly affect the accuracy. There are 26.877 and 18.169 percent reduction in the free parameters (connection weights of the neural network including biases) for sonar and radar ionosphere databases respectively. Consequently, although the results were dependent on the problems, it was seen that the sensitivity analysis could be successfully used as a method to reduce the dimensionality


of networks. Particularly, ionosphere database results shows that removing more than one input with low sensitivity values from a large network does not greatly affect the accuracy. This may provide significant impetus for neural network circuit

implementations in practice. It is observed that, for sonar data, 5L error norm yields

the better classification results than 2L norm on the test data set before dimensionality

reduction. However, the result shows the effectiveness of cross-entropy norm in the case

of dimensionally reduced data. As one can see, 4L norm performs well when compared

to other type of norms on the radar ionosphere data set before dimensionality reduction

and 2L in the case of dimensionality reduced data.

REFERENCES [1] Ripley B.D., (1995) ‘ Pattern recognition and Neural Networks’, Cambridge Univ. Press.

[2] Haykin S., (1994) ‘Neural Networks: A Comprehensive Foundation’, McMillan, New York.

[3] Jain A.K., and Chandrasekaran B.,(1982) ‘Dimensionality and Sample size considerations in

pattern recognition practice’, Handbook of Statistics, P.R. Krishnaiah and L.N. Kanal, eds.,

Amsterdam:North-Holland, Vol. 2, pp.835-855.

[4] Lippman R.P.,(1987) ‘An introduction to Computing with Neural Nets’, IEEE ASSP Magazine ,

pp. 4-22.

[5] Rumelhart D.E, Hinton G.E., and Williams R.J.,(1986) ‘Learning Internal Representations by

Error Propagation’, Parallel Distributed Processing: Explorations in the Microstructure of

Cognition, MIT Press, pp 318-362.

[6] Charalambous C.,(1992) ‘Conjugate Gradient Algorithm for Efficient Training of Artificial’,

Neural Networks, IEE Proceedings-G, Vol. 139, pp. 301-310.

[7] Johanson E.M., Dowla F.U., and Goodman D.M.,(1992) ‘Backpropagation Learning for

Multilayer Feed-forward Neural Networks using the Congugate Gradient Method’, International

Journal of Neural Systems, Vol. 2, pp. 291-301.

[8] Moller M.F.,(1993) ‘A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning’,

Neural Networks, Vol. 6, pp. 525-533.

[9] Smagt P.P.,(1997) ‘Minimisation Methods for Training Feed-forward Neural Networks’,

Neural Networks, Vol. 7, pp. 1-11.

[10] Falas T.,(1995) ‘Neural Networks in Empirical Accounting Research: An Alternative to

Statistical Models’, Neural Networks World, Vol. 5, pp. 419-432.


[11] Kalman B.L., and Kwasny S.C.,(1991) ‘A Superior Error Function for Training Neural

Networks’, International Joint Conference on Neural Networks.

[12] White H.,(1992) ‘Artificial Neural Networks: Approximation and Learning Theory’,

Blackwell, Cambridge.

[13] Siu S., and Cowan C.F.N.,(1993) ‘Performance Analysis of the Lp norm Backpropagation

Algorithm for Adaptive Equalisation’, IEE Proceedings-F Vol. 140, pp. 43-47.

[14] Sang-Hoon Oh,(1997) ‘Improving the error backpropagation algorithm with a modified error

functions’, IEEE Neural Networks, vol. 8, pp. 799-803.

[15] Shannon, C. E.,(1948) ‘A Mathematical Theory of Communication’, The Bell System

Technical Journal, 27(3), pp. 379-423.

[16] Bishop, C.M.,(1995) ‘Neural Networks for Pattern Recognition’, Oxford University Press,

Oxford,.

[17] Gorman, R. P., and Sejnowski, T. J.,(1988) ‘Analysis of Hidden Units in a Layered Network

Trained to Classify Sonar Targets’, in Neural Networks, Vol. 1, pp. 75-89.

[18] Newman, D.J., Hettich, S., Blake, C.L. and Merz, C.J., (1988) ‘UCI Repository of machine

learning databases’ [http://www.ics.uci.edu/mlearn/MLRepository.html]. Irvine, CA: University of

California, Department of Information and Computer Science.

[19] Salankar, S. S., and Patre, B. M.,(2006) ‘Sonar Signal Classification using a Neural Network

Approach’, Proc. of the National Conference, NEST-06, L.D. Institute of Engineering and

Technology, Alwar-Rajasthan(India), pp.01-05.

[20] Sigillito Vince, ([email protected]),(1989) ‘John Hopkins University Ionosphere

database’, Applied Physics Laboratory, Johns Hopkins University, Johns Hopkins Road, Laurel,

MD 20723.

[21] Sigillito, V.G., Wing, S.P., Hutton, L.V., and Baker, K.B.,(1989) ‘Classification of radar

returns from the ionosphere using neural networks’, Johns Hopkins APL Technical Digest, Vol.10,

pp.262-266.

[22] Salankar, S.S. and Patre, B.M., (2006) ‘RBF Neural Network based Model as an Optimal

Classifier for the classification of Radar returns from the ionosphere’, Proc. IEEE International

Conference, ICIT 2006, Mumbai, pp.2043-2048.

[23] Principe, J.C., Euliano, N.R., and Lefebvere, W.C.,(2000) ‘Neural and Adaptive Systems:

Fundamental Through Simulations’, John Wiley and Sons, Inc.

[24] Salankar, S.S. and Patre, B.M.,(2007) ’Effects of Dimensionality Reduction and Different Error

Norms on MLP Neural Network based classifier on Sonar Signal Returns’, Proc. International

Conference, RACE-07, College of Engineering, Bikaner-Rajasthan (India), March 24-25, pp. 1-6.


Suresh S. Salankar, Principal, J. L. Chaturvedi College of Engineering, Nagpur, India, E-mail: [email protected] PhD in Electronics & Telecommunication Engineering from the SRTM University Nanded, India in 2008. His research interests are in the design and evaluation of learning algorithms for Pattern recognition applications. This includes, in particular, neural network classifiers, support vector machines classifier and classifier combing strategies. He has published several papers in these areas. He is a member of the Institution of Engineers (India) and Indian society for Technical Education.

Balasaheb M. Patre was born in Basmathnagar, India in 1965. He received B. E. and

M. E. degree in Instrumentation and Control Engineering in 1986 and 1990 respectively

from Marathwada University, Aurangabad and subsequently doctorate degree (Ph. D.) in

Systems and Control Engineering from IIT, Bombay in 1998. He has published seventy

five papers in the National/International conferences/journals. He has presented his

research work at Cambridge University, UK and in Germany. He is a life member ISTE and Instrument

Society of India, member of IETE, IEEE, and IET. He is reviewer for several International Journals. His area

of interest includes robust control, VSS, interval arithmetic applications in robust control, intelligent control

etc. Presently he is working as Professor of Instrumentation Engineering at SGGS Institute of Engineering

and Technology, Nanded, India.


292

AN ANALYTICAL FRAMEWORK FOR CRITICAL LITERATURE

REVIEW OF SUPPLY CHAIN DESIGN

DARSHAN KUMAR, OM PAL SINGH, JAGDEV SINGH

Abstract There can be little dispute that supply chain design is an area of importance in

the field of engineering, yet there have been few literature reviews on this topic. Over the

past decade, the traditional purchasing and logistics functions have evolved into a broader

strategic approach to materials and distribution management in engineering, known as

supply chain management. This paper sets out not to review the mathematical models

developed in supply chain literature per se, but rather to contribute to a critical theory

debate through the presentation and use of a framework for the categorisation of literature

linked to supply chain design through mathematical modelling. In addition, this article

attempts to clearly describe supply chain management since the literature is replete with

buzzwords that address elements or stages of this new management philosophy. The present

study is based on the analysis of a large number of publications on supply design.

Key Words, Supply chain design, analytic hierarchy process, time-compression, JIT.

1. Introduction

A supply chain may be defined as an integrated process, wherein a number of various business entities, i.e., suppliers, manufacturers, distributors, retailers and end users work together in an effort to acquire raw material, convert them into specified final products, deliver these final products to retailers and then to the end users. Thus, supply chain is comprised of two basic, integrated processes: (a) The Production Planning and Inventory control Process, and (b) The Distribution and Logistics Process. This chain is traditionally characterized by a forward flow of material and a backward flow of information. From a practical standpoint, the supply chain concept arose from a number of changes in the manufacturing environment, the shrinking resources of manufacturing bases, shortened product life cycles, the levelling of the playing field within manufacturing and the globalization of market economies. The current interest has sought to extend the traditional supply chain to include “reverse logistics”, to include the product recovery for the purpose of recycling, remanufacturing and reuse [1]. The



REVIEW OF SUPPLY CHAIN DESIGN 293

supply chain process can be described by figure 1. During the 1990s, many manufacturers and service providers sought to collaborate with their suppliers to upgrade their purchasing and supply management functions from a

clerical role to an integral part of a new phenomenon known as supply chain design [2]. Wholesalers and retailers have also integrated their physical distribution and logistics functions into the transportation and logistics perspective of supply chain design to enhance competitive advantage [3]. This article reviews the literature base and development of supply chain design on the bases of mathematical models developed in the different fields of engineering, using analytic hierarchy process (AHP), time-compression and just-in-time (JIT) approach. This paper has been organized as follows. In, section 2, suply chain has been defined. In section 3, evolution of supply chain has been discussed. In section 4, literature review of papers dealing with modelling developed in the different fields of engineering using analytic hierarchy process (AHP), linear and non-linear programming, time-compression and just-in-time (JIT) approach have been discussed. Conclusion of the paper has been summed up in section 5. 2. The supply chain defined

The new view of supply chain management and one that should take hold of in the new millennium is the inter-organizational approach. The goals of entire supply chain become the common objectives of each firm. Cost and service improvements that were not achievable by individual firms will now be attained by the companies acting together. Thus, managing the coalition of firms becomes very important to ensure that the supply chain runs smoothly [4]. Supply chain management has been defined by various experts.

294 D. KUMAR, O.P. SINGH AND J. SINGH

A few of these definitions are given below in table 1[5]: Table 1. Definitions of supply chain management

Authors Definition

Tan et al. (1998) Supply chain management encompasses materials/supply

management from the supply of basic raw materials to final product (and possible recycling and re-use). Supply chain management focuses on how firms utilise their suppliers' processes, technology and capability to enhance competitive advantage. It is a management philosophy that extends traditional intra-enterprise activities by bringing trading partners together with the common goal of optimisation and efficiency.

Berry et al. (1994) Supply chain management aims at building trust, exchanging information on market needs, developing new products, and reducing the supplier base to a particular original equipment manufacturer(OEM) so as to release management resources for developing meaningful, long term relationship.

Jones and Riley (1985) An integrative approach to dealing with the planning and control of the material flow from supplier to end-user.

Saunders (1995) External Chain is the total chain of exchange from original source of raw material, through the various firms involved in extracting and processing raw materials, manufacturing, assembling, distributing and retailing to ultimate end customers.

Ellram (1991) A network of firms interacting to deliver product or service to the end customer, linking flows from raw material supply to final delivery.

Christopher (1992) Network of organisations that are involved, through upstream and downstream linkages, in the different processes and activities that produce value in the form of products and services in the hands of the ultimate consumer.

Lee and Billington (1992) Networks of manufacturing and distribution sites that procure raw materials, transform them into intermediate and finished products, and distribute the finished products to customers.


Kopczak (1997) The set of entities, including suppliers, logistics services providers, manufacturers, distributors and resellers, through which materials, products and information flow.

Lee and Ng (1997) A network of entities that starts with the suppliers' supplier and ends with the customers’ custom the production and delivery of goods.

According to Singh and Chand [6], supply chain is comprised of following three

basic integrated processes: (1) procurement process (2) production planning and inventory control processes and (3) distribution and logistics processes. Procurement process encompasses the purchasing of raw materials from the identified suppliers and their storage and control at the manufacturing facilities. Production planning processes encompasses the entire manufacturing processes including production scheduling and material flow system design and control. Inventory control describes the design and management of storage policy and procedures for raw materials, work-in-process inventories and finished products. Distribution and logistics process determines how products are replenished and transported from plants to distribution centres. These products may be transported directly to the customers. These processes interact with one another in an integrated supply chain. They defined supply chain with the help of a block

diagram, shown as figure 2.


3. Evolution of supply chain In the 1950s and 1960s, most manufacturers emphasized on mass production to minimize unit production cost as the primary operations strategy, with little product or process flexibility. New product development was slow and relied exclusively on in-house technology and capacity. 'Bottleneck' operations were cushioned with inventory to maintain a balanced line flow, resulting in huge investment in work in process (WIP) inventory. Sharing technology and expertise with customers or suppliers was considered too risky and unacceptable and little emphasis appears to have been placed on cooperative and strategic buyer-supplier partnership. The purchasing function was generally regarded as being a service to production, and managers paid limited attention to issues concerned with purchasing. In the 1970s, Manufacturing Resource Planning was introduced and managers realized the impact of huge WIP on manufacturing cost, quality, new product development and delivery lead-time [2]. The term supply chain management was originally introduced by consultants in the early 1980s. Since the early 1990s, academics have attempted to give structure to supply chain management [7-8]. More recently, just-in-time (JIT), time compression and other management initiatives are being utilized to improve manufacturing efficiency and cycle time. In the fast-paced JIT manufacturing environment with little inventory to cushion production or scheduling problems, buyer-supplier relationship has become an important factor to be taken care of by the strategic management. The concept of supply chain management helped the top management relationship with their immediate suppliers. Once this relationship established, experts in transportation and logistics carried the concept of materials management a step further to incorporate the physical distribution and transportation functions. This resulted in the integrated logistics concept, also known as supply chain management. More and more work on the evolution of supply chain management continued into the 1990s with drastic improvements in IT sectors. This broadened the supplier efficiency to include more sophisticated reconciliation of cost and quality considerations.

More recently, many manufacturers and retailers have adopted this concept of supply chain management to improve their profits. Manufacturers now commonly exploit supplier strengths and technology in support of new product design and development. Retailers seamlessly integrate their physical distribution function with transportation partners to successfully achieve results in the field of JIT. A key facilitating mechanism in the evolution of supply chain management is a customer-focus corporate vision, which drives change throughout a firm's internal and external linkages, as shown in figure 3.


4. A framework of literature review on different models developed for supply chain design It has been observed from the review of open literature that researchers have developed various models in the field of supply chain using different techniques. Chan and Lee [9] have very nicely summed up these techniques in their book. They have given detailed models for independent policies for buyers and vendors, considering various options. Various models in the field of supply chain have been classified and discussed below.

4.1 Analytic Hierarchy Process (AHP) models

The Analytic Hierarchy Process (AHP) was originally developed by Satty in 1980. It is a revolutionary breakthrough which empowers people to relate intangibles to tangibles, the subjective to the objective and to link both to their purposes. It offers a way to integrate complexity, set the right objectives, establishes their priorities and determines the overall value of each alternative solution. The AHP uses hierarchical decision models and it has a sound mathematical basis. AHP is based on the following


three principles: decomposition, comparative judgments and the synthesis of priorities. AHP starts by decomposing a complex, multi criteria problem into a hierarchy where each level consists of a few manageable elements which are then decomposed into another set of elements. The second step is to use a measurement methodology to establish priorities among the elements within each level of hierarchy. The third step in using AHP is to synthesise the priorities of the elements to establish overall priorities for the decision alternatives. Many researchers used this methodology in different fields to select one alternative from the many available depending upon different selection criteria, as studied from the open literature. Korpela et al. [10] utilized AHP for supporting the process re-engineering approach called “PROPER” and demonstrated with the help of illustrative examples. They used AHP to compare the performance level of a company to that of the best-in-class companies and also for analysing customer requirements, taking reliability, flexibility and relationship as main service elements and for benchmarking the logistics operations taking reliability, flexibility, lead time, customer effectiveness and value-addition as logistic critical success factors. It was concluded that owing to its flexibility to support many types of problems, the AHP shows good potential in supporting PROPER-based supply chain development process. Elsewhere, Korpela et al. [11] presented an approach based on the analytic hierarchy process (AHP) and mixed integer linear programming for similar study i.e., for customer requirement. An integrated framework was presented for an approach which enables a customer-oriented evaluation of each alternative link and node in the logistic network and optimized the overall customer service capability of the network. To evaluate the importance of the customers, the criteria considered were the long term profitability potential, the possibility of establishing a partnership-type relationship with the customer, the volume purchased by certain customer and the long term financial viability of a customer. From the study, conclusion drawn was that, that integrating the analytic hierarchy process (AHP) and mixed integer linear programming expands the scope of the traditional approaches to a more customer oriented direction by implementing customers’ preferences on the decision process. Similar combination of AHP and linear programming has also been applied by Ghodsypour and O’Brien [12] to study the problems related to supplier selection, including both qualitative and quantitative factors. Here supplier selection problem has been divided between two categories, one, when there is no constraint and two, when there are some limitations in suppliers’ capacity, quality etc. Real quantitative data has been used to improve the system’s consistency. Five steps have been used in application of AHP: define the criteria; calculate the weights of the criteria; rate the alternatives; compute the overall score for each alternative and built the linear model based on the results achieved. They took cost,


quality and service as the main criteria for the supplier selection to maximize the total value of purchasing and concluded that AHP enables management to make a trade-off between several tangible and intangible priorities, but in this method, errors can creep in due to perception or biased behaviour of the decision making managers and independent nature of attributes used. This problem was solved by Murlidharan et al. [13], who used AHP for selection and rating of vendors, by getting the rating done by a group of decision makers for continuous evaluation of vendors, following the principle of anonymity and integrated the method with a managerial tool-Delphi method. For further better results, emphasis has been laid on establishment of confidence limits in group decision making. The persons, whose opinions fall outside the group’s confidence limit, were further studied to understand the source of variation. It was concluded that AHP applied this way, resulted in better communication leading to clearer understanding among the members of decision making groups and hence greater commitment to the chosen alternative. Supplier-customer relationship was further studied by Korpela et al. [14] from different angle using the same methodology of combination of AHP and mixed integer programming. Here, using already published data, an AHP model has been developed to prepare a sales plan for a company where the risk related to the Supplier-customer relationship in the decision process has also been included. In this work, mixed integer programming (MIP) has been used to divide the production capacity between the customers. The factors considered were profitability, relationship and volume of sales from a particular customer. The conclusion drawn from this work was that, that traditionally, the focus had been on company’s own quantitative view-points, but approach used here enabled focusing on the customers’ viewpoint. Still, another criterion in supplier selection was used by Handfield et al. [15], i.e., environmental performance indicators (EPI). They illustrated the use of the analytic hierarchy process (AHP) as a decision support model that included relevant environmental criteria and that could be readily applied to a variety of industry applications. They identified a number of EPIs from the literature. To assess the overall ranking of criteria individually and for group consensus, a Delphi group study was conducted and a model was then created that refined and consolidated the set of measures to include those that could be easily accessed and were important from environmental standpoint. Pilot tests were conducted on an automotive manufacturer, a paper manufacturer and an apparel manufacturer and examined how AHP could be incorporated into a comprehensive information system supporting Environmentally Conscious Purchasing (ECP). It was concluded that the model developed could be further improved as some purchasing managers did not value all the environmental measures given to them. Second limitation stated was data availability. They proposed that for viable solution to these drawbacks, system of


equations could be to develop and this way, environmental costs should be integrated with the “Total Cost of Ownership (TCO)”. This concept was applied by Bhutta and Huq [16], who applied AHP in the supplier selection process and compared it with the total cost ownership (TCO) method. It was concluded that TCO tends to focus more on the pricing issues and ignores qualitative issues, its strength being the ability to use the same model to evaluate suppliers across the board, based on lowest transaction costs and AHP helps in comparing seemingly incomparable issues to select optimal supplier. But, in this model, capacity constraint of the supplier has not been considered, as AHP does not consider such constraints. This problem was addressed by Wang et al. [17]. They developed an integrated analytic hierarchy process (AHP) and pre-emptive goal programming (PGP) based multi-criteria decision-making methodology to take into account both qualitative and quantitative factors in supplier selection. AHP was used to match product characteristics to qualitatively determine supply chain strategy and PGP to mathematically determine the optimal order quantity from chosen suppliers. PGP model considered capacity constraint of the supplier also.

Till early 2000, no feedback from the customers was incorporated in the process and the attributes considered were largely independent. These problems can be overcome by using more general form of AHP, called Analytic Network Process (ANP). Agarwal and Shankar [18] used Analytic Network Process (ANP), which incorporates feedback and interdependent relationships among decision attributes and alternatives. They made a model to aid the decision makers in prioritizing the options related to improvement in the supply chain management by considering three factors: market sensitiveness, information driven and process integration. Considering the additional benefits of Analytic Network Process (ANP) given above, it has also been used to study the effect of environmental factors on strategic planning considering life cycle, operational life cycle, performance measures and environmentally influential organizational policy elements [19]. In this paper, it has been concluded that the major disadvantage of this methodology is the large amount of decision-maker input required, for the analysis of which some software should be developed. Also, the computational and data requirements still make “what-if” analysis geometrically more cumbersome. These problems have been addressed by Min and Melachrinoudis [20] who applied AHP to present a real-world case study involving the re-location of a combined manufacturing and distribution (warehousing) facility for a firm which primarily manufactures and distributes home improvement hardware. The authors designed the configuration of supply chain networks and assessed the viability of the proposed sites from supply chain perspective using the analytic hierarchy process (AHP). The various location factors of proposed sites, such as site characteristics, cost, traffic access, market opportunity,


quality of living at the proposed sites, local incentives were studied and these factors were used in designing an AHP model. Based on the study of designed model, they were able to reach the conclusion that traffic access was the most important criteria for the company under study. The relative advantages of other candidate sites have also been mentioned. The model enabled the location planner to evaluate ‘what-if’ scenarios associated with shifts in the company’s management philosophy and competitive positions and also enabled the location planner to determine the extent of conflicts among the competing objectives. The problem of large data analysis cited in [19] was taken up by Yang and Kuo [21], who proposed an AHP and data envelopment analysis (DEA) methodology to solve a plant layout design problem. To avoid large input-data collection problem, they adopted a computer aided layout planning tool to facilitate the layout alternatives generation process as well as quantitative performance data and then applied AHP to collect qualitative performance data. Finally, DEA has been used to solve the layout design problem by simultaneous considering both quantitative and qualitative performance data. This methodology was also used for an anonymous leading packaging company to illustrate its efficiency and effectiveness. But, a large data had to be collected for the development of such models, making these processes cumbersome. To avoid this problem of large data collection, Poyhonen and Hamalainen [22], collected data through internet. They collected data for different persons who selected the particular attributes in the job evaluation task and applied AHP for the evaluation of the same. The subjects were allowed to create the alternatives and attribute themselves in their experiment. Each subject used five methods to assess attribute weights- one version of AHP, direct point allocation, simple multi-attribute rating technique, swing weighting and trade-off weighting. The results computed with these different methods were compared and it was concluded that weights differ because decision makers choose their responses from a limited set of numbers and the spread of weights and the inconsistency between the preference statements depend on the number of attributes that a decision maker considers simultaneously. AHP has also been utilized to select the most appropriate technology for seawater desalination [23].

AHP is mainly used in crisp decision applications with a very unbalanced scale of judgement and to overcome this problem, AHP has been used by various researchers along with fuzzy logic application for the purpose of comparison. A new and general decision making method for evaluating weapon systems using fuzzy AHP based on entropy weight has been used [24]. But, the method used is very subjective and calculations are very complicated. Simpler way to solve the same problem of evaluating weapon systems has been used by Chen [25]. Here, fuzzy AHP has been used by a new, but, simpler way and it has been concluded that the method used is simpler and faster.


But the shortcoming in the method used is that the criterion’s score has not been normalized. The same problem was solved, taking this shortcoming into consideration by Cheng, using a new method- ranking fuzzy numbers [26].Yet another method, fuzzy logarithmic least square method has been used in AHP to solve a theoretical problem [27]. Constraints have not been considered in the problem solved. Salo [28] used fuzzy ratio comparison in hierarchy decision models through linear programming and considered constraints also. Fuzzy synthetic method has also been applied for evaluation of customer loyalty [29]. AHP based on fuzzy simulation has also been proposed, characterizing fuzzy linguistic variables as triangular fuzzy variables [30-31]. Carrera and Mayorga [32] proposed a fuzzy inference system for supplier selection problem for new product development to handle the impreciseness and uncertainty, considering technological level, economical situation, production capacity and market share as strategic options. However, the researchers stated that they were not able to give an answer whether decision makers do indeed take into account the attribute ranges in assessing the weights, which might give variation in the results achieved and this may be taken for the future study.

4.2 Models using Linear and Non Linear Programming

The development of an integrated supply chain requires the management of material and information flows to be viewed from the three perspectives: strategic, tactical and operational. At each level the use of facilities, people, finance and systems must be coordinated and harmonized as a whole. Researchers have developed mathematical models using different aspects of these perspectives. Arntzen et al. [33] considered strategic decisions, such as location of customers and suppliers, location and availability of inexpensive skilled labour, cost of various transportation modes, export regulations etc. for a computer company- Digital Equipment Corporation to develop a mathematical model and Pyke & Cohen [34] considered strategic decisions in a firm to avoid conflicts of the production staff with those of marketing staff. For this purpose, a model of an integrated production-distribution system was prepared, that comprised of a single station model of a factory, a stockpile of finished goods and a single retailer. For similar purposes, Talluri et al. [35] utilized another approach for mathematical model development, i.e., data envelopment analysis for improvements in strategic decisions to find out efficient candidates for designing, manufacturing and distribution etc. and proposed a two-phased quantitative framework to aid the decision making process in effectively selecting an efficient and a compatible set of partners. Same approach of data envelopment analysis, mixed with another operations research technique, multi-objective programming was used by Weber et al. [36] also to prepare a mathematical model for


studying another part of strategic decisions, i.e., JIT supply and an approach for evaluating a number of vendors was presented to employ in a procurement situation. Similar work of supplier selection was considered by Talluri and Sarkis [37] to develop a mathematical model for the supplier performance evaluation and monitoring processes, which assisted in maintaining effective customer-supplier linkages, considering price, quality, delivery and flexibility as variables apart from JIT supply. The model and its applications have been demonstrated through a previously published illustrative case example. Further, Talluri and Baker [38] presented a multi-phase mathematical programming approach for effective supply chain design considering potential suppliers, manufacturers and distributors. The model developed was based on game theory concepts and linear & integer programming methods, taking cost, product variety, quality and lead time as main attributes. Same area of manufacturers and distributors has also been analysed for competitive behaviour using Nash equilibrium theory by Bing and Dao-li [39]. Chan et al. [40] developed a model to find the vendor's optimal production policy, considering two-level supply chain of retailer and manufacturer. In the model developed, it has been assumed that the manufacturer has full knowledge of his demand throughout the time interval which is equal to the retailer's optimal production rate in the time interval considered. Yet another part of strategic decisions, i.e., to react on time and on the design of systematic decision-making processes for the supply chain by using control laws to manage the dynamic system was studied by Perea et al. [41]. A mathematical model was developed for a polymer manufacturer for the analysis of the ability of enterprise systems, taking customer satisfaction and inventory oscillation as variables. Another important aspect of strategic decisions for vendor–buyer synchronization was studied by Chan and Kingsman considering single-vendor multi-buyer supply chain [42]. A mathematical model was developed to achieve synchronization by scheduling the actual delivery days of the buyers and coordinating them with the vendor’s production cycle whilst allowing the buyers to choose their own lot sizes and order cycles. Three examples have been considered to show that the synchronization policy works by using the mathematical model developed. Another important aspect of strategic decisions was taken up by Vidal et al. [43] to present a mathematical model for optimization of a global supply that maximizes the after tax profits of a multinational corporation, using linear programming technique. Decision variables considered were transfer prices and transportation costs. Location of distribution centres was identified as an area for future study. Hammel et al. [44] took up this area as research field and developed a mathematical model for the re-engineering of Hewlett-Packard’s CD-RW supply chain. They considered technological innovation and delivering innovative products at competitive prices as variables for locating distribution


centres and the business was able to save $50m annually along with quality improvement and introduction of new products at some new distribution centres. Same area of locating distribution centres, along with bill of materials, being another important aspect of strategic decisions, was studied by Yan et al. [45]. A strategic production-distribution mathematical model was developed for supply chain design, with consideration of bills of materials, considering logical constraints. The operations research tool used was mixed integer programming; using purchasing cost, production cost, transportation and distribution cast as variables. The developed model was used on an international computer company in Southeast Asia for illustration. Instead of locating distribution centres, Chauhan et al. [46] developed a mathematical model for strategic level decision when production-distribution of a new market opportunity has to be launched in an existing supply chain. The tool used was mixed integer linear programming, considering cost associated with production enhancement to the existing capacity for new market and additional transportation cost.

Mathematical modelling on operational level considerations, to determine the operating parameters required to be monitored for day to day planning of various echelons of the supply chain has been another field of research. Wong et al. [47] remodelled the dynamics of an already developed model so that optimal production rate obtained becomes a continuous function of time instead of having jump discontinuities. Cohen and Lee [48] presented a comprehensive mathematical model to answer how production and distribution control policies could be coordinated to achieve synergies in performance and how material input, work-in-process and finished goods availability affect costs, lead-times and flexibility. For illustrative purposes, a problem has been examined that consisted of two finished products, three raw materials, one plant, two production lines within the plant and three distribution centres. Conflict between production and marketing departments due to varying interests has not been considered and efficiency improvement in process has been mentioned as future scope of work. Pyke and Cohen [49] considered this conflict. Batch size is wanted to be bigger by production section to reduce set-up costs and work force change cost, whereas distribution section wants it to be small so that response to the changing market demands is quick and later on mathematical models have also been developed to streamline the operations to improve the efficiency and responsiveness of a supply chain in a fine chemical industry and for a consumer goods industry [50-51]. Emphasis was on integrated information framework that would address issues of resource availability, lot sizing and plant responsiveness. Chan et al. [52] and Sabri & Beamon [53] also proposed mathematical model for efficiency improvement of the system. A model of a typical, single channel logistics network has been developed after assessing each of the order


release mechanism and proposed a new order release approach, which was found to be superior and led to improved supply chain performance and multi-objective supply chain to allow use of a performance measurement system that included cost, customer service levels and flexibility in volume or delivery of products. Persson and Olhager [54] also considered quality, lead-times and costs as the key performance parameters to develop mathematical model for a firm manufacturing mobile communication systems. Ways have been proposed to increase the understanding of the interrelationships among above mentioned parameters and other parameters, relevant for the design of supply chain structure. For further improvements in efficiency, Vergera et al. [55] developed an algorithm using the economic delivery and scheduling model and analyzed supply chain dealing with multi-components. Another approach, called algebraic targeting approach to locate the optimum production rate for aggregate planning in a supply chain has also been reported. The proposed algorithm was tested and it was shown that the model provided near optimal solutions for a wide range of problems [56].

4.3 Models using Time Compression and JIT Approaches

A great deal of progress has been achieved in improving the existing business setup by focusing on cost reduction and quality improvement. As revealed by the literature surveyed, time reduction of production cycle is another fundamental element through which competitive advantage can be achieved. In ‘time-compression’, material is made to move faster rather than moving the people faster throughout the process. This approach looks at the flow of product and reduces the time period during which no work is being carried upon it, a period found to be significantly large part of the total production time. The commonly used time related performance measures are lead time, delivery time, cycle time, processing time, waiting time, transportation time etc. The essence of time-compression is to reduce or eliminate the maximum possible idle time out of these parts of material flow. The main groups of non value-adding time are queuing time, rework time, decision making time, waiting time etc. and emphasis has been laid to reduce these time-periods. Researchers have tried to reduce total production time by working on various strategies: refocus on sequence of activities, synchronization of lead times and capacities, reduction in number of process steps, combination of activities, minimizing material delay and waiting time, reducing the sub lot size, keeping low level of work in progress inventory, overlapping more than one activities and reducing variety in the inbound flow of material etc. Beesley [57] stated that the key to achieving time compression is to remove waste and refocus the sequence of the activities so that the time consumption is reduced for the total supply chain system. A tool, time-based-process-mapping has been developed that helped to answer the questions


such as who the end user was, identified the core processes that took place to serve the end user and related the core processes and the resources utilized to the consumption of time. Further time compression has been achieved by optimizing information & material flows and by workflow management in the supply chain and the results achieved have been verified using a simulation model [58-59]. It has been concluded that to maximize production cycle time compression, a supply chain must re-design order information usage strategy. This way of time compression through fast information flow to the suppliers and customers has further been utilized by Salvador et al. [60]. It has been illustrated through both model development and empirical analysis of 164 plants. It has been suggested that when an organization interacts with suppliers and with customers on quality management issues, the organization would improve its time performances indirectly as a result of complete mediation by internal practices for quality improvement, low management, inter-unit coordination and vertical coordination. Interactions with suppliers and customers should improve time-related performances in terms of delivery punctuality and throughput time. Time compression while working with suppliers can also be achieved with better and timelier information about orders, new products and special needs and through focusing on minimum reasonable inventory (MRI) [61]. It has been stressed that production should look to manufacture in economic batch quantities so as to achieve economies of scale. Further emphasis has been on reduction and elimination of delays and proper design of feedback loops. On the basis of industrial studies, it has been confirmed that collapsing cycle times drive the business into more competitive scenario. Along with reduction in inventory, time compression is possible through standardization of the products; working on not value added inspection and internal transport also [62]. The ratio of lead time to value added time has been minimized by reducing the queuing effect to its minimum. Jones and Towill [63] further stressed upon total cycle time (TCT) compression in an agile supply chain through reduction in information lead time. Time compression has been applied on a fashion supply chain. Emphasis has been on the use of IT for information flow from one end to another. It has been concluded that in the information enriched supply chain, each player receives the marketplace data directly. While working with suppliers, another aspect studied for total cycle compression has been through concentrating on purchasing and transportation processes [64]. A set of total cycle time factors related to purchasing and transportation has been extracted to produce a conceptual model. Five objects of managerial attention identified for time compression have been the nature of relationship with important suppliers including transportation carriers, the characteristics of the physical process of transportation, the characteristics of the information system that processes purchasing and transportation transactions and information, the composition


and functioning of management decision-making teams and the characteristics of strategy and strategy formulation for purchasing and transportation.

Ramasesh [65] et al. reduced total cycle time of production through lot streaming, overlapping of process steps and by reducing work-in-process inventories. Lot streaming provides an opportunity to lower the cost of work-in-process inventories. Economic production lot size model has been presented that minimized the total relevant cost by using lot streaming in the illustrative numerical examples. Through computational analysis, the potential for substantial cost saving by the application of lot-streaming model has been demonstrated. Nieuwenhuyse and Vandaele [66] also worked on time compression by determining the optimal number of sub lots in a single-product, deterministic flow shop with overlapping operations. Formal expressions have been derived for the calculation of gaps and transfer batch lead times and a cost model has been formulated that considered the inventory holding cost, the transportation cost and the gap cost per production cycle. Conclusion drawn was that, that idling of machines is not technologically prohibited but does entail a penalty. Hoque ang Goyal [67] also developed a model for time compression through decreasing sub lot size. In this work, it has been proposed that for minimum in-process inventory the production flow should be synchronized by shifting the lot from a stage to next in equal shipment sizes. The model developed has been compared with already developed model based on the assumption of transferring the lot in equal sized batches through all stages. Two numerical problems have also been solved following the algorithm developed in the model and cost reduction was concluded. Overlapping of various activities was again proposed a way of time compression, along with reduction in variety of the inbound flow of material during the process for an automobile industry [68]. It was proposed that on the basis of a specific product configuration decided by consumer, it was possible to deduct the materials required for the production planning by applying a simple and fast algorithm on each line in the complex bill-of-material and certain combinations of options could actually be produced in connection with order placement. Reduction in lead time and elimination in variety of the inbound flow of materials was concluded. Jayaram et al. [69] also studied time compression through the time based performance of first tier suppliers GM, Ford and Chrysler in North America. Different engineering tools for time compression at different stages of the product were suggested, e.g., CAD/CAM, Concurrent Engineering (CE), Design for manufacturing (DFM), Standardization of product and process design, Preventive Maintenance, Just-in-time (JIT) purchasing, supplier partnering etc. Similar approach of reducing number of steps for time compression has been used by Gehani [70]. Stress has been on the reduction in the series of inspections and approvals required, either at shop floor or in field service. Use of Kanban style clip boards where each


shop-floor worker and supervisor is given information, on an ongoing basis, about how fast they are doing their work, and how much more work they need to do during the rest of that day has been suggested by citing some examples. Further stress to compress cycle time has been on reduction in serial departmentalization of organizations’ structure. Bhattacharya [71] et al. also worked on similar techniques of time compression. Time based process analysis, designed to eliminate non-vale added time and hence render the resultant system more responsive to the needs of customer has been suggested. Total production time has been divided into two parts-technical time and managerial time and ways have been suggested to compress time in both cases for a garment manufacturing company. They suggested eliminating non-value-added time like waiting and breakdown from each activity in the process; launch smaller batches so that the total time taken for each activity was reduced and individual part throughput time is reduced; Run activities in parallel. Time consumed as a result of managerial factors, referred to as managerial time could be compressed in two ways- Integrating the elements that made up the process so that there were fewer hand-offs and deleting those decision processes that contributed to delays, off-line from the core process. Hum and Sim [72] also studied the effect of synchronization of lead times for time compression. It has been suggested that every manager should measure the elapsed time in every step from product conceptualization to product consumption and then work continuously to reduce all critical time intervals. The study has been done in view of introducing new product in a competitive market. Total cycle time compression has also been achieved by Singh and Chand [73-74] for the electronic industry. A model was also developed to reduce inventory cost considering ordering cost, inventory holding cost and backorder penalty cost. Appreciable reduction in total cycle time was concluded after study of layouts of various production shops and process flow charts for different subassemblies and instrument assembly. Usage of optimal sub-lot size instead of fixed sub-lot size was suggested and based on the suggestions made, revised process flow charts were prepared for showing the time compression in the process. But, it should be kept in mind that analysis of a project, proposed for lead time reduction is important. Consequences should be analysed by representatives of variety of business functions [75].

JIT has been another area of interest of the researchers in the field of supply chain management. Miltenburg [76] studied theoretically how JIT reduces cost, inventory, cycle time and improves quality, with a mathematical framework. The mechanism developed was able to identify and reduce waste also. Apart from these benefits, JIT supply of bought out component brings with it the possibility and necessity of improving the business cycle, when applied in an automotive industry [77-78]. It has been pointed out that electronic data interchange between supplier and manufacturer is required to


make the cycle work. Its application enables the industry to have improved cash management and facilitates in generating exact delivery schedules-an important parameter in the field of automotive industry. The impact of supplier-manufacturer linkages in JIT environment has been further explored by Wafa et al. [79] along with the roles of information, communication and relationship with vendors. It has been shown that successful implementation of JIT resulted in production flexibility, reduction in inventory of finished goods, work-in-process and raw material significantly.

Researchers [80-82] have analysed the inventory costs of JIT and economic order quantity (EOQ) purchasing also through comparison of the two methods by developing mathematical models. Existing data has been used for illustration of the developed models. It has been pointed out that under what conditions one system was superior to the other from cost perspective. JIT has been found less costly alternative when the level of annual demand for the inventory item was lower than a certain limit and when the demand increased beyond this limit, EOQ became less costly. It has been cautioned that stock outs caused by JIT ordering policy could add to the costs along with loss of price discounts. This problem was taken up by Min and Pheng [83] to develop a mathematical model for a concrete industry, taking into account the price discount also. The developed model has been compared with the other models where price discount has not been considered and it has been concluded that EOQ with price discount could prove to be better option when demand is more than a certain value, whereas Singh and Chand [84] showed that JIT purchasing is better and economical when compared with EOQ purchasing for an electronics industry. A mathematical model has been developed considering ordering cost, inventory holding cost at central stores and backorder penalty cost. Using this model it has been concluded that unit cost of controlling raw material reduced by 53-59% along with reduction in production cost. For the future scope of study, purchasing through internet and information flow during JIT purchasing has been suggested. But, when supply lead times are uncertain, use of dual-sourcing technique offers savings in inventory holding, the magnitude of which depends on the level of uncertainty in lead times [85].

5. Conclusion

As discussed earlier, the development and evolution of supply chain management owes much to the purchasing and supply management, and transportation and logistics literature. But, for the optimum use of resources, integration of supply chain at purchase, manufacturing, storage and delivery to customer levels is the need of the hour. Genuinely integrated supply chain management requires a massive commitment by all members of the value chain for its successful implementation. Integrating the purchasing with


manufacturing and delivery functions can create a closely linked set of processes. It allows organizations to deliver products and services to both internal and external customers in a more timely and effective manner and thus help in achieving the corporate goals. The results can still be better if time compression and JIT approaches are also added to integrated supply chain implementation, as discussed above. Although supply chain management has been discussed along separate paths of mathematical models development, it has eventually merged into a unified body of literature with a common goal of increased efficiency, better customer satisfaction and better financial results. One of the most significant findings from our literature analysis has been the relative lack of integrated supply chain design for the whole process when compared to design of supply chain for individual sections and can be taken as future scope of study. Other related areas where work still needs to done are simulation-based optimization methodologies; optimization under uncertainty; incorporation of negotiation abilities etc.

REFERENCES [1] Benita M. Beamon, ‘Supply Chain design and analysis: Models and methods’, International

Journal of Production Economics 55 (1998), 281-294.

[2] Tan, K.C.,’A framework of supply chain management literature’, European Journal of

Purchasing and Supply Management 7 (2001), 39-48.

[3] Ballou, R., Gilbert, S., and Mukherjee, A., ‘New Managerial Challenges from Supply Chain

Opportunities’, International Marketing Management 29 (2000), 7-18.

[4] Lancioni, R.A., ‘New Developments in Supply Chain Management for the Millennium’,

Industrial Marketing Management 29 (2000), 1-6.

[5] Croom, S., Romano, P., and Giannakis, M., ‘Supply chain management: an analytical

framework for critical literature review’, European Journal of Purchasing & Supply Management

6 (2000), 67-83.

[6] Singh, O.P., and Chand, S., ‘Supply Chain Management: Emerging trends’, Northern region

convention on Supply Chain Management through Information Technology, Feb. 13-15, 2003

MNNIT Allahabad.

[7] Lambert, Douglas M., and Cooper, Martha C., ‘Issues in Supply Chain Management’,

Industrial Marketing Management 29 (2000), 65-83.

[8] Lambert, Douglas M., and Cooper, Martha C., ‘Issues in Supply Chain Management’,

Industrial Marketing Management, 29(2000), 65-83.

[9] Chan, C., and Lee, H.W.J., ‘Successful strategies in supply chain management’, IDEA Group

Publishing, USA, 2005

[10] Korpela, J., Lehmusvaara, A., and Tuominen, M., ‘An analytic approach to supply chain

development’, International Journal of Production Economics 71 (2001), 145-155.


[11] Korpela, J., Lehmusvaara, A., and Tuominen, M., ‘Customer service based design of the

supply chain’, International Journal of Production Economics 69 (2001), 193-204.

[12] Ghodsypour, S.H., and O’Brian C., ‘A decision system for supplier selection using an

integrated analytic hierarchy process and linear programming’, International Journal of Production

Economics 56-57 (1998), 199-212.

[13] Murlidharan, C., Anantharaman, N., and Deshmukh, S.G., ‘Vendor rating in purchasing

scenario: a confidence interval approach’, International Journal of Operations and Production

Management 21 (2001) 1305-1325.

[14] Korpela, J., Kylaheiko, K., Lehmusvaara, A., and Tuominen, M.,’An analytic approach to

production capacity allocation and supply chain design’, International Journal of Production

Economics 78 (2002), 187-195.

[15] Handfield, R., Walton, S.V., Sroufe, R., and Melnyk, S.A., ‘Applying environmental criteria in

the application of the Analytic Hierarchy Process’, European Journal of Operational Research 141

(2002), 70-87.

[16] Bhutta, K.S., and Huq, F., ‘Supplier selection problem: a comparison of the total cost of

ownership and analytic hierarchy process approaches’, International Journal of Supply Chain

Management 7 (2002), 126-135.

[17] Wang, G., Huang, S.H., and Dismukes, J.P., ‘Product-driven supply chain selection using

integrated multi-criteria decision-making methodology’, International Journal of Production

Economics 91 (2004), 1-15.

[18] Agarwal, A., and Shankar, R., ‘Analyzing alternatives for improvement in supply chain

performance’, Work Study 51 (2002) 32-37.

[19] Sarkis, J., ‘A strategic decision framework for green supply chain management’, Journal of

Cleaner Production 11 (2003), 397-409.

[20] Min, H, and Melachrinoudis, E.,’The relocation of a hybrid manufacturing /distribution

facility from supply chain perspective: a case study’, Omega, International Journal of Management

Science 27 (1999), 75-85.

[21] Yang. T., and Kuo, C., ‘A hierarchical AHP/DEA methodology for the facilities layout design

problem’, European Journal of Operational Research 147 (2003), 128-136.

[22] Poyhenen, M., and Hamalainen, R.P., ‘Theory and methodology on the convergence of

multiattribute weighting methods’, European Journal of Operational Research 129 (2001),

569-585.

[23] Hajeeh, M., and Othman, A.A., ‘Application of analytic hierarchy process in the selection of

desalination plants’, Desalination 174 (2005), 97-108.

[24] Mon, D., ‘Evaluating weapon system using fuzzy analytic hierarchy process based on entropy

weight’, IEEE 7 (1995) 591-598.

[25] Chen, S., ‘Evaluating weapon systems using fuzzy arithmetic operations’, International


Journal of Fuzzy Sets and Systems 77 (1996), 591-598.

[26] Cheng, C., ‘Evaluating weapon systems using ranking fuzzy numbers’, International Journal

of Fuzzy Sets and Systems 107 (1999), 25-35.

[27] Ruoning, X., and Xiaoyan, Z., ‘Fuzzy logarithmic least squares ranking method in analytic

hierarchy process’, International Journal of Fuzzy Sets and Systems 77 (1996), 175-190.

[28] Salo, A.A., ‘On fuzzy comparisons in hierarchical decision models’, International Journal of

Fuzzy Sets and Systems 84 (1996), 21-32.

[29] Lu, Y., Han, J., Li, F., and Li, S., ‘Fuzzy synthetic evaluation on customer loyalty based on

analytic hierarchy process’, Proceedings of Fourth International Conference on Machine Learning

and Cybernetics, Guangzhou, August 18-21, 2005, 2706-2710.

[30] Ning, Y., Tang, W., and Zhang, Y., ‘Analytic hierarchy process based on fuzzy simulation’,

IEEE International Conference on Fuzzy Systems (2005), 546-550.

[31] Wu, Y.L., Chang, M.C.S., and Yang, P.C., ‘The use of fuzzy analytic hierarchy process to

confer the core factors of 3G mobile telecommunications’ content use’, Proceedings of the IEEE

IEEM (2007), 238-241.

[32] Carrera, D.A., and Mayorga, R.V., ‘Supply chain management: a modular fuzzy inference

system approach in supplier selection for new product development’, Journal of Intelligent

Manufacturing, 19(2008), 1-12.

[33] Arntzen, B.C., Brown, G.G., Harrison, T.P., and Trafton, L.L., ‘Global supply chain

management at Digital Equipment Corporation’, Interfaces 25 (1995), 69-93.

[34] Pyke, D.F., and Cohen, M.A., ‘Multiple integrated production-distribution systems’, European

Journal of Operational Research 74 (1994), 18-49.

[35] Talluri, S., Baker, R.C., and Sarkis, J., ‘A Framework for Designing Efficient Value Chain

Networks’, International Journal of Production Economics 62 (1999), 133-144.

[36] Weber, C.A., Current, J., and Desai, A., ‘An optimization approach to determining the number

of vendors to employ’, International Journal of Supply Chain Management 5 (2000), 90-98.

[37] Talluri, S., and Sarkis, J., ‘A model for performance monitoring of suppliers’, International

Journal of Production Research 40 (2002), 4257-4269.

[38] Talluri, S., and Baker, R.C., ‘A multi-phase mathematical programming approach for effective

supply chain design’, European Journal of Operational Research 141 (2002), 544-558.

[39] Bing, X., and Dao-li, Z., ‘Multi-commodity flow supply chain network equilibrium model

with stochastic choice’, System Engineering-Theory & Practice, 27(2007), 82-90.

[40] Chan, C., Lee, H.W.J., and Wong, K.H., ‘Optimal feedback production for a two level supply

chain’, International journal of Production Economics, 113(2008), 619-625.

[41] Perea, E., Grossmann, I., Ydstie, E., and Tahmassebi, T., ‘Dynamic modeling and classical

theory for supply chain management’, Computers and Chemical engineering 24 (2000),

1143-1149.


[42] Chan, C., and Kingsman, B.G., ‘Co-ordination in a single-vendor multi-buyer supply chain by

synchronizing delivery and production cycles’, Transport Research, Part E, 43(2007), 90-111.

[43] Vidal, C.J., and Goetschalckx, M., ‘Theory and methodology: A global supply chain model

with transfer pricing and transportation cost allocation’, European Journal of Operational Research

129 (2001), 134-158.

[44] Hammel, T., Kuettner, D., and Phelp, T., ‘The re-engineering of Hewlett-Packard’s CD-RW

supply chain’, International Journal of Supply Chain Management 7 (2002), 113-118.

[45] Yan, H., Yu, Z., and Cheng, T.C.E., ‘A strategic model for supply chain design with logical

constraints: formulation and solution’, Computers and Operations Research 30 (2003), 2135-2155.

[46] Chauhan, S.S., Nagi, R., and Proth, J.M.,’Strategic capacity planning in supply chain design

for a new market opportunity’, International Journal of Production Research 40 (2004), 1-10.

[47] Wong, K.H., Chan, C., and Lee, H.W.J., ‘Optimal feedback production for a single echelon

supply chain’, Discrete and Continuous Dynamical Systems - Series B, 6(2006), 1431-1444.

[48] Cohen, M.A., and Lee, H.L., ‘Strategic analysis of integrated production-distribution systems:

Models and methods’, Operations Research 36 (1988), 216-228.

[49] Pyke, D.F., and Cohen, M.A., ‘Performance characteristics of stochastic integrated

production-distribution systems’, European Journal of Operational Research 68 (1993), 23-48.

[50] Voudouris, V.T., ‘Mathematical programming techniques to bottleneck the supply chain of

fine chemical industries’, Computers and Chemical engineering 20 (1996), 1269-1274.

[51] Camm, J.D., Chorman, T.E., Dill, F.A., Evans, J.R., Sweeney, D.J., and Wegryn, G.W.,

‘Blending OR/MS, judgment, and GIS: Restructuring P&G’s supply chain’, Interfaces 27 (1997),

128-142.

[52] Chan, F.T.S., Tang, N.K.H., Lau, H.C.W., and Ip, R.W.L., ‘A simulation approach in supply

chain management’, Integrated Manufacturing Systems 13 (2002), 117-122.

[53] Sabri, E.H., and Beamon, B.M., ‘A multi-objective approach to simultaneous strategic and

operational planning in supply chain design’, Omega, International Journal of Management

Science 28 (2000), 581-598.

[54] Persson, F., and Olhager, J., ‘Performance simulation of supply chain designs’, International

Journal of Production Economics 77 (2002), 231-245.

[55] Vergera, F.E., Khouja, M., and Michalewicz, Z., ‘An evolutionary algorithm for optimizing

material flow in supply chains’, Computers and Industrial Engineering 43 (2002), 407-421.

[56] Foo, D.C.Y., Ooi, M.B.L., Tan, R.R., and Tan, J.S., ‘A heuristic-based algebraic targeting

technique for aggregate planning in supply chains’, Computers and Chemical Engineering 32

(2008), 2217–2232.

[57] Beesley, A., ‘Time compression in the supply chain’, Industrial Management and Data

Systems 96 (1996), 12-16.

[58] Jones, R.M., and Towill, D.R., ‘Time compression in the supply chain: information


management is the vital ingredient’, Logistics Information Management 11 (1998), 93-104.

[59] Sun, D., Zhang, X., and Yu, M., ‘Applications of workflow in supply chain management: A

case study’, IEEE, (2006), 540-544.

[60] Salvador, F., and Forza, C., ‘Supply chain interactions and time-related performances’,

International Journal of Operations and Production Management 21 (2001), 461-475.

[61] Towill, D.R., ‘Time compression and supply chain management - a guided tour’, International

Journal of Supply Chain Management 1 (1996), 15-27.

[62] Andries, B., and Gelders, L., ‘Time-based manufacturing logistics’, Logistics Information

Management 8 (1995), 30-36.

[63] Jones, R.M., and Towill, D.R., ‘Total cycle time compression and the agile supply chain’,

International Journal of Production Economics 62 (1999), 61-73.

[64] Ng, B., Ferrin, B.G., and Pearson, J.N., ‘The role of purchasing/transportation in cycle time

reduction’, International Journal of Operations and Production Management 17 (1997), 575-591.

[65] Ramasesh, R.V., Duncan, H.F., Fong, K.H., and Hayya, J.C., ‘Lot streaming in multistage

production systems’, International Journal of Production Economics 66 (2000), 199-211.

[66] Nieuwenhuyse, I.V., and Vandaele, N., ‘Determining the optimal number of sublots in a

single-product, deterministic flow shop with overlapping operations’, International Journal of

Production Economics 92 (2004), 221-239.

[67] Hoque, M.A., and Goyal, S.K., ‘On lot streaming in multistage production systems’,

International Journal of Production Economics 95 (2005), 195-202.

[68] Nielsen, N.P.H., and Holmstrom, J., ‘Design for speed: a supply chain perspective on design

for manufacturability’, Computer Integrated Manufacturing Systems 8 (1995), 223-228.

[69] Jayaram, J., Vickery, S.K., and Droge, C., ‘An empirical study of time-based competition in

the North America automotive supplier industry’, International Journal of Operations and

Production Management 19 (1999), 1010-1033.

[70] Gehani, R.R., ‘Time-based management of technology’, International Journal of Operations

and Production Management 15 (1995), 19-35.

[71] Bhattacharya, A.K., Jina, J., and Walton, A.D., ‘Production market, turbulence and time

compression’, International Journal of Operations and Production Management 16 (1996), 34-47.

[72] Hum, S.H., and Sim, H.H., ‘Time-based competition: literature review and implications for

modelling’, International Journal of Operations and Production Management 16 (1996), 75-90.

[73] Singh, O.P., and Chand, S., ‘Time Compression Strategies in Supply Chain: Insight from

Electronic Industry’, Udyog Pragati, 32(2008), 39-58.

[74] Singh, O.P., and Chand, S., ‘Supply Chain Design of an Electronic Industry Incorporating

Time Compression’, International Journal of Services and Operations Management, 4(2008),

553-579.

[75] Wouters, M.J.F., ‘Economic evaluation of lead time reduction’, International journal of


Production Economics, 22(1991), 111-120.

[76] Miltenburg, J., ‘A theoretical framework for the understanding why JIT reduces cost and cycle

time and improves quality’, International journal of Production Economics, 30-31(1993), 195-204.

[77] O’Brien, C., and Head, M., ‘Developing a full business environment to support just-in-time

logistics’, International journal of Production Economics, 42(1995), 41-50.

[78] Jina, J., ‘Automated JIT based materials management for lot manufacture’, International

journal of Operations and Production Management, 16(1996), 62-75.

[79] Wafa, M.A., Yasin, M.M., and Swinehart, K., ‘The impact of supplier proximity on JIT

success: an informational perspective’, International journal of Physical Distribution & Logistics

Management, 26(1996), 23-34.

[80] Fazel, F., ‘A comparative analysis of inventory costs of JIT and EOQ purchasing’,

International journal of Physical Distribution & Logistics Management, 27(1997), 496-504.

[81] Fazel, F., Fischer, K.P., and Gilbert, E.W., ‘JIT purchasing vs. EOQ with a price discount: An

analytical comparison of inventory costs’, International journal of Production Economics,

54(1998), 101-109.

[82] Schniederjans, M.J., and Cao, Q., ‘A note on JIT purchasing vs. EOQ with a price discount:

An expansion of inventory costs’, International journal of Production Economics, 65(2000),

289-294.

[83] Min, W., and Pheng, L.S., ‘EOQ, JIT and fixed costs in the ready-mixed concrete industry’,

International journal of Production Economics,.

[84] Singh, O.P., and Chand, S., ‘Just-in-Time Practices in Supply Chain: Insight from Electronic

Industry’, Icfai journal of Supply Chain Management, 4(2007), 39-56.

[85] Ramasesh, R.V., ‘Procurement under uncertain supply lead times-A dual-sourcing technique

could save costs’, International journal of Production Economics, 21(1991), 59-68.

Darshan Kumar, Assistant Professor in Mechanical Engineering Department of Beanr

College of Engineering & Technology, Gurdaspur (Punjab), India, received the B.E.

degrees in Mechanical Engineering from Thapar Institute of Engineering & Technology,

Patiala, Punjab (India) in 1990 and M.Tech. degree in Mechanical Engineering from Punjab

Technical University, Jalandhar Punjab (India) in 2003. Presently, he is pursuing his PhD

degree from Punjab Technical University, Jalandhar Punjab (India) in the field of Supply

chain design. He is also a member of professional bodies like ISTE and IE India. E-mail:

[email protected]



Dr. Om Pal Singh, Professor in Mechanical Engineering Department of Beanr College of

Engineering & Technology, Gurdaspur (Punjab), India, did his graduation and post

graduation from University of Roorkee, Roorkee (India). He did is Doctoral inSupply Chain

Design field from MNNIT Allahabad,U.P., India and presently working as Professor and

Head of Mechanical Engineering at Beant College of Engineering & Technology, Gurdaspur

(Pb.), India. He has held many administrative responsibilities like member of Board of

Studies, Coordinator Mechanical Engg. Department etc. He has vast experience of 18 years

in his field. Research interest includes Supply Chain Management, Operations Research and

Welding Technology. He has contributed in ISIJ International, Int. J. Services and

Operations Management, Trans. Indian Inst. Met., Icfai Journal of Management Research,

Icfai Journal of Supply Chain Management, OPSEARCH and PARADIGM etc.

E-mail: [email protected]

Dr. Jagdev Singh, Assistant Professor in Mechanical Engineering Department of Beant

College of Engineering & Technology, Gurdaspur (Punjab), India, received the B.E. and

M.E. degrees in Mechanical Engineering from Punjab University, Chandigarh (India) in

1988 and 1998, respectively and completed his Ph.D. from Punjab Technical University,

Jalandhar Punjab (India). His current research interests centre on the fuzzy logic and its

applications in mechanical engineering systems; Supply chain design & management. He is

also life member of professional bodies like ISTE and IE India. E-mail:

[email protected]



volume 6, number 2, pages 113{127 - university of albertazhiyongz/read/binder1.pdf · 2010. 5....

Documents