a levenberg-marquardt learning applied for recurrent ... · of the polymerization process in a...

A Levenberg–Marquardt Learning Appliedfor Recurrent Neural Identification andControl of a Wastewater TreatmentBioprocessIeroham S. Baruch,∗ Carlos R. Mariaca-Gaspar†

Department of Automatic Control, CINVESTAV-IPN, Av. IPN No2508,A.P. 14-740, Col. Zacatenco, 07360 Mexico D.F., Mexico

The paper proposed a new recurrent neural network (RNN) model for systems identification andstates estimation of nonlinear plants. The proposed RNN identifier is implemented in direct andindirect adaptive control schemes, incorporating a noise rejecting plant output filter and recurrentneural or linear-sliding mode controllers. For sake of comparison, the RNN model is learned bothby the backpropagation and by the recursive Levenberg–Marquardt (L–M) learning algorithm.The estimated states and parameters of the RNN model are used for direct and indirect adaptivetrajectory tracking control. The proposed direct and indirect schemes are applied for real-timecontrol of wastewater treatment bioprocess, where a good, convergence, noise filtering, and lowmean squared error of reference tracking is achieved for both learning algorithms, with priorityof the L–M one. C© 2009 Wiley Periodicals, Inc.

1. INTRODUCTION

The rapid growth of available computational resources led to the developmentof a wide number of neural networks (NN)-based modelling, identification, predic-tion, and control applications,1−10 especially for wastewater treatment bioprocesses.The main network property, namely the ability to approximate complex nonlinearrelationships without prior knowledge of the model structure, makes them a very at-tractive alternative to the classical modelling and control techniques. Among severalpossible network architectures the ones most widely used are the feedforward NN(FFNN),3,8,11−14 and recurrent NN (RNN).1,9,12,13,15,16 There exists a fuzzy-neural6

and a hybrid or modular17 neural network applications too. The most of the FFNNand RNN are learned using the offline or online versions of the backpropagation

∗Author to whom all correspondence should be addressed: e-mail: [email protected].

†e-mail: [email protected].

INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, VOL. 24, 1094–1114 (2009)C© 2009 Wiley Periodicals, Inc. Published online in Wiley InterScience

(www.interscience.wiley.com). • DOI 10.1002/int.20377

RECURRENT NEURAL NETWORK MODEL FOR SYSTEMS IDENTIFICATION 1095

(BP)9,12,16,18,19 or Levenberg–Marquardt11,20−26 learning algorithms. In a FFNN,the signals are transmitted only in one direction, starting from the input layer, subse-quently through the hidden layers to the output layer, which requires applying a tapdelayed global feedbacks and a tap delayed inputs to achieve a nonlinear autoregres-sive moving average (NARMAX) neural dynamic plant model. A RNN has localfeedback connections to some of the previous layers. Such a structure is suitable al-ternative to the FFNN when the task is to model dynamical systems. The NN-basedtechniques were successfully applied in several engineering areas as optimizationof the polymerization process in a twin-screw extruder reactor and acetic anhydrideplant.4 In Ref. 3, a comparative study of linear, nonlinear, and neural-network-basedadaptive controllers for a class of fed-batch baker’s and brewer’s yeast fermentationwas done. The paper proposed to use the method of neural identification control,given in Ref. 8 and applied FFNNs (multilayer perceptron and radial basis functionsNN). The proposed control gives a good approximation of the nonlinear plants dy-namics, better with respect to the other methods of control, but the applied staticNNs have a great complexity, and the plant order has to be known. The applicationof RNNs could avoid these problems and could reduce significantly the size of theapplied NNs.

In some early papers of Baruch and coauthors,1,18,27 the state-space approachis applied to design a RNN in a universal way, defining a Jordan canonical twoor three layer RNN model, named recurrent trainable neural network (RTNN) anda backpropagation (BP) algorithm of its learning. This NN model is a parametricone, and it serves as a parameter estimator and a system state predictor, whichpermits to use the estimated parameters and states directly for process control. Inone previous paper,1 this general RTNN approach is applied in direct and indi-rect neural control schemes for identification and control of continuous wastewa-ter treatment fermentation bioprocess (see also Ref. 28) where unfortunately theplant and measurement noises affected the control precision. In the proposed pa-per, we goes ahead applying the same RTNN topology for the same wastewatertreatment plant,1,28,29 incorporated in a direct and indirect RNN control scheme,containing a low pass noise filter, and applying a second-order learning. Thelearning algorithm used is the more precise second-order recursive Levenberg–Marquardt (L–M) one,21,25,26 applied to the combined plant and filter dynamicsidentification.

The paper is organized as follows. In the first section of methodology, thearchitecture and the learning mechanism of a discrete-time Jordan canonical RTNNare discussed and implemented for process parameter and state estimation. Here,both the backpropagation (BP) and the recursive L–M learning algorithm are de-rived for the RTNN topology. Finally, several RTNNs are incorporated in processdynamics identification, direct and indirect adaptive neural control scheme withnoise rejecting filter. In a separate section, we introduce a mathematical model ofthe wastewater treatment and formulate the control objectives. Finally, in the lastsection, the practical relevance of the proposed control scheme is illustrated bycomparative graphical simulation results, where some statistical analysis also hasbeen done.

International Journal of Intelligent Systems DOI 10.1002/int

1096 BARUCH AND MARIACA-GASPAR

2. THE RTNN MODEL

2.1. The RTNN Architecture and the Backpropagation Algorithmof Its Learning

Figure 1 shows the block-diagram of the RTNN topology. The RTNN model isdescribed by the following equations1,18,27:

X(k + 1) = JX(k) + BU(k); J = block-diag(Ji); |Ji |< 1, i = 1, . . . , N (1)

Z(k) = �[X(k)] (2)

Y (k) = �[CZ(k)] (3)

where X(k) is a N-state vector; U (k) is a M-input vector; Y (k) is a L-outputvector; Z(k) is a L-vector—output of the hidden layer; �(.), �(.) are vector-valuedactivation functions such as saturation, sigmoid, or hyperbolic tangent, which havecompatible dimensions; J is a weight-state block-diagonal matrix with elements Ji ;the inequality in (1) is a stability preserving condition, imposed on the weights Ji ;B and C are weight input and output matrices with compatible dimensions and blockstructure, corresponding to the block structure of J . The given RTNN model is acompletely parallel parametric one, with such parameters the weight matrices J , B,C, and the state vector X(k). The RTNN topology has a linear time varying structureproperties, such as controllability, observability, reachability, and identifiability,which are considered in Refs. 27 and 18. These properties of the RTNN structuresignify that starting from the block-diagonal matrix structure of J , we can find acorrespondence in the block structure of the matrices B and C, that is show us howto find out the ability of learning of this RTNN. The main advantage of this discreteRTNN (which is really a Jordan canonical RNN model) is of being an universalhybrid neural network model with one or two feedforward layers, and one recurrenthidden layer, where the weight matrix J is a block-diagonal one. So, the RTNNposses a minimal number of learning weights and the performance of the RTNN isfully parallel. The described RTNN architecture could be used as one step ahead statepredictor/estimator and systems identifier. Another property of the RTNN model isthat it is globally nonlinear but locally linear. That is why the matrices J , B, C,generated by learning, could be used to design a linear control laws.1 Furthermore,the RTNN model is robust, due to the dynamic weight adaptation law, based on thesensitivity model of the RTNN and the performance index to be minimized, which

Figure 1. Block-diagram of the RTNN topology.



is as follows:

ξ (k) = (1/2)�j [Ej (k)]2, j ∈ C; ζ = (1/Ne)�kξ (k) (4)

Here, the performance index ξ (.) is an instantaneous mean squared error (MSE)and it is a nonlinear function of the weight matrices of the output and the hiddenRTNN layers, respectively; the performance index ζ is the total MSE for one epochof learning with dimension Ne. The general RTNN–BP learning algorithm, writtenin vector-matricial form, is given by the following equation:

W (k + 1) = W (k) + η �W (k) + α �W (k − 1); |Wij | < Wo (5)

where W is the weight matrix, being modified (J , B, C); �W is the weight matrixcorrection (�J , �B, �C), which is defined as �W (k); η is a learning rate normal-ized parameter’s diagonal matrix, and α is a momentum term normalized learningparameter’s diagonal matrix. The inequality in (5) represents an antiwindup con-dition, where the learned weight Wij is restricted in a specified region Wo. Themomentum term of this learning algorithm is used when some error oscillationsoccurred. The general BP learning algorithm (5) could be applied online, in realtime, where the instantaneous MSE ξ (.) is minimized with respect to the RTNNweights each learning iteration, or offline, where the total MSE ζ is minimized oncefor epoch of learning Ne. The weight matrix elements update for the discrete timemodel of the RTNN has been derived and applied in Ref. 27, but here it will beexpressed in vector-matricial form,1,18 using the diagrammatic method of Wan andBeaufays.19 The BP learning algorithm used the error prediction adjoint topology,designed by means of the diagrammatic rules, based on the RTNN topology, givenby Equations 1–3 and illustrated in Figure 1. Figure 2 shows the block-diagram ofthe adjoint RTNN.

The weight update algorithm is as follows:

�C(k) = E1(k)ZT (k); E1(k) = �′[Y (k)]E(k); E(k) = Yp(k) − Y (k) (6)

�J (k) = E3(k)XT (k); E3(k) = �′[Z(k)]E2(k); E2(k) = CT (k)E1(k) (7)

�vJ (k) = E3(k) ⊗ X(k) (8)

�B(k) = E3(k)UT (k) (9)

Figure 2. Block-diagram of the adjoint RTNN topology used for the BP algorithm of learning.



where �J , �B, �C are weight corrections of the of the learned matrices J , B, C,respectively; E(k) = Yd (k) – Y (k) is an L-error vector of the output RTNN layer,where Yp is a desired target vector (plant output for identification tasks) and Y is aRTNN output vector, both with dimensions L; X is a N-state vector, and Ej is a j therror vector with respective dimension; �′, �′ are diagonal Jacobean matrices withappropriate dimensions, which elements are derivatives of the respective activationfunctions. Equation 7 represents the learning of the feedback weight matrix of thehidden layer, which is supposed as a full (N × N) matrix. Equation 8 gives thelearning solution when this matrix is diagonal, where vJ is an N-vector, which isthe diagonal of the matrix J . Equations 1–3 together with Equations 6–9 form aBP-learning procedure, where the functional algorithm 1–3 represented the forwardstep, executed with constant weights, and the learning algorithm 6–9 representedthe backward step, executed with constant signal vector variables. This learningprocedure, denoted by Y = (L, M , N , Yp, U , X, J , B, C, E), could be executedonline or offline. It uses as input data the RTNN model dimensions L, M , N , and thelearning data vectors Yp, U , and as output data—the X-state vector, and the matrixweight parameters J , B, C.

2.2. Stability Proofs of the Learning Algorithm

The stability and the properties of the BP–RTNN learning algorithm, given byEquation 5, are proved in one theorem and one lemma.

2.2.1. Theorem of Stability

Let the RTNN with Jordan canonical structure, is given by Equations 1–3 andthe nonlinear plant model is as follows:

Xp·(k + 1) = F [Xp(k), U (k)] (10)

Y ∗(k) = G[Xp(k)] (11)

where {Y ∗, Xp, U} are output, state, and input variables with dimensions L, Np,M , respectively; F (.), G(.) are vector-valued nonlinear functions with respectivedimensions. Under the assumption of RTNN identifiability made, the applicationof the BP learning algorithm for J , B, C, in general matricial form, described byEquation 5, and the learning rates η(k), α(k) (here they are considered as time-dependent and normalized with respect to the error) are derived using the followingLyapunov function18,27:

L(k) = ||J (k)||2 + ||B(k)||2 + ||C(k)||2 (12)

Then, the identification error is bounded, i.e.

�L(k) ≤ −η(k)|E(k)|2 − α(k)|E(k − 1)|2 + d; �L(k) = L(k) − L(k − 1)

(13)



where all the unmodelled dynamics, the approximation errors, and the perturbationsare represented by the d-term, and the complete proof of that theorem is given inRef. 18.

2.2.2. Lemma of the Rate of Convergence

Let �Lk is defined by Equation 13. Then, applying the limit’s definition, theidentification error bound condition is obtained as

limk→∞

1

k

k∑t=1

(|E(t)|2 + |E(t − 1)|2) ≤ d (14)

Proof. Starting from the final result of the theorem of RTNN stability:

�L(k) ≤ −η(k)|E(k)|2 − α(k)|E(k − 1)|2 + d

And iterating from k = 0, we get

L(k + 1) − L(0) ≤ −k∑

t=1

|E(t)|2 −k∑

t=1

|E(t − 1)|2 + dk (15)

k∑t=1

(|E(t)|2 + |E(t − 1)|2) ≤ dk − L(k + 1) + L(0) ≤ dk + L(0) (16)

From here we can see that the term d must be bounded by weight matrices andlearning parameters, in order to obtain

�L(k) ∈ L(∞) (17)

As a consequence

J (k) ∈ L(∞), B(k) ∈ L(∞), C(k) ∈ L(∞) (18)

�

2.3. Recursive Levenberg–Marquardt Algorithm of the RTNN Learning

The RTNN architecture is described in a state-space form and serves as a one-step ahead state predictor/estimator; therefore, it is suitable for identification andcontrol purposes.1,27 The recursive Levenberg–Marquardt algorithm,21,25,26 derived



for the RTNN topology could be considered as a continuation of the BP one. Thegeneral recursive L–M updating rule is described by the following equation:

W (k + 1) = W (k) + P (k)DY [W (k)]E[W (k)] (19)

where W (.) is the general updated weight matrix (J , B, C) of the RTNN model;P (.) can be interpreted as the covariance matrix of weights estimate W (.); DY[W (.)]is the Jacobian matrix, which is defined as the derivative of the RTNN output withrespect to the weight; finally, E[W (.)] is the error of approximation; k is the iterationnumber. The approximation error is given by

E[W (k)] = Yp(k) − Y (k) (20)

where Yp(.) and Y (.) are plant and RTNN outputs, respectively. The local gradientcomponents could be derived using the block-diagram of Figure 3, representing theadjoint RTNN but with different input, which is D = I .

Following the block-diagram of Figure 3, we could obtain the correspondingto (J , B, C) elements of the Jacobean matrix, derived as

DY [W (k)] = (∂/∂W )Y (k) = {DY [Cij (k)]; DY [Jij (k)], DY [Bij (k)]} (21)

DY [Cij (k)] = D1,i(k)Zj (k) (22)

D1,i(k) = �′[Yi(k)]Di(k) (23)

DY [Jij (k)] = D2,i(k)Xj (k) (24)

DY [Bij (k)] = D2,i(k)Uj (k) (25)

D2,i(k) = �′[Zi(k)]CiD1,i(k) (26)

The P (.) matrix is computed recursively by the following equation:

P (k) = α−1(k){P (k − 1) −P (k − 1)�[W (k)]S−1[W (k)]�T [W (k)]P (k − 1)}

(27)

Figure 3. Block-diagram of the adjoint RTNN, used for the L–M algorithm of learning.



where the S(.) and �(.) matrices are given as

S[W (k)] = α(k) (k) + �T [W (k)] P (k − 1)�[W (k)] (28)

�T [W (k)] =[

DYT [W (k)]0 · · · 1 · · · 0

];

(k)−1 =[

1 00 ρ

]; 10−4 ≤ ρ ≤ 10−6;

(29)0.97 ≤ α(k) ≤ 1; 103 ≤ P (0) ≤ 106

The matrix �(.) has dimension (Nw × 2), where Nw is the number of weights. Thesecond row of �(.) has only one unity element (the others are zero). The position ofthat element is computed by

i = k mod (Nw) + 1; k > Nw (30)

Next, the RTNN topology together with both given above algorithms of learning isapplied for the wastewater treatment plant model identification and control.

3. ADAPTIVE NEURAL CONTROL SYSTEMS DESIGN

3.1. Direct Adaptive Neural Control

The block-diagram of the control system is given in Figure 4. The controlscheme contained three RTNNs. The RTNN-1 is a plant identifier, learned by theidentification error Ei = Yd − Y , which estimates the state vector. The RTNN-2and RTNN-3 are feedback and feedforward neural controllers, respectively, bothlearned by the control error Ec = R − Yd . The control vector is a sum of bothcontrol RTNN actions Uf b, Uff , outputs of the corresponding RTNN controllers.1

Figure 4. Block-diagram of the direct adaptive neural control system.



The control scheme contained also a low pass noise filter of the plant output. Let usto linearize the plant equations 10 and 11 and the activation functions of the RTNNcontrollers. So we could write the following z-transfer functions, representing theplant input to output (Wp), the filter input to output (W ∗), the RTNN identifier (inputto state P i), the feedback controller (estimated state to feedback control Q1), andthe feedforward controller (reference to feedforward control Q2), as follows:

WP (z) = CP (zI − Jp)−1Bp;W ∗(k) = C∗(zI − J ∗)−1B∗;P i(z) = (zI − J i)−1Bi

(31)

Q1(z) = Ccf b

(zI − J c

f b

)−1Bf b;

Q2(z) = Ccff

(zI − J c

ff

)−1Bff

(32)

Following the block-diagram of Figure 4, we could connect all z-transfer functions31 and 32 in one closed-loop system equation, given in the z-operational form:

Yp(z) = W ∗(z)Wp(z)[I + Q1(z)P i(z)

]−1Q2(z)R(z) + V3(z) (33)

where V3(.) is a generalized noise term, given as

V3(z) = W ∗(z)[Wp(z)V1(z) + V2(z)] (34)

The RTNN topology is controllable and observable, and both—the BP and the L–Malgorithms of learning—are convergent,26,27 so the identification and control errorstend to zero:

Ei(k) = Yp(k) − Y i(k) → 0; Ec(k) = R(k) − Yp(k) → 0; k → ∞ (35)

This means that each transfer function given by Equations 31 and 32 is stable withminimum phase. From (33), it is seen that the dynamics of the stable low pass filteris independent of the dynamics of the plant and it does not affect the stability ofthe closed-loop system. The closed-loop system is stable, and the RTNN-2 feed-back controller compensates the combined plant-plus-filter dynamics. The RTNN-3feedforward controller dynamics is an inverse dynamics of the closed-loop system,which assure a precise reference tracking in spite of the presence of process andmeasurement noises.

3.2. Indirect Adaptive Sliding Mode Control Systems Design

The block diagram of this control is given in Figure 5. It contained a RTNNidentifier and an indirect adaptive linear controller. Here the indirect adaptive controlis viewed as a sliding mode control (SMC) one, designed using the parameters andstates issued by the RTNN identifier. Let us suppose that the studied nonlinear plant



Figure 5. Block-diagram of the indirect adaptive neural control system.

possess the structure, given by Equations 10 and 11, where L = M is supposed. Thelinearization of the activation functions of the learned identification RTNN model,which approximates the plant (see Equations 1–3), leads to the following linear localplant model:

X(k + 1) = JX(k) + BU(k) (36)

Y (k) = CX(k) (37)

Let us define the following sliding surface with respect to the output tracking error:

S(k + 1) = E(k + 1) +P∑

i=1

γiE(k − i + 1); |γi | < 1 (38)

where S(.) is the sliding surface error function, E(.) is the systems output trackingerror, γi are parameters of the desired error function, and P is the order of the errorfunction. The additional inequality in (38) is a stability condition, required for thesliding surface error function. The tracking error is defined as

E(k) = R(k) − Y (k) (39)

where R(k) is a L-dimensional reference vector and Y (k) is an output vector withthe same dimension. The objective of the sliding mode control systems design is tofind a control action that maintains the systems error on the sliding surface, whichassure that the output tracking error reaches zero in P steps, where P < N . So, thecontrol objective is fulfilled if

S(k + 1) = 0 (40)

The iteration of the error (39) gives

E(k + 1) = R(k + 1) − Y (k + 1) (41)



Now, let us to iterate (37) and to substitute (36) in it so to obtain the input/outputlocal plant model, which yields

Y (k + 1) = CX(k + 1) = C[JX(k) + BU(k)] (42)

From (38), (40), and (41), it is easy to obtain

R(k + 1) − Y (k + 1) +P∑

i=1

γiE(k − i + 1) = 0 (43)

The substitution of (42) in (43) gives

R(k + 1) − CJX(k) − CBU(k) +P∑

i=1

γiE(k − i + 1) = 0 (44)

As the local approximation plant model (36), (37), is controllable, observable, andstable (see Refs. 1, 18, 27), the matrix J is diagonal, and L = M , the matrix product(CB) is nonsingular, and the plant states X(k) are smooth nonincreasing functions.Now, from (44) it is possible to obtain the equivalent control capable of to lead thesystem to the sliding surface, which yields

Ueq(k) = (CB)−1

[−CJX(k) + R(k + 1) +

P∑i=1

γiE(k − i + 1)

](45)

Following Ref. 30 the SMC avoiding chattering is taken using a saturation functioninside a bounded control level Uo, taking into account plant uncertainties. So, theSMC takes the form

S(k + 1) = 0U(k) ={Ueq(k), if ‖Ueq(k)‖ < U0

−U0Ueq(k)/‖Ueq(k)‖, if ‖Ueq(k)‖ ≥ U0(46)

The proposed SMC copes with the characteristics of the wide class of plant modelreduction neural control with the reference model, defined in Ref. 8 and representsan indirect adaptive neural control, given in Ref. 1.

4. DESCRIPTION OF THE BIOLOGICAL WASTEWATERTREATMENT BIOPROCESS

A sketch of the biological wastewater treatment bioprocess system is givenin Figure 6. It contained a bioreactor and a settler with a recirculation feedback.Wastewater treatment28,29 is performed in an aeration tank (bioreactor), in whichthe contaminated water is mixed with biomass in suspension (activated sludge), andthe biodegradation process is then triggered in the presence of oxygen. The tank



Figure 6. Sketch of the wastewater treatment bioprocess plant.

is equipped with a surface aeration turbine, which supplies oxygen to the biomass,and additionally changes its suspension into a homogeneous mass. After sometime, the biomass mixture and the remaining substrate go to a separating chamber,where the biologic flocks (biologic sludge) are separated from the treated effluent.The treated effluent is then led to a host environment. The aim is good settlingof the biomass in the settler and high conversion of the entering organic materialin the bioreactor. The main objective of the control system is to keep the recyclebiomass concentration close to the reference signal, and this should be achieved inthe presence of disturbances and measurement noise acting on the recycle flow rate.A complete description of the plant is previously presented in Refs. 1 and 28. Thesensor dynamics is given by the following equation:

TmmX(t) = −Xm(t) + XR(t) + n(t) (47)

The main equations of the plant used in the simulations are given by

XR(t) =(

q(t)

q(t)+ μ(t, S(t)) − Fin(t)

V− cd + q(t) − 1

V

)XR(t) (48)

S(t) = − 1

Y (t)μ(t, S(t))

1

q(t)XR(t) + Fin(t)

VSin − Fin(t) + FR(t)

VS(t) (49)

where the state variables are X(t), biomass concentration; XR(t), the dynamicsof the concentration of the biomass in the settler; S(t), the substrate measured bythe chemical oxygen demand (COD); V is the reactor volume; FR represents therecycle flow rate (manipulated variable), Fin is the influent flow rate; Sin is theinfluent substrate concentration (potential disturbance, also expressed as COD), andY > 0 is the yield coefficient. Here cdXR denotes the decay rate of the biomassconcentration (which is added in the model to simulate biomass mortality), withcd > 0 as the decay rate parameter. The variable μ(·) denotes the specific growthrate, and it is modeled by the following Monod-type equation:

μ(S(t)) = μm(t)S(t)

Km(t) + S(t)(50)

where μm(·) is the maximum growth rate and Km(·) is the half-saturation constantof biodegradable organic matter. The time-varying parameters are given by the



following equations:

μm(t) = 0.2 + 0.1 sin(2πt/3 + 4π/3) (51)

Km(t) = 90 + 30 sin(πt/2) (52)

Y (t) = 0.6 + 0.1 sin(πt/3 + π/3) (53)

q(t) = 4 + sin(πt/6) (54)

cd (t) = 10−4(25 + 5 · sin(πt/12)) (55)

where the parameter q(t) is considered as continuously differentiable and boundedfunction with bounded inverse, bounded derivative, and q(t) > 1 for all t ≥ 0. Theplant output scaling equation is given by

yp = (Xm − 11400)/5700 (56)

where Xm is the real measured output. The scaled reference signal equation

r(k) = 0.5 sin

(πk

12

)(57)

And finally, the scaled plant input equation is as follows:

FR = (0.5U − yp)1.71 × 108 (58)

where yp is the scaled output of the bioreactor. Note that the recycle flow rate FR isa function of the control variable U , computed by the feedback control with respectto the estimated state and the feedforward control with respect to the scaled plantreference r(k).

5. SIMULATION AND EXPERIMENTAL RESULTS

5.1. Simulation Results, Obtained with the Direct Adaptive Neural Control

All simulations are performed using the following set of equations: the processdescription (47), (48), (49); the Monod-type equation (50); the time variable plantparameters (51)–(55); the plant output scaling equation (56); the scaled referencesignal equation (57); and the scaled plant input equation (59). In some simulations,a 10% process and measurements noises (both with variance SEED = 1200), areadded. In other simulations, 20 runs of the control program are performed withdifferent variance of the added noises and some statistics such as mean and varianceMSE% values are computed and given in Table III. The process is simulated over aperiod of 40 h, which gives an idea about its periodic behavior (a typical period isabout 24 h) and the period of discretization is set to T0 = 0.01 h (it is 1 h of the processtime). The parameters of RTNN learning are α = 0.01, η = 0.95. The identificationRTNN has topology (1, 2, 1). The activation functions of the hidden and output



Figure 7. Comparative graphical results obtained from the direct adaptive neural control (usingthe L–M RTNN learning) with and without plant output filter: (a) Comparison of the output ofthe plant and the reference signal using filter, (b) same as (a) but without filter, (c) comparison ofthe output of the plant and the outputs of the identification RTNNs using filter, (d) same as (c) butwithout filter, (e) combined control signal using filter, (f) same as (e) but without filter, (g) statesof the RTNN of identification using filter, (h) same as (g) but without filter, (i) mean squared errorof control (MSE) using filter, and (j) same as (i) but without filter.



network layers are hyperbolic tangents. The graphical simulation results, obtainedfrom the direct adaptive neural control (L–M RTNN learning) with and withoutfilter, are given in Figures 7a–7j. The results during the first and the last instantsof the simulation using Levenberg–Marquardt and backpropagation algorithms aregiven in Figures 8a–8h. The results show a good convergence of the system outputto the desired trajectory after approximately 2.1 h and a good filtration of the noisethat makes a MSE% reduction to about 1.3%.

Figure 8. Comparative graphical results of the first and the last instants of the simulationobtained from the direct adaptive neural control using the backpropagation algorithm and theLevenberg–Marquardt algorithm with and without filter. (a) and (c): The first and the last instantsof simulation using the backpropagation algorithm and filter. (e) and (g): The first and the lastinstants of simulation using the backpropagation algorithm without filter. (b) and (d): The firstand the last instants of simulation using the Levenberg–Marquardt algorithm and filter. (f) and(h): The first and the last instants of simulation using the Levenberg–Marquardt algorithm withoutfilter.



Figure 9. Comparative graphical results obtained from the indirect adaptive sliding mode control(using the L–M RTNN learning) with and without plant output filter: (a) comparison of the outputof the plant and the reference signal using filter, (b) same as (a) but without filter, (c) comparisonof the output of the plant and the outputs of the identification RTNNs using filter, (d) same as(c) but without filter, (e) combined control signal using filter, (f) same as (e) but without filter,(g) states of the RTNN of identification using filter, (h) same as (g) but without filter, (i) meansquared error of control (MSE) using filter, and (j) same as (i) but without filter.



5.2. Simulation Results, Obtained with the IndirectAdaptive Neural Control (SMC)

The same initial conditions and equations of the plant are used in those simu-lations. In all of them, the severe realistic conditions such as measurement noise aretaken into account by generating a stochastic signal added to the process input andoutput—the same as in the previous simulation experiment. The simulation dura-tion, the reference signal, the period of discretization, and the learning parameters

Figure 10. Comparative graphical results of the first and the last instants of the simulation ob-tained from the indirect adaptive sliding mode control using the backpropagation and Levenberg–Marquardt algorithms with and without filter. (a) and (c): The first and the last instants of simulationusing the backpropagation algorithm and filter. (e) and (g): The first and the last instants of simu-lation using the backpropagation algorithm without filter. (b) and (d): The first and the last instantsof simulation using the Levenberg–Marquardt algorithm and filter. (f) and (h): The first and thelast instants of simulation using the Levenberg–Marquardt algorithm without filter.



Table I. Final means squared error (%) of direct and indirect control (ξav) for 20 runs of thecontrol program (with filter) using Levenberg–Marquardt algorithm.

Direct No. 1 2 3 4 5 6 7 8 9 10control MSE 1.231 1.344 1.291 1.301 1.237 1.287 1.349 1.253 1.304 1.262design No. 11 12 13 14 15 16 17 18 19 20using L–M MSE 1.300 1.311 1.263 1.266 1.304 1.334 1.259 1.268 1.295 1.290

Indirect No. 1 2 3 4 5 6 7 8 9 10control MSE 1.370 1.319 1.407 1.261 1.340 1.297 1.390 1.305 1.361 1.258design No. 11 12 13 14 15 16 17 18 19 20using L–M MSE 1.308 1.331 1.321 1.342 1.351 1.268 1.397 1.373 1.379 1.383

are the same as given earlier. The identification RTNN has topology (1, 2, 1). Theactivation functions of the hidden and output network layers are hyperbolic tangents.The results, obtained from the indirect sliding mode control with and without thenoise rejecting filter (L–M RTNN learning), are given in Figures 9a–9j. The resultsduring the first and the last instants of the simulation using Levenberg–Marquardtand backpropagation algorithms are given in Figures 10a–10h. The results show agood convergence of the system output to the desired trajectory after approximately1.5 h and a good filtration of the noise that makes a MSE% reduction to about 1.5%.

Finally, the behavior of the control system in the presence of 10% whiteGaussian noise added to the plant output and input has been studied accumulatingsome statistics of the final MSE% (ξav) for multiple run of the control program (withdifferent variance of the noise for each run), which results are given in Tables Iand II for 20 runs of the direct and indirect control programs (incorporating filter)using the BP and L–M learning and depicted graphically in Figures 11a–11d. Themean average cost for all runs (ε) of control, the standard deviation (σ ) with re-spect to the mean value, and the deviation (�) are computed using the followingformulae:

ε = 1

n

n

�k=1

ξavk; σ =

√1

n

n

�i=1

�2i ; � = ξav − ε (59)

where k is the run number and n is equal to 20. Tables III presents the comparativeresults of the mean average MSE% value ε and the standard deviations σ for the

Table II. Final means squared error (%) of direct and indirect control (ξav) for 20 runs of thecontrol program (with filter) using Backpropagation algorithm.

Direct No. 1 2 3 4 5 6 7 8 9 10control MSE 1.538 1.517 1.401 1.358 1.351 1.464 1.498 1.511 1.477 1.400design No. 11 12 13 14 15 16 17 18 19 20using BP MSE 1.378 1.480 1.539 1.513 1.536 1.412 1.403 1.457 1.382 1.392

Indirect No. 1 2 3 4 5 6 7 8 9 10control MSE 1.455 1.608 1.649 1.472 1.574 1.476 1.512 1.477 1.494 1.529design No. 11 12 13 14 15 16 17 18 19 20using BP MSE 1.477 1.498 1.635 1.528 1.552 1.468 1.454 1.481 1.618 1.625



Figure 11. MSE of control (with filter) obtained during 20 run program simulations: (a) Usingthe Levenberg–Marquardt algorithm for a direct control; (b) using the Levenberg–Marquardtalgorithm for an indirect control; (c) using the backpropagation algorithm for a direct control;(d) using the backpropagation algorithm for an indirect control.

Table III. Mean average cost for all runs (ε) of control and the standard deviation (σ ) withrespect to the mean value using Levenberg–Marquardt and backpropagation algorithms for directand indirect control (using filter).

Algorithm Method Mean average cost (ε) Standard deviation (σ )

Levenberg–Marquardt Direct 1.2879 0.0320Indirect 1.3386 0.0443

Backpropagation Direct 1.4508 0.0630Indirect 1.5295 0.0644

20 runs of the program control over the four experiments. Results given in Table IIIshow that the state and parameter estimations using the L–M learning algorithmgave better results with respect to the results obtained using the BP learning, andthe control results applying the direct control scheme are superior to with respect tothe indirect one.

6. CONCLUSIONS

In this paper, a RTNN model and a dynamic Levenberg–Marquardt learningalgorithm are proposed and applied to real-time identification and state estimation ofa nonlinear wastewater treatment bioprocess plant. The proposed RTNN model hasa Jordan canonical structure, which permits to use the generated vector of estimatedstates directly for process control. The obtained parameters and states are used todesign a feedback direct and indirect (SMC) adaptive control laws. Both, the directand indirect controls with noise filter are able to force the system to track a time-varying process-dependent reference signal in noisy plant conditions. It performs



very well under restrictive conditions of periodically acting disturbances, parameteruncertainties, and inevitable sensor dynamics. The comparative simulation results,obtained with a continuous wastewater treatment bioprocess plant model, confirmthe applicability of the proposed identification and control methodology using boththe BP and L–M RTNN learning, giving priority to the L–M learning.

Acknowledgments

Doctoral student Carlos R. Mariaca G. is thankful to CONACyT, Mexico for the fellowshipreceived during his study in the DCA, CINVESTAV-IPN, Mexico.

References

1. Baruch IS, Georgieva P, Barrera-Cortes J, Feyo de Azevedo S. Adaptive recurrent neuralnetwork control of biological wastewater treatment. Int J Intell Syst 2005;20(2):173–194.Special issue on soft computing for modeling, simulation and control of nonlinear dynamicalsystems, O. Castillo and P. Melin, guest editors.

2. Boger ZI. Application of neural networks to water and wastewater treatment plant operation.ISA Trans 1992;31(1):25–33.

3. Boskovic JD, Narendra KS. Comparison of linear, nonlinear and neural-network-based adap-tive controllers for a class of fed-batch fermentation processes. Automatica 1995;31:817–840.

4. Geeraerd AH, Herremans CH, Cenens C, Van Impe JF. Application of artificial neuralnetworks as a nonlinear modular modelling technique to describe bacterial growth in chilledfood products. Int J Food Microbiol 1998;44:49–68.

5. Kriger C, Tzoneva R. Neural networks for prediction of wastewater treatment plant influentdisturbances. In: Proc of the AFRICON, 2007; pp 1–7.

6. Liu C, Qiao J, Zhang F. The control of wastewater treatment process based on fuzzy neuralnetwork. In: Proc of the Sixth World Congress on Intelligent Control and Automation,WCICA, 2006. Vol 2, pp 9347–9351.

7. Miler WT III, Sutton RS, Werbos PJ. Neural networks for control. London: MIT Press;1992.

8. Narendra KS, Parthasarathy K. Identification and control of dynamic systems using neuralnetworks. IEEE Trans Neural Networks 1990;1(1):4–27.

9. Su Hong-Te, McAvoy ThJ, Werbos P. Long-term predictions of chemical processes usingrecurrent neural networks: a parallel training approach. Ind Eng Chem Res 1992;31:1338–1352.

10. Tabatabaei TS, Farooq T, Guergachi A. Support vector machines based approach for chem-ical phosphorus removal process in wastewater treatment plant. In: Proc of the CanadianConference on Electrical and Computer Engineering, 2006. pp 318–322.

11. Hagan M. Neural network design. Boston, MA: PWS Publishing; 1996.12. Haykin S. Neural networks, a comprehensive foundation. New York: Macmillan College;

1994.13. Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop PJ. Neural network for control systems—a

survey. Automatica 1992;28(6):1083–1112.14. Pao SA, Phillips SM, Sobajic DJ. Neural net computing and intelligent control systems. Int

J Control 1992;56:263–289.15. Chen S, Billings SA. Neural networks for nonlinear dynamics system modeling and identi-

fication. Int J Control 1992;56:319–346.16. Jin L, Gupta M. Stable dynamic backpropagation learning in recurrent neural networks.

IEEE Trans Neural Networks 1999;10:1321–1334.



17. Melin P, Castillo O. Hybrid intelligent systems for pattern recognition. Heidelberg, Germany:Springer-Verlag; 2005.

18. Nava FR, Baruch IS, Poznyak A, Nenkova B. Stability proofs of advanced recurrent neuralnetworks topology and learning. Comptes Rendus (Proceedings of the Bulgarian Academyof Sciences) 2004;57(1):27–32.

19. Wan E, Beaufays F. Diagrammatic method for deriving and relating temporal neural networksalgorithms. Neural Comput 1996;8:182–201.

20. Ampazis BN, Perantonis SJ. Two highly efficient second-order algorithms for trainingfeedforward networks. IEEE Trans Neural Networks 2002;13(5):1064–1074.

21. Asirvadam VS, McLoone SF, Irwing GW. Parallel and separable recursive Levenberg–Marquardt training algorithm. In: Proc of 12th IEEE Workshop on Neural Networks forSignal Processing, 2002. pp 129–138.

22. Chen AT, Han D, Au FTK, Tham LG. Acceleration of Levenberg–Marquardt training ofneural networks with variable decay rate. In: Proc of the Int Joint Conf on Neural Networks,Vol 3, 2003. pp 1873–1878.

23. Hagan M, Menhaj M. Training feedforward networks with the Marquardt algorithm. IEEETrans Neural Networks 1994;5(6).

24. Lourakis MLA, Argyros AA. Is Levenberg–Marquardt the most efficient optimization algo-rithm for implementing bundle adjustment? In: Proc of the 10th IEEE Int Conf on ComputerVision, 2005. pp 1526–1531.

25. Ngia LS, Sjoberg J. Efficient training of neural nets for nonlinear adaptive filtering using arecursive Levenberg–Marquardt algorithm. IEEE Trans Signal Process 2000;48:1915–1927.

26. Ngia LS, Sjoberg J, Viberg M. Adaptive neural nets filter using a recursive Levenberg–Marquardt search direction. IEEE Signals Syst Comp 1998;1:697–701.

27. Baruch IS, Flores JM, Nava F, Ramirez IR, Nenkova B. An advanced neural networktopology and learning, applied for identification and control of a D.C. motor. In: Proc of theFirst Int IEEE Symp on Intelligent Systems. Sgurev V, Jotsov V, editors. Varna, Bulgaria;2002; pp 289–295.

28. Georgieva P, Ilchmann A. Adaptive λ-tracking control of activated sludge processes. Int JControl 2001;74(12):1247–1259.

29. Sundstrom DW, Klei HE. Wastewater treatment. Englewood Cliffs, NJ: Prentice-Hall, 1979.30. Young KD, Utkin VI, Ozguner U. A control engineer’s guide to sliding mode control. IEEE

Trans Control Syst Technol 1999;7(3):328–342.


a levenberg-marquardt learning applied for recurrent ... · of the polymerization process in a...

Documents