a reliable multi-objective control strategy for batch processes based on bootstrap aggregated neural...

Available online at www.sciencedirect.com

www.elsevier.com/locate/jprocont

Journal of Process Control 18 (2008) 720–734

A reliable multi-objective control strategy for batch processes basedon bootstrap aggregated neural network models

Ankur Mukherjee, Jie Zhang *

School of Chemical Engineering and Advanced Materials, Newcastle University, Newcastle upon Tyne NE1 7RU, UK

Received 19 March 2007; received in revised form 19 August 2007; accepted 16 November 2007

Abstract

This paper presents a reliable multi-objective optimal control method for batch processes based on bootstrap aggregated neural net-works. In order to overcome the difficulty in developing detailed mechanistic models, bootstrap aggregated neural networks are used tomodel batch processes. Apart from being able to offer enhanced model prediction accuracy, bootstrap aggregated neural networks canalso provide prediction confidence bounds indicating the reliability of the corresponding model predictions. In addition to the processoperation objectives, the reliability of model prediction is incorporated in multi-objective optimisation in order to improve the reliabilityof the obtained optimal control policy. The standard error of the individual neural network predictions is taken as the indication ofmodel prediction reliability. The additional objective of enhancing model prediction reliability forces the calculated optimal control pol-icies to be within the regions where the model predictions are reliable. By such a means, the resulting control policies are reliable. Theproposed method is demonstrated on a simulated fed-batch reactor and a simulated batch polymerisation process. It is shown that byincorporating model prediction reliability in the optimisation criteria, reliable control policy is obtained.� 2007 Elsevier Ltd. All rights reserved.

Keywords: Batch processes; Multi-objective optimisation; Neural networks; Model reliability

1. Introduction

Batch or semi-batch processes are suitable for theresponsive manufacturing of high value added products[1]. In the operation of batch processes, it is desirable tomeet a number of objectives, which are usually conflictingto each other. The relative importance of the individualobjectives usually changes with market conditions. Tomaximise the profit from batch process manufacturing,multi-objective optimal control should be applied to batchprocesses.

The performance of multi-objective optimal controldepends on the accuracy of the process model. Developingdetailed mechanistic models is usually very time consumingand may not be feasible for agile responsive manufactur-ing. Data based empirical models, such as neural network

0959-1524/$ - see front matter � 2007 Elsevier Ltd. All rights reserved.

doi:10.1016/j.jprocont.2007.11.008

* Corresponding author. Tel.: +44 191 2227240; fax: +44 191 2225292.E-mail address: [email protected] (J. Zhang).

models [2] and nonlinear partial least square models[3–5], and hybrid models [6] have to be utilised. Stackedneural networks, also known as aggregated neural net-works, have been shown to possess better generalisationcapability than single neural networks [7,8] and are usedin this paper to model batch processes. An additional fea-ture of stacked neural networks is that they can also pro-vide prediction confidence bounds indicating thereliability of the corresponding model predictions [9].Due to model-plant mismatches, the ‘‘optimal” controlpolicy calculated from a neural network model may notbe optimal when applied to the actual process [10]. Thusit is import that the calculated optimal control policyshould be reliable. Zhang [10] proposes a reliable optimalcontrol approach for batch processes through incorporat-ing model prediction confidence into the optimisationobjective function in a single objective optimisation frame-work. However, single objective optimisation may not beefficient in handling multiple process operating objectives.

mailto:[email protected]

X Y

Fig. 1. A bootstrap aggregated neural network.

A. Mukherjee, J. Zhang / Journal of Process Control 18 (2008) 720–734 721

This paper presents a multi-objective optimal controlmethod for batch processes based on bootstrap aggregatedneural network models. In addition to the process opera-tion objectives, the reliability of model prediction is incor-porated in multi-objective optimisation in order to improvethe reliability of the obtained optimal control policy. Thestandard error of the individual neural network predictionsis taken as the indication of model prediction reliability.The goal attainment method [11] is used to solve themulti-objective optimisation problem where, in additionto the process operation objectives, minimising the stan-dard error of the individual neural network predictions istaken as the additional optimisation objectives.

The proposed method is demonstrated on a simulatedfed-batch reactor and a simulated batch polymerisationprocess. It is shown that by incorporating model predictionreliability in the optimisation criteria, reliable control pol-icy is obtained even under unknown process variations.

The paper is organised as follows. Section 2 brieflyintroduces bootstrap aggregated neural networks. Amulti-objective optimal control strategy incorporatingmodel prediction confidence bounds as extra objectives ispresented in Section 3. Applications of the proposed tech-nique to a fed-batch reactor and a batch polymerisationprocess are given in Sections 4 and 5 respectively. Section6 contains some concluding remarks.

2. Bootstrap aggregated neural networks

A limitation of neural network models is that they canlack generalisation when applied to unseen data. Severaltechniques have been developed to improve neural net-work generalisation capability, such as regularisation[12], early stopping [13], Bayesian learning [14], trainingwith both dynamic and static process data [15], and com-bining multiple networks [2,7,8,16]. Among these tech-niques, combining multiple networks is a very promisingapproach to improving model predictions on unseen data.The emphasis of this approach is on the generalisationaccuracy on future predictions (i.e. predictions on unseendata). When building neural network models, it is quitepossible that different networks perform well in differentregions of the input space. By combining multiple neuralnetworks, prediction accuracy on the entire input spacecould be improved.

A diagram of bootstrap aggregated neural networks isshown in Fig. 1, where several neural network models aredeveloped to model the same relationship. Instead of select-ing a ‘‘best” single neural network model, these individualneural networks are combined together to improve modelaccuracy and robustness. The overall output of the aggre-gated neural network is a weighted combination of theindividual neural network outputs. This can be representedby the following equation:

f ðX Þ ¼Xn

i¼1

wifiðX Þ ð1Þ

where f(X) is the aggregated neural network predictor, fi(X)is the ith neural network, wi is the aggregating weight forcombining the ith neural network, n is the number of neu-ral networks, and X is a vector of neural network inputs.Proper determination of the stacking weights is essentialfor good modelling performance. Since the individual neu-ral networks are highly correlated, appropriate stackingweights could be obtained through principal componentregression [8].

The aggregated neural network shown in Fig. 1 can beunderstood as a bigger neural network. However, the aggre-gated neural network is not trained in one go as a bigger sin-gle network. Each individual network is trained on abootstrap re-sampling replication of the original trainingdata. The trained individual networks are then combined,e.g. using principal component regression. If this bigger net-work is trained in one go as a single neural network on theoriginal training data, then the trained neural network maynot give desired model representation due to over-fitting orunder-fitting which may occur during the training process.This problem can be overcome to certain extent by trainingthe individual networks separately on bootstrap re-sam-pling replications of the original training data.

Another advantage of bootstrap aggregated neural net-work is that model prediction confidence bounds can becalculated from individual network predictions [9]. Thestandard error of the predicted value is estimated as

re ¼1

n� 1

Xn

i¼1

½fiðX Þ � yðX Þ�2( )1=2

ð2Þ

where yðX Þ ¼Pn

i¼1fiðX Þ=n. Assuming that the individualnetwork prediction errors are normally distributed, the95% prediction confidence bounds can be calculated asy(xi; �) ± 1.96re. A narrower confidence bound, i.e. smallerre, indicates that the associated model prediction is morereliable.

3. Reliable multi-objective optimal control

The proposed reliable multi-objective optimal controlstrategy is shown in Fig. 2. A bootstrap aggregated neural

BatchProcess

NN Model

Multi-objectiveoptimiser

GoalU

Disturbances

Y(tf)

)(ˆftY

σe(tf)

Fig. 2. Reliable multi-objective optimal control.

722 A. Mukherjee, J. Zhang / Journal of Process Control 18 (2008) 720–734

network is developed from process operational data and itpredicts the end of batch product quality variables from thecontrol policy for the batch. The bootstrap aggregated neu-ral network model can be represented as follows:

Y ðtfÞ ¼ f ðUÞ ð3Þ

In Fig. 2 and Eq. (3), Y(tf) is a vector of product qualityvariables at the end of a batch, tf is the batch end time,U = [u1, u2, . . . ,uN] is a vector of control actions during abatch (i.e. the control policy), Y ðtfÞ is a vector of predictedproduct quality variables at the end of a batch, and f(�) is anonlinear function represented by a bootstrap aggregatedneural network. In addition to predicting the final productquality variables, the bootstrap aggregated neural networkcan also predict the standard errors of model predictions asgiven in Eq. (2), which can also be represented as a func-tion of the control policy

reðtfÞ ¼ gðUÞ ð4Þ

where re(tf) is standard errors in predicting Y(tf) and g(�) isa nonlinear function determined by Eqs. (2) and (3).

In addition to the process operation objectives, minimis-ing the standard errors of model predictions which is equiv-alent to minimising the model prediction confidence bound(i.e. maximising the model prediction reliability) is incorpo-rated as the additional control objective. This is illustratedusing the goal attainment multi-objective optimisationmethod as follows:

F ðuÞ ¼h½Y ðtfÞ�reðtfÞ

� �ð5Þ

minU ;c

c

subject to F iðUÞ � W ic 6 F �i ð6ÞEqs:ð3Þandð4ÞuL 6 uj 6 uU j ¼ 1; 2; . . . ;N

where h[Y(tf)] is a vector of the process operational objec-tives, c is a scalar variable, Wi is the weighting parameterfor the ith objective, F �i is the desired goal value for theith objective, and U=[u1, u2, ... ,uN] is the control policy.

The goal attainment multi-objective optimisationmethod [11] involves expressing a set of design goals,

F � ¼ fF �1; F �2; . . . ; F �mg, which is associated with a set ofobjectives, F(x) = {F1(x), F2(x) ... ,Fm(x)}, where m is thenumber of objectives. The problem formulation allowsthe objectives to be under- or over-achieved enabling thedesigner to be relatively imprecise about the initial designgoals. The relative degree of under- or over-achievementof the goals is controlled by a vector of weighting coeffi-cients, W = {W1, W2 ... , Wm}. The term Wic introducesan element of slackness into the problem and implies thatthe goals have not to be rigidly met. The weighting vector,W, enables the designer to express a measure of the relativetrade-offs between the objectives. Hard constraints can beincorporated into the design by setting a particular weight-ing factor to zero (i.e., Wi = 0).

Predictions of the product quality variables form thebootstrap aggregated neural network under the controlpolicy calculated from the above multi-objective optimisa-tion are reliable because of the narrow confidence bounds.The formulation of the optimal control strategy impliesthat the control policy will try to stay in the region wherethe model is reliable. As a consequence, the obtained con-trol policy is reliable.

4. Application to a fed-batch reactor

4.1. A fed-batch reactor

The fed-batch reactor is taken from [17]. The followingreaction system:

Aþ B!k1 C

Bþ B!k2 D

is conducted in an isothermal semi-batch reactor. Theobjective in operating this reactor is, through addition ofreactant B, to convert as much as possible of reactant A

to the desired product, C, in a specified time tf = 120 min.It would not be optimal to add all B initially as the secondorder side-reaction yielding the undesired species D will befavoured at high concentration of B. To keep this unde-sired species low, the reactor is operated in semi-batchmode where B is added in a feed stream with concentrationbfeed = 0.2 moles/litre. Based on the reaction kinetics andmaterial balances in the reactor, the following mechanisticmodel can be developed

d½A�dt¼ �k1½A�½B� �

½A�V

u ð7Þ

d½B�dt¼ �k1½A�½B� � 2k2½B�2 þ

bfeed � ½B�V

u ð8Þ

d½C�dt¼ k1½A�½B� �

½C�V

u ð9Þ

d½D�dt¼ 2k2½B�2 �

½D�V

u ð10Þ

dVdt¼ u ð11Þ


In the above equations, [A], [B], [C], and [D] denote,respectively, the concentrations of A, B, C, and D, V isthe current reaction volume, u is the reactant feed rate,and the reaction rate constants have the nominal valuek1 = 0.5 and k2 = 0.5. At the start of reaction, the reactorcontains [A](0) = 0.2 moles/litre of A, no B ([B](0) = 0)and is fed to 50% (V(0) = 0.5).

4.2. Modelling of the fed-batch reactor using bootstrap

aggregated neural networks

In this study, a fixed batch time of 120 min is consideredas in [18]. Since it is usually difficult to measure the productquality variables frequently during a batch, it is a generalpractice to measure the product quality variables only atthe end of a batch. The batch duration is divided into 10equal intervals and within each interval the reactant feedrate is kept constant as in [18]. Dividing the batch durationinto more intervals will increase the degree of freedom inthe control policy with the potential of improved perfor-mance. However, this will increase the network training(due to increased network inputs) and batch optimisation(due to increased control actions within a batch) computa-tion effort. Dividing the batch duration into fewer intervalswill reduce the computation effort in network training andbatch optimisation. However, this may reduce the achiev-able control performance due to reduced degree of freedomin the control policy. The objective in operating this pro-cess is to maximise the amount of the final product[C](tf)V(tf) and simultaneously minimise the amount ofundesired species [D](tf)V(tf). Neural network model forthe prediction of [C](tf)V(tf) and [D](tf)V(tf) at the finalbatch time are of the form

y1 ¼ f1ðUÞ ð12Þy2 ¼ f2ðUÞ ð13Þ

0 5 10 15 200

2

4x 10

-3

MS

E (

Tra

inin

g)

CcV

0 5 10 15 200

2

4x 10

-3

MS

E (

Val

idat

ion)

0 5 10 15 200

2

4

6x 10

-3

MS

E (

Tes

ting)

Network number

Fig. 3. Model errors of individual ne

where y1 = [C](tf)V(tf), y2 = [D](tf)V(tf), U = [u1, u2,... ,u10]T is the reactant feed rate, f1 and f2 are nonlinearfunctions represented by neural networks.

In this study, 50 batches of simulated process opera-tional data were generated with the reactant feed rate ran-domly distributed in the range [0, 0.01]. Of the 50 batchesof data, 40 batches were used to develop neural networkmodels and the remaining 10 batches were used as unseentesting data.

Two bootstrap aggregated neural networks each con-taining 20 neural networks were developed for predicting[C](tf)V(tf) and [D](tf)V(tf) respectively. Each individualneural network has a single hidden layer with 10 hiddenneurons. Hidden neurons use the sigmoid activationfunction whereas the output layer neuron uses the linearactivation function. The Levenberg–Marquardt trainingalgorithm with ‘‘early stopping” was used in this study totrain the networks. For training each network, bootstrapre-sampling with replacement [19] was used to generate areplication of the 40 batches of process data. Half of thereplication was used as training data while the other halfwas used as the validation data. The validation data wasused in determining the number of hidden neurons andused in the ‘‘early stopping” mechanism. The training datafor the individual networks are different due to bootstrapre-sampling so as to ensure that different individual net-works are obtained. Because of the different magnitudesof the model input and output data, the data for the neuralnetwork training, validation and testing were pre-processedto be in the range [�1, 1].

Fig. 3 shows the mean squared errors (MSE) for theindividual networks on training, validation, and testingdata sets for the two neural net models. It can be observedfrom Fig. 3 that the individual neural network errors onthe training, validation and testing data sets are inconsis-tent in that networks giving small errors on the training

0 5 10 15 200

2

4

6x 10

-3

MS

E (

Tra

inin

g)

CdV

0 5 10 15 200

2

4

6x 10

-3

MS

E (

Val

idat

ion)

0 5 10 15 200

2

4

6x 10

-3

MS

E (

Tes

ting)

Network number

tworks for the fed-batch reactor.

Table 1Bootstrap aggregated neural network prediction accuracy for the fed-batch reactor

[C](tf)V(tf) [D](tf)V(tf)

Mean squaredtesting error

Standardprediction error

Mean squaredtesting error

Standardprediction error

0.0034 0.0020 0.0040 0.0028


data may not give small errors on the testing data. Table 1shows the MSE of the bootstrap aggregated neural net-work models and the standard error from the individualnetwork predictions on the testing data.

4.3. Multi-objective optimal control of the fed-batch reactor

The objective in operating the process is to maximise theamount of the final desired product [C](tf)V(tf) and simul-taneously minimise the amount of the final undesired spe-cies [D](tf)V(tf). In order to obtain reliable control policyfrom the aggregated neural network model, minimisationof the standard error of individual network predictionsare introduced as additional objectives in the optimisationproblem. This can be formulated as a multi-objective opti-misation problem which is solved using the goal attainmentmethod

F ðUÞ ¼

�CcðtfÞV ðtfÞCdðtfÞV ðtfÞre; CcðtfÞre; Cd

ðtfÞ

26664

37775 ð14Þ

minU ;c

c ð15Þ

subject to F iðUÞ � W ic 6 F �i0 6 uj 6 0:01 j ¼ 1; 2; . . . ; 10

V ðtf 6 1:00Þ

where c is a scalar variable, Wi is the weighting parameterfor the ith objective, F �i is the desired goal value for the ithobjective, U=[u1, u2, ... ,u10] is the sequence of the reactantfeed rates into the reactor, V is the reaction volume,re; CcðtfÞ and re; Cd

ðtfÞ denote the standard prediction errorsof the individual networks within the two bootstrap aggre-gated neural network models. The objective function F(U)maximises the amount of product, [C](tf)V(tf), minimisesthe amount of by-product, [D](tf)V(tf), and also minimisethe standard errors of the neural network model prediction(i.e. maximise the reliability of model predictions).

Two cases with the following goal values are consideredhere:

Case I: F* = [�0.065 0.015 0.001 0.001]T

Case II: F* = [�0.075 0.020 0.001 0.001]T

Case I emphasises on less by-product generationwhereas Case II stresses on producing more of the desirableproduct. The weights used in the goal attainment algorithmfor the maximisation of final amount of product and min-imisation of the final amount of by-product are taken as

being complementary to each other with the summationof them being equal to 0.8 (W1 = 0.8 �W2). The weightsfor the standard prediction errors for the final amountsof product and by-product are taken as W3 = 0.04 andW4 = 0.03 respectively. In order to demonstrate the advan-tage of the proposed technique, 50 solutions of the optimalcontrol problem were computed by varying the weights on[C](tf)V(tf) and [D](tf)V(tf), W1 and W2 respectively, ran-domly and uniformly within [0, 0.8].

Fig. 4 shows one of the 50 computed optimal controlprofiles for Case I and one of the 50 computed optimalcontrol profiles for Case II. The difference in the optimalcontrol trajectories can be seen from the plots, which isdue to the different objectives. The corresponding trajecto-ries for the process variables for the two investigated casesare shown in Figs. 5 and 6 respectively. The optimisationand simulation results for the same sample solution aregiven in Table 2. The values for the neural network modelpredicted and mechanistic model calculated (i.e. the actualprocess) values and other parameters are presented in thetable for the considered sample solution. The mechanisticmodel calculated output represents the end point qualityvalues when the calculated ‘‘optimal” control profile isapplied to the actual process (i.e. the simulation on themechanistic model). The relative error is defined as theabsolute difference between the neural network and mech-anistic model predictions divided by the mechanistic modelpredictions. The results of multi-objective optimisationwithout considering model prediction confidence are alsogiven for the purpose of comparison.

The results shown in Table 2 clearly signify the effect ofincorporation of the minimisation of standard predictionerrors in the multi-objective optimisation. Except for[C](tf)V(tf) in Case II, the proposed method results in muchless relative errors between the neural network model andthe mechanistic model. This indicates that the optimal con-trol policies calculated under the proposed method is reli-able in the sense that the performance on the actualprocess (mechanistic model simulation) is close to that pre-dicted by the neural network model. For Case I, though theneural network predicted value of [D](tf)V(tf) is much bet-ter and that of [C](tf)V(tf) is marginally worse if model pre-diction confidence bound is not included in the objectivefunction, the resulting control profile is not reliable andmay not provide ‘‘optimal” performance if it is applied tothe actual process. This is verified from the simulationresults on the mechanistic model (representing the actualprocess). The actual product quality variable values,[C](tf)V(tf) and [D](tf)V(tf), are overall better when thestandard prediction errors are minimised as part of theoptimisation objectives. For the study in Case II, thoughthe neural network predictions for [C](tf)V(tf) is betterand for [D](tf)V(tf) is only marginally worse if standardprediction errors are not included in the optimisationobjective function, the resulting optimal control profile isnot expected to be reliable. The simulation results on themechanistic model indicate that there is a large reduction

0 20 40 60 80 100 1200

0.1

0.2

Conce

ntr

atio

n

[A]

0 20 40 60 80 100 1200

0.05

0.1

Conce

ntratio

n

0 20 40 60 80 100 1200.4

0.6

0.8

1

Time (min)

Volu

me (m

3 )

[B][C][D]

Fig. 5. Process variable profiles under optimal control for the fed-batch reactor: Case I.

0 20 40 60 80 100 1200

0.002

0.004

0.006

0.008

0.01

Time (min)

U (

m3 /m

in)

Control Inputs - Case I

0 20 40 60 80 100 1200

0.002

0.004

0.006

0.008

0.01

Time (min)

Control Inputs - Case II

U (

m3 /m

in)

Fig. 4. Optimal control profile for the fed-batch reactor.


in [D](tf)V(tf) and the value of [C](tf)V(tf) is only marginallyreduced when the standard prediction errors is incorpo-rated in the optimisation.

Fig. 7 shows the relative errors of the bootstrap aggre-gated neural network model predictions under the two dif-ferent optimal control profiles: considering confidencebounds (s) and not considering confidence bounds (*). Itcan be seen from Fig. 7 that the neural network predictionsunder the control profiles calculated by considering themodel prediction confidence bounds are generally moreaccurate than those under the control profiles calculatedwithout considering the model prediction confidence

bounds. Thus, the optimisation results incorporating themodel prediction confidence bounds are more reliable thanthose without incorporating the model prediction confi-dence bounds.

Fig. 8 shows the Pareto solutions obtained for the opti-misation results with and without model prediction confi-dence bounds. The standard prediction errors of theindividual neural network models are also given in Fig. 8.It can be concluded from Fig. 8 that the resulting end pointquality variable values are better when the minimisation ofthe standard prediction errors is incorporated as an addi-tional objective.

0 20 40 60 80 100 1200

0.1

0.2

Co

ncen

trat

ion

[A]

0 20 40 60 80 100 1200

0.05

0.1

Con

cent

ratio

n

0 20 40 60 80 100 1200.5

1

1.5

Time (min)

[B][C][D]

Vo

lum

e (

m3 )

Fig. 6. Process variable profiles under optimal control for the fed-batch reactor: Case II.

Table 2Sample optimisation and simulation results in the fed-batch reactor case study

Cases Weights Neural network model Mechanistic model Relative errors

[C](tf)V(tf) [D](tf)V(tf) [C](tf)V(tf) [D](tf)V(tf) [C](tf)V(tf) (%) [D](tf)V(tf) (%)

I [0.416, 0.384, 0.04, 0.03]T 0.05974 0.02486 0.06023 0.02705 0.81 8.10[0.416, 0.384]T 0.05898 0.02067 0.06140 0.02832 3.94 27.01

II [0.416, 0.384, 0.01, 0.01]T 0.06082 0.02532 0.06193 0.02707 1.79 6.46[0.416, 0.384]T 0.06202 0.02639 0.06239 0.03034 0.59 13.02

0 10 20 30 40 500

20

40

60

80

100

Solution Number

Re

lativ

e e

rror

s (%

)

Case I - Cc(tf)*V(tf)

0 10 20 30 40 500

100

200

300

Solution Number

Case I - Cd(tf)*V(tf)

0 10 20 30 40 500

5

10

15

20

Solution Number

Case II - Cc(tf)*V(tf)

0 10 20 30 40 500

10

20

30

40

Solution Number

Case II - Cd(tf)*V(tf)

Re

lativ

e e

rror

s (%

)

Re

lativ

e e

rror

s (%

)R

ela

tive

err

ors

(%)

Fig. 7. Bootstrap aggregated neural network model prediction accuracy for the fed-batch reactor under the optimal control policies.


Fig. 9 illustrates the percentage improvement in thequality variables due to the incorporation of the minimisa-tion of the standard prediction errors in the multi-objectiveobjective optimisation. It can be seen from Fig. 9 that

improvements are obtained in most of the cases. The num-ber of cases with improvement (out of 50) is presented inTable 3. It can be seen from Table 3 that the proposed reli-able multi-objective control strategy results in improve-

-0.07 -0.06 -0.05 -0.04 -0.03 -0.020

0.01

0.02

0.03Case I

-Cc(tf)*V(tf)

Cd

(tf)

*V(t

f)

-0.065 -0.06 -0.055 -0.050.015

0.02

0.025

0.03

0.035

0.04Case II

-Cc(tf)*V(tf)

Cd

(tf)

*V(t

f)

2 3 4 5 6

x 10-3

0

0.002

0.004

0.006

0.008

0.01Case I

Standard Deviation in Cc(tf)V(tf)Sta

ndar

d D

evi

atio

n in

Cd(

tf)V

(tf)

1 2 3 4 5 6

x 10-3

0

2

4

6

8x 10

-3 Case II


ndar

d D

evi

atio

n in

Cd(

tf)V

(tf)

Fig. 8. Comparison of Pareto solutions for the fed-batch reactor (s: with confidence bound; *: without confidence bound).

20 0 20 40 60 80–300

–250

–200

–150

–100

–50

0

50

Percentage Improvement in Cc(tf)*V(tf)

Per

cent

age

Impr

ovem

ent i

n C

d(tf)

*V(t

f)

Case I

–5 0 5 10 15 20–70

–60

–50

–40

–30

–20

–10

0

10

20

Percentage Improvement in Cc(tf)*V(tf)

Per

cent

age

Impr

ovem

ent i

n C

d(tf)

*V(t

f)

Case II

Fig. 9. Improvement in solutions by using confidence bound as additional objectives in the fed-batch reactor case study.


ment in either [C](tf)V(tf) or [D](tf)V(tf) in most of thecases. No cases with no improvement in [C](tf)V(tf) andno improvement in [D](tf)V(tf) under Case I occur andthere are only 2 cases under Case II that the proposedmethod does not improve the performance.

In order to investigate the robustness of the proposedmethod to process variations, process variations wereintroduced by randomly varying the values of k1 and k2

assuming a normal random distribution with a standarddeviation of 0.05 and a mean value (nominal value) of

0.50. Fig. 10 shows the relative errors of the bootstrapaggregated neural networks under process variations. Itcan be seen that under the proposed multi-objectiveoptimal control strategy, the model prediction errors aresmall leading to reliable optimal control. Fig. 11 showsthe Pareto solutions obtained for the optimisation resultswith and without model prediction confidence boundsunder process variations. The standard deviations of theindividual network predictions are also shown. It can beseen from Fig. 11 that the solutions obtained by the

Table 3Improvement in solutions by incorporating confidence bounds as extraobjectives in the fed-batch reactor case study

Criteria considered Number ofcases

CaseI

CaseII

Improvement in [C](tf)V(tf) 38 8Improvement in [D](tf)V(tf) 12 40Improvement in [C](tf)V(tf) and [D](tf)V(tf) 0 0No improvement in [C](tf)V(tf) and no improvement in

[D](tf)V(tf)0 2


proposed multi-objective optimal control strategy areoverall better than those without considering model predic-tion confidence under random process variations. Table 4summarises the number of cases with improvement (outof 50) under random process variations. It can be seenfrom Table 4 that the proposed reliable multi-objectivecontrol strategy works well even under random processvariations.

5. Application to a batch polymerisation process

5.1. A batch polymerisation process

This example involves a thermally initiated bulk poly-merisation of styrene in a batch reactor. The differentialequations describing the polymerisation process are givenby Kwon and Evans through reaction mechanism analysisand laboratory testing [20]. Gattu and Zafiriou [21] reportthe parameter values of the first principle model. Donget al. [22] use it to demonstrate batch-to-batch optimisa-

0 10 20 30 40 500

20

40

60

Solution Number

Re

lativ

e e

rror

s (%

)

Case I - Cc(tf)*V(tf)

0 10 20 30 40 500

1

2

3

4

Solution Number

Case II - Cc(tf)*V(tf)

Re

lativ

e e

rror

s (%

)

Fig. 10. Relative errors of bootstrap aggregated neural network models un* – without model prediction confidence bounds).

tion. The differential equations for this process are givenbelow:

dx1

dt¼ f1 ¼

q20q

Mmð1� x1Þ2 expð2x1 þ 2vx2

1ÞAm exp � Em

uT ref

� �

ð16Þ

dx2

dt¼ f2 ¼

f1x2

1þ x1

1� 1400x2

Aw expðB=uT refÞ

� �ð17Þ

dx3

dt¼ f3 ¼

f1

1þ x1

Aw expðB=uT refÞ1500

� x3

� �ð18Þ

With

q ¼ 1� x1

r1 þ r2T c

þ x1

r3 þ r4T c

; q0 ¼ r1 þ r2T c;

T c ¼ uT ref � 273:15 ð19Þ

where x1 is the conversion, x2 = xn/xnf and x3 = xw/xwf

are, respectively, the dimensionless number-average andweight–average chain lengths (NACL and WACL),u = T/Tref is the control variable, T is the absolute temper-ature of the reactor and Tc is the temperature in degree Cel-sius, Aw and B are coefficients in the relation betweenWACL and temperature obtained from experiments, Am

and Em are, respectively, the frequency factor and activa-tion energy of the overall monomer reaction, the constantsr1 to r4 are density-temperature corrections, and Mm and vare the monomer molecular weight and polymer–monomerinteraction parameter. Table 5 gives the reference valuesused to obtain the dimensionless variables as well as thevalues of reactor parameters. The final time tf is fixed tobe 313 mins [21]. The initial values of the states arex1(0) = 0, x2(0) = 1, and x3(0) = 1.

0 10 20 30 40 500

50

100

150

Solution Number

Case I - Cd(tf)*V(tf)

0 10 20 30 40 500

10

20

30

Solution Number

Case II - Cd(tf)*V(tf)

Re

lativ

e e

rror

s (%

)R

ela

tive

err

ors

(%)

der process variations (s – with model prediction confidence bounds,

Table 4Improvement in solutions by incorporating confidence bounds as extraobjectives in the fed-batch reactor case study (under process variations)

Criteria considered Number ofcases

CaseI

CaseII

Improvement in [C](tf)V(tf) 30 0Improvement in [D](tf)V(tf) 22 49Improvement in [C] (tf)V(tf) and [D](tf)V(tf) 2 0No improvement in [C](tf)V(tf) and no improvement in

[D](tf)V(tf)6 1

Table 5Parameter values of the batch polymerisation process

Am 4.266 � 105 m3/kmol sAw 0.033454B 4364 KEm 10103.5 KMm 104 kg/kmolr1 0.9328 � 103 kg/m3

r3 1.0902 � 103 kg/m3

r2 �0.87902 kg/m3 �Cr4 �0.59 kg/m3 �CTref 399.15 Ktf 313 minxnf 700xwf 1500v 0.33

-0.07 -0.06 -0.05 -0.04 -0.030.005

0.01

0.015

0.02

0.025

0.03Case I

-Cc(tf)*V(tf)

Cd

(tf)

*V(t

f)

-0.065 -0.06 -0.055 -0.050.02

0.025

0.03

0.035

0.04Case II

-Cc(tf)*V(tf)

Cd

(tf)

*V(t

f)

1 2 3 4 5

x 10-3

0

2

4

6

8x 10

-3 Case I

Standard Deviation in Cc(tf)V(tf)

Sta

ndar

d D

evi

atio

n in

Cd

(tf)

V(t

f)

1 2 3 4 5

x 10-3

0

2

4

6

8x 10

-3 Case II


ndar

d D

evi

atio

n in

Cd

(tf)

V(t

f)Fig. 11. Comparison of Pareto solutions for fed-batch reactor under process variations (s – with confidence bound, * – without confidence bound).


5.2. Modelling of the batch polymerisation process using

neural networks

The batch reaction time for the process is 313 min and isdivided into 10 equal intervals as in [23]. Within each timeinterval, the reactor temperature is maintained at a con-stant level. As discussed earlier, in the selection of the num-ber of intervals in a batch, there is trade-off between thedegree of freedom in the control policy and the computa-tional effort in network training and batch optimisation.Since the polymer quality variables are difficult to measureon-line, it is assumed here that the polymer quality mea-surements are only collected at the end of a batch anddetermined via off-line laboratory analysis. The reactortemperature set points during a batch form a control trajec-tory for the reactor. According to process knowledge, thedegree of monomer conversion, NACL and WACL arethe most important parameters for the quantification ofthe polymer quality. The objective for operating this reac-tor is to maximise the degree of conversion of the polymer,

x1(tf), and also making the end point values of dimension-less number average chain length NACL, x2(tf), and theweight average chain length WACL, x3(tf) approachingto unity. Hence, three separate bootstrap aggregated neuralnetwork models were built for the prediction of end pointvalues of these variables. The neural network models forthe prediction of the end point polymer quality variablesare as follows:

x1ðtfÞ ¼ f1ðUÞ ð20Þx2ðtfÞ ¼ f2ðUÞ ð21Þx3ðtfÞ ¼ f3ðUÞ ð22Þ

where U = [u1, u2, . . . , u10]T is a vector of reactor tempera-ture during a batch.

The data to build the neural network models resemblingan industrial situation were obtained by adding randomperturbations to the control variable, u. These nominal

Table 6Bootstrap neural network prediction accuracy on the testing data for thebatch polymerisation process

X1(tf) X2(tf) X3(tf)

MSE Standarderror

MSE Standarderror

MSE Standarderror

0.0210 0.0448 0.0343 0.0327 0.0208 0.0219


control policies may be obtained from different processoperators in the concerned batch operation. In this study,the control policy reported in [23] is taken and random uni-form variations on the control variable are added to it. Themethod of bootstrap re-sampling with replacement [19] wasused to generate 200 replications of the process data andwas then utilised to train 3 separate neural network modelsfor prediction of end point values of x1(tf), x2(tf) and x3(tf)respectively.

Of the 200 batches generated by bootstrap re-sampling,120 batches were used for training and 40 batches wereutilised for validation of the neural network model. Thetesting of the developed models was performed onthe remaining 40 batches of unseen testing data. For theprediction of each polymer quality variable via the boot-strap aggregated neural network model, 20 individualneural networks were used. The neural networks for theprediction of x1(tf), x2(tf) and x3(tf) contain 10, 14 and12 hidden neurons respectively. Hidden neurons use thesigmoid function as the transfer function and outputlayer neurons use linear transfer function. The neuralnetwork weights and bias are initialised as random num-bers uniformly distributed in (�0.1, 0.1). Each individ-ual neural network was trained using the Levenberg–Marqudt algorithm with regularisation and early stopping.Because of the different magnitudes of the model inputand output data, the data for the neural network training,validation and testing were pre-processed to be in therange [�1, 1].

Fig. 12 shows the MSEs from the individual networkson the training, validation, and testing data sets. It canbe observed from Fig. 12 that the network errors on the

0 5 10 15 200

0.02

0.04

0.06

0.08

MS

E (

Tra

inin

g)

x1

0 5 10 15 200

0.02

0.04

0.06

0.08

MS

E (

Val

idat

ion)

0 5 10 15 200

0.01

0.02

0.03

MS

E (

Tes

ting)

0 50

0.02

0.04

0.06

MS

E (

Tra

inin

g)

0 50

0.02

0.04

0.06

0.08

MS

E (

Val

idat

ion)

0 50

0.02

0.04

0.06

MS

E (

Tes

ting)

Network number Netw

Fig. 12. Model errors of individual network

training and testing data are inconsistent for the differentindividual neural network models. For example, network9 for predicting x2 gives very good performance on thetraining and validation data, however, its performance onthe unseen testing data is among the worst of the individualnetworks. Table 6 shows the MSE and standard error ofthe bootstrap aggregated neural network models on theunseen testing data. Comparing these with the results givenin Fig. 12, it can be concluded that using bootstrap aggre-gated neural network models results in greater model accu-racy and more robust models.

5.3. Multi-objective optimal control

The objective in operating this batch polymerisationreactor is to achieve a highest possible conversion of theraw materials and also approach the desired values ofdimensionless number average and weight average chainlengths (NACL and WACL) which are equal to 1.0. Theproposed reliable optimal control strategy is used to calcu-late the optimal control policy. The multi-objective optimi-sation problem to be solved using the goal attainmentalgorithm is formulated as:

10 15 20

x2

10 15 20

10 15 20

0 5 10 15 200

0.01

0.02

0.03

0.04

MS

E (

Tra

inin

g)

x3

0 5 10 15 200

0.02

0.04

0.06

MS

E (

Val

idat

ion)

0 5 10 15 200

0.01

0.02

0.03

MS

E (

Tes

ting)

ork number Network number

s for the batch polymerisation process.


F ðUÞ ¼

�x1ðtfÞk½1� x2ðtfÞ�2

k½1� x3ðtfÞ�2

re;x1ðtfÞ

re;x2ðtfÞ

re;x3ðtfÞ

26666666664

37777777775

ð23Þ

minU ;c

c ð24Þ

subject to F iU � W ic 6 F �i0:93486 6 uj 6 1:18539 j ¼ 1; 2; . . . ; 10

where c is a scalar variable, Wi are the weighting parame-ters, F �i are design goal values, U = [u1, u2, . . . , u10]T is thesequence of the reactor temperature. The control variable(reactor temperature) is denoted by u = T/Tref, with theabsolute temperature of the reactor being denoted by T

and the temperature in degree Celsius being denoted byTc. The temperature Tc, is constrained within the range100 �C 6 Tc 6 200 �C, which is translated to the boundsfor the manipulated input u, as shown in Eq. (24).re;x1ðtfÞ; re;x2

ðtfÞ; and re;x3ðtfÞ denote the individual

standard prediction errors from the three bootstrap aggre-gated neural network models. k is a scalar weighting in theobjective function having a value of 100,000. This scalarweighing is used to force the end point NACL and WACLvalues to be close to the ideal value of 1.0 and hence haveminimum deviation in these objectives. The presentedobjective function F(U) maximises the end point monomerconversion denoted by x1(tf), and minimises the deviationof the end point NACL and WACL values from 1.0 whichis achieved by using the following objectives k[1 � x2(tf)]

2

and k[1 � x3(tf)]2 for NACL and WACL respectively.

0 0.2 0.4 0.6 0.80.94

0.96

0.98

1

1.02

1.04

1.06

Time

U

Contro

Fig. 13. Optimal control profile for t

Simultaneously, it also minimises the standard predictionerrors from the aggregated neural network models forthese three quality variables. The neural network modelprediction under the optimal control policy obtained bysolving the objective function in Eqs. (23) and (24) has anarrow model prediction confidence bound and thus thecomputed optimal control policy is reliable.The followinggoal values were selected in this study:

F � ¼ ½�0:80 0:00 0:00 0:001 0:001 0:001�T

In order to demonstrate the advantage of the pro-posed technique, 50 solutions of the optimal control prob-lem were computed by varying the weights on x1(tf),k[1 � x2(tf)]

2 and k[1 � x3(tf)]2. In computing the 50

solutions, the weight for the maximisation of conversion,W1, was taken as a random number following uniformdistribution within [0, 1], the weights on the deviationsof NACL and WACL values from 1.0, W2 and W3 respec-tively, were taken as random numbers from a uniformdistribution within [0, 0.1], to lay additional emphasis onachieving the required ideal value of 1, and the weightsfor the standard prediction errors, W4, W5, and W6, weretaken as 0.04, 0.03 and 0.03, respectively. Experimentalinvestigations indicate that the above weight values onre;x1ðtfÞ, re;x2

ðtfÞ and re;x3ðtf Þ result in good perfor-

mance. Process model mismatch conditions were intro-duced in the model by varying the value of Aw, thecoefficient in the relation between WACL and tempera-ture, by assuming a normal random distribution with astandard deviation of 0.01 around its nominal value of0.033454.

Fig. 13 shows one of the 50 computed optimal controlprofiles. The corresponding trajectory for the process vari-

1 1.2 1.4 1.6 1.8 2

x 104 (Sec)

l Inputs

he batch polymerisation process.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 104

0

0.2

0.4

0.6

0.8

X1

x 104

0.95

1

1.05

1.1

X2

x 104

0.95

1

1.05

1.1

Time (sec)

X3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Fig. 14. Process variable profiles under optimal control for the batch polymerisation process.

Table 7Sample optimisation and simulation results in the batch polymerisationcase study

Weights [0.359, 0.0154, 0.0415,0.04, 0.03, 0.03]T

[0.359, 0.0154,0.0415]T

Neuralnetworkmodel

x1(tf) 0.7300 0.7599k[1- x2(tf)]

2 0.0121 0.0017k[1- x3(tf)]

2 0.0250 0.0036re;x1ðtf Þ 0.0324 0.0324

re;x2ðtf Þ 0.0222 0.0222

re;x3ðtf Þ 0.0132 0.0180

Mechanisticmodel

x1(tf) 0.7169 0.8151k[1 � x2(tf)]

2 0.2294 2.3040k[1 � x3(tf)]

2 0.0009 0.0090


ables for the investigated case is shown in Fig. 14. The opti-misation and simulation results for the same sample solu-tion are presented in Table 7. The neural network modelpredicted values and mechanistic model calculated (i.e. rep-resenting the actual process) values and other parametersare presented in Table 7 for the considered sample solution.The mechanistic model calculated outputs represent theend point quality values when the calculated ‘‘optimal”control profile is applied to the actual process (i.e. the sim-ulation on the mechanistic model).

The values shown in Table 7 indicate the effect of incor-poration of the minimisation of standard prediction errorsas an objective in the multi-objective optimisation. Thoughthe neural network predicted values of x1(tf), k[1 – x2(tf)]

2

and k[1 – x3(tf)]2 are better if no confidence bound criterion

is included in the optimisation objectives, the resulting con-trol profile is not reliable and may not provide ‘‘optimal”performance if it is applied to the actual process. The sim-ulation results on the mechanistic model (i.e. representing

the actual process) show that the end point quality values,k[1 – x2(tf)]

2 and k[1 – x3(tf)]2 and hence consequently, the

values of x2(tf) and x3(tf) are better when the standard pre-diction errors are minimised as additional optimisationobjectives. However, the degradation of the value ofx1(tf) occurs because of the addition of the new objectivesand may be explained on the very basis of the nature ofmulti-objective optimisation, in which the multiple conflict-ing objectives of different solutions may not be comparedwith each other.

Fig. 15 shows the absolute relative errors between thebootstrap aggregated neural network model predictionsof end point quality variables and the corresponding valuesfrom the actual process outputs under the computed opti-mal control profiles. The neural network predictions arefairly accurate with the mean relative errors of the neuralnetwork models being 2.03%, 1.11% and 0.84% for x1(tf),k[1 – x2(tf)]

2 and k[1 – x3(tf)]2 respectively.

Fig. 16 illustrates the percentage improvement in thequality variables due to the incorporation of the standardprediction errors in the original objective function. It canbe concluded that there is a consequent improvement inthe end point quality variables for x2(tf) and x3(tf) sincemost of the obtained solutions show a positive improve-ment in either of the end point quality values. However,there is no improvement in x1(tf). The non-improvementof the values of x1(tf) indicates that the multi-objectiveoptimisation routine is not able to achieve improved solu-tions for all the objectives concurrently and requires fur-ther investigation into the suitability of weight values.The number count of the improvements in the solutionsis presented in Table 8. It can be seen from Table 8 thatthe proposed method improves the performance in 49 outof 50 cases.

0 5 10 15 20 25 30 35 40 45 500

2

4

6

8X1(tf)

0 5 10 15 20 25 30 35 40 45 500

5

10

15X2(tf)

0 5 10 15 20 25 30 35 40 45 500

5

10

Solution Number

X3(tf)

Erro

rsE

rror

sE

rror

s

Fig. 15. Relative errors of bootstrap aggregated neural network models for the batch polymerisation process.

-0.2

-0.1

0

-0.01

0

0.01

0.02-2

0

2

4

6

8

10

x 10-3

% Diff in X1(tf)

Improvement by Confidence Bound

% Diff in X2(tf)

% D

iff in

X3(tf)

Fig. 16. Improvement in solutions by incorporating confidence bounds asextra objectives in the batch polymerisation process.

Table 8Improvement in solutions by incorporating confidence bounds as extraobjectives in the batch polymerisation case study

Criteria considered Number of cases

Improvement in x1(tf) 0Improvement in x2(tf) 49Improvement in x3(tf) 49Improvement in x1(tf), x2(tf) and x3(tf) 0


6. Conclusions

A reliable multi-objective optimal control strategy usingbootstrap aggregated neural network is proposed in thispaper. Bootstrap aggregated neural networks can not onlyprovide enhanced model prediction performance, but alsoprovide model prediction confidence bounds. Minimisingmodel prediction confidence bounds is taken as additionalobjectives in calculating the control policy. By this means,the calculated optimal control policies are forced to be inthe regions where the model predictions are reliable. Thisleads to reliable optimal control policies. Applications toa simulated fed-batch process and a simulated batch poly-merization process demonstrate that the proposed tech-nique can significantly enhance the reliability of thecontrol profiles. The proposed method can be extendedto on-line batch control in that, at each sampling time,the control policy for the remaining batch duration is re-calculated taking into account of the on-line processmeasurements.

References

[1] D. Bonvin, Optimal operation of batch reactors: a personal view,Journal of Process Control 8 (1998) 355–368.

[2] Z. Ahmad, J. Zhang, Combination of multiple neural networks usingdata fusion techniques for enhanced nonlinear process modelling,Computers and Chemical Engineering 30 (2) (2006) 295–308.

[3] S.J. Qin, T.J. McAvoy, Nonlinear PLS modeling using neuralnetwork, Computers and Chemical Engineering 16 (4) (1992) 379–391.

[4] C. Li, H. Ye, G. Wang, J. Zhang, A recursive nonlinear PLSalgorithm for adaptive nonlinear process modelling, Chemical Engi-neering and Technology 28 (2) (2005) 141–152.


[5] S.J. Zhao, J. Zhang, Y.M. Xu, A nonlinear projection to latentstructures method and its applications, Industrial and EngineeringChemistry Research 45 (11) (2006) 3843–3852.

[6] Y. Tian, J. Zhang, A.J. Morris, Modeling and optimal control of abatch polymerization reactor using a hybrid stacked recurrent neuralnetwork model, Industrial and Engineering Chemistry Research 40(21) (2001) 4525–4535.

[7] D.V. Sridhar, R.C. Seagrave, E.B. Bartlett, Process modelling usingstacked neural networks, AIChE Journal 42 (1996) 2529–2539.

[8] J. Zhang, A.J. Morris, E.B. Martin, C. Kiparissides, Inferentialestimation of polymer quality using stacked neural networks,Computers and Chemical Engineering 21 (1997) s1025–s1030.

[9] J. Zhang, Developing robust non-linear models through bootstrapaggregated neural networks, Neurocomputing 25 (1999) 93–113.

[10] J. Zhang, A reliable neural network model based optimal controlstrategy for a batch polymerisation reactor, Industrial and Engineer-ing Chemistry Research 43 (4) (2004) 1030–1038.

[11] F.W. Gembicki, Vector Optimisation for Control with Performanceand Parameter Sensitivity Indices, PhD Thesis, Case Western ReserveUniversity, Cleveland, USA, 1974.

[12] C. Bishop, Improving the generalisation properties of radial basisfunction neural networks, Neural Computation 13 (1991) 579–588.

[13] C. Bishop, Neural Networks for Pattern Recognition, OxfordUniversity Press, Oxford, 1995.

[14] D.J.C. MacKay, Bayesian interpolation, Neural Computation 4(1992) 415–447.

[15] J. Zhang, Developing robust neural network models by using bothdynamic and static process operating data, Industrial and Engineer-ing Chemistry Research 40 (1) (2001) 234–241.

[16] D.H. Wolpert, Stacked generalization, Neural Networks 5 (1992)241–259.

[17] P. Terwiesch, D. Ravemark, B. Schenker, D.W.T. Rippin, Semi-batchprocess optimization under uncertainty: theory and experiments,Computers and Chemical Engineering 22 (1998) 201–213.

[18] J. Zhang, Modelling and optimal control of batch processes usingrecurrent neuro-fuzzy networks, IEEE Transactions on FuzzySystems 13 (4) (2005) 417–427.

[19] B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans,Society for Industrial and Applied Mathematics, Philadelphia, 1982.

[20] Y.D. Kwon, L.B. Evans, A coordinate transformation method for thenumerical solution of nonlinear minimum-time control problems,AIChE Journal 21 (1975) 1158–1164.

[21] G. Gattu, E. Zafiriou, A methodology for on-line setpoint modifica-tion for batch reactor control in the presence of modeling error,Chemical Engineering Journal 75 (1999) 21–29.

[22] D. Dong, T.J. McAvoy, E. Zafiriou, Batch-to-batch optimizationusing neural network models, Industrial and Engineering ChemistryResearch 35 (1996) 2269–2276.

[23] Z. Xiong, J. Zhang, Product quality trajectory tracking in batchprocesses using iterative learning control based on time-varyingperturbation models, Industrial and Engineering Chemistry Research42 (26) (2003) 6802–6814.

a reliable multi-objective control strategy for batch processes based on bootstrap aggregated neural...

Documents