gation hybrid learning algorithm for interval copy author€¦ · gation hybrid learning algorithm...

International Journal of Hybrid Intelligent Systems 11 (2014) 125–135 125DOI 10.3233/HIS-130188IOS Press

Orthogonal-least-squares and backpropa-gation hybrid learning algorithm for intervalA2-C1 singleton type-2 Takagi-Sugeno-Kangfuzzy logic systems

Gerardo M. Méndeza,∗, J. Cruz Martinezb, David S. Gonzáleza and F. Javier Rendón-EspinozabaCentro de Manufactura Avanzada, Corporación Mexicana de Investigación en Materiales SA de CV – COMIMSA,Saltillo, Coah, MéxicobDepartamento de Economía y Administración, Instituto Tecnológico de Nuevo León, Cd. Guadalupe, N.L., México

Abstract. A novel learning methodology based on a hybrid mechanism for training interval singleton type-2 Takagi-Sugeno-Kang fuzzy logic systems uses recursive orthogonal least-squares to tune the type-1 consequent parameters and the steepestdescent method to tune the interval type-2 antecedent parameters. The proposed hybrid-learning algorithm changes the intervaltype-2 model parameters adaptively to minimize some criteria function as new information becomes available and to matchdesired input-output data pairs. Its antecedent sets are type-2 fuzzy sets, its consequent sets are type-1 fuzzy sets, and its inputsare singleton fuzzy numbers without uncertain standard deviations. As reported in the literature, the performance indices ofhybrid models have proved to be better than those of the individual training mechanisms used alone. Experiments were carriedout involving the application of hybrid interval type-2 Takagi-Sugeno-Kang fuzzy logic systems for modeling and prediction ofthe scale-breaker entry temperature in a hot strip mill for three different types of coils. The results demonstrate how the intervaltype-2 fuzzy system learns from selected input-output data pairs and improves its performance as hybrid training progresses.

Keywords: Type-2 Takagi-Sugeno-Kang fuzzy logic systems, hybrid-learning mechanism, OLS-BP training methods, ANFIS,temperature prediction

1. Introduction

All Interval type-2 Takagi-Sugeno-Kang fuzzy logicsystems are capable of approximating any real contin-uous function on a compact set to arbitrary accuracy.To use interval type-2 fuzzy systems as identifiers fornonlinear dynamic systems, it is necessary to updateand tune the fuzzy parameters so that they perform thedesired nonlinear mappings. This paper reports the de-

∗Corresponding author: Gerardo M. Méndez, Centro de Manu-factura Avanzada, Corporación Mexicana de Investigación en Ma-teriales SA de CV – COMIMSA, Saltillo, Coah, México. E-mail:[email protected].

velopment of a hybrid training algorithm to train fuzzysystems to match desired input-output data pairs.

Interval type-2 (IT2) fuzzy logic systems (FLS)are a mature technology [1–4]. The processes of fi-nancial systems [5–7], hot strip mills (HSM) [8,9], autonomous mobile robots [10], intelligent con-trollers [11–15], plant monitoring and diagnostics [16–18], edge detection [19] and forecasting of popula-tion growth [20] are characterized by high uncertainty,nonlinearity, and time-varying behavior [21]. Intervaltype-2 fuzzy sets (FS) make it possible to model the ef-fects of uncertainties [22–24] and to minimize them byoptimizing the parameters of an interval type-2 fuzzyset during the learning process.

1448-5869/14/$27.50 c© 2014 – IOS Press. All rights reserved

AUTH

OR

COPY

126 G.M. Méndez et al. / Orthogonal-least-squares and backpropagation hybrid learning algorithm

Fig. 1. Schematic view of the IT2 SFLS firing set calculation.(Colours are visible in the online version of the article; http://dx.doi.org/10.3233/HIS-130188)

In [1], both single-pass and steepest-descent meth-ods are presented as IT2 Mamdani FLS learning meth-ods, but only steepest-descent is presented as a learn-ing mechanism for IT2 Takagi-Sugeno-Kang (TSK)FLS systems. When the steepest descent method isused in both Mamdani and TSK FLS, none of the an-tecedent and consequent parameters of the IT2 FLSis fixed at the start of the training process; they aretuned using the steepest-descent method exclusively.In [1], hybrid learning algorithms based on recur-sive parameter-estimation methods such as recursiveleast-squares (RLS), recursive square-root filters (RE-FIL) [25] (a Kalman-type filter), and recursive orthog-onal least-squares (OLS) are not presented as IT2 FLSlearning mechanisms.

A feedforward neural network (FFNN) is a layeredarchitecture, and its parameters (weights) can be opti-mized using the method of steepest descent, called inthis case the backpropagation (BP) algorithm. In thisalgorithm, the output error is propagated in a backwarddirection from the output layer down into the inner lay-ers, hence the name “backpropagation”. In an FLS, theoutput error is also propagated from the output layerdown into the inner layers, and therefore this algorithmis also referred to as a backpropagation algorithm.

The aim of this work is to present a novel OLP-BP-based hybrid learning mechanism for antecedent

Fig. 2. Schematic view of the IT2 NSFLS1 firing set calculation.(Colours are visible in the online version of the article; http://dx.doi.org/10.3233/HIS-130188)

and consequent parameter tuning for interval singletontype-2 TSK FLS systems.

Here, the names of such hybrid-adapted systems willbe abbreviated based on the input type: the name IT2TSK SFLS will be used for interval singleton type-2TSK FLS systems with inputs modeled as crisp num-bers as shown in Fig. 1; IT2 TSK NSFLS1 will be usedfor interval type-1 non-singleton type-2 TSK FLS sys-tems with inputs modeled as type-1 fuzzy numbers asshown in Fig. 2; and IT2 TSK NSFLS2 will be usedfor interval type-2 non-singleton type-2 TSK FLS sys-tems with inputs modeled as type-2 fuzzy numbers asshown in Fig. 3. In addition, the names of such hybrid-adapted systems will be abbreviated based on the an-tecedent and consequent fuzzy types, where “A” rep-resents antecedent and “C” represents the consequent.A2-C1 represents the most general case of an inter-val type-2 TSK FLS when its antecedents are type-2fuzzy sets, but its consequents are type-1 fuzzy sets;A1-C1 represents the case of an interval type-2 TSKFLS when its antecedents and its consequents are bothtype-1 fuzzy sets; and A2-C0 represents the case ofan interval type-2 TSK FLS when its antecedents aretype-2 fuzzy sets, but its consequents are crisp num-bers. It is important to mention that this paper focuseson the case of an interval A2-C1 TSK SFLS system,so the A2-C1 specification can be omitted in the abbre-viated name; i.e., the hybrid interval A2-C1 singletontype-2 TSK FLS is called hybrid IT2 TSK SFLS.

The hybrid algorithm for IT2 Mamdani-type FLSsystems have been presented elsewhere [8,9,26–28]

AUTH

OR

COPY

G.M. Méndez et al. / Orthogonal-least-squares and backpropagation hybrid learning algorithm 127

Fig. 3. Schematic view of the IT2 NSFLS2 firing set calculation.(Colours are visible in the online version of the article; http://dx.doi.org/10.3233/HIS-130188)

with three combinations of hybrid learning method:RLS-BP, REFIL-BP, and OLS-BP.

For the case of TSK-type fuzzy logic systems,the hybrid algorithm method for interval singletonIT2 TSK SFLS [29,30] and for interval type-1 non-singleton IT2 TSK NSFLS1 [31,32] has been pre-sented using both RLS-BP and REFIL-BP.

In addition, in [33] there is an introduction to a type-2 TSK FLS design based on the type-1 or type-2 na-tures of the antecedent memberships and consequentparameters.

In [34] there are three interval type-2 fuzzy neu-ral networks (IT2FNN) with hybrid learning algorithmtechniques (gradient descent and gradient descent withadaptive learning rate).

In [35] it is presented a hybrid approach for im-age recognition combining type-2 fuzzy logic, modularneural networks, and the Sugeno integral.

Studies of IT2 TSK SFLS using hybrid OLP-BPlearning mechanisms as a training method have notbeen found in the literature.

In this research, an IT2 TSK SFLS system withOLP-BP hybrid learning mechanism has been devel-oped and implemented for modeling and prediction ofthe transfer-bar temperature in a hot strip mill. To en-able direct comparison of the performance and func-tionality of the proposed hybrid mechanism, the sameinput-output data set as used in [26,32] was used. Per-

formance has been experimentally examined under thesame conditions as in previous work.

This paper is organized as follows. Section 2 givesthe fundamentals of IT2 TSK fuzzy-logic systems, us-ing recursive orthogonal least-squares, and backprop-agation estimation algorithms. Section 3 presents theprocess of constructing the hybrid IT2 TSK SFLS sys-tem for temperature prediction. Section 4 presents theexperimental results, and Section 5 summarizes theconclusions.

2. Basis of the learning methodology

2.1. IT2 TSK fuzzy logic systems

A type-2 fuzzy set [1], denoted by A, is character-ized by a type-2 membership functionμA (x, u), wherex ∈ X and u ∈ Jx ⊆ [0, 1]:

A = {((x, u), μA(x, u))|∀x∈X, ∀u∈Jx⊆[0, 1]}(1)

in addition, 0 � μA (x, u) � 1. This means that at aspecific value of x, say x′, there is no longer a singlevalue, as for a type-1 membership function (u′) [1];instead, the type-2 membership function takes on aset of values called the primary membership of x′,u ∈ Jx ⊆ [0, 1]. It is possible to assign an ampli-tude distribution to this set of points. This amplitudeis called the secondary grade of a general type-2 fuzzyset. When the values of the secondary grade are thesame and equal to one, the function is an interval type-2 membership function [22–24].

An IT2 TSK fuzzy logic system having p inputsx1 ∈ X1, . . . , xp ∈ Xp and one output y ∈ Y can bedescribed by fuzzy IF-THEN rules that represent theinput-output relations of the system and that can be ex-pressed as:

Ri : IF x1 is F i1 and . . . and xp is F i

p

THEN Y i = Ci0 + Ci

1x1 + Ci2x2 + . . .+ Ci

pxp

(2)

where i = 1, . . . ,M ; Cij(j = 0, 1, . . . , p) are conse-

quent type-1 fuzzy sets (C1); Y i, the output of the ithrule, is also a type-1 fuzzy set, and F i

k(k = 0, 1, . . . , p)are interval type-2 antecedent fuzzy sets (A2) de-scribed by Gaussians with uncertain means [36–38].

AUTH

OR

COPY


In an IT2 TSK fuzzy logic system, the firing set ofthe ith rule is F i (x), where

F i (x) =∏p

k=1μF i

k(xk) =

[f i (x) , f

i(x)

](3)

and

μF ik(x) =

[μF i

k

(xk), μF ik(x)

]k = 1, . . . , p (4)

is the kth active branch of the ith rule, where

f i (x) = μF i

1

(x1) ∗ . . . ∗ μF ip

(xp) (5)

and

fi(x) = μF i

1(x1) ∗ . . . ∗ μF i

p(xp) (6)

is the firing set of the IT2 TSK fuzzy logic system. Theith consequent Ci

j is an interval set of the form:

Cij =

[cij + sij , c

ij − sij

](7)

where cij denotes the center of Cij and sij denotes the

spread of Cij (i = 1, . . . ,M and j = 1, . . . , p). Then

the consequent of Ri, Y i =[yil , y

ir

], is an interval set

and can be expressed as:

yil =∑p

j=1cijxj + ci0 −

∑p

j=1|xj |sij − si0 (8)

and

yir =∑p

j=1cijxj + ci0+

∑p

j=1|xj | sij + si0. (9)

The output of an IT2 TSK fuzzy logic system is aninterval type-1 set YTSK,2 = [yl, yr]. It is then possibleto calculate this set using the average yl and yr; theoutput is:

YTSK, 2 =yl + yr

2, (10)

where yl is

yl =

∑Mi=1 f

il y

il

M∑i=1

f il

=

L∑i=1

fiyil +

M∑i=L+1

f iyil

L∑i=1

fi+

M∑i=L+1

f i

=∑M

i=1yilp

il (x) = yT

l pl (x)

(11)

and yr is

yr =

∑Mi=1 f

iry

ir

M∑i=1

f ir

=

R∑i=1

f iyir+M∑

i=R+1

fiyir

R∑i=1

f i+M∑

i=R+1

fi

=∑M

i=1yirp

ir (x) = yT

r pr (x)

(12)

L is the index of the rule-ordered FBF expansions atwhich yl is a minimum, and R is the index at whichyr is a maximum. L and R are calculated using thealgorithm presented in [1].

In addition, if yil = yir, this represents the case of anA2-C0 IT2 TSK FLS system. In Eq. (2), when Ci

j(j =0, 1, . . . , p) are consequent type-1 fuzzy sets (C1), Y i,the output of the ith rule, is also a type-1 fuzzy set, andF ik(k = 0 1, . . . , p) are type-1 antecedent fuzzy sets

(A1), which is the case of an A1-C1 IT2 TSK FLS.This is equal to a type-1 TSK FLS because they bothprovide identical results.

2.2. The recursive orthogonal least-squares method

As mentioned above, a brief presentation of the ba-sic principles of the specific OLS method is given inthis section. Suppose that, as in [25], a particular sys-tem has one input u (k) and one output y (k) with anadditive noise e (k), measured during a certain numbert of time periods of length T ; it is then possible to de-scribe its dynamic behavior using the following differ-ences model:

y (k) =n∑

j=1

ajy (k − j)+

n∑j=0

bju (k − j)+ e(k)

(13)

where k = 1, 2, 3 . . . t; aj , bj ∈ R and n = systemorder. This can be written in more compact form:

y (k) = pT z (k) + e (k) (14)

where:

pT = [b0, a1, b1, . . . , an, bn] (15)

is the parameter estimation matrix of size 2n+ 1 and:

zT (k) = [u (k) , y (k − 1) , u (k − 1) ,

. . . , y (k − n) , u (k − n)](16)

AUTH

OR

COPY


is the measurements vector of size 2n+ 1.The model Eq. (14) can be expressed for t input-

output data pairs as:

YT (t) = PTZT (t) +ET (t) (17)

where the output vector of size t, is:

YT (t) = [y (1) , y (2) , . . . , y(t)] (18)

the measurements matrix of size (2n+ 1)× t is:

ZT (t)=

⎡⎢⎢⎢⎢⎢⎢⎣

u(1), u(2), . . . u(t)y(0), y(1), . . . y(t− 1)u(0), u(1), . . . u(t− 1). . . . . . . . . . . .. . . . . . . . . . . .

y(1− n), y(2− n), . . . y(t− n)u(1− n), u(2− n), . . . u(t− n)

⎤⎥⎥⎥⎥⎥⎥⎦

(19)

and the noise vector of size t is:

ET (t) = [e (1) , e (2) , . . . , e(t)] (20)

For the estimation of P, it is required to minimize thenext criteria:

J = (Y(t) − Z(t)P(t))T I(Y(t) − Z((t))P(t))

(21)

The symmetric and positive matrix C(t+ 1) of size(2n+ 1)× (2n+ 1) is defined as:

C(t+ 1) =[ZT (t+ 1)Z(t+ 1)

]−1(22)

this works as a covariance attenuation matrix of theidentification process. On the other hand, the linearequation system

Ax = b (23)

where A is a matrix of size m×n, x is a vector of sizen, b is a vector of size m, and m > n, does not havean exact solution, and can be written as:

Ax− b = e (24)

where e, a vector of size m, is the error of any solutionof Eq. (23). If:

ATA = FTF (25)

where F is any upper or lower triangular matrix of sizen, then Eq. (23) can be written as:

Fx =(FT

)−1ATb (26)

A least-squares solution can be found using Eq. (26).Now, considering the orthogonal transformation or ro-tational matrix defined as:

TT = T−1 (27)

Rewriting Eq. (24) as:

[A : b]

[x−1

]= e (28)

where D = [A : x] is a matrix of size m × (n+ 1),

and x′ =[x−1

]is a vector of size n+1. Now applying

the orthogonal transformation matrix T to Eq. (28), wecan obtain:

TDx′ = Te (29)

If D′ = TD = F is a triangular matrix, and b′ =

Te =(FT

)−1ATb then Eqs (26) and (29) are equiv-

alent.F the resulting upper or lower triangular matrix of

size n is the squared-root of Eq. (25).It is possible to apply the orthogonal transformation

solution to equations system for parameters identifica-tion of discrete models. The least-squares solution ofEq. (17) can be expressed as

[ZT (t)Z(t)

]PT = ZT (t)Y(t) (30)

this can be obtained through the orthogonal transfor-mations algorithm. This equation can be reduced to anequivalent triangular system:

F(t+ 1)P(t+ 1) = q(t+ 1) (31)

where the upper triangular matrix F(t+ 1), of propersize 2n + 1, is the square root of ZT (t+ 1)Z (t+ 1),and q (t+ 1) is a vector of size of 2n + 1. For eachperiod of time, the above algorithm reduces to zero onerow of the compound vector [zT (t+ 1)y(t+ 1)], ofsize 2n+ 2. The parameters of P(t+ 1) can easily becalculated by use of the REDCO routine given in [25].

AUTH

OR

COPY


2.3. The backpropagation method

As explained in [40], a squared error measure for thepth input-output pair can be defined as:

Ep = k∑

(dk − xk)2 (32)

where dk is the desired output for node k and xk is theactual output for node k when the input part of the pthdata pair is presented. To find the gradient vector, anerror term εi for node i is defined as:

εi =∂+Ep

∂xi(33)

By the chain rule of calculus, a recursive formula forεi can then be written as:

εi =

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

−2 (di − xi)∂xi

∂xi

=− 2 (di − xi)xi (1− xi)∂xi

∂xi=∑

j,i<j

∂+Ep

∂xj

∂xj

∂xi

=xi (1− xi)∑

j,i<j εjωij

(34)

where ωij is the weight of the connection from nodei to node j and ωij is zero if there is no direct con-nection. Then the weight update factor ωki for onlinelearning is:

Δωki = −η∂+Ep

∂ωki= −η

∂+Ep

∂xi

∂xi

∂ωki= −ηεixk

(35)

where η is the learning rate, which affects the con-vergence speed and stability of the weights during thelearning process.

2.4. Additional remarks on hybrid IT2 TSK SFLSsystems

The proposed hybrid algorithm uses the recursiveorthogonal least-squares during the forward pass forconsequent parameter tuning and backpropagation dur-ing the backward pass for antecedent parameter tuning.This approach is similar to Sugeno ANFIS type-1 [39],which uses a hybrid RLS-BP mechanism as a learningrule in type-1 fuzzy logic systems.

The membership functions of the hybrid IT2 TSKSFLS fuzzy-logic system are Gaussian functions. Theantecedent parameters are tuned using the backpropa-gation training method, while the consequent param-

eters are tuned using the recursive orthogonal least-squares training method.

To specify the hybrid IT2 TSK SFLS completely us-ing the training data, the design criterion is as follows:given N input-output data training pairs, the hybrid al-gorithm for E training epochs should minimize the fol-lowing error function:

e(t) =1

2

[fs2

(x(t)

)− y(t)

]2(36)

where fs2(x(t)

)is the IT2 system prediction, y(t) is

the real output value, and x(t) is the input vector at timet of the system under identification.

The universal approximation theorem [1] do not in-dicate how many inputs, what inputs, how many rulesand how many fuzzy sets for each input variable mustbe used to construct an optimal and stable IT2 TSKFLS. Universal approximation theorem imply that byusing enough inputs, enough fuzzy sets and enoughrules, the IT2 TSK FLS controller can uniformly ap-proximate any real continuous nonlinear function toarbitrary degree of accuracy. There are an enormousnumber of possibilities for an IT2 TSK FLS. The de-sign degrees of freedom that control the accuracy ofIT2 TSK FLS are number of inputs, number of rulesand number of fuzzy sets for each input variable. Con-sider the ith input variable xi, were xi ∈ Xi = [X−

i ,X+

i ]. It is intuitively obvious that dividing the inter-val

[X−

i , X+i

]into 200 overlapping regions will lead

to greater resolution, and consequently greater accu-racy than dividing the interval into 20 overlapping re-gions. If there are p inputs, each of which is dividedinto r overlapping regions, then a complete IT2 TSKFLS must contain pr rules. As resolution parameter in-creases, the size of the FLS becomes enormous. All ofthese degrees of freedom introduce uncertainties in thefinal IT2 TSK FTS models. However, this is the realand hidden power of the IT2 TSK FLS; they are thebase of the heuristic construction of the system throughthe expert’s knowledge.

3. Process of constructing the hybrid IT2 TSKSFLS

3.1. The design process

Because of the complexities and high uncertain-ties involved in rolling operations, the development ofmathematical theories for the hot strip mill has beenlargely restricted to two-dimensional models of heat

AUTH

OR

COPY


Fig. 4. Schematic view of the hot strip mill facility where experiment was carried out. (Colours are visible in the online version of the article;http://dx.doi.org/10.3233/HIS-130188)

losses in flat rolling operations. The most critical stagein the hot strip mill process is the finishing mill (FM),see Fig. 4.

The slab leaves the furnace at about ∼1280◦C andit is transported to the roughing mill by the transfertable. The reversing roughing stand gives in severalpasses the initial thickness reduction to the slab, from∼250 mm to ∼25 mm. The product of the roughingmill is called transfer bar. The transfer bar is taken tothe finishing mill where final gage, finishing temper-ature and final width specifications have to be fulfill.Due to the transfer bar length, the transfer stage be-tween roughing mill and finishing mill is ∼120 m. Dur-ing the traveling time from roughing mill to finishingmill, scale formation on the transfer bar surface takesplace. The scale breaker washes out the scale in orderto allow proper rolling of the bar.

So far, several mathematical-model-based systemshave been proposed for setting up a finishing mill.A model-based setup system calculates the finishing-mill working references needed to obtain the desiredgauge, width, and temperature at the finishing-mill exitstand [41]. It is very important for the model to knowthe finishing-mill entry temperature accurately. An er-ror in this temperature will propagate throughout theentire finishing mill. In this application, the inputs ofthe hybrid IT2 TSK SFLS (OLS-BP) systems are thesurface temperature of the transfer bar and the timerequired for the transfer-bar head to reach the scale-breaker (SB) entry zone. Currently, the surface tem-perature is measured using a pyrometer located at theroughing-mill (RM) exit. This measurement is affectedby noise produced by transfer-bar scale growth, envi-ronmental water and steam, and pyrometer location,calibration, resolution, and repeatability. The transfer-bar head-end traveling time is calculated by mathemat-ical modeling using the estimated finishing-mill threadspeed. This estimate has associated with it an inherentmodeling uncertainty.

IT2 FLS is a probed technology that accounts for allthe components of the uncertainty of a process mea-surement, which may be grouped into two categoriesaccording to the method used to estimate their numeri-cal values [42]. The type “A” evaluation of standard un-certainty may be based on any valid statistical methodfor treating data. Examples are calculating the standarddeviation of the mean of a series of independent ob-servations. The type “B” evaluation of standard uncer-tainty is usually based on scientific judgment using allthe relevant information available, which may includeprevious measurement data, experience with or gen-eral knowledge of the behavior and property of rele-vant materials and instruments, manufacturer’s specifi-cations, data provided in calibration and other reports,and uncertainties assigned to reference data taken fromhandbooks.

The architecture of the hybrid IT2 TSK SFLS (OLS-BP) model for temperature prediction is defined sothat the parameters are continuously optimized. Theantecedents are the roughing-mill exit temperature x1

and the traveling time x2. For the case of the input vari-able x1, its antecedent-input space (the universe of dis-course) was divided into three fuzzy sets, and for thecase of x2, its space was also divided into three fuzzysets, leading to nine fuzzy rules (3 × 3).

Each rule of the hybrid IT2 TSK SFLS (OLS-BP)temperature model was characterized by six antecedentparameters (three for each input variable); and six con-sequent parameters (two for each input variable plustwo for the crisp number), giving a total of twelve pa-rameters per rule.

3.1.1. Input-output data pairsFrom an industrial hot strip mill, noisy input-output

pairs for three different coil types were collected andused as training and testing data. The inputs were thenoisy measured transfer-bar surface temperature at the

AUTH

OR

COPY


roughing-mill exit side, a universe of discourse whichis bounded (an interval set) from 700◦C to 1, 300◦C,and the estimated transfer-bar traveling time fromthe roughing-mill exit to the entry point of the scalebreaker, a universe of discourse which is bounded from1 Section 100 sec. The output is the noisy transfer-barentry surface temperature measured at the entry pointof the scale breaker, a universe of discourse which isalso bounded from 600◦C to 1, 120◦C.

3.1.2. Fuzzy rule baseThe hybrid IT2 TSK SFLS fuzzy rule base consists

of a set of IF-THEN rules that represents the systemmodel. The system has two inputs x1 ∈ X1, x2 ∈ X2

and one output y ∈ Y . The rule base has M = 9 rulesof the form:

Ri : IFx1 is F i1 and x2 is F i

2 ,

THEN Y i = Ci0 + Ci

1x1 + Ci2x2

(37)

where Y i is the output of the ith rule, which is a fuzzytype-1 set, and the parameters Ci

j , i = 1, 2, 3, . . . , 9and j = 0, 1, 2, are consequent type-1 FS sets.

3.1.3. Input membership functionsThe primary membership function for each input to

the hybrid IT2 TSK SFLS system are singletons theform:

μXk(xk) = 1 (38)

where μXk(xk) is centered at the measured input xk =

x′k.

3.1.4. Antecedent membership functionsThe primary membership functions for each an-

tecedent are FSs described by Gaussians with uncer-tain means:

μik(xk) = exp

[−1

2

[xk −mi

k

σik

]2]= F i

k (39)

where mik ∈ [

mik1,m

ik2

]is the uncertain mean, k =

1, 2 (the number of antecedents), i = 1, 2, . . . 9 (thenumber of M-rules), and σi

k is the standard deviation.The fuzzy sets of the antecedent membership functionscan be determined according to linguistic rules fromhuman experts or be chosen arbitrarily in the inputspace [16].

3.1.5. Consequent membership functionsEach consequent is an interval type-1 fuzzy set with

Y i =[yil , y

ir

], where

yil =∑p

j=1cijxj + ci0−

∑p

j=1|xj |sij − si0 (40)

and

yir =∑p

j=1cijxj+ci0+

∑p

j=1|xj | sij+si0 (41)

cij denotes the center (mean) of Cij , sij denotes the

spread of Cij , i = 1, 2, 3, . . . , 9, j = 0, 1, 2, and yil and

yir are the consequent parameters.

3.2. Initial adjustment process

After the construction of the hybrid IT2 TSK SFLStemperature forecaster, the next step was the offline su-pervised adjustment of the system, in which the param-eters of the hybrid IT2 TSK SFLS (OLS-BP) tempera-ture model were adjusted using a data set of N input-output training data pairs to minimize the followingtraining error function e

(t)ts after E training epochs:

e(t)ts =

1

2

[ftss2

(x(t)ts

)− y

(t)ts

]2(42)

where ftss2(x(t)

)is the output at training phase

(the IT2 TSK SFLS-calculated scale-breaker temper-ature), x(t)

ts is the input training vector (the measuredroughing-mill exit temperature x1 and the measuredtransfer-bar translation time x2), and y

(t)ts is the out-

put training value (the measured scale-breaker entrytemperature). The mechanism for antecedent parame-ter adjustment uses the backpropagation method, whileconsequent parameters are adjusted uses the recursiveorthogonal least-squares method.

3.3. Setup process

The next step after construction and initial adjust-ment of the system is the online setup process (pro-ductive process) that calculates for each bar the scale-breaker entry temperature, ftss2

(x(m)

)as a function

of the measured roughing-mill exit temperature x1 andthe predicted transfer-bar translation time from theroughing-mill exit to the scale-breaker entry x2. At theend of the setup procedure, the hybrid IT2 TSK SFLSfuzzy model checks the physical finishing-mill limitson force, power, speed, draft, gap, and current. If alimiting condition is encountered, the hybrid IT2 TSKSFLS (OLS-BP) scale-breaker temperature predictorgenerates an alarm and does not execute the subsequentadaptation process.

AUTH

OR

COPY


3.4. Feedback process

After the online setup process, the online feedback-adaptation process take place, in which the hybrid IT2TSK SFLS (OLS-BP) scale-breaker entry temperaturepredictor adapts its parameters from one bar to another,and calculates the setup error function e

(m)ts using:

e(m)ts = ftss2 (x

m)− y(m)ts (43)

where ftss2(x(m)

)is the strip temperature at the scale-

breaker entry point as calculated by the previous setupprocess; x(m)

ts is the vector of input-variable values (thevalidated measured transfer-bar surface temperature atthe roughing-mill exit x1 and the predicted transfer-bartranslation time, x2); y(m)

ts is the validated measuredtransfer-bar surface temperature at the scale-breakerentry point (feedback). The rolling time is approxi-mately four minutes, so every four minutes, the hybridIT2 TSK SFLS (OLS-BP) temperature model adaptsits antecedent parameters using Eq. (35) and adapts itsconsequent parameters using Eqs (29)–(31).

The bar-to-bar parameter adaptation procedure en-ables the model to respond to all changing mill con-ditions: known (planed) and unknown (uncertainties).The hybrid IT2 TSK SFLS temperature model inhibitsfeedback if the strip runs outside operation conditions.

4. Results

The hybrid IT2 TSK SFLS (OLS-BP) was used topredict the scale-breaker entry temperature. Three dif-ferent sets of data for the three different coil types weretaken from a real mill. Each of these data sets was splitinto two sets: one for the initial adjustment process,and the other for the setup validation and feedback pro-cesses. Eighty-seven type A, sixty-eight type B, andtwenty-eight type C input-output data pairs were usedfor the initial adjustment process. The performanceevaluation of each of the IT2 TSK FLS system is basedon the root mean squared error (RMSE) criterion [1]:

RMSE =

√1

n

∑n

k=1[Y(k)− fs2(x(k))]

2 (44)

where Y (k) is the output value from the input-outputdata vector, i.e., the measured scale-breaker entry tem-perature value used for the initial adjustment, whichis different from the test data, but from the same coiltype, and fs2(x(k)) is the temperature predicted by theIT2 TSK FLS system.

Fig. 5. IT2 TSK SFLS systems RMSE error: (*) BP-BP, (•) RLS-BP,(x) REFIL-BP, (-) OLS-BP. (Colours are visible in the online versionof the article; http://dx.doi.org/10.3233/HIS-130188)

A period of fifty epochs was chosen for display pur-poses of the RMSEs of IT2 TSK FLS systems trainedusing type C products. Experiments show that theRMSE converges to a specific ε value. The use of onlyvalidated and bounded data guarantees that the conver-gence of the IT2 TSK FLS tested systems depends onlyon their feedback gain values, as has been proved ex-perimentally in this research and as can be observed inFig. 5. For this experiment, at epoch one, the hybridIT2 TSK SFLS OLS-BP had better performance thanthe other hybrid IT2 TSK SFLS systems. It reached agood RMSE value of ε = 4.3◦C after only two epochsand trended very quickly to its converged RMSE valueof ε = 3.7◦C. Compared to the non-hybrid IT2 TSKSFLS (BP-BP), to the hybrid IT2 SFLS (RLS-BP) sys-tems, and to the hybrid IT2 SFLS (REFIL-BP) sys-tems, the proposed IT2 TSK SFLS (OLS-BP) hybridapproach was clearly better for temperature prediction.The hybrid IT2 TSK SFLS (OLS-BP) system achievedhigh performance after only two epochs of training.

5. Conclusion

This paper presents a hybrid learning mechanism forIT2 TSK SFLS systems using OLS and BP learningmethods. The hybrid IT2 TSK SFLS antecedent mem-bership functions and consequent centroid absorbedthe uncertainty introduced by the real data (tempera-ture, and traveling-time measurements) used to tunethe hybrid system. The BP-BP, RLS-BP, REFIL-BPand the proposed OLP-BP hybrid training mechanisms

AUTH

OR

COPY


were tested using real industrial application. The re-sults showed that the hybrid algorithm (OLS-BP) canbe used to tune the IT2 TSK SFLS systems and topredict robustly the scale-breaker entry temperature ofsteel bars in real time while they are being rolled. Com-parisons of RMSEs show that the IT2 TSK SFLS sys-tems using the hybrid OLS-BP method outperformedthe BP-BP and the RLS-BP learning methods. It wasrequired only two training epochs to the hybrid trainingalgorithm to match the desired input-output data pairs.

References

[1] J.M. Mendel, Uncertain rule-based fuzzy logic systems: In-troduction and new directions, Prentice-Hall, Upper SaddleRiver NJ, (2001).

[2] J.M. Mendel, Advances in type-2 fuzzy sets and systems, In-formation Sciences 177(1) (2007), 84–110.

[3] J.M. Mendel and H. Wu, New results about the centroid ofan interval type-2 fuzzy set, including the centroid of a fuzzymendel granule, Information Sciences 177(2) (2007), 360–377.

[4] F. Lui, An efficient centroid type-reduction strategy for gen-eral type-2 fuzzy logic system, Information Sciences 178(9)(2008), 2224–2236.

[5] M.A. Hernández and G.M. Méndez, Modeling and predictionof the mxn-usa exchange rate using interval singleton type-2fuzzy logic systems, Proc IEEE Int Conf on Fuzzy Systems,Vancouver BC, Canada, 6 (2006), 10556–10559.

[6] V. Anastasakis and N. Mort, Prediction of the GSP-USD ex-change rate using statistical and neural network models, ProcIASTED Int Conf on Artificial Intelligence and Applications,Benalmádena, Spain, (2003), 493–498.

[7] A. Lendasse, E. de Boot, V. Wertz and V.M. Verleysen, Non-linear financial time series forecasting: application to the bel20 stock market index, European Journal of Economics andSocial Systems 14 (2000), 81–91.

[8] M. Méndez, A. Cavazos, L. Leduc and R. Soto, Hot stripmill temperature prediction using hybrid learning intervalsingleton type-2 FLS, Proc IASTED Int Conf on Modelingand Simulation, Artificial Intelligence and Applications, PalmSprings, CA, (2003), 380–385.

[9] M. Méndez, A. Cavazos, L. Leduc and R. Soto, Modeling ofa hot strip mill temperature using hybrid learning for intervaltype-1 and type-2 non-singleton type-2 fuzzy logic systems,Proc IASTED Int Conf on Artificial Intelligence and Applica-tions, Benalmádena, Spain, (2003), 529–533.

[10] H.A. Hagras, A hierarchical type-2 fuzzy logic control archi-tecture for autonomous mobile robots, IEEE Transactions onFuzzy Systems 12(4) (2004), 524–539.

[11] R. Sepulveda, O. Castillo, P. Melin and O. Montiel, Anefficient computational method to implement type-2 fuzzylogic in control applications, Advances in Soft Computing 41(2007), 45–52.

[12] R. Sepulveda, O. Castillo, P. Melin, A. Rodriguez-Diaz and O.Montiel, Experimental study of intelligent controllers underuncertainty using type-1 and type-2 fuzzy logic, InformationSciences 177(10) (2007), 2023–2048.

[13] M.Y. Hsiao, T.H. Li, J.Z. Lee, C.H. Chao and S.H. Tsai, De-sign of interval type-2 fuzzy sliding-mode controller, Infor-mation Sciences 178(6) (2008), 1696–1716.

[14] O. Castillo, P. Melin, A.A. Garza, O. Montiel and R.Sepúlveda, Optimization of interval type-2 fuzzy logic con-trollers using evolutionary algorithms, Soft Comput 15(6)(2011), 1145–1160.

[15] N. Cázarez-Castro, L.T. Aguilar and O. Castillo, Fuzzy logiccontrol with genetic membership function parameters opti-mization for the output regulation of a servomechanism withnonlinear backlash, Expert Syst Appl 37(6) (2010), 4368–4378.

[16] O. Castillo and P. Melin, A new hybrid approach for plantmonitoring and diagnostics using type-2 fuzzy logic and frac-tal theory, Proceedings, FUZZ’ 2003, St. Louis MO, (2003),102–107.

[17] O. Castillo, G. Huesca and F. Valdez, Evolutionary computingfor optimizing type-2 fuzzy systems in intelligent control ofnon-linear dynamic plants, Proc IEEE NAFIPS 05 Int Conf(2005), 247–251.

[18] P. Melin and O. Castillo, An intelligent hybrid approach forindustrial quality control combining neural networks, fuzzylogic and fractal theory, Journal of Information Sciences177(23) (2007), 1543–1557.

[19] P. Melin, O. Mendoza and O. Castillo, An improved methodfor edge detection based on interval type-2 fuzzy logic, ExpertSyst Appl 37(12) (2010), 8527–8535.

[20] C.L. Ramírez, O. Castillo, P. Melin and A.R. Díaz, Simula-tion of the bird age-structured population growth based on aninterval type-2 fuzzy cellular structure, Information Sciences181(3) (2011), 519–535.

[21] D. Wu and J.M. Mendel, Uncertainty measures for inter-val type-2 fuzzy sets, Information Sciences 177(23) (2007),5378–5393.

[22] Q. Liang and J.M. Mendel, Interval type-2 fuzzy logic sys-tems: theory and design, Transactions on Fuzzy Systems 8(2000), 535–550.

[23] R.I. John, Embedded interval valued type-2 fuzzy sets, Proc2002 IEEE Int. Conf on Fuzzy Systems 1 & 2, Honolulu,Hawaii, (2002), 1316–1321.

[24] J.M. Mendel and R.I. John, Type-2 fuzzy sets made simple,IEEE Transactions on Fuzzy Systems 10(2) (2002), 117–127.

[25] A. Aguado, Temas de identificación y control adaptable,La Habana, Cuba, Instituto de Cibernética, Matemáticas yFísica, (2000).

[26] G.M. Méndez and I.L. Juárez, Orthogonal back-propagationhybrid learning algorithm for interval type-1 non-singletontype-2 fuzzy logic systems, WSEAS Trans on Syst 3(4)(2005),1109–2777.

[27] G.M. Méndez, A. Cavazos, R. Soto and L. Leduc, Entry tem-perature prediction of a hot strip mill by hybrid learning type-2 FLS, Journal of Intelligent and Fuzzy Syst 17(6) (2006),583–596.

[28] G.M. Méndez and M. de los Angeles Hernandez, Hybridlearning for interval type-2 fuzzy systems based on orthogo-nal least-squares and back-propagation methods, InformationSciences 179(13) (2009), 2146–2157.

[29] G.M. Méndez and O. Castillo, Interval type-2 tsk fuzzy logicsystems using hybrid learning algorithm, Proc IEEE Int Confon Fuzzy Systems (2005), 230–235.

[30] G.M. Méndez and I.L. Juarez, First-order interval type-2tsk fuzzy logic systems using a hybrid learning algorithm,WSEAS Trans on Comp 4 (2005), 378–384.

[31] G.M. Méndez, Interval type-1 non-singleton type-2 tsk fuzzylogic systems using the hybrid training method RLS-BP,Analysis and Design of Intelligent Systems Using Soft Com-puting Techniques, Springer-Verlag, (2007), 36–44.

AUTH

OR

COPY


[32] G.M. Méndez and M. de los Angeles hernandez, interval type-2 ANFIS, innovations in hybrid intelligent systems, Springer-Verlag, (2007), 64–71.

[33] Q. Liang and J.M. Mendel, An introduction to type-2 tskfuzzy logic systems, Proceedings, International Fuzzy Sys-tems Conference (FUZZ-IEEE ’99) 3 (1999), 1534–1539.

[34] J.R. Castro, O. Castillo, P. Melin and A. Rodríguez-Díaz, Ahybrid learning algorithm for a class of interval type-2 fuzzyneural networks, Information Sciences, Special Section onHigh-Order Fuzzy Sets 179(13) (2009), 2175–2193.

[35] O. Mendoza, P. Melin and G. Licea, A hybrid approach forimage recognition combining type-2 fuzzy logic, modularneural networks and the sugeno integral, Information Sciences179(13) (2009), 2078–2101.

[36] J.M. Mendel, On the importance of interval sets in type-2fuzzy logic systems, Proc Joint 9th IFSA World Congress

and 20th NAFIPS Int Conf, Vancouver, BC, Canada, (2001),1647–1652.

[37] J.B. Turksen, Interval-valued fuzzy uncertainty, Proc FifthIFSA World Congress (1993), 5–38.

[38] J.B. Turksen, Interval-valued fuzzy sets and fuzzy connec-tives, Interval Computations 4 (1993), 125–142.

[39] J.-S.R. Jang, C.-T. Sun and E. Mizutani, Neuro-fuzzy andsoft computing: A computational approach to learning andmachine intelligence, Prentice-Hall, Upper Saddle River NJ,(1997).

[40] J.-S.R. Jang and C.-T. Dun, Neuro-fuzzy modeling and con-trol, Proc of the IEEE 83, (1995), 378–406.

[41] G. Electric, Models Reference Manual 1 Roanoke VA, 1993.[42] B.N. Taylor and C.E. Kuyatt, Guidelines for Evaluating and

Expressing the Uncertainty of NIST Measurement Results,Technical note 1297, Gaitherburg, MD, NIST, 1994.

AUTH

OR

COPY

gation hybrid learning algorithm for interval copy author€¦ · gation hybrid learning algorithm...

Documents