a genetic-based neuro-fuzzy approach for modeling and control of dynamical systems

8/6/2019 A Genetic-Based Neuro-Fuzzy Approach for Modeling and Control of Dynamical Systems

1/12

756 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

A Genetic-Based Neuro-Fuzzy Approach forModeling and Control of Dynamical Systems

Wael A. Farag, Victor H. Quintana, and Germano Lambert-Torres

Abstract Linguistic modeling of complex irregular systemsconstitutes the heart of many control and decision making sys-tems, and fuzzy logic represents one of the most effective algo-rithms to build such linguistic models. In this paper, a linguistic(qualitative) modeling approach is proposed. The approach com-bines the merits of the fuzzy logic theory, neural networks, andgenetic algorithms (GAs). The proposed model is presented in afuzzy-neural network (FNN) form which can handle both quan-titative (numerical) and qualitative (linguistic) knowledge. Thelearning algorithm of an FNN is composed of three phases. Thefirst phase is used to find the initial membership functions of thefuzzy model. In the second phase, a new algorithm is developedand used to extract the linguistic-fuzzy rules. In the third phase,

a multiresolutional dynamic genetic algorithm (MRD-GA) isproposed and used for optimized tuning of membership functionsof the proposed model. Two well-known benchmarks are used toevaluate the performance of the proposed modeling approach,and compare it with other modeling approaches.

Index TermsDynamic control, fuzzy logic, genetic algorithms,modeling, neural networks.

I. INTRODUCTION

THE principle of incompatibility, formulated by Zadeh

[1], explains the inadequacy of traditional quantitative

techniques when used to describe complex systems. Zadeh has

suggested a linguistic (qualitative) analysis for these systems

in place of a quantitative analysis. Accordingly, linguisticmodeling of complex systems has become one of the most

important issues [2][5]. A linguistic model is a knowledge-

based representation of a system; its rules and inputoutput

variables are described in a linguistic form which can be easily

understood and handled by a human operator; in other words,

this kind of representation of information in linguistic models

imitates the mechanism of approximate reasoning performed

in the human mind.

The fuzzy set theory formulated by Zadeh [6] has been

considered an appropriate presentation method for linguistic

terms and human concepts since Mamdanis pioneering work

in fuzzy control [7]. This work has motivated many researchers

to pursue their research in fuzzy modeling [2][5], [8][13].

Fuzzy modeling uses a natural description language to form a

system model based on fuzzy logic with fuzzy predicates.

Manuscript received February 28, 1997; revised October 10, 1997.W. A. Farag and V. H. Quintana are with the Electrical and Computer

Engineering Department, University of Waterloo, Waterloo, Ontario, Canada,N2L 3G1.

G. Lambert-Torres is with Escola Federal de Engenharia de Itjuba, Itajuba,MG, Brazil.

Publisher Item Identifier S 1045-9227(98)06185-2.

The knowledge representation in fuzzy modeling can be

viewed as having two classes. The first (class A) as suggested

by Takagi and Sugeno in [11] can represent a general class

of static or dynamic nonlinear systems. It is based on fuzzy

partition of input space and it can be viewed as the expansion

of piecewise linear partition represented as

If is and is and is

then (1)

where denotes the th fuzzy rule, and

is the input and is the output of thefuzzy rule . , are fuzzy

membership functions which can be bell-shaped, trapezoidal,

or triangular, etc., and usually they are not associated with

linguistic terms. From (1), it is noted that Takagi and Sugeno

approach approximates a nonlinear system with a combination

of several linear systems by decomposing the whole input

space into several partial fuzzy spaces and representing each

output space with a linear equation. This type of knowledge

representation does not allow the output variables to be de-

scribed in linguistic terms which is one of the drawbacks

of this approach. Another drawback is that the parameter

identification of this model is carried out iteratively using a

nonlinear optimization method [10], [11]. The implementationof this method is not an easy task [8], [12], [13], because the

problem of determining the optimal membership parameters

involve a nonlinear programming problem.

The second class of knowledge representation (class B) in

fuzzy models is developed by Mamdani [14] and used by Lin

and Lee [15] and Sugeno and Yasukawa [5]. The knowledge

is presented in these models as

If is and is and is

then is (2)

where , are fuzzy mem-bership functions which are bell-shaped, trapezoidal, or tri-

angular, etc., and usually associated with linguistic terms.

This approach has some advantages over the first approach.

The consequent parts are presented by linguistic terms, which

makes this model more intuitive and understandable and gives

more insight into the model structure. Also, this modeling

approach is easier to implement than the first approach [13].

This second form (class B) of knowledge representation will

be adopted throughout this paper as we are more concerned

with the linguistic modeling approaches.

10459227/98$10.00 1998 IEEE


2/12

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH 757

Many studies regarding finding the rules and tuning the

membership function parameters of fuzzy models have been

reported [2][5], [8][13]. Neural networks are integrated with

fuzzy logic in a form of fuzzy neural networks (FNNs)

and used to build fuzzy models [15][22]. Many algorithms

have been proposed to train these FNNs [15][22], and Jang

et al. [19] have reviewed the fundamental and advanced

developments in neuro-fuzzy synergisms for modeling and

control. Lin and Lee [15] have proposed a three-phase learning

algorithm. In the first phase, they have used the self-organizing

feature map algorithm for coarse-identification of fuzzy model

parameters. In the second phase, they have used a competitive

learning technique to find the rules. And in the third phase

they have used the backpropagation algorithm for fine-tuning

the parameters.

In this paper, we propose a new approach for building lin-

guistic models for complex dynamical systems. The structure

of the model is formed using a five-layer fuzzy neural network.

The parameter identification of the fuzzy model is composed

of three phases. The first phase uses the Kohonens self

organizing feature map algorithm to find the initial parametersof the membership functions. A new algorithm is proposed,

in the second phase, to find the linguistic rules. The third

phase fine-tunes the membership functions parameters using a

new genetic algorithm (GA) called proposed multiresolutional

dynamic genetic algorithm (MRD-GA). The method used inthis work builds a linguistic model in a general framework

known as the black box approach in systems theory. That is,

the model is built for a system without a priori knowledge

about the system provided that numerical inputoutput data is

given.

This paper is organized as follows. In Section II, a

brief overview on conventional genetic algorithms is given.

Section III illustrates the structure of the neuro-fuzzy model.Section IV describes the hybrid learning algorithm. Sections V

and VI present the simulation results of two benchmarks.

Section VII concludes the work done in this paper.

II. OVERVIEW OF GENETIC ALGORITHMS

GAs are powerful search optimization algorithms based on

the mechanics of natural selection and natural genetics. GAs

can be characterized by the following features [23], [24]:

a scheme for encoding solutions to the problem, referred

to as chromosomes or strings;

an evaluation function (referred to as a fitness function)

that rates each chromosome relative to the others in thecurrent set of chromosomes (referred to as a population);

an initialization procedure for a population of chromo-

somes (strings);

a set of operators which are used to manipulate the genetic

composition of the population (such as recombination,

mutation, crossover, etc.);

a set of parameters that provide initial settings for the

algorithm and operators as well as the algorithms termi-

nation conditions.

A candidate solution (in a GA) for a specific problem is

called a chromosome and consists of a linear list of genes,

where each gene can assume a finite number of values (alleles).

A population consists of a finite number of chromosomes.

The genetic algorithm evaluates a population and generates a

new one iteratively, with each successive population referred

to as a generation. Given an initial population , the

GA generates a new generation based on the previous

generation as follows [24]:

Initialize Population at time

Evaluate

While (not terminate-condition) do

begin

:Increment generation

select from

recombine :apply genetic operators

(crossover, mutation)

evaluate

end

end.

The GA uses three basic operators to manipulate the genetic

composition of a population: reproduction, crossover, and

mutation. Reproduction is a process by which the most highly

rated chromosomes in the current generation are reproduced in

the new generation. Crossover operator provides a mechanism

for chromosomes to mix and match attributes through random

processes. For example, if two chromosomes (parents)

are selected at random (such as

and ) and an arbitrary crossover

site is selected (such as 3), then the resulting two

chromosomes (offspring) will be and

after the crossover operation takes

place. Mutation is a random alteration of some gene values in

a chromosome. Every gene in each chromosome is a candidatefor mutation, and its selection is determined by the mutation

probability.

GAs provide a means to optimize ill-defined, irregular

problems. They can be tailored to the needs of different situ-

ations. Because of their robustness, GAs have been success-

fully applied to generate ifthen rules and adjust membership

functions of fuzzy systems [25][28].

The GA described above is a conventional GA, meaning

the one in which the parameters are kept constant while the

optimization process is running (static GA). In our approach,

we introduce a new dynamic GA (MRD-GA) in which some

of its parameters as well as the problem configuration change

from one generation to next (while the optimization process isrunning) as will be discussed later in Section IV-C.

III. THE NEURO-FUZZY (NF) MODEL TOPOLOGY

The NF model is built using a multilayer fuzzy neural net-

work shown in Fig. 1. The system has a total of five layers as

proposed by Lin and Lee [15]. A model with two inputs and a

single output is considered here for convenience. Accordingly,

there are two nodes in layer 1 and one node in layer 5.

Nodes in layer 1 are input nodes that directly transmit input

signals to the next layer. Layer 5 is the output layer. Nodes in

layers 2 and 4 are term nodes and they act as membership


3/12


Fig. 1. Topology of the neuro-fuzzy model.

functions to express the inputoutput fuzzy linguistic variables.

A bell-shaped function is adopted to represent the membership

function, in which the mean value and the variance are

adjusted through the learning process. The two fuzzy sets of

the first and the second input variables consist of and

linguistic terms, respectively. The linguistic terms, such

as positive large (PL), positive medium (PM), positive small

(PS), zero (ZE), negative small (NS), negative medium (NM),

negative large (NL), are numbered in descending order in the

term nodes. Hence, nodes and nodes are included

in layers 2 and 4, respectively, to indicate the inputoutput

linguistic variables.

Each node of layer 3 is a rule node and represents a

single fuzzy control rule. In total, there are nodes

in layer 3 to form a fuzzy rule base for two linguistic inputvariables. The links of layers 3 and 4 define the preconditions

and consequences of the rule nodes, respectively. For each

rule node, there are two fixed links from the input term nodes.

Layer 4 links, encircled in dotted line, are adjusted in response

to varying control situations. By contrast, the links of layers 2

and 5 remain fixed between the inputoutput nodes and their

corresponding term nodes.

The NF model can adjust the fuzzy rules and their member-

ship functions by modifying layer 4 links and the parameters

that represent the bell-shaped membership functions for each

node in layers 2 and 4. For convenience we use the following

notation to describe the functions of the nodes in each of the

five layers.the net input value to the th node in layer .

the output value of the th node in layer .

the mean and variance of the bell-shaped func-

tion of the th node in layer .

the link that connects the output of the th node

in layer 3 with the input to the th node in

layer 4.

Layer 1: The nodes of this layer directly transmit input

signals to the next layer. That is

(3)

Layer 2: The nodes of this layer act as membership func-

tions to express the linguistic terms of input variables. For a

bell-shaped function, they are

for

for(4)

for

(5)

Note that layer 2 links are all set to unity.

Layer 3: The links in this layer are used to perform pre-

condition matching of fuzzy rules. Thus, each node has two

input values from layer 2. The correlation-minimum inference

procedure is utilized here to determine the firing strengths of

each rule. The output of the nodes in this layer is determined

by fuzzy AND operation. Hence, the functions of the layer

are as follows:

for

(6)for (7)

The link weights in this layer are also set to unity.

Layer 4: Each node of this layer performs the fuzzy OR

operation to integrate the field rules leading to the same output

linguistic variable. The functions of the layer are expressed as

follows:

(8)

for (9)

The link weight in this layer expresses the association of

the th rule with the th output linguistic variable. It can takeonly two values; either one or zero.

Layer 5: The node in this layer computes the output signal

of the NF model. The output node together with layer 5 links

act as a defuzzifier. The center of area defuzzification scheme,

used in this model, can be simulated by

(10)

(11)

Hence, the th link weight in this layer is .

IV. THE PROPOSED HYBRID LEARNING ALGORITHM

In this section we present a three-phase learning scheme for

the above NF model. The proposed scheme is an extension

of the ideas presented in [15] and [21]. It is quite convenient

to divide the task of constructing the fuzzy model into the

following subtasks: locating initial membership functions,

finding the ifthen rules, and optimal tuning of the membership

functions. In phase one and two of the proposed scheme,

unsupervised learning algorithms are used to perform the


4/12


first and the second subtasks. In phase three, a supervised

learning scheme is used to perform the third subtask. To

initiate the learning scheme, training data and the desired or

selected coarse of fuzzy partition (i.e., the size of the term

set of each inputoutput linguistic variable) must be provided

from the outside world. For more details about the structure

identification of fuzzy models refer to Sugeno et al. [29].

A. Learning-Phase One

The problem for the self-organized learning can be stated as:

Given the training input data , the desired

output value , the fuzzy partitions

and , and the desired shapes of membership

functions, we want to locate the membership functions. In

this phase, the network works in a two-sided manner; that

is, the nodes and the links at layer 5 are in the updown

transmission mode manner (follow the dotted lines in Fig. 1)

so that the training input and output data are fed into this

network from both sides.

The centers (or means) and the widths (or variances) of

the membership functions are determined by a self-organized

learning technique that is analogous to statistical clustering.

This serves to allocate network resources efficiently by placing

the domains of membership functions covering only those

regions of the inputoutput space where data are present.

Kohonens self-organized feature-map (SOM) algorithm is

adapted to find the center of the membership function [15]

(12)

(13)

for (14)

where is a monotonically decreasing scalar learning rate,

and . This adaptive formulation runs independently

for each input and output linguistic variable. The determination

of which of s is can be accomplished in constant

time via a winner-take-all circuit [15].

Once the centers of membership functions are found, their

widths can be determined by the -nearest-neighbors heuris-

tic, by minimizing the following objective function with

respect to the widths s:

(15)

where is an overlap parameter that usually ranges from 1.0

to 2.0. Since our third learning phase will optimally adjust

the centers and the widths of the membership functions, the

widths can be simply determined by the first-nearest-neighbor

heuristic at this stage as

(16)

B. Learning-Phase Two

After the parameters of the membership functions have been

found, the training signals from both external sides can reach

the outputs of the term nodes at layer two and layer four.

Furthermore, the outputs of term nodes at layer two can be

transmitted to rule-nodes through the initial architecture of

layer-three links. Thus we can get the firing strength of each

rule-node. Based on these rule-firing strengths (denoted ass) and the outputs of term-nodes at layer four (denoted

as s), we want to decide the correct consequence-link

for each rule node (from the connected layer-four-links)

to find the rules. A new algorithm is proposed here

to perform this task. We refer to this algorithm as maximum

matching-factor algorithm (MMFA). The MMFA is described

as follows.

Step 1: For each layer-three-rule node we construct

matching factors. In this case, we have matching

factors. Each matching factor is denoted as , where the

subscript is the rule-node-index ( ), and

the subscript is the output-linguistic-variable-index (output-

term-node-index) ( ).Step 2: is calculated according to the following pseu-

docode:

(no. of available training examples)

if is the maximum element

in the set

Otherwise.

endend

end.

Step 3: After calculating all the s using the previous

code considering all the available training patterns, the rules

consequences can be determined form these factors according

to the following pseudocode:

For

Find the maximum matching-factor

from the set

Find the corresponding term-node index

of

Delete all the layer-four-links of the th

rule-node except the one connecting it

with the term-node of index ,

end

Step 4: From the above algorithm, only one term in the out-

put linguistic variables term set can become the consequence

of a certain fuzzy-logic rule. If all the matching factors of a

rule-node are very small (meaning that this rule has small or

no effect on the output), then all the corresponding links are

deleted, and this rule-node is eliminated.


5/12


C. Learning-Phase Three

After the fuzzy rules are found, the whole network structure

is established, and the third-learning phase is started in order to

optimally adjust the parameters of the membership functions.

Optimization, in the most general form, involves finding the

most optimum solution from a family of reasonable solutions

according to an optimization criterion. For all but a few trivial

problems, finding the global optimum can never be guaranteed[30, ch. 14]. Hence, optimization in the last three decades

has focused on methods to achieve the best solution per unit

computational cost.

The problem for this supervised learning phase can be

stated as: Given the training input data ,

the desired output value , the fuzzy

partitions and , the desired shapes of membership

functions, and the fuzzy rules, adjust the parameters of the

membership functions optimally. In this phase, the network

works in the feedforward manner; that is, the nodes and the

links at layer four are in the down-up transmission mode.

A new approach is proposed here to use GAs for the op-

timization of the membership functions parameters. Problem-specific knowledge is used to tailor a GA to the needs of this

learning phase. The main attribute of the proposed approach

is that the fuzzy-model configuration is dynamically-adapted

while the optimization process is running. Accordingly, the

MRD-GA changes its search space with the change of the

problem configuration and with the advance of generations.

The GA search space is monotonically getting narrower and

narrower, while the model parameters are getting closer and

closer to the optimal values.

The GA is coded using the well-structured language C .

The program allows the user to define the values for population

size (pop size), maximum number of generations (max gen),probability of crossover (pcross), and probability of mutation

(pmut). In order to select the individuals for the next genera-

tion, the tournament selection method is used. In this method,

two members of the population are selected at random and

their fitness values compared. The member with higher fitness

advances to the next generation. An advantage of this method

is that it needs less computational effort than other methods.

Also, it does not need a scaling process (like the roulette

wheel selection). However, the particulars of the reproduction

scheme are not critical to the performance of the GA; virtually,

any reproduction scheme that biases the population toward the

fitter strings works well [25].

The MRD-GA uses decimal-integer strings to encode themodel parameters. The decimal strings are considered a more

suitable representative method than the binary strings. This

representation allows the use of a more compact-size strings.

The number of alleles (individual locations which make up

the string) is determined from the total number of fuzzy sets

used to partition the spaces of the inputoutput variables.

For the model configuration shown in Fig. 1, we have (

) membership functions. Each bell-shaped

membership function is defined by two parameters (the center

, and the width ). To optimize the membership functions,

we have to optimize ( ) parameters. Thus, the GA uses

strings of length alleles. It is allowed for each allele

to take any value in the set [ ]. To convert the

allele-value to a new center or width of a certain membership

function, we use the following procedure.

Step 1: The initial values of the centers and widths of the

fuzzy controller are entered to the GA program, for example,

( ) and ( ).

Step 2: The new centers and widths are calculated from

the allele values as

(17)

(18)

where and are the new center and width values, respec-

tively, is the value of the th allele in the string, and

and are the offsets of the centers and widths, respectively.

It is recommended to set these offsets to very small values

(around 0.001). This allows a more stable convergence of the

MRD-GA.

Step 3: If the allele value of any center or width equals

five then no change occurs. If is greater than five then

positive change occurs (the center or width increases). If it isless than five then negative change occurs (the center or width

decreases).

The MRD-GA uses the mean squared error (MSE) (the error

is the difference between the actual output and the estimated

output by the fuzzy model) as a fitness function. Simply,

for each chromosome (1/MSE) is considered as the fitness

measure of it. The MSE is calculated from data points as

MSE (19)

where is the actual value, and is the estimated value.

Each generations, the offset values ( and ) decrease

according to the following decaying functions:(20)

(21)

where and are the modifying factors for the centers

and widths, respectively. The decaying functions can take any

decaying shape such as, for example, an exponential decay.

The usual GA terminating condition is a maximum allowable

generations or a certain value of MSE required to be reached.

In this GA algorithm, the stopping criteria is the execution of a

certain number of generations without any improvement in the

best fitness value. In this criteria, you do not need to specify

a required MSE value (which usually unknown in advance) or

a required number of generations (where there is no granteethat this number will produce an appropriate solution).

The MRD-GA pseudocode is shown at the bottom of the

next page.

This GA offers exciting advantages over the conventional

GA [31][33]. It allows a dynamic increase in the resolution

of the search space (by decreasing and ) as the model

parameters approach their optimal values. It also changes the

nature of the model-identification problem from a static type to

a dynamic type (by adapting and continuously) which

decreases the chances of the GA premature convergence. The

MRD-GA has also advantages over the backpropagation (BP)


6/12


algorithm [15], [21]. The MRD-GA allows to obtain interme-

diate solutions, which the BP usually cannot offer; also, the

GA does not suffer from convergence problems with the same

degree that the BP suffers (i.e., the MRD-GA is more robust).

V. THE FIRST NUMERICAL EXAMPLE

The proposed modeling algorithm is examined using thewell-known example of system identification given by Box

and Jenkins [34]. The process is a gas furnace with a single

input and a single output : gas flow rate and CO con-

centration, respectively. The data set consists of pairs

of inputoutput measurements. The sampling interval is 9 s.

The inputs to the fuzzy model are selected to be

and , respectively [35]. The input variable

is modified to be , where

is the average of all the s. Each of the input

variables is intuitively partitioned into seven linguistic sets

( ). The output of the fuzzy model is , where

the actual process output is . The model

output is partitioned into nine linguistic sets ( ).The gas furnace is modeled using the fuzzy-neural network

shown in Fig. 1. The SOM algorithm (Section IV-A) is used to

determine the initial centers and widths of the 23 member func-

tions of the inputoutput variables. The three scaling factors

of this model are determined as Gu 0.658, Gy 0.227,

and Go 7.909. The resultant membership functions after

finishing this learning phase are shown in Fig. 2.

The MMFA algorithm (Section IV-B) is used to find the

linguistic fuzzy rules of the gas furnace model. Out of the 49

rules of the fuzzy model, 37 rules are only considered. The

rest are deleted because they have very small matching factors

(less than 1% of the highest matching factor of all the rules).

The 37 rules are shown in Table I.The MRD-GA (Section IV-C) is then applied to optimize

the parameters of the gas furnace model. The algorithm

parameters are set as follows: pcross 0.9, pmut 0.1,

TABLE ITHE COMPLETE FUZZY ASSOCIATIVE MEMORY MATRIX WITH THE RULES

TABLE IITHE COMPUTATION TIME OF EXAMPLE 1

chromosome-length 46, , ,, , and . To study the pop size

effect on the search performance of the proposed GA, the GA

is applied four times with different population sizes (pop size

80, 50, 30, and 12). After finishing the second learning

phase and before applying the GA, the model has an MSE

value of 0.937. This MSE value is decreased to 0.111 after

4972 generations of the MRD-GA using pop_size 50 (note

that the MSE value reached 0.15 after only 900 generations).

The computation time elapsed to perform the whole learning

scheme is roughly determined as shown in Table II. The

resultant membership functions and the model output are

shown in Figs. 3 and 4, respectively. The MSE decrease rates

using different population sizes are also shown in Fig. 5.In Table III, our fuzzy model is compared with other models

identified from the same data. It can be seen that our model

outperforms all the other models in its class (class B). In

Initialize . Population at time .

Initialize best fit = 0. :The best fitness value.

Evaluate .

Search for the best fitness of and assign best fit to it.

While (not terminate-condition) do

Begin

:Increment generation.

If then modify (decrease) and as given in (20), and (21).

Select from using tournament selection criteria.

Recombine :apply genetic operators (crossover, mutation).

Evaluate .

Search for the best fitness of and compare it with best fit, if larger then do

Begin

Assign best fit to the best fitness value of .

Adapt the centers and the widths and according

to the state of the chromosome having the best fitness using (17), (18).

End

End

End.


7/12


Fig. 2. The normalized membership functions after SOM.

Fig. 3. The optimized membership functions after MRD-GA.

comparison with class A models, Sugenos model [10] has

less MSE value using six inputs but, in the same time, has

much higher MSE value using the same inputs used by our

model ( , and ). Also, This model is quite

difficult to build [8], [12], [13]. The most difficult aspect

lies in the identification of the premise structure, mainly

the membership functions of the input variables. For each

membership function, at least two or three parameters have

to be calculated through a nonlinear programming procedure.

The choice and computation of these membership functions are

rather tricky and subjective so that it is possible for different

designers to sometimes get completely different results.

Wangs model (class A) [8] has comparable results and

less number of rules; however, the number of rules does


8/12


Fig. 4. Output of the gas furnace fuzzy model.

Fig. 5. The MRD-GA convergence rates with different population sizes.

not necessarily give a reliable indication of the number of

unknown parameters of the model. For example, in Wangs

model [8] five class A rules are used with two inputs, the

number of unknown parameters in this case (in both thepremise and consequent parts) are 35. In our model, 37 class

B rules are used with two inputs and 46 unknown parameters.

Bearing in mind that our model shows about 30% reduction in

the MSE value and provides a linguistic description for the gas

furnace system; these two advantages, in our view, compensate

for the difference in the number of parameters (46 versus 35).

VI. THE SECOND NUMERICAL EXAMPLE

This example is taken from Narendra et al. [36] in which

the plant to be identified is given by the second-order highly

nonlinear difference equation

(22)

Training data of 500 points are generated from the plant

model, assuming a random input signal uniformly dis-

tributed in the interval [ ]. This data is used to build a

linguistic-fuzzy model for this plant.

The plant is modeled using the FNN described in

Section III. The model has three inputs , , and ,

and a single output . The inputs and are intuitively

partitioned into five fuzzy linguistic spaces NL, NS, ZE, PS,

PL , the input is partitioned into three fuzzy spaces N,

Z, P and the output is partitioned into 11 fuzzy spaces

NVL, NL, NM, NS, NVS, ZE, PVS, PS, PM, PL, PVL .


9/12


TABLE IIICOMPARISON OF OUR MODEL WITH OTHER MODELS

The SOM algorithm described in Section IV-A is used to

determine the initial centers and widths of the 24 membership

functions of the inputoutput variables of the fuzzy model.

The four scaling factors of this fuzzy model are determined

from this learning phase as Gu 0.7476, Gy 0.4727,

Gyy 0.6261, and Go 5.5781.

According to the structure of this fuzzy-neural network, the

number of rules (rule nodes in the third layer) is .

The MMFA described in Section IV-B is used to find the 75

rules of this fuzzy model and the results are shown in Table IV.

The MRD-GA (Section IV-C) is applied to optimize the

parameters of the dynamic-system model. The algorithm pa-rameters are set as follows: pop size , pmut 0.05,

chromosome-length , , ,

, , and . After finishing the second

learning phase and before applying the MRD-GA, the model

has an MSE value of 0.2058. This MSE value is decreased to

0.0374 after 3517 generations using a single point crossover

with pcross value of 0.9 (note that the MSE value reached

0.06 after only 470 generations). The computation time used

to perform this learning process is illustrated in Table V. The

MSE decay rates using different crossover probabilities are

shown in Fig. 6.

After the learning process is finished, the model is tested by

applying a sinusoidal input signal to the

fuzzy model. The output of both the fuzzy model and the actual

model are shown in Fig. 7. The fuzzy model has a good match

with the actual model with a MSE of 0.0403. Another test

is carried out using an input signal .

The result is shown in Fig. 8 and the MSE in this case is

0.0369. After extensive testing and simulations, the fuzzy

model proved a good performance in forecasting the output of

the complex-dynamic plant. Remember that in this example

only 500 data points are used to build the model; while in

[36], 100 000 data points have been used to identify a neuralnetwork model. It can be expected that the performance of the

identified fuzzy model may be further improved if the number

of data points used to build the model is increased.

In order to compare our modeling approach with that of

Sugenos [10], [11] and Wangs [8] approaches, both of

these approaches are implemented. The Sugenos approach is

implemented using the MATLAB fuzzy-logic tool box. The

approach [37] applies the least-squares algorithm (LSA) and

the backpropagation gradient descent method for identifying

linear (consequent) and nonlinear (premise) parameters of

the class A fuzzy rules, respectively. The core function of


10/12


Fig. 6. The MRD-GA convergence rates with different crossover rates.

Fig. 7. Testing of the fuzzy model versus the actual model.

this algorithm is implemented using an-optimized-for-speed

C code. Wangs approach is implemented using the C

programming language. The approach uses the fuzzy C-means

(FCM) clustering algorithm [38] to find the premise parameters

of the class A fuzzy rules, then it applies the least squares

algorithm to find the consequent linear parameters of the rules.

Table VI compares our modeling approach with both of

Sugenos and Wangs approaches. The models are learned

from the previously generated 500 data pairs and tested by

applying a sinusoidal input signal . All the

experiments are carried out on a Pentium 166-MHz PC. The

comparison shows the advantages of our modeling approach.


11/12


Fig. 8. Testing of the fuzzy model versus the actual model.

TABLE IVTHE COMPLETE FAM MATRICES WITH THE FUZZY RULES

VII. CONCLUSION

This paper presents a neuro-fuzzy approach for linguistic

modeling of complex-dynamical systems, in which a proposed

genetic algorithm plays a central role. One of the advantages of

the method presented is that it divides the learning algorithm

into three phases. The initial membership functions are found

in the first phase using Kohonens self organizing feature

maps. In the second phase, a new algorithm is proposed to

extract and optimize the fuzzy rules for the neuro-fuzzy model.

TABLE VTHE COMPUTATION TIME OF EXAMPLE 2

TABLE VI

A MODELING COMPARISON USING EXAMPLE 2

In the third phase, the membership functions are fine-tunedusing the proposed MRD-GA. The performance of the neuro-

fuzzy approach is tested using two benchmarks, and compared

with other models. The approach shows a good performance

in building accurate linguistic fuzzy models.

ACKNOWLEDGMENT

The authors wish to thank the anonymous reviewers for

their valuable and meticulous comments. The guidance and

help of Dr. A. Y. Tawfik at Wilfrid Laurier University are

also gratefully acknowledged.


12/12


REFERENCES

[1] L. Zadeh, Outline of a new approach to analysis of complex systemsand decision processes, IEEE Trans. Syst., Man, Cybern., vol. SMC-3,pp. 2844, 1973.

[2] R. M. Tong, The construction and evaluation of fuzzy models, Advances in Fuzzy Set Theory and Applications, M. M. Gupta, R. K.Ragade, and R. R. Yager, Eds. Amesterdam, The Netherlands: North-Holland, 1979, pp. 559576.

[3] W. Pedrycz, An identification algorithm in fuzzy relational systems,

Fuzzy Sets Syst., vol. 13, pp. 153167, 1984.[4] C. W. Xu and Y. Z. Lu, Fuzzy model identification and self-learningfor dynamic systems, IEEE Trans. Syst., Man, Cybern., vol. 17, pp.683689, 1987.

[5] M. Sugeno and T. Yasukawa, A fuzzy-logic-based approach to qual-itative modeling, IEEE Trans. Fuzzy Syst., vol. 1, pp. 731, Feb.1993.

[6] L. A. Zadeh, Fuzzy sets, Inform. Contr., vol. 8, pp. 338353, 1965.[7] E. H. Mamdani and S. Assilian, An experiment in linguistic synthesis

with a fuzzy logic controller, Int. J. ManMachine Studies, vol. 7, no.1, pp. 113, 1975.

[8] L. Wang and R. Langari, Complex systems modeling via fuzzy logic,IEEE Trans. Syst., Man, Cybern., vol. 26, pp. 100106, Feb. 1996.

[9] W. Pedrycz and J. V. de Oliveira, Optimization of fuzzy models, IEEETrans. Syst., Man, Cybern., vol. 26, no. 4, pp. 627636, Aug. 1996.

[10] M. Sugeno and K. Tanaka, Successive identification of a fuzzy modeland its application to prediction of a complex system, Fuzzy Sets Syst.,vol. 42, pp. 315334, 1991.

[11] T. Takagi and M. Sugeno, Fuzzy identification of systems and itsapplications to modeling and control, IEEE Trans. Syst., Man, Cybern.,vol. 15, 1985.

[12] L. Wang and R. Langari, Building Sugeno-type models using fuzzydiscretization and orthogonal parameter estimation techniques, IEEETrans. Fuzzy Syst., vol. 3, pp. 454458, Nov. 1995.

[13] E. Kim, M. Park, S. Ji, and M. Park, A new approach to fuzzymodeling, IEEE Trans. Fuzzy Syst., vol. 5, pp. 328337, Aug. 1997.

[14] E. Mamdani, Advances in the linguistic synthesis of fuzzy controllers,Int. J. ManMachine Studies, vol. 8, pp. 669678, 1976.

[15] C. T. Lin and C. S. G. Lee, Neural-network-based fuzzy logic controland decision system, IEEE Trans. Comput., vol. 40, pp. 13201336,Dec. 1991.

[16] H. Nomura, I. Hayashi, and N. Wakami, A self-tunning method of fuzzycontrol by descent method, Int. Fuzzy Syst. Assoc., Brussels, Belgium,1991, pp. 155158.

[17] L. X. Wang and J. Mendel, Generating fuzzy rules by learning from

examples, IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 14141427,1992.[18] C. C. Hung, Building a neuro-fuzzy learning control system, AI

Expert, pp. 4049, Nov. 93.[19] J. S. R. Jang and C. T. Sun, Neuro-fuzzy modeling and control, Proc.

IEEE, vol. 83, no. 3, pp. 378406, Mar. 1995.[20] Y. Lin and G. A. Cunningham, A new approach to fuzzy-neural system

modeling, IEEE Trans. Fuzzy Syst., vol. 3, pp. 190198, May 1995.[21] W. A. Farag, V. H. Quintana, and G. Lambert-Torres, Neural-network-

based self-organizing fuzzy-logic automatic voltage regulator for asynchronous generator, in Proc. 28th Annu. NAPS, Cambridge, MA,Nov. 1012, 1996, pp. 289296.

[22] R. Langari and L. Wang, Fuzzy models, modular networks, and hybridlearning, Fuzzy Sets Syst., vol. 79, pp. 141150, 1996.

[23] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Ma-chine Learning. Reading, MA: Addison-Wesley, 1989.

[24] Z. Miachalewicz, GA + Data Structure = Evolution Programming.New York: Springer-Verlag, 1994.

[25] C. L. Karr and E. J. Gentry, Fuzzy control of pH using geneticalgorithms, IEEE Trans. Fuzzy Syst., vol. 1, pp. 4653, Feb. 1993.

[26] D. Park, A. Kandel, and G. Langholz, Genetic-based new fuzzyreasoning models with application to fuzzy control, IEEE Trans. Syst.,

Man, Cybern., vol. 24, pp. 3947, Jan. 1994.[27] A. Homaifar and E. McCormick, Simultaneous design of membership

functions and rule sets for fuzzy controllers using genetic algorithms, IEEE Trans. Fuzzy Syst., vol. 3, May 1995.

[28] H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, Selecting fuzzyifthen rules for classification problems using genetic algorithms, IEEETrans. Fuzzy Syst., vol. 3, pp. 260270, Aug. 1995.

[29] M. Sugeno and G. T. Kang, Structure identification of fuzzy model,Fuzzy Sets Syst., vol. 28, pp. 1533, 1988.

[30] T. J. Ross, Fuzzy Logic with Engineering Applications. New York:McGraw-Hill, 1995.

[31] M. Lee and H. Takagi, Dynamic control of genetic algorithms usingfuzzy logic techniques, in Proc. 5th Int. Conf. Genetic Algorithms, Univ.Illinois, Urbana-Champaign, July 1721, 1993, pp. 7683.

[32] A. N. Aizawa and B. W. Wah, Dynamic control of genetic algorithmsin a noisy environment, Proc. 5th Int. Conf. Genetic Algorithms, Univ.Illinois, Urbana-Champaign, July 1721, 1993, pp. 4849.

[33] J. Kim and B. P. Zeigler, Designing fuzzy logic controllers using amultiresolutional search paradigm, IEEE Trans. Fuzzy Syst., vol. 4, pp.213226, Aug. 1996.

[34] G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting andControl, 2nd ed. San Francisco, CA: Holden-Day, 1976.

[35] R. M. Tong, Synthesis of fuzzy models for industrial processesSomerecent results, Int. J. General Syst., vol. 4, pp. 143162, 1978.

[36] K. S. Narendra and K. Parthasarathy, Identification and control of dy-namical systems using neural networks, IEEE Trans. Neural Networks,vol. 1, Mar. 1990.

[37] J.-S. R. Jang, ANFIS: Adaptive-network-based fuzzy inference sys-tem, IEEE Trans. Syst., Man, Cybern., vol. 23, pp. 665685, 1993.

[38] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algo-rithms. New York: Plenum, 1981.

Wael A. Farag received the B.Sc. and M.Sc. de-grees from Cairo University, Egypt, in 1990 and1993, respectively, both in electrical engineering.Currently, he is a Ph.D. candidate in the Electricaland Computer Engineering Department at the Uni-versity of Waterloo, Ontario, Canada.

From 1990 to 1994, he worked as an AssistantLecturer in the Electrical Engineering Departmentat Cairo University. His research interests includemicroprocessor-based systems, fuzzy logic and neu-ral networks, evolutionary algorithms, and control

systems.

Victor H. Quintana (M73SM80) received the

Dipl. Ing. degree from the State Technical Univer-sity of Chile in 1959, the M.Sc. degree in electricalengineering from University of Wisconsin, Madi-son, in 1965, and the Ph.D. degree in electricalengineering from the University of Toronto, Ontario,Canada, in 1970, respectively.

Since 1973, he has been with the University ofWaterloo, Department of Electrical and ComputerEngineering, where he is currently a Full Professor.His main research interests are in the areas of

numerical optimization techniques, state estimation, and control theory asapplied to power systems.

Dr. Quintana is an Associate Editor of the International Journal of EnergySystems and a member of the Association of Professional Engineers of theProvince of Ontario.

Germano Lambert-Torres received the Ph.D. degree in electrical engineeringfrom Ecole Polytechnique de Montreal, Canada.

Since 1983, he has been with the Escola Federal de Engenharia de Itajuba(EFEI), Brazil, where he is a Full Professor and Associate Chairman ofgraduate Studies. During the 1995 to 1996 academic year, he was a VisitingProfessor at University of Waterloo, Canada. He has served as a consultant formany power industries in South-America countries. He has published morethan 200 chapter books, journal articles, and conference papers on intelligentsystems applied to power system problem solving.

a genetic-based neuro-fuzzy approach for modeling and control of dynamical systems

Documents