on direct construction of fuzzy systems

Fuzzy Sets and Systems 112 (2000) 165–171www.elsevier.com/locate/fss

On direct construction of fuzzy systemsMing Ma a, Yanqing Zhang a, Gideon Langholz b, Abraham Kandelb;∗;1

aComputer Science and Engineering Department, University of South Florida, 4202 East Fowler Avenue, Tampa,FL 33620-5350, USA

bDepartment of Electrical Engineering-Systems, Tel-Aviv University, Tel-Aviv 69978, Israel

Received April 1997; received in revised form November 1997

Abstract

In this paper, we show that the fuzzy logic system presented by Wang and Mendel (1992) is a linear expansion of thefuzzy rule base if fuzzy events are represented by triangular basis functions. Therefore, based on linear regression theory, wepropose a direct approach for constructing a fuzzy logic system from input–output data. The e�ectiveness of this approachis demonstrated by simulations. c© 2000 Elsevier Science B.V. All rights reserved.

Keywords: Fuzzy logic systems; Fuzzy rule base; Linear regression; Linear expansion; Linguistic variables

1. Introduction

Fuzzy logic systems have been successfully appliedin many �elds. The issue of how to form a fuzzy rulebase directly from a given data set has attracted manyresearchers [2–6, 8, 10–15]. Here, we consider thisissue by showing how to construct a piecewise linearfuzzy logic system with a minimal number of fuzzyrules under a given squared error.Various di�erent learning (non-constructive) ap-

proaches, such as in [6, 8] were applied to the con-struction of fuzzy rule bases representing a givenfuzzy logic system. However, these non-constructiveapproaches may not guarantee �nding proper fuzzy

∗ Corresponding author. Tel: +1-813-974-4232; fax: +1-813-974-5456;E-mail address: [email protected]. (A. Kandel)1 On leave from the Department of Computer Science and

Engineering, University of South Florida, Tampa, FL33620, USA.

rules for a given squared error. On the other hand, aconstructive approach, based on uniform continuityof the process f for constructing a piecewise linearfuzzy logic system, was proposed in [12]. Unfor-tunately, this constructive approach depends on theprocess f rather than on the fuzzy logic system itself,and is not e�ective in constructing a piecewise lin-ear fuzzy logic system with near-optimal or optimalnumber of fuzzy rules.In order to overcome the weaknesses of the

constructive approach in [12], we propose a directconstructive approach for constructing a single-input–single-output piecewise linear fuzzy logic systemwith near-optimal number of fuzzy rules under a �xedsquared.In Section 2, we recall some preliminaries on

fuzzy logic systems and present a direct construc-tive approach based on linear regression. The directconstructive algorithm is considered in Section 3. InSection 4, several simulation examples are presented,followed by the conclusions in Section 5.

0165-0114/00/$ - see front matter c© 2000 Elsevier Science B.V. All rights reserved.PII: S 0165 -0114(97)00387 -4

166 M. Ma et al. / Fuzzy Sets and Systems 112 (2000) 165–171

2. Constructing fuzzy logic systems

We �rst recall some basic concepts regarding fuzzylogic systems.A fuzzy rule base consists of a collection ofm fuzzy

IF–THEN rules in the following form:

R(k): If x1 is Ak1 and : : : and xn is Akn ;

THEN y is Gk; (1)

where Aki and Gk are fuzzy sets in Ui⊂R and V ⊂R,

respectively, and �x=(x1; x2; : : : ; xn)T ∈U1×U2× · · ·×Un and y∈V are linguistic variables [13] referredto as the input and output to the fuzzy logic system,respectively.For defuzzi�cation, one may use the center average

defuzzi�er de�ned as

y=∑m

k=1 yk�Gk (yk)∑m

k=1 �Gk (yk)

; (2)

where yk is the center of the fuzzy set Gk , that is, thepoint in V at which Gk(y) achieves its maximum.It was showed in [13] that fuzzy logic systems with

center average defuzzi�er, product-inference rule, andsingleton fuzzi�er are given by the following form:

f( �x)=

∑mk=1 y

k [∏ni=1 �Aki (xi)]∑m

k=1[∏ni=1 �Aki (xi)]

: (3)

In the following, we will show that the fuzzy logicsystem of Eq. (3) is a linear expansion of �red fuzzyrules if the membership functions �Aki and �Gk arerepresented by triangular fuzzy sets. We only considerthe single-input–single-output case.Let a0 =infU and an+1 =supU , namely, U = [a0;

an+1]. Suppose that

a06a1¡a2¡ · · ·¡an6an+1; (4)

and

�Ai(x)=

x − ai−1ai − ai−1 ; x∈ [ai−1; ai];ai+1 − xai+1 − ai ; x∈ [ai; ai+1];0 otherwise:

(5)

For any x∈U , there exist i0 and � such thatai06x¡ai0+1; x= �ai0 + (1− �)ai0+1: (6)

Therefore, for a given x only the following two rulesin the fuzzy rule base are �red:

If x is Ai0 ; then y is bi0 ; (7)

If x is Ai0+1; then y is bi0+1: (8)

This implies that the fuzzy logic system of Eq. (3) canbe simpli�ed as

g(x)=∑m

i=1 bi�Ai(x)∑mi=1 �Ai(x)

(9)

=bi0�Ai0 (x) + bi0+1�Ai0+1 (x)

�Ai0 (x) + �Ai0+1 (x)(10)

= �bi0 + (1− �)bi0+1: (11)

Since the fuzzy logic system given by Eq. (3) canbe simpli�ed as a linear expansion of fuzzy rules,the question of how to use a given input–output dataset {(xi; yi): i=1; 2; : : : ; N} in order to form a fuzzyrule base becomes the issue of how to choose typ-ical points {(ak ; bk): k =1; 2; : : : ; m} such that thepiecewise linear expansion of (ak ; bk) can approximate{(xi; yi): i=1; 2; : : : ; N} as close as possible. Practi-cally, we hope that the fuzzy rule base contains asfew as possible fuzzy rules with the same approxima-tion error. To do so, we suggest a direct constructiveapproach based on linear regression.Following [9], assume that N pairs of observations

(x1; y1); : : : ; (xN ; yN ) have been collected on an inde-pendent variable X and a dependent variable Y , wherexi may be either distinct or not. The linear modelrelating Y to X is given by

yi= � + �(xi − �x) + ei; i=1; 2; : : : ; N; (12)

where

�x=1N

N∑i=1

xi: (13)

The constraint that the least-squares estimators of �and � follow is given by

S(�; �)=N∑i=1

[yi − � − �(xi − �x)]2: (14)

Since the ei’s are uncorrelated and have the same vari-ance, an unweighted sum of squared deviations seems

M. Ma et al. / Fuzzy Sets and Systems 112 (2000) 165–171 167

reasonable. Partial di�erentiation leads to the estima-tors

�0 =1N

N∑i=1

yi= �y; (15)

and

�0 =∑N

i=1(yi − �y)(xi − �x)∑Ni=1(xi − �x)2

: (16)

Hence, the least-squares predicted value of Y corre-sponding to X = xi is

Yi= �y + �0(xi − �x): (17)

The deviations of the values of the observed dependentvariable from their predictions are denoted by

zi=yi − Yi: (18)

Suppose the set {(xi; yi): i=1; 2; : : : ; N} is ourinput–output observation data. Without loss ofgenerality, suppose that xi¡xi+1. Let � be anapproximation error threshold. We begin withthe observation (x1; y1). For the observations(x1; y1); (x2; y2); : : : ; (xi; yi), the linear regressionthrough (x1; y1) is given by

Y = � + �(x − �x); (19)

with

�=y1 − �(x1 − �x); (20)

�=∑i

k=1[yk − y1][xk − x1]∑ik=1[xk − x1]2

; (21)

where

�x=1i

i∑k=1

xk : (22)

Denote

Ei= max16k6i

|zk | (23)

and iterate i from 2. Find i1 such that Ei1¡� andEi1+1¿�, then execute linear regression of the obser-vations (xi1+1; yi1+1); : : : ; (xi; yi) through (xi1 ; Yi1 ), andrepeat the above procedure until (xN ; yN ). Then, basedon {(x1; y1); (xi1 ; Yi1 ); : : : ; (xip ; Yip); (xN ; yN )}, we can

form the fuzzy rule base of the fuzzy logic system,represented by Eq. (3), that approximates the observa-tions {(xi; yi): i=1; 2; : : : ; N} with an error less thanthe given threshold �. The constructive algorithm ispresented in the next section.

3. A constructive algorithm

Assuming that there are N data pairs (x[i]; y[i]);i=1; 2; : : : ; N , and that the required maximum squarederror is Em. The direct construction algorithm is de-scribed below, wherek is the counter of fuzzy rules.p is the pointer to point the �rst data pair

(x[p]; y[p]) for the current linear regression function.q is the counter of used data pairs.n is the counter of current data pairs used in the

linear regression.Ec is the current squared error.E is the total squared error.�E is the mean squared error.� is the parameter of the current linear regression

function.

Step 1 (Initialization): k =0; p=1; �=y[1];E=0:0.Step 2: Set: n=1; k = k + 1.Step 3: n= n+ 1.Calculate:

�=

∑p+ni=p (� − y[i])(x[p]− x[i])∑p+n

i=p (x[p]− x[i])2: (24)

Then calculate n values of the current linear regressionfunction:

f[i] = � + �(x[i]− x[p]); (25)

where i=p;p + 1; : : : ; p + n. Finally, calculate thecurrent squared error:

Ec =p+n∑i=p+1

(f[i]− y[i])2: (26)

Step 4: Set: q=p + n. Calculate the total squarederror:

E=E + Ec: (27)


Then calculate the mean squared error:

�E=Eq: (28)

If �E6Em, store the parameters of the currentlinear regression function: a[k] = �; b[k] = � andc[k] = x[p], and goto Step 5.Else goto Step 6.Step 5: If q6N , then goto Step 3.Else goto Step 6.Step 6: Set: p= q− 2 and

�= b[k] + a[k](x[p]− c[k]): (29)

Step 7: If q6N , then goto Step 2.Else goto Step 8.Step 8: Set: c[k + 1]= x[N ]. The �nal constructed

fuzzy system is given by

f(x)= b[i] + a[i](x − c[i]); (30)

where x∈ [c[i]; c[i + 1]] for i=1; 2; : : : ; k. The inputmembership functions are given by

�A1 (x)

=

{1− x−c[1]

c[2]−c[1] for c[1]6x6c[2];

0 otherwise;(31)

�Aj (x)

=

x−c[ j−1]c[ j]−c[ j−1] for c[j − 1]6x6c[j];1− x−c[ j]

c[ j+1]−c[ j] for c[j]6x6c[j + 1];

0 otherwise;

(32)

�Ak (x)

=

{ x−c[k]c[k+1]−c[k] for c[k]6x6c[k + 1];

0 otherwise;

(33)

where j=2; : : : ; k.Step 9: End.

Compared with the constructive approach proposedin [12], the algorithm here depends only on certain dis-crete input–output data instead of on a continuous pro-cess. Since the system has knowledge only on certain

points (xi; yi); i=1; 2; : : : ; N , and is a piecewise lin-ear expansion of fuzzy rules, the proposed algorithmhas the advantage that it will stop partitioning fuzzyrules when the linear approximation is satis�ed in thesubinterval. This will reduce the number of fuzzy rulesin the rule base. Another advantage of the algorithmis that the created linguistic terms represented by par-ticular triangular membership functions in the fuzzyrule base have easily interpreted physical meanings.

4. Simulations

4.1. A nonlinear function approximation

We consider a simple function y=1=x on [a; b] fora¿0. The constructive approach in [12] requires atleast 190 rules to approximate the process even if theerror threshold is �=0:5. Actually, in order to obtain∣∣∣∣f(x)− f

(110

)∣∣∣∣¡�; (34)

for �=0:5, we require x¡ 19:5 , that is,

�6 19:5 − 1

10 (35)

= 1190 : (36)

Therefore, the number N of rules is

N¿[1�

]=190: (37)

We have implemented the new direct constructivealgorithm in order to construct the fuzzy system f(x).The simulation results, shown in Table 1 and in Figs. 1and 2, clearly indicate that our fuzzy system is moree�cient than the fuzzy system in [12] and can ef-fectively approximate a nonlinear function under any

Table 1Comparisons between the direct constructive fuzzy model and thefuzzy model in [12]

Absolute error (E) Number of rules Number of rulesFuzzy system in [12] Our fuzzy system

0.05 190 90.10 190 70.25 190 50.50 190 4


Fig. 1. Observations (dashed line) and predicted values (solid line) generated by our fuzzy system with 5 fuzzy rules.

Fig. 2. Observations (dished line) and predicted values (solid line) generated by our fuzzy system with 7 fuzzy rules.


Fig. 3. Observations (dashed line) and predicted values (solid line) generated by our fuzzy system with 44 fuzzy rules.

accuracy for any required error under the absolute er-ror criteria de�ned by

EA= max0:16x610

|Y (x)− f(x)|: (38)

4.2. Box and Jenkins’s gas furnace modelidenti�cation

Box and Jenkins’s gas furnace data [1] is a fre-quently used benchmark for checking performancesof both non-fuzzy and fuzzy logic identi�cation algo-rithms. The data set has 296 pairs of inputs (gas owrates) and outputs (concentrations of CO2).The simulation results, under the absolute error

EA= max16k6296

|Yk − yk | (39)

and the mean squared error

E=1296

296∑k=1

(Yk − yk)2; (40)

shown in Table 2 and Fig. 3 indicate that our fuzzysystem is able to perform complex nonlinear tasks by

Table 2Direct constructive fuzzy models with di�erent rules for givenabsolute errors

Absolute error (EA) Mean squared error (E) Number of rules

1.0 0.2542 440.9 0.2054 460.8 0.1517 460.7 0.1313 530.6 0.1023 580.5 0.0644 650.4 0.0474 750.3 0.0239 890.2 0.0106 1200.1 0.0016 180

using piecewise linear functions and that it also canapproximately �nd important turning points at whichan increasing (decreasing) function changes to a de-creasing (increasing) function (see Fig. 2). Impor-tantly, for any given absolute error (or mean squarederror), our algorithm can directly construct a piece-wise linear fuzzy system with the near-optimal num-ber of fuzzy rules according to the piecewise linearregression analysis.


5. Conclusions

Due to their essential nonlinearity, many proposedfuzzy logic systems use a variety of learning algorithmin order to construct the fuzzy rule base. In order toobtain a fuzzy rule base with a clear physical meaning,Wang et al. suggested the constructive approach asdescribed in [12]. Unfortunately, the number of rulesproduced by that approach is predetermined by theprocess f rather than by the constructive procedure,resulting in many redundant rules. By making use ofthe properties of a piecewise linear fuzzy logic sys-tem, the new constructive approach proposed in thispaper can generate an e�cient fuzzy rule base. Thee�ciency of our approach is clearly demonstrated bythe examples provided in the paper. Although the sim-ulation Example 4.2 shows that all of the constructivefuzzy rules are almost centered at the local maximumand minimum, the simulation Example 4.1 indicatesthat it is not a natural law of our constructive approach.This demonstrates the di�erence between time-seriesapproach and our approach.

References

[1] G.E.P. Box, G.M. Jenkins, Time Series Analysis, Forecastingand Control, Holden Day, San Francisco, 1976.

[2] J.J. Buckley, Y. Hayashi, Numerical relationships betweenneural networks, continuous functions, and fuzzy systems,Fuzzy Sets and Systems 60 (1993) 1–8.

[3] S.S.L. Chang, L. Zadeh, On fuzzy mapping and control, IEEETrans. System, Man and Cybernet. 2 (1972) 30–34.

[4] D. Driankov, H. Hellendoorn, M. Reinfrank, An Introductionto Fuzzy Control, Springer, Berlin, 1993.

[5] B. Heshmaty, A. Kandel, Fuzzy linear regression and itsapplications to forecasting in uncertain environments, FuzzySets and Systems 15 (1985) 159–191.

[6] J.-S.R. Jang, ANFIS: adaptive-network-based fuzzy inferencesystem, IEEE Trans. System, Man Cybernet. 23 (1993)665–685.

[7] A. Kandel, Fuzzy Mathematical Techniques withApplications, Addison-Wesley, New York, 1986.

[8] C.C. Lee, Fuzzy logic in control systems: fuzzy logiccontrollers. Part I and II, IEEE Trans. System, Man Cybernet.20 (1990) 404–435.

[9] D.F. Morrison, Applied Linear Statistical Methods, Prentice-Hall, Englewood Cli�s, NJ, 1983.

[10] W. Siler, H. Ying, Fuzzy control theory: the linear case,Fuzzy Sets and Systems 33 (1989) 275–290.

[11] T. Takagi, M. Sugeno, Fuzzy identi�cation of systems and itsapplications to modeling and control, IEEE Trans. System,Man Cybernet. 15 (1985) 116–132.

[12] P. Wang et al., Constructing fuzzy systems, Proc. VI IFSACongr. San Paulo, 1995.

[13] L.-X. Wang, J.M. Mendel, Fuzzy basis functions, universalapproximation and orthogonal least-squares learning, IEEETrans. Neural Networks 3 (1992) 807–814.

[14] L.-X. Wang, Adaptive Fuzzy Systems and Control Designand Stability Analysis, PTR Prentice-Hall, 1994.

[15] X.-J. Zeng, M.G. Singh, Approximate theory of fuzzy systems– SISO case, IEEE Trans. Fuzzy Systems 2 (1994) 162–176.

on direct construction of fuzzy systems

Documents