Co-Evolution: An approach to automatic generation of Fuzzy Systems

Anderson Francisco Talon Federal University of São Carlos –

UFSCar anderson_talon@dc.ufscar.br

Heloisa de Arruda Camargo Federal University of São Carlos –

UFSCar heloisa@dc.ufscar.br

Abstract

This work focus on the problem of automatic generation of fuzzy systems by means of evolutionary computation, specifically using the approach of co-evolution. Co-evolution is based on the idea of modular modeling of the problem subcomponents. In this work the subcomponents are represented by species, which have a collaborative relation among them. The fuzzy system to be created performs fuzzy pattern classification. Basically, the environment is composed by four different species, which have a hierarchical collaboration both in the generation of the species and in the fitness determination of the individuals of these species. These species are organized in levels, where the contribution in the specie generation happens from the lowest to highest levels and the contribution in the fitness determination happens from the highest to lowest levels. The fitness calculation includes evaluations of rules compactness, what was demonstrated to improve the system interpretability. 1. Introduction

The solution of practical problems many times requires the construction of hybrid systems, capable of integrating techniques and methodologies from different research areas. The Soft Computing, which embraces fuzzy systems, neural networks and evolutionary computation, is a promising alternative for the generation of hybrid systems.

The fuzzy systems [1] have been successfully applied to several areas, due to its ability to properly represent and process knowledge that is naturally imprecise. Since these systems do not provide an inherent mechanism for automatic construction from data sets, there has been a growing interest in extending fuzzy systems with learning capabilities. In the context of soft computing the attempts to create hybrid approaches aiming at constructing fuzzy systems originated two successful approaches: the neural fuzzy systems and the genetic fuzzy systems.

Genetic fuzzy systems are fuzzy systems extended by a learning procedure based on Genetic Algorithms.

Genetic algorithms were introduced by John Holland [2]. His initial objective was to study the adaptation phenomena as is occur in nature and extract its mechanisms to reproduce it in computational systems.

Genetic algorithms have been used in the task of fuzzy knowledge base definition from several points of view, such as: fuzzy rules definition, reduction on the number of previously defined rules, tuning of fuzzy sets by the parameters optimization, redundancy elimination, among others. A collection of significant and complete approaches and new trends can be found in [3]. There are different possibilities to genetic fuzzy systems design, such that some of them are called the classical approaches and others explore new directions. Among the new directions reported in the literature, the one that uses coevolutionary algorithms has proven to be prominent. The main idea of the coevolutionary approach is the co-adapted evolution of different species influenced by the interaction among the species.

The objective of the work described here is to investigate a coevolutionary approach for fuzzy systems generation, which is based on the developments described in [4, 5, 6] and focus on the generation of fuzzy classification systems, as opposed to the original work, that investigates mainly problems of function approximation.

This paper is organized as follows: section 2 describes the coevolutionary genetic systems developed in this work; section 3 presents the experimental results and section 4 discusses conclusions and future work. 2. Coevolutionary Classification System

In traditional genetic algorithm, the complete solution to a problem is coded in one individual and the population evolves in an isolated way. The individuals in one population compete among each other and there is no stimulus for co-adaptation. An interesting alternative approach to this kind of

Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06)0-7695-2662-4/06 $20.00 © 2006

representation is the coevolutionary approach, where parts of the solution are represented in different populations called species in the coevolutionary jargon. These populations are designed in a way that promotes the coevolution through mutual adaptation. The coevolution approach presents a number of advantages with relation to the single population version, since it is conceived based on the idea of splitting the problem in subproblems, avoiding the search space explosion for each population individually and because of the exploration of the collaboration mechanism among populations, leading the individuals to co-adapt in the search for the best solution. The general structure of the learning system adopted here was proposed in [4, 5, 6], where the main focus was in the generation of TSK fuzzy models and in function approximation. The approach developed in the work here described is dedicated to the automatic generation of fuzzy classification systems.

The coevolutive approach used here adopts distinct populations organized in a hierarchical way in evolutive modulus. Individuals in different populations, representing four distinct species, code different parts of the fuzzy system. The four populations are required so that all parameters in the fuzzy system can be defined automatically, in a collaborative way. The learning system includes:

• A population of fuzzy partitions in the first level (I);

• A population of fuzzy rules in the second level (II);

• A population of fuzzy rule bases in the third level (III);

• A population of fuzzy systems in the forth level (IV).

Individuals in upper levels are designed from individuals in the lower levels. The individuals in level IV (fuzzy systems) include a fuzzy rule base from level III and a fuzzy partition form level I. The chromosomes in this level also has a gene that codes the reasoning method used in the classification, which could be either the classical fuzzy reasoning method or the general fuzzy reasoning method and a gene that codes the t-norm used in the reasoning method, which could be the minimum, algebraic product or limited difference operators. As long as the populations are genetically isolated, each one has a genetic algorithm in the background, which is responsible for the population evolution. Although, all populations belong to the same environment, the fitness of individuals are determined together according to the application domain. The structure of the environment and the defined populations can be seen in Fig. 1.

In the process of fitness calculation for the individuals belonging to different species, relations among modules are established: the fitness of one

individual is calculated based on the fitness of individuals belonging to other populations.

Fig. 1: Coevolutive model

Fig. 2 illustrates the influence of the fitness value

of individuals in one population in the fitness calculation of individuals in other populations, according to the hierarchical organization of the populations in the environment. The fitness values are evaluated from the higher levels to the lower levels.

Fig. 2: Hierarchical relation among populations

According to the scheme represented in Figure 2,

the fitness values are calculated in the following sequence:

• Fitness evaluation of individuals in level IV (Fuzzy System). The fitness value is calculated based on the performance of the system in the classification task, meaning that the best individuals are the ones that posses the higher classification rates;

• Fitness evaluation for individuals in level III (Fuzzy Rule base). The fitness value for an individual in this level is calculated from the highest fitness value among the values of individuals in level IV (fuzzy systems) that use the rule base represented by this individual and a value inversely proportional to the number of fuzzy rules in this rule base. With this formula, the best rule bases are the ones that participate in the fuzzy systems with best performance and contain a small number of rules;

• Fitness evaluation for individuals in level II (Fuzzy Rule). The fitness value for an

Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06)0-7695-2662-4/06 $20.00 © 2006

individual in level II is calculated based on the average value of all fitness values of the individuals in level III (fuzzy rule bases) that use that rule represented by this individual and a value inversely proportional to the number of linguistic variable appearing in the rule antecedent. This calculation will assign a higher fitness value to the rules with a lesser number of variable, or the more general rules;

• Fitness evaluation for individuals in level I (Fuzzy Partition). The fitness value for an individual in this level is calculated based on the highest fitness value of the individuals in level IV (Fuzzy Systems) that use the fuzzy partition represented by this individual and a value inversely proportional to the number of sets in each fuzzy partition. The best individuals are the ones that appear in the best fuzzy systems, with a small number of sets in each partition.

The additional values inversely proportional to the number of rules, variables and sets used in the fitness calculation for individuals in populations III, II and I respectively, are supposed to guide the search in the direction of more compact and clear knowledge bases, since it forces the reduction of the number of rules, variables and sets. 3. Experimental Results

The learning system based on coevolution describe in this paper were run on two data sets: the iris data set [7] and the concentrical spirals data set.

For all results presented here, the parameters were defined empirically in previous experiments and fixed at 10% for the mutation rate, 90% for the crossover rate and 5% for the elitism rate for all populations.

The iris data set is one of the most used in inductive machine learning [8]. The data set contains three classes, each one with 50 instances, characterizing one type of plant from the Iris species. The instances are described by 4 numerical and continuous attributes related to measures from plants belonging to each class. The attributes and classes are:

• First Attribute: sepal length (SL) in cm; • Second attribute: sepal width (SW) in cm; • Third attribute: petal length (PL) in cm; • Fourth attribute: petal width (PW) in cm; • Class: Iris Setosa, or Iris Versicolor, or Iris

Virginica. The data set was split in training and test set at a

50% rate. The experiments on this data set were run with populations size:

• Population I size: 10; • Population II size: 240; • Population III size: 70;

• Population IV size: 30. The maximum number of rules in a rule based

was limited to 4 and the number of generations was defined as 100. The fuzzy partitions were defined with a maximum of two fuzzy sets.

The best fuzzy system found uses the classical fuzzy reasoning method, and the standard intersection (min) as the t-norm to calculate the compatibility degree between a pattern and a rule antecedent in the reasoning process.

The fuzzy system presents the fuzzy partitions shown in Fig. 3 and the following rule base:

IF SL is SMALL AND PL is SMALL AND PW is SMALL THEN Class = Iris Setosa

IF SL is SMALL AND SW is SMALL AND PW is SMALL THEN Class = Iris Versicolor

IF SL is BIG AND SW is BIG AND PL is SMALL THEN Class = Iris Virginica

IF SL is BIG AND PL is BIG THEN Class = Iris Virginica

Fig. 3: Fuzzy partitions for the iris data set

The best fuzzy system found for iris data set

misclassified only 2 instances and, from a total of 50 runs, a obtained a precision of 94,28% and a standard deviation of 1,10%.

A comparison among approaches found in literature with iris data set can be seeing in Table 1. Table 1: Results in iris data set. Approach Rules Misclassified ANFIS [9] 16 6 CoevolGFS [4, 5, 6] 4 3 CoESys* 4 2 * Co-Evolutionary Classification System.

The other data set used was the concentrical spirals data set. The problem of concentrical spirals is very popular in the literature when the subject is classification. The main goal is classify the right points between two concentrical spirals that begin in point (0.0). The Fig. 4 illustrates both spirals.

Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06)0-7695-2662-4/06 $20.00 © 2006

Fig. 4: Concentrical spirals

The concentrical spirals data set was created with

97 instances to each spiral. All instances (194) were used in the training phase. The experiments on this data set were run with populations size:

• Population I size: 20; • Population II size: 30; • Population III size: 80; • Population IV size: 100. The maximum number of rules in a rule base was

limited to 10 and the number of generations was defined as 529. The fuzzy partitions were defined with a maximum of three fuzzy sets.

The best fuzzy system found uses the general fuzzy reasoning method, and the limited difference as the t-norm to calculate the compatibility degree between a pattern and a rule antecedent in the reasoning process.

The fuzzy system presents the fuzzy partitions shown in Fig. 5 and the following rule base:

IF X1 is LOW THEN +1 IF X1 is MEDIUM THEN +1 IF X2 is MEDIUM THEN –1 IF X1 is LOW AND X2 is MEDIUM THEN +1 IF X1 is MEDIUM AND X2 is LOW THEN –1 IF X1 is MEDIUM AND X2 is HIGH THEN –1 IF X1 is HIGH AND X2 is LOW THEN –1 IF X1 is HIGH AND X2 is MEDIUM THEN +1 IF X1 is HIGH AND X2 is HIGH THEN –1

Fig. 5: Fuzzy partitions for the spirals data set

The best fuzzy system found to concentrical data

set, classified correctly all instances and, from a total of 10 runs, obtained a precision of 97,83% and a standard deviation of 1,02%. 4. Final Considerations

This article described a coevolutionary approach for the generation of fuzzy rule based systems. The coevolution approach presents a number of

advantages with relation to the single population version (traditional AG), since it is conceived based on the idea of splitting the problem in subproblems, avoiding the search space explosion for each population individually and because of the exploration of the collaboration mechanism among populations, leading the individuals to co-adapt in the search for the best solution. The results obtained have demonstrated that this is a promising approach to solve the problem of knowledge bases generation from instance data sets in the classification subject. The final analysis also indicates that the knowledge bases generated through this process preserve interpretability, while achieving a high performance rate. In future works the authors plan to investigate the system performance with larger data sets, and also compare the developed approach to others found in the literature. 5. References [1] G. Klir, B. Yuan, “Fuzzy Sets and Fuzzy Logic – Theory and Applications”, Prentice-Hall, 1995. [2] J. H. Holland, “Adaptation in Natural and Artificial Systems”, 1975, University of Michigan Press. [3] O. Cordón, F. Gomide, F. Herrera, F. Hoffman, L. Magdalena (eds.) “Genetic Fuzzy Systems: new developments”, (special issue) Fuzzy Set and Systems 141, 2004. [4] M. R. Delgado, F. V. Zuben, F. Gomide, “Hierarchical Genetic Fuzzy Systems”, Information Sciences – Special Issue on Recent Advances in Genetic Fuzzy Systems, 2001, Vol. 136, N. 1-4, pp. 29-52. [5] M. R. Delgado, F. V. Zuben, F. Gomide, “Coevolutionary Design of Takagi-Sugeno Fuzzy Systems”, IEEE, 2002. [6] M. R. Delgado, F. V. Zuben, F. Gomide, “Coevolutionary genetic fuzzy systems: a hierarchical collaborative approach”, Fuzzy Sets and Systems 141, 2004, pp. 89-106. [7] C. J. Merz, P. M. Murphy, “UCI repository of machine learning databases”. Irvine, CA: University of California, Department of Information and Computer Science, [http://www.ics.uci.edu/~mlearn], 1998. [8] R. O. Duda, P. E. Hart, “Pattern Classification and Scene Analysis”, John Wiley & Sons, New York, 1973. [9] J. S. Jang, “ANFIS: Adaptive-Network-based Fuzzy Inference Systems”, IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, N. 3, pp. 665-685, 1993.

Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06)0-7695-2662-4/06 $20.00 © 2006