xcs: current capabilities and future challenges

XCS: Current capabilities and futurechallenges

Martin V. ButzDepartment of Cognitive Psychology (III)

University of Würzburg, Germanyhttp://www-illigal.ge.uiuc.edu/~butz

[email protected]

02/23/2006

05/16/2006 XCS: Current Capabilities & Future Challenges 2Martin V. Butz

Overview

1. The XCS Classifier System

2. XCS - Capabilities

3. XCS - Future Challenges

4. Summary & Conclusions



1. The XCS Classifier System1. Framework

2. Evolutionary Pressures

3. Computational Complexity

4. General Learning Intuition2. XCS – Capabilities

3. XCS – Future Challenges



The XCS Classifier System1 The XCS Classifier System

• Learning classifier system– Rule-based representation of condition-action-predictions– Steady-state GA for evolution of conditions– Gradient-based techniques for estimation of predictions

• Major characteristics:– Q-learning based reinforcement learning– Relative accuracy-based fitness– Action-set restricted selection, that is, niche selection– Panmictic (population-wide) deletion

Goal:

Learn a complete maximally accurate,maximally general predictive model.


Rule Evaluation

• Gradient-based techniques for derivation of predictionand error estimates

• Q-learning derived update

• Propagation of reward possible

• Rule quality depends on inverse of error estimate

• Accuracy of rule prediction determines fitness

1 The XCS Classifier System


Evolutionary Algorithm

• Fixed population size

• Steady-state genetic algorithm

• Two reproductions and deletions per iteration– Reproduction in action set based on fitness

– Deletion from whole population based on coverage

• Genetic operators:– Mutation

– Recombination



Learning Suitability

• XCS represents its solution by a collection of sub-solutions(that is, the population of classifiers).

• XCS learns an effective problem space clustering(subspaces) in its conditions.

• Clusters (subspaces) evolve to enable maximally accuratepredictions.– Accuracy can be bounded (error threshold ε0 and population size

relation).– Basically any form of prediction is possible (e.g. reward, next

sensory input, function value).



XCS Power

• Combination of– Gradient-based techniques (to generate predictions)– Evolutionary techniques (to generate features / clusters)

• Advantage:– Usage of gradient-based error-feedback learning where possible– Usage of evolutionary techniques

• Where error-feedback is hard or impossible to propagate into• Where error-feedback learning gets easily stuck in local optima

• Thus:– Combine best approximation technique (error-feedback learning) with

best evolutionary technique (representation and operators)



2. XCS – Current Capabilities


2. XCS – Current Capabilities

1. Binary and Real-world Classification Problems

2. Function Approximation Problems

3. (Multistep) Reinforcement Learning Problems3. XCS – Future Challenges



The Multiplexer Problem

……

0110000

0100101

1100010

0101101

0011011

0000111

1001000

0000000

ClassProblem instance

101000010##0#

……………

101000101#1##

100001#1##

100101#0##

101000001#0##

1010001001###

1000001###

1001000###

1010000000###

FεRAC

Optimal solution representation

2 XCS – Current Capabilities


XCS Performance in MP 702 XCS – Current Capabilities


Hierarchical Classification Problem

• Hierarchical problems with low order dependencies(“building blocks”) and further high-order dependencies

• BB structures are re-used on the higher level to deriveproblem class.

• Example: Hierarchical 3-parity, 6-multiplexer problem:

2 XCS – Current Capabilities


XCS/BOA Performance2 XCS – Capabilities


Classification of Real-World Datasets

• Conditions are coded with attributes dependent on type ofattribute in dataset (interval coding or Binary coding).

• Experiments in 42 datasets (from UCI and other sources)• Comparisons with ten other ML systems (pairwise t-test)• XCS learns competitively but it is a much more general

learning system.

24/99/149/189/1514/77/619/125/930/138/095%

23/88/139/179/1113/75/619/125/829/138/099%

SMO

(rad.)

SMO

(pol.3)

SMO

(poly)

IB3IB1PARTNaïve

Bayes

C4.51-RMaj.XCS

2 XCS – Capabilities


Piecewise Linear Function Approximation

0 0.2 0.4 0.6 0.8 1x

0 0.2 0.4 0.6 0.8 1

y

-1-0.8-0.6-0.4-0.2

0 0.2 0.4 0.6 0.8

1

f(x,y)

2 XCS – Capabilities


Performance in 3D Sinusoidal Function2 XCS – Capabilities


Performance Evaluation in Maze62 XCS – Capabilities


Performance Maze6 plus Irrelevant Bits2 XCS – Capabilities


3. XCS – Future Challenges


2. XCS – Performance Demonstration

3. XCS – Future Challenges1. Representation & Operators

2. Niching

3. Dynamic Problems

4. Compactness of solution / population

5. Hierarchical classifier systems4. Summary & Conclusions


Representation & Operators

• Different representations of conditions– Binary, real-valued, mixed– Kernels, bases– Combinations, Hybrids

• Different representations of predictions– Constant, Linear, Polynomial– State prediction, property prediction– Control variable prediction

• Most suitable operators for representations– Approximation operators (use best gradient method)– Genetic operators (mind XCS problem bounds)

4.1 XCS –Future Challenges


XCS and Niching

• Currently:– Niching is done occurrence-based

– Number of classifiers in large problem niches unnecessarylarge

– Number of classifiers in small but hard to approximate problemspaces potentially too small -> niche loss

• Additional balancing mechanisms might be necessary!



Dynamic Problems

• Currently: XCS was applied mainly to static problems– Makes iterative, adaptive approach not really necessary

• Dynamic problems– Concept class changes

– Reward distribution changes

– Problem sampling changes

• Question:– How quickly can XCS adapt to these changes?

– Can we improve adaptation?



Compactness of Solution / Population

• Problem in XCS:– Population sizes get rather big.

– Final solution is not very compact

– Solutions indicate overfitting problems in dataminig problems

• Main generalization mechanism purely based onoccurrence frequency

• Borders between different classes are ill-defined.

• How can we efficiently compact the population online orusing post-processing (some approaches available)?


Hierarchical Classifier System

• Local structures are often used by many higher-orderstructures (decomposability of environment, problems, etc.)

• Can we build higher-level classifier structures that build onevolving lower-level structures….

• The hierarchical boolean function problems as a start?


4. Summary and Conclusions


2. XCS – Performance Demonstration

3. XCS – Future Challenges4. Summary & Conclusions


Conclusions

• XCS is designed to– Cluster the problem space to enable– Maximally accurate predictions

• XCS is a highly flexible learning system– Conditions of various types possible– Predictions of various types possible

• Major XCS challenges lie in the further development of– Representation & Operators– Niching– Dynamic Problems– Compactness of solution / population– Hierarchical classifier systems

• XCS has a big potential due to the combination of– Gradient-based update mechanisms– Evolutionary-based feature extraction mechanisms

4 Summary and Conclusions

Thank You for Your Attention

xcs: current capabilities and future challenges

Technology