lecture24

Introduction to MachineIntroduction to Machine LearningLearning

Lecture 24Lecture 24Learning Classifier Systems

Albert Orriols i Puightt // lb t i l thttp://www.albertorriols.net

[email protected]

Artificial Intelligence – Machine Learningg gEnginyeria i Arquitectura La Salle

Universitat Ramon Llull

Recap of Lecture 23Michigan-style LCSg y

EnvironmentSensorialstate RewardAction

Learning Classifier 1Classifier 2

Online rule evaluator:• XCS: Q-Learning (Sutton & Barto, 1998)

Uses Widrow-Hoff delta rule

state

Any Representation:Classifier System

Classifier 2

Classifier n

Uses Widrow-Hoff delta ruley pproduction rules,

genetic programs,perceptrons,

SVMs

Rule evolution:

SVMs

GeneticAlgorithm

Rule evolution: Typically, a GA (Holland, 75; Goldberg, 89) applied on the population.

Slide 2Artificial Intelligence Machine Learning

Recap of Lecture 23Main characteristics of XCS

Population-based method

I d d t l ifiIndependent classifiers

Works under a reinforcement learning paradigm, but can be l li d t i d l i ( d t f tialso applied to supervised learning (and to function

approximation)

Cl ifi l d b ti l ithClassifiers evolved by a genetic algorithm


Today’s Agenda

Examples of domainsAnother step toward cognitive systems

Anticipatory Classifier System


Applications of LCSExamples of domainsp

Reinforcement learning

S i d l iSupervised learning

[Function approximation – not seen herein]

Real-life applicationsReal life applicationsData Mining

Modeling market traders

Autonomous robotics

Modeling artificial ecosystems

Slide 5

…

Artificial Intelligence Machine Learning

Example in Reinf. LearningExample: simple maze problemp p p

XCS solves more complex reinforcement learning prob :XCS solves more complex reinforcement learning prob.:Complex mazes

Slide 6

Mountain car


Example in Reinf. LearningPerformance in the Maze6 problem (Butz et al.)p ( )


Example in Sup. LearningSolving large, non-linear, complex boolean functions

000 0#######:0000 0#######:0000 1#######:1001 #0######:0001 #1######:1001 #1######:1010 ##0#####:0010 ##1#####:1011 ###0####:0011 ###0####:0011 ###1####:1100 ####0###:0100 ####1###:1100 ####1###:1101 #####0##:0101 #####1##:1110 ######0#:0

Solving multiplexer problems up to 135 input variables

110 ######1#:1111 #######0:0111 #######1:1


Solving multiplexer problems up to 135 input variables

Current Real-Life ApplicationsData mining

Most important application domain of LCSs

John H. HolmesJohn H. HolmesEpidemiologic study by means of LCSsHidden relationships among variables p gdiscovered by LCSs

Xavier Llorà et al.Better than Human Capability in Diagnosing Prostate Cancer Using Infrared Spectroscopic imagingg g

Many other applications and miners:GALE GAX GAssist UCS MIPSGALE, GAX, GAssist, UCS, MIPS …

See: Bull, Bernadó-Mansilla & Holmes (eds) Learning Classifier Systems in Data mining. Springer (2008)


Current Real-Life ApplicationsModeling market tradersg

LETS project: Evolving artificial traders for successful market trading (Sonia Schulenburg et al, 2007)ad g (So a Sc u e bu g et a , 00 )

Evolutionary economics:Evolutionary economics:Create trend followersand value investors

Let them interact

Evolve a population ofstrategies


Current Real-Life ApplicationsAutonomous Robotics

Robot shaping: Early efforts of Marco Dorigo and Marco Colombetti (1997)

Small mobile robots equipped with sensors and motorsRobots connected in real time by various sorts of modem cableRobots controlled by LCS, ICS, running on desktop computersConstant stream of positive/negative rewards (bucket brigade)

Tasks solved: Following lightsGather food and run homeHunt around for a light hidden behind and obstacle

Impressive results, high performance

Recent applications to model robotic problems performed in the University of West England

Slide 11

University of West England


Current Real-Life ApplicationsModeling Artificial Ecosystemsg y

Eden: Artificial Life environment (Jon McCromak, 2004)

Model of an environment where evolvable rule basedModel of an environment where evolvable rule-basedclassifier systems drive agent behavior.

Autonomous LCSs or agents compete for limitedAutonomous LCSs or agents compete for limitedresources.

Agents can:gMake and listen to soundsForage for foodgEncounter predatorsMate with each other

Goal: Maintain the audience in tension without fitness needing the

Slide 12

audience explicitly perform fitness selection


Toward Cognitive SystemsCognitive systemsg y

Cognitive systems are natural or artificial information processing systems, including those responsible for perception, p ocess g sys e s, c ud g ose espo s b e o pe cep o ,learning, reasoning and decision-making and for communication and action

LCS originally devised as cognitive systemsLCS originally devised as cognitive systems

A step furtherAnticipatory LCS


Anticipatory LCSAnticipations influence cognitive systemsp g y

LCS learned:LCS learned:Conditions x actions x prediction

Anticipatory learning processes learnAnticipatory learning processes learnCondition x action x effect relations

Let’s see the Anticipatory LCS (ACS2)


ACSENVIRONMENTENVIRONMENT

C1 A1 E1

Population [P]

Match Set [M]

A1σt σt+1

C1 – A1 – E1C2 – A2 – E2C3 – A3 – E3C4 – A1 – E4

C1 – A1 – E1C3 – A3 – E3C4 – A1 – E4

Match Set [M]

C1 – A1 – E1

Action Set [A]

C5 – A5 – E5C6 – A6 – E6C7 – A1 – E7C8 – A8 – E8

C4 – A1 – E4C6 – A6 – E6C7 – A1 – E7C9 – A9 – E9

C4 – A1 – E4C7 – A1 – E7

…C8 – A8 – E8C9 – A9 – E9

…

…

compareAnticipatory Learning Process


Next Class

Bi i t f h t h fBig picture of what we have seen so far

New challenges in machine learning


Introduction to MachineIntroduction to Machine LearningLearning

Lecture 24Lecture 24Learning Classifier Systems

Albert Orriols i Puightt // lb t i l thttp://www.albertorriols.net

[email protected]

Artificial Intelligence – Machine Learningg gEnginyeria i Arquitectura La Salle

Universitat Ramon Llull

lecture24

Education