optimizing the state eval heuristic of abalone using evolutionary algorithms benjamin rhew november...
Post on 30-Jan-2016
221 views
TRANSCRIPT
![Page 1: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/1.jpg)
Optimizing the State Eval Heuristic of Optimizing the State Eval Heuristic of Abalone using Evolutionary AlgorithmsAbalone using Evolutionary Algorithms
Benjamin RhewNovember 28, 2005
![Page 2: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/2.jpg)
The Roadmap
Why Should We Optimize?Why Use an Evolutionary Algorithm?Previous WorkAbalone ExplainedExperimental SetupWhat is a Two-Pool EA?Evolutionary Algorithm DetailsFuture Work
![Page 3: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/3.jpg)
Why Should We Optimize?
Heuristics are everywhere
Spam filters
Speech recognition software
Have limitations
Slow, but good solution
Fast, but poor solution
So optimize!
![Page 4: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/4.jpg)
Why Use an Evolutionary Algorithm?
Heuristic development takes time
Improbable for a person to design the optimal heuristic
Need to test many different heuristics
Therefore, an EA is called for
![Page 5: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/5.jpg)
Previous Work
There is a lot of previous work, mainly involving chess heuristics
Chess-specific algorithms
Works mainly on chess playing programs
Do not play among themselves
Population dynamics
This approach should be more general
![Page 6: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/6.jpg)
Abalone Explained
Board Game developed in the 1990s
Sumo Wrestling with Marbles...
Program Demo
![Page 7: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/7.jpg)
Experimental Setup
Main Idea – the fitness of an individual depends on how well that individual plays games
Randomly chosen
Play 2 games, one as each color
Takes a long time
Therefore, a steady-state two-pool EA with high selective pressure is used
![Page 8: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/8.jpg)
What is a Two-Pool EA?
An EA where individuals are in two groups
Can use many separations
Male/Female
Predator/Prey
Child/Adult
In this case, child/adult is used
![Page 9: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/9.jpg)
Evolutionary Algorithm Details
Parent Selection – stochastic, based on fitnessSurvival Selection – stochastic/elitist, with worst individual having highest chance of dyingInitial Population – small, 20 individualsRecombination – N-point crossoverMutation – Gaussian Random VariableFitness – number of wins divided by the number of games playedInitialization – small random floats near 1
![Page 10: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/10.jpg)
Evolutionary Algorithm Details
Evolutionary Process – individual will not be killable in the first 10 generations of its life
Three possible Gene Representations:
Take each part of the previous heuristic and multiply it by a constant
Subdivide parts of the previous heuristic and multiply each part by a constant
Completely subdivide previous heuristic and multiply each part by a constant
![Page 11: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/11.jpg)
Evolutionary Algorithm Details
Gene 1:
board1, board2, mymarbles, oppmarbles, age
Gene 2:
board1_1...board1_5, board2_1...board2_5, mymarbles, oppmarbles, age
Gene 3:
board1_1_1...board1_9_9, board2_1_1...board2_9_9, mymarbles, oppmarbles, age
![Page 12: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/12.jpg)
Evolutionary Algorithm Details
Gene 1:
Gene 2:
Gene 3:
![Page 13: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/13.jpg)
Future Work
Results
Compare different fitness function strategies
Compare different gene representations
Use Genetic Programming to evolve new heuristics from scratch
![Page 14: Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005](https://reader036.vdocument.in/reader036/viewer/2022062500/56649d545503460f94a31949/html5/thumbnails/14.jpg)
Questions?