cap6938 neuroevolution and artificial embryogeny real-time neat
DESCRIPTION
CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT. Dr. Kenneth Stanley February 22, 2006. Generations May Not Always Be Appropriate. When a population is evaluated simultaneously Many are observable at the same time Therefore, entire population would change at once - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/1.jpg)
CAP6938Neuroevolution and Artificial Embryogeny
Real-time NEAT
Dr. Kenneth Stanley
February 22, 2006
![Page 2: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/2.jpg)
Generations May Not Always Be Appropriate
• When a population is evaluated simultaneously – Many are observable at the same time– Therefore, entire population would change at
once– A sudden change is incongruous, highly
noticeable
• When a human interacts with one individual at a time– Want things to improve constantly
![Page 3: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/3.jpg)
Steady State GA: One Individual Is Replaced at a Time
• Start by evaluating entire first generation• Then continually pick one to remove, replace it
with child of the best
Start:Evaluate All
1f
2f
3f
4f
5f
6f
7f
8f
1) Remove poor individual
2) Create offpsring from good parents
3) Replace removed individual
Repeat…
![Page 4: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/4.jpg)
Steady State During Simultaneous Evaluation: Similar but not Identical
• Several new issues when evolution is real-time– Evaluation is asynchronous – When to replace?– How to assign fitness?– How to display changes
![Page 5: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/5.jpg)
Regular NEAT Introduces Additional Challenges for Real Time
• Speciation equations based on generations
• No “remove worst” operation defined in algorithm
• Dynamic compatibility thresholding assumes generations
![Page 6: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/6.jpg)
Speciation Equations Based on Generations
![Page 7: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/7.jpg)
How to Remove the Worst?
• No such operation in generational NEAT
• Worst often may often be a new species– Removing it would destroy protection of
innovation– Loss of regular NEAT dynamics
![Page 8: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/8.jpg)
Dynamic Compatibility Thresholding Assumes A Next
Generation
![Page 9: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/9.jpg)
Real-time NEAT Addresses Both the Steady State and Simultaneity Issues• Real-time speciation
• Simultaneous and asynchronous evaluation
• Steady state replacement
• Fast enough to change while a game is played
• Equivalent dynamics to regular NEAT
![Page 10: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/10.jpg)
Main Loop (Non-Generational)
![Page 11: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/11.jpg)
Choosing the Parent Species
![Page 12: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/12.jpg)
Finally: How Many Ticks Between Replacements?
• Intuitions:– The more often replacement occurs, the fewer are eligible – The larger the population, the more are eligible– The high the age of maturity, the fewer are eligible
![Page 13: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/13.jpg)
rtNEAT Is Implemented In NERO
• Download at http://nerogame.org• rtNEAT source soon available (TBA)• Simulated demos have public appeal
– Over 50,000 downloads– Appeared on Slashdot– Best Paper Award in Computational Intelligence and
Games– Independent Games Festival Best Student Game
Award– rtNEAT licensed– Worldwide media coverage
![Page 14: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/14.jpg)
NERO: NeuroEvolving Robotic Operatives
• NPCs improve in real time as game is played
• Player can train AI for goal and style of play
• Each AI Unit Has Unique NN
![Page 15: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/15.jpg)
NERO Battle Mode
• After training, evolved behaviors are saved
• Player assembles team of trained agents
• Team is tested in battle against opponent’s team
![Page 16: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/16.jpg)
NERO Training: The Factory
• Reduces noise during evaluation– All evaluations start out similarly
• Robot bodies produced by “factory”
• Each body sent back to factory to respawn
• Bodies retain their NN unless chosen for replacement
• NN’s have different ages– Fitness is diminishing average of spawn trials:
![Page 17: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/17.jpg)
NERO Inputs and Outputs
![Page 18: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/18.jpg)
Enemy/Friend Radars
![Page 19: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/19.jpg)
Enemy On-Target Sensor
![Page 20: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/20.jpg)
Object Rangefinder Sensors
![Page 21: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/21.jpg)
Enemy Line-of-Fire Sensors
![Page 22: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/22.jpg)
Further Applications?
• New kinds of games
• New kinds of AI in games
• New kinds of real-time simulations
• Training applications
• Interactive steady-state evolution
![Page 23: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/23.jpg)
Next Topic: Improving the neural model
• Adaptive neural networks
• Change over a lifetime
• Leaky integrator neurons and CTRNN
Homework due 2/27/06: Working genotype to phenotype mapping. Genetic representation completed. Saving and loading of genome file I/O functions completed. Turn in summary, code, and examples demonstrating that it works.
Evolutionary Robots with On-line Self-Organization and Behavioral Fitness by Dario Floreano and Joseba Urzelai (2000)Evolving Adaptive Neural Networks with and Without Adaptive Synapses by Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen (2003)
![Page 24: CAP6938 Neuroevolution and Artificial Embryogeny Real-time NEAT](https://reader031.vdocument.in/reader031/viewer/2022020417/5681379a550346895d9f3d7d/html5/thumbnails/24.jpg)
Project Milestones (25% of grade)
• 2/6: Initial proposal and project description• 2/15: Domain and phenotype code and examples• 2/27: Genes and Genotype to Phenotype mapping • 3/8: Genetic operators all working• 3/27: Population level and main loop working• 4/10: Final project and presentation due (75% of grade)