artificial life/agents creatures: artificial life autonomous software agents for home entertainment...
TRANSCRIPT
![Page 1: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/1.jpg)
Artificial Life/AgentsCreatures: Artificial Life Autonomous Software Agents for Home EntertainmentStephen Grand, 1997
Learning Human-like Opponent Behaviour for Interactive Computer GamesChristian Bauckhage, Christian Thurau, and Gerhard Sagerer, 2003
Evolving Neural Network Agents in the NERO Video GameKenneth O. Stanley, Bobby D. Bryant, Risto Miikkulainen, 2005
![Page 2: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/2.jpg)
Creatures
• 1997 state of the art Artificial Life game• Creatures with neural network brains and an
evolving genome• Learn by punishment/reward reinforcement• Can learn rudimentary ‘verb-object’ language• Sense of sight, sound, touch• Complex biochemistry (metabolism, immune
system, genetically encoded morphology)
![Page 3: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/3.jpg)
Creatures’ Brains
• Hebbian learning• ~1000 neurons, ~5000 synapses• Organised into ‘lobes’:
![Page 4: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/4.jpg)
Characteristics
• Designed for efficiency (runs on 1997 commodity hardware)
• Limited number of neurons• Brain model is also limited, restricts potential
functions
![Page 5: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/5.jpg)
Learning Human-like Opponent Behaviour
![Page 6: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/6.jpg)
Learning Human-like Opponent Behaviour
• Neural-network control system for a Quake II bot
• Offline, supervised learning• Feed-forward, back-propagation learning,
multilayer perceptron network• One network for moving, one for aiming• Trained to learn one path, then multiple
paths, then moving and aiming
![Page 7: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/7.jpg)
Advantages
• Potentially cheaper and faster than scripting bots
• Generalises to novel situations• More efficient than on-line learning bots• Good introduction to learning agents
![Page 8: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/8.jpg)
Problems
• Paper is horribly structured and hard to read• Assumption: only the agent’s current
state/environmental influences matter!• Experiments didn’t work very well• Bots still static, can’t learn opponent tactics• Maybe difficult to get training data
![Page 9: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/9.jpg)
Evolving Neural Network Agents in NERO
![Page 10: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/10.jpg)
Evolving Neural Network Agents in NERO
• Online, reinforcement learning• Agent fitness increased by learning and
evolution• Player can train teams of bots to compete
against each other in increasingly complex training scenarios
• Won Best Paper Award at the IEEE Symposium on Computer Intelligence and Games
![Page 11: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/11.jpg)
The Network
![Page 12: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/12.jpg)
Learning Method
• rtNeat: basically, a technique for evolving increasingly complex neural networks
• Benefits over traditional RL:– Diversity increased/maintained through speciation– Can keep a memory of past events
• Player provides customised fitness function• NERO removes the worst agents, breeds the
best ones
![Page 13: Artificial Life/Agents Creatures: Artificial Life Autonomous Software Agents for Home Entertainment Stephen Grand, 1997 Learning Human-like Opponent Behaviour](https://reader035.vdocument.in/reader035/viewer/2022062517/56649f145503460f94c28fc7/html5/thumbnails/13.jpg)
• Currently NERO is quite simple• Paper presents no quantitative results, but
results seem promising