video games as environments for learning and planning: what’s … · • dota 2 • starcraft 2....
TRANSCRIPT
![Page 1: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/1.jpg)
Video Games As Environments For Learning And Planning:
What’s Next?Julian Togelius
![Page 2: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/2.jpg)
![Page 3: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/3.jpg)
A very selective history• Othello
• Backgammon
• Checkers
• Chess
• Go
• Poker
• Super/Infinite Mario Bros
• Ms. Pac-Man
• Crappy Atari games
• Montezuma’s Revenge
• Dota 2
• StarCraft 2
![Page 4: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/4.jpg)
What’s next?
What’s left?
![Page 5: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/5.jpg)
How we will think of this
![Page 6: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/6.jpg)
Characteristics of games
![Page 7: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/7.jpg)
Characteristics of games
• Number of players1, 1.5, 2, many; competitive, cooperative, both?
• Stochasticity
• Observability
• Action space and branching factor
• Time granularityFew turns, many turns, continuous?
![Page 8: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/8.jpg)
Branching factor• Checkers: 3
• Pac-Man: 4
• Super Mario Bros: 16
• Chess: 35
• Go: 350
• Civilization: ?
• Fortnite: ?
![Page 9: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/9.jpg)
Characteristics of the game-AI interface
• How is the game state represented?Pixels, text, simulated sensors, some internal representation?
• Is there a forward model?Is it fast? Accurate?
• Do you have time to train?How much?
• How many games are you playing?
![Page 10: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/10.jpg)
Characteristics ofcommon benchmarks
• Classic board games: lowish branching factor, few turns, two-player adversarial, complete information, no stochasticity, structured game state representation, forward model readily available
• Poker: lots of hidden information, stochastic, otherwise like above
• Atari 2600: low branching factor, longish, single-player, mostly complete information, no stochasticity (!), pixels, no/slow forward model
• StarCraft: insane branching factor, long games, two-player adversarial, hidden information, some stochasticity, pixels (more or less), no forward model
![Page 11: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/11.jpg)
Another way of thinking
Which cognitive abilities are exercised by a particular game?
Which cognitive abilities that humans have are not tested by current game-based AI benchmarks?
If we create AI that can play games that require certain abilities of humans, will agents that solve these games need to have the same abilities?
![Page 12: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/12.jpg)
![Page 13: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/13.jpg)
Which challenges are next?
![Page 14: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/14.jpg)
Wait a minute, why are we doing this?
To improve specific AI capabilities?
To achieve AGI?
To make games better?
To keep unemployed AI researchers off the streets?
![Page 15: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/15.jpg)
Which challenges are next?• Role-playing games
• Roguelikes
• Multiplayer games
• 4X games
• Open world games
• Gardening/building games
• Generalization/Transfer
• Human-like playing
• Resource-constrained playing
• Things that are not game-playing
![Page 16: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/16.jpg)
Role-playing games
Text as input; very long game length; reasoning about motives, emotions and social practices
![Page 17: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/17.jpg)
Text adventure games
![Page 18: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/18.jpg)
Roguelikes
Handling radical uncertainty, generalization
![Page 19: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/19.jpg)
Multiplayer games
Multiplayer dynamics (example: Pommerman)
![Page 20: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/20.jpg)
4X games
Example: Civilization 6. Ridiculous branching factor, n-player, extremely long game time.
![Page 21: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/21.jpg)
![Page 22: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/22.jpg)
So many game genres• Dating simulators
• Rhythm games
• Word games
• Grand strategy war-games
• Life simulations
• Weirdo casual games of all kinds
• Things that come out of the global game jam
![Page 23: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/23.jpg)
Generalization/Transfer
Example: Obstacle Tower.Small branching factor, single-player, pixel inputs with very complex visuals, thoroughly randomized puzzles and visuals, potentially endless game time
![Page 24: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/24.jpg)
Generalization/Transfer
Example: General Video Game AI
![Page 25: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/25.jpg)
Human-like playing
• Playing games to win is one thing - how about playing them like a human?
• Same strengths, same weaknesses, same preferences
• Very useful for the game industry
![Page 26: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/26.jpg)
Resource-constrained playing
• How to do it without 5000 TPUs
• Academics like this
• Obvious practical applications
![Page 27: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/27.jpg)
Things that are not game-playing
• Procedural content generation
• Player experience modeling
• Game modeling
• Game generation
• Ultimately, inventing new games for AI to play automatically
![Page 28: Video Games As Environments For Learning And Planning: What’s … · • Dota 2 • StarCraft 2. What’s next? What’s left? How we will think of this. Characteristics of games](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb707d1106bb828cf5c4329/html5/thumbnails/28.jpg)