tools for artificial intelligence
TRANSCRIPT
SOME TOOLS FOR ARTIFICIAL INTELLIGENCE
Olivier Teytaud --- [email protected]
NUTN, Tainan, 2011
Tao (Inria, Cnrs, Lri, Paris-Sud)
People:Permanent staff: 11
~15 ph.D. Students
In Universit Paris-SudLargest campus in France
Faculty of sciences: mathematics, computer science, physics, chemistry, biology, earth and space sciences ==> 12000 students
Inria affiliation: Around 50 years old
Devoted to research in comp. science
Tao (Inria, Cnrs, Lri, Paris-Sud)
Reservoir computing
Optimal decision making under uncertainty
Optimization
Autonomic computer
Machine learning
Communication not always so easy:
Many of you speak Chinese + Taiwanese. So English = third language.
I am French. English = second language.
I work mainly in mathematical aspects of computer science, more than computer science.
Difficulties might also be an enrichment.
Feel free to interrupt me as much as useful.
NUTN, Tainan, 2011
Communication not always so easy:
Many of you speak Chinese + Taiwanese. So English = third language.
I am French. English = second language.
I work mainly in mathematical aspects of computer science, more than computer science.
Difficulties might also be an enrichment.
Feel free to interrupt me as much as useful.
NUTN, Tainan, 2011
Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be directly useful. I switched to Statistics.
3) I had fun, but I wanted to be more directly useful. Switched to Operational Research, in industry.- Many applications.- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence: - Mathematics. - Challenges (in particular games). - Applications.
Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be directly useful. I switched to Statistics.
3) I had fun, but I wanted to be more directly useful. Switched to Operational Research, in industry.- Many applications.- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence: - Mathematics. - Challenges (in particular games). - Applications.
Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be directly useful. I switched to Statistics.
3) I had fun, but I wanted to be more directly useful. Switched to Operation Research, in industry.- Many applications.- My favorite: electricity generation.
4) Now (40 dangerously approaching), Artificial Intelligence: - Mathematics. - Challenges (in particular games). - Applications.
Goes back to militaryapplication around world war II,when UK resisted to Hitler thanksto optimized radars.Now essentially civil applications.
Vita in a nutshell:
1) First research: mathematical logic
2) I had fun, but I wanted to be directly useful. I switched to Statistics.
3) I had fun, but I wanted to be more directly useful. Switched to Operational Research, in industry.- Many applications.- My favorite: electricity generation.
4) Now (40 years old soon...), Artificial Intelligence: - Mathematics. - Beautiful challenges (in particular games). - Applications.
Outline of what I'll discuss:
1) Some concepts:- simplified problems- toolboxes for these problems
2) Principle: - reducing real problems to groups of artificial problems - small problems might be considered as artificial and useless when considered alone. - but when you solve a clearly stated small problem, usually you can find an application for this solution. - we will see applications as well.
==> For the moment let's see big applications
3) I'll also show some works on which contributors are welcome.
EXAMPLES OF APPLICATIONS
ELECTRICITY GENERATION
ELECTRICITY GENERATIONThe case of France
Data: - climate model (stochastic) - model of electricity demand (stochastic) - model of power plants
Each day we receive: - electricity consumption - weather information - info on faults
Each day, we decide how to distribute the production among the power plants. (also: schedule long-term investiments)
Data: - climate model (stochastic) - model of electricity demand (stochastic) - model of power plants (PP): nuclear PP (NPP), thermal PP (TPP), Hydroelectric PP (HPP)...
Each day we receive: - electricity consumption - weather information - info on faults
Each day, we decide how to distribute the production among the power plants.
DATA(climate,plants,economy)STRATEGYPROGRAM
Daily information
Decisions
Electricsystem
One of the most important industrial problem you can imagine:how to produce energy ?
France has specific elements:- heavily nuclearized (most nuclearized country in the world)- often cooled by rivers (do not work in case of droughts ==> hard to predict) - we must schedule maintenance - we must take long-term decisions (building new NPP ? Removing ?)- also hydroelectricity: - should we use water now ? - should we keep it for winter (in France, high consumption is in winter)
DATA(climate,plants,economy)STRATEGYPROGRAM
Daily information
Decisions
Electricsystem
Problem 1: Taiwan is very different from France :-)
Almost no nuclear power plant ? Cooled by sea ?
Electrically connected to other countries ? (France might be connected to Africa)
Sun sufficient for massive photo-voltaic units ?
Wind much stronger than in France - can be used ?
Other questions ?
Electricity consumption dominated by air conditioning ?
Maybe electric cars in the future ?
Climate maybe more regular ? Problem easier than in France ?
==> I don't know==> I'd like to work on it (energy is an important concern, in Taiwan as well lack of independence ?)==> Need Chinese-reading persons==> Other (Taiwan-independent) concern: tackling partial observation in energy generation problem
GAME OF GO(with Nutn)
GOOD NEWS: we had a lot of progress with **generic** algorithms (algorithms which can be used for many things).
The revolution in Go which occurred in 2007-2009 is a major breakthrough in Artificial Intelligence.
We'll see that in details.
I am a little bit tired of the game of Go, because I have no recent progress, and recent progress in the community comes from Go expertise, which is only useful for Go...
Problem 2: Solving unsolved situations in Go
Now computers are much stronger than in the past.
However, they still misunderstand some trivial situations (in particular, liberty races).
You have an idea ? Tell me :-)
We have a solver in France (not for playing Go; aimed at provably solving), that we would like to test on various situations. We do not play Go. If you are 5kyu or better, you can contribute.
URBAN RIVALS
17 Millions registered users. Important company.
URBAN RIVALS- Choose 4 cards, your opponent chooses 4 Cards- Each player gets 12 Pilz (i.e. strength points) - Each player gets health points.- Each turn: - each player chooses a card - each player uses pilz (each used pilz is lost forever, but it gives strength) - read cards, apply rules==> no more health point ? ==> you're dead.
Urban Rivals
==> Partial information because you don't observe your opponent's decisions
==> There are on the shell algorithms and programs for full information games, but not for partial information games.
==> We used a (provable) combination of MCTS and EXP3
==> Immediately human level performance
==> suggests that maths can help ==> still possible works: - automatic choice of cards ? - reducing comp. cost ?
POKEMONS
Second most lucrative video game.
Meta-gaming: choosing your deck.
POKEMONS: Problem 3
Second most lucrative video game.
Meta-gaming: choosing your deck.In-gaming: playing with your set of cards.
Problem 4: Solving MineSweeper.
Looks like a trivial boring problem. Certainly not indeed.
Many papers with the same approach (so-called CSP technique)
We could outperform these algorithms thanks to a probabilistic approach.
But my approach only works on small board (or huge computational cost) ==> we want to extend.
Quite similar to electricity generation (yes, I believe in this)
Find an optimalmove ?
Game applications can be considered as childish.Shouldn't we focus on more important things ?
However:
- If you have a breakthrough in an important game, people will trust you. Doors will be opened when you will propose new algorithms for real-world applications.
- Testing ideas on a nuclear power plant is more dangerous than testing ideas on a game of Go.
- It's easier to compare approaches in games than in electricity generation.
INTRODUCTION IS OVER.
NOW TECHNICAL STUFF.
REMARKS, QUESTIONS ?
TODAY, GAMES.
1) HOW TO SOLVE THEM
2) C IMPLEMENTATION
ONE FUNDAMENTAL TOOL: ZERMELOConsider the following game:
- there are 5 sticks;- in turn, each player removes 1 or 2 sticks;- the player which removes the last stick looses.
Example:Player I: IIIIIPlayer II: IIIPlayer I: I ==> looses!
How should I play ?
ONE FUNDAMENTAL TOOL: ZERMELOZermelo proposed a solution (for full-information games).
Born in 1871.
1900-1905: major contributions in logic.
1913: major contribution to games in 1913.
1931: Optimized navigation (from games to applications).
Resigned in 1935 (he did not like Hitler).
Died in 1953.
ONE FUNDAMENTAL TOOL: ZERMELO543
32
21
LOSS!
WIN!
WIN!
21
LOSS!
WIN!
WIN!
WIN!
LOSS!
ZERMELO: I HAVETHE OPTIMAL STRATEGY!543
32
21
LOSS!
WIN!
WIN!
21
LOSS!
WIN!
WIN!
WIN!
LOSS!
ZERMELO: not limited to win/loss games.Can work on games with continuous rewards.New rule: if the game contains 4, reward is multiplied by 2.543
32
21
0
1
2
21
0
2
2
2
0
2
YELLOW NODES:LABEL = MINIMUMOF CHILDREN's LABELSBLUE NODES:LABEL = MAXIMUMOF CHILDREN's LABELS
struct gameState { int *descriptionOfState; int numberOfLegalMoves; int * legalMoves; int turn; // 1 if player 1 plays, -1 otherwiseint result; // final reward, if numberOfLegalMoves=0};struct gameState next(struct gameState s,int move) { RULES };
double zermeloValue(struct gameState s){ int i;double value; double maxValue=-MAXDOUBLE; if (s.numberOfLegalMoves==0) return(s.turn * s.result); for (i=0;imaxValue) maxValue=value;} return s.turn*maxValue; //we return value for player 1}
ZERMELO: C CODE
struct gameState { int *descriptionOfState; int numberOfLegalMoves; Int * legalMoves; int turn; // 1 if player 1 plays, -1 otherwiseint result; // final reward, if numberOfLegalMoves=0};struct gameState next(struct gameState s,int move) { RULES };double zermeloValue(struct gameState s){ int i;double value; double maxValue=-MAXDOUBLE; if (s.numberOfLegalMoves==0) return(s.turn * s.result); for (i=0;imaxValue) maxValue=value;} return s.turn*maxValue; //we return value for player 1}ZERMELO: C CODE
Last week: Zermelo algorithm.
What is Zermelo ? = Simplest algorithm for solving 1Player or 2Player games. = Recursive algorithm = Conveniently (but slowly) implemented with struct
This week
= a bit more on Zermelo algorithm = C development: static random variables
Future weeks
Still some C implementation (or other languages ? as you wish) Still some (not always easy) algorithmsModels of applicationsI hope I can convince you that operational research / artificial intelligence are useful and fun.
Zermelo again.
What does the zermeloValue() function returns ?
===> The reward in case of perfect play.===> A perfect strategy.
===> Gods can run Zermelo algorithms: perfect play.==> humans have no time for this.==> Can we design a new version in case it is too slow ?
Let's see a pseudo-code, instead of a code.
double zermeloValue(struct gameState s){ if (s is end of game) then return score. else { If (play 1 plays) then return max(zermeloValue(children))Elsereturn min(zermeloValue(children)) }
}
ZERMELO: A NATURAL CONCEPT,THE DEPTH.5(0)4(1)3(1)
3(2)2(2)
2(2)1(2)
LOSS!
WIN!
WIN!
2(3)1(3)
LOSS!
WIN!
WIN!
WIN!
LOSS!
double zermeloValue(struct gameState s){ static int depth=0; int i;double value; double maxValue=-MAXDOUBLE; if (s.numberOfLegalMoves==0) return(s.turn * s.result);depth++; for (i=0;imaxValue) maxValue=value;}depth--; return s.turn*maxValue; //we return value for player 1
}ZERMELO: C CODE FOR THE DEPTH
Sometimes it is too slow.
Then, what can I do ?
Etc... too big!
We will not gobelow this depth.
We will not go But, what shouldbelow this depth. zermeloFunction return ?
double zermeloValue(struct gameState s){ static int depth=0; int i;double value; double maxValue=-MAXDOUBLE; if (s.numberOfLegalMoves==0) return(s.turn * s.result);if (depth>5) return drand48();depth++; for (i=0;imaxValue) maxValue=value;}depth--; return s.turn*maxValue; //we return value for player 1
}
Should we return a random number ?
double zermeloValue(struct gameState s){ static int depth=0; int i;double value; double maxValue=-MAXDOUBLE; if (s.numberOfLegalMoves==0) return(s.turn * s.result);if (depth>5) return heuristicValue(s);depth++; for (i=0;imaxValue) maxValue=value;}depth--; return s.turn*maxValue; //we return value for player 1
}A function writtenby some expert ofthe game.
This idea is a main contributionby Shannon (for European chess).
Shannon 1916-2001Noble prize (not Nobel!)
Works in:- Logic- Games (also: artificial mouse for mazes)- Financial analysis
SHANNON and games
double heuristicValue(struct gameState s){
if (!strcmp(gameName,chineseChess))
{/******/ Return 0.1*(nbOfBlackElephants(s) nbOfRedElephants(s) ) +0.1*(nbOfBlackGuards(s) - nbOfWhiteGuards(s) ) +0.03*(nbOfBlackPieces(s) - nbOfWhitePieces(s) ) +0.01*(nbOfBlackPawns(s) - nbOfWhitePawns(s) );}else{ assert(0); }
}
Zermelo's algorithm is too slow.MINIMAX: an approximation of Zermelo's algo.Thanks to Wikipedia
ALPHA-BETA (thks WIKIPEDIA)
ALPHA-BETA
PRINCIPLE OF ALPHA-BETA:
In zermeloFunction, considering a opponent node, if I know:
- THAT AT PREVIOUS DEPTH, I CAN REACH SCORE ALPHA=6,
- THAT IN CURRENT STATE MY OPPONENT CAN ENSURE SCORE BETA THIS IS A ALPHA-CUTOFF==> OTHER PLAYER: BETA-CUTOFF (just exchange players)
ALPHA-BETA (thks WIKIPEDIA)
EXAMPLE OF GAME (we can discuss why it is a good game)
- Randomly generate a 4x4 matrix with 0 and 1 (K=4). 0 0 1 1 1 0 0 1 0 1 1 1 1 0 0 0- Player one removes top part or bottom part 0 1 1 1 1 0 0 0- Player two removes left part or right part 0 1 1 0- Player one removes top part of bottom part 0 1- Player two removes left part or right part 0 ==> Player one wins if 1, player two wins if 0!
POSSIBLE HOME WORK
1) ZERMELO:can you implement it on a simple game ?
2) MINIMAX: can you add a heuristic function ? Which heuristic function ?Experiments: plot a graph: X(depth) = computation time of minimax (divided by Zermelo's computation time) Y(depth) = win rate against Zermelo
3) ALPHA-BETACan you modify it ==> alpha-beta pruning ? Plot a graph for various sizes: X = number of visited nodes Y = average winning rate of alpha-beta vs minimax Or X = depthY = average winning rate of a-b vs a-b with depth -1
APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.CAN WE FIND A REALLY USEFUL APPLICATION ?
APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:- water
APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:- water- plants (which need water during summer'sheat wave)
APPLICATION OF ZERMELO
WE HAVE SEEN THE 5-STICKS GAME.CAN WE FIND A REALLY USEFUL APPLICATION ?
I have:- water- plants (which need water during summer'sheat wave)
Actions = giving water to plants, or not.
APPLICATION OF ZERMELO
I have:- water- plants (which need water during summer'sheat wave)
Each day, I choose an action.State = { date +water level in stock + water level in plants }
Reward = quality / quantity of production.
Zermelo ==> optimal sequence of actions ==> optimal stock level.
IMPORTANT REMARK:
- Maybe this does not look serious.
- But heat waves are a serious problem.
- Here the problem is simplified, but the concepts for the real application are the same.
- Applying this just requires a computer anddatas/models about plants/water resources.==> if you can apply Zermelo variants correctly, you can help for a better world.
However, the nextState function is randomized ==> we need a Zermelo for this case
double zermeloValue(struct gameState s){ int i;double value; static int depth=0;If (s.turn==0) { value=0;double total=0;for (i=0;i5) return heuristicValue(s);depth++; for (i=0;imaxValue) maxValue=value;}depth--; return s.turn*maxValue; //we return value for player 1}s.turn == 0: action is randomly chosen.
This is Zermelo, adapted tostochastic games.
References: - Mass- Bellman
The problem:
Solving Matrix Games.
A solution:
EXP3.
ONE MORE TOOL: MATRIX GAMES
What is a (0-sum) Matrix Game ?
Example:
1 0 0 M= 0 1 1 1 0 1
- You choose (privately) a row (i is 1, 2 or 3).- In same time, I choose (privately) a column (j=1, 2 or 3).- My reward: M(i,j)- Your reward: -M(i,j)
I want a 1, you want a 0.Given M, how should I play ?
What is a (0-sum) Matrix Game ?
Example: rock-paper-scissor Rock Paper Scissor Rock 0 -1 1 M= Paper 1 0 -1 Scissor -1 1 0
- You choose (privately) a row (i is 1, 2 or 3).- In same time, I choose (privately) a column (j=1, 2 or 3).- My reward: M(i,j)- Your reward: -M(i,j)
I want a 1, you want a 0.Given M, how should I play ?
Given M, how should I play ?
Nash (diagnosed with paranoid schizophrenia)got a Nobel prize for his work around that.
Principle of a Nash equilibrium: - pure strategy = fixed strategy (e.g. play scissor) - mixed strategy = randomized strategy (e.g. play scissor with probability and play rock with probability - choose the mixed strategy such that
The worst possible score against any opponent strategy is maximum
==> Nash strategy ==> EXP3: algorithm for finding Nash strategies.
IMPORTANT FACTS ON GAMES:
- Turn-based, full-information games, solvers exist: - Too slow for chess, Go. - Ok for 8x8 checkers. ==> Zermelo ==> variants: Minimax, Alpha-beta, play reasonably well many games
- Matrix games:- Nash strategies = wort-case optimal - Nash strategies = randomized strategies
A BETTER EXAMPLE ? POKEMON.
Each player chooses 2 pokemons among the 3 possible ones (real life: 3 or 4 among hundreds).
A BETTER EXAMPLE ? POKEMON.
Three possibilities:
A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing a row in a 3x3 matrix game):
Player 1
Player 2
Check whowins (by somefull-observationgame-solver).
A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing a row in a 3x3 matrix game):
Player 1
Player 2
P1 P2 P2
P2 P1 P1
P1 P2 P1
A BETTER EXAMPLE ? POKEMON.
Three possibilities (the same as choosing a row in a 3x3 matrix game):
Player 1
Player 2
1 0 0
0 1 1
1 0 1
EXP3 principle for Nash equilibrium of KxK matrix M:
- choose a number N of iterations - S1=null vector - S2=null vector - at each iteration t=1, ..., t=N: { - compute p1 as a function of S1 // we will see how - compute p2 as a function of S2 // we will see how
- randomly draw i according to probability distribution p1 - randomly draw j according to probability distribution p2
- define r=M(i,j) in the matrix
- S1(i)+= r / p1(i) - S2(j)+=(1-r) / p2(j)
- Player1Nash(i)+= (1/N); - Player2Nash(j)+= (1/N); }
EXP3 principle for Nash equilibrium of KxK matrix M:
- choose a number N of iterations - S1=null vector - S2=null vector - at each iteration t=1, ..., t=N: { - compute p1 as a function of S1 // we will see how - compute p2 as a function of S2 // we will see how
- randomly draw i according to probability distribution p1 - randomly draw j according to probability distribution p2
- define r=M(i,j) in the matrix
- S1(i)+= r / p1(i) - S2(j)+=(1-r) / p2(j)
- Player1Nash(i)+= (1/N); - Player2Nash(j)+= (1/N); }
==> see C source code
Q&A: (my questions, and also yours)
Q: Who cares about matrix games ?A: Useful for many things. Unfortunately, it's usually a building block inside more complex algorithms. We will see examples, but later.
Q: Is a Nash strategy optimal ?A: It depends for what... It is optimal in a worst case sense (i.e. against a very strong opponent). Not necessarily very good against a weak opponent.
Muokkaa otsikon tekstimuotoa napsauttamalla
Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso