interactive genetic algorithms in multi-agents systems : smart grids as an application
DESCRIPTION
TRANSCRIPT
+
Interactive Genetic Algorithms in Multi-Agents Systems Smart grids as an application
Thesis presented by : Yomna Hassan
Academic Advisor: Dr. Jacob CrandallRSC committee: Dr. Davor Svetinovic, Dr. Iyad Rahwan
Images: available at Google images
+Introduction
Power systems
+Introduction
+Introduction
+Repeated matrix games
+Repeated matrix games
Player 1
Player 2
+Repeated matrix games
+Motivation (Drawbacks of existing methodologies)
Learn slowly for real time systems.
Unable to handle dynamic environments.
Unable to adapt to multi-agent environments.
Does not consider human input.
Too much human input.
+GIGA-WoLF and Q-learning in repeated matrix games
+
+Genetic algorithms
What are genetic algorithms?
Genetic algorithms in Matrix games
Interactive genetic algorithms
+Genetic algorithm
+Algorithm structure
+Chromosome
+Selection
Death for being unfit
Fitness
+Mutation
+Crossover
+Genetic algorithm in Matrix games
D. Gong et al.,Interactive Genetic Algorithms with Individual Fitness,Journal of Universal Computer Science, vol. 15, no. 13 (2009)
+Interactive genetic algorithm
+Objectives
Study the applicability of using genetic algorithms in repeated matrix games.
Study how human input affect the performance of genetic algorithms in repeated matrix games.
+Experimental setup
Experiments
Genetic algorithms in
repeated matrix games
Interactive genetic
algorithms in repeated matrix
games
Three-player prisoner’s dilemma
+Algorithm structure
Chromosome in population represents strategy to be taken following a certain history pattern.
+Evaluation criteria
Average final payoffs: Study conducted for 10 iterations per each (game, opponent) pair.
Average payoff per generation.
+Genetic algorithms (GAs) in repeated matrix games
BasicGA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Genetic algorithms (GAs) in repeated matrix games
Basic GA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Genetic algorithms (GAs) in repeated matrix games
Basic GA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Genetic algorithms (GAs) in repeated matrix games
Basic GA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Genetic algorithms (GAs) in repeated matrix games
Basic GA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Results
Self-play: a similar trend to GIGA, cooperation is really bad
+Results
+Results
+Results
+
Basic GA
GA with history
propagationGA with stopping
condition
GA with dynamic mutation
GA with dynamic mutation and stopping condition
+Highlights1
The performance of the algorithms (even with varying mutation rate and taking stability of environment into consideration) is not consistently well.
Slow convergence to solution, not adapting to opponents.
+Interactive genetic algorithms (IGAs) in repeated matrix games
+Interactive genetic algorithms (IGAs) in repeated matrix games
+Human input
Our choice of what is Acceptable is to unify our experiment.
Human input choice is situational and opponent dependent.
+Results
+Results
+Interactive genetic algorithms (IGAs) in repeated matrix games
+Results
+
+Highlights 2
Human input, even though it may not enhance the final payoff in all the situations, it shows consistency per several iterations.
Variations of IGA performed differently within different games and against different opponents. Fitness that propagates through generations (propagation of history) is not always the best option.
Human input quality is variable over different games and opponents. In our simulations, our knowledge of the games and the opponents contributed into the effectiveness of the human input. This shows that an expert human will enhance the performance of an IGA more than a novice.
+Three-player prisoner’s dilemma
+Results
+Results
+Results
5.1
3.5
1.97
Self play: 4.75
+Conclusions
GAs are an effective learning technique in static systems.
GAs, when improved through implementing a dynamic mutation technique well enough in dynamic systems, except in the presence of other intelligent agents of slower learning rate, such as Q-learning.
Effective human input improves the performance of GAs in multi-agent (dynamic systems in general).
The studied forms of GAs and IGAs maintains its effectiveness in environments with higher numbers of players.
+Future work
Test the effectiveness of interactive genetic algorithms in more complex environments against a wider range of opponents.
Experiment with other techniques used for improving the adaptability of genetic algorithms within dynamic systems.
Explore the effectiveness of human input with various interaction rates. This is in order to reach the most effective rate of interaction.
Study the use of actual human input on GAs and discuss their effectiveness using participants with various backgrounds.
+Questions
+Supplementary slides
+