geometric interpretation of crossover alberto moraglio [email protected] bctcs 2005
Post on 21-Dec-2015
217 views
TRANSCRIPT
![Page 2: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/2.jpg)
Contents
I – Quick Preliminaries
II – Geometric Interpretation of Crossover
Extremely quick overview of its implications:
III – Unification of Major Representations
IV – Crossover Principled Design
V – Is Biological Recombination Geometric?
VI – Unity of Evolutionary Search
![Page 3: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/3.jpg)
I. Quick Preliminaries
![Page 4: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/4.jpg)
Evolutionary Algorithms…
• Are function optimizers
• Mimic biological evolution
• Are robust, hence preferred for real world problems
• Have little theory to explain how and why they work
• There are various flavours
![Page 5: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/5.jpg)
Evolutionary Algorithm Template
Problem & representation independent
![Page 6: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/6.jpg)
Standard representations & EAs flavours/dialects
• Binary strings (genetic algorithms, the classic)• Real code vectors (evolution strategies, continuous
optimization)• Permutations (order-based GAs, combinatorial
optimization)• Parse trees (genetic programming, evolution of computer
programs)Algorithmically irrelevant differences:
name/authorship/solution interpretation/domain of application
Algorithmically relevant differences:solution representation/genetic operators
![Page 7: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/7.jpg)
What is crossover?
CrossoverIs there any
commonaspect ?
Is it possible to give arepresentation-
independent definitionof crossover and
mutation?
100000011101000
100111100011100
100110011101000
100001100011100
![Page 8: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/8.jpg)
Mutation & Crossover for binary strings
• Mutation = bit flip at random position101001 101101
• Crossover = selection crossover point at random swap tails
1010|01 1010001110|00 111001
1*10|0* 1*100*• All offspring match the parent schema
![Page 9: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/9.jpg)
II. Geometric Interpretation of
Crossover
![Page 10: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/10.jpg)
Genetic operators & Neighbourhood structure
• Forget the representation and consider the neighbourhood structure (= search space structure)
• Mutation: offspring are “close to” their parent in the direct neighbourhood
![Page 11: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/11.jpg)
Direct Neighbour Mutation
000
001
010
011
100
101
111
110
Representation: Binary String
Move: Bit Flip
Neighbourhood: Hamming
Representation + Move = Neighbourhood
?
Mutation: Offspring in the direct neighbourhoodWhat is crossover?
![Page 12: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/12.jpg)
Neighbourhood and Crossover
Crossover idea: combining parents genotypes to get children genotypes “somewhere in between” them
Topologically speaking, “somewhere in between” = somewhere on a shortest path
Why on a shortest path?
![Page 13: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/13.jpg)
Shortest Path Crossover011001
010001 011101 011011
010101 011111
010011
010111
D0 : P1
D2 : P2
D1
Parent1: 011101
Parent2: 010111
Children: 01*1*1
Children are on shortest paths
More than one shortest path in general
![Page 14: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/14.jpg)
Interpretation & Generalization
• Traditional mutation & crossover have a natural interpretation in the neighbourhood structure in terms of closeness and betweenness
• Given any representation plus a notion of neighbourhood (move), mutation & crossover operators are well-defined
![Page 15: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/15.jpg)
From graphs to geometry
• Forget the neighbourhood structure and consider the metric space (= space with a notion of distance)
• The distance in the neighbourhood is the length of the shortest path connecting two solutions
• Mutation Direct neighbourhood Ball• Crossover All shortest paths Line
Segment
![Page 16: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/16.jpg)
Balls & SegmentsIn a metric space (S, d) the closed ball is the set of the form
where x belongs to S and r is a positive real number called the radius of the ball.
In a metric space (S, d) the line segment or closed interval is the set of the form
where x and y belong to S and are called extremes of the segment and identify the segment.
}),(|{);( ryxdSyrxB
)},(),(),(|{];[ yxdyzdzxdSzyx
![Page 17: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/17.jpg)
Squared balls & Chunky segments
33
000 001
010 011
100 101
111
110
B(000; 1)Hamming space
3
B((3, 3); 1)Euclidean space
3
B((3, 3); 1)Manhattan space
Balls
1
2
1
2
000 001
010 011
100 101
111
110
[000; 011] = [001; 010]2 geodesics
Hamming space
1 3
[(1, 1); (3, 2)]1 geodesic
Euclidean space
1 3
[(1, 1); (3, 2)] = [(1, 2); (3, 1)]infinitely many geodesics
Manhattan space
Line segments
![Page 18: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/18.jpg)
Uniform Mutation & Uniform Crossover
Uniform topological crossover:
Uniform topological ε-mutation:
|],[|
]),[(}2,1|Pr{),|(
yx
yxzyPxPzUXyxzfUX
],[}0),|(|{)],(Im[ yxyxzfSzyxUX UX
|),(|
)),((}|Pr{)|(
xB
xBzxPzUMxzfUM
),(}0)|(|{)](Im[ xBxzfSzxUM M
Genetic operators have a geometric nature
![Page 19: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/19.jpg)
Representation independentand rigorous definition of
crossover and mutation in the neighbourhood seen as a
geometric space…
![Page 20: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/20.jpg)
This is cheating! I have generalized from a single example
of solution representation!
![Page 21: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/21.jpg)
III. Unification of Major Representations &
Operators
![Page 22: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/22.jpg)
Minkowski spaces – real vectors
22
2
B((2, 2); 1)Euclidean space
2
B((2, 2); 1)Manhattan space
Balls
2
2
B((2, 2); 1)Chessboard space
1
2
1
2
1 3
[(1, 1); (3, 2)]1 geodesic
Euclidean space
1 3
[(1, 1); (3, 2)] = [(1, 2); (3, 1)]infinitely many geodesics
Manhattan space
Line segments
1
2
1 3
[(1, 1); (3, 2)]infinitely many geodesics
Chessboard space
Representation: real vectors
Neighbourhoods: continuous (3 types)
Distances: Minkowski distances
Implementation: algebraic manipulation of real vector (equation of line passing through two points)
Pre-existing recombination operators:- both blend crossovers and discrete crossovers fit geometric definition- extended blend crossovers do not fit
![Page 23: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/23.jpg)
Hamming spaces – binary strings
00 01 02
10 11 12
20 21 22
00 01 02
10 11 12
20 21 22
B(00;1)Hamming space H(2,3)
[00;11]=[01;10]2 geodesics
Hamming space H(2,3)
000 001
010 011
100 101
111110
B(000; 1)Hamming space H(3,2)
000 001
010 011
100 101
111
110
[000; 011] = [001; 010]2 geodesics
Hamming space H(3,2)
Representation: binary/multary strings
Neighbourhoods: bit-flip/site substitution
Distances: Hamming distances
Implementation: symbolic manipulation of multary strings (mask-based crossovers)
Pre-existing recombination operators:- all binary crossovers fit the geometric definition
![Page 24: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/24.jpg)
Cayley spaces - permutationsRepresentation: permutations
Neighbourhoods: adj. swap, swap, reversal, insertion
Distances: corresponding distances
Implementation: “minimal permutation sorting by X move” algorithms:- adj. swap = bubble sort- swap = selection sort - insertion = insertion sort - reversal = approximated MPS by reversals (NP-Hard))
Pre-existing recombination operators:various pre-existing crossover operators are sorting algorithm in disguise (because sorting permutations is easier than sorting vectors of other items)
abc
bac acb
bca cab
cba
B(abc; 1)Adjacent swap space
abc
bac acb
bca cab
cba
[abc; bca]1 geodesic
Adjacent swap space
B(abc; 1)Swap space & Reversal
space
abc
bac acb
bca cab
cba
abc
bac acb
bca cab
cba
[abc; bca]3 geodesics
Swap space & Reversal space
B(abc; 1)Insertion space
[abc; bca]1 geodesic
Insertion space
abc
bac acb
bca cab
cba
abc
bac acb
bca cab
cba
![Page 25: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/25.jpg)
Syntactic tree spaces
Representation: syntactic tree (lisp expression)
Neighbourhood: weighted sub-tree neighbourhood
Distance: structural distance
Implementation: - sub-tree swap crossover - common region mask based crossover
Pre-existing recombination operators:- traditional crossover (non-geometric)- homologous crossover - the geometric framework can help to clarify what is the landscape and distance related to homologous crossover and a distance connected with a geometric crossover which traditional crossover is an approximation
+
sin +
x x x
*
* *
y x*
yy
Parent 1 Parent 2
y
+
sin
x
*
*
yy
x
AlignmentCrossover Point
Swap
*
*
yy
+
x x
Offspring 1Offspring 2
![Page 26: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/26.jpg)
Significance of Unification
• Most of the pre-existing crossover operators for major representations fit geometric definition
• Established pre-existing operators have emerged from experimental work done by generations of practitioners over decades
• Geometric crossover compresses in a simple formula an empirical phenomenon
![Page 27: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/27.jpg)
IV. Crossover Principled Design
![Page 28: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/28.jpg)
Crossover Principled Design
• Domain specific solution representation is effective
• Problem: for non-standard representations it is not clear how crossover should look like
• But: given a combinatorial problem you may know already a good neighbourhood structure
• Geometric Interpretation of Crossover Give me your neighbourhood definition and I give you a crossover definition
![Page 29: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/29.jpg)
+ = ?
Crossover Design Example
![Page 30: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/30.jpg)
Non-labelled graph neighbourhood
MOVE: Insert/remove an edge
Fixed number of nodes
0
1
2
1
2
3
![Page 31: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/31.jpg)
+
Offspring
![Page 32: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/32.jpg)
V. Is Biological Recombination
Geometric?
![Page 33: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/33.jpg)
Levenshtein spaces – sequences
Representation: multary sequences (DNA/amino acids)
Neighbourhood: insertion + deletion + substitution (compound edit move)
Distance: Levenshtein distance
Implementation: inexact sequence alignment (dynamic programming) and sites exchange (crossover mask)
Pre-existing recombination operators:- none- it could be a good crossover for linear GP- it could be a better model of biological crossover to study molecular evolution because it keeps into account the inexact alignment due to molecular annealing of DNA strands that producesevolution of size variation
Parent1=AGCACACAParent2=ACACACTA
best inexact alignment (with gaps):
AGCA|CAC-A Child1=AGCACACTAA-CA|CACTA Child2=ACACACA
![Page 34: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/34.jpg)
A simple model of (homologous) biological recombination fits the
geometric definition under a DNA distance used in bioinformatics
![Page 35: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/35.jpg)
VI. Unity of Evolutionary Search
![Page 36: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/36.jpg)
Example of evolutionary search
![Page 37: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/37.jpg)
Abstract convex evolutionary search
Main result: an evolutionary algorithm using geometric crossover with any probability distribution, any kind of representation, any problem, any selection and replacement mechanism, does the same search: convex search
Proof based on abstract convexity (axiomatic geodesic convexity) and axiomatization of search process (abstract search process)
![Page 38: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/38.jpg)
…Nearly Over!
![Page 39: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/39.jpg)
Future work
THEORY: Generalizing and accommodating pre-existent theories into geometric framework (schema theorem, fitness landscapes, representation theories…)
PRACTICE: Testing crossover principled design on important problems with non-standard representation (problem domain representation)
![Page 40: Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6c5503460f94a4bb5a/html5/thumbnails/40.jpg)
Questions?