automatic algorithm invention
DESCRIPTION
Presented at 28C3 in Berlin, Germany December 2011. Automated creation of image filters using Cartesian Genetic Programming (CGP). An introduction to doing CGP in C++.TRANSCRIPT
![Page 1: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/1.jpg)
Automatic Algorithm Invention
by Wes Faler of Part-Time Scientists for 28C3, Berlin Germany, 2011
![Page 2: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/2.jpg)
<presentation>
Why
What
How
Code and Details
Scaling
State
Your turn!
![Page 3: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/3.jpg)
Why
Part Time Scientists needs a De-Bayer algorithm for our images.
![Page 4: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/4.jpg)
What - Bayer Image Filter
Source: Wikimedia
![Page 5: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/5.jpg)
Optimization – GA, PS, SA, LS
Given: ◦ An existing system structure
Inputs Output(s) Formula structure
◦ Test cases ◦ Constraints
Create: ◦ The optimal set of parameters
a Red + b Blue + c Green0 + d Green1
![Page 6: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/6.jpg)
Invention – GP
Given: ◦ A partial existing system structure
Inputs
Output(s)
Formula structure
◦ Test cases
◦ Constraints
Create: ◦ The optimal set of parameters
◦ An equation or algorithm
![Page 7: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/7.jpg)
Invention - GP Source: Wikimedia
![Page 8: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/8.jpg)
How – CGP
Cartesian Genetic Programming (CGP) ◦ Generates equations like circuits.
◦ Parallelizable results.
◦ FPGA friendly.
![Page 9: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/9.jpg)
How – CGP
Make a random “circuit”.
Grid layout.
+
*
+
&&
-
>
![Page 10: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/10.jpg)
How – CGP
Score the circuit.
+
*
+
&&
-
> Test #1: Terrible Test #2: Terrible Test #3: Terrible Score: Terrible*3
![Page 11: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/11.jpg)
How – CGP
Make random changes and rescore.
+
*
+
+
-
/ Test #1: Terrible Test #2: Terrible-4 Test #3: Terrible Score: Terrible*3-4
![Page 12: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/12.jpg)
How - CGP
Start with 1 parent “circuit”.
500
![Page 13: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/13.jpg)
How - CGP
Start with 1 parent “circuit”.
Make mutant children.
500 500
![Page 14: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/14.jpg)
How - CGP
Start with 1 parent “circuit”.
Make mutant children.
Score everyone.
500 500
500
725
![Page 15: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/15.jpg)
How - CGP
Start with 1 parent “circuit”.
Make mutant children.
Score everyone.
Promote the best child that isn’t worse than the parent.
500 500
500
725
500
![Page 16: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/16.jpg)
How - CGP
Start with 1 parent “circuit”.
Make mutant children.
Score everyone.
Promote the best child that isn’t worse than the parent.
500 500
500
725
500
85
97
85
![Page 17: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/17.jpg)
Neutral Search - Scores
500 500
500
725
500
85
97
85
500 500
500
725
500
400
817
400
New < Old
New <= Old
![Page 18: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/18.jpg)
Neutral Search – Mutations
0 0
1
1
1
2
2
2
0 0
1
1
0
1
1
1
New < Old
New <= Old
![Page 19: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/19.jpg)
Terminology
Ni
No
Nr
Nc
+ Operator type Constant Input indexes
Gene
![Page 20: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/20.jpg)
Code
Individual ◦ Representation
◦ Random creation
Execution
![Page 21: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/21.jpg)
Test Cases
Ni No Fitness (NP )
![Page 22: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/22.jpg)
Testing one test case
Ni No Fitness
Calculate Fitness
Typical vector operation: Fitness = | actual – expected |2
![Page 23: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/23.jpg)
Finish the loop
Ni No Fitness (NP )
∑ ∑ ∑ ∑ Sum fitness
Select best Promote Mutate
![Page 24: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/24.jpg)
Code
Test cases
Population
Mutator
![Page 25: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/25.jpg)
Demo Explained
Test cases Validation Image
Bayer Image
![Page 26: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/26.jpg)
Demo
CGP-based De-Bayer filter
![Page 27: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/27.jpg)
Scaling
Ni
No
Nr
Nc
O(No2+)
![Page 28: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/28.jpg)
Scaling – Simple Chromosomes
![Page 29: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/29.jpg)
Scaling – Complex Chromosomes
![Page 30: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/30.jpg)
Beyond functions - State
State data saved for next run.
![Page 31: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/31.jpg)
2 chromosomes with private and shared state
class Widget { private: float fState[2]; float gState[2]; float sharedState[4]; public: void f(const float input[7], float output[3]); void g(const float input[4], float output[1]); };
![Page 32: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/32.jpg)
Tips
Visualize!
Tests should give partial success
Increase population
Keep mutation rate low enough
![Page 33: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/33.jpg)
Future
CGP on GPU ◦ Especially validation!
Islands
More data visualizations
Parameter tuning with PSO
Open source
![Page 34: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/34.jpg)
</presentation>
Danke!
<shameless recruiting ad>
◦Join us at ptscientists.com! </shameless recruiting ad>
Wes Faler of Part-Time Scientists ◦ [email protected]
![Page 35: Automatic algorithm invention](https://reader034.vdocument.in/reader034/viewer/2022051314/557cbd90d8b42a09218b45d2/html5/thumbnails/35.jpg)
Resources
Julian Miller (inventor of CGP) ◦ http://sites.google.com/site/julianfrancismiller/professional
CGP Book ◦ “Cartesian Genetic Programming”
◦ http://www.springer.com/computer/theoretical+computer+science/book/978-3-642-17309-7
“Evolved to Win” e-book ◦ http://www.moshesipper.com/etw/