Genetic Algorithms and Genetic Algorithms and Neural NetworksNeural Networks

MIT Splash 2006MIT Splash 2006

Jack CarrozzoJack Carrozzo

ApplicationsApplications

GAs need a configuration string/array of GAs need a configuration string/array of data: 01001010, actgtggcata, data: 01001010, actgtggcata, sdlkdsdflk0flk3ds, 0xdeadbabe…sdlkdsdflk0flk3ds, 0xdeadbabe…

Examples: the Traveling Salesman Problem, Examples: the Traveling Salesman Problem, Puzzles, Mazes, Games (Chess…), Puzzles, Mazes, Games (Chess…), problems where you are striving and can problems where you are striving and can get close to a goalget close to a goal

The Golden FunctionThe Golden Function

The “Fitness Function” is the deciding factor The “Fitness Function” is the deciding factor in using genetic algorithms: if you can’t in using genetic algorithms: if you can’t come up with one, the problem is not come up with one, the problem is not solvable by genetics.solvable by genetics.

This function rates a given genetic string by This function rates a given genetic string by how close it comes to solving the problem how close it comes to solving the problem or reaching the goal. Bad combinations or reaching the goal. Bad combinations are weeded out in this way.are weeded out in this way.

Finding FitnessFinding Fitness

The fitness function describes how far from The fitness function describes how far from out goal we are. Say perhaps we’re out goal we are. Say perhaps we’re designing an antenna to be most efficient designing an antenna to be most efficient at 2.54 GHz. If we use positive sorting at 2.54 GHz. If we use positive sorting (wait 5 minutes) and the range/power as (wait 5 minutes) and the range/power as our fitness function, successive our fitness function, successive permutations will lead to antennas with a permutations will lead to antennas with a greater range to power ratio.greater range to power ratio.

The AlgorithmThe Algorithm

If we are searching for a solution set to our If we are searching for a solution set to our problem, we first assign our solution problem, we first assign our solution threads random values within our threads random values within our expected range. We sort these in order of expected range. We sort these in order of their fitness to the problem; the best fit we their fitness to the problem; the best fit we keep, the next best we split, and the worst keep, the next best we split, and the worst we regenerate from random. we regenerate from random.

On ThreadsOn Threads

The generally accepted method for threads The generally accepted method for threads is to use 4 or 8 on a “standard” sort of is to use 4 or 8 on a “standard” sort of problem with one CPU. There are times problem with one CPU. There are times where we may have many more CPUs, in where we may have many more CPUs, in which case we assign each thread its own which case we assign each thread its own processor (16, 32, 64…). Anyone here processor (16, 32, 64…). Anyone here with a box like that, you are l33t.with a box like that, you are l33t.

Sorting The ThreadsSorting The Threads

During each propagation, we run the fitness During each propagation, we run the fitness function on each solution thread, and function on each solution thread, and create a list in order of how well each create a list in order of how well each configuration solves our problem. The configuration solves our problem. The sorting can be either negative or positive sorting can be either negative or positive depending on the problem.depending on the problem.

CrossbreedingCrossbreeding

Let’s say we have 8 solution threads, sorted Let’s say we have 8 solution threads, sorted in order of fitness. A good rule of thumb is in order of fitness. A good rule of thumb is to keep the first two, cross the next two, to keep the first two, cross the next two, randomly generate half the first and randomly generate half the first and second half of the next two respectively, second half of the next two respectively, and finally recreate the final two from and finally recreate the final two from scratch (randomly).scratch (randomly).

The LoopThe Loop

That’s it! Solving GAs is as simple as That’s it! Solving GAs is as simple as repeating the previous steps many times repeating the previous steps many times until your error or fitness is either a) within until your error or fitness is either a) within a good enough range or b) not changing a good enough range or b) not changing anymore (no better matches are being anymore (no better matches are being found).found).

A Note on MaximaA Note on Maxima

As you may recall from Calculus, there are As you may recall from Calculus, there are two types of maxima: local and absolute. If two types of maxima: local and absolute. If you only use crossbreeding and half-you only use crossbreeding and half-random recreation (ie, no full random random recreation (ie, no full random recreation) as your thread modification recreation) as your thread modification methods, there is a good chance that you methods, there is a good chance that you will get stuck at a fairly good solution, but go will get stuck at a fairly good solution, but go no higher. Full random threads ensure there no higher. Full random threads ensure there is a chance to break out of local minima.is a chance to break out of local minima.

Case Study: Traveling SalesmanCase Study: Traveling Salesman

Case Study: PokerCase Study: Poker

My solution string was an ugly 240-char My solution string was an ugly 240-char array with data defining what the bot would array with data defining what the bot would do with a given hand, game options, and do with a given hand, game options, and recent game history. Had it run for a long recent game history. Had it run for a long time more, it would have done pretty time more, it would have done pretty well…well…

Neural NetsNeural Nets

Look at all those pretty circles and lines!Look at all those pretty circles and lines!

The PartsThe Parts

Nodes: Keep values to pass to the other Nodes: Keep values to pass to the other nodes, as well as take input or give output. nodes, as well as take input or give output. Each node applies the “activation Each node applies the “activation function”.function”.

Links: The connections between nodes. Links: The connections between nodes. Each link has a specific “weight” that Each link has a specific “weight” that changed the values as they pass to the changed the values as they pass to the other node.other node.

The Forward PassThe Forward Pass

1)1) Set the hidden layer’s values to the sum Set the hidden layer’s values to the sum of each input node multiplied by their of each input node multiplied by their perspective weights. perspective weights.

2)2) Run the activation function on the hidden Run the activation function on the hidden nodes.nodes.

3)3) Do the same from the hidden to output Do the same from the hidden to output layer.layer.

Or mathematically…Or mathematically…

VV1J1J==((vv0i0i*w*wiJiJ))

LOLZ OMG WTF?LOLZ OMG WTF?

So… you just ran that, and BOOM you get a So… you just ran that, and BOOM you get a number out. Is it what you expected?number out. Is it what you expected?

NO!NO!

It’s completely useless… but we aim to fix It’s completely useless… but we aim to fix that ;-)that ;-)

Training: The Backwards PassTraining: The Backwards Pass

The most-used type of training is called The most-used type of training is called “Back Propagation”, because we calculate “Back Propagation”, because we calculate the specific error for each node and the specific error for each node and propagate backwards to fix each layer. propagate backwards to fix each layer. This is why we refer to this type of net as This is why we refer to this type of net as “back propagation neural networks”.“back propagation neural networks”.

Training: Finding GradientsTraining: Finding Gradients

If j is an output node: If j is an output node:

jj==’(V’(Vjj)()(jj)) = expected-actual= expected-actual

If j is a hidden node:If j is a hidden node:

jj==’(V’(Vjj)()( w wjkjk**kk))

’ ’ is the derivative of the activation function, which is the derivative of the activation function, which is is ’(x)=x(x-1)’(x)=x(x-1)

Training: Fixing the weightsTraining: Fixing the weights

The change in each weight, or The change in each weight, or wwii, is as , is as

follows:follows:

wwii==xxiiii

And thus the weight update is:And thus the weight update is:

wwjj= w= wjj + + wwjj

Architecture: Size, LengthArchitecture: Size, Length

How big should your net be? How many How big should your net be? How many hidden layers? Net memory?hidden layers? Net memory?

How much training do you need to do? Data How much training do you need to do? Data set size? Overtraining?set size? Overtraining?

Simple Test: XORSimple Test: XOR

XOR Truth Table:XOR Truth Table:0,0 0,0 0 00,1 0,1 1 11,0 1,0 1 11,1 1,1 0 0

(If one and only one input it high, go high, (If one and only one input it high, go high, else go low)else go low)

Computability With NNsComputability With NNs

Neural Nets are best used to draw Neural Nets are best used to draw conclusions in data that general conclusions in data that general mathematics cannot find, and can’t be mathematics cannot find, and can’t be seen by the human eye:seen by the human eye:

Gold DigGold DigText to SpeechText to SpeechCameras: finding tanks, driving Cameras: finding tanks, driving

cross-cross- countrycountryReading HandwritingReading Handwriting

w00tw00t

Have fun, if you do something awesome (or Have fun, if you do something awesome (or really, anything at all) I want to hear about really, anything at all) I want to hear about

it!it!

ThanksThanks

Jack CarrozzoJack Carrozzo

jack@crepinc.comjack@crepinc.com