![Page 1: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/1.jpg)
POKEMON HACKS
Jodie Ashford
Josh Baggott
Chloe Barnes
Jordan bird
![Page 2: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/2.jpg)
Why pokemon?
1997 – “The Pokemon Problem”
• an episode of the anime caused 685 children to have
seizures
• Professor Graham Harding from Aston University was
flown to japan
• He created a set of rules for tv that prevented this from
happening again in the future
What a legend
![Page 3: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/3.jpg)
Part #1
Machine learning on
brain activity to predict
whether someone is
watching pokemon
![Page 4: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/4.jpg)
Since all conscious human activities are
triggered in the brain, we can use EEG to
discern what a person is:
• Thinking
• Doing
• Feeling
Muse EEG HEADBAND
![Page 5: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/5.jpg)
Recorded 60 seconds of each
state for 4 people
120 seconds per person = 8 minutes of raw data
60 seconds watching pokemon
60 seconds watching a tutorial on Microsoft word from 1978
(it was very boring)
![Page 6: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/6.jpg)
Brainwave data is dynamic
and temporal, but we
need static data
Time windowing technique
Data point = the statistics of a time window
![Page 7: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/7.jpg)
Temporal Extraction of
0.5 second windows of EEG data,
every 0.25s interval
![Page 8: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/8.jpg)
Statistics are extracted from each
time window to produce one row of
data
• Mean value
• 0.25s and 0.5s window
• Max value
• 0.25s and 0.5s window
• Min value
• 0.25s and 0.5s window
• Standard deviation
• Statistical moments
• 3rd, 4th order
• Mean value distance
• Max value distance
• Min value distance
• Log-covariance
• Shannon entropy
• Log-energy entropy
• Accumulative energy
features
![Page 9: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/9.jpg)
We used a premade script to do this
part!
WatchingOr
Not watching(CLASS)
2500+ attributes for
each time window…
The dataset was huge
57 Megabytes!
![Page 10: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/10.jpg)
USING THIS DATA WE TRAINED VARIOUS
MACHINE LEARNING MODELS
• Naïve Bayes
• Bayesian Network
• J48 Tree • Java implementation of the C4.5 Algorithm
• Random Tree
• Random Forest
• MLP NEURAL NETWORK
• SUPPORT VECTOR MACHINE (SVM)
For this we performed machine learning
on the google cloud platform
![Page 11: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/11.jpg)
RESULTS
Model Prediction Accuracy (%)
Naive Bayes 60.45
Bayesian Network 75.11
J48 Tree 85.24
Random Tree 80.82
Random Forest 94.07
MLP Neural Network 86.42
Support Vector
Machine71.76
![Page 12: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/12.jpg)
But JUST how accurate is it?
• We record data at 150hz (150 times per second)
• At 94.07% this means that in 10 seconds of
recording, we misclassify only around half a
second…
WE. CAN. READ. YOUR. MIND.
![Page 13: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/13.jpg)
NOTE:
SINCE THE FRONTAL LOBE (AF7, AF8) is
CLOSELY RELATED TO EMOTIONs, WE
ARE ACTUALLY PROBABLY LEARNING TO
CLASSIFY A POSITIVE EMOTIONAL
EXPERIENCE
![Page 14: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/14.jpg)
Part #2
Designing a Pokemon
game with Machine
Learning
![Page 15: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/15.jpg)
Generating images of Pokemon
Sprites from various Pokemon games were
compiled into a large image dataset
![Page 16: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/16.jpg)
Generating images of Pokemon
A Deep Convolutional Generative Adversarial Neural
Network (DCGAN) trained on this data for 3 hours and
created new images inspired by what it had seen
![Page 17: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/17.jpg)
How does a DCGAN work?
1. Team Rocket forge a painting
2. Professor Oak learns to spot their forgery
3. Team Rocket have to learn produce a better forgery
4. Professor Oak has to learn to spot a better forgery
5. Repeat
VS.
![Page 18: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/18.jpg)
How does a DCGAN work?
1. Generator forges a image
2. Discriminator learns to spot the forgery
3. Generator has to learn produce a better forgery
4. Discriminator has to learn to spot a better forgery
5. Repeat
VS.
![Page 19: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/19.jpg)
Generating images of Pokemon
Generation 1 to 200
(3 hours of learning)
![Page 20: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/20.jpg)
Generating images of Pokemon
The process worked quite well and made great progress,
but it would take a lot longer to generate Pokemon with
more discernible features
![Page 21: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/21.jpg)
Generating names and descriptions of Pokemon
We fed the whole Pokedex (names and descriptions) to
Long Short Term Memory neural networks to generate new
text
![Page 22: POKEMON HACKS - Jordan J. Birdjordanjamesbird.com/wp-content/uploads/2018/11/aston... · 2018. 11. 13. · Sprites from various Pokemon games were compiled into a large image dataset](https://reader036.vdocument.in/reader036/viewer/2022090506/601bd65bb2452d3dec42f354/html5/thumbnails/22.jpg)
Some examples of the horrible abominations we made
Mowirup
They flock to the stars and mountains. This Pokemon glows, it does not construct
silk.
Bigabble
Very powerful, it absorbs filthy atmosphere. They check the air. It nurses the
food colony. Rampant birds.
Moomstu
It chews to show strength. It starts rivers to sleep. Capable to twitch
coughing bouncy gas. Shock and unknown fangs hide in the tree.