deep learning: a brief overview on the possibility for ... · deep learning for studies of galaxy...
TRANSCRIPT
Deep Learning: a brief overview on the possibilityfor astrophysics
Francois-Xavier Dupe
(LIS/Aix-Marseille Universite, France)
Journee SKA @ LAM
Introduction
What is Machine Learning?
The aim of Machine Learning is to build a mathematical functionwhich solve a human task.
Today tasks include
classification/regression;
representation (or feature) learning;
transfert learning;
reinforcement learning;
. . .
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 2 / 25
Introduction
What is Machine Learning?
The aim of Machine Learning is to build a mathematical functionwhich solve a human task.
Today tasks include
classification/regression;
representation (or feature) learning;
transfert learning;
reinforcement learning;
. . .
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 2 / 25
Introduction
Some history about Machine Learning
1956: the Dartmouth workshop
Proposal
We propose that a 2 month, 10 man study of artificial intelligence becarried out during the summer of 1956 at Dartmouth College inHanover, New Hampshire. The study is to proceed on the basis of theconjecture that every aspect of learning or any other feature ofintelligence can in principle be so precisely described that a machinecan be made to simulate it. An attempt will be made to find how tomake machines use language, form abstractions and concepts, solvekinds of problems now reserved for humans, and improve themselves.We think that a significant advance can be made in one or more ofthese problems if a carefully selected group of scientists work on ittogether for a summer.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 3 / 25
Introduction
Some history about Machine Learning (2)
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 4 / 25
Introduction
A brief timeline
from Andrew L. Beam
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 5 / 25
Introduction
Today’s talk
1 Deep learning
2 Interactions with astrophysics
3 What next?
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 6 / 25
Deep learning
Today’s talk
1 Deep learning
2 Interactions with astrophysics
3 What next?
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 7 / 25
Deep learning
What is it?
Deep learning ⇒ hierarchical learning with high order features.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 8 / 25
Deep learning
What is it?
Deep learning ⇒ hierarchical learning with high order features.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 8 / 25
Deep learning
The location of deep learning
From https://www.machinecurve.com/index.php/2017/09/30/
the-differences-between-artificial-intelligence-machine-learning-more/
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 9 / 25
Deep learning
The zoo
From http://www.asimovinstitute.org/neural-network-zoo/
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 10 / 25
Deep learning
AlexNet
ImageNet Classification with Deep Convolutional Neural Networks byA. Krizhevsky et al (NIPS 2012)
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 11 / 25
Deep learning
AlexNet (results)
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 12 / 25
Deep learning
Long Short-Time Memory (LSTM)
Example of a recurrent neural network.
From http://colah.github.io/posts/2015-08-Understanding-LSTMs/
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 13 / 25
Deep learning
Long Short-Time Memory (LSTM): details
Input gate: encode theinput data.
Output gate: create thenext output (from theinput).
Forget gate: removeinformation from inputdata.
Block input: output datafrom another block.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 14 / 25
Deep learning
Generative Adversarial Networks (GAN)
Idea: building a generator that can fool a discriminator
Generator: a NN thatproduce new data fromnoise.
Discriminator: aclassifier which distinguishfake data from true.
A set of real samples.
Generative adversarial nets by I. Goodfellow et al (NIPS 2014)
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 15 / 25
Deep learning
Generative Adversarial Networks (GAN): some results
From https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 16 / 25
Interactions with astrophysics
Today’s talk
1 Deep learning
2 Interactions with astrophysics
3 What next?
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 17 / 25
Interactions with astrophysics
Some examples
Star-galaxy Classification Using Deep Convolutional Neural Networksby Edward J. Kim Robert J. Brunner (MNRAS 2016)
Idea: Use ConvNet on the reduced calibrated pixel values.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 18 / 25
Interactions with astrophysics
Some examples
Star-galaxy Classification Using Deep Convolutional Neural Networksby Edward J. Kim Robert J. Brunner (MNRAS 2016)
Idea: Use ConvNet on the reduced calibrated pixel values.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 18 / 25
Interactions with astrophysics
Some examples (2)
Fast Cosmic Web Simulations with Generative Adversarial Networksby A.C. Rodrıguez et al (arXiv:1801.09070)
Idea: use deep learning to avoid the computational cost of N-bodysimulation.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 19 / 25
Interactions with astrophysics
Some examples (2)
Fast Cosmic Web Simulations with Generative Adversarial Networksby A.C. Rodrıguez et al (arXiv:1801.09070)
Idea: use deep learning to avoid the computational cost of N-bodysimulation.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 19 / 25
Interactions with astrophysics
Some other examples from arXiv
Deep learning for studies of galaxy morphology (arXiv:1701.05917)
Detecting Solar-like Oscillations in Red Giants with Deep Learning(arXiv:1804.07495)
Lunar Crater Identification via Deep Learning (arXiv:1803.02192)
Deep learning from 21-cm images of the Cosmic Dawn (arXiv:1805.02699)
Fast Point Spread Function Modeling with Deep Learning (arXiv:1801.07615)
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 20 / 25
What next?
Today’s talk
1 Deep learning
2 Interactions with astrophysics
3 What next?
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 21 / 25
What next?
Explainable Machine Learning
Deep learning a black-box?
Source: Department of Defense, Advanced Research Projects Agency.
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 22 / 25
What next?
Evaluation
One of the open question in Machine Learning: how to fullyevaluate a method?
No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.
By OswaldLR - From: A Corny Concerto (2).png, Public Domain
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 23 / 25
What next?
Evaluation
One of the open question in Machine Learning: how to fullyevaluate a method?
No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.
Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.
By OswaldLR - From: A Corny Concerto (2).png, Public Domain
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 23 / 25
What next?
Evaluation
One of the open question in Machine Learning: how to fullyevaluate a method?
No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.
By OswaldLR - From: A Corny Concerto (2).png, Public DomainF.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 23 / 25
Conclusion
Take away message
Evaluation is primordial! Must be coherent with the task.
Learning representation is a way to avoid some bias.
Deep learning asks for big datasets, but scale very well.
There is no free lunch!
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 24 / 25
Conclusion
Take away message
Evaluation is primordial! Must be coherent with the task.
Learning representation is a way to avoid some bias.
Deep learning asks for big datasets, but scale very well.
There is no free lunch!
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 24 / 25
Conclusion
Take away message
Evaluation is primordial! Must be coherent with the task.
Learning representation is a way to avoid some bias.
Deep learning asks for big datasets, but scale very well.
There is no free lunch!
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 24 / 25
Conclusion
Take away message
Evaluation is primordial! Must be coherent with the task.
Learning representation is a way to avoid some bias.
Deep learning asks for big datasets, but scale very well.
There is no free lunch!
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 24 / 25
Conclusion
Thank you for your attention.
Any questions?
F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 25 / 25