Upload File

deep learning concepts from theory to practice learning concept… · deep learning i specialized...

  • Home
  • Documents
  • Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans
14
Deep Learning Concepts from Theory to Practice January 19, 2016 Patrick Oliver GLAUNER and Dr. Radu STATE [email protected], [email protected] SEDAN Lab, SnT - Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg

Upload: others

Post on 22-May-2020

12 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Deep Learning Concepts from Theory to Practice

January 19, 2016

Patrick Oliver GLAUNER and Dr. Radu STATE

[email protected], [email protected]

SEDAN Lab,SnT - Interdisciplinary Centre for Security, Reliability and Trust,University of Luxembourg

mailto:[email protected]
mailto:[email protected]
Page 2: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Biography

I Joined SnT in September 2015 as a PhD studentI Collaboration with Choice Technologies Holding on detection of

non-technical losses (NTLs)I MSc in Machine Learning from Imperial College LondonI Previously worked at CERN and SAP

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 1 / 13

Page 3: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Motivation

Definition (Artificial Intelligence)"AI is the science of knowing what to do when you don’t know what todo." (Peter Norvig)a

ahttp://www.youtube.com/watch?v=rtmQ3xlt-4A4m45

Definition (Machine Learning)Machine Learning is the field of study that gives computers the abilityto learn without being explicitly programmed.

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 2 / 13

http://www.youtube.com/watch?v=rtmQ3xlt-4A4m45
Page 4: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Motivation

I Deep Learning attracted major IT companies including Google,Facebook, Microsoft and Baidu to make significant investments

I Learning features from data rather than modeling themI Specialized artificial intelligences have started to outperform

humans on certain tasksI Advances have been raising many hopes about the future of

machine learning

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 3 / 13

Page 5: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Agenda

1. Neural networks2. Deep Learning3. Event-driven stock prediction4. Conclusions and outreach

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 4 / 13

Page 6: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Neural networks

Figure: Neural network with two input and output units 1.

1Bishop, Christopher M.: Pattern Recognition and Machine Learning. Springer.2007.

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 5 / 13

Page 7: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Neural networks

Figure: History of neural networks 2.

2Deng, Li and Yu, Dong: Deep Learning Methods and Applications. Foundationsand Trends in Signal Processing, 7 (3-4), 197-387. 2014.

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 6 / 13

Page 8: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Deep Learning

Figure: Deep neural network layers learning complex feature hierarchies 3.

3The Analytics Store: Deep Learning.http://theanalyticsstore.com/deep-learning/. Retrieved: March 1, 2015.

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 7 / 13

http://theanalyticsstore.com/deep-learning/
Page 9: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Deep Learning

I Specialized artificial intelligences based on Deep Learning havestarted to outperform humans on certain tasks

I Training time can accelerated using GPUs 4 5 or a distributedenvironment, such as Apache Spark 6

4Bergstra, J.; Breuleux, O.; Bastien, F.; Lamblin, P.; Pascanu, R.; Desjardins, G.;Turian, J.; Warde-Farley, D. and Bengio, Y.: Theano: A CPU and GPU MathExpression Compiler. Proceedings of the Python for Scientific Computing Conference(SciPy) 2010. June 30 - July 3, Austin, TX. 2010.

5NVIDIA: TESLA. http://www.nvidia.com/object/tesla-servers.html.Retrieved: August 20, 2015.

6Apache Spark. http://spark.apache.org/. Retrieved: November 3, 2015.P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 8 / 13

http://www.nvidia.com/object/tesla-servers.html
http://spark.apache.org/
Page 10: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Deep Learning: DeepMindI Founded in 2010 in LondonI Created a neural network that learns how to play video games in a

similar fashion to humansI Acquired by Google in 2014, estimates range from USD 400

million to over GBP 500 millionI Now being used in Google’s search engine

Figure: Google DeepMind 7.

7http://deepmind.com/. Retrieved: January 15, 2016.P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 9 / 13

http://deepmind.com/
Page 11: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Event-driven stock prediction

Figure: Example news influence of Google Inc. 8.

8Ding, Xiao; Zhang, Yue; Liu, Ting and Duan, Junwen: Deep Learning forEvent-Driven Stock Prediction. Proceedings of the Twenty-Fourth International JointConference on Artificial Intelligence (IJCAI 2015), Buenos Aires, Argentina. 2015.

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 10 / 13

Page 12: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Event-driven stock prediction

Figure: Accuracies of prediction for selected stocks from S&P 500 9.

9Ding et al. (2015)P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 11 / 13

Page 13: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Event-driven stock predictionI This model combines influence of long-term, mid-term and

short-term events on stock price movementsI It significantly outperforms state-of-the-art models by an extra 6%

of accuracy, in particular for stocks with low amount of news

Table: Compared to baseline from Feb. to Nov. 2013 using 35,603 news 10.

10Ding et al. (2015)P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 12 / 13

Page 14: Deep Learning Concepts from Theory to Practice Learning Concept… · Deep Learning I Specialized artificial intelligences based on Deep Learning have started to outperform humans

Conclusions and outreach

I Deep neural networks can learn complex feature hierarchiesI Significant speedup of training due to GPU accelerationI About to be applied to the detection of NTLsI Has been successfully applied to stock predictionI Promising methods, lots of potential to be applied to FinTechI SEDAN Lab is happy to provide more details on Deep Learning

and to discuss potential joint projects!

P. GLAUNER and R. STATE (SnT) Deep Learn. Concepts f. Theory to Practice January 19, 2016 13 / 13


20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner Deep Learning Overview 2 ... coursera ... 20151216 Computer Vision Meetup Deep Learning

Deep Learning I - Korea Universitydsba.korea.ac.kr/wp/wp-content/seminar/Deep learning/Deep Learning... · Deep Learning I Deep learning is research on learning models with multilayer

NPDL Global Report - New Pedagogies for Deep Learning · Deep Learning 7 The 6Cs 7 Deep Learning Progressions 9 Impact 16 Key Findings: Deep Learning 18 New Pedagogies 19 Learning

Deep Learning for Depth Learning CS 229 Course Project ...cs229.stanford.edu/...DepthLearning_FinalReport.pdf · Deep Learning for Depth Learning CS 229 Course Project, ... Deep Learning

Deep Learning - Oxford Statisticspalamara/teaching/SML19/HT19_lecture15.pdf · SB2b/SM4 - Deep Learning Deep Learning Infrastructure! Computational Infrastructure critical to deep

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning

Binary Deep Learning - TAUweb.eng.tau.ac.il/.../wp-content/uploads/2017/03/Binary-Deep-Learning.pdf · Binary Deep Learning Deep Learning Seminar, School of Electrical Engineering,

Tutorial: Deep Reinforcement Learning · Outline Introduction to Deep Learning Introduction to Reinforcement Learning Value-Based Deep RL Policy-Based Deep RL Model-Based Deep RL

Modern Deep Learning through Bayesian Eyesmlg.eng.cam.ac.uk/yarin/PDFs/2015_UCL_Bayesian_Deep_Learning_t… · Modern Deep Learning through Bayesian Eyes ... deep learning models

Demystifying Deep Learning - MathWorks · Demystifying Deep Learning. 2 What’s Deep Learning and why should I care? A practical approach to Deep Learning (for images) –Transfer

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Deep materials informatics: Applications of deep learning in … · deep learning: † Deep learning requires big data: In many cases, the biggest limiting factor for applying deep

Deep Learning - South Dakota State UniversityDeep Learning Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine

DEEP LEARNING FOR LONG-TERM VALUE INVESTING€¦ · COMPANY STRUCTURE Joint Venture between ‣ Asset manager since 1994 ‣ Value philosophy ‣ Funds outperform on the long run

Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

Introducing Deep Learning with MATLAB - MathWorks · Introducing Deep Learning with MATLAB7 How A Deep Neural Network ... relevant features of an image. With deep learning, ... Deep

Lecture 3: Deep Learning Optimizations · UVA DEEP LEARNING COURSE EFSTRATIOS GAVVES DEEP LEARNING OPTIMIZATIONS -3 Empirical Risk Minimization

Lecture 3: Deep Learning Optimizations - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture3.pdf · Lecture 3: Deep Learning Optimizations Deep Learning @ UvA. UVA DEEP LEARNING

Deep Reinforcement Learning applied to the game Bubble Shooter · the past chess computers were created, which were able to outperform hu-mans. The chess computer DeepBlue beat the

Introducing Deep Learning with MATLAB - Systematics · Introducing Deep Learning with MATLAB6 Inside a Deep Neural Network ... Deep learning is a subtype of machine learning. With

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Deep Learning with H2O - University of Idahostevel/504/Deep Learning with...Deep Learning with H2O. . 4Deep Learning Overview Unlike the neural networks of the past, modern Deep Learning

Learning Deep Learning

Machine Learning Pipelines with Apache Spark and Intel BigDL...2019/01/24  · Deep Learning proved to be able to outperform other ML methods • See paper: “Topology classification

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Deep Learning in real world @Deep Learning Tokyo

Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

Abstract - arXiv · Recently, the emergence of deep learning has enabled the development of new intelligent diagnostics based on computer vision. CNNs outperform many state-of-the-art

Lecture 1: Introduction to Deep Learning · uva deep learning course –efstratios gavves introduction to deep learning - 1

Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

Deep neural networks outperform human expert's capacity in

Hybrid Convolutional Recurrent Neural Networks Outperform CNN … · 2020. 2. 3. · study, four deep learning algorithm architectures, including two convention deep learning models

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Intro to Deep Learning - bpTT Bright Minds … · 17/08/2020  · Intro to Deep Learning Deep Sequence Modeling Deep Computer Vision Deep Generative Modeling Deep Learning Applications

UNSUPERVISED DEEP LEARNING - TAUweb.eng.tau.ac.il/.../uploads/2016/12/Unsupervised-Deep-Learning.pdf · UNSUPERVISED DEEP LEARNING Erez Aharonov Noam Eilon Deep Learning Seminar School