U Kang 1

Large Scale Data Analysis UsingDeep Learning

Course Introduction

U KangSeoul National University

U Kang 2

In This Lecture

Motivation to study deep learning

Administrative information for this course

U Kang 3

Outline

Deep Learning

Course Information

U Kang 4

Deep Learning as a Machine Learning

Machine Learning (ML)

Given x (predictor) and y (response), ML learns a function f() from data, such that y = f(x)

E.g., x = image, y = category

This learned function f() can be used to classify a new example x’

This is different from a typical programming where you want to compute y, given x and f()

Deep learning provides good performances in learning f() for many problems

Learns non-linear functions

U Kang 5

Deep Learning as a Machine Learning

Data Size

Accuracy Deep Learning

Other machine

learning

U Kang 6

Learning Tasks

Image classification

Speech recognition

Text classification

…

“Taxi”

Hello, dear

International politics

U Kang 7

Main Idea

Most perception (input processing) in the brain may use one learning algorithm

Design learning methods that mimic the brain

U Kang 8

Neurons In the Brain

U Kang 9

Neural Network

[LeCun et al.,

Nature 2015]

U Kang 10

Convolutional Neural Net (CNN)

U Kang 11

Representation Learning

Typical machine learning

Deep learning

Input OutputExtract

Features

x y

Input Output

x y

Extract

Features

Extract

Features

Extract

Features

Classifier

… Classifier

U Kang 12

Human Level Object Recognition

ImageNet ~ 15M labeled images, ~ 22K categories

Top-5 Error rates (1.2 million images, 1k categories) Non-CNN based method (~2012): 26.2 %

Alexnet (2012): 15.3 %

GoogLeNet (2014): 6.66 %

Resnet (2015): 3.57%Human level:

5.1% error

U Kang 13

Human-Level Face Recognition

DeepFace

97% accuracy

~ Human-level

[Taigman et al.

CVPR 2014]

U Kang 14

Computer Game

Deepmind

U Kang 15

Computer Game

Deepmind [Nature 2015]

https://www.youtube.com/watch?v=V1eYniJ0Rnk

U Kang 16

AlphaGo

[Silver et al., Mastering the game of Go with

deep neural networks and tree search, Nature 2016]

U Kang 17

Neural Artist

[Gatys et al., Image Style Transfer Using

Neural Networks, CVPR 2016]

U Kang 18

Machine Translation

U Kang 19

Topics in This Course (tentative)

(ch. 1) Introduction (ch. 2) Linear Algebra (ch. 3) Probability and Information Theory (ch. 4) Numerical Computation (ch. 5) Machine Learning Basics (ch. 6) Deep Feedforward Networks

(ch. 7) Regularization for Deep Learning (ch. 8) Optimization for Training Deep Models (ch. 9) Convolutional Networks (ch. 10) Sequence Modeling: Recurrent and Recursive Nets (ch. 11) Practical Methodology (ch. 12) Applications

U Kang 20

Outline

Deep Learning

Course Information

U Kang 21

M1522.001600, Spring 2017

http://datalab.snu.ac.kr/~ukang/courses/17S-DL/

Lecture slide: at least 1 hour before the lecture

Office hour

Mon. 11:00 – 12:00

Class meets: Mon, Wed 14:00 – 15:15, 301-101

U Kang 22

Textbook

Deep Learning (Ian Goodfellow, Yoshua Bengio, and Aaron Courville)

Available at http://www.deeplearningbook.org

U Kang 23

Prerequisites

Basic probability

Average, std. deviation, typical distributions, MLE, …

Basic linear algebra

Rank, singular value decomposition

Programming language

Python, (C++, Java)..

Machine Learning or Artificial Intelligence

Basic understandings of machine learning

U Kang 24

Grading

10% Attendance and Quiz (random)

30% Project

30% Midterm

30% Final

(+5% Participation)

U Kang 25

Late Policy

For all deliverables (homework, code, …)

No delay penalties, for medical etc. emergencies (bring doctor's note)

Each person has 4 'slip days' total, for the whole semester. 10% per day of delay, after that

U Kang 26

Project

A good opportunity to solve real world problems using deep learning

Team

A group of 3-4 persons

If you cannot find your team mate, discuss with the TA and/or instructor

U Kang 27

Project

Topic Improve the current status-of-the-art in deep learning

Deep learning applications E.g., Novel applications

Deep learning implementations E.g., Fast implementations using GPU

Method of deep learning E.g., New regularization method E.g., Achieve the best score in object recognition competition

We will provide some candidates, but feel free to propose your own topic

Feel free to discuss with the TA and/or the instructor before the proposal

U Kang 28

Project

Advice

Start your project from day 1: today!

Think of the data you are interested in (e.g. image, audio, text, graph, etc.), and how to get it

Read related papers

Find your team mates

Data should be ready very soon

In the worst case, it should be ready until the end of March

If you plan to collect the data later, you might not get it until the semester ends

U Kang 29

Project

Schedule and grading

Project proposal (due April 3) : 10%

Progress report (due May 3): 20%

Final report and presentation (due June 7) : 70%

U Kang 30

Advice

Read each chapter before class It is ok to encounter something you don’t understand. Just

mark it, and later you will understand it when you come back.

“Understand” intuitions of main ideas Do not memorize without understanding Improve your problem solving skills

Active participation encouraged All questions are right; ask many questions

Use office hours (instructor and/or TA)

Enjoy this course, and study hard!

U Kang 31

Questions?