introduction to deep learningintroduction to deep learning optimization cnn introduction to nn...
TRANSCRIPT
![Page 1: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/1.jpg)
Introduction to Deep Learning
1I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 2: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/2.jpg)
2
Lecturers
PhDs
I2DL: Prof. Niessner, Prof. Leal-Taixé
Prof. Dr. Laura Leal-Taixé
Prof. Dr. Matthias Niessner
Patrick Dendorfer
AndreasRössler
The Team
![Page 3: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/3.jpg)
What is Computer Vision?
• First defined in the 60s in artificial intelligence groups
• “Mimic the human visual system”
• Center block of robotic intelligence
3I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 4: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/4.jpg)
Hubel and Wiesel• David Hubel and Torsten Wiesel
were neurobiologists from Harvard Medical School
• Experiment revealed several secrets of the human vision system
• Won 2 Nobel prizes
4I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 5: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/5.jpg)
Hubel and Wiesel Experiment• Recorded electrical activity from
individual neurons in the brains of cats.
• Slide projector to show specific patterns to the cats noted specific patterns stimulated activity in specific parts of the brain.
• Results: Visual cortex cells are sensitive to the orientation of edges but insensitive to their position
5I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 6: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/6.jpg)
6I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 7: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/7.jpg)
Computer Vision
Few Decades Later…
7I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 8: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/8.jpg)
Computer Vision
Physics PsychologyBiology
MathematicsEngineering Computer
scienceArtificial Intelligence
ML
Neuroscience
AlgorithmsOptimization
NLPSpeech
Robotics
OpticsImage
processing
8I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 9: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/9.jpg)
Pre 2012
Image Classification
9I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 10: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/10.jpg)
AAwesome magic box
Become magicians Post 2012Open the box
Image Classification
10I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 11: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/11.jpg)
Why Deep Learning?
11I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 12: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/12.jpg)
Deep Learning History
12I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 13: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/13.jpg)
The Empire Strikes Back
13
0
5
10
15
20
25
30
2010 2011 2012AlexNet
2013 2014VGGNet
Human 2015ResNet
2016Ensemble
2017SENet
ILSVRC top-5 error on ImageNet
Deep Learning Approaches
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 14: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/14.jpg)
• MNIST digit recognition dataset
• 107 pixels used in training
• ImageNet image recognition dataset
• 1014 pixels used in training
1998LeCunet al.
2012Krizhevskyet al.
What Has Changed?
14I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 15: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/15.jpg)
Big Data
Models know where to learn from
Hardware
Models are trainable
Deep
Models are complex
What Made this Possible?
15I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 16: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/16.jpg)
Deep Learning Recognition
16
ACM Turing Award 2019 (Nobel Prize of Computing)Yann LeCun, Geoffrey Hinton, and Yoshua Bengio
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 17: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/17.jpg)
Credits: Dr. Pont-Tuset, ETH Zurich
Deep Learning and Computer Vision
17I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 18: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/18.jpg)
Credits: Dr. Pont-Tuset, ETH Zurich
Deep Learning and Computer Vision
18I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 19: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/19.jpg)
Deep Learning Today
19Object Detection
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 20: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/20.jpg)
Self-driving cars
Deep Learning Today
20I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 21: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/21.jpg)
AlphaGo
Machine translation
Emoticon suggestion
Deep Learning Today
21I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 22: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/22.jpg)
Deep Learning Today
22Alpha StarI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 23: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/23.jpg)
Deep Learning Today
23
Automated Text Generation [Karpathy et al.]
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 24: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/24.jpg)
Deep Learning Today
24
Google Assistant (Google IO’19)I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 25: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/25.jpg)
Healthcare, cancer detection
Deep Learning Today
25I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 26: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/26.jpg)
Deep Learning Market
[…] market research report Deep Learning Market […] “ the deep learning market is expected to be worth USD 1,722.9 Million by 2022.
26I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 27: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/27.jpg)
Deep Learning Job Perspective• Excellent Job Perspectives!
– Automation requires ML/DL -> growth!– Top-notch companies will gladly hire you!
• Many industries now:– IT-Companies– Cars, Logistic, Health Care, etc…– Manufacturing / Robotics, etc…
27I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 28: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/28.jpg)
But: Also Challenging!• High-level understanding is not enough
– Need proper theory background– Need proper practical skillsets
• Can be competitive!– Many good people– Downloading scripts / running code not enough – Deeper understanding often requires PhDs
28I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 29: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/29.jpg)
Deep Learning Culture
29I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 30: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/30.jpg)
Deep Learning Memes
30I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 31: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/31.jpg)
Deep Learning Memes
31I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 32: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/32.jpg)
Deep Learning Memes
32I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 33: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/33.jpg)
Deep Learning Memes
33I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 34: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/34.jpg)
Deep Learning Memes
34I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 35: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/35.jpg)
Deep Learning Memes
35I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 36: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/36.jpg)
Deep Learning at TUM
36I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 37: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/37.jpg)
Many TUM Research Labs use DL• Visual Computing Lab (Prof. Niessner):
– Research in computer vision, graphics, and machine learning• Dynamic Vision and Learning Group (Prof. Leal-Taixe)
– Research on Computer Vision; e.g., video editing/segmentation etc.• 3D Understanding Lab (Dr. Dai):
– Research in 3D scenes and its semantics.• Computer Vision Group (Prof. Cremers)
– Research in computer vision and pattern recognition• Data Mining and Analytics Lab (Prof. Günnemann)
– Research methods for robust machine learning• Computer Aided Medical Procedures (Prof. Navab)
– Research in machine learning for medical applications• And probably many more
37I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 38: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/38.jpg)
Our Research Labs• Visual Computing Lab (Prof. Niessner):
https://niessnerlab.org/publications.html– Twitter: https://twitter.com/MattNiessner– Youtube: https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q– Facebook: https://www.facebook.com/matthias.niessner
• Dynamic Vision and Learning Lab (Prof. Leal-Taixé):https://dvl.in.tum.de/publications.html– Twitter: https://twitter.com/lealtaixe– Youtube: https://www.youtube.com/channel/UCQVCsX1CcZQr0oUMZg6szIQ
38I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 39: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/39.jpg)
[Caelles et al., CVPR’ 17] One-Shot Video Object Segmentation
Deep Learning at TUM
39I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 40: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/40.jpg)
Deep Learning at TUM
40[Dosovitskiy et al., ICCV’ 15] FlowNetI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 41: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/41.jpg)
Deep Learning at TUM
41
CC3
CC2
CC1
Reshape Conv+BN+ReLU Pooling Upsample Concat Score
DDFF
[Hazirbas et al., IJCV’18] Deep Depth From Focus.I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 42: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/42.jpg)
Deep Learning at TUM• Video anonymization
42
[Maximov et al., CVPR 2020] CIAGAN: Conditional identity anonymization generative adversarial networks.I2DL: Prof. Niessner, Prof. Leal-Taixé
So
urce
Control identity
![Page 43: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/43.jpg)
Deep Learning at TUM• Multiple object tracking with graph neural networks
43
[Brasó and Leal-Taixé, CVPR 2020] Learning a Neural Solver forMultiple Object Tracking.I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 44: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/44.jpg)
Deep Learning at TUM
44[Yang et al., ECCV’ 18] Deep Virtual Stereo Odometry I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 45: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/45.jpg)
Deep Learning at TUM
45[Xie et al. Siggraph’ 18] tempoGANI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 46: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/46.jpg)
Deep Learning at TUM
47[Dai et al., CVPR’17] ScanNet
ScanNet Stats:-Kinect-style RGB-D sensors-1513 scans of 3D environments-2.5 Mio RGB-D frames-Dense 3D, crowd-source MTurk labels-Annotations projected to 2D frames
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 47: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/47.jpg)
Deep Learning at TUM
48[Hou et al., CVPR’19] 3D Semantic Instance Segmentation
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 48: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/48.jpg)
Deep Learning at TUM
49[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 49: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/49.jpg)
Deep Learning at TUM
50[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 50: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/50.jpg)
Deep Learning at TUM
51[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 51: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/51.jpg)
Deep Learning at TUM
52[Thies et al., Siggraph’19 ]: Neural TexturesI2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 52: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/52.jpg)
Deep Learning at TUM
53
[Roessler et al., ICCV’19 ]: Face Forensics++
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 53: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/53.jpg)
Context of Other Lectures at TUM
54I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 54: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/54.jpg)
Introduction to Deep Learning
Optimization
CNN
Introduction to NN
Machine Learning
basics
Back-propagation RNN
55I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 55: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/55.jpg)
Deep Learning at TUM
56
Intro to Deep
Learning(Niessner,
Leal-Taixe)DL for Physics(Thuerey)
ADL for Vision (Niessner,
Leal-Taixe)
DL for Medical Applicat.
(Menze)
DL in Robotics
(Bäuml)
Machine Learning(Günnemann)
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 56: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/56.jpg)
Why is I2DL so Important?• Many of the other lectures / practical require it!
– Often only limited spots are available (e.g., in the prestigious Advanced Deep Learning for Computer Vision Class)
• Always preparation for guided research / IDP– Most topics require it – For career in AI/DL these are the best ways to get into
57I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 57: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/57.jpg)
What’s New?
• Last semester: – Re-vamped content of slides / improved design– Re-organization of lecture structure
• This semester:– Re-vamping of the exercises (incl. sessions)– Earlier intro to pyTorch, more but smaller exercises
58I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 58: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/58.jpg)
Introduction to Deep Learning
59I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 59: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/59.jpg)
Moodle• Announcements via Moodle - IMPORTANT!
– Sign up in TUM online for access: https://www.moodle.tum.de/
– We will share common information (e.g., regarding exam)– Ask content questions online so others benefit– Don’t post solutions– TAs will monitor Moodle (expect answers within hours)
• We have a team of 11 TAs
60I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 60: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/60.jpg)
Email• Email list:
• Do NOT email us personally!– Cannot handle so many emails / hence will be ignored
• Email list for organizational questions only!– Content questions -> Moodle or Office Hours
61
I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 61: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/61.jpg)
Website• Links and slides will be shared on website
62
https://niessner.github.io/I2DL/I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 62: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/62.jpg)
About the Lecture• Theory lectures• Every Monday 14:15-15:45
– Lectures are virtual and will be uploaded to youtube:https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q
• Practical sessions– Every Thursday 8:30-10:00 – There will be practical exercises
• Guest lecture TBD
63I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 63: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/63.jpg)
Grading System• Exam: TBA
• Review: TUM Exam
• Important: no retake exam
• Practice: More information on Thursday!
• Bonus 0.3 + questions in the final exam
• Do not miss the first practical session !!!
64I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 64: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/64.jpg)
Preliminary Syllabus
Lecture 1: Introduction to the lecture, Deep Learning, Machine Learning.Lecture 2: Machine Learning Basics, Linear regression, Maximum LikelihoodLecture 3: Introduction to Neural Networks, Computational GraphsLecture 4: Optimization and BackpropagationLecture 5: Scaling Optimization to large Data, Stochastic Gradient DescentLecture 6: Training Neural Networks ILecture 7: Training Neural Networks IILecture 8: Training Neural Networks IIILecture 9: Introduction to CNNsLecture 10: CNNs architectures; CNNs for object detectionLecture 11: Recurrent Neural Networks (RNNs) Lecture 12: Autoencoders, VAEs and GANsLecture 13: Reinforcement LearningLecture 14: Guest Lecture (TBD)
66I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 65: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/65.jpg)
Other Administrative• External students welcome
– Will get Certificate / Schein at the end– Send email to list and we will add you to moodle
• Lectures will be recorded and put on youtube– https://www.youtube.com/channel/UCXN2nYjVT0cR9G
61RPEzK5Q
• Again: check announcements on moodle
67I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 66: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/66.jpg)
Office Hours
• We will have dedicated office hours regarding– Theoretical help (e.g., specific lecture questions)– Help on exercises
• Will be announced with first tutorial session and on Moodle
68I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 67: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/67.jpg)
Upcoming Lecture
• Next Lecture: Lecture 2: Machine Learning basics
• This Thursday: 1st Practical Session– if you want bonus, do not miss it!
69I2DL: Prof. Niessner, Prof. Leal-Taixé
![Page 68: Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é](https://reader030.vdocument.in/reader030/viewer/2022033120/5ec6507b12933a21da5edf5f/html5/thumbnails/68.jpg)
See you next week
70I2DL: Prof. Niessner, Prof. Leal-Taixé