![Page 1: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/1.jpg)
Getting Started with DeepLearning using Scala
@OE_uia
![Page 2: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/2.jpg)
Who I am:
Taisuke Oe (TW: @OE_uia, GITHUB: taisukeoe)
● Android App Developer in Scala● ND4S creator● ND4J contributor
![Page 3: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/3.jpg)
Agenda / Disclaimer
What I’m going to talk:
- What is Deep Learning?- What is Neural Network?- How Neural Netowork works over training.- DeepLearning4j architecture
What I’m NOT going to talk (in detail):
- Type of Neural Network- Hyper parameter and its tuning
今日は深層学習とニューラルネットワークの基礎、 DeepLearning4jのアーキテクチャについて触れます
![Page 4: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/4.jpg)
What is Machine Learning?
"Field of study that gives computers the ability to learn without being explicitly programmed."
Arthur Samuel (1959)
機械学習とは、明示的にプログラムされなくてもコンピューターに学ぶ能力を付与する研究分野のこと。
![Page 5: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/5.jpg)
What is Deep Learning?
- A category of Machine Learning,
especially for images, text, audio, etc.
- Accurate and reasonably fast.
- Train “Neural Networks” with updating its parameters
over iterations.- Neural Network has a layered structure, and learns
features in each layer
深層学習は、ニューラルネットワークを使う機械学習の一分野で、画像、文章、音などを対象とする。
![Page 6: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/6.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4j
ND4J
DeepLearning4jの主な3つのモジュール。 DeepLearning4j, ND4J, Canova。
![Page 7: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/7.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4j
ND4J
General Vectorization Library.
Vectorize raw data to INDArray.
DeepLearning Framework.
Construct Neural Network based on configuration.
N-dimensional Arraycalculation Library.
DeepLearning4jが設定からニューラルネットを構築。 Canovaは生データをベクター化。 ND4Jはそれらの計算基盤。
![Page 8: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/8.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4jImages
Text
Audio
ND4J
Canovaは、画像やテキスト、オーディオなどの生データを解釈する。
![Page 9: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/9.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4jImages
Text
Audio
INDArray
ND4J
disclaimer: it’s a simplified example very much.
例えば、Canovaが生の画像を計算しやすいように N次元行列(この例では2次元)に変換する。
![Page 10: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/10.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4jImages
Text
Audio
INDArray
ND4J
INDArray
ND4S
disclaimer: it’s a simplified example very much.
場合によっては、ND4SでN次元行列を計算ないしは学習しやすいように変換する。
![Page 11: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/11.jpg)
DEMOhttps://github.com/taisukeoe/matsuri-mnist-example
DEMO in REPL of:
https://github.com/deeplearning4j/nd4s
![Page 12: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/12.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4j
ND4J
N dimensional Sample Data
disclaimer: it’s a simplified example very much.
DeepLearning4jによって構築されたニューラルネットが N次元行列の入力を受け取り、結果を表す N次元行列を返す
![Page 13: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/13.jpg)
DeepLearning4j Archtecture
Canova DeepLearning4j
ND4J
N dimensional Sample Data
disclaimer: it’s a simplified example very much.
この例では出力された行列の各列が各クラスに分類される確率を表す。 2が90%、3が10%という具合。
![Page 14: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/14.jpg)
How DeepLearning works
N dimensional Sample Data
Parameters in Neural Network (in this picture, Multi Layer Perceptron) got updated over iterations
# of samples
トレーニングの際。サンプルをミニバッチごとに分けて入力させる。ニューラルネット内の状態が更新され続ける。
![Page 15: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/15.jpg)
How DeepLearning works
N dimensional Sample Data
# of samples
Neural Network hasstates which are updated over iteration.
DataSet has states like a iterator.Spit out samples in each mini-batch.
トレーニングの際。サンプルをミニバッチごとに分けて入力させる。ニューラルネット内の状態が更新され続ける。
![Page 16: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/16.jpg)
How DeepLearning works
N dimensional Sample Data
# of samples
Neural Network hasstates which are updated over iteration.
Go to the detail:How neuron unit behaves:
トレーニングを通じて更新される、ニューラルネット内の「状態」の中身を見ていきましょう。
![Page 17: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/17.jpg)
How Neuron works
構成単位ニューロン。各 (x:入力)に(w:重み)を乗じて(b:バイアス)を足したものを、活性化関数 fに渡して出力計算。
![Page 18: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/18.jpg)
How Neuron works
mutable!
mutable!
immutable!
(w 重み)と(b バイアス)は変数で、トレーニングごとに更新される。 活性化関数 fは不変。
![Page 19: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/19.jpg)
How Neural Net work in each layer
(全結合な)ニューラルネットの各層は、重み行列、バイアス行列、活性化関数からなる関数で表現される
![Page 20: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/20.jpg)
Further information:
- Type of Neural Network - Convolutional Neural Network
- Recurrent Neural Network
- etc
- Backpropagation- Loss Function- Learning rate - Other hyper parameters
深層学習 http://www.amazon.co.jp/dp/B018K6C99A/
Coursera https://www.coursera.org/course/neuralnets
(w:重み)と(b:バイアス)を更新する方法は、誤差逆伝搬法と損失関数、学習係数などで決定される。詳しくは書籍などで。
![Page 21: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/21.jpg)
How to improve results?
- Choose appropriate Neural Network(s)- Tune hyper parameters of Neural Network(s).- Pre-process input data to be efficiently trained.
学習結果を改善させるには?ニューラルネットワークのタイプを選ぶ、超パラメータを調整、入力データのプリプロセシング。
![Page 22: Getting Started with Deep Learning using Scala](https://reader034.vdocument.in/reader034/viewer/2022042722/5889fbb81a28ab0f388b5891/html5/thumbnails/22.jpg)
Summary
- Deep Learning is a category of Machine Learning, which is trained by Neural Networks with three or more layers.
- Neural Network maps input feature matrix to output matrix which stands for possibilities of each class in classification purpose.
- DeepLearning4j: Deep Learning framework in JVM- Nd4j: N-dimensional array calculation library- Canova: General vectorization library- Nd4s: NumPy-like + Scala-like API for ND4J
深層学習は、3層以上からなるニューラルネットワークで行う機械学習のこと。N次元行列の生データを加工するなどして、パフォーマンスを上げる。