practice: vae and gan - deep generative models
TRANSCRIPT
![Page 1: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/1.jpg)
Practice: VAE and GANHao Dong
Peking University
1
![Page 2: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/2.jpg)
2
Practice: VAE + GAN
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 3: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/3.jpg)
3
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 4: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/4.jpg)
4
Hello World: MNIST Classification
28x28x1 = 784 binary values/image
28
28
MNIST dataset
ā¢ Image X is a list of row vectors:>>> X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 784))>>> print(X_train.shape)ā¦ (50000, 784)
ā¢ Image X is a list of images:>>> X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 28, 28, 1))>>> print(X_train.shape)ā¦ (50000, 28, 28, 1)
If RGB image, we will have 3 channels
(height, width, channels)
![Page 5: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/5.jpg)
5
Hello World: MNIST Classification
ā¢ Simple Iteration
![Page 6: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/6.jpg)
6
Hello World: MNIST Classification
ā¢ Dataset API
![Page 7: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/7.jpg)
7
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 8: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/8.jpg)
8
Introduction of VAE
!šæ!šX
L2
KLD
ā¢ Two network architectures
ā¢ Two loss functions
ā¢ Reparameterization trick
![Page 9: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/9.jpg)
9
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 10: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/10.jpg)
10
VAE Architecture
!šæ!šX
L2
KLD
28
28
28x28x1 or 784
š„~šššš”š
![Page 11: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/11.jpg)
11
VAE Architecture
!šæ!šX
L2
KLD
ā¢ Architecture of Encoder
![Page 12: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/12.jpg)
12
VAE Architecture
!šæ!šX
L2
KLD
ā¢ Architecture of Decoder/Generator
![Page 13: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/13.jpg)
13
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 14: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/14.jpg)
14
VAE Training
!šæ!šX
L2
KLD
Reconstruction Loss
![Page 15: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/15.jpg)
15
VAE Training
!šæ!šX
L2
KLD
KL-Divergence loss
![Page 16: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/16.jpg)
16
VAE Training
!šæ!šX
L2
KLD
Training Pipeline
![Page 17: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/17.jpg)
17
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 18: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/18.jpg)
18
VAE Interpolation
![Page 19: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/19.jpg)
19
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 20: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/20.jpg)
20
Sampling
!šæZ
![Page 21: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/21.jpg)
21
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 22: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/22.jpg)
22
Introduction of DCGAN
ā¢ Two network architectures
ā¢ Two loss functions!šæ
X
real
fakeZ
![Page 23: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/23.jpg)
23
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 24: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/24.jpg)
24
DCGAN Architecture
!šæ
X
real
fakeZ
28
28
28x28x1 or 784
š§~š(0,1)
š„~šššš”š
![Page 25: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/25.jpg)
25
DCGAN Architecture
!šæ
X
real
fakeZ
Architecture of generator
![Page 26: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/26.jpg)
26
DCGAN Architecture
!šæ
X
real
fakeZ
Architecture of discriminator
![Page 27: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/27.jpg)
27
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 28: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/28.jpg)
28
DCGAN Training
!šæ
X
fake
realZ
Loss of discriminator
![Page 29: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/29.jpg)
29
DCGAN Training
!šæ realZ
Loss of generator
![Page 30: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/30.jpg)
30
DCGAN Training
Training pipeline
!šæ
X
real
fakeZ
![Page 31: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/31.jpg)
31
ā¢ Hello World: MNIST Classification
ā¢ Introduction of VAEā¢ VAE Architectureā¢ VAE Trainingā¢ VAE Interpolationā¢ Sampling
ā¢ Introduction of DCGANā¢ DCGAN Architectureā¢ DCGAN Trainingā¢ DCGAN Interpolation
![Page 32: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/32.jpg)
32
DCGAN Interpolation
!šæ
Z1
Z2
(1-a) Z1+a Z2
![Page 33: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/33.jpg)
33
More
![Page 34: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/34.jpg)
34
Improved GANLSGANWGAN
WGAN-GPBiGAN
VAE-GANā¦
with MNIST
![Page 35: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/35.jpg)
35
Pix2PixCycleGAN
SRGANā¦
with other datasets
![Page 36: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/36.jpg)
36
Proposal Your Projects
![Page 37: Practice: VAE and GAN - Deep Generative Models](https://reader034.vdocument.in/reader034/viewer/2022051600/627f944bc00c4803562b5b00/html5/thumbnails/37.jpg)
Thanks
37