neural style transfer · 3/14/2019 · neural style transfer author: microsoft office user created...
TRANSCRIPT
![Page 1: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/1.jpg)
![Page 2: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/2.jpg)
![Page 3: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/3.jpg)
![Page 4: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/4.jpg)
Loss
Function
![Page 5: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/5.jpg)
![Page 6: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/6.jpg)
𝐿𝑡𝑜𝑡𝑎𝑙 𝑐, 𝑠, 𝑥 = 𝛼𝐿𝑐𝑜𝑛𝑡𝑒𝑛𝑡( 𝑐, 𝑥) + 𝛽𝐿𝑠𝑡𝑦𝑙𝑒( 𝑠, 𝑥)
![Page 7: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/7.jpg)
![Page 8: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/8.jpg)
Target content, output
from a VGG mid-layerWhite noise
𝐿𝑐𝑜𝑛𝑡𝑒𝑛𝑡 𝑐, 𝑥, 𝑙 =1
2
𝑖,𝑗,𝑘
(𝐹𝑖,𝑗,𝑘𝑙 − 𝐶𝑖,𝑗,𝑘
𝑙 )2
![Page 9: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/9.jpg)
𝑛𝐶 = 5
𝑛𝐻 =4
𝑛𝑊 = 4
𝐶𝑜𝑛𝑡𝑒𝑛𝑡 𝐼𝑚𝑎𝑔𝑒
𝑛𝐶 = 5
𝑛𝐻 =4
𝑛𝑊 = 4
𝑊ℎ𝑖𝑡𝑒 𝑁𝑜𝑖𝑠𝑒
![Page 10: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/10.jpg)
Target
content
White
noise
Minimize
![Page 11: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/11.jpg)
![Page 12: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/12.jpg)
𝐿𝑠𝑡𝑦𝑙𝑒 𝑠, 𝑥, 𝑙 =1
(2𝑛𝑊𝑛𝐻𝑛𝑐)2
𝑘,𝑘′
(𝐺𝑘,𝑘′𝑙 , −𝑆𝑘,𝑘′
𝑙 )2
White noise
Target style,
output from a
VGG CNN layer
𝐺𝑘,𝑘′𝑙 =
𝑖,𝑗
𝑎𝑖,𝑗,𝑘𝑙 𝑎𝑖,𝑗,𝑘′
𝑙
![Page 13: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/13.jpg)
𝑛𝐶 = 5
𝑛𝐻 =4
𝑛𝑊 = 4
𝑆𝑡𝑦𝑙𝑒 𝐼𝑚𝑎𝑔𝑒
Correlation
of 0.89
![Page 14: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/14.jpg)
𝑛𝐶
𝑛𝐶 = 5
0.89
𝐺𝑘,𝑘′𝑙 =
𝑖,𝑗
𝑎𝑖,𝑗,𝑘𝑙 𝑎𝑖,𝑗,𝑘′
𝑙
![Page 15: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/15.jpg)
𝑛𝐶 = 5
𝑛𝐻 =4
𝑛𝑊 = 4
𝑆𝑡𝑦𝑙𝑒 𝐼𝑚𝑎𝑔𝑒
Correlation
of 0.2
![Page 16: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/16.jpg)
𝑛𝐶
0.89 0.2
𝐺𝑘,𝑘′𝑙 =
𝑖,𝑗
𝑎𝑖,𝑗,𝑘𝑙 𝑎𝑖,𝑗,𝑘′
𝑙
𝑛𝐶
![Page 17: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/17.jpg)
𝑛𝐶 = 5
𝑛𝐻 =4
𝑛𝑊 = 4
𝑆𝑡𝑦𝑙𝑒 𝐼𝑚𝑎𝑔𝑒
Correlation
of 0.7
![Page 18: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/18.jpg)
𝑛𝐶
0.89 0.2 0.7
𝐺𝑘,𝑘′𝑙 =
𝑖,𝑗
𝑎𝑖,𝑗,𝑘𝑙 𝑎𝑖,𝑗,𝑘′
𝑙
𝑛𝐶
![Page 19: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/19.jpg)
𝐿𝑠𝑡𝑦𝑙𝑒 𝑠, 𝑥, 𝑙 =1
(2𝑛𝑊𝑛𝐻𝑛𝑐)2
𝑘,𝑘′
(𝐺𝑘,𝑘′𝑙 − 𝑆𝑘,𝑘′
𝑙 )2
White Noise
Gram Matrix
Target Style
Gram Matrix
, 𝑙
![Page 20: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/20.jpg)
Target
content
White
noise
Minimize
Min Min Min Min Min
G G G G G G G G G G
![Page 21: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/21.jpg)
Deeper
layers for
style
Increasing 𝛼 𝛽 ratio
𝐿𝑡𝑜𝑡𝑎𝑙 = 𝛼𝐿𝑐𝑜𝑛𝑡𝑒𝑛𝑡 + 𝛽𝐿𝑠𝑡𝑦𝑙𝑒
𝟏𝟎−𝟓 𝟏𝟎−𝟐
![Page 22: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/22.jpg)
![Page 23: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/23.jpg)
Epoch 100 Epoch 300 Epoch 500
![Page 24: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/24.jpg)
![Page 25: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/25.jpg)
![Page 26: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/26.jpg)
![Page 27: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/27.jpg)
![Page 28: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/28.jpg)
![Page 29: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/29.jpg)
![Page 30: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/30.jpg)
![Page 31: Neural Style Transfer · 3/14/2019 · Neural Style Transfer Author: Microsoft Office User Created Date: 3/13/2019 5:38:10 PM](https://reader033.vdocument.in/reader033/viewer/2022060520/604e9d46a75c7f4efa6c62e7/html5/thumbnails/31.jpg)
𝑛𝐶
0.89 0.2 0.7
• Diagonal: which
channels are most
active
• Off-diagonal: which
channel pairs co-occur
𝑛𝐶