stereoscopic style transfer - tng technology consulting · hardware hacking team tng technology...
TRANSCRIPT
Stereoscopic Style TransferStereoscopic Style TransferKunst ist das, was man siehtKunst ist das, was man sieht
Edgar DegasEdgar Degas
@Mar�nFoertsch @originalone1984
What is this all about?What is this all about?
@Mar�nFoertsch @originalone1984
Hardware Hacking TeamHardware Hacking TeamTNG Technology Consulting GmbHTNG Technology Consulting GmbH
Jonas Mayer
Thomas Endres
Mar�n Förtsch
Thomas Reifenberger
Florian Gather
Markus Spanier
Christoph Bergemann
Holger Frydrich
Mathias Arens
Andreas Würl
Mirco Michel
Leopold Kellers
AgendaAgendaIntroduc�onDeep Learning in a NutshellStyle TransferCode & Technology
RetrospectionRetrospectionAugmented Rift (2015)Augmented Rift (2015)
@Mar�nFoertsch @originalone1984
RetrospectionRetrospectionAugmented Rift (2015)Augmented Rift (2015)
@Mar�nFoertsch @originalone1984
IntroductionIntroductionStyle TransferStyle Transfer
@Mar�nFoertsch @originalone1984
1. Von Vincent van Gogh ‐ bgEuwDxel93‐Pg at Google Cultural Ins�tute, zoom level maximum, Gemeinfrei, h�ps://commons.wikimedia.org/w/index.php?curid=254982862. Photo by Mar�n Förtsch, "Window to the World", Mesa Arch, Canyonlands, 2015
IntroductionIntroductionStyle TransferStyle Transfer
@Mar�nFoertsch @originalone1984
IntroductionIntroductionStyle TransferStyle Transfer
@Mar�nFoertsch @originalone1984
1. Edvard Munch ‐ WebMuseum at ibiblioPage URL: h�p://www.ibiblio.org/wm/paint/auth/munch/munch.scream.jpg, Gemeinfrei, h�ps://commons.wikimedia.org/w/index.php?curid=37610298
IntroductionIntroductionStyle TransferStyle Transfer
@Mar�nFoertsch @originalone1984
1. Pablo Picasso, Frau mit grünem Hut, h�ps://www.alber�na.at/site/assets/files/1456/9_pablo_picasso_‐_frau_mit_gruenem_hut.720x0.jpg
IntroductionIntroductionDeepArt.ioDeepArt.io
@Mar�nFoertsch @originalone1984
IntroductionIntroductionHow Paintings workHow Paintings work
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
1. 2. By BruceBlaus ‐ Own work, CC BY 3.0, h�ps://commons.wikimedia.org/w/index.php?curid=28761830By Original by en:User:Chris 73, updated by en:User:Diberri, converted to SVG by �Zom ‐ Own work, CC BY‐SA 3.0
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
1. By JacoTen [CC BY‐SA 3.0 (h�ps://crea�vecommons.org/licenses/by‐sa/3.0)], from Wikimedia Commons
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
1. By JacoTen [CC BY‐SA 3.0 (h�ps://crea�vecommons.org/licenses/by‐sa/3.0)], from Wikimedia Commons
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
1. h�ps://medium.com/@arvind_70185/gradient‐descent‐for‐machine‐learning‐52ce08c96296
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
IntroductionIntroductionDeep Learning in a NutshellDeep Learning in a Nutshell
@Mar�nFoertsch @originalone1984
Style TransferStyle TransferOffline Role Play Game (Content Image)Offline Role Play Game (Content Image)
@Mar�nFoertsch @originalone1984
1. Von A.Savin (Wikimedia Commons ∙ WikiPhotoSpace) ‐ Eigenes Werk, CC BY‐SA 3.0, h�ps://commons.wikimedia.org/w/index.php?curid=21174383
Style TransferStyle TransferOffline Role Play Game (Style Image)Offline Role Play Game (Style Image)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical Implementation
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer WorkflowStyle Transfer Workflow
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Workflow by Gatys et al. (2015)Style Transfer Workflow by Gatys et al. (2015)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationFully Connected vs. Convolutional Neural Network LayerFully Connected vs. Convolutional Neural Network Layer
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationVGG19 WorkflowVGG19 Workflow
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationVGG19 - Live Demo!VGG19 - Live Demo!
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationCalculating Content Loss with VGG19Calculating Content Loss with VGG19
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationCalculating Content Loss with VGG19Calculating Content Loss with VGG19
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationCalculating Style Loss with VGG19Calculating Style Loss with VGG19
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Inside (VGG19 Layer 2_2)Style Transfer Inside (VGG19 Layer 2_2)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Inside (VGG19 Layer 2_2)Style Transfer Inside (VGG19 Layer 2_2)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Inside (VGG19 Layer 4_2)Style Transfer Inside (VGG19 Layer 4_2)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Inside (VGG19 Layer 4_2)Style Transfer Inside (VGG19 Layer 4_2)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationStyle Transfer Inside (VGG19 Layer 5_2)Style Transfer Inside (VGG19 Layer 5_2)
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationProblems are Time Consumption and NondeterminismProblems are Time Consumption and Nondeterminism
@Mar�nFoertsch @originalone1984
Technical ImplementationTechnical ImplementationFast Style Transfer Workflow by Ulyanov et al. (2016)Fast Style Transfer Workflow by Ulyanov et al. (2016)
@Mar�nFoertsch @originalone1984
Code & TechnologyCode & TechnologyArchitecture of the CNNArchitecture of the CNN
@Mar�nFoertsch @originalone1984
Training DatasetsTraining DatasetsMNISTMNIST
@Mar�nFoertsch @originalone1984
1. h�ps://www.researchgate.net/publica�on/322408789_Impact_of_Training_Set_Batch_Size_on_the_Performance_of_Convolu�onal_Neural_Networks_for_Diverse_Datasets?_sg=I0‐QwjF9AALHMI‐AiFT1rOM5b6lmp08QDQxIG
Training DatasetsTraining DatasetsCOCO Common Objects in ContextCOCO Common Objects in Context
@Mar�nFoertsch @originalone1984
Demonstration TimeDemonstration TimeRealtime Style TransferRealtime Style Transfer
@Mar�nFoertsch @originalone1984
1. h�ps://medium.com/data‐science‐group‐iitr/ar�s�c‐style‐transfer‐with‐convolu�onal‐neural‐network‐7ce2476039fd
Code & TechnologyCode & TechnologyDell VisorDell Visor
@Mar�nFoertsch @originalone1984
Code & TechnologyCode & TechnologyArchitecture OverviewArchitecture Overview
@Mar�nFoertsch @originalone1984
Code & TechnologyCode & TechnologyTrainingTraining
@Mar�nFoertsch @originalone1984
Code & TechnologyCode & TechnologyLimitationsLimitations
@Mar�nFoertsch @originalone1984
State of the ArtState of the ArtTemporal Style Transfer for VideosTemporal Style Transfer for Videos
@Mar�nFoertsch @originalone1984
1. h�ps://www.youtube.com/watch?v=pW6nZXeWlGM
State of the ArtState of the ArtRealtime Human Pose EstimationRealtime Human Pose Estimation
@Mar�nFoertsch @originalone1984
1. h�ps://www.youtube.com/watch?v=pW6nZXeWlGM
Experiment TimeExperiment TimeLet's do something crazy!Let's do something crazy!
@Mar�nFoertsch @originalone1984
Experiment TimeExperiment TimeLet's do something crazy!Let's do something crazy!
@Mar�nFoertsch @originalone1984
Experiment TimeExperiment TimeLet's do something crazy!Let's do something crazy!
@Mar�nFoertsch @originalone1984
Experiment TimeExperiment TimeLet's do something crazy!Let's do something crazy!
@Mar�nFoertsch @originalone1984
Experiment TimeExperiment TimeLet's do something crazy!Let's do something crazy!
@Mar�nFoertsch @originalone1984