Upload File

generative adversarial networks - sds...unpaired image-to-image translation using cycle-consistent...

  • Home
  • Documents
  • Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation
18
Generative Adversarial Networks: When fake never looked so real Evan Ntavelis 1,2 Dr. Iason Kastanis 1 Philipp Schmid 1 {ens, iks, psd}@csem.ch 1. Robotics & Machine Learning CSEM SA 2. Computer Vision Lab ETH Zürich

Upload: others

Post on 19-Jun-2020

4 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

Generative

Adversarial Networks:

When fake never

looked so real

Evan Ntavelis1,2

Dr. Iason Kastanis1

Philipp Schmid1

{ens, iks, psd}@csem.ch

1. Robotics & Machine Learning

CSEM SA

2. Computer Vision Lab

ETH Zürich

Page 2: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

2

CSEM at a glance – Close to industry

N A L

MZ

Zürich

Muttenz

Neuchâtel

Alpnach

Landquart

83.0Turnover

(mio CHF)

450Persons

175Industrial

clients

64European

projects

https://youexec.com/plus
https://youexec.com/plus
Page 3: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

3

Technologies in focus at CSEM

https://youexec.com/plus
https://youexec.com/plus
Page 4: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

4

Unpaired Image-to-Image Translation using

Cycle-Consistent Adversarial Networks

Zhu et al. 2017

https://youexec.com/plus
https://youexec.com/plus
https://junyanz.github.io/CycleGAN/
Page 5: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

5

AttnGAN: Fine-Grained Text to Image Generation

with Attentional Generative Adversarial Networks}

Xu et al 2018

https://youexec.com/plus
https://youexec.com/plus
Page 6: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

6

A Style-Based Generator Architecture

for Generative Adversarial Networks

Karras et al. 2018

https://youexec.com/plus
https://youexec.com/plus
Page 7: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

7

Semantic Image Synthesis with

Spatially-Adaptive Normalization

Park et al. 2019

https://youexec.com/plus
https://youexec.com/plus
Page 8: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

8

Source: datagrid.co.jp 2019

https://youexec.com/plus
https://youexec.com/plus
Page 9: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

9

Few-Shot Adversarial Learning of

Realistic Neural Talking Head Models

Zakharov et al. 2019

https://youexec.com/plus
https://youexec.com/plus
Page 10: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

10

Generative Adversarial Nets

• Introduced in 2014 by Ian

Goodfellow

• Rapidly Adopted

• Unprecedented Generational

Quality

https://youexec.com/plus
https://youexec.com/plus
Page 11: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

11

Generative Adversarial Nets

• An adversarial game between

two subnets:

• The Generator

• The Discriminator

https://youexec.com/plus
https://youexec.com/plus
Page 12: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

12

• In the era of Fake News do highly realistic images harbor dangers to

the society?

Deep Fakes

https://youexec.com/plus
https://youexec.com/plus
Page 13: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

13

Defense Mechanisms

https://youexec.com/plus
https://youexec.com/plus
Page 14: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

14

How can we use GANs in the industry?

The important question…

https://youexec.com/plus
https://youexec.com/plus
Page 15: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

15

• Gathering data is tedious and

costly

• Good quality labels require

even more effort

The Problem

https://youexec.com/plus
https://youexec.com/plus
Page 16: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

16

• Adversarial Domain Adaptation

• Train on a simulated data and

adapt for the use case

• Data Augmentation

• Learn how to generate new

samples to train with

• Generate images with desired

attributes

A Solution Using Adversarial Networks

Sources: CyCADA: Cycle-Consistent Adversarial Domain Adaptation

Hoffman et al. 2017,

GAN-based Synthetic Medical Image Augmentation

for increased CNN Performance

in Liver Lesion Classification

Frid-Adar et al, 2018

https://youexec.com/plus
https://youexec.com/plus
Page 17: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

17

• GANs are not a panacea

• Nascent technology

• Difficult to train

• Require abundance of data

• Clever schemes may reduce the

effort

• Yet, very promising results

• Worth the effort!

But…

https://youexec.com/plus
https://youexec.com/plus
Page 18: Generative Adversarial Networks - SDS...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Zhu et al. 2017. 5 AttnGAN: Fine-Grained Text to Image Generation

18

Are you interested in being part of a highly stimulating environment

working on the latest Deep Learning Technologies?

We are hiring!

That’s all folks!

https://youexec.com/plus
https://youexec.com/plus

COCO-FUNIT: Few-Shot Unsupervised Image Translation with a ... · Keywords: Image-to-image translation, Generative Adversarial Networks 1 Introduction Image-to-Image translation [1,2]

COMP9444 Neural Networks and Deep Learning 9b. Generative Adversarial … · 2020. 7. 25. · COMP9444 20T2 Generative Adversarial Networks 13 Generative Adversarial Networks GAN

Generative Adversarial Networks for Extreme Learned Image …openaccess.thecvf.com/content_ICCV_2019/papers/Agustsson... · 2019-10-23 · Generative Adversarial Networks for Extreme

A Survey of Image Synthesis and Editing with Generative ...kun/papers/gan_survey_final.pdf · 2 Generative Adversarial Networks Generative adversarial networks (GANs) were proposed

Generative Adversarial Networks - TJ Machine LearningGenerative Adversarial Networks Justin Zhang July 2017 1 Introduction Generative Adversarial Networks (GANs) are a seminal type

Boundless: Generative Adversarial Networks for Image ...openaccess.thecvf.com/.../Teterwak_Boundless...Boundless: Generative Adversarial Networks for Image Extension - Supplementary

BILINEAR REPRESENTATION FOR LANGUAGE-BASED IMAGE EDITING … · BILINEAR REPRESENTATION FOR LANGUAGE-BASED IMAGE EDITING USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS Xiaofeng

Text to Image Synthesis Using Generative Adversarial Networks

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Collaborative Sampling in Generative Adversarial … › pdf › 1902.00813.pdfGenerative Adversarial Networks Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) consist

Perceptual Adversarial Networks for Image-to-Image ... · PDF filetraining process. Through combining the perceptual adversarial loss and the generative adversarial loss, we presented

Mode Seeking Generative Adversarial Networks for Diverse ... · Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis Qi Mao∗1, Hsin-Ying Lee∗2, Hung-Yu Tseng∗2,

Optimizing Generative Adversarial Networks for Image Super ... · Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization ... reconstructions

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Mode Seeking Generative Adversarial Networks for …openaccess.thecvf.com/content_CVPR_2019/papers/Mao_Mode...Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Image Representation Learning with Generative Adversarial ... · Image Representation Learning with Generative Adversarial Networks Keywords machine learning, generative models, image

Hierarchical Generative Adversarial Networks for Single ......Hierarchical Generative Adversarial Networks for Single Image Super-Resolution Weimin Chen1∗, Yuqing Ma2∗, Xianglong

Discriminative Region Proposal Adversarial Networks for ......Our image-to-image translation model, called Discriminative Region Proposal Adversarial Networks (DRPAN), is composed

Generative Adversarial Networks (GANs) - Emanuele Sansone · List of recent papers on Generative Adversarial Networks: PAPERs about GANs 20142016 Generative Adversarial Networks ICLR

Generative Adversarial Networksbhiksha/courses/deeplearning/Spring.2018/... · 2018. 3. 29. · Generative Adversarial Networks Generative adversarial networks (GANs) are relatively

Distilling portable Generative Adversarial Networks for ... AAAI GAN Distillation.pdf · Distilling portable Generative Adversarial Networks for Image Translation Hanting Chen1, Yunhe

Generative Adversarial Networks (GANs)

Generative Adversarial Networks (part 2)deeplearning.cs.cmu.edu/document/slides/bstriner_gans_part_2.pdf · Unrolled Generative Adversarial Networks Unrolled Generative Adversarial

Image-to-Image Translation with Conditional Adversarial Networks€¦ · Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola, Jun-Yan Zhu, Tinghui Zhou

Unsupervised Image Reconstruction using Deep Generative ...web.stanford.edu/~ekcole/sedona.pdfUnsupervised Image Reconstruction using Deep Generative Adversarial Networks Elizabeth

Image Difficulty Curriculum for Generative Adversarial Networks … · 2020. 2. 25. · Image Difficulty Curriculum for Generative Adversarial Networks (CuGAN) Petru Soviany1, Claudiu

Generative Adversarial Networks for Hyperspectral Image ...static.tongtianta.site/paper_pdf/488448f6-1a35-11e9-89a1-00163e08bb86.pdfAbstract—A generative adversarial network (GAN)

Image Colorization using Generative Adversarial Networks

ST-GAN: Spatial Transformer Generative Adversarial Networks ......ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing Chen-Hsuan Lin1* Ersin Yumer2,3*

Location Conditional Image Generation using …...Location Conditional Image Generation using Generative Adversarial Networks Mayur Saxena Department of Computer Science University

Generative Adversarial Networks, and Applications Adversarial Networks, and Applications Outline: •Generative Models vs Discriminative Models (Background) •Generative Adversarial

CPGAN: Content-Parsing Generative Adversarial Networks for ... · CPGAN: Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis Jiadong Liang1 ;y, Wenjie Pei2,

Generative Adversarial Networks - sahin olut...Image to text (Image captioning - Generative Adversarial Text to Image Synthesis) There are many applications which are not covered above

StarGAN: Unified Generative Adversarial Networks for Multi ...€¦ · StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation Yunjey Choi1,2

Unpaired Image-To-Image Translation Using Cycle …openaccess.thecvf.com/content_ICCV_2017/papers/Zhu...Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks