Download - NeuralRestoration
! Goal:Restoreunseendamagedimages/photographswithacontent-awareneuralnet
CanArtificialIntelligencerivalhumansinSemanticImageReconstruction?
OriginalImage DamagedImage RestoredImage
ArtRestorationwithDeepLearning
! Howwouldyoufillinthemissinginformation?
! Canonetrainacomputertohavethenecessarycognitiontofillinthemissingstructural,textualandcontentinformation?
WeighingVariousArchitectures
Non-ParametricImageMatching
VariationalSampling
ConvolutionalNeuralNet
AdversarialNetworks
ContextualAwareness ✔ ✔ ✔ ✔PerceptualAwareness ✗ ✔ ✔ ✔HighResolution ✔ ✗ ✗ ✔LargeMissingPatches ✗ ✗ ✔ ✔EasytoTrain ✔ ✔ ✗ ✗
DataMiningandImageAugmentation
Scrapeandaugment200kartsy.netimages
1 Retrieveand
pre-processimages
2
Trainautoencoderasimagegenerator&
extractlatentfeatures
3 Simultaneouslytrainan“adversarial”neuralnettodiscriminate
realvs.“fake”images
4 ImageRestoration
Application
5
RetrievalandPre-ProcessingPipeline
! Imagesareresizedandnormalizedforzeromean/unitvariance
! 1,000imagesaresetasideeachforvalidationandtestset
TargetImage TrainingImage Arbitrarilysizedmaskisplacedrandomlyto“corrupt”thetrainingimage
ImageGeneratorArchitecture
4 hidden convolution layers
4 hidden deconvolution layers
1024 8 x 10 feature maps
MSE only
Replace final dense layers. Connect output units to input units as autoencoder
Not good!
DeepConvolutionalGenerativeAdversarialNetworks(DCGANs)
G(MSE) + D(Fake_BCE)
Real Image
Fake Image
D(Real_BCE)
D(Fake_BCE)
Generator, G() Discriminator, D()
No heuristic cost functions
needed!!!
KeyModelInsights
! GenerativeAdversarialNetworks,sinceitsintroductionbyIanGoodfellowin2014,isknowntobeverytrickytotrain
! Anumberofstabilizingmeasureswereintroduced,basedonthegroundbreakingpaper“UnsupervisedRepresentationLearningwithDeepConvolutionalGenerativeAdversarialNetworks”byAlecRadford,LukeMetzandSoumithChintala
! Themainbenefitforusingadversarialnetworksisthemodel’sabilitytolearnitsowncostfunctioninanunsupervisedfashion
ArtWorld–ImageRestorationApp
Step 1 Step 2 Step 3
Drag or Upload flawed image
to website
Place user defined mask on damaged area
Hit repair button. Receive repaired image instantly
OtherCommercialApplicationsoftheDCGANsArchitecture
! Learnhigh-orderfunctionslikereasoning,planningandprediction
! Dimensionreduction/LatentFeatureExtraction(morepowerfulthanPCA)
! Generatesuperresolutionorup-sampledimages
! Forwardvideoprediction
ContactInformation
EdChin
! Email:[email protected]
! Linkedin:! https://www.linkedin.com/in/edwin-chin-62392b1
! Repo:github.com/echin6/my_recent_projects