the curse of dimensionality and image recognition€¦ · image source: ‘explaining and...
TRANSCRIPT
![Page 1: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/1.jpg)
THECURSEOFDIMENSIONALITYANDIMAGERECOGNITION
BRANDONEDWARDS
![Page 2: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/2.jpg)
OUTLINE
• Imageclassification• Worst-casetestimages(adversarialexamples)• Defenseagainstadversarialattacks
• ’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz,
Maithra Rughu,MartinWattenberg,IanGoodfellow(2018)ICLRPaper
• Relevancetoadversarialexamplesinimageclassification
![Page 3: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/3.jpg)
IMAGECLASSIFICATION
![Page 4: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/4.jpg)
IMAGECLASSIFICATION
ImageSource:“ImageNetClassificationwithDeepConvolutionalNeuralNetworks”,AlexKrizhevsky,IlyaSutskever,GeoffreyE.Hinton,2012
• ImageNet(ILSVRC):1000classes;training-1.2million,validation-50k,test-150k• ~83%successforgroundtruthinthetop5classes• Currenttop5performance>95%
![Page 5: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/5.jpg)
MODELFUNCTION
ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=#classes
![Page 6: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/6.jpg)
ADVERSARIALEXAMPLES‘CLOSE’IMAGESTHATCLASSIFY‘INCORRECTLY’
![Page 7: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/7.jpg)
DIGITALATTACK
ImageSource:‘ExplainingandHarnessingAdversarialExamples’,ICLR2015,Goodfellow,Shlens,Szegedy
• AttackaboveisonGoogLeNet (ImageNet)(>94%top5accuracy).• Theperturbationisclearlysmallbyhumanstandards.
• Digitaladversarialattackscenario:Phishingdetection
![Page 8: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/8.jpg)
PHYSICALATTACKImageSource:“AccessorizetoaCrime:RealandStealthyAttacksonState-of-the-ArtFaceRecognition”,MahmoodSharif,Sruti Bhagavatula,Lujo Bauer,MichaelK.Reiter;CCS2016
• Attackagainstpre-trainedfacialrecognitionmodel• 88%ofimageswithglassesclassifiedasMilla Jovovich• Meanconfidencewas78%.
• Thisperturbationislarger,butwoulditberaisesuspicion?
![Page 9: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/9.jpg)
DEFENSEAPPROACHESLearnthe(distributional)differencebetweenadversarialexamplesand‘natural’data.
• Preprocessing(removingperturbation)– JPEG,neuralnetworkde-noiser
• Detectionofadversarialexamples
Imposeconstraintsonmodelfunctiontolimitlocalchanges
• RegularizationorLipschitzconstraints
Consideradversarialexamplesduringtraining
• Adversarialtraining
Improvemodelinotherways
• Capsulenetworks?
![Page 10: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/10.jpg)
ADVERSARIALSPHERESPAPER
![Page 11: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/11.jpg)
ADVERSARIALSPHERESPAPER
’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz, MaithraRughu,MartinWattenberg,IanGoodfellow (2018)ICLR
• Simpleclassificationtask
• Experimentalmodelresults
• Theoreticalresultsrelatingmodelaccuracyandproximityofadversarialexamples.
![Page 12: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/12.jpg)
CLASSIFICATIONTASKANDEXPERIMENTALRESULTS
• Twospherescenteredattheorigininℝ)(R=1andR=1.3).• Trainanartificialneuralnetwork.• ModelInput:Apointinℝ)• ModelOutput:”probability”ofbeingclosertotheinnersphere
• ExperimentalFocusond=500• Trainonpointsuniformlysampledfrombothspheres• Testonpointsuniformlysampledfromtheinnersphere• Hightestaccuracy,butcloseadversarialexamplesremain
![Page 13: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/13.jpg)
THEORETICALRESULT
• Non-zeroerrorimpliesarbitrarilycloseadversarialexamplesforlargeenoughdimensiond.
• Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
![Page 14: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/14.jpg)
Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
SketchofProof
LetEbethesetofmisclassifiedpoints,so𝜇 𝐸 = 𝑞 > 0 [𝜇 𝑆8 = 1].
Let𝑑 𝐸 = 𝔼;~=>𝑑(𝑥, 𝐸) (averagedistancetoE).
Maximum𝑑 𝐸 occursfora”cap”(intersectionof𝑆8 withahalf-space)[Figiel et.al.1977].
![Page 15: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/15.jpg)
Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
SketchofProof(Continued)
(larged)anycoordinateon𝑆8 hasdistribution:𝑁(0,1/𝑑).– [Poincaré\Lévy]
Asmallbandaroundan“equator”containsthemajorityofspherevolume.Thecapboundaryisthusclosetotheequator– wherethemajorityofpointslie.
![Page 16: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/16.jpg)
ADDITIONALTHOUGHTS
Adversarialproblemisworse
Theasymptoticresultsignoretheadversarialexamplesthatmaybefoundofftheinnersphere.
ModelErrorasafunctionofNumberofTrainingPoints
Learnmodelsthatusestheradiusofpoints- perfectmodelsforALLdimensions.I.E.Domainspecificfeaturesmayprovidethelowerrordesired.
![Page 17: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/17.jpg)
LESSONSFORIMAGECLASSIFICATION?
• Insightintocurrentdefenseideas• Caution:Adversarialexamplescouldlieonthedatadistribution.
• Confirmation:Lipschitzconstraintsandadversarialtrainingwouldhelphere
• Imagesmaybedifferent:shapeofindividualclassdistributions
• Couldbebetterinsomeways,worseinothers
• Ex:𝐵C×𝐼)FC forsmallk.Betterforin-distributionexamples,butmoresurfaceareacouldallowmoreoff-surface?
![Page 18: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/18.jpg)
SUMMARY
• Thespherestoyproblemprovidesinsightrelatedtocurrentadversarialimagedefensetechniques.
• LargedimensionsCANleadtoverystricterrorrequirementsinordertoavoidclose‘adversarialexamples’.
• Domainspecificlowdimensionalfeaturecreationorotherconstraintscouldprovidethelowerrorneededto‘pushoff’adversarialexamplesfortheaveragetestpoint.
![Page 19: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/19.jpg)
THANKYOU
![Page 20: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/20.jpg)
NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS
Proof:
• LetEbethesetofmiss-classifiedpointsontheinnersphere,witherrorrate𝜇(𝐸) =q.ForcalculatinganupperboundwereplaceEwitha“cap”𝐸′ with𝜇 𝐸′ = 𝜇(𝐸).Withoutlossofgenerality,weassume:
E′ = {𝑥 ∈ 𝑆8: 𝑥K > 𝛽/ 𝑑� } forsome𝛽 > 0.
• Thenq = 𝜇 𝐸′ ≅ ℙ 𝑁 0, K)> Q
)�= ℙ 𝑁 0,1 > 𝛽 = 1 − Φ(𝛽) ,where
Φ isthestandardnormalcdf.
![Page 21: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/21.jpg)
NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS
Proof(Continued):
• Thus𝛽 = ΦFK(1 − 𝑞).
• Notethat𝑑 𝐸′ : = 𝔼;~=>𝑑 𝑥, 𝐸′ ≤ 𝔼 max 2� Q)�− 𝑁 0, K
), 0
= 𝑂(Z[\(KF])
)�).
• Finally,forfixedqwehave𝑑 𝐸 ≤ 𝑑 𝐸^ = 𝑂( K)�).
![Page 22: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/22.jpg)
TWOEXPERIMENTALMODELSPiecewiseLinearModel
• TwolinearlayernetworkwithReLU activations.Mini-batchstochasticgradientdescentwasusedwithbatchsize50.Batchnormalizationwasperformedatthetwohiddenlayers.
QuadraticModel(Ellipsoidal)
• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobe
expressedas∑ 𝛼c)cdK 𝑥c`.
• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.
![Page 23: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/23.jpg)
STRUCTUREDFEATUREDETECTION
• INPUT:Rawpixelvalues(2-Darray)structure.
• INTERMEDIATEFEATUREVARIABLES:
• LocalFeatures:Edges,Textures,…,Ears,Eyes,...
• GlobalFeatures:Face,Body,…
• Keyoperations:Convolutions, Down-Sampling,Up-Sampling,…
• Built-ininvariance:Shift,Scale,…
• FINALLAYERS:Usefeaturevaluestocomputeclassconfidencevalues.
ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=# classes
![Page 24: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/24.jpg)
TWOTRAININGMODES
• [Online](makesurematchestheirstatements)Uniformlysamplefromtheinnerandoutsphereforeachnewtrainingpoint.
• [Batch]UniformlysamplefromtheinnerandoutsphereforNpointseach.Iterateoverthese2Npointsrepeatedlyduringtraining.
![Page 25: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/25.jpg)
FORLARGEDIMENSIONS,CLOSEADVERSARIALEXAMPLESAREFOUND.
• Piecewiselinearmodel(Onlinetraining– 25millionpointspersphere)withd=500.Noerrorwasobservedin10milliontestpoints.
• Note:Volumeofthismisclassifiedspaceontheinnersphereissmall!!!
• Note:d=60wastheobservedpointwheretheexperimentabovestartedtohaveadversarialexamples.
![Page 26: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/26.jpg)
ALINEARINCREASEINADVERSARIALDISTANCEREQUIRESANEXPONENTIALDECREASEINERRORRATE
Experimentalmodelerror
rateestimates,𝑞 v.s.𝑑(𝐸).
Theupperbound𝑑(𝐸^) is
thesolidblackplot.
ImageSource
`AdversarialSpheres’ICLR2018
![Page 27: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/27.jpg)
MOREINFOONQUADRATICNETWORK
• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.– Thedecisionboundarywillbeanellipsoid.
• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)
cdK 𝑥c`.
• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.
![Page 28: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/28.jpg)
QUADRATICMODELOBSERVATIONS
• A rotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)
cdK 𝑥c`.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
Online:50milliontrainingpoints(samesetupfromReLU experiment)
All𝛼c wereinrangeforaperfectclassifier– ie perfectclassifier
![Page 29: THE CURSE OF DIMENSIONALITY AND IMAGE RECOGNITION€¦ · Image Source: ‘Explaining and Harnessing Adversarial Examples’, ICLR 2015, Goodfellow, Shlens, Szegedy • Attack above](https://reader033.vdocument.in/reader033/viewer/2022050217/5f6382cc732115248b533a19/html5/thumbnails/29.jpg)
QUADRATICMODELOBSERVATIONSBatch:batchsize1million
Noerrorsin20milliontestpoints
Adversarialexamplesarefound
394/500𝛼c areoutofrange.
Withhighprobability,the
effectsofthebad𝛼c cancel
eachotherout.
ImageSource:`AdversarialSpheres’ICLR2018