cs4501: introduction to computer vision filtering
Post on 16-Oct-2021
3 Views
Preview:
TRANSCRIPT
CS4501:IntroductiontoComputerVision
Filtering,Frequency,andEdges
Various slides from previous courses by: D.A.Forsyth(Berkeley/UIUC),I.Kokkinos(Ecole Centrale/UCL).S.Lazebnik (UNC/UIUC),S.Seitz(MSR/Facebook),J.Hays(Brown/GeorgiaTech),A.Berg(StonyBrook/UNC),D.Samaras(StonyBrook).J.M.Frahm (UNC),V.Ordonez(UVA).
• ImageProcessing:Brightness• ImageFiltering:MeanFilter• ImageBlurring• ImageGradients:TheSobelOperator
LastClass
• RecaponConvolutionalOperations• ImageGradients:TheSobelOperator• FrequencyDomain• FilteringinFrequency• GoogleColaboratory
Today’sClass
What we see What a computer seesSource: S. Narasimhan
ImagesasMatrices
ImagesasFunctionsz = 𝑓(𝑥, 𝑦)
• Thedomainofxandyis[0,img-width)and[0andimg-height)• x,andyarediscretizedintointegervalues.
BasicImageProcessing
𝐼 𝛼𝐼
𝛼 > 1
BasicImageProcessing
𝐼 𝛼𝐼
0 < 𝛼 < 1
Imagefiltering:Convolutionoperator
ImageCredit:http://what-when-how.com/introduction-to-video-and-image-processing/neighborhood-processing-introduction-to-video-and-image-processing-part-1/
𝑘(𝑥, 𝑦)
𝑔 𝑥, 𝑦 =11𝑘 𝑢, 𝑣 𝑓(𝑥 − 𝑢, 𝑦 − 𝑣)�
7
�
8
Image filtering: Convolution operatore.g.meanfilter
ImageCredit:http://what-when-how.com/introduction-to-video-and-image-processing/neighborhood-processing-introduction-to-video-and-image-processing-part-1/
𝑘(𝑥, 𝑦)
𝑘(𝑥, 𝑦) =
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
[.,.]h[.,.]f
Imagefiltering111111111],[g ××
Credit: S. Seitz
],[],[],[,
lnkmflkgnmhlk
++=å
Image filtering: Convolution operatorImportantfilter:gaussian filter(gaussian blur)
ImageCredit:http://what-when-how.com/introduction-to-video-and-image-processing/neighborhood-processing-introduction-to-video-and-image-processing-part-1/
𝑘(𝑥, 𝑦)
𝑘(𝑥, 𝑦) =
1/16 1/8 1/16
1/8 1/4 1/8
1/16 1/8 1/16
Image filtering: Convolution operatore.g.gaussian filter(gaussian blur)
ImageCredit:http://what-when-how.com/introduction-to-video-and-image-processing/neighborhood-processing-introduction-to-video-and-image-processing-part-1/
SharpeningFilter
Original
111111111
000020000 -
Sharpening filter- Accentuates differences with local average
Source: D. Lowe
SharpeningFilter
Source: D. Lowe
Image filtering: Convolution operatorImportantFilter:Sobeloperator
ImageCredit:http://what-when-how.com/introduction-to-video-and-image-processing/neighborhood-processing-introduction-to-video-and-image-processing-part-1/
𝑘(𝑥, 𝑦)
𝑘(𝑥, 𝑦) =
1 0 -1
2 0 -2
1 0 -1
SobelinX
-101
-202
-101
Vertical Edge(absolute value)
Sobel
SlidebyJamesHays
SobelinY
-1-2-1
000
121
Horizontal Edge(absolute value)
Sobel
SlidebyJamesHays
Sobeloperatorsareequivalentto2Dpartialderivativesoftheimage• Verticalsobel operator– PartialderivativeinX(width)• Horizontalsobel operator– PartialderivativeinY(height)
• Cancomputemagnitudeandphaseateachlocation.
• Usefulfordetectingedges
https://en.wikipedia.org/wiki/Sobel_operator
Sobelfiltersare(approximate)partialderivativesoftheimage
𝜕𝑓(𝑥, 𝑦)𝜕𝑥
= lim=→?
𝑓 𝑥 + ℎ, 𝑦 − 𝑓(𝑥, 𝑦)ℎ
𝑓(𝑥, 𝑦) beyourinputimage,thenthepartialderivativeis:Let
𝜕𝑓(𝑥, 𝑦)𝜕𝑥
= lim=→?
𝑓 𝑥 + ℎ, 𝑦 − 𝑓(𝑥 − ℎ, 𝑦)2ℎ
Also:
Butdigitalimagesarenotcontinuous,theyarediscrete
ΔD𝑓[𝑥, 𝑦] = 𝑓[𝑥 + 1, 𝑦] − 𝑓[𝑥, 𝑦]
𝑓[𝑥, 𝑦] beyourinputimage,thenthepartialderivativeis:Let
ΔD𝑓[𝑥, 𝑦] = 𝑓[𝑥 + 1, 𝑦] − 𝑓[𝑥 − 1, 𝑦]Also:
Butdigitalimagesarenotcontinuous,theyarediscrete
ΔD𝑓[𝑥, 𝑦] = 𝑓[𝑥 + 1, 𝑦] − 𝑓[𝑥, 𝑦]
𝑓[𝑥, 𝑦] beyourinputimage,thenthepartialderivativeis:Let
ΔD𝑓[𝑥, 𝑦] = 𝑓[𝑥 + 1, 𝑦] − 𝑓[𝑥 − 1, 𝑦]Also:
-1 1
-1 0 1
k(x,y)=
k(x,y)=
SobelOperatorsSmoothinYandthenDifferentiateinX
1 0 -1k(x,y)=
1
2
1
* =
1 0 -1
2 0 -2
1 0 -1
SimilarlytodifferentiateinY
Frequency
FigurebyNationalInstruments
Anyfunctioncanbeapproximatedbyapolynomialfunction
…ifyouletyourpolynomialhaveahighdegree
…ANDyoucancomputethederivativesoftheoriginalfunctioneasily.
TaylorSeriesexpansion
Difficultinpractice
https://brilliant.org/wiki/taylor-series-approximation/
JeanBaptiste JosephFourier(1768-1830)hadcrazyidea(1807):Any univariate functioncanberewrittenasaweightedsumofsines andcosinesofdifferentfrequencies.
• Don’tbelieveit?– NeitherdidLagrange,Laplace,Poissonandotherbigwigs
– NottranslatedintoEnglishuntil1878!
• Butit’s(mostly)true!– calledFourierSeries– therearesomesubtlerestrictions
...themannerinwhichtheauthorarrivesattheseequationsisnotexemptofdifficultiesand...his
analysistointegratethemstillleavessomethingtobedesiredonthescoreofgeneralityandeven
rigour.
Laplace
LagrangeLegendre
SlidebyJamesHays
Example
SlidebyEmmanuelAgu
DiscreteFourierTransform
𝐹 𝑢 = 1 𝑓 𝑥 exp −2𝜋𝑖𝑥𝑢𝑁
NOP
DQ?
𝐹 𝑢 = 1 𝑓 𝑥 cos −2𝜋𝑥𝑢𝑁
+ 𝑖 sin −2𝜋𝑥𝑢𝑁
NOP
DQ?
InverseDiscreteFourierTransform
𝐹 𝑢 = 1 𝑓 𝑥 exp −2𝜋𝑖𝑥𝑢𝑁
NOP
DQ?
𝑓 𝑥 =1𝑁1 𝐹 𝑢 exp 2𝜋𝑖
𝑥𝑢𝑁
NOP
7Q?
DiscreteFourierTransform
Moregenerallyforimages(2DDFTandiDFT)
𝐹 𝑢, 𝑣 = 1 1 𝑓 𝑥, 𝑦 exp −2𝜋𝑖𝑥𝑢𝑀+𝑦𝑣𝑁
NOP
WQ?
XOP
DQ?
𝑓 𝑥, 𝑦 =1𝑀𝑁
1 1𝐹 𝑢, 𝑣 exp 2𝜋𝑖𝑥𝑢𝑀+𝑦𝑣𝑁
NOP
8Q?
XOP
7Q?
DiscreteFourierTransform- Visualization
SlidebyA.Zisserman
FourierTransform
• Fouriertransformstoresthemagnitudeandphaseateachfrequency• Magnitudeencodeshowmuchsignalthereisataparticularfrequency• Phaseencodesspatialinformation(indirectly)• Formathematicalconvenience,thisisoftennotatedintermsofrealandcomplexnumbers
22 )()( ww IRA +±=)()(tan 1
wwf
RI-=Amplitude: Phase:
SlidebyJamesHays
ImageFilteringintheFrequencyDomain
SlidebyA.Zisserman
ImageFilteringintheFrequencyDomain
SlidebyA.Zisserman
ImageFilteringintheFrequencyDomain
SlidebyA.Zisserman
TheConvolutionTheorem• TheFouriertransformoftheconvolutionoftwofunctionsistheproductoftheirFouriertransforms
• Convolution inspatialdomainisequivalenttomultiplication infrequencydomain!
]F[]F[]F[ hghg =*
]]F[][F[F* 1 hghg -=
Howcanthisbeuseful?
Questions?
39
top related