image processing in freq. domain restoration / enhancement inverse filtering match filtering /...
TRANSCRIPT
![Page 1: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/1.jpg)
Image Processing in
Freq. Domain
• Restoration / Enhancement
• Inverse Filtering
• Match Filtering / Pattern Detection
• Tomography
![Page 2: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/2.jpg)
Enhancement v.s. Restoration
• Image Enhancement: – A process which aims to improve bad
images so they will “look” better.
• Image Restoration: – A process which aims to invert known
degradation operations applied to images.
![Page 3: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/3.jpg)
Enhancement vs. Restoration
• “Better” visual representation
• Subjective
• No quantitative measures
• Remove effects of sensing environment
• Objective
• Mathematical, model dependent quantitative measures
![Page 4: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/4.jpg)
Typical Degradation SourcesTypical Degradation Sources
Low Illumination
Atmospheric attenuation(haze, turbulence, …)
Optical distortions(geometric, blurring)
Sensor distortion(quantization, sampling,
sensor noise, spectral sensitivity, de-mosaicing)
![Page 5: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/5.jpg)
Image Preprocessing
Enhancement Restoration
SpatialDomain
Freq.Domain
Point operations Spatial operationsFiltering
• Denoising• Inverse filtering• Wiener filtering
![Page 6: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/6.jpg)
Examples
Hazing
![Page 7: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/7.jpg)
Echo image Motion Blur
![Page 8: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/8.jpg)
Blurred image Blurred image + additive white noise
![Page 9: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/9.jpg)
Reconstruction as an Inverse ProblemReconstruction as an Inverse Problem
Distortion
Hnfg Hf
n noise
measurements
Original image
Reconstruction Algorithmg f̂
![Page 10: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/10.jpg)
• Typically:– The distortion H is singular or ill-posed.– The noise n is unknown, only its statistical properties
can be learnt.
g f̂ ng 1H
So what is the problem?
![Page 11: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/11.jpg)
Key point: Stat. Prior of Natural Images
fPfgPgfPfxx
maxargmaxargˆ MAP estimation:
likelihood prior
![Page 12: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/12.jpg)
Image space
measurements
•From amongst all possible solutions, choose the one that maximizes the a-posteriori probability:
Bayesian Reconstruction (MAP)Bayesian Reconstruction (MAP)
P(g|f)
P(f)
Most probable solutionP(f | g)P(g | f) P(f)
![Page 13: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/13.jpg)
Bayesian Denoising• Assume an additive noise model :
g=f + n
• A MAP estimate for the original f:
• Using Bayes rule and taking the log likelihood :
gfPff
|maxargˆ
fPfgPgP
fPfgPf
fflog|logminarg
|maxargˆ
![Page 14: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/14.jpg)
Bayesian Denoising
• If noise component is white Gaussian distributed:
g=f + n where n is distributed ~N(0,)
R(f) is a penalty for non probable f
fRfgff
2minargˆ
data term prior term
![Page 15: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/15.jpg)
Inverse Filtering
• Degradation model:
g(x,y) = h(x,y)*f(x,y)
G(u,v)=H(u,v)F(u,v)
F(u,v)=G(u,v)/H(u,v)
![Page 16: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/16.jpg)
Inverse Filtering (Cont.)
Two problems with the above formulation:1. H(u,v) might be zero for some (u,v).
2. In the presence of noise the noise might be amplified:
F(u,v)=G(u,v)/H(u,v) + N(u,v)/H(u,v)
Solution: Use prior information
FRGHFFF
2minargˆ
data term prior term
![Page 17: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/17.jpg)
Option 1: Prior Term• Use penalty term that restrains high F values:
where
• Solution:
FEFF
minargˆ
22 FGHFFE
022 *
FGHFH
F
FE
GHH
HF
*
*
ˆ0ˆ1),(
ˆ1),(
FvuH
HGFvuH
![Page 18: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/18.jpg)
Degraded Image (echo)
![Page 19: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/19.jpg)
F=G/H
![Page 20: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/20.jpg)
GHH
HF
*
*
ˆ
![Page 21: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/21.jpg)
Degraded Image (echo+noise)
![Page 22: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/22.jpg)
GHH
HF
*
*
ˆ
![Page 23: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/23.jpg)
• The inverse filter is C(H)= H*/(H*H+ )
• At some range of (u,v):
S(u,v)/N(u,v) < 1 noise amplification.
-1 -0.5 0 0.5 1 1.5 20
2
4
6
8
10
12
14
16
18
H
C(H
)
=10-3
![Page 24: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/24.jpg)
Option 2: Prior Term1. Natural images tend to have low energy at high frequencies
2. White noise tend to have constant energy along freq.
where FEF
Fminargˆ
2222 FvuGHFFE
50 100 150 200 250
40
60
80
100
120
140
160
180
200
220
240
![Page 25: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/25.jpg)
• Solution:
• This solution is known as the Wienner Filter• Here we assume N(u,v) is constant.• If N(u,v) is not constant:
022 22*
FvuGHFH
F
FE
GvuHH
HF
22*
*
ˆ
GvuNvuHH
HF
),(ˆ
22*
*
![Page 26: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/26.jpg)
Degraded Image (echo+noise)
![Page 27: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/27.jpg)
Wienner Filtering
![Page 28: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/28.jpg)
Wienner Previous
![Page 29: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/29.jpg)
Degraded Image (blurred+noise)
![Page 30: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/30.jpg)
Inverse Filtering
![Page 31: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/31.jpg)
Using Prior (Option 1)
![Page 32: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/32.jpg)
Wienner Filtering
![Page 33: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/33.jpg)
Matched Filter in Freq. Domain• Pattern Matching:
– Finding occurrences of a particular pattern in an image.
• Pattern:– Typically a 2D image fragment.
– Much smaller than the image.
![Page 34: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/34.jpg)
• Image Similarity Measure:– A function that assigns a nonnegative real value to two
given images.
– Small measure high similarity– Preferable to be a metric distance (non-negative, identity,
symmetric, triangular inequality)
• Can be combined with thresholding:
Image Similarity Measures
d( - ) ≥ 0
1 ( , )( , )
0
d P Q thresholdf P Q
otherwise
![Page 35: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/35.jpg)
• Scan the entire image pixel by pixel.
• For each pixel, evaluate the similarity between its local neighborhood and the pattern.
The Matching Approach
![Page 36: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/36.jpg)
• Given:– k×k pattern P
– n×n image I
– kxk window of image I located at x,y - Ix,y
• For each pixel (x,y), we compute the distance:
• Complexity O(n2k2)
The Euclidean Distance as a Similarity Measure
1
0,
2
2
2
,2,2
,,1
1,
k
ji
yxyxE
jiPjyixIk
PIk
PId
![Page 37: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/37.jpg)
• Convolution can be applied rapidly using FFT.
• Complexity: O(n2 log n)
FFT Implementation
Fixed 2( * )I P 2 * mask of 1'sI k k
2
,2,2 1
, PIk
PId yxyxE
jiPjyixIjiPjyixIk
ji
,,2,, 21
0,
2
![Page 38: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/38.jpg)
Naïve FFT
Time Complexity
Space
Integer Arithmetic Yes No
Run time for 16×16 1.33 Sec. 3.5 Sec.
Run time for 32×32 4.86 Sec. 3.5 Sec.
Run time for 64×64 31.30 Sec. 3.5 Sec.
2( log )O n n2 2( )O n k2n2n
Performance table for a 1024×1024 image, on a 1.8 GHz PC:
Naïve vs. FFT
![Page 39: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/39.jpg)
0
0.5
1
1.5
2
x 107
![Page 40: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/40.jpg)
• NCC:– A similarity measure, based on a normalized cross-
correlation function.
– Maps two given images to [0,1] (absolute value).
– Measures the angle between vectors Ix,y and P
– Invariant to intensity scale and offset.
Normalized Cross Correlation
11
11,
,
,,
PPII
PPIIPId
yx
yxyxNC
![Page 41: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/41.jpg)
• Note that
• Thus,
• The above expression can be implemented efficiently using 3 convolutions.
Efficient Implementation
YXnYXYYXX 11
11
11,
,,
,,,
PPII
PPIIPId
yxyx
yxyxyxNC
22,
22,,
,2
,
PkPPIkII
PIkPI
yxyxyx
yxyx
![Page 42: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/42.jpg)
0
0.5
1
1.5
2
x 107
Euclidean distance similarity measure
0
0.5
1
1.5
NCC similarity measure
![Page 43: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/43.jpg)
10 20 30 40 50 60 70
10
20
30
40
50
60
0.2
0.4
0.6
0.8
1
1.2
10 20 30 40 50 60 70
10
20
30
40
50
60
1
2
3
4
5
6
7
8
9
10
11
x 106
Euclidean distance similarity measure
NCC similarity measure
![Page 44: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/44.jpg)
Computer Tomography using FFT
![Page 45: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/45.jpg)
• In 1901 W.C. Roentgen won the Nobel Prize (1st in physics) for his discovery of X-rays.
CT Scanners
Wilhelm Conrad Röntgen
![Page 46: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/46.jpg)
• In 1979 G. Hounsfield & A. Cormack, won the Nobel Prize for developing the computer tomography.
• The invention revolutionized medical imaging.
CT Scanners
Allan M. Cormack
Godfrey N. Hounsfield
![Page 47: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/47.jpg)
f(x,y)
1
2
Tomography: Reconstruction from Projection
![Page 48: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/48.jpg)
• Projection: All ray-sums in a direction
• Sinogram: collects all projections
Projection & Sinogram
P(t)
f(x,y)
t
y
x
X-rays Sinogramt
![Page 49: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/49.jpg)
CT Image & Its Sinogram
K. Thomenius & B. Roysam
![Page 50: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/50.jpg)
The Slice Theorem
spatial domain frequency domain
f(x,y)
1
x
y
1
u
vFourier
Transform
![Page 51: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/51.jpg)
The Slice Theorem
f(x,y) = object
g(x) = projection of f(x,y) parallel to the y-axis: g(x) = f(x,y)dy
F(u,v) = f(x,y) e -2i(ux+vy) dxdyFourier Transform of f(x,y):
Fourier Transform at v=0 : F(u,0) = f(x,y) e -2iuxdxdy
= [ f(x,y)dy] e -2iuxdx
= g(x) e -2iux dx = G(u)
The 1D Fourier Transform of g(x)
![Page 52: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/52.jpg)
• Interpolate (linear, quadratic etc) in the frequency space to obtain all values in F(u,v).
• Perform Inverse Fourier Transform to obtain the image f(x,y).
Interpolation Method
u
v
F(u,v)
![Page 53: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/53.jpg)
![Page 54: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/54.jpg)
![Page 55: Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography](https://reader035.vdocument.in/reader035/viewer/2022062308/56649e245503460f94b1227a/html5/thumbnails/55.jpg)
THE END