chapter 5 – image pre-processing
DESCRIPTION
5.1 Brightness Transformations 5.2 Geometric Transformations 5.3 Local Pre-processing 5.4 Image Restoration. Chapter 5 – Image Pre-processing. Objectives of image pre-processing: (a) Suppress image information that is not relevant to later work - PowerPoint PPT PresentationTRANSCRIPT
Chapter 5 – Image Pre-processing
5-1
5.1 Brightness Transformations
5.2 Geometric Transformations
5.3 Local Pre-processing
5.4 Image Restoration
5-2
• Objectives of image pre-processing:
(a) Enhancing information that is useful for later analysis
(b) Suppress image information that is not relevant to later work
(3) Image Enhancement, Image Restoration
• Classes of Image Pre-processing Methods
Categorization:
(1) Point processing, Neighborhood processing
(2) Position invariant, Position variant
(a) Brightness Transformations
(b) Geometric transformations
5.1 Brightness Transformations
5-3
5-4
○ Point Processing• Histogram Equalization (HE)
5-5
14 2( 5) 2
9 5 5 9
y x
x
( ) ( ) ,x a
y d c cb a
a x b
Transform function
e.g.,
5-6
1 1
2 2
5-7
Theorem: Let T be a differentiable strictly increasing
or strictly decreasing function.
( ) ( )s rdr
p s p rds
( ) ( )s rp s ds p r drorThen,
Let r be a random variable having density
Let having density ( )s T r sp
rp
Proof: Let : the distribution functions of r and s
(a) T strictly increasing
1 1 11 1( ( )) ( ) ( )
( ) ( ( )) ( ( ))rs r r
dP T s dT s dT sP s P T s p T s
ds ds ds
r sP ,P
1 1( ) ( ) ( ( ) ) ( ( )) ( ( ))s rP s P s P T s P T s P T s s r r
1 1( ) ( ),
dT s dT sds ds
1
1 ( )( ) ( ( ))s r
dT sp s p T s
ds
1 1 11 1( ( )) ( ) ( )
( ) ( ( )) ( ( ))( )rs r r
dP T s dT s dT sP s P T s p T s
ds ds ds
1 1( ) ( ) ( ( ) ) ( ( )) 1 ( ( ))s rP s P s P T s P T s P T s s r r
1 1( ) ( ),
dT s dT sds ds
1
1 ( )( ) ( ( ))s r
dT sp s p T s
ds
(b) T strictly decreasing
5-8
Let transform function be
Then
( )T r0
( ) ( )r
rs T r p w dw
( )r
dsp r
dr
1( ) ( )| |, ( ) ( )| | 1
( )s r s r
r
drp s p r p s p r
ds p r
5-9
Called equalization or linearization.
5-10
○ Example
Let 1 0 1
( )0 elsewherer
r rp r
Since 2
0
1( ) ( 1)
2
r
s T r w dw r r
the transform function T is
5-11
Since
From
i.e., is a uniform distribution
1( )=1 1 2r T s s
0 1r , 1 1 2r s , 1
1 2
drds s
1( ) ( )| | ( 1)
1 2
1 ( 1+ 1 2 +1)
1 2
1 1 2 1
1 2
s r
drp s p r r
ds s
ss
ss
( )s
p s
21
2s r r
5-12
Discrete case:
Let , ,
Transform:
n
nrp k
k )( 0 1k
r
0( )
kj
k kj
ns T r
n
0
Scale : ( 1)k
jk
j
ns L
n
1 2 1k , , ,L
5-13
5-14
○ Examples:
5-15
Specified Histogram Equalization (SHE)
-- Specify the shape of the histogram that we wish the processed image to have.
Input image SHE HE
5-16
5-17
Procedure:
Given: input image (I) and specification ( )
1. Compute the probability density of gray
levels r of the input image I
2. Compute from
3. Compute from
4. Compute
5. Transform I into O by
zp
rp
0( ) ( )
r
rT r p w dw r
p
0( ) ( )
z
zG z p t dt z
p
1( ( ))z G T r1( ( ))z G T r
z : gray levels of the output image O
5-18
Discrete case:
0 0( ) ( )
k k j
k k r jj j
ns T r p r ,
n
0( ) ( )
k
k z i ki
G z p z s , 1 2 1k , , ,L
1 1( ) ( ( ))k k k
z G s G T r
1 2 1k , , ,L
5-19
Example: Given image I of size 64 by 64 with 8 gray levels 0 1 7
, , ,r r r
Histogram of input image I:
0 0( ) ( )
k k j
k k r jj j
ns T r p r ,
n Transformation function:
5-20
Specified histogram:
Transformation function:0
( ) ( )k
k z i ki
G z p z s ,
5-21
Inverse transformation function: 1 1( ( )) ( )k k k
z G T r G s
1 1( ( )) ( )k k k
z G T r G s Output image O:
5-22
Histogram of output image O:
5-23
Input histogram Equalized histogram
Specified histogramOutput histogram
5-24
5.2. Geometric Transformations• Geometric Distortion Types :
a. Variable distance,
b. Panoramic c. Skew,
e. Scale, f. Perspective• Geometric Transformation
Scene grid Distorted grid image Recovered grid image
Two steps: i) Pixel coordinate transformation ii) Brightness interpolation
5-25
5.2.1. Pixel Coordinate TransformationsTransformation model: ( , ), ( , ), ( , )x y x yT T x T x y y T x y T
Bilinear transformation:0 1 2 3 0 1 2 3, x a a x a y a xy y b b x b y b xy
0 0
,m m r
r krk
r k
x a x y
0 0
m m rr k
rkr k
y b x y
Polynomial transformation:
0 1 2 0 1 2, x a a x a y y b b x b y Affine transformation:
Rotation : cos sin , sin cosx x y y x y
Scale change : , x ax y bx
Skewing : tan , x x y y y
5-26
Example: 0 1 2 3 x a a x a y a xy
1 1 1 1 2 2 2 2(( , ), ( , )), (( , ), ( , ))x y x y x y x y
3 3 3 3 4 4 4 4(( , ), ( , )), (( , ), ( , ))x y x y x y x y
1 0 1 1 2 1 3 1 1 1 0 1 1 2 1 3 1 1, x a a x a y a x y y b b x b y b x y
2 0 1 2 2 2 3 2 2 2 0 2 2 2 2 3 2 2, x a a x a y a x y y b b x b y b x y
3 0 1 3 2 3 3 3 3 3 0 1 3 2 3 3 3 3, x a a x a y a x y y b b x b y b x y
4 0 1 4 2 4 3 4 4 4 0 1 4 2 4 3 4 4, x a a x a y a x y y b b x b y b x y
Needs at least 4 pairs of corresponding points to determine the parameters
Bilinear transform
0 1 2 3 0 1 2 3, , , , , , ,a a a a b b b b
0 1 2 3y b b x b y b xy
5-27
01 1 1 1 1
11 1 1 1 1
22 2 2 2 2
32 2 2 2 2
03 3 3 3 3
13 3 3 3 3
24 4 4 4 4
34 4 4 4 4
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
ax y x y x
ax y x y y
ax y x y x
ax y x y y
bx y x y x
bx y x y y
bx y x y x
bx y x y y
A x b Solve x by the least square error method.
Application: Image Registration
Steps: 1. Detect salient points of images 2. Determine the point correspondences between the two images 3. Compute the parameters of the transformation functions
5-28
Blind areas around a vehicle
Window pillars
Height of vehicle
Driver’s position
Application: Bird’s-View Image Generation
Summary of blind areas
5-29
System Configuration
Fish-eye camera:
Scene Image5-30
F DT D TT
D BT
F BT
iF BT
1,2,3,4i
5-31
5-32
5.2.2. Brightness Interpolation
5-33
1( , ) ( , )x y x y T
(a) Nearest-Neighbor Interpolation
1 2 1( ) ( ) ( ),
1
F f x f x f x
a
2 1( ) (1 ) ( )F af x a f x
5-34
(b) Linear interpolation
Bilinear interpolation
( , ) ( 1, ) (1- ) ( , )
( ( 1, 1) (1- ) ( 1, ))
(1- )( ( , 1) (1- ) ( , ))
( 1, 1) (1- ) ( 1, )
(1- ) ( , 1) (1- )(1 ) ( , )
f x y f x y f x y
f x y f x y
f x y f x y
f x y f x y
f x y f x y
◎ Generalization
○ Interpolation function R
0
0 if 0.5
( ) 1 if 0.5 0.5
0 if 0.5
R
○ Examples:
1
1 if 0( )
1 if 0R
5-35
1 2( ) (1 ) ( ) ( )f x f x f x
1 2( ) ( ) ( ) (1 ) ( )f x R f x R f x
○ Substituting into
NN-interpolation
( )R 0 ( )R
0 0If 0.5, then ( ) 1 and (1 ) 0R R
1 2 1( ) 1 ( ) 0 ( ) ( )f x f x f x f x
0 0If 0.5, then ( ) 0 and (1 ) 1R R
0 1 0 2( ) ( ) ( ) (1 ) ( )f x R f x R f x
5-36
0
0 if 0.5
( ) 1 if 0.5 0.5
0 if 0.5
R
1 2 2( ) 0 ( ) 1 ( ) ( )f x f x f x f x
○ Substituting into linear interpolation
1( )R
1 1 1 2
1 2
( ) ( ) ( ) (1 ) ( )
(1 ) ( ) ( )
f x R f x R f x
f x f x
5-37
1
1 if 0( )
1 if 0R
( )R
3 1 3 2
3 3 3 4
( ) ( 1 ) ( ) ( ) ( )
(1 ) ( ) (2 ) ( )
f x R f x R f x
R f x R f x
○ Cubic interpolation function3 2
33 2
1.5 2.5 1 if 1( )
0.5 2.5 4 2 if 1< 2R
5-38
5-39
-- Applies a function to a neighborhood of each pixel-- Different functions different objectives e.g., noise removal (smoothing), edge detection, corner detection
5.3 Local (Neighborhood) Pre-Processing
Neighborhood (window, mask)
5-40
1 1 11
= 1 1 1 9
1 1 1
h
1-D case:
Mean filter Smoothed dataInput data
2-D case:
• Linear Smoothing Filters
5.3.1 Image Smoothing Function + Window = Filter
5-41
2
2
( )
21( )
2
x x
h x e
11
( ) ( )2
/ 2 1/ 2
1( )
(2 ) | |
T
nh e
x-x x-x
x
1D:
2D:
Gaussian Smoothing
1 2 11
= 2 4 2 16
1 2 1
h
Discrete case:
5-42
1 2 3 ,nx x x x ix
nx
: mask elements
。 Maximum filter:
1x
。 Minimum filter:
。 K-nearest neighbors (K-NN) mean filter
• Non-linear Smoothing Filters
5-43
。 Median filter / 2nx
5-44
5-45
。 Smoothing by a rotating masker
2
2
( , ) ( , )
2
2
( , ) ( , )
1 1( , ) ( , )
1 1 ( , ) ( , )
i j R i j R
i j R i j R
g i j g i jn n
g i j g i jn n
Dispersion
5-46
5.3.2 Edge Detectors
-- Edges are important information for image
understanding Origin of edges
Line drawing
5-47
Step edge (jump edge)
Ramp edge
Roof edge (crease edge)
Smooth edge
Line
Typical edge profiles:
5-48
○ First Derivatives
0
0
1 D case :
lim
( ) ( ) lim
In a discrete case, 1
( 1) ( )
or ( ) ( 1)
1 or ( ( 1) ( 1))
2
x
x
df f
dx xf x x f x
xx
dff x f x
dxf x f x
f x f x
5-49
2 2( , ) ( ) ( )g g
g x yx y
/( , ) ( , )( , )
/i + j =
g xg x y g x yg x y
g yx y
2D case:
Gradient
Magnitude Direction
1tan /g g
x y
5-50
。 Roberts operator:
-1 0 1 -1 -2 -1
-2 0 2 , 0 0 0
-1 0 1 1 2 1x yP P
1 0 0 0 1 0
0 -1 0 , -1 0 0
0 0 0 0 0 0x yP P
。 Sobel operator: 。 Robinson operator:
1 1 1 1 1 1
1 2 1 , 1 2 1
1 1 1 1 1 1x yP P
。 Kirsch operator:3 3 3 5 3 3
3 0 3 , 5 0 3
5 5 5 5 3 3x yP P
。 Prewitt filters
1 0 1
1 0 1
1 0 1xP
1 1 1
0 0 0
1 1 1yP
5-51
Edge image Binary image Thinning
Vertical HorizontalInput
5-52
Laplacian:
Laplaceoperator:
Invariant under rotation (isotropic filter)
2 22
2 2
( , ) ( , )( , )
f x y f x yf x y
x y
0 1 0 0 0 0 0 1 0
1 4 1 1 2 1 0 2 0
0 1 0 0 0 0 0 1 0
5.3.3 Second Derivatives
5-53
Step edge:
Ramp edge:
0 + , + 00 － , － 0+ － , － +
Zero crossing
5-54
。 Other Laplacian masks
11
1 4 11
1
。 Second derivatives are sensitive to noise
Example: Edge detection by taking zero crossings after a Laplace filtering
Marr-Hildreth methodSmooth the input image using a Gaussian before Laplace filtering
5-55
。 Gaussian smooth + Laplace filtering = Laplacian of Gaussian (LOG): 2G
2
221( )
2
x
G x e
2 2
2 22 2
2 2 22 2 2
1 1( ) ( 1)
2 2
x xd x
G e edx
2 2 2( ) ( ) ( )I G G I G I
Difference of Gaussian (DOG):1 2G G
0 0 1 0 0
0 2 2 1 0
1 2 16 2 1
0 1 2 1 0
0 0 1 0 0
Mexican hat
5-56
5-57
○ Scale Space Filtering
Larger scale fewer noises, less precise in location
Smaller scale more noises, more precise in location
2 2/ 2( , ) xG x e 2( ) ,G I
Step 1: Edge detection
(i) Horizontal direction
(ii) Vertical direction v vE I G 2
223( )
2
xx
G x e
,h hE I G
2
221( ) ,
2
x
G x e
5.3.5 Canny Edge Detector
5-58
(iii) Edge magnitude
Edge direction
2 2h vE E E
1tan ( )vp
h
E
E
Step 2: Non-maximum suppression
For each pixel p,
(i) Quantize to
0, 45, 90 or 135 degrees
(ii) Along
p is marked if its edge magnitude
is larger than both its two neighbors
p is ignored otherwise
p
pp
pE
5-59
Step 3: Hysteresis thresholding
For each marked pixel p,
(i) If > or
(ii) If and p is adjacent to an
edge pixel
p is considered as an edge pixel
Step 4: Repeat steps (1) - (3) for ascending
Step 5: Synthesize edges at multiple scales
HtpE
L p Ht E t
5-60
5-61
5-62
5-63
5.3.8 Pre-processing in the Frequency Domain
5-64
Low pass filtrationOriginal image
High pass filtration Band pass filtration
5-65
Homomorphic filtering
,f i r
log log logz f i r
Z I R
S H Z H I H R
sexp( )g s
5-66
5.3.9 Line Detection
Line Finding Operators
Reinforcement of Linear Structure UsingParameterized Relaxation Labeling
J.S. Duncan & T. BirkholzerIEEE PAMI, vol. 14, no. 5, pp. 502-515, 1992
1. Edge Reinforcement
(a) (c) (e) (g)
(b) (d) (f) (h)
5-67
2. Edge Reinforcement with Thinning
(a) (c) (e) (g)
(b) (d) (f) (h)
3. Bar Reinforcement
5-70
5.3.10 Detection of Corners
: approximates curvature
Basic idea: corners possess large curvatures
Harris corner detector
f: image, W: image patch2
( , )
( , ) ( ( , ) ( , ))i i
W i i i ix y W
S x y f x y f x x y y
A corner point will have a high response of
for all ( , )WS x y ( , )x y
5-71
2
( , )
2
( , )
2
( , )
( , ) ( ( , ) ( , ))
( , ) ( , )( , ) ( , )
( , ) ( , )
i i
i i
i i
W i i i ix y W
i i i ii i i i
x y W
i i i i
x y W
S x y f x y f x x y y
xf x y f x yf x y f x y
yx y
xf x y f x y
yx y
( , )
[ ] [ ] ( , )i i
Wx y W
fx xf fx
x y x y A x yf y yx yy
( , ) ( , )
( , ) ( , )
i i i i
i i i i
f x x y y f x y
xf x y f x y
yx y
From Taylor approximation
5-72
( , )
2
2( , ) ( , )
2
2( , ) ( , )
( , )
i i
i i i i
i i i i
Wx y W
x y W x y W
x y W x y W
f
f fxA x y
f x yy
f f f
x yx
f f f
x y y
Harris matrix
Let 1 2, :
1 2If , : 1. Both small: no edge and corner2. One large and one small: ridge3. Both large: corner
eigenvalues
of WA
5-73
5-74
5-75
Moravec Detector
11
1 1
1MO( , ) ( , ) ( , )
8
ji
k i l j
i j f k l f i j
which is maximal in pixels with high contrast.
2 2 31 2 3 4 5 6 7
2 2 38 9 10
( , )
f i j c c x c y c x c xy c y c x
c x y c xy c y
Image function f(i,j) is approximated in the neighborhood of pixel (i,j)
Zuniga-Haralick Detector2 22 6 2 3 5 3 4
2 2 3/ 22 3
2( )ZH( , )
( )
c c c c c c ci j
c c
Kitchen-Rosenfeld Detector
2 22 6 2 3 5 3 4
2 22 3
2( )KR( , )
( )
c c c c c c ci j
c c
5-76
5.4 Image RestorationObjective: reconstruct or recover from degradation
(e.g., moving, distortion).
Idea: modeling the degradation
( , ) [ ( , )] ( , )g x y f x y n x y H
5.4.1 Degradation Model
Mathematically,
Assume ( , ) 0 ( , ) [ ( , )]n x y g x y f x y H
1
5-77
Image:
If H is linear, i.e.,
( , ) ( , ) ( , )f x y f x y d d
( , ) [ ( , ) ( , ) ]g x y f x y d d H
Degraded image:
If H is homogeneous, i.e.,
( , ) [ ( , ) ( , )]g x y f x y d d H
( , ) ( , ) [ ( , )]g x y f x y d d H
1 1 2 2 1 1 2 2[ ( , ) ( , )] [ ( , )] [ ( , )]k g x y k g x y k g x y k g x y H H H
[ ( , )] [ ( , )]kg x y k g x yH H
( , ) [ ( , )]g x y f x yH
2-78
Let ( , , , ) [ ( , )]h x y x y H
( , ) ( , ) ( , , , )g x y f h x y d d
If H is position invariant, i.e.,
( , ) ( , ) ( , )g x y f h x y d d
( , ) ( , ) ( , )g x y f x y h x y
( , , , ) [ ( , )] ( , )h x y x y h x y H
In discrete case,1 1
0 0( , ) ( , ) ( , )
M N
e e em n
g x y f m n h x m y n
: PSF
2-79
1
0( ) ( ) ( ), 0, 1, , 1
M
e e em
g x f m h x m x M
Consider 1D case,
In matrix form, ,Hg f where
(0) ( 1) , (0) ( 1)T T
e e e ef f M g g M f g
(0) ( 1) ( 1)
(1) (0) ( 2)
( 1) ( 2) (0)
e e e
e e e
e e e
h h h M
h h h M
H
h M h M h
2-80
( )eh xSince is periodic,
(0) ( 1) (1)
(1) (0) (2)
( 1) ( 2) (0)
e e e
e e e
e e e
h h M h
h h h
H
h M h M h
H is a circulant matrix
( ) ( ),e eh x h M x
2-81
Define2
( ) (0) ( 1)exp[ 1 ]
2 ( 2)exp[ 2 ]
2 (1)exp[ ( 1) ]
e e
e
k h h M j kM
h M j kM
he j M kM
2 2 2[ 1 ] [ 2 ] [ ( 1) ]
( ) [ 1 ]j k j k j M k TM M Mk e e e
W
0, 1, , 1k M ( ) ( ) ( ),H k k k W W
5.4.2 Circulant Matrices
2-82
(0) ( 1) (1) 1
(1) (0) (2) 1
(0)
( 1) ( 2) (0) 1
e e e
e e e
e e e
h h M h
h h h
h M h M h
HW
e.g., k = 0
(0) ( 1) (2) (1)
(1) (0) ( 1) (2)
( 1) ( 2) (1) (0)
e e e e
e e e e
e e e e
h h M h h
h h h M h
h M h M h h
2-83
1
1
(0) (0) ( (0) ( 1) (1))
1
e e eh h M h
W
(0) ( 1) (1)
(0) ( 1) (1)
e e e
e e e
h h M h
h h M h
(0) (0) (0)H W W
2-84
1(0) ( 1) (1)2
exp[ 1]
(1)
2exp[ ( 1)]( 1) (0)
e e e
e e
h h M h
jM
H
j Mh M hM
W
2 2(0) ( 1)exp[ 1] (1)exp[ ( 1)]
2 2(1) (0)exp[ 1] (2)exp[ ( 1)]
2 2( 1) ( 2)exp[ 1] (0)exp[ ( 1)]
e e e
e e e
e e e
h h M j h j MM M
h h j h j MM M
h M h M j h j MM M
For k = 1
2-85
2 2(1) (1) (0) ( 1)exp[ 1] ( 2)exp[ 2]e e eh h M j h M j
M M W
1
2exp[ 1]
2(1)exp[ ( 1)]
2exp[ ( 1)]
e
jM
h j MM
j MM
2 2(0) ( 1)exp[ 1] (1)exp[ ( 1)]
2 2 2(0)exp[ 1] (1)exp[ ( 1) ]
2 2(0)exp[ ( 1)] (1)exp[ ( 1)]
e e e
e e
e e
h h M j h j MM M
h j h j M jM M M
h j M h j MM M
(1) (1) (1)H W W
2-86
1 and HW WD D W HW
( ) ( ) ( ),H k k kW WFrom 0, 1, , 1k M
i.e., formed by the M eigenvectors of of H,
(0) (0) (0),H W W (1) (1) (1),H W W
( 1) ( 1) ( 1),H M M M W W
[ (0) (1) ( 1)]W M W W W
12 1 2( , ) exp[ ], ( , ) exp[ ]W k i j ki W k i j ki
M M M
1 *,W W where * denotes conjugate transpose
: a diagonal matrix and ( , ) ( )D k k k
2-87
2 2 exp[ ( ) ] exp[ ]j M i k j ik
M M
Q
1
0
2 2 ( ) (0) (1)exp[ ] (2)exp[ 2 ]
2 ( 1)exp[ ( 1) ]
( ) exp[ 2 / ]
e e e
e
M
ex
k h h j k h j kM M
h M j M kM
h x j kx M
: the DFT of ( )eh x( )k
2( ) (0) ( 1)exp[ 1 ] ( 2)
2 2 exp[ 2 ] (1)exp[ ( 1) ]
e e ek h h M j k h MM
j k he j M kM M
2-88
2 2( , ) exp[ ], ( , ) exp[ ]M Nw i m j im w k n j kn
M M
• Block Circulant Matrices
Define (1,1) . . (1, )
. . . .
. . . .
( ,1) . . ( , )MN MN
W W M
W
W M W M M
( , ) ( , ) , ( , ) ( , ).M N N NW i m w i m W W k n w k n
where
, 0, 1, , 1; , 0, 1, , 1i m M k n N
( , )W i j are N by N matrices and
2-89
1 1 11( , ) ( , )M NW i m w i m W
M ；
1 11( , ) ( , )N NW k n w k n
N
The inverse matrix 1W
1 1THW WD H WDW H WD W ； ； ；Likewise,
is the DFT of ( , )D k k ( , ).eh x y1 ,D W HW where
1 2( , ) exp[ ]Nw k n j kn
N
1 2( , ) expMw i m j im
M
◎ Diagonalization
• 1-D case: Hg f-1 -1 -1, H WDW W DW g f f g f
1,H WDW From and
-1
2
1 1 1 . . . 1 (0)2 2 2 (1)1 exp[ ] exp[ 2] . . . exp[ ( 1)]
. . . . . . .
1 . . . . . . .
2 2 21 exp[ ] . . exp[ ] . exp[ ( 1)]
. . . . . . .
2 21 exp[ ( 1)] . . . . exp[ ( 1) ]
e
e
f
fj j j MM M M
WM
j k j ki j k MM M M
j M j MM M
f
( )
( -1)
e
e
f k
f M
2-90
-1
1
0
1 2( ) {[ (0) (1)exp[ ] ( 1)
2 1 2 exp[ ( 1)]} ( )exp[ ] ( )
e e e
M
ei
W k f f j k f MM M
j k M f i j ki F kM M M
f
2-91
: the DFT of ( )ef i-1 ( (0), , ( -1))TW F F M f F : the DFT of f
: the DFT of gSimilarly, -1W g G1
0
1
0
2( , ) ( ) ( )exp[ ]
1 2( ) exp[ ] ( )
M
ei
M
e ei
D k k k h i j kiM
M h i j ki MH kM M
is the DFT of sequence ( )eh x
-1( ) ( )W k F kf
( )eH k
2-92
-1 -1 ,W DWg fFrom
( ) ( ) ( ),eG k MH k F k 0, 1, , 1k M
• 2-D case:
( , ) ( , ) ( , ) ( , )eG u v MNH u v F u v N u v
0, 1, , 1;u M 0, 1, , 1v N
Including noise term, ( ) ( ) ( ) ( )eG k MH k F k N k
( , ) ( , ) ( , ) ( , )eG u v H u v F u v N u v
Ignore the scale factor MN,
5-93
( , ) ( , ) ( , ) ( , )eG u v H u v F u v N u v
( , ) ( , )( , )
( , ) ( , )e e
G u v N u vF u v
H u v H u v
Low-pass filtering:( , )
( , ) ( , )( , )e
G u vF u v L u v
H u v
Constrained division:( , )
if ( , )( , )( , )
( , ) if ( , )
ee
e
G u vH u v d
H u vF u v
G u v H u v d
5.4.3 Inverse Filtering
2-94
5.4.4 Algebraic Approach to Restoration
A. Unconstrained restorationB. Constrained restoration
A. Unconstrained restoration
2 ˆ( )ˆ ˆ ˆ( ) , 0 2 ( )ˆ
TJJ H H H
f
f g f g ff
,H g f nFrom H n g f
Problem: Find f̂2 2ˆmin minH g f ns.t.
Let
1 1 1 1
ˆ ˆ2 ( ) 2 2 0
ˆ
ˆ ( ) ( )
T T T
T T
T T T T
H H H H H
H H H
H H H H H H H
g f g + f =
f = g
f g g g
‧
2-95
B. Constrained restoration
2 22ˆ ˆmin Q subject to H f n g f
Using the method of Lagrange multipliers,2 2 2ˆ ˆ ˆ( ) ( )J Q H f f g f n
ˆ( ) ˆ ˆ0 2 2 ( )ˆ
T TJQ Q H H
f
f g ff
where Q is a linear operator.
11ˆ ( )T T TH H Q Q H
f g
Problem: Find f̂ s.t.
2-96
5.4.5 Winer Filtering
{ }, { }T Tf nR E R E ff nn : correlation matrices
of f and nThe ij-th element of fR is given by { }i jE f f
We hope noise-to-signal ratio /n fR R to be small.
{ } { }, { } { }i j j i i j j iE f f E f f E n n E n n Q
and f nR R : real symmetric matrices
For images, pixels within 20 to 30 pixels can generally be correlated. A typical correlation matrix has a bound of nonzero elements about the main diagonal and zeros in the right upper and left lower corner regions.
2-97
can be made to approximate block and f nR Rcirculant matrices and can be diagonalized by
1 1, f nR WAW R WBW 1t
f nQ Q R RLet1ˆ ( )T T TH H rQ Q H f gSubstitute into
1 1ˆ ( )T Tf nH H rR R H f g
From 1 * 1 and ,TH WDW H WD W * 1 1 * 1( )( )TH H WD W WDW WD DW
1 1 1 * 1ˆ ( )f nWDD W rR R WD W f g
5-98
From 1 1 1 1 1 1( ) ( )f nR R WAW WBW WA BW
* 1 1 1 1 * 1ˆ ( )WD DW rWA BW WD W f g
1 1 * 1 1 1 1 * 1ˆ ( )W W WD DW rWA BW WD W f g
* 1 1 1 1 * 1 1 1
* 1 1 1
( ) [ ( ) ]
( )
WD DW rWA BW W D D rA B W
W D D rA B W
Q
1 1 * 1 1 1 * 1
* 1 1 * 1
ˆ ( )
( )
W W W D D rA B W WD W
D D rA B D W
f g
g
* 1 1 *ˆ ( )F D D rA B D G
5-99
2
2
2
( , )ˆ ( , ) ( , )( , ) [ ( , ) / ( , )]
( , )1 ( , )
( , ) ( , ) [ ( , ) / ( , )]
e
e f
e
e e f
H u vF u v G u v
H u v r S u v S u v
H u vG u v
H u v H u v r S u v S u v
,eD MNH Ignore M, N
A and B are diagonal matrices derived from the
and , respectively.f nR R
2( , ) ( , ) ( , )e e eH u v H u v H u v
where ( , ) : Power spectrum of noise
( , ) : Power spectrum of image n
f
S u v n
S u v f
5-100
(when no noise )( , ) 0S u v Ideal inverse filter
If ( , ) and ( , ) are unknown, approximatefS u v S u v2
2
( , )1ˆ ( , ) ( , )( , ) ( , )
e
e e
H u vF u v G u v
H u v H u v k
where k : constant
( , )ˆ ( , )( , )e
G u vF u v
H u v
2
2
( , )1ˆ ( , ) ( , )( , ) ( , ) [ ( , ) / ( , )]
e
e e f
H u vF u v G u v
H u v H u v r S u v S u v
Parametric Wiener filter
2-101
Different k’s
5-102
Image f(x,y) undergoes planar motion
: the components of motion
T : the duration of exposure
Fourier transform,
○ Applications -- Motion Deblurring
0 0( ) and ( )x t y t
0 00( , ) ( ( ), ( ))
Tg x y f x x t y y t dt
2 ( )
2 ( )0 00
( , ) ( , )
[ ( ( ), ( )) ]
j ux vy
T j ux vy
G u v g x y e dxdy
f x x t y y t dt e
dxdy
5-103
2 ( )0 00
0 0 0 0
( , ) [ ( ( ), ( ))
]
( , ) ( , )exp[ 2 ( )
(translation pr
T j ux vyG u v f x x t y y t e
dxdy dt
f x x y y F u v j ux vy
Q
operty)
( , ) ( , ) ( , ), ( , ) ( , ) / ( , )G u v H u v F u v F u v G u v H u v
0 0
0 0
2 ( ( ) ( ))
0
2 ( ( ) ( ))
0
( , ) ( , )
( , )
T j ux t vy t
T j ux t vy t
G u v F u v e dt
F u v e dt
0 02 ( ( ) ( ))
0Let ( , )
T j ux t vy tH u v e dt
5-104
Suppose uniform linear motion: 0 0( ) / , ( ) 0x t at T y t
0 02 ( ( ) ( )) 2 /
0 0( , )
sin( )
T Tj ux t vy t j uat T
j ua
H u v e dt e dt
Tua e
ua
Note H vanishes at u = n/a (n: an integer)
Restore image by the inverse or Wiener filter
5-105
○ Defocusing 1 ( )( , )
J arH u v
ar
○ Atmospheric turbulence2 2 5 / 6- ( )( , ) c u vH u v e