chapter 5 – image pre-processing 5-0 5.1 brightness transformations 5.2 geometric transformations...
TRANSCRIPT
Chapter 5 – Image Pre-processing
5-1
5.1 Brightness Transformations
5.2 Geometric Transformations
5.3 Local Pre-processing
5.4 Image Restoration
5-2
Objectives of image pre-processing:
(a) Suppress image information that is not relevant to later work (b) Enhancing information that is useful for later analysis
(3) Image Enhancement, Image Restoration
Classes of Image Pre-processing Methods
Categorization:
(1) Point processing, Neighborhood processing
(2) Position invariant, Position variant
(a) Brightness Transformations(b) Geometric transformations
5.1 Brightness Transformations
5-3
5-4
○ Point Processing
• Histogram Equalization
5-5
14 2( 5) 2
9 5 5 9
y x
x
( ) ( ) ,x a
y d c cb a
a x b
Transform function
e.g.,
5-6
1 1
2 2
5-7
Theorem: Let T be a differentiable strictly increasing
or strictly decreasing function.
( ) ( )s rdr
p s p rds
( ) ( )s rp s ds p r drorThen,
Let r be a random variable having density
Let having density ( )Ts r sp
rp
Proof: Let : the distribution functions of r and s
(a) T strictly increasing
1 1 11 1( ( )) ( ) ( )
( ) ( ( )) ( ( ))rs r r
dP T s dT s dT sP s P T s p T s
ds ds ds
r sP ,P
1 1( ) ( ) ( ( ) ) ( ( )) ( ( ))s rP s P s P T s P T s P T s s r r
1 1( ) ( ),
dT s dT sds ds
1
1 ( )( ) ( ( ))s r
dT sp s p T s
ds
1 1 11 1( ( )) ( ) ( )
( ) ( ( )) ( ( ))( )rs r r
dP T s dT s dT sP s P T s p T s
ds ds ds
1 1( ) ( ) ( ( ) ) ( ( )) 1 ( ( ))s rP s P s P T s P T s P T s s r r
1 1( ) ( ),
dT s dT sds ds
1
1 ( )( ) ( ( ))s r
dT sp s p T s
ds
(b) T strictly decreasing
5-8
Example: Let x: a random variable with uniform distribution on (0, 1). Find the density g of 1 log(1 ), 0y x
( ) 0Y y 0y 0y For ,
Ans: Let Y : the distribution of y. Since y is a positive random variable,
for
1( ) ( ) ( log(1 ) )
(log(1 ) ) (1 )
( 1 ) 1
y x
x x
x
y
y y
Y y P y P y
P y P e
P e e
0
( ) ( )0 0
ye yg y Y y
y
The density of y:
5-9
Let transform function be
Then
( )T r0
( ) ( )r
rs T r p w dw
( )r
dsp r
dr
1( ) ( )| |, ( ) ( )| | 1
( )s r s r
r
drp s p r p s p r
ds p r
5-10
Called equalization or linearization.
5-11
○ Example
Let 1 0 1
( )0 elsewherer
r rp r
Since 2
0
1( ) ( 1)
2
r
s T r w dw r r
the transformation function T is
5-12
Since
From
i.e., is a uniform distribution
1( )=1 1 2r T s s
0 1r , 1 1 2r s , 1
1 2
drds s
1
( ) ( )| | ( 1)1 2
1 ( 1+ 1 2 +1)
1 2
1 1 2 1
1 2
s r
drp s p r r
ds s
ss
ss
( )s
p s
21
2s r r
5-13
Discrete case:
Let , ,
Transformation:
n
nrp k
k )( 0 1k
r
0( )
kj
k kj
ns T r
n
0
Scale : ( 1)k
jk
j
ns L
n
1 2 1k , , ,L
5-14
○ Example: L = 16, n = 360
5-15
5-16
○ Examples:
5-17
5-18
5-19
Specified Histogram Equalization-- Specify the shape of the histogram that we wish the processed image to have.
Input image Histogram specification
Histogram equalization
5-20
5-21
rLet : gray levels of the input image I
z : gray levels of the output image O
rp : the probability density function of r
that can be estimated from I
zp : the given specified probability density
function of z that we wish O to have
Let0
( ) ( )r
rs T r p w dw and
0( ) ( )
z
zG z p t dt s
Then ( ) ( )G z T r s and 1 1( ) ( ( ))z G s G T r
Both are known( ), ( )T r G z
5-22
Procedure:
Given: input image (I), specification ( ) 1. Compute from I
2. Compute from
3. Compute from
4. Compute
5. Transform I into O by
zp
rp
0( ) ( )
r
rT r p w dw r
p
0( ) ( )
z
zG z p t dt z
p
1( ( ))z G T r1( ( ))z G T r
5-23
Discrete case:
0 0( ) ( )
k k j
k k r jj j
ns T r p r ,
n
0( ) ( )
k
k z i ki
G z p z s , 1 2 1k , , ,L
1 1( ) ( ( ))k k k
z G s G T r
1 2 1k , , ,L
5-24
Example: Given image I of size 64 by 64 with 8 gray levels 0 1 7
, , ,r r r
Histogram of input image I:
0 0( ) ( )
k k j
k k r jj j
ns T r p r ,
n Transformation function:
5-25
Specified histogram:
Transformation function:0
( ) ( )k
k z i ki
G z p z s ,
5-26
Inverse transformation function: 1 1( ( )) ( )k k k
z G T r G s
1 1( ( )) ( )k k k
z G T r G s Output image O:
5-27
Histogram of output image O:
5-28
Input histogram Equalized histogram
Specified histogramOutput histogram
5-29
5.2. Geometric Transformations
Two steps: i) Pixel coordinate transformation ii) Brightness interpolationApplications: Remotely sensed image registration Bird-view generation Document skew
Scene grid Distorted grid image Recovered grid image
A geometric transform is a vector function T defined by ( , ), ( , ), ( , )x y x yT T x T x y y T x y T
5-30
Remotely Sensed Image Registration
• Blind areas around a vehicle
Window pillars
Height of vehicle
Driver’s position
Bird’s-Eye View Image Generation
Summary of blind areas
5-31
• System Configuration
32
Fish-eye camerawith wide-anglelens
Scene Image5-32
33
F DT D TT
D BT
F BT
iF BT
1,2,3,4i
5-33
34
Experiments
5-34
35
Four major tasks: (i) bird-view image generation
(ii) Parking space detection, (iii) path planning,
and (iv) automatic parking
Automatic Parking System
5-35
• Automatic parking
365-36
5-37
5.2.1. Pixel Coordinate Transformations
( , ),xx T x y ( , )yy T x y( , ),x yT TT
Geometric distortion types :
a. variable distance, b. panoramic
c. skew, e. scale, f. perspective
Transformation model:
where
◎ Image Enlargement
Step 1: Zero interleave
2
(( 1) / 2,( 1) / 2) if , : odd( , )
0 otherwise
m i j i jm i j
5-38
Step 2: Filling
(a) NN interpolation
(b) Bilinear interpolation
(c) Bicubic interpolation
1 1 0
1 1 0
0 0 0
1 2 11
2 4 24
1 2 1
1 4 6 4 1
4 16 24 16 41
6 24 36 24 664
4 16 24 16 4
1 4 6 4 1
5-39
5-40
(a) NN interpolation
(b) Bilinear interpolation
5-41
Inputimage
5-42
0 1 2 0 1 2, x a a x a y y b b x b y
Bilinear transformation:
Affine transformation:
Rotation : cos sin
sin cos
x x y
y x y
Scale change :
Skewing :
, x ax y bx
tan , x x y y y
0 1 2 3 0 1 2 3, x a a x a y a xy y b b x b y b xy
0 0
,m m r
r krk
r k
x a x y
0 0
m m rr k
rkr k
y b x y
Polynomial transformation:
5-43
Example:
0 1 2 3 x a a x a y a xy
1 1 1 1 2 2 2 2(( , ), ( , )), (( , ), ( , ))x y x y x y x y
3 3 3 3 4 4 4 4(( , ), ( , )), (( , ), ( , ))x y x y x y x y
1 0 1 1 2 1 3 1 1 1 0 1 1 2 1 3 1 1, x a a x a y a x y y b b x b y b x y
2 0 1 2 2 2 3 2 2 2 0 2 2 2 2 3 2 2, x a a x a y a x y y b b x b y b x y
3 0 1 3 2 3 3 3 3 3 0 1 3 2 3 3 3 3, x a a x a y a x y y b b x b y b x y
4 0 1 4 2 4 3 4 4 4 0 1 4 2 4 3 4 4, x a a x a y a x y y b b x b y b x y
Needs at least 4 pairs of corresponding points to determine the parameters
Bilinear transform
0 1 2 3 0 1 2 3, , , , , , ,a a a a b b b b
0 1 2 3y b b x b y b xy
5-44
01 1 1 1 1
11 1 1 1 1
22 2 2 2 2
32 2 2 2 2
03 3 3 3 3
13 3 3 3 3
24 4 4 4 4
34 4 4 4 4
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
1 0 0 0 0
0 0 0 0 1
ax y x y x
ax y x y y
ax y x y x
ax y x y y
bx y x y x
bx y x y y
bx y x y x
bx y x y y
A x b Solve x by the least square error method.
Image Registration:
Steps: 1. Detect salient points of images 2. Determine the point correspondences between the two images 3. Compute the parameters of the transformation functions
5-45
5.2.2. Brightness Interpolation
5-46
1( , ) ( , )x y x y T
(a) Nearest-Neighbor Interpolation
1 2 1( ) ( ) ( ),
1
F f x f x f x
a
2 1( ) (1 ) ( )F af x a f x
5-47
(b) Linear Interpolation
( , ) ( 1, ) (1- ) ( , )
( ( 1, 1) (1- ) ( 1, ))
(1- )( ( , 1) (1- ) ( , ))
( 1, 1) (1- ) ( 1, )
(1- ) ( , 1) (1- )(1
f x y f x y f x y
f x y f x y
f x y f x y
f x y f x y
f x y
=
) ( , )f x y
( , ) ( , 1) (1- ) ( , )f x y f x y f x y
( 1, ) ( 1, 1) (1- ) ( 1, )f x y f x y f x y
( , ) ( 1, ) (1- ) ( , )f x y f x y f x y
。 Bilinear Interpolation
5-48
◎ Generalization
○ Interpolation function R
0
0 if 0.5
( ) 1 if 0.5 0.5
0 if 0.5
R
○ Examples:
1
1 if 0( )
1 if 0R
5-49
1 2( ) (1 ) ( ) ( )f x f x f x
1 2( ) ( ) ( ) (1 ) ( )f x R f x R f x
○ Substituting into
NN-interpolation
( )R 0 ( )R
0 0If 0.5, then ( ) 1 and (1 ) 0R R
1 2 1( ) 1 ( ) 0 ( ) ( )f x f x f x f x
0 0If 0.5, then ( ) 0 and (1 ) 1R R
0 1 0 2( ) ( ) ( ) (1 ) ( )f x R f x R f x
5-50
0
0 if 0.5
( ) 1 if 0.5 0.5
0 if 0.5
R
1 2 2( ) 0 ( ) 1 ( ) ( )f x f x f x f x
○ Substituting into linear interpolation
1( )R
1 1 1 2
1 2
( ) ( ) ( ) (1 ) ( )
(1 ) ( ) ( )
f x R f x R f x
f x f x
5-51
1
1 if 0( )
1 if 0R
( )R
3 1 3 2
3 3 3 4
( ) ( 1 ) ( ) ( ) ( )
(1 ) ( ) (2 ) ( )
f x R f x R f x
R f x R f x
○ Cubic interpolation function3 2
33 2
1.5 2.5 1 if 1( )
0.5 2.5 4 2 if 1< 2R
5-52
○ Bi-cubic Interpolation
-- Apply cubic interpolation first along the rows and then down the columns
5-53
5-54
-- Applies a function to a neighborhood of each pixel-- Different functions different objectives e.g., noise removal (smoothing), edge detection, corner detection
5.3 Local (Neighborhood) Pre-Processing
5-55
Neighborhood (window, mask)
Function + Window = Filter
5-56
( , )
( , ) ( , ) ( , ) m n w
g i j f i m j n h m n
Convolution:
Filtration(Filtering)
5-57
Objective: noise removal
1 1 11
= 1 1 1 9
1 1 1
h
1-D case:
Mean filter Smoothed dataInput data2-D case:
• Linear Smoothing Filters
5.3.1 Image Smoothing
5-58
5-59
2
2
( )
21( )
2
x x
h x e
11
( ) ( )2
/ 2 1/ 2
1( )
(2 ) | |
T
nh e
x-x x-x
x
1D:
2D:
Gaussian Smoothing
1 2 11
= 2 4 2 16
1 2 1
h
Discrete case:
5-60
。 Mean Filters
( , ) ( , )
1( , ) ( , )
mns t w x y
g x y f s tmn
(i) Arithmetic mean:
(ii) Geometric mean:
( , ) ( , )
( , )1
( , )mns t w x y
mng x y
f s t
1
( , ) ( , )
( , ) ( , )mn
mn
s t w x y
g x y f s t
(iii) Harmonic mean:
5-61
(v) Alpha-trimmed mean filter
i) Order elements,
ii) Trim off end elements
iii) Take mean1
( ) /( 2 )n m
i
i m
x n m
(iv) Contra-harmonic mean:1
( , ) ( , )
( , ) ( , )
( , )
( , )( , )
mn
mn
Q
s t w x yQ
s t w x y
f s t
g x yf s t
0 : Arithmetric, 1: HarmonicQ Q
5-62
Assume noise n(x,y) is Gaussian, uncorrelated and has zero mean. ( , ) ( , ) ( , )g x y f x y n x y
。 Image Averaging
1
1
1
1( , ) ( , )
1[ ( , ) ( , )]
1( , ) ( , )
M
ii
M
ii
M
ii
g x y g x yM
f x y n x yM
f x y n x yM
{ ( , )} ( , )E g x y f x y
5-63
1 2 3 ,nx x x x ix
nx
: mask elements
。 Maximum filter:
1x
。 Minimum filter:
。 K-nearest neighbors (K-NN) mean filter
• Non-linear Smoothing Filters
5-64
。 Median filter / 2nx
5-65
5-66
。 Smoothing by a rotating masker
5-67
2
2
( , ) ( , )
2
2
( , ) ( , )
1 1( , ) ( , )
1 1 ( , ) ( , )
i j R i j R
i j R i j R
g i j g i jn n
g i j g i jn n
Dispersion
5-68
5.3.2 Edge Detectors
-- Edges are important information for image
understanding Origin of edges
Line drawing
5-69
Step edge (jump edge)
Ramp edge
Roof edge (crease edge)
Smooth edge
Line
Typical edge profiles:
5-70
○ Derivatives
0
0
1 D case :
lim
( ) ( ) lim
In a discrete case, 1
( 1) ( )
or ( ) ( 1)
1 or ( ( 1) ( 1))
2
x
x
df f
dx xf x x f x
xx
dff x f x
dxf x f x
f x f x
5-71
2 2( , ) ( ) ( )g g
g x yx y
/( , ) ( , )( , )
/i + j =
g xg x y g x yg x y
g yx y
2D case:
Gradient
Magnitude Direction
1tan /g g
x y
5-72
。 Prewitt filters
Consider Horizontal filter: , Smooth filter:
Combine
Vertical filter: , Smooth filter:
Combine
( 1) ( 1)f x f x
[-1 0 1] [1 1 1]
1 1 0 1
1 1 0 1 1 0 1
1 1 0 1xP
-1
0
1
1 1 1 1
0 1 1 1 0 0 0
1 1 1 1yP
[1 1 1]
5-73
Edge image Binary image Thinning
Vertical HorizontalInput
5-74
。 Roberts operator:
-1 0 1 -1 -2 -1
-2 0 2 , 0 0 0
-1 0 1 1 2 1x yP P
1 0 0 0 1 0
0 -1 0 , -1 0 0
0 0 0 0 0 0x yP P
。 Sobel operator:
。 Robinson operator:1 1 1 1 1 1
1 2 1 , 1 2 1
1 1 1 1 1 1x yP P
。 Kirsch operator:3 3 3 5 3 3
3 0 3 , 5 0 3
5 5 5 5 3 3x yP P
Sobel operator
Roberts operator
5-75
Laplacian:
Laplaceoperator:
Invariant under rotation (isotropic filter)
2 22
2 2
( , ) ( , )( , )
f x y f x yf x y
x y
0 1 0 0 0 0 0 1 0
1 4 1 1 2 1 0 2 0
0 1 0 0 0 0 0 1 0
5.3.3 Zero-Crossings of Second Derivatives
5-76
Step edge:
Ramp edge:
5-77
Zero crossing
0 + , + 00 － , － 0+ － , － +
5-78
。 Other Laplacian masks
11
1 4 11
1
。 Second derivatives are sensitive to noise
Example: Edge detection by taking zero crossings after a Laplace filtering
Marr-Hildreth methodSmooth the input image using a Gaussian before Laplace filtering
5-79
。 Gaussian smooth + Laplace filtering = Laplacian of Gaussian (LOG): 2G
2
221( )
2
x
G x e
2 2
2 22 2
2 2 22 2 2
1 1( ) ( 1)
2 2
x xd x
G e edx
2 2 2( ) ( ) ( )I G G I G I
Difference of Gaussian (DOG):1 2G G
0 0 1 0 0
0 2 2 1 0
1 2 16 2 1
0 1 2 1 0
0 0 1 0 0
Mexican hat
5-80
5-81
1 2( , ) ( ) ( )h x y h x h y
1 2
( , ) ( , ) ( , )
( ) ( ) ( , )
N N
m N n N
N N
m N n N
g x y h m n f x m y n
h m h n f x m y n
e.g., Laplacian filter1 2 1 1
2 4 2 2 [1 - 2 1]
1 2 1 1
Separable Filters
Convolution:
n × n filter:
2 (n × 1) filters:
2 multiplicationsn2 1 additionsn
2 multiplicationsn2 2 additionsn
5-82
5.4.3 Scale Space Filtering
Larger scale fewer noises, less precise in location
Smaller scale more noises, more precise in location
2 2/ 2( , ) xG x e ( , ) ( )* ( , )F x f x G x
Step 1: Edge detection
(i) Horizontal direction
(ii) Vertical direction v vE I G 2
223( )
2
xx
G x e
,h hE I G
2
221( ) ,
2
x
G x e
5.3.5 Canny Edge Detector Criteria:
a. Low error rate of detection:
no missing and extra edges
b. Localization of edges: precise in edge position
c. Single response: one-pixel width edges
5-83
(iii) Edge magnitude
Edge direction
2 2h vE E E
1tan ( )vp
h
E
E
Step 2: Non-maximum suppression
For each pixel p,
(i) Quantize to
0, 45, 90 or 135 degrees
(ii) Along
p is marked if its edge magnitude
is larger than both its two neighbors
p is ignored otherwise
p
pp
pE
5-84
Step 3: Hysteresis thresholding
For each marked pixel p,
(i) If > or
(ii) If and p is adjacent to an
edge pixel
p is considered as an edge pixel
Step 4: Repeat steps (1) - (3) for ascending
Step 5: Synthesize edges at multiple scales
HtpE
L p Ht E t
5-85
5-86
5-87
5-88
5.3.7 Edges in Multi-Spectral Images
Methods:
(a) Applied to individual components
(b) Applied to combination of component images
(i) difference or (ii) ratio
5-89
5.3.8 Pre-processing in the Frequency Domain Fourier Transform
Spatial Domain Frequency Domain
5-90
5-91
Low pass filtration
5-92
High Pass Filtration
5-93
Band pass filtration
5-94
Gaussian Frequency Filters
2low
0
1 ( , )( , ) exp( ( ) )
2
D u vG u v
D high low( , ) 1 ( , )G u v G u v
5-95
Spatial counterparts
Spatial filters
Frequency filters
5-96
Periodic noise removal
5-97
Butterworth frequency filters
low
0
1( , )
( , )1
nB u vD u v
D
high low( , ) 1 ( , )B u v B u v
5-98
Homomorphic filtering
,f i r
log log logz f i r
Z I R
S H Z H I H R
sexp( )g s
5-99
5.3.9 Line Detection
Line Finding Operators
Reinforcement of Linear Structure UsingParameterized Relaxation Labeling
J.S. Duncan & T. BirkholzerIEEE PAMI, vol. 14, no. 5, pp. 502-515, 1992
1. Edge Reinforcement
(a) (c) (e) (g)
(b) (d) (f) (h)
5-100
2. Edge Reinforcement with Thinning
(a) (c) (e) (g)
(b) (d) (f) (h)
3. Bar Reinforcement
5-103
5.3.10 Detection of Corners
: approximates curvature
Basic idea: corners possess large curvatures
Harris corner detector
f: image, W: image patch2
( , )
( , ) ( ( , ) ( , ))i i
W i i i ix y W
S x y f x y f x x y y
A corner point will have a high response of
for all ( , )WS x y ( , )x y
5-104
2
( , )
2
( , )
2
( , )
( , ) ( ( , ) ( , ))
( , ) ( , )( , ) ( , )
( , ) ( , )
i i
i i
i i
W i i i ix y W
i i i ii i i i
x y W
i i i i
x y W
S x y f x y f x x y y
xf x y f x yf x y f x y
yx y
xf x y f x y
yx y
( , )
[ ] [ ] ( , )i i
Wx y W
fx xf fx
x y x y A x yf y yx yy
( , ) ( , )
( , ) ( , )
i i i i
i i i i
f x x y y f x y
xf x y f x y
yx y
From Taylor approximation
5-105
( , )
2
2( , ) ( , )
2
2( , ) ( , )
( , )
i i
i i i i
i i i i
Wx y W
x y W x y W
x y W x y W
f
f fxA x y
f x yy
f f f
x yx
f f f
x y y
Harris matrix
Let 1 2, :
1 2If , : 1. Both small: no edge and corner2. One large and one small: ridge3. Both large: corner
eigenvalues
of WA
5-106
5-107
5-108
Moravec Detector
11
1 1
1MO( , ) ( , ) ( , )
8
ji
k i l j
i j f k l f i j
which is maximal in pixels with high contrast.
2 2 31 2 3 4 5 6 7
2 2 38 9 10
( , )
f i j c c x c y c x c xy c y c x
c x y c xy c y
Image function f(i,j) is approximated in the neighborhood of pixel (i,j)
Zuniga-Haralick Detector2 22 6 2 3 5 3 4
2 2 3/ 22 3
2( )ZH( , )
( )
c c c c c c ci j
c c
Kitchen-Rosenfeld Detector
2 22 6 2 3 5 3 4
2 22 3
2( )KR( , )
( )
c c c c c c ci j
c c
5-109
5.3.11. Maximally Stable External Regions Harris corner detector can be invariant to rotation andtranslation but variant to scale change and projective transformation.
Maximally Stable External Regions (MSER) are invariant to translation, rotation, similarity and affine transformations.To detect (MSER): Maximal regions: union all connected components of all frames of a sequence of thresholded I with frame t corresponding to threshold t. Minimal regions: obtained by inverting the intensity of I and running the same process.
5-110
5-111
5-112
5.4.1 Degradation Model
5.4.2 Diagonalization of Circulant
and Block-Circulant Matrices
5.4.3 Inverse Filtering
5.4.4 Algebraic Approach to Restoration
5.4.5 Wiener Filter
5.4 Image Restoration
Objective: reconstruct or recover from degradation
(e.g., moving, distortion).
Idea: modeling the degradation
5-113
( , ) [ ( , )] ( , )g x y f x y n x y H
5.4.1 Degradation Model
○
Problem: Given g(x,y) and some knowledge
about degradation H and noise n, obtain
an approximation to f(x,y).
Mathematically,
5-114
Assume ( , ) 0 ( , ) [ ( , )]n x y g x y f x y H
Image:
If H is linear, i.e.,
( , ) ( , ) ( , )f x y f x y d d
( , ) [ ( , ) ( , ) ]g x y f x y d d H
Degraded image:
If H is homogeneous, i.e.,
( , ) [ ( , ) ( , )]g x y f x y d d H
( , ) ( , ) [ ( , )]g x y f x y d d H
1 1 2 2 1 1 2 2[ ( , ) ( , )] [ ( , )] [ ( , )]k g x y k g x y k g x y k g x y H H H
[ ( , )] [ ( , )]kg x y k g x yH H
2-115
Let ( , , , ) [ ( , )]h x y x y H
( , ) ( , ) ( , , , )g x y f h x y d d
If H is position invariant, i.e.,
( , ) ( , ) ( , )g x y f h x y d d
( , ) ( , ) ( , )g x y f x y h x y
( , , , ) [ ( , )] ( , )h x y x y h x y H
( , ) [ ( , )]g x y f x y H
In discrete case,1 1
0 0( , ) ( , ) ( , )
M N
e e em n
g x y f m n h x m y n
: PSF
2-116
1
0( ) ( ) ( ), 0, 1, , 1
M
e e em
g x f m h x m x M
Consider 1D case,
In matrix form, ,Hg f where
(0) ( 1) , (0) ( 1)T T
e e e ef f M g g M f g
(0) ( 1) ( 1)
(1) (0) ( 2)
( 1) ( 2) (0)
e e e
e e e
e e e
h h h M
h h h M
H
h M h M h
2-117
( )eh xSince is periodic,
(0) ( 1) (1)
(1) (0) (2)
( 1) ( 2) (0)
e e e
e e e
e e e
h h M h
h h h
H
h M h M h
: circulant matrix
( ) ( ),e eh x h M x
2-118
◎ Diagonalization
Define2
( ) (0) ( 1)exp[ 1 ]
2 ( 2)exp[ 2 ]
2 (1)exp[ ( 1) ]
e e
e
k h h M j kM
h M j kM
he j M kM
2 2 2[ 1 ] [ 2 ] [ ( 1) ]
( ) [ 1 ]j k j k j M k TM M Mk e e e
W
0, 1, , 1k M ( ) ( ) ( ),H k k k W W
• Circulant Matrices
2-119
(0) ( 1) (1) 1
(1) (0) (2) 1
(0)
( 1) ( 2) (0) 1
e e e
e e e
e e e
h h M h
h h h
h M h M h
HW
e.g., k = 0
(0) ( 1) (2) (1)
(1) (0) ( 1) (2)
( 1) ( 2) (1) (0)
e e e e
e e e e
e e e e
h h M h h
h h h M h
h M h M h h
2-120
1
1
(0) (0) ( (0) ( 1) (1))
1
e e eh h M h
W
(0) ( 1) (1)
(0) ( 1) (1)
e e e
e e e
h h M h
h h M h
(0) (0) (0)H W W
2-121
1(0) ( 1) (1)2
exp[ 1]
(1)
2exp[ ( 1)]( 1) (0)
e e e
e e
h h M h
jM
H
j Mh M hM
W
2 2(0) ( 1)exp[ 1] (1)exp[ ( 1)]
2 2(1) (0)exp[ 1] (2)exp[ ( 1)]
2 2( 1) ( 2)exp[ 1] (0)exp[ ( 1)]
e e e
e e e
e e e
h h M j h j MM M
h h j h j MM M
h M h M j h j MM M
For k = 1
2-122
2 2(1) (1) (0) ( 1)exp[ 1] ( 2)exp[ 2]e e eh h M j h M j
M M W
1
2exp[ 1]
2(1)exp[ ( 1)]
2exp[ ( 1)]
e
jM
h j MM
j MM
2 2(0) ( 1)exp[ 1] (1)exp[ ( 1)]
2 2 2(0)exp[ 1] (1)exp[ ( 1) ]
2 2(0)exp[ ( 1)] (1)exp[ ( 1)]
e e e
e e
e e
h h M j h j MM M
h j h j M jM M M
h j M h j MM M
(1) (1) (1)H W W
2-123
1 and HW WD D W HW
( ) ( ) ( ),H k k kW WFrom 0, 1, , 1k M
i.e., formed by the M eigenvectors of of H,
(0) (0) (0),H W W (1) (1) (1),H W W
( 1) ( 1) ( 1),H M M M W W
[ (0) (1) ( 1)]W M W W Wwhere
12 1 2( , ) exp[ ], ( , ) exp[ ]W k i j ki W k i j ki
M M M
1 *,W W where * denotes conjugate transpose
: a diagonal matrix and ( , ) ( )D k k k
2-124
2 2 exp[ ( ) ] exp[ ]j M i k j ik
M M
1
0
2 2 ( ) (0) (1)exp[ ] (2)exp[ 2 ]
2 ( 1)exp[ ( 1) ]
( ) exp[ 2 / ]
e e e
e
M
ex
k h h j k h j kM M
h M j M kM
h x j kx M
: the DFT of ( )eh x( )k
2( ) (0) ( 1)exp[ 1 ] ( 2)
2 2 exp[ 2 ] (1)exp[ ( 1) ]
e e ek h h M j k h MM
j k he j M kM M
2-125
2 2( , ) exp[ ], ( , ) exp[ ]M Nw i m j im w k n j kn
M M
• Block Circulant Matrices
Define (1,1) . . (1, )
. . . .
. . . .
( ,1) . . ( , )MN MN
W W M
W
W M W M M
( , ) ( , ) , ( , ) ( , ).M N N NW i m w i m W W k n w k n
where
, 0, 1, , 1; , 0, 1, , 1i m M k n N
( , )W i j are N by N matrices and
2-126
1 1 11( , ) ( , )M NW i m w i m W
M ；
1 11( , ) ( , )N NW k n w k n
N
The inverse matrix 1W
1 1THW WD H WDW H WD W ； ； ；Likewise,
is the DFT of ( , )D k k ( , ).eh x y1 ,D W HW where
1 2( , ) exp[ ]Nw k n j kn
N
1 2( , ) expMw i m j im
M
◎ Effects of Diagonalization on the Degradation Model
• 1-D case: Hg f-1 -1 -1, H WDW W DW g f f g f
1,H WDW From and
-1
2
1 1 1 . . . 1 (0)2 2 2 (1)1 exp[ ] exp[ 2] . . . exp[ ( 1)]
. . . . . . .
1 . . . . . . .
2 2 21 exp[ ] . . exp[ ] . exp[ ( 1)]
. . . . . . .
2 21 exp[ ( 1)] . . . . exp[ ( 1) ]
e
e
f
fj j j MM M M
WM
j k j ki j k MM M M
j M j MM M
f
( )
( -1)
e
e
f k
f M
2-127
-1
1
0
1 2( ) {[ (0) (1)exp[ ] ( 1)
2 1 2 exp[ ( 1)]} ( )exp[ ] ( )
e e e
M
ei
W k f f j k f MM M
j k M f i j ki F kM M M
f
2-128
: the DFT of ( )ef i-1 ( (1), , ( -1))TW F F M f F : the DFT of f
: the DFT of gSimilarly, -1W g G1
0
1
0
2( , ) ( ) ( )exp[ ]
1 2( ) exp[ ] ( )
M
ei
M
e ei
D k k k h i j kiM
M h i j ki MH kM M
is the DFT of sequence ( )eh x
-1( ) ( )W k F kf
( )eH k
2-129
-1 -1 ,W DWg fFrom ( ) ( ) ( ),eG k MH k F k0, 1, , 1k M
• 2-D case:
( , ) ( , ) ( , ) ( , )eG u v MNH u v F u v N u v
0, 1, , 1;u M 0, 1, , 1v N
Including noise term, ( ) ( ) ( ) ( )eG k MH k F k N k
( , ) ( , ) ( , ) ( , )eG u v H u v F u v N u v
Ignore the scale factor MN,
5-130
( , ) ( , ) ( , ) ( , )eG u v H u v F u v N u v
( , ) ( , )( , )
( , ) ( , )e e
G u v N u vF u v
H u v H u v
Low-pass filtering:( , )
( , ) ( , )( , )e
G u vF u v L u v
H u v
Constrained division:( , )
if ( , )( , )( , )
( , ) if ( , )
ee
e
G u vH u v d
H u vF u v
G u v H u v d
5.4.3 Inverse Filtering
2-131
5.4.4 Algebraic Approach to Restoration
A. Unconstrained restorationB. Constrained restoration
A. Unconstrained restoration
2 ˆ( )ˆ ˆ ˆ( ) , 0 2 ( )ˆ
TJJ H H H
f
f g f g ff
,H g f nFrom H n g f
Find f̂2 2ˆmin minH g f ns.t.
Let
1 1 1 1
ˆ ˆ2 ( ) 2 2 0
ˆ
ˆ ( ) ( )
T T T
T T
T T T T
H H H H H
H H H
H H H H H H H
g f g + f =
f = g
f g g g
2-132
B. Constrained restoration 22 2 ˆmin Q subject to H f n g f
Using the method of Lagrange multipliers,2 2 2ˆ ˆ ˆ( ) ( )J Q H f f g f n
ˆ( ) ˆ ˆ0 2 2 ( )ˆ
T TJQ Q H H
f
f g ff
where Q is a linear operator on f.
11ˆ ( )T T TH H Q Q H
f g
2-133
5.4.5 Winer Filtering
{ }, { }T Tf nR E R E ff nn : correlation matrices
of f and nThe ij-th element of fR is given by { }i jE f f
We hope noise-to-signal ratio /n fR R to be small.
{ } { }, { } { }i j j i i j j iE f f E f f E n n E n n
and f nR R : real symmetric matrices
For images, pixels within 20 to 30 pixels can generally be correlated. A typical correlation matrix has a bound of nonzero elements about the main diagonal and zeros in the right upper and left lower corner regions.
2-134
can be made to approximate block and f nR Rcirculant matrices and can be diagonalized by
1 1, f nR WAW R WBW 1t
f nQ Q R RLet1ˆ ( )T T TH H rQ Q H f gSubstitute into
1 1ˆ ( )T Tf nH H rR R H f g
From 1 * 1 and ,TH WDW H WD W * 1 1 * 1( )( )TH H WD W WDW WD DW
1 1 1 * 1ˆ ( )f nWDD W rR R WD W f g
5-135
From 1 1 1 1 1 1( ) ( )f nR R WAW WBW WA BW
* 1 1 1 1 * 1ˆ ( )WD DW rWA BW WD W f g
1 1 * 1 1 1 1 * 1ˆ ( )W W WD DW rWA BW WD W f g
* 1 1 1 1 * 1 1 1
* 1 1 1
( ) [ ( ) ]
( )
WD DW rWA BW W D D rA B W
W D D rA B W
1 1 * 1 1 1 * 1
* 1 1 * 1
ˆ ( )
( )
W W W D D rA B W WD W
D D rA B D W
f g
g
* 1 1 *ˆ ( )F D D rA B D G
5-136
2
2
2
( , )ˆ ( , ) ( , )( , ) [ ( , ) / ( , )]
( , )1 ( , )
( , ) ( , ) [ ( , ) / ( , )]
e
e f
e
e e f
H u vF u v G u v
H u v r S u v S u v
H u vG u v
H u v H u v r S u v S u v
,eD MNH Ignore M, N
A and B are diagonal matrices derived from the
and f nR R
2( , ) ( , ) ( , )e e eH u v H u v H u v
where ( , ) : Power spectrum of noise
( , ) : Power spectrum of image n
f
S u v n
S u v f
5-137
(when no noise )( , ) 0S u v Ideal inverse filter
If ( , ) and ( , ) are unknown, approximatefS u v S u v2
2
( , )1ˆ ( , ) ( , )( , ) ( , )
e
e e
H u vF u v G u v
H u v H u v k
where k : constant
( , )ˆ ( , )( , )e
G u vF u v
H u v
2
2
( , )1ˆ ( , ) ( , )( , ) ( , ) [ ( , ) / ( , )]
e
e e f
H u vF u v G u v
H u v H u v r S u v S u v
Parametric Wiener filter
2-138
Different k’s
5-139
Image f(x,y) undergoes planar motion
: the components of motion
T : the duration of exposure
Fourier transform,
○ Applications -- Motion Deblurring
0 0( ) and ( )x t y t
0 00( , ) ( ( ), ( ))
Tg x y f x x t y y t dt
2 ( )
2 ( )0 00
( , ) ( , )
[ ( ( ), ( )) ]
j ux vy
T j ux vy
G u v g x y e dxdy
f x x t y y t dt e
dxdy
5-140
2 ( )0 00
0 0 0 0
( , ) [ ( ( ), ( ))
]
( , ) ( , ) exp[ 2 ( )
(translation pro
T j ux vyG u v f x x t y y t e
dxdy dt
f x x y y F u v j ux vy
perty)
( , ) ( , ) ( , ), ( , ) ( , ) / ( , )G u v H u v F u v F u v G u v H u v
0 0
0 0
2 ( ( ) ( ))
0
2 ( ( ) ( ))
0
( , ) ( , )
( , )
T j ux t vy t
T j ux t vy t
G u v F u v e dt
F u v e dt
0 02 ( ( ) ( ))
0Let ( , )
T j ux t vy tH u v e dt
5-141
Suppose uniform linear motion: 0 0( ) / , ( ) 0x t at T y t
0 02 ( ( ) ( )) 2 /
0 0( , )
sin( )
T Tj ux t vy t j uat T
j ua
H u v e dt e dt
Tua e
ua
Note H vanishes at u = n/a (n: an integer)
Restore image by the inverse or Wiener filter
5-142
○ Defocusing
1 ( )( , )
J arH u v
ar
○ Atmospheric turbulence2 2 5 / 6- ( )( , ) c u vH u v e