ee565 advanced image processing copyright xin li 20081 why do we need image model in the first...
TRANSCRIPT
EE565 Advanced Image Processing Copyright Xin Li 2008
1
Why do we Need Image Model in the first place? Any image processing algorithm has to
work on a collection (class) of images instead of a single one
Mathematical model gives us the abstraction of common properties of the images within the same class
Model is our hypothesis and images are our observation data In physics, can F=ma explain the
relationship between force and acceleration? In image processing, can this model fit this class of images?
EE565 Advanced Image Processing Copyright Xin Li 2008
2
Statistical vs. Deterministic They are different languages invented
by mathematicians to facilitate the communication of scientific results (just like English vs. Chinese spoken by people in different countries)
None is better than other – pick up the one you feel most comfortable with
We adopt a statistical language most of the time in this class
EE565 Advanced Image Processing Copyright Xin Li 2008
3
The Curse of Dimensionality
Even for a small-size image such as 64-by-64, we need to model it by a random process in 4096-dimensional space (R4096) whose covariance matrix is sized by 4096-by-4096
More importantly, we ask ourselves: do we need to consider all pixels simultaneously?
EE565 Advanced Image Processing Copyright Xin Li 2008
4
A Simple Idea: Locality
The conditional pdf is determined by a local neighborhood
),...,|(),...,|( 111 Nkkkkk XXXPXXXP N past samples
EE565 Advanced Image Processing Copyright Xin Li 2008
5
Parametric vs. Nonparametric
N
nknknk wXaX
1
non-parametricsampling
Input image
Xk1
2 3 4
5
678
Parametric model
EE565 Advanced Image Processing Copyright Xin Li 2008
6
Spatial vs. Wavelet
EE565 Advanced Image Processing Copyright Xin Li 2008
7
Complete vs. Overcomplete
EE565 Advanced Image Processing Copyright Xin Li 2008
8
Marginal PDF of wavelet coefficients
where
Laplacian
Gaussian
P: shape parameter: variance parameter
EE565 Advanced Image Processing Copyright Xin Li 2008
9
Joint PDF of Wavelet Coefficients
Neighborhood I(Q): {Left,Up,cousin and aunt}
X=
Y=
Joint pdf of two correlated random variables X and Y
Can you use this model to interpret why EZW works?
EE565 Advanced Image Processing Copyright Xin Li 2008
10
Good Bad
Spatially Fixed vs. Adaptive Models
EE565 Advanced Image Processing Copyright Xin Li 2008
11
Locality Revisited
Input image
),...,|(),...,|( 111 Nkkkkk XXXPXXXP N past samples
The definition of local neighborhood has to be relative
EE565 Advanced Image Processing Copyright Xin Li 2008
12
Application I: Image Denoising
Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering
Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering
From local to nonlocal denoising
EE565 Advanced Image Processing Copyright Xin Li 2008
13
Linear Frequency Weighting
),(),(
),(),(
),,(),(),(ˆ
2121
2121
212121
wwSwwS
wwSwwH
wwYwwHwwX
WX
X
22
2
,ˆwx
xaaYX
FT
Power spectrum |X|2
EE565 Advanced Image Processing Copyright Xin Li 2008
14
Spatially Adaptive Wiener Filtering of Wavelet Coefficients
Basic assumption: image source is modeled by a nonstationary Gaussian process
Signal variance is locally estimated from the windowed noisy observation data
]1
,0max[ˆ 2
1
22w
N
iix y
N
T
TN=T2
Recall
EE565 Advanced Image Processing Copyright Xin Li 2008
15
Wavelet Thresholding
DWT IWTThresholdingY X
~
otherwise
TnYifnYnX
0
|][|][][
~Hard thresholding
Soft thresholding
TnY
TnYTnY
TnYTnY
nX
|][|0
][][
][][
][~
Noisysignal
denoisedsignal
EE565 Advanced Image Processing Copyright Xin Li 2008
16
Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are
modeled by a Gaussian random variable with zero mean and spatially varying variance
Apply Wiener filtering to wavelet coefficients, i.e.,
][][
][][
~22
2
nYn
nnX
estimated in the same wayas spatial-domain (Slide 15)
EE565 Advanced Image Processing Copyright Xin Li 2008
17
Translation Invariant Denoising
Noisy image
Tce Tce-1ThresholdingWD =
shift(mK,nK) WD shift(-mK,-nK)
shift(m1,n1) WD shift(-m1,-n1)
Avg
denoised image
(mk,nk): a pair of integers, k=1-K (K: redundancy ratio)
EE565 Advanced Image Processing Copyright Xin Li 2008
18
Further Improvements Gaussian scalar mixture (GSM) based
denoising (Portilla et al.’ 2003) Instead of estimating the variance, it
explicitly addresses the issue of uncertainty with variance estimation
Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band
coefficients (refer to the posted paper)
EE565 Advanced Image Processing Copyright Xin Li 2008
19
Nonlocal Patch-based Denoising
WD
T T-1ThresholdingWD =
Noisy patches Denoised patches
EE565 Advanced Image Processing Copyright Xin Li 2008
20
Application II: Texture Syntehsis
Spatial-domain models Parametric autoregressive model Nonparametric resampling based
Wavelet-domain models Histogram matching based Parametric models based joint-
statistics
EE565 Advanced Image Processing Copyright Xin Li 2008
21
Spatial-DomainParametric Texture Synthesis
Gaussian noise image -- w Gabor Filter -- g
Output of Gabor Filter,x=g**w Synthesis Result from AR model,Xn(z)=W(z)/A(z)
EE565 Advanced Image Processing Copyright Xin Li 2008
22
Nonparametric Texture Synthesis
EE565 Advanced Image Processing Copyright Xin Li 2008
23
Wavelet-domain Histogram Matching
EE565 Advanced Image Processing Copyright Xin Li 2008
24
Wavelet-DomainParametric Texture Models
original
synthesized
EE565 Advanced Image Processing Copyright Xin Li 2008
25
Other Applications
Interpolation Spatial-domain covariance-based models PDE-based (nonlinear diffusion) models
Coding Statistical modeling of wavelet
coefficients Dual to wavelet-based image denoising
Data hiding DCT-domain human vision model
EE565 Advanced Image Processing Copyright Xin Li 2008
26
Summary on Theory Image models are at the foundation of
any image processing algorithm Statistical models help us deal with the
uncertainty in observation data Appropriate image representation (e.g.,
prediction/transform) facilitates the modeling task
Spatial adaptation is important – to have a good model for a wide class of images
Localized models are popular and powerful but nonlocal models might prevail later
EE565 Advanced Image Processing Copyright Xin Li 2008
27
Summary on Practice MATLAB provides a user friendly
platform for testing your ideas You can see what you have done
Experimental efficiency is important Avoid loops and test small-size images
C/C++ programming skills are a plus Efficient implementation could make a
difference
EE565 Advanced Image Processing Copyright Xin Li 2008
28
Beyond Image Processing I will discuss something important than
Wiener filtering or wavelet coding It is about you and your career
If you are a MS student, your master thesis will be your selling point in your job hunting
If you are a PhD student, you need to have a desire for first-class research
It all depends on your perspective - how you want to look at it
EE565 Advanced Image Processing Copyright Xin Li 2008
29
Where is Your Talent?
Outsider advantage: EZW, Turbo codes, Youtube, …
A tradeoff among mathematical capabilities, physics intuitions, programming skills, management style …
Selling your work could be even more important than doing the work itself
EE565 Advanced Image Processing Copyright Xin Li 2008
30
Follow your heart and
enjoy what you do!
Final Words