image deconvolution, denoising and...
TRANSCRIPT
Image deconvolution, denoisingand compression
T.E. Gureyev and Ya.I.Nesterets15.11.2002
CONVOLUTION
),,(),)((),( yxNyxPIyxD +!=
,'')','()','(),)(( !! ""=# dydxyyxxPyxIyxPI
where D(x,y) is the experimental data (registered image),I(x,y) is the unknown "ideal" image (to be found),P(x,y) is the (usually known) convolution kernel,* denotes 2D convolution, i.e.
and N(x,y) is the noise in the experimental data.
In real-life imaging, P(x,y) can be the point-spread functionof an imaging system; noise N(x,y) may not be additive.
Poisson noise: )}/(),({Poisson),( 22
avavavav IyxIIyxD !!=
EXAMPLE OF CONVOLUTION
* =
← 3%Poisson noise
←10%Poisson noise
DECONVOLUTION PROBLEM
(*) ),(),)((),( yxNyxPIyxD +!=
Deconvolution problem: given D, P and N, find I(i.e. compensate for noise and the PSF of the imaging system)
Blind deconvolution: P is also unknown.
Equation (*) is mathematically ill-posed, i.e. its solution maynot exist, may not be unique and may be unstable with respectto small perturbations of "input" data D, P and N. This is easyto see in the Fourier representation of eq.(*)
)(# ),(ˆ),(ˆ),(ˆ),(ˆ !"!"!"!" NPID +=
1) Non-existence: ?),(ˆ 0),(ˆ),(ˆ ,0),(ˆ =!==" #$#$#$#$ INPD
2) Non-uniqueness: anyINDP =!== ),(ˆ ),(ˆ),(ˆ ,0),( "#"#"#"#
3) Instability: !"#"#"#!"# /)],(ˆ),(ˆ[),(ˆ ),( NDIP $=%=
A SOLUTION OF THE DECONVOLUTION PROBLEM
)(# ),(ˆ),(ˆ),(ˆ),(ˆ !"!"!"!" NPID +=
)(! ),(ˆ/)],(ˆ),(ˆ[),(ˆ !"!"!"!" PNDI #=
We assume that 0),(ˆ !"#P (otherwise there is a genuine lossof information and the problem cannot be solved). Theneq.(!) provides a nice solution at least in the noise-free case(as in reality the noise cannot be subtracted exactly).
(*)-1 =
Convolution:
Deconvolution:
NON-LOCALITY OF (DE)CONVOLUTION
!! ""=# '')','()','(),)(( dydxyyxxPyxIyxPI
The value of convolution I*P at point (x,y) depends on allthose values I(x',y') within the vicinity of (x,y) where P(x-x', y-y')≠0. The same is true for deconvolution.
Convolution with asingle pixel widemask at the edges
Deconvolution (the error isdue to the non-locality andthe 1-pixel wide mask)
EFFECT OF NOISE
(*)-1 =
3% noise in theexperimental data with regularization→
without regularization→
In the presence of noise, the ill-posedness of deconvolutionleads to artefacts in deconvolved images:The problem can be alleviated with the help of regularization
)!(! ]|),(ˆ/[|),(ˆ),(ˆ),(ˆ 2* !"#"#"#"# += PPDI
PDI ˆ/ˆ),(ˆ !"#
EFFECT OF NOISE. II
(*)-1 =
10% noise in theexperimental data with regularization→
without regularization→
In the presence of stronger noise, regularization may not beable to deliver satisfactory results, as the loss of high frequencyinformation becomes very significant. Pre-filtering (denoising)before deconvolution can potentially be of much assistance.
DECONVOLUTION METHODSTwo broad categories
(1) Direct methodsDirectly solve the inverse problem (deconvolution).Advantages: often linear, deterministic, non-iterative and fast.Disadvantages: sensitivity to (amplification of) noise, difficultyin incorporating available a priori information.Examples: Fourier (Wiener) deconvolution, algebraic inversion.
(2) Indirect methodsPerform (parametric) modelling, solve the forward problem(convolution) and minimize a cost function.Disadvantages: often non-linear, probabilistic, iterative and slow.Advantages: better treatment of noise, easy incorporation ofavailable a priori information.Examples: least-squares fit, Maximum Entropy, Richardson-Lucy, Pixon
DIRECT DECONVOLUTION METHODSFourier (Wiener) deconvolutionBased on the formulaRequires 2 FFTs of the input image (very fast)Does not perform very well in the presence of noise
Convolution with 10% noise
]|),(ˆ/[|),(ˆ),(ˆ),(ˆ 2* !"#"#"#"# += PPDI
ITERATIVE WIENER DECONVOLUTIONThe method proposed by A.W.StevensonBuilds deconvolution as
Requires 2 FFTs of the input image at each iterationCan use large (and/or variable) regularization parameter α
Convolution with 10% noise
!"
=
"#+=1
0
)()( ),(),(n
k
k
knDPIWDWI $$
Richardson-Lucy (RL) algorithm
If PSF is shift invariant then RL iterative algorithm is written as
I(i+1) = I(i) Corr(D / (I(i) ∗ P ), P)
Correlation of two matrices is Corr(g, h)n,m= ∑i ∑j gn+i,m+j hi,j
Usually the initial guess is uniform, i.e. I(0) = 〈D〉
Advantages:1. Easy to implement (no non-linear optimisation is needed)2. Implicit object size (In
(0) = 0 ⇒ In(i) = 0 ∀i)
and positivity constraints (In(0) > 0 ⇒ In
(i) > 0 ∀i)Disadvantages:
1. Slow convergence in the absence of noise and instability in thepresence of noise
2. Produces edge artifacts and spurious "sources"
No noise
50 RL iterations
1000 RL iterations
OriginalImage
Data
RL
3% noise
20 RL iterations
6 RL iterations
OriginalImage
Data
RL
Joint probability of two events p(A, B) = p(A) p(B | A)
p(D, I, M) = p(D | I, M) p(I, M) = p(D | I, M) p(I | M) p(M)
= p(I | D, M) p(D, M) = p(I | D, M) p(D | M) p(M)
I – image M – model D – data
p(I | D, M) = p(D | I, M) p(I | M) / p(D | M)
p(I, M | D) = p(D | I, M) p(I | M) p(M) / p(D)
p(I | D , M) or p(I, M | D) – inference
p(I, M), p(I | M) and p(M) – priors
p(D | I, M) – likelihood function
Bayesian methods
(1)
(2)
Goodness-of-fit (GOF) and Maximum Likelihood (ML) methods
Assumes image prior p(I | M) = const and results in maximizationwith respect to I of the likelihood function p(D | I, M)
In the case of Gaussian noise (e.g. instrumentation noise) p(D | I, M) = Z‑1 exp(‑χ2 / 2) (standard chi-square distribution)
where χ2 = ∑k( (I ∗ P)k ‑ Dk )2 / σk2, Z = ∏k(2πσk
2)1/2
In the case of Poisson noise (count statistics noise)
! Without regularization this approach typically produces imageswith spurious features resulting from over-fitting of the noisydata
( ) ( )( )!
"#"=
kk
k
D
k
D
PIPIMIDp
k exp),|(
3% noise
Original Image
DataNo noise
Deconvolution
GOF
Maximum Entropy (ME) methods(S.F.Gull and G.J.Daniell; J.Skilling and R.K.Bryan)
ME principle states that a priori the most likely image is the onewhich is completely flat
Image prior: p(I | M) = exp(λS),
where S = -∑i pi log2 pi is the image entropy, pi = Ii / ∑i Ii
GOF term (usually): p(D | I, M) = Z‑1 exp(‑χ2 / 2)
The likelihood function: p(I | D, M ) ∝ exp(‑L + λS), L = χ2 / 2
+ tends to suppress spurious sources in the data
- can cause over-flattenning of the image
! the relative weight of GOF and entropy terms is crucial
Original Image Data (3% noise)ME deconvolution
λ = 2 λ = 5 λ = 10
p(I | D, M) ∝ exp(-L + λS)
Pixon method(R.C.Puetter and R.K.Pina, http://www.pixon.com/)
Image prior is p(I | M) = p({Ni}, n, N) = N! / (nN ∏Ni!)
where Ni is the number of units of signal (e.g. counts) in the i-thelement of the image, n is the total number of elements,
N = ∑i Ni is the total signal in the image
Image prior can be maximized by1. decreasing the total number of cells, n, and2. making the {Ni} as large as possible.
I (x) = (Ipseudo∗ K) (x) = ∫ dy K( (x – y) / δ(x) ) Ipseudo(y)
K is the pixon shape function normalized to unit volume
Ipseudo is the “pseudo image”
3% noise
Original Image
DataNo noise
Deconvolution
Pixon deconvolution
Original RL GOF
No noise
Pixon Wiener IWiener
Original
3% noise
ME
Pixon IWienerWiener
RL
DOES A METHOD EXIST CAPABLE OF BETTERDECONVOLUTION IN THE PRESENCE OF NOISE ???
1) Test images can be found on "kayak" in"common/DemoImages/DeBlurSamples" directory
2) Some deconvolution routines have been implementedonline and can be used with uploaded images. Theseroutines can be found at "www.soft4science.org" in the"Projects… On-line interactive services… Deblurring on-line" area