vector valued image regularization with pde’s: a common framework for different applications
DESCRIPTION
Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications. CVPR 2003 Best Student Paper Award. Definitions. Scalar images: Intensity images Vector valued images: RGB, HSV, YIQ… Regularization: Finding approximate solution for ill-posed problems. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/1.jpg)
Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications
CVPR 2003 Best Student Paper Award
![Page 2: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/2.jpg)
Definitions
Scalar images: Intensity images Vector valued images: RGB, HSV, YIQ… Regularization: Finding approximate solution
for ill-posed problems.
Let G be 2x2 matrix
dc
baG
2
1,)(
iiigtrace G
![Page 3: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/3.jpg)
Divergence and Curl
Vectors are obtained from image gradients I
y
A
x
AAAdiv yx
)(
Divergence defines expansion or contraction per unit volume
),( yxA vector function
yyxxy
xII
I
IdivIdivI
)(
Curl of a vector field is defined by cross product between vectors
![Page 4: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/4.jpg)
Structure Tensor
n
iyx
y
xn
i
Tii II
I
III
11
G
![Page 5: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/5.jpg)
Structure Tensor
n
iyx
y
xn
i
Tii II
I
III
11
G
One dimensional image (gray level)
yyyx
yxxxgray IIII
IIIIG
)1,(),(
),1(),(
yxIyxII
yxIyxII
y
x
![Page 6: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/6.jpg)
Structure Tensor
yyyx
yxxx
yyyx
yxxx
yyyx
yxxxcolor
TTTcolor
BBBB
BBBB
GGGG
GGGG
RRRR
RRRR
BBGGRR
G
G
reigenvecto
eigenvalue
color
2
1
2
1
φ
φλ
φ
φG
Eigenvalue and eigenvectors of G (spectral elements)
Can be computed using Matlab functions
![Page 7: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/7.jpg)
x derivative y derivative
![Page 8: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/8.jpg)
More Insight (hessian&tensor)
Hessian of image I
Laplacian of I
Tensor: Matrix of matrices. They abuse the notation: Symmetric semi-positive definite matrix
yyyx
xyxx
II
IIH
yyxx IItraceII )()div( H
![Page 9: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/9.jpg)
xx derivative yy derivative
![Page 10: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/10.jpg)
tracexx derivative yy derivative
![Page 11: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/11.jpg)
Image Regularization Functional minimization: Euler-Lagrange equations
Divergence expression: Diffusion of pixel values from high to low concentration
Oriented Laplacians: Image smoothing in eigenvector directions weighted by corresponding eigenvalue.
dINnRI
))((min:
)( itI IDdiv
vvuutI IcIc 21
![Page 12: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/12.jpg)
Variational Problem
Define variational problem:
dIIIIFIE yxyx ),,,()(
0
yx I
F
dy
d
I
F
dx
d
I
FEuler-Lagrange equation
Solution using classic iterative method:
)1()1()1()1()()(
00
tI
F
dy
dt
I
F
dx
dt
I
FtItI
t
tI
II
yx
t
![Page 13: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/13.jpg)
Variational Problem Horn&Schunck Example
0 tyx fvfuf dtdy
dtdx vu
dxdyvvuufvfufvuvuE yxyxtyx22222),,,(
022 avguu
yyxxxtyx uuffvfuf
022
avgvv
yyxxytyx vvffvfuf
tyxxt
avgt fvfuf
fuu
1
yx u
E
yu
E
xu
E
yx v
E
yv
E
xv
E
tyxyt
avgt fvfuf
fvv
1
![Page 14: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/14.jpg)
Defining Energy Functional
Functional: (minimization based)
dIERI
),(min)(3:
Euler-Lagrange is given by (relation to divergence based methods)
)div(div i
i
ii ID
I
I
t
I
y
x
TTD
22
increasing function)()(, afbfab
![Page 15: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/15.jpg)
D Matrix
TTD
22 is 2x2.
Eigenvalues and eigenvectors of D are
22 21
21 uu
iTTi I
t
I
22div
![Page 16: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/16.jpg)
Old School Laplacian Approach
TT ccT 222111 vvuu
vvuu IcIct
I21
Second order image derivatives in directions of eigenvectors of G at that point (structure tensor)(Edge preserving smoothing)
1 tt II
)( ii Ttracet
IH
Solution of this PDE can be given by:
),(
)0()(
tii GIItt
T
)4
exp(4
1)(
1),(
ttG t xxT
xT
![Page 17: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/17.jpg)
Laplacian Example (constant T)
Constant T, such that direction (eigenvectors are same)
![Page 18: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/18.jpg)
Laplacian Example
![Page 19: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/19.jpg)
Laplacian Example (varying T)
T is not constant and but independent of image content. There are similarities with bilateral filtering.
![Page 20: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/20.jpg)
Laplacian Example (varying T)
![Page 21: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/21.jpg)
)( ii IDdivt
I
Relation Between T and D
cb
baD
)()()( DdivIDtraceIDdiv Tiii H
Homework Due 19 January 2005
Show the following: (Page 3 of the paper):
![Page 22: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/22.jpg)
Relation Between T and D
)( ii Ttracet
IH
)()( DdivIDtracet
I Tii
i
H
Ti
Ti
Ti
Ti
Ti
Itrace
IGtrace
IGdivtraceIDdivtraceDdivI
)()()(
2
2
![Page 23: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/23.jpg)
Relation Between T and D
3
1
3
1
3
1
3
12
2
2
2
)(
jjjjj
j jjjj
jjjj
jjj
jjj
j jjjjjj
jjjjjj
j jjj
jjj
III
IIII
IIII
III
III
IIIIII
IIIIII
III
IIIdivGdiv
yyyxyx
xyyxxx
yyxxy
yyxxx
xyxyxxyyy
xyyyyxxxx
yyx
yxx
H
Let’s just solve for div(G). Rest can be solved similarly:
yyxxy
xII
I
IdivIdivI
)(
![Page 24: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/24.jpg)
Relation Between T and D
Using trace property:
B
AAAB
2)(
T
tracetrace
3
1
)()(j
jij
iji QDtraceIDdiv H
Kronecker func.
2
2
2
Tijijij
Tij
TT
Tij
TT
jTi
ij
II
II
II
PPQ
G
P
Hessian
TT ffD
2
2
2
1 ),(),(
![Page 25: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/25.jpg)
Rewriting the Formula
3
1
)()(j
jij
iji QDtraceIDdiv H
)()(
)(3
1
AHtraceIDdiv
jtraceIDdivj
iji
HA
333231
232221
131211
AAA
AAA
AAA
A
3
2
1
H
H
H
H
super matrix notation
![Page 26: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/26.jpg)
Unified Expression
)(AHtracet
I
iti
ti
iti
ti
TT
traceII
traceII
H
TH
**
**
****
1
1
1
1
1
1
)(
),(1 f
),(2 f
![Page 27: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/27.jpg)
Numerical Implementation
Conventional approach Compute image Hessians and Gradients
Proposed method Use local filtering by Gaussians (2nd page)
),(
)0()()( t
iiii GIItracet
Itt
TTH
)4
exp(4
1)(
1),(
ttG t xxT
xT
Similar to bilateral filtering
![Page 28: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/28.jpg)
Numerical Implementation
1. Compute local convolution mask defining local geometry by T.
2. Estimate trace(THi) in local neighborhood of x.
3. Apply filtering for each trace(AijHj) in the vector trace(AH)
),( tG T
)1,1(),1()1,1(
)1,(),()1,(
)1,1(),1()1,1(
)1,1(),1()1,1(
)1,(),()1,(
)1,1(),1()1,1(
),()(
yxGyxGyxG
yxGyxGyxG
yxGyxGyxG
yxiyxiyxi
yxiyxiyxi
yxiyxiyxi
yxtrace iTH
![Page 29: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/29.jpg)
Comparison Between Two Implementations
![Page 30: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/30.jpg)
Noise Removal and Image impainting
![Page 31: Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications](https://reader035.vdocument.in/reader035/viewer/2022062518/568146ac550346895db3c6e9/html5/thumbnails/31.jpg)
Reconstruction, VisualizationSuperscale Images