representation and modeling of natural scenes ying nian wu ucla department of statistics
DESCRIPTION
Representation and Modeling of Natural Scenes Ying Nian Wu UCLA Department of Statistics. http://www.stat.ucla.edu/~ywu/research/. Song Chun Zhu. Stefano Soatto. Wu, Zhu, Liu, IJCV 2000; Zhu, Liu, Wu, PAMI 2000. observed image. synthesized image. Malik and Perona, late 80s. Image I. - PowerPoint PPT PresentationTRANSCRIPT
Representation and Modeling of Natural Scenes
Ying Nian WuUCLA Department of Statistics
http://www.stat.ucla.edu/~ywu/research/
Song Chun Zhu Stefano Soatto
observed image synthesized image
Wu, Zhu, Liu, IJCV 2000; Zhu, Liu, Wu, PAMI 2000
Malik and Perona, late 80s
Image I ,,, lyxB ,,, lyxB
,,,,,, , lyxlyx BIf
Histogram matching (Heeger and Bergen, mid 90s)
Filter response
Filtered image ),,( ,,,, yxfF lyxl
Histogram ,lh
Julesz ensemble
Image universe
Draw random samples from the Julesz ensemble
),,(})(:{)(
, lhhhIhIh
l
DDD
2ZD
Global statistical property
Image lattice
Zhu, Liu, Wu, PAMI 2000
0D
2ZD})(:{)( hIhIh DDD
})|(,exp{)(
1);|(0000
DDDD IIHZ
hIIp
Local statistical property
Gibbs (1902): equivalence of ensemblesExponential family model
Julesz ensemble
Markov random field
Small patch
Large lattice
Wu, Zhu, Liu, IJCV 2000
Iobs from a unknown hc Isyn ~ h with h= Isyn with h= histogram
h= histograms h= histograms h= histograms
JmJmmm BcBcBcI ...2211
Data: a collection of natural image patches },...,1,{ MmIm
Learning: basis },...,{ 1 JBB
Linear representation:
),...,( 1 mJmm ccI
Sparseness of coefficients linear bases
Olshausen & Field: Sparse coding
Mallat and Zhang: matching pursuitCandes and Donoho: curvelets
mjJ
j mjm
mj
BcI
indepcpc
1
),(~
),0()1()( 20 Ncp
Two-Level Generative Model
Mixture prior for sparseness
Bell & Sejnowski (96)Lewiki & Olshausen (99)Olshausen & Millman (00)Pece (01) George & McCulloch (95)
i
lyxi iiiiBcI
S
,,,
~
Wu, Zhu, Guo, ECCV 2002.
Model fitting (EM-type iteration)Estimate S based on I and Sketch Model (MCMC)
Fit Sketch Model on S
Sketch Model
},...,1),,,,,({ nilcyxsS iiiiii
SimplificationEstimate S from I using matching pursuit (Mallat & Zhang)
Fit Sketch Model on S (ignoring c and e)
Math representations of sketch
},...,1),,,,({ nilyxsS iiiii List:
)},,({ ,,,, yxyxyxyx lsS Bit-map:
Causal model for sketch
)|()( ),(, yxNy
yxx
SspSp
)}|,(),(exp{)|(
1)|( ,,2,,1,0),(
),(,),(
iyxyxSsyxyxyxyxN
yxNyx sllSZ
SspyxNi
Pairwise interactions
Soatto,Doretto,Wu, ICCV 2001
Modeling dynamic scenesData:
Model: time series
Representation: principal components (Fourier bases)
Autoregressive model
},...,,{ 21 TIII
ttt
ttt
CWIqAWW
1
Fourier’s solution to heat equation
Soatto,Doretto,Wu, ICCV 2001
World W = (W_high, W_low)
Image I
Knowledge K
Why generative modeling?Representing knowledgeUnsupervised learning of causesModel selection as explaining awayModel checking by synthesis
Physics model and image-based rendering
P(W; K)P(I | W; K) P(W | I; K)