light, surface and feature in color images
DESCRIPTION
Light, Surface and Feature in Color Images. Lilong Shi Postdoc at Caltech Computational Vision Lab, Simon Fraser University. Topics. Color Constancy Surface Reflectance Model Feature Analysis. Light Surface Feature. Color Formation. sensor sensitivity. camera response. - PowerPoint PPT PresentationTRANSCRIPT
Light, Surface and Feature
in Color Images
Lilong ShiPostdoc at Caltech
Computational Vision Lab, Simon Fraser University
Topics
Color Constancy
Surface Reflectance Model
Feature Analysis
Color Formation
reflectance spectral
Illum. power distribution BGRkdSEp kk ,,)()()(
camera response
sensor sensitivity
Light
Surface
Feature
Color Constancy
Light
Surface
Feature
Automatic White Balance
AWB
Light
Surface
Feature
Canonical
Color Constancy Methods
Retinex Theory (McCann64)
MaxRGB/White-Patch (Land77): max(R)
Gray-World (Buchsbaum80): mean(R)
Shades-of-Gray (Finlayson04): [mean(Rp)]1/p
Gray-Edge Hypothesis (Weijer07): mean(edge(R))
Non-Negative Matrix Factorization (Shi07)
Light
Surface
Feature
=
Color Constancy Methods
Gamut Mapping (Forsyth90)
Color by Correlation (Finlayson01)
Neural Network (Cardei02)
Support Vector Regression (Xiong06)
Thin Plate Spline (Shi11)
Light
Surface
Feature
Color Constancy Methods
Classification-based (Bianco09)
Scene-based (Gijsenij11)
Light
Surface
Feature
Color Constancy Evaluation
MethodInpu
tTrain speed Para.
Relative Performance
Assumptions
Max-RGB img no very fast none poor white surface
Gray-World img no very fast none poor average gray
Shades-of-Gray img no moderate one moderate/good average gray
Edge-based Hyp. img no fast none moderate average gray
Color-by-Corre. hist yes fast a few moderate candidate illums
Neural-Network hist yes moderate some good none
Sup. Vector Reg. hist yes dep. train some moderate/good none
Thin-Plate-Spline thum yes dep. train a few good none
Light
Surface
Feature
Blackbody Radiator Lights
Tungsten lamps, sunrise/sunset, sky light
Planckian locus
Narrowband sensors
Light
Surface
Feature
T
c
eIcTE2
51),(
2
51( ) ( ) ( ) , { , , }i
c
Ti i i i ip S E S Ic e i R G B
Surface Reflectance Model
LIS Coordinate (Finlayson 01)
Light
Surface
Feature
Achromatic Surface
Detection in LIS
Light
Surface
Feature
GraySurface
Skin Color Model
Skin: melanin + hemoglobin
Skin Reflectance (Hiraoka et al 93)
Under blackbody illumination
Light
Surface
Feature
)]()()()()(exp[)( hhhmmm llS
pigment density
absorbance length in epidermis/dermi
s
absorbance of other material
c1ωσσΠ
bτb
SEp
hhmmhm ),,,(
Skin Color Locus
Linear model
m is melanin basis,
h is hemoglobin basis,
is blackbody radiator basis,
c is a constant vector
Light
Surface
Feature
Skin Tone Correction
Even simpler model:
Tone correction
Preserve melanin
16 different illum + camera
calibrations
( , )m m m τ Π σ ω
Light
Surface
Feature
Features
Textures, edges, corner, blobs, etc..
Colors
Integrated by Quaternion
Light
Surface
Feature
Quaternion
Real, complex, quaternion (q = a + bi + cj + dk)
Non-commutative (pq ≠ qp)
Convolution, Correlation, Fourier, Wavelet, etc
SVD, EVD, PCA
Light
Surface
Feature
Texture Feature Extraction
Light
Surface
Feature
QPCA
Image-specific quaternion texture
basisSampled sub-windows
Texture Feature Extraction
Light
Surface
Feature
Single quaternion
A texture patch
1st QPCA Basisma
gn
itu
de
re
al la
ye
rre
d la
ye
rg
re
en
la
ye
rb
lu
e la
ye
r
T
Texture Feature 1st Feature
Light
Surface
Feature
magnitude
real layer i layer j layer k layer
Segmentation
Quaternion Hoang(05)
Light
Surface
Feature
Segmentation
Light
Surface
Feature
Color Curvature
Light
Surface
Feature
Iso-luminance
Color -> Gray
Cancellation in combining +/- derivatives
Light
Surface
Feature
Hessian Descriptor
2nd order local shape
Principle Curvature
Light
Surface
Feature
2
22
2
2
2
),(
y
I
xy
Iyx
I
x
I
yxH eigenvectors: (e1, e2 )
eigenvalues: |1|<|2|
e1
e2
1
λ2
e1
e2 λ2
1
Curvature
Tubular, vessel-like structures [Frangi98]
With eigen-values blobness: backgroundness: vesselness: R and S
Gray image, 2 λ’s; RGB image, 6λ’s
Light
Surface
Feature
Color Curvature
Quaternion-valued Hessian
QSVD on H 2 real singular values
Light
Surface
Feature
k
y
B
xy
Byx
B
x
B
j
y
G
xy
Gyx
G
x
G
i
y
R
xy
Ryx
R
x
R
HQ
2
22
2
2
2
2
22
2
2
2
2
22
2
2
2
QT
Q UVHQ
Curvature Detection
Light
Surface
Feature
Frangi Quaternion
Future Works
Content-based color constancy
Color blob/points detection
Possibilities …
Light
Surface
Feature