![Page 1: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/1.jpg)
ter Haar Romeny, FEV
Nuclei of fungus cell Paramecium Caudatum
Spatial gradient Illumination spectrum-invariant gradient
Color RGB original
Color-Scale Differential Structure
Geusebroek et al, LNCS 1852, 459-464, 1999
![Page 2: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/2.jpg)
ter Haar Romeny, FEV
The color of an object depends on
• color of the illuminating light• illumination intensity• sensor sensitivity• direction of surface normal• surface reflectance properties
Assumptions:• Scene is uniformly illuminated• light source is colored• surface has Lambertian reflectance
![Page 3: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/3.jpg)
ter Haar Romeny, FEV
What causes color ?
Lamp
objectcolor
spectral
color
![Page 4: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/4.jpg)
ter Haar Romeny, FEV
400 450 500 550 600 650 700
500 1000 1500 2000
50000
100000
150000
200000
250000
300000
350000
400 450 500 550 600 650 700 400 450 500 550 600 650 700 400 450 500 550 600 650 700
Object reflectance function for the observed spectrum for a resp. 2500K, 6500K and 10,000K light source:
Spectrum reflected froman arbitrary object
8 hc 5E hc
kT 11
Emissionspectrum ofblack bodyradiator
![Page 5: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/5.jpg)
ter Haar Romeny, FEV
Color receptive fields
x
y
x
![Page 6: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/6.jpg)
ter Haar Romeny, FEV
Self-organization:receptive fieldsfrom Eigenpatches(12x12 pixels)
![Page 7: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/7.jpg)
ter Haar Romeny, FEV
Colour receptivefields fromEigenpatches
x
![Page 8: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/8.jpg)
ter Haar Romeny, FEV
Hering basis
0 10 20 30 40 50 60 70 80 90-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
Idea Koenderink: Gaussian derivatives of zero, first and second order in the wavelength domain
wavelength
RFsensitivity
How can wemeasure color?
![Page 9: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/9.jpg)
ter Haar Romeny, FEV
Taylor color model
Luminance
Blue-yellowness
Purple-greenness
300 400 500 600 700 800 90000.10.20.30.40.50.60.70.80.9
LM
S
Cone sensitivity
![Page 10: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/10.jpg)
ter Haar Romeny, FEV
Spatial color
Color scale-space starts by probing this space.
y
x
xy
s
Energy densities cannot be measured at a point, …… one probes a certain volume
![Page 11: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/11.jpg)
ter Haar Romeny, FEV
Reflectance of light
Lamp
objectcolor
spectral
color
What are invariantproperties?
![Page 12: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/12.jpg)
ter Haar Romeny, FEV
Reflectance model
![Page 13: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/13.jpg)
ter Haar Romeny, FEV
Transparent materials
![Page 14: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/14.jpg)
ter Haar Romeny, FEV
)()),,(1()()( 2 RvsneE f
The reflected spectrum is:
v = viewing directionn = surface patch normals = direction of illuminationf = Fresnel front surface reflectance coefficient in vR = body reflectance
![Page 15: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/15.jpg)
ter Haar Romeny, FEV
),())(1)(,(),( 2 xRxxexE f
Because of projection of the energy distribution on the image plane the vectors n, s and v will depend on the position at the imaging plane. So the energy at a point x is then related to:
We assume an illumination with a locally constant color:
),())(1)(()(),( 2 xRxxiexE f
![Page 16: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/16.jpg)
ter Haar Romeny, FEV
Aim: describe material changes independentof the illumination.
Rxxie
exRxxi
E
f
f
2
2
))(1)(()(
),())(1)((
),())(1)(()(),( 2 xRxxiexE f
Bothequationshavemanycommonterms
![Page 17: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/17.jpg)
ter Haar Romeny, FEV
R
xR
e
e
E
EE
),(
1
)(
11ˆ
The normalized differential
determines material changes independent of the viewpoint, surface orientation, illumination direction, illumination intensity and illumination color!
![Page 18: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/18.jpg)
ter Haar Romeny, FEV
The derivative jet to x and forms a complete family of geometric invariants:
0
ˆˆ ( ; 515 ; 55 )
n
n
EE G nm nm
These are observed properties, so we convolve with Gaussian derivatives
ˆn m
n m
E
x
![Page 19: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/19.jpg)
ter Haar Romeny, FEV
2 2ˆ ˆx yE E
Color edges can be defined as the thresholding of the spatial gradient (color-invariant equivalent of Lw):
2 2ˆ ˆx yE E
Color invariants
![Page 20: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/20.jpg)
ter Haar Romeny, FEV
Spatial color model and tracing color edges in microscopy
Influence of illumination color temperature on edge strength, scale is 3.0 px.
Skin tissue section illuminated by a halogen bulb at 4000 K (top) and 2600 K (bottom) color temperature.
![Page 21: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/21.jpg)
ter Haar Romeny, FEV
1
ee
x. ex, y, eshortnotatione exzx, y, exx, y, ezx, y,
e2
x,. ex, y, eshortnotation1
e3e2 exzzx, y, 2 e exzx, y, ezx, y, exx, y, 2 ezx, y, 2 e ezzx, y,
Color-invariant multi-scale structural operators
![Page 22: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/22.jpg)
ter Haar Romeny, FEV
Simplifyx2 y2 x, 2 y, 2;shortnotation 1
ex, y, 6 ex, y, 2ex, y, exzx, y, exx, y, ezx, y, 2
ex, y, 2ex, y, eyzx, y, eyx, y, ezx, y, 2 ex, y, 2 exzzx, y, 2 exx, y, ezx, y, 2 ex, y, 2 exzx, y, ezx, y, exx, y, ezzx, y, 2 ex, y, 2 eyzzx, y, 2 eyx, y, ezx, y, 2 ex, y, 2 eyzx, y, ezx, y, eyx, y, ezzx, y, 2
Total edge strength
![Page 23: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/23.jpg)
ter Haar Romeny, FEV
[im,] 1e
e
color invariant 1e
e
[im,]
first wavelength derivative of
[im,] 22 second wavelength derivative of
g[im,] x2 y2 yellow-blue edges
g [im,] 2x2 2
y
2 red-green edges
g[im,] x2 y2
2x2 2
y
2 total color edge strength
Somecolor
differentialinvariants
![Page 24: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/24.jpg)
ter Haar Romeny, FEV
Feulgen stain,red-green edges
Paramecium caudatum, Feulgen and Fast green stain
Color canny, red-green normalized edges, scale 3
![Page 25: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/25.jpg)
ter Haar Romeny, FEV
Hematoxylin eosin stain
Pituitary gland, sheep, adenohypophysis 40x
Cell: E<0, E > 0, scale 1.0
Nuclei: E <0, E > 0, E +E < 0, scale 3.0
additional constraint added to refine selection
![Page 26: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/26.jpg)
ter Haar Romeny, FEV
Safranin O stain
E > 0, E > 0, scale sigma 1.0
Safranin O stain for proteoglycans (mouse knee
joint)
Courtesy of Koen Gijbels and Paul Stoppie
![Page 27: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/27.jpg)
ter Haar Romeny, FEV
Oil red O stain
Oil red O stain of fat emboli in
lung
E > 0, E > 0, scale 1.5
![Page 28: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/28.jpg)
ter Haar Romeny, FEV
PAS stain
Lww > 0, Lvv Lww-Lvw2 > 0, E -E > 0, scale sigma 2.0
P.A.S. stain for carbohydrates (goblet cells, gut)
carbohydrates stain magenta - elliptic patches
![Page 29: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/29.jpg)
ter Haar Romeny, FEV
Blood smear
Blood smear, Giemsa stain, 100x, JPEG
compression
RBC: E > 0, E +E > 0, scale 0.5
Leucocytes: E < 0, scale 12
Leucocyte nuclei: E < 0, E > 0, scale
3
![Page 30: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/30.jpg)
ter Haar Romeny, FEV
Blue-yellow edges
Note the complete absence of detection of black-white edges.
![Page 31: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/31.jpg)
ter Haar Romeny, FEV
Color edges can also be defined as the zero-crossings of the second order derivative in the spatial gradient direction (color-invariant equivalent of Lww):
0ˆˆ
ˆˆˆˆˆ2ˆˆˆ
22
22
yx
xxxxyxyyyyww
EE
EEEEEEEE
Second order color invariants
![Page 32: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/32.jpg)
ter Haar Romeny, FEV
Color invariant edge detection
Luminance gradientedge detection
![Page 33: Ter Haar Romeny, FEV Nuclei of fungus cell Paramecium Caudatum Spatial gradient Illumination spectrum -invariant gradient Color RGB original Color-Scale](https://reader035.vdocument.in/reader035/viewer/2022081602/551a730f550346b52d8b5032/html5/thumbnails/33.jpg)
ter Haar Romeny, FEV
Conclusions
• Color ‘scale-space’ compatible with classical luminance
scale-space
• The model enables the design of practical image analysis
‘color reasoning’ solutions, e.g. invariance for illumination
• The color-scale invariant differential operators are building
blocks for a differential geometry on color images