![Page 1: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/1.jpg)
Mark S. Drew and Amin Yazdani Salekdeh
School of Computing Science,Simon Fraser University,Vancouver, BC, Canada
{mark/ayazdani}@cs.sfu.ca
Multispectral Image Invariant to Illumination Colour, Strength, and
Shading
![Page 2: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/2.jpg)
Table of ContentsIntroductionRGB Illumination InvariantMultispectral Image FormationSynthetic Multispectral ImagesMeasured Multispectral ImagesConclusion
2
![Page 3: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/3.jpg)
IntroductionInvariant Images – RGB:
Information from one pixel, with calibrationInformation from all pixels – use entropy
New Multispectral data: Information from one pixel without
calibration, but knowledge of narrowband sensors peak wavelengths
3
![Page 4: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/4.jpg)
RGB Illumination Invariant
4
Removing Shadows from Images, ECCV 2002Graham Finlayson, Steven Hordley, and Mark Drew
![Page 5: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/5.jpg)
-0.5 0 0.5 1 1.5 2 2.5 3-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
log(r/g)lo
g(b
/g)
An example, with delta function sensitivities
400 450 500 550 600 650 7000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Wavelength
Re
lati
ve
Se
ns
itiv
ity
B
W R
YG
PNarrow-band
(delta-function sensitivities)
Log-opponent chromaticities for 6 surfaces under 9 lights
RGB…
![Page 6: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/6.jpg)
Deriving the Illuminant Invariant
-0.5 0 0.5 1 1.5 2 2.5 3-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
log(r/g)
log
(b/g
)
Log-opponent chromaticities for 6 surfaces under 9 lights
This axis is invariant to illuminant colour
Rotate chromaticities
RGB…
![Page 7: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/7.jpg)
Normalized sensitivities of a SONY DXC-930 video
camera
An example with real camera data
400 450 500 550 600 650 7000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Wavelength
Re
lativ
e S
en
siti
vity
Log-opponent chromaticities for 6
surfaces under 9 different lights
RGB…
![Page 8: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/8.jpg)
Deriving the invariant
Log-opponent chromaticities
The invariant axis is now only approximately illuminant
invariant (but hopefully good enough)
Rotate chromaticities
RGB…
![Page 9: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/9.jpg)
Image FormationIllumination : motivate using theoretical
assumptions, then test in practicePlanck’s Law in Wien’s approximation:
Lambertian surface S(), shading is , intensity is I
Narrowband sensors qk(), k=1..31, qk()=(-k)
Specular: colour is same as colour of light (dielectric):
9
Multispectral
![Page 10: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/10.jpg)
To equalize confidence in 31 channels, use a geometric-mean chromaticity:
Geometric Mean Chromaticity:
with
Multispectral Image Formation …
10
{ }
![Page 11: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/11.jpg)
Multispectral Image Formation …
11
sensor-dependent
illumination-dependent
surface-dependent
So take a log to linearize in (1/T) !
![Page 12: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/12.jpg)
Logarithm:
12
Multispectral Image Formation …
known because, in special case of multispectral, *know* k !
Only sensor-unknown is ! ( spectral-channel gains)
klog
![Page 13: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/13.jpg)
If we could identify at least one specularity, we could recover log k ??
Nope, no pixel is free enough of surface colour .So (without a calibration) we won’t get log k, but
instead it will be the origin in the invariant space.Note: Effect of light intensity and shading removed:
31D 30-DNow let’s remove lighting colour too: we know 31-
vector (ek – eM) (-c2/k - c2/M)
Projection to (ek – eM) removes effect of light, 1/T : 30D 29-D
13
Multispectral Image Formation …
![Page 14: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/14.jpg)
Algorithm:
-Form 31-D chromaticity k
- Take log
- Project to (ek – eM) using projector Pe
![Page 15: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/15.jpg)
Algorithm:
What’s different from RGB? For RGB have to get “lighting-change
direction”(ek – eM) either from (i)calibration, or (ii) internal evidence (entropy) in the
image.
For multispectral, we know (ek – eM) !
![Page 16: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/16.jpg)
First, consider synthetic images, for understanding:
16
Camera: Kodak DSC 420
31 sensor gains qk()
Surfaces: 3 spheres, reflectances from Macbeth ColorChecker
Carry out all in 31-D, but show as camera would see it.
![Page 17: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/17.jpg)
Synthetic Images
17
Under red light, P2800
Under blue light, P10500
shading, for light 1, for light 2
![Page 18: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/18.jpg)
Synthetic Images
18
Spectral invariant
Original: not invariant
![Page 19: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/19.jpg)
Measured Multispectral Images
19
Under D75 Under D48
Invt. #1 Invt. #2
![Page 20: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/20.jpg)
Measured Multispectral Images
20
In-shadow, In-light
After invt. processing
![Page 21: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/21.jpg)
Measured Multispectral Images
21
![Page 22: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/22.jpg)
Measured Multispectral Images
22
![Page 23: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/23.jpg)
Measured Multispectral Images
23
![Page 24: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/24.jpg)
ConclusionA novel method for producing illumination
invariant, multispectral image Successful in removing effects of
Illuminant strength, colour, and shading
24
Next: removing shadows from remote-sensing data.
![Page 25: Multispectral Image Invariant to Illumination Colour, Strength, and Shading](https://reader033.vdocument.in/reader033/viewer/2022051418/56815849550346895dc5a043/html5/thumbnails/25.jpg)
25Multispectral Images Invariant to Illumination Colour, Strength and Shading
Thanks!
Funding: Natural Sciences and Engineering Research Council of Canada