![Page 1: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/1.jpg)
1
Formation et Analyse d’ImagesSession 2
Daniela Hall
7 October 2004
![Page 2: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/2.jpg)
2
Course Overview
• Session 1: – Homogenous coordinates and tensor notation– Image transformations– Camera models
• Session 2:– Camera models– Reflection models– Color spaces
• Session 3:– Review color spaces– Pixel based image analysis– Gaussian filter operators
• Session 4:– Scale Space
![Page 3: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/3.jpg)
3
Course overview
• Session 5:– Contrast description– Hough transform
• Session 6:– Kalman filter– Tracking of regions, pixels, and lines
• Session 7:– Stereo vision – Epipolar geometry
• Session 8: exam
![Page 4: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/4.jpg)
4
Session Overview
1. Camera model
2. Light
3. Reflection models
4. Color spaces
![Page 5: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/5.jpg)
5
Camera model
• Projective model
Ti
Trr
r
Tccc
c
Tsss
s
jiP
yxP
zyxP
zyxP
)1,,(
)1,,(
)1,,,(
)1,,,(
Scene coordinates
Camera coordinates
Image coordinates
![Page 6: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/6.jpg)
6
Camera model
• Transformation from scene to camera coordinates
• Projection of camera coordinates to retina coordinates
• Transformation from retina coordinates to image coordinates
• Composition (camera model) sis
scs
rc
ir
i
rir
i
crc
r
scs
c
PMPTMCP
wPCP
PMP
PTP
The camera model is the composition of the transformations thattransform Ps to Pi
![Page 7: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/7.jpg)
7
Transformation Scene - Camera
• (xs,ys,zs) is position of the origin of the camera system with respect to the scene coordinates (translation).
• R is the orientation of the camera system with respect to the scene system (3d rotation).
1000s
s
scsc
sz
y
x
RT
![Page 8: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/8.jpg)
8
3d rotation
• Around x-axis (counter-clockwise)
• Around y-axis
• Around z-axis
• General )()()(,
1000
1000
0100
00)cos()sin(
00)sin()cos(
)(
1000
0)cos(0)sin(
0010
0)sin(0)cos(
)(
1000
0)cos()sin(0
0)sin()cos(0
0001
)(
aRbRcRRtz
ty
tx
RT
cc
cc
cR
bb
bb
bR
aa
aaaR
xyz
z
y
x
![Page 9: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/9.jpg)
9
Projection Camera-Retina
• Imagine a 1D camera in a 2D space.
• The transformation MRc can be found by considering
similar triangles
z(xc ,zc )
F
x
xr
O
)()(
)()(
c
cr
c
cr
c
cr
c
cr
zF
Fyy
zF
y
F
y
zF
Fxx
zF
x
F
x
![Page 10: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/10.jpg)
10
Projection Camera-Retina
1
0
0
/1
0
0
0
1
0
0
0
1
1
,
F
M
z
y
x
M
w
wy
wx
PMP
RC
c
c
c
RCr
rRR
CR
![Page 11: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/11.jpg)
11
Transformation Retina-Image• A frame: the image is composed of pixels (picture
elements)
• Pixels are in general not squared. There physical sizes depends on the used material.
i columns
j rows
(0,0)
(i-1,j-1)
![Page 12: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/12.jpg)
12
Intrinsic camera parameters
• F: focal distance
• Ci , Cj : Optical image center (in pixels)
• Di , Dj : Physical size of the pixel on the retina (in pixel/mm)
• i, j : image coordinates (in pixels)• Transformation Retina-Image
jjr
iir
CDyj
CDxi
1100
0
0
1r
r
jj
ii
y
x
CD
CD
j
i
![Page 13: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/13.jpg)
13
Camera modelSI
SSC
SRC
IR
I PMPTMCP
1S
S
S
IS z
y
x
M
w
wj
wi
SS
IS
SS
IS
SS
IS
SS
IS
PM
PM
mzsmysmxsm
mzsmysmxsm
w
wjj
PM
PM
mzsmysmxsm
mzsmysmxsm
w
wii
3
2
3
1
)(
)(
34333231
24232221
)(
)(
34333231
14131211
Equation:
Image coordinates
![Page 14: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/14.jpg)
14
Transformation image-scene
• Problem: we need to know depth zs for each image position.
• This process of finding MSI is called
calibration.
• MSI has 12 coefficients.
• MSI is homogenous. 11 degrees of freedom.
ISI
IRI
CR
SC
S PMPCMTP
![Page 15: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/15.jpg)
15
Calibration
1. Construct a calibration object whose 3D position is known.
2. Measure image coordinates
3. Determine correspondences between 3D point RS
k and image point PIk.
4. We have 11 DoF. We need at least 5 ½ correspondences.
![Page 16: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/16.jpg)
16
Calibration
• For each correspondence scene point RSk and image
point PIk
• which gives following equations for k=1, ..., 6
• from wich MIS can be computed
Skh
IS
Skh
IS
k
kkk RM
RM
w
iwi
3
1
)(
)(
Skh
IS
Skh
IS
k
kkk RM
RM
w
jwj
3
2
)(
)(
0))(())((
0))(())((32
31
Skh
ISk
Skh
IS
Skh
ISk
Skh
IS
RMjRM
RMiRM
![Page 17: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/17.jpg)
17
Calibration using many points
• For k=5 ½ M has one solution.– Solution depends on precise measurements of
3D and 2D points. – If you use another 5 ½ points you will get a
different solution.
• A more stable solution is found by using large number of points and do optimisation.
![Page 18: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/18.jpg)
18
Calibration using many points
• For each point correspondence we know (i,j) and R.
• We want to know MIS.
Solve equation with your favorite algorithm (least squares, levenberg-marquart, svd,...)
0))(())((
0))(())((32
31
Skh
ISk
Skh
IS
Skh
ISk
Skh
IS
RMjRM
RMiRM
0
34
33
32
31
24
23
22
21
14
13
12
11
10000
00001
0000000000
0000000000
m
m
m
m
m
m
m
m
m
m
m
m
jzjyjxjzyx
iziyixizyx
![Page 19: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/19.jpg)
19
Application: Rectifying images
![Page 20: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/20.jpg)
20
Applications
![Page 21: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/21.jpg)
21
Homographie: projection from one plane to another
• Homographie HBA is bijective QB = HB
A PA
jj
jjB
jj
jjB
A
j
j
jAB
AB
B
PHPHy
PHPHx
P
H
H
H
PH
w
wy
wx
32
31
3
2
1
/
/
![Page 22: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/22.jpg)
22
Homography computation
• H can be computed from 4 point correspondences.
Ps1 Ps2
Ps3Ps4
Rd1 Rd2
Rd3Rd4Source image (observed)
Destination image (rectified)
4,3,2,1 and 2,1 with ,0 iSRHPRHP DiSD
SiDiSD
Si
![Page 23: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/23.jpg)
23
Homography computation
• H is 3x3 matrix and has 8 degrees of freedom (homogenous coordinates)
• gives 8 equations and one solution for H.
4,3,2,1 and 2,1 with ,0 iSRHPRHP DiSD
SiDiSD
Si
![Page 24: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/24.jpg)
24
Session Overview
1. Camera model
2. Light
3. Reflection models
4. Color spaces
![Page 25: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/25.jpg)
25
Light
• N: surface normal• i angle between incoming light and normal• e angle between normal and camera• g angle between light and camera
camera
light
N
egi
![Page 26: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/26.jpg)
26
Spectrum
• Light source is characterised by its spectrum. • The spectrum consists of a particular quantity of photons
per frequency. • The frequency is described by its wavelength• The visible spectrum is 380nm to 720nm• Cameras can see a larger spectrum depending on their
CCD chip
f1
![Page 27: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/27.jpg)
27
• Albedo is the fraction of light that is reflected by a body or surface.
• Reflectance function:
Albedo
light received
light emitted
Irradiance
Radiance),,( geiR
camera
light
N
egi
![Page 28: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/28.jpg)
28
Session Overview
1. Camera model
2. Light
3. Reflection models
4. Color spaces
![Page 29: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/29.jpg)
29
Reflectance functions
• Specular reflection– example mirror
• Lambertian reflection– diffuse reflection, example paper, snow
![Page 30: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/30.jpg)
30
Specular reflection
else 0,
gei and ei if ,1),,( geiR
camera
lightN
egi
![Page 31: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/31.jpg)
31
Lambertian reflection
)cos(),,( igeiR
![Page 32: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/32.jpg)
32
Di-chromatic reflectance model
• the reflected light R is the sum of the light reflected at the surface Rs and the light reflected from the material body RL
• Rs has the same spectrum as the light source• The spectrum of Rl is « filtered » by the material (photons are
absorbed, this changes the emitted light)• Luminance depends on surface orientation• Spectrum of chrominance is composed of light source
spectrum and absorption of surface material.
),,(),,(),,( geiRgeiRgeiR LLSs
![Page 33: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/33.jpg)
33
Color perception
• The retina is composed of rods and cones.• Rods - provide "scotopic" or low intensity vision.
– Provide our night vision ability for very low illumination, – Are a thousand times more sensitive to light than cones, – Are much slower to respond to light than cones, – Are distributed primarily in the periphery of the visual field.
![Page 34: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/34.jpg)
34
Color perception
• Cones - provide "photopic" or high acuity vision. – Provide our day vision, – Produce high resolution images, – Determine overall brightness or darkness of images, – Provide our color vision, by means of three types of cones:
• "L" or red, long wavelength sensitive, • "M" or green, medium wavelength sensitive, • "S" or blue, short wavelength sensitive.
• Cones enable our day vision and color vision. Rods take over in low illumination. However, rods cannot detect color which is why at night we see in shades of gray.
• source: http://www.hf.faa.gov/Webtraining/VisualDisplays/
![Page 35: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/35.jpg)
35
Color perception
• Rod Sensitivity- Peak at 498 nm.• Cone Sensitivity- Red or "L" cones peak at 564 nm. - Green or "M"
cones peak at 533 nm. - Blue or "S" cones peak at 437 nm.
• This diagram shows the wavelength sensitivities of the different cones and the rods. Note the overlap in sensitivity between the green and red cones.
![Page 36: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/36.jpg)
36
Camera sensitivity
• observed light intensity depends on:– source spectrum: S(λ)– reflectance of the observed point (i,j): P(i,j,λ)– receptive spectrum of the camera: c(λ)– p0 is the gain
400 600 800 1000 nm
S(λ)
λ
vidicon
CCD
0
0 )()(),,(),( cSjiPpjip
![Page 37: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/37.jpg)
37
Classical RGB camera
• The filters follow a convention of the International Illumination Commission.
• They are functions of λ: r(λ), g(λ), b(λ)• They are close to the sensitivity of the human
color vision system.
![Page 38: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/38.jpg)
38
Color pixels
0
0
0
)()(),,(0),(
)()(),,(0),(
)()(),,(0),(
),(
),(
),(
),(
dbSjiPbjiB
dgSjiPgjiG
drSjiPrjiR
jiB
jiG
jiR
jiP
![Page 39: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/39.jpg)
39
Color bands (channels)
• It is not possible to perceive the spectrum directly.
• Color is a projection of the spectrum to the spectrum of the sensors.
0
0
0
)()(0
)()(0
)()(0
dbSbB
dgSgG
drSrR
![Page 40: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/40.jpg)
40
Session Overview
1. Camera model
2. Light
3. Reflection models
4. Color spaces
![Page 41: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/41.jpg)
41
Color spaces
• RGB color space
• CMY color space
• YIQ color space
• HLS color space
![Page 42: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/42.jpg)
42
RGB color space
• A CCD camera provides RGB images
• The luminance axis is r=g=b (diagonal)
![Page 43: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/43.jpg)
43
CMY color space
• Cyan, magenta, yellow
• CMYK: CMY + black color channel
B
G
R
B
G
R
Y
M
C
max
max
max
![Page 44: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/44.jpg)
44
YIQ color space
• This is an approximation of– Y: luminance, – I: red – cyan, – Q: magenta - green
• Used US TVs (NTSC coding). Black and white TVs display only Y channel.
B
G
R
Q
I
Y
31.052.021.0
32.028.06.0
11.059.03.0
![Page 45: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/45.jpg)
45
HLS space
• Hue, luminance, saturation space.
• L=R+G+B
• S=1-3*min(R,G,B)/L
elsex
gbxT
BGBRGR
BRGRx
,2
if ,
)))(()(
))()((5.0(cos
2
1
L
S T
![Page 46: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/46.jpg)
46
Color distribution
• Color distribution can be studied by histograms.• A histogram is a multi-dimensional table.• We define a function from the continuous color
space to the discrete histogram space.• Then each pixel of the image increments a cell in
the histogram.• Example: We define a histogram of RGB (3D)
with 32 cells/dimension. The pixel value (210,180,100) increments cell (6,5,3)
![Page 47: 1 Formation et Analyse d’Images Session 2 Daniela Hall 7 October 2004](https://reader030.vdocument.in/reader030/viewer/2022032607/56649ed45503460f94be433c/html5/thumbnails/47.jpg)
47
Colors of a surface
• A reflection has the color of the light source (Rs)• Which color is near the border of the reflection?
– Rs and Rb are mixed.
– A color histogram can be used to study this mix.
– The histogram should contain two axis (in theory).
– But reflectance in the real world is more complex than only Rs and Rb. You also have inter reflectance between neighboring objects.