1 formation et analyse d’images daniela hall 19 septembre 2005 [email protected]
TRANSCRIPT
2
Course material
• course slides (.pdf)– http://www-prima.inrialpes.fr/perso/Hall/Courses/FAI0
5/• References on the slides• last years documents
– http://www-www-prima.inrialpes.fr/perso/Hall/Courses/FAI04/
• CVonline:– http://homepages.inf.ed.ac.uk/rbf/CVonline/
• Contact: email [email protected]
3
Course Overview
• Session 1: – Overview– Human vision – Homogenous coordinates– Camera models
• Session 2:– Tensor notation– Image transformations
• Session 3:– Reflection models– Color spaces
• Session 4:– Pixel based image analysis
• Session 5:– Gaussian filter operators– Scale Space
4
Course overview
• Session 6:– Contrast description– Hough transform
• Session 7:– Kalman filter
• Session 8:– Tracking of regions, pixels, and lines
• Session 9:– Stereo vision
• Session 10:– Epipolar geometry
• Session 11: exercises and questions
5
Session Overview
1. Course overview2. Image formation in the eye and the camera
1. The eye2. The retina3. Capacities of the eye
3. Homogenous coordinates4. Camera models
1. Pinhole camera2. Perspective projection3. Mathematical formulation
6
Image formation in the eye
• Biological vision: process of using light reflected from surrounding world for modifying behaviour.
• Humans require conscious understanding of the 3d world from 2d projections on the retina.
• Surrounding environment is interpreted by visual input.
Ref: CVonline/LOCAL_COPIES/OWENS/LECT1/
7
The eye
aqueous humor
8
The eye
• Light enters the eye through the transparent cornea, passes through the aqueous humor, the lens and the vitreous humor, where it forms the image on the retina.
• Accomodation: muscular adjustment of the lens that focuses the image directly on the retina.
9
The retina
• Complex tiling of photoreceptors (rods and cones).
• Photoreceptors stimulated by light transmit electrical signals to the brain via the optic nerve.
• Location of the optic nerve on the retina has no photoreceptors. No light is perceived within this region (blind spot).
10
The retina
• Rods and cones communicate through several layers of cells to the ganglion cells
• Synapses: junctions between layers
• Rods and cones are situated at the back of the retina.
• Light passes through the different layers and the signal is transmitted back via synaptic junctions to the optic nerve.
11
The retina
• Ganglion cell responds to the photostimulus according to a receptive field.
• The spatial organization of the receptive field on the retinal ganglion cell is circular symmetric (either excitatory center and inhibitory suround or inverse). Such cells are known as mexican hat operators
• Other spatial organization are – simple cells (orientation sensitive receptive fields),– complex cells (non-linear combination of even and odd responses)
and – end-stopped cells (simple differentiation operators).
12
Capacities of the eye
• Sensitivity and performance approach the limits set by quantum physics.
• Eye can detect a single photon• Eye adapts to ranges in light of many orders of magnitude• No camera can even partially match this performance• Inputs from left and right are processed in the optic
chiasma.• The slight difference of viewpoint of the eyes is used to
deduce depth. • Nerve fibers lead from the optic chiasma to the striate
cortex (the seat of visual processing in the brain).
13
Session Overview
1. Course overview2. Image formation in the eye and the camera
1. The eye2. The retina3. Capacities of the eye
3. Camera models1. Pinhole camera2. Homogenous coordinates3. Perspective projection4. Mathematical formulation
14
Camera model
• Physical geometry
The pinhole camera is the simplest. It has a infinitesimally smallhole through which light enters and forming an inverted image on the camera surface (retina). To simplify things, we model a pinhole camera by placing the retina between the camera center and the object (projective model)
15
Camera model
• Projective model
Ti
Trr
r
Tccc
c
Tsss
s
jiP
yxP
zyxP
zyxP
)1,,(
)1,,(
)1,,,(
)1,,,(
Scene coordinates
Camera coordinates
Image coordinates
16
Change of coordinate systems
• Transformation from scene to camera coordinates
• Projection of camera coordinates to retina coordinates
• Transformation from retina coordinates to image coordinates
• Composition
• This mapping from 3 dimensions to 2 is called perspective projection
sis
scs
rc
ir
i
rir
i
crc
r
scs
c
PMPTMCP
wPCP
PMP
PTP
17
Camera parameters
• 4 intrinsic parameters:– 2 for the origin of the image coordinate frame– 2 for the scale of the axes– Focal length F
• 6 extrinsic parameters:– 3 for the 3D position of the center of projection– 3 for the orientation of the image plane
18
Intrinsic camera parameters
• F: focal distance
• Ci , Cj : Optical image center (in pixels)
• Di , Dj : Physical size of the pixel on the retina (in pixel/mm)
• i, j : image coordinates (in pixels)• Transformation Retina-Image
jjr
iir
CDyj
CDxi
1100
0
0
1r
r
jj
ii
y
x
CD
CD
j
i
19
Homogenous coordinates
• Allow to manipulate n-dim vectors in a n+1-dim space
• A point p can be written as vector • In homogenous coordinates we add a scaling factor
• To transform the homogenous coordinates in normal coordinate, divide by the n+1 coordinate w.
y
xp
1
y
x
w
wy
wx
ph
20
Homogenous coordinates
• we note
• Proof:
11
:, y
x
by
x
aba
1/
/
/
1
y
x
aa
aay
aax
a
ay
ax
y
x
a
21
Translation
• Classic • Homogenous coordinates
y
x
tyy
txx
12
12
1
1
1
100
10
01
1
2
2
y
x
ty
tx
y
x
22
Scaling
• Classic • Homogenous coordinates
12
12
ryy
sxx
1
1
1
100
00
00
1
2
2
y
x
r
s
y
x
x
y
ppl
5.7
6
55.1
32,
5
3 lpp
23
Rotation (clockwise)
• Classic • Homogenous coordinates
1)cos(1)sin(
1)sin(1)cos(
2
2
yaxay
yaxax
1
1
1
100
0)cos()sin(
0)sin()cos(
1
2
2
y
x
aa
aa
y
x
x
y
p
pl
5
7.8
087.0105.0
05.01087.0,
0
1030lpp
24
Translation and rotation
• Classic • Homogenous coordinates
tyyaxay
txyaxax
1)cos(1)sin(
1)sin(1)cos(
2
2
1
1
1
100
)cos()sin(
)sin()cos(
1
2
2
y
x
tyaa
txaa
y
x
25
Translation, rotation and scaling
• Classic • Homogenous coordinates
tyyarxasy
txyarxasx
1)cos(1)sin(
1)sin(1)cos(
2
2
1
1
1
100
)cos()sin(
)sin()cos(
1
2
2
y
x
tyaras
txaras
y
x
26
3d rotation
• Around x-axis (counter-clockwise)
• Around y-axis
• Around z-axis
• General )()()(,
1000
1000
0100
00)cos()sin(
00)sin()cos(
)(
1000
0)cos(0)sin(
0010
0)sin(0)cos(
)(
1000
0)cos()sin(0
0)sin()cos(0
0001
)(
aRbRcRRtz
ty
tx
RT
cc
cc
cR
bb
bb
bR
aa
aaaR
xyz
z
y
x
27
Transformation Scene - Camera
• (xs,ys,zs) is position of the origin of the camera system with respect to the scene coordinates (translation).
• R is the orientation of the camera system with respect to the scene system (3d rotation).
1000s
s
scsc
sz
y
x
RT
28
Transformation Camera-Retina
• Imagine a 1D camera in a 2D space.
• The transformation MRc can be found by considering
similar triangles
z(xc ,zc )
F
x
xr
O
)()(
)()(
c
cr
c
cr
c
cr
c
cr
zF
Fyy
zF
y
F
y
zF
Fxx
zF
x
F
x
29
Transformation Camera-Retina
1
0
0
/1
0
0
0
1
0
0
0
1
1
,
F
M
z
y
x
M
w
wy
wx
PMP
RC
c
c
c
RCr
rRR
CR
30
Transformation Retina-Image• A frame: the image is composed of pixels (picture
elements)
• Pixels are in general not squared. There physical sizes depends on the used material.
i columns
j rows
(0,0)
(i-1,j-1)
31
Exercise
• Soit une camera avec un axe optique a la position (Ci,Cj) [pixels] et une taille de pixel de Di [mm/ligne], Dj [mm/colonne]. L'horloge du numerisateur est mal regle: chaque ligne d l'image est decalee vers la droite par k pixels par ligne.
• Ecrivez la matrice de projection, Cir de l'image
vers la retine en coordonees homogenes.