lecture 2 camera calibration
TRANSCRIPT
![Page 1: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/1.jpg)
1
Lecture 2: Camera Calibration
Lecture 2Camera Calibration
Joaquim SalviUniversitat de Girona
Visual Perception
![Page 2: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/2.jpg)
2
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 3: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/3.jpg)
3
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 4: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/4.jpg)
4
Lecture 2: Camera Calibration
– Dense reconstruction – Visual inspection
– Object localization – Camera localization
2.1 Calibration Introduction
• Some applications of this capability include
![Page 5: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/5.jpg)
5
Lecture 2: Camera Calibration
Image courtesy of C. Taylor
“The Scholar of Athens,” Raphael, 1518
2.1 Calibration Introduction – Perspective Imaging
![Page 6: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/6.jpg)
6
Lecture 2: Camera Calibration
WZ
WY
WX
WO
wP
Image Plane
{ }W
uP
IY
IX
IO
{ }I
Focal Point
WZ
WY
WX
WO
wP
Image Plane
{ }W
uP
IY
IX
IO
{ }I
Focal Point
1
I
u
I I
u u
X
P Y
1
W
w
W
W w
w W
w
X
YP
Z
In pixels
In metrics?
wl
wl
0
W
w
W
W w
w W
w
X
Yl
Z
2.1 Calibration Introduction
![Page 7: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/7.jpg)
7
Lecture 2: Camera Calibration
Modelling
G(X) X ?
Calibration
X !!!
Modelling:
• Determine the equation that approximates the camera behaviour.
• Define the set of unknowns in the equation (camera parameters).
• The camera model is an approximation of the physics & optics of the camera.
Calibration:
• Get the numeric value of every camera parameter.
G(X)
2.1 Calibration Introduction
![Page 8: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/8.jpg)
8
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 9: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/9.jpg)
9
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 10: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/10.jpg)
10
Lecture 2: Camera Calibration
2.2 Pinhole Model
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX{ }R
RO
Image
coordinate
system
![Page 11: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/11.jpg)
11
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 1: World to Camera)
Camera
coordinate
system
World
coordinate
system
CY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
C
WK
Step 1
![Page 12: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/12.jpg)
12
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 2: Projection)
Camera
coordinate
system
World
coordinate
system
CY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uPf
Step 2
wX
wY
wZ
uX
uY
RY
RX{ }R
RO
![Page 13: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/13.jpg)
13
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
Camera
coordinate
system
World
coordinate
system
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
RY
RX{ }R
RO
Step 3
dP
![Page 14: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/14.jpg)
14
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
dP
uP dr
CY
CX
Observed position
Ideal
projection
dr: radial distortion
a
b
Radial distortion effect (a: negative, b: positive)
Radial Distortion
![Page 15: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/15.jpg)
15
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
Axis with
maximum
radial
distortion
Axis with
minimum
tangential
distortion
CY
CX
Ideal
projection
Observed
position
dr: radial distortion
dt: tangential distortion
dPuP
dr
CY
CX
dt
Radial and Tangential Distortion
Image with distortionImage without distortion
![Page 16: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/16.jpg)
16
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 4: Camera to Image)
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX{ }R
RO
Image
coordinate
system
Step 4
dP
![Page 17: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/17.jpg)
17
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.2 Pinhole Model
![Page 18: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/18.jpg)
18
Lecture 2: Camera Calibration
2.2 Calibration Methods (I)
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
![Page 19: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/19.jpg)
19
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 20: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/20.jpg)
20
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 21: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/21.jpg)
21
Lecture 2: Camera Calibration
2.3 The Method of Hall
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
![Page 22: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/22.jpg)
22
Lecture 2: Camera Calibration
World
coordinate
system
WZ
WYWX
WO
wP
Image plane
{ }W
uP
IY
IXIO{ }IImage
coordinate
system
2.3 The Method of Hall - Modelling
![Page 23: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/23.jpg)
23
Lecture 2: Camera Calibration
11 12 13 14
21 22 23 24
31 32 33 341
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A A
Assume light is captured on the image plane by a linear projection
The matrix is defined up to a scale factor Multiple Solutions
A component is fixed to the unity Unique Solution
11 12 13 14
21 22 23 24
31 32 33 11
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A
2.3 The Method of Hall - Modelling
![Page 24: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/24.jpg)
24
Lecture 2: Camera Calibration
11 12 13 14
31 32 33
21 22 23 24
31 32 33
1
1
W W W
I w w w
u W W W
w w w
W W W
I w w w
u W W W
w w w
A X A Y A Z AX
A X A Y A Z
A X A Y A Z AY
A X A Y A Z
11 12 13 14
21 22 23 24
31 32 33 11
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A
11 31 12 32 13 33 14
21 31 22 32 23 33 24
W I W W I W W I W I
w u w w u w w u w u
W I W W I W W I W I
w u w w u w w u w u
A X A X X A Y A X Y A Z A X Z A X
A X A Y X A Y A Y Y A Z A Y Z A Y
2.3 The Method of Hall - Calibration
![Page 25: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/25.jpg)
25
Lecture 2: Camera Calibration
2 1
2
1 0 0 0 0
0 0 0 0 1
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
Q X Y Z X X X Y X Z
Q X Y Z Y X Y Y Y Z
2 1
2
I
i u i
I
i u i
B X
B Y
T
11 12 13 14 21 22 23 24 31 32 33A A A A A A A A A A A A
1
t tA Q Q Q B
QA B
Pseudoinverse leads to a unique solution:
1A Q B
Obtaining 11 unknowns and each 2D point gives two equations
So, at least 6 points are needed. More points leads to a more accurate solution.
11 31 12 32 13 33 14
21 31 22 32 23 33 24
W I W W I W W I W I
w u w w u w w u w u
W I W W I W W I W I
w u w w u w w u w u
A X A X X A Y A X Y A Z A X Z A X
A X A Y X A Y A Y Y A Z A Y Z A Y
2.3 The Method of Hall - Calibration
![Page 26: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/26.jpg)
26
Lecture 2: Camera Calibration
Camera
coordinate
system
CY
CX CZ
CO{ }C
IY
IXIO{ }I
RY
RX
RO
{ }R
World
coordinate
system
WZ
WYWX
WO{ }W
Reconstruction
Area
Image of the calibrating pattern
2 1
2
1 0 0 0 0
0 0 0 0 1
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
Q X Y Z X X X Y X Z
Q X Y Z Y X Y Y Y Z
2 1
2
I
i u i
I
i u i
B X
B Y
1t t
A Q Q Q B
2.3 The Method of Hall - Calibration
![Page 27: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/27.jpg)
27
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 28: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/28.jpg)
28
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 29: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/29.jpg)
29
Lecture 2: Camera Calibration
2.4 The Method of Faugeras-Toscani
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
![Page 30: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/30.jpg)
30
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.4 The Method of Faugeras-Toscani
![Page 31: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/31.jpg)
31
Lecture 2: Camera Calibration
• Extrinsic parameters: Model the situation and orientation of the camera with
respect to a world co-ordinate system.
• Intrinsic parameters: Model the behaviour of the internal geometry and the optical
characteristics of the camera.
u
w
v
Yc
Xc
Zc
Oc
Oi
(u0, v0)
Pu
P
image
co-ordinate
system
(píxels)
retinal
co-ordinate
system
(mm.)
Image plane
Retinal plane
Yr
Xr
Zr
w
World
co-ordinate
system
WZ
WY
WX
WO { }W
Camera
co-ordinate system
2.4 The Method of Faugeras-Toscani
![Page 32: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/32.jpg)
32
Lecture 2: Camera Calibration
Yc
Xc
Zc
Zw
YwOc
Ow
Pw
Camera
co-ordinate
system World
co-ordinate system
Retinal Plane
K
Xw
X
C
W Y
Z
t
T t
t
11 12 13
21 22 23
31 32 33
, , ,C
W
C
W
R Rot X Rot Y Rot Z
r r r
R r r r
r r r
C W
w w
C C W C
w W w W
C W
w w
X X
Y R Y T
Z Z
1 1
C W
Cw w
W
P PK
3 3 3 1
1 30 1
C C
C W Wx x
W
x
R TK
2.4 Extrinsic Parameters
![Page 33: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/33.jpg)
33
Lecture 2: Camera Calibration
CPw
CPu
Yc
Xc
Zc
Oc C
f
PZc
Yu
PYc XuPXc
C
C w
u C
w
C
C w
u C
w
XX f
Z
YY f
Z
2.4 The Intrinsic Parameters: Ideal Projection
![Page 34: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/34.jpg)
34
Lecture 2: Camera Calibration
pixel
Retinal
plane
(0, 0)
Yr
Xr (0, 0)
(Xd, Yd)
Image
Plane
(Xp, Yp)
R C
d u u
R C
d v u
X k X
Y k Y
2.4 The Intrinsic Parameters: Pixel Conversion
![Page 35: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/35.jpg)
35
Lecture 2: Camera Calibration
Yr
Xr
V
U(0, 0)
Principal point
(u0,v0)
Computer image
co-ordinate
system
Camera
co-ordinate
system
0
0
I R
d d
I R
d d
X X u
Y Y v
2.4 The Intrinsic Parameters: Principal Point
![Page 36: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/36.jpg)
36
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.4 The Method of Faugeras-Toscani
![Page 37: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/37.jpg)
37
Lecture 2: Camera Calibration
Real projection on the image plane (Xi, Yi)
(Xw, Yw, Zw) 3D object point with respect to world co-ordinate system
Affine transformation.
Modelled parameters: R, T
(Xc, Yc, Zc) 3D object point with respect to camera co-ordinate system
Perspective transformation.
Modelled parameter: f
(Xu, Yu) Ideal projection on the retinal plane
Pixel adjustment
Modelled parameters: ku, kv
(Xp, Yp) Real projection on the image plane
Adaptation to the computer image buffer
Modelled parameters: u0, v0
2.4 The Method of Faugeras-Toscani
![Page 38: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/38.jpg)
38
Lecture 2: Camera Calibration
C
C w
u C
w
C
C w
u C
w
XX f
Z
YY f
Z
R C
d u u
R C
d v u
X k X
Y k Y
0
0
I R
d d
I R
d d
X X u
Y Y v
0
0
C
I w
u u C
w
C
I w
u v C
w
XX k f u
Z
YY k f v
Z
0
0
0 0
0 0
0 0 1 01
C
I w
u u C
I w
u v C
w
Xs X u
Ys Y v
Zs
vv
uu
fk
fk
2.4 The Method of Faugeras-Toscani - Modelling
![Page 39: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/39.jpg)
39
Lecture 2: Camera Calibration
11 12 13
0
21 22 23
0
31 32 33
0 0
0 0
0 0 1 00 0 0 1 1
W
xI w
u u W
yI w
u v W
z w
r r r t Xs X u
r r r t Ys Y v
r r r t Zs
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
Intrínsecs Extrínsecs
2.4 The Method of Faugeras-Toscani - Modelling
![Page 40: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/40.jpg)
40
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 41: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/41.jpg)
41
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 42: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/42.jpg)
42
Lecture 2: Camera Calibration
11 12 13
0
21 22 23
0
31 32 33
0 0
0 0
0 0 1 00 0 0 1 1
W
xI w
u u W
yI w
u v W
z w
r r r t Xs X u
r r r t Ys Y v
r r r t Zs
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
Intrínsecs Extrínsecs
2.5 The Method of Faugeras-Toscani – Modelling
![Page 43: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/43.jpg)
43
Lecture 2: Camera Calibration
1 14 3 34
2 24 3 34
0
0
W I W
w u w
W I W
w u w
A P A X A P A
A P A Y A P A
31 14
34 34 34
32 24
34 34 34
I W W I
u w w u
I W W I
u w w u
AA AX P P X
A A A
AA AY P P Y
A A A
1 1 2
3 2 2
I W W I
u w w u
I W W I
u w w u
X T P C T P X
Y T P C T P Y
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Calibration
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
343
242
141
AA
AA
AA
1343
242
141
w
W
u
I
u
I
P
AA
AA
AA
s
Ys
Xs
![Page 44: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/44.jpg)
44
Lecture 2: Camera Calibration
1
2
3
1
2
T
T
X T
C
C
B QX
1 3
1 3
0 1 0
0 0 1
t tW I W
w u w xi i i
t tI W W
x u w wi i i
P X PQ
Y P P
I
u i
I
u i
XB
Y
1
t tX Q Q Q B
2.5 The Method of Faugeras-Toscani – Calibration
1 1 2
3 2 2
I W W I
u w w u
I W W I
u w w u
X T P C T P X
Y T P C T P Y
11 unknowns
minimum 6 points
![Page 45: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/45.jpg)
45
Lecture 2: Camera Calibration
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
3 1r 2
1zt
T
2.5 The Method of Faugeras-Toscani – tz
𝑅 =
𝑟1𝑟2𝑟3
![Page 46: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/46.jpg)
46
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Intrinsics
02
33130
33310
32121
·· u
t
rr
t
r
t
ru
t
r
t
r
t
r
t
ru
t
rTTTT
z
u
zzzzz
u
zz
t
2
1zt
T
2
2
210
T
TTu
t
![Page 47: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/47.jpg)
47
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2 31 2
0 02 2
2 2
1 2 2 3
2 2
2 2
tt
t t t t
u v
T TT Tu v
T T
T T T T
T T
2.5 The Method of Faugeras-Toscani – Intrinsics
![Page 48: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/48.jpg)
48
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Extrinsics
tt
t
tt
t
u
z
z
u
zz
TT
T
T
TTTTr
TTT
T
T
TTTTr
tu
t
rTr
t
ru
t
rT
21
2
2
2
21211
221
2
2
2
2
212110
311
10
31
1
![Page 49: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/49.jpg)
49
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2 1 2
1 1 22
1 2 2
2 2 3
2 3 22
2 3 2
2
3
2
t
t t
t
t t
T T Tr T T
T T T
T T Tr T T
T T T
Tr
T
2 1 2
1 2
1 2 2
2 2 3
2 2
2 3 2
2
1
t
x t t
t
y t t
z
T T Tt C
T T T
T T Tt C
T T T
tT
2.5 The Method of Faugeras-Toscani – Extrinsics
![Page 50: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/50.jpg)
50
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 51: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/51.jpg)
51
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 52: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/52.jpg)
52
Lecture 2: Camera Calibration
2.6 The Method of Faugeras-Toscani with distortion
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
![Page 53: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/53.jpg)
53
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 5
Step 4
2.6 The Method of Faugeras-Toscani with distortion
![Page 54: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/54.jpg)
54
Lecture 2: Camera Calibration
Ideal
projection
Observed
position
drdt
Xr
Yr
dr: radial distortion
dt: tangential distortion
PuPd
2.6 Lens Distortion
![Page 55: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/55.jpg)
55
Lecture 2: Camera Calibration
a
b
Radial distorsion effect Tangential distorsion effect
Xr
Axe of a
maximum
tangential
distortion
Axe of a
minimum
tangential
distortion
Radial distorsion is the most important and usually the only considered in
calibration.
2.6 Lens Distortion
![Page 56: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/56.jpg)
56
Lecture 2: Camera Calibration
X X Du d x Y Y Du d y
D X k rx d 1
2 D Y k ry d 1
2r X Yd d 2 2
2 4
1 2
2 4
1 2
2 2
C
x d
C
y d
C C
d d
D X k r k r
D Y k r k r
r X Y
k1 is the most important component
and usuallly sufficient in most
applications.
2.6 Lens Distortion
Model of Faugeras-Toscani with distortion:
![Page 57: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/57.jpg)
57
Lecture 2: Camera Calibration
u
w
v
Yc
Xc
Zc
Oc
Oi
(u0, v0)
Pu
P
Camera
co-ordinate system
image
co-ordinate
system
f Pd
retinal
co-ordinate
system
Image plane
Retinal plane
Yr
Xr
Zr
X
f
P
P
u Xc
Zc
Y
f
P
P
u Yc
Zc
X X Du d x Y Y Du d y
D X k rx d 1
2 D Y k ry d 1
2r X Yd d 2 2
X k Xp u d Y k Yp v d
X X ui p 0 Y Y vi p 0
2.6 The Method of Faugeras-Toscani with distortion
![Page 58: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/58.jpg)
58
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 5
Step 4
2.6 The Method of Faugeras-Toscani with distortion
![Page 59: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/59.jpg)
59
Lecture 2: Camera Calibration
(Xw, Yw, Zw) 3D object point with respect to world co-ordinate system
Affine transformation.
Modelled parameters: R, T
(Xc, Yc, Zc) 3D object point with respect to camera co-ordinate system
Perspective transformation.
Modelled parameter: f
(Xu, Yu) Ideal projection on the retinal plane
Radial lens distortion.
Modelled parameter: k1
(Xd, Yd) Real projection on the retinal plane
Pixel adjustment
Modelled parameters: ku, kv
(Xp, Yp ) Real projection on the image plane
Adaptation to the computer image buffer
Modelled parameters: u0, v0
(Xi, Yi) Real projection on the image plane
2.6 The Method of Faugeras-Toscani with distortion
![Page 60: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/60.jpg)
60
Lecture 2: Camera Calibration
2
1
2
1
C
C Cw
d dC
w
C
C Cw
d dC
w
Xf X k r X
Z
Yf Y k r Y
Z
0
0
I
dC
d
u
I
dC
d
v
X uX
k
Y vY
k
1 1
C W
w w
C W
Cw w
WC W
w w
X X
Y YK
Z Z
r X Yd d 2 2
The model is NON-LINEARIterative minimisation:
• Newton-Raphson
• Levenberg-Marquardt
2.6 The Method of Faugeras-Toscani with distortion
![Page 61: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/61.jpg)
61
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 62: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/62.jpg)
62
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
![Page 63: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/63.jpg)
63
Lecture 2: Camera Calibration
Hall Faugeras Faugeras distorted
Tsai Weng
Transformation
matrix
Step 3Lens
Distortion
Step 2Projection
Step 1World2camera
Transformation
with , , , tx, ty and tz
Projection with f
Radial distortion with k1
UndistortedMultiple
distortion
k1, g1, g2, g3, g4
Transformation with
u0, v0 , ku and kv
Transformation
with u0, v0 and sx Transformation
with u0, v0,
ku and kv
Step 4Camera2image
C C W C
w W w WP P T R
,
C C
C Cw w
u uC C
w w
X YX f Y f
Z Z
C C
w uP P
C C
u dP P
C I
d dP P
C C
u dP P
0
0
I C
d u d
I C
d v d
X k X u
Y k Y v
2 2
1
2 2
1
C C C C C
u d u u u
C C C C C
u d u u u
X X k X X Y
Y Y k Y X Y
1'
0
1
0
I C
d x x d
I C
d y d
X s d X u
Y d Y v
=
W C
w wP P
I W
d wP P A
2.7 Experimental Comparison - Methods
![Page 64: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/64.jpg)
64
Lecture 2: Camera Calibration
wP
Optical Ray
3Dd
2.7 Experimental Comparison - Accuracy Evaluation
• 3D Measurement
– Distance with respect to the optical ray
– Normalized Stereo Calibration Error
• 2D Measurement
– Accuracy of distorted image coordinates
– Accuracy of undistorted image
coordinates
1 22 2
2 2 21
ˆ ˆ1
NSCEˆ 12
C C C Cn
w w w wi i i i
Ci w u vi
X X Y Y
n Z
Camera
coordinate
system
World
coordinate
system
fCY
CX
WZ
WX
WO
wP
Image plane
{ }W
uP
dP
IX{ }I
RX
{ }R
RO
Image
coordinate
system
ˆuP
ˆdP
0 0,u v
WYCZ
CO { }C
IY
IO
RY
uP
ˆuP
0 0,u v
dd
Observed Point
Linear Projection
- distortion
+ distortion
ud
ˆdP
dP
![Page 65: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/65.jpg)
65
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Synthetic Images (I)
2D distorted image (pix.) 2D undistorted image (pix.)
Mean Standard
desviation
Max Min Mean Standard
desviation
Max Min
1 Hall 0.2676 0.1979 1.2701 0.0213 0.2676 0.1979 1.2701 0.0213
2 Faugeras 0.2689 0.1997 1.2377 0.0075 0.2689 0.1997 1.2377 0.0075
3 Faugeras with distortion 0.0840 0.0458 0.2603 0.0081 0.0834 0.0454 0.2561 0.0080
4 Tsai 0.0838 0.0457 0.2426 0.0035 0.0832 0.0453 0.2386 0.0035
5 Weng 0.0845 0.0455 0.2608 0.0019 0.0843 0.0443 0.2584 0.0129
2D distorted
0
0,05
0,1
0,15
0,2
0,25
0,3
1 2 3 4 5
pix
.
Mean Standard deviation
2D undistorted
0
0,05
0,1
0,15
0,2
0,25
0,3
1 2 3 4 5p
ix.
Mean Standard deviation
![Page 66: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/66.jpg)
66
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Synthetic Images (II)
3D position (mm) NSCE
Mean Standard
desviation
Max Min
1 Hall 0.1615 0.1028 0.5634 0.0113 n/a
2 Faugeras 0.1811 0.1357 0.8707 0.0147 0.6555
3 Faugeras NR with distortion 0.0566 0.0307 0.1694 0.0055 0.2042
4 Tsai optimized 0.0565 0.0306 0.1578 0.0087 0.2037
5 Weng 0.0570 0.0305 0.1696 0.0088 0.2064
Normalized Stereo Calibration Error
Normalized Stereo Calibration Error
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
1 2 3 4 5
NSCE
3D position
0
0,05
0,1
0,15
0,2
1 2 3 4 5
mm
.
Mean Standard deviation
![Page 67: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/67.jpg)
67
Lecture 2: Camera Calibration
Computing Time
160 punts 1800 punts
• Hall 1 ms 70 ms
• Faugeras 1 ms 70 ms
• Faugeras with distortion 10 ms 380 ms
• Tsai 10 ms 530 ms
• Weng 51 ms 4216 ms
Pentium III at 1 GHz.
2.7 Experimental Comparison: Synthetic Images (III)
![Page 68: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/68.jpg)
68
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Real Images (I)
Camera
coordinate
system
CY
CX CZ
CO{ }C
IY
IXIO{ }I
RY
RX
RO
{ }R
World
coordinate
system
WZ
WYWX
WO{ }W
Reconstruction
Area
Image of the calibrating pattern
3D position (mm) NSCE
Mean Standard
desviation
Max Min
Hall 0.5219 0.2595 1.1370 0.0143 n/a
Faugeras 0.7782 0.4253 2.0210 0.0187 4.0649
Faugeras with distortion 0.4967 0.3367 1.5642 0.0094 2.5489
Tsai 0.4815 0.3023 1.4014 0.0093 2.4836
Weng 0.4740 0.2904 1.2669 0.0087 2.4556
![Page 69: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/69.jpg)
69
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Real Images (II)
3D position (mm) NSCE
Mean Standard
desviation
Max Min
Hall 1.5698 0.9842 8.9249 0.0247 n/a
Faugeras 1.6187 0.9856 8.8812 0.0302 2.0175
Faugeras with distortion 0.9930 0.5660 3.2386 0.0154 0.9909
Tsai 0.9927 0.5655 3.2311 0.0153 0.9908
Weng 0.9896 0.5724 3.3526 0.0149 0.9869
Image of the calibration patternStereo camera over a mobile robot
![Page 70: Lecture 2 Camera Calibration](https://reader034.vdocument.in/reader034/viewer/2022042517/589e9c721a28ab9f728b67d7/html5/thumbnails/70.jpg)
70
Lecture 2: Camera Calibration
2.7 Experimental Comparison - Conclusions
• Implementation of 5 of the most used camera calibration
methods
– Notation was unified
– The methods were compared in terms of model and
calibration
• The accuracy of non-linear methods is better than linear
methods
• Modelling of radial distortion is quite sufficient when high
accuracy is required
• Accuracy measuring methods obtain similar results if they are
relatively compared
Additional bibliography:
J. Salvi, X. Armangué and J. Batlle. A Comparative Review of Camera Calibrating Methods with Accuracy Evaluation. Pattern Recognition, PR, pp. 1617-1635, Vol. 35, Issue 7, July 2002.