calibration dorit moshe. 2 in today’s show how positions in the image relate to 3d positions in...
Post on 21-Dec-2015
214 views
TRANSCRIPT
![Page 1: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/1.jpg)
Calibration
Dorit Moshe
![Page 2: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/2.jpg)
2
In today’s show
How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify more precisely the
relationship between a camera, the objects it observes, and the pictures of these objects
We start by briefly recalling elementary notions of analytical Euclidean geometry.
We then introduce the various physical parameters that relate the world and camera coordinate frames, and present as an application various methods for estimating these parameters, a process known as geometric camera calibration.
We also present along the way a linear least-squares technique for parameter estimation
![Page 3: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/3.jpg)
3
Motivation
How positions in the image relate to 3D positions in the world?
The reconstruction of 3D image is not trivial. We have to reconstruct the third coordinate!
32
z
y
x
y
x
![Page 4: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/4.jpg)
4
Example
Rabbit or Man?
The information lays within the 3rd coordinate
Markus Raetz, Metamorphose II, 1991-92
![Page 5: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/5.jpg)
5
2D projections are not the “same” as the real object as we usually see everyday!
![Page 6: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/6.jpg)
6
But…
![Page 7: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/7.jpg)
7
Introduction
Camera calibration: estimation of the unknown values in a camera model.Intrinsic parametersIntrinsic parameters - Link the frame coordinates of
an image point with its corresponding camera coordinates
Extrinsic parametersExtrinsic parameters - define the location and orientation of the camera coordinate system with respect to the world coordinate system
![Page 8: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/8.jpg)
8
Euclidean Geometry - reminder
Orthonormal coordinate Orthonormal coordinate frame (frame (FF)) is defined by
a point O in E3 and three unit vectors i, j and k orthogonal to each other
kOPz
zkyjxiOPjOPy
iOPx
3
z
y
x
p
![Page 9: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/9.jpg)
9
Transformations
FP - the coordinate vector of the point P in the frame F Let’s consider two frames : A and B (A) = (OA, iA, jA, kA)
(B) =(OB, iB, jB, kB)
How can we express BP as a function of AP? Let us suppose that the basis vectors of both coordinate systems are
parallel to each other, i.e., iA = iB, jA = jB and kA = kB, but the origins OA and OB are distinct
We say that two coordinate systems are separated by a pure translation,
POOOPO AABB
![Page 10: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/10.jpg)
10
Pure translation
BP = AP + BOA
![Page 11: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/11.jpg)
11
Pure rotation
When the origins of the two frames coincide, i.e.,OA = OB = O, we say that the frames are separated by a pure rotation.
![Page 12: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/12.jpg)
12
We define the rotation matrixrotation matrix
) ( kjiR AB
AB
ABA
B
BABABA
BABABA
BABABABA R
kkkjki
jkjjji
ikijii
...
...
...
TB
A
TB
A
TB
A
k
j
i
cos|||| baba
![Page 13: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/13.jpg)
13
It means that
for pure rotation
PRP ABA
B
BP BP
![Page 14: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/14.jpg)
14
The inverse of a rotation matrix is equal to its transpose Its determinant is equal to 1 – the transform preserves the
volume. Not every transformation that preserves
the volume keeps the sign. For example - reflection
This ortho normal transform preserves length and angles.
0 1
1 0
![Page 15: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/15.jpg)
15
Example (Pure rotation)
kA=kB=k
The vector iB is obtained by applying to the vector iA a counterclockwise rotation of angle θ about k.
100
0cossin
0sincos
RBA
![Page 16: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/16.jpg)
16
Translation and rotation- rigid transformation
ABAB
AB OPRP
![Page 17: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/17.jpg)
17
As a single matrix equation:
BP APTransformationRotation
![Page 18: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/18.jpg)
18
Homogenous coordinates
Add an extra coordinate and use an equivalence relation
For 3D, equivalence relation k*(X,Y,Z,T) is the same as (X,Y,Z,T)
Motivation – it will be possible to write the action of a perspective camera as a matrix
![Page 19: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/19.jpg)
19
Homogenous/Non-Homogenous transformation for 3D point
Non-homogenous to homogenous – add “1” as the 4th coordinate:
1
z
y
x
z
y
x
Homogenous to non- homogenous – devide 1st 3 coordinates by the 4th
z
y T1
x
T
z
y
x
![Page 20: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/20.jpg)
20
Homogenous/Non-Homogenous transformation for 2D point
Non-homogenous to homogenous – add “1” as the 3rd coordinate:
1
y
x
y
x
Homogenous to non- homogenous – devide 1st 2 coordinates by the 3rd
y 1 x
zz
y
x
![Page 21: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/21.jpg)
21
Camera calibration
Use the camera to tell you things about the world
Relationship between coordinates in the world and coordinates in the image : geometric calibrationgeometric calibration
(We will not discuss here the relationship between intensities in the world and intensities in the image : photometric camera calibration.)
![Page 22: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/22.jpg)
22
Three coordinate systems involved
Camera: perspective projection.Image: intrinsic/internal camera parametersWorld: extrinsic/external camera parameters
![Page 23: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/23.jpg)
23
The camera perspective equation
The coordinates (x, y, z) of a scene point P observed by a pinhole camera are related to its image coordinates (x’, y’) by the perspective equation
We have by similar triangles (x,y,z)-> (f x/z, f y/z, -f ). Ignoring the third coordinate, we get (x,y,z)-> (f x/z, f y/z)
P’P
![Page 24: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/24.jpg)
24
Intrinsic parameters
Relate the camera’s coordinate system to the idealized coordinate system
We can associate with a camera two different image planes: the first one is a normalized planenormalized plane located at a unit distance from the pinhole. We attach to this plane its own coordinate system with an origin located at the point where the optical axis pierces it. According to the perspective eq :
Perspective projectionPerspective projection::C
zy
zx
v
u
ˆ
ˆ
![Page 25: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/25.jpg)
25
f
Intrinsic parameters(cont)
1
![Page 26: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/26.jpg)
26
The second is the physical retinaphysical retina. It is located at a distance f ≠1 from the pinhole, and the image coordinates (u,v) are usually expressed in pixel units.
Pixels are usually rectangular, so the camera has two additional scale parameters k and l, and:
zy
zx
lfv
kfu f is a distance in meters
lk11 dimentions sPixel’
Define : units pixelin expressed , lfkf
Intrinsic parameters(cont)
Meterk,l 1 *pixel units
![Page 27: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/27.jpg)
27
The actual origin of the camera coordinate system is at a corner C of the retina, and not at its center. It adds two parameters u0 and v0 that define the position (in pixel units) of C0 in the retinal coordinate system.
0
0
vv
uu
zy
zx
Intrinsic parameters(cont)
![Page 28: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/28.jpg)
28
The camera coordinate system may also be skewed, due to some manufacturing error, so the angle θ between the two image axes is not equal to 90°.
0
0
sin
cot
vz
y
θ
βv
uz
yθα
z
xαu
Intrinsic parameters(cont)
![Page 29: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/29.jpg)
29
Using homogenous coordinates:
Intrinsic parameters(cont)
3x4 matrix
![Page 30: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/30.jpg)
30
The physical size of the pixels and the skew are always fixed for a given camera, and they can in principle be measured during manufacturing
Intrinsic parameters(cont)
![Page 31: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/31.jpg)
31
Extrinsic parameters
Relate the camera’s coordinate system to a fixed world coordinate system and specify its position and orientation in space.
We consider the case where the camera frame (C) is distinct from the world frame (W).
Non-homogenous coordinates
Homogenous coordinates
![Page 32: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/32.jpg)
32Twww
WC
CW
zyxP
dinatespixel coorp
Ot
RR
t)R KM
whereΜPz
1p
)1,,,(
(
Extrinsic parameters(cont)
![Page 33: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/33.jpg)
33
M can be defined with 11 free coefficients5 are intrinsic parameters – α,β,u0,v0,θ
6 are extrinsic – the 3 angles defining R, 3 coordinates of t
Combining extrinsic and intrinsic calibration parameters
3 raws of R
3 coords of t
M is only defined up to scale in this setting!!
![Page 34: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/34.jpg)
34
Rewriting the equation
PΜz
1p
1. . .m . . .
. . .m . . .
. . .m . . .1
13
2
1
z
y
x
T
T
T
w
w
w
zv
u
Pixel coordinates
World coordinates
![Page 35: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/35.jpg)
35
Z is in the camera coordinate system, but we can solve it, cause
z
Pm 31
And we get
Relation between image positions, u,v to points at 3D positions in P (homogenous coordinates)
![Page 36: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/36.jpg)
36
Calibration methods
Techniques for estimating the intrinsic and extrinsic parameters of a camera
Suppose that a camera observes nn geometric features such as points or lines with known positions in some fixed world coordinate system.
We will:Compute the perspective projection
matrix M associated with the camera in this coordinate system
Compute the intrinsic and extrinsic parameters of the camera from this matrix
Which features should we choose? Which features should we choose?
![Page 37: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/37.jpg)
37
A linear approach to camera calibration
For each feature point, i, we have :
0)(
0)(
32
31
ii
ii
Pmvm
Pmum
For n features, we will get 2n equations
![Page 38: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/38.jpg)
38P =0m
A linear approach to camera calibration(cont)
![Page 39: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/39.jpg)
39
A linear approach to camera calibration(cont)
When n>=6, the system is over-constrained, i.e. there is no non-zero vector m in R12 that satisfies exactly these equations.
On the other hand, a zero vector is always a solution. According to the linear least-squares methods, we want to
compute the value of the unit vector m that minimizes |Pm|2.
In particular, estimating the vector m reduces to computing the eigenvectors and eigenvalues of the 12x12 matrix PTP
![Page 40: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/40.jpg)
40
Linear least squares methods
Let us consider a system of n equations, p unknowns:
A is n x p matrix with coefficients aij, x = (x1,..,xp)T
There is no single solution if n≥p. The non trivial solution exists only if A is non singular.
We will try to find vector x that minimize E (the error measure):
![Page 41: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/41.jpg)
41
Linear least squares methods(cont)
Need to impose a contraint on x, since x=0 yields the minimum. Since E(λx) = λ2E(x), we will use the contraint |x2|=1.
E=xT(ATA)x , where ATA (pxp) matrix is positive smmetric matrix It can be diagonalized in an orthonormal basis of eigenvectors e i (i=1,..,p) associated
with eigenvalues 0≤ λ1 ≤ λ1 ≤ … ≤ λp
we can write x as x=μ1e1+… μnen, that (μ12 +…+ μp
2 )=1
: e1 minimizes the error E. It is the eigenvector
associated with the minimum eigenvalue of ATA (λ1 )
![Page 42: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/42.jpg)
42
Recovering the intrinsic and extrinsic parameters
Once we have the M matrix, we can recover the intrinsic and extrinsic parameters in a simple mathematical process, described in Forsyth&Ponce, section 6.3.1
![Page 43: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/43.jpg)
43
Camera Calibration with a Single Image
Sometimes more than one view of the same picture is used to estimate calibration parameters (For example, Stereo)
Most Camera parameters can be estimated from the measurements of a single image when sufficient geometric object knowledge is available.
The object knowledge used in the approach described here consists of parallelism and perpendicularity assumptions of straight object edges. In buildings parallel and perpendicular edges
are usually abundant. Therefore, this method is often applicable for historic imagery of possibly demolished buildings taken with an unknown camera.
![Page 44: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/44.jpg)
44
Targets for camera calibration
![Page 45: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/45.jpg)
45
Projection of each point gives us two equations and there are 11 unknowns. 6 points in general position are sufficient for calibration.
![Page 46: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/46.jpg)
46
The 6 anchor points clicked by the user are represented in green. If the user had clicked more accurately, they should lie exactly at the corners of the small white squares. Using these 6 points and the corresponding 3D anchor points, the program computes an initial estimate of the projection matrix.
http://www-sop.inria.fr/robotvis/personnel/lucr/detecproj.html
![Page 47: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/47.jpg)
47
We take as input a set of at least 6 non-coplanar 3D anchor points, and their 2D images. The 2D coordinates do not need to be very accurate, they are typically obtained manually by a user who clicks their approximate position.
![Page 48: Calibration Dorit Moshe. 2 In today’s show How positions in the image relate to 3D positions in the world? We will use analytical geometry to quantify](https://reader030.vdocument.in/reader030/viewer/2022032704/56649d585503460f94a373d8/html5/thumbnails/48.jpg)
48
Summary
We saw the goal of calibration We mentioned Euclidean Geometry We learned about internal/external camera parameters We learned how to compute them from a given set of
points We saw an example of calibration by one picture only We will see (stereo lecture) computing 3D coordinates
from more than one picture (More than one view).