Download - Old summary of camera modelling
![Page 1: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/1.jpg)
1
Old summary of camera modelling
• 3 coordinate frame• projection matrix• decomposition• intrinsic/extrinsic param
![Page 2: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/2.jpg)
2
World coordinate frame: extrinsic parameters
11
1w
w
w
c
c
c
ZYX
ZYX
0tR
Finally, we should count properly ...
13343 0
tR0IKC
![Page 3: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/3.jpg)
3
‘abstract’ camera: projection from P3 to P2
This is the most general camera model without considering optical distortion
As lines are preserved so that it is a linear transformation and can be represented by a 3*4 matrix
23: PP C
c33
34333231
24232221
14131211
CC
cccccccccccc
Math: central proj. Physics: pin-hole
‘new’ way of looking at ‘old’ modeling
![Page 4: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/4.jpg)
4
• 11 d.o.f.• Rank(P) = ?• ker(P)=c• row vectors, planes• column vectors, directions• principal plane: w=0• calibration, 6 pts• decomposition by QR, • K intrinsic (5). R, t, extrinsic (6)• geometric interpretation of K, R, t (backward from u/x=v/y=f/z to P)• internal parameters and absolute conic
Properties of the 3*4 matrix P
![Page 5: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/5.jpg)
5
It is the image of the absolute conic, prove it first!
1T KK
TKK
Point conic:
The dual conic:
What is the calibration matrix K?
![Page 6: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/6.jpg)
6
01cos))((2)()( 002
20
2
201T
vuvu
T
avvuuvvuu
uKKu
2002
20
2
20 1cos))((2)()( i
avvuuvvuu
vuvu
),( 00 vu
vu
![Page 7: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/7.jpg)
7
Don’t forget: when the world is planar …
10
34333231
24232221
14131211
zyx
cccccccccccc
wvu
1343231
242221
141211 yx
ccccccccc
wvu
A general plane homography!
![Page 8: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/8.jpg)
8
Camera calibration
ii Xu Given
• Estimate C• decompose C into intrinsic/extrinsic
from image processing or by hand
![Page 9: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/9.jpg)
9
Calibration set-up:
3D calibration object
![Page 10: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/10.jpg)
10
),,(),( ii iiiii zyxvu Xu
34333231
24232221
34333231
14131211
czcycxcczcycxcv
czcycxcczcycxcu
iii
iiii
iii
iiii
The remaining pb is how to solve this ‘trivial’ system of equations!
![Page 11: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/11.jpg)
11
Review of some basic numerical algorithms
• linear algebra: how to solve Ax=b?• (non-linear optimisation)• (statistics)
![Page 12: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/12.jpg)
12
Linear algebra review
• Gaussian elimination • LU decomposition
• orthogonal decomposition • QR (Gram-Schmidt)• SVD (the high(est)light of linear algebra!)
![Page 13: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/13.jpg)
13
Solving (full rank) square matrix linear sys Ax =b = elimination = LU factorization
1. factor A into LU2. solve Lc = b (lower triangular, forward substitution)3. solve Ux=c (upper tri., backward substitution)
![Page 14: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/14.jpg)
14
Solving for Least squares solution for Ax=b, min||Ax-b|| = pseudo-inverse x = (A^TA)-1(A^T A)b (theoretically, but not numerically)
Numerically, QR does it well: as A^TA= R^TR,
Orthogonal bases and Gram-Schmidt A = QR
![Page 15: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/15.jpg)
15
Solving for homogeneous system Ax=o subject to ||x||=1,
It is equivalent to min||Ax||, i.e. x^T A^T A x,
the solution is the eigenvector of A^TA associated with the smallest eigenvalue
Triangular systems not bad, but diagonal system is better!
Diagonalization = eigen vectors => doable for symmtric matrices
![Page 16: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/16.jpg)
16
SVD gives orthogonal bases for all subspaces
TVUA nm
• row space: first Vs• null space: last Vs• col space: first Us• null space of the trans : last Us
A x = b, pseudo-inverse, x = A+ b for both square system and least squares sol.Even better with homogeneous sys: A x =0, x = v_n !
You get everything with svd:
![Page 17: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/17.jpg)
17
Linear methods of computing P
• p34=1• ||p||=1• ||p3||=1
Geometric interpretation of these constraints
![Page 18: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/18.jpg)
18
Decomposition
• analytical by equating K(R,t)=P• (QR (more exactly it is RQ))
![Page 19: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/19.jpg)
19
zT
zyvTT
v
zxuTT
u
ttvtvtutu
3
0302
0301
43
rrrrr
C
1. Renormalise by c32. tz = c343. r3 = c34. u0 = c1^T c35. v0 = c2^T c36. alpha u7. alpha v8. …
![Page 20: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/20.jpg)
20
Linear, but non-optimal,but we want optima, but non-linear,
methods of computing P
![Page 21: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/21.jpg)
21
))()((min 2
34333231
242322212
34333231
14131211
czcycxcczcycxcv
czcycxcczcycxcu
iii
iiii
iii
iiii
How to solve this non-linear system of equations?
![Page 22: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/22.jpg)
22
(Non-linear iterative optimisation)
• J d = r from vector F(x+d)=F(x)+J d• minimize the square of y-F(x+d)=y-F(x)-J d = r – J d • normal equation is J^T J d = J^T r (Gauss-Newton)• (H+lambda I) d = J^T r (LM)
Note: F is a vector of functions, i.e. min f=(y-F)^T(y-F)
![Page 23: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/23.jpg)
23
Using a planar pattern
Cf. the paper by Zhengyou Zhang (ICCV99), Sturm and Maybank (CVPR99)
(Homework: read these papers.)
Why? it is more convenient to have a planar calibration pattern than a 3D calibration object, so it’s very popular now for amateurs.
![Page 24: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/24.jpg)
24
10
34333231
24232221
14131211
zyx
cccccccccccc
wvu
1343231
242221
141211
yx
ccccccccc
wvu
• first estimate the plane homogrphies Hi from u and x, 1. How to estimate H? 2. Why one may not be sufficient?
• extract parameters from the plane homographies
![Page 25: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/25.jpg)
25
z
y
xu
tttrur
cccccccccccc
********13011
34333231
24232221
14131211 H
Relationship between H and parameters:
How to extract intrinsic parameters?
![Page 26: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/26.jpg)
26
(How to extract intrinsic parameters?)
1TKK
0)(1T
xHKKHx TT
The absolute conic in image
The (transformed) absolute conic in the plane:
The circular points of the Euclidean plane (i,1,0) and (-i,1,0) go thru this conic: two equations on K!
![Page 27: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/27.jpg)
27
It turns the camera into an spherical one, or angular/direction sensor!
Direction vector:
Angle between two rays ...
uKd -1
What does the calibration give us?
uKx -1Normalised coordinates:
![Page 28: Old summary of camera modelling](https://reader035.vdocument.in/reader035/viewer/2022062811/568160ef550346895dd02531/html5/thumbnails/28.jpg)
28
Summary of calibration
1. Get image-space points2. Solve the linear system 3. Optimal sol. by non-linear method4. Decomposition by RQ