introduction à la vision artificielle iii
DESCRIPTION
Introduction à la vision artificielle III. Jean Ponce Email : [email protected] Lecture given by Josef Sivic < Josef.Sivic @ ens.fr > Planches après les cours sur : http://www.di.ens.fr/~ ponce/introvis/lect3.pptx http://www.di.ens.fr/~ ponce/introvis/lect3.pdf - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/1.jpg)
Introduction à la vision artificielle III
Jean PonceEmail: [email protected]
http://www.di.ens.fr/~ponce/introvis/lect3.pptxhttp://www.di.ens.fr/~ponce/introvis/lect3.pdfhttp://www.di.ens.fr/~ponce/introvis/sbook.pdf• Remplissez la feuille avec nom, email, affiliation• Le premier exo est du le 6 octobre http://www.di.ens.fr/willow/teaching/introvis14/assignment1/
![Page 2: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/2.jpg)
Camera geometryand calibration II
• Intrinsic and extrinsic parameters• Strong (Euclidean) calibration• Degenerate configurations• What about affine cameras?
![Page 3: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/3.jpg)
The Intrinsic Parameters of a Camera
The calibration Matrix
The PerspectiveProjection Equation
![Page 4: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/4.jpg)
The Extrinsic Parameters of a Camera
![Page 5: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/5.jpg)
Explicit Form of the Projection Matrix
Note:
M is only defined up to scale in this setting!!
![Page 6: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/6.jpg)
Explicit Form of the Projection Matrix
Note:
M is only defined up to scale in this setting!!
![Page 7: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/7.jpg)
Theorem (Faugeras, 1993)
![Page 8: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/8.jpg)
Observations:
•
is the equation of a plane of normal direction a1
• From the projection equation, it is also the plane defined by: u = 0
• Similarly: • (a2,b2) describes the plane defined by: v = 0• (a3,b3) describes the plane defined by:
That is the plane passing through the pinhole (z = 0)
Geometric Interpretation
33
11
3
1
bZYX
a
bZYX
a
PmPmu
T
T
T
T
011
b
ZYX
aT
vu
Projection equation:
![Page 9: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/9.jpg)
u
v
a3
C
Geometric Interpretation: The rows of the projection matrix describe the three planes defined by the image coordinate system
a1
a2
![Page 10: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/10.jpg)
p P
Other useful geometric properties
Q: Given an image point p, what is the direction of the corresponding ray in space?
A:
Q: Can we compute the position of the camera center ?
A:
pAw 1
bA 1
![Page 11: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/11.jpg)
Linear Camera Calibration
![Page 12: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/12.jpg)
Linear Systems
A
A
x
x b
b=
=
Square system:• unique solution
• Gaussian elimination
Rectangular system ??• underconstrained: infinity of solutions
Minimize |Ax-b|2
• overconstrained: no solution
![Page 13: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/13.jpg)
How do you solve overconstrained linear equations ??
![Page 14: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/14.jpg)
Homogeneous Linear Systems
A
A
x
x 0
0=
=
Square system:• unique solution: 0
• unless Det(A)=0
Rectangular system ??• 0 is always a solution
Minimize |Ax| under the constraint |x| =12
2
![Page 15: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/15.jpg)
How do you solve overconstrained homogeneous linear equations ??
The solution is e .1
E(x)-E(e1) = xT(UTU)x-e1T(UTU)e1
= 112+ … +qq
2-1
> 1(12+ … +q
2-1)=0
![Page 16: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/16.jpg)
Example: Line Fitting
Problem: minimize
with respect to (a,b,d).
• Minimize E with respect to d:
• Minimize E with respect to a,b:where
• Done !!
n
![Page 17: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/17.jpg)
Note:
• Matrix of second moments of inertia
• Axis of least inertia
![Page 18: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/18.jpg)
Linear Camera Calibration
![Page 19: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/19.jpg)
Once M is known, you still got to recover the intrinsic andextrinsic parameters !!!
This is a decomposition problem, not an estimationproblem.
• Intrinsic parameters
• Extrinsic parameters
r
![Page 20: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/20.jpg)
Degenerate Point ConfigurationsAre there other solutions besides M ??
• Coplanar points: (λ, μ, ν ) = (π,0,0) or (0,π,0) or (0,0,π )• Points lying on the intersection curve of two quadricsurfaces = straight line + twisted cubic
Does not happen for 6 or more random points!
![Page 21: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/21.jpg)
Analytical Photogrammetry
Non-Linear Least-Squares Methods• Newton• Gauss-Newton• Levenberg-Marquardt
Iterative, quadratically convergent in favorable situations
![Page 22: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/22.jpg)
Weak-Perspective Projection
Paraperspective Projection
What about Affine Cameras?
![Page 23: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/23.jpg)
Orthographic Projection
Parallel Projection
More Affine Cameras
![Page 24: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/24.jpg)
Weak-Perspective Projection Model
r(p and P are in homogeneous coordinates)
p = A P + b (neither p nor P is in hom. coordinates)
p = M P (P is in homogeneous coordinates)
![Page 25: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/25.jpg)
Theorem: All affine projection models can be represented by affine projection matrices.
Definition: A 2x4 matrix M = [A b], where A is a rank-2 2x3 matrix, is called an affine projection matrix.
![Page 26: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/26.jpg)
22101 tRΜ
skzr
General form of the weak-perspective projection equation:
Theorem: An affine projection matrix can be written uniquely (up to a sign amibguity) as a weak perspective projection matrix as defined by (1).
(1)
![Page 27: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/27.jpg)
Applications: Mobile Robot Localization (Devy et al., 1997)
![Page 28: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/28.jpg)
(Rothganger, Sudsang, & Ponce, 2002)
![Page 29: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/29.jpg)
Applications: Calibration for sports TV (Grau, BBC R&D)
![Page 30: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/30.jpg)
Calibration with a wand in uncontrolled environments
Difficulties: • Cameras are knocked overnight• They are not rigidly mounted
![Page 31: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/31.jpg)
Calibration from image features only
How? • Find the pitch lines• Compute the intrinsic & extrinsic parameters
![Page 32: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/32.jpg)
• Radiométrie• Radiance• Irradiance• BRDF• Photometric stereo
• Couleur• Radiométrie spectrale• Couleur des sources et des surfaces• Trichromacies• L’oeil• Espaces de couleurs• Modèle bichromatique
Lumière, dégradés, couleurs et ombres
![Page 33: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/33.jpg)
Light and shadows
[These slides courtesy of S. Narasimhan, CMU.]
![Page 34: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/34.jpg)
Reflections
![Page 35: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/35.jpg)
![Page 36: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/36.jpg)
![Page 37: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/37.jpg)
![Page 38: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/38.jpg)
Reflections and refractions
![Page 39: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/39.jpg)
Refraction
Caustics
![Page 40: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/40.jpg)
Is glass really transparent?
![Page 41: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/41.jpg)
![Page 42: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/42.jpg)
Interreflections
![Page 43: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/43.jpg)
Scattering
![Page 44: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/44.jpg)
![Page 45: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/45.jpg)
![Page 46: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/46.jpg)
![Page 47: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/47.jpg)
More complex stuff
![Page 48: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/48.jpg)
![Page 49: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/49.jpg)
![Page 50: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/50.jpg)
The two main types of reflection: diffuse and specular
![Page 51: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/51.jpg)
Shading and orientation (distant light sources)Interreflections and soft shadows
![Page 52: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/52.jpg)
Area sources and soft shadows
![Page 53: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/53.jpg)
Indoor scene with some directional componenentbut almost no shadows: small occluders and largesources
![Page 54: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/54.jpg)
Image Formation: Radiometry
What determines the brightness of an image pixel?
The lightsource(s)
The surfacenormal
The surfacepropertiesThe optics
The sensorcharacteristics
![Page 55: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/55.jpg)
d2P = L( P, v ) dA dw
DEFINITION: The radiance is the power traveling at some pointin a given direction per unit area perpendicular to this direction,per unit solid angle.
![Page 56: Introduction à la vision artificielle III](https://reader036.vdocument.in/reader036/viewer/2022081502/568166c3550346895ddacf44/html5/thumbnails/56.jpg)
d2P = L( P, v ) dA dw d2P = L( P, v ) cosq dA dw
DEFINITION: The radiance is the power traveling at some pointin a given direction per unit area perpendicular to this direction,per unit solid angle.