Download - Triangulation methods mihaylova
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Triangulation MethodsSeminar work
Robotics and Medicine SS 09Institut fur Prozessrechentechnik, Automation und Robotik
(IPR)
Zlatka MihaylovaSupervisor: M.Phys. Matteo Ciucci
July 13, 2009
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
IntroductionThe human visual perception systemEpipolar geometry
Triangulation methods3D point reconstructionComputation of the Fundamental matrix F
Practical examplesActive triangulation
ConclusionAppliance in the medical roboticsClosing words
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
Stereovision
I Why are we able to percept the relative distance to all objects?
I Why is it so important to measure the distance to andbetween the objects?
I How can another point of view help in solving this problem?
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
Stereovision principle
fbx1
d
I Disparity p = b fd , where f represents the lens focal length
I p is proportional to the stereoscopic base b and inverselyproportional to d - the distance to the measured object.
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
Basics of the epipolar geometry
A B
C
e e’
c c’
�
epipolar line
baseline
epipolar line
epipolar plane
I The baseline connects camera centers A and B and intersectsthe image planes in the epipoles e and e ′.
I The epipolar plane π is defined by the camera centers and the3D object point C .
I The ambiguous projection of C .
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
Basics of the epipolar geometry
A B
C
e e’
c c’
�
epipolar line
baseline
epipolar line
epipolar plane
I The baseline connects camera centers A and B and intersectsthe image planes in the epipoles e and e ′.
I The epipolar plane π is defined by the camera centers and the3D object point C .
I The ambiguous projection of C .
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
Basics of the epipolar geometry
A B
C
e e’
c c’
�
epipolar line
baseline
epipolar line
epipolar plane
I The baseline connects camera centers A and B and intersectsthe image planes in the epipoles e and e ′.
I The epipolar plane π is defined by the camera centers and the3D object point C .
I The ambiguous projection of C .Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
The Fundamental Matrix F and the camera matrices P , P ′
I F is a 3× 3 matrix representing the mapping between a pointin the first image and epipolar line in the second image.
I For all pairs of image points c and c ′ the correspondencecondition holds:
c ′T
Fc = 0 (1)
I The camera matrices P and P ′ satisfy the conditions c = PCand c ′ = P ′C for every point correspondence
I In the case, when we deal with calibrated cameras, it iscleverer to compute the Essential Matrix E , which isspecialization of F :
F = P ′−T
EP−1 (2)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
The Fundamental Matrix F and the camera matrices P , P ′
I F is a 3× 3 matrix representing the mapping between a pointin the first image and epipolar line in the second image.
I For all pairs of image points c and c ′ the correspondencecondition holds:
c ′T
Fc = 0 (1)
I The camera matrices P and P ′ satisfy the conditions c = PCand c ′ = P ′C for every point correspondence
I In the case, when we deal with calibrated cameras, it iscleverer to compute the Essential Matrix E , which isspecialization of F :
F = P ′−T
EP−1 (2)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
The human visual perception systemEpipolar geometry
The Fundamental Matrix F and the camera matrices P , P ′
I F is a 3× 3 matrix representing the mapping between a pointin the first image and epipolar line in the second image.
I For all pairs of image points c and c ′ the correspondencecondition holds:
c ′T
Fc = 0 (1)
I The camera matrices P and P ′ satisfy the conditions c = PCand c ′ = P ′C for every point correspondence
I In the case, when we deal with calibrated cameras, it iscleverer to compute the Essential Matrix E , which isspecialization of F :
F = P ′−T
EP−1 (2)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
General approach
Algorithm:
I Take two images of the scene, separated by a baseline
I Identify the point correspondences in the images
I Apply the triangulation rules: compute F , P and P ′
I Find these two lines, which intersection defines the searchedworld point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
General approach
Algorithm:
I Take two images of the scene, separated by a baseline
I Identify the point correspondences in the images
I Apply the triangulation rules: compute F , P and P ′
I Find these two lines, which intersection defines the searchedworld point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
General approach
Algorithm:
I Take two images of the scene, separated by a baseline
I Identify the point correspondences in the images
I Apply the triangulation rules: compute F , P and P ′
I Find these two lines, which intersection defines the searchedworld point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
General approach
Algorithm:
I Take two images of the scene, separated by a baseline
I Identify the point correspondences in the images
I Apply the triangulation rules: compute F , P and P ′
I Find these two lines, which intersection defines the searchedworld point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
Identification the point correspondences in the images
The most difficult part is finding the point correspondencesautomatically! Robust pattern matching algorithm needed!
I Harris corner detector: simple but scales dependentI Successful combination of Harris and Laplacian detectors:
www.robots.ox.ac.uk/∼vgg/research/affine/det evalfiles/mikolajczyk ijcv2004.pdf
I Laplacian and Difference of Gaussian (DoG) ”points ofinterest” detectors
I Salient region detector: www.robots.ox.ac.uk/∼vgg/research/affine/det eval files/kadir04.pdf
I Maximally stable extremal regions (MSER)(http://www.robots.ox.ac.uk/∼vgg/research/affine/det eval files/matas bmvc2002.pdf - specially developed for the stereoproblem analysis)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
Algorithms for computing F
Having F computed gives us the possibility to estimate the scenepoints. There are some algorithms available:
I Eight point algorithm: F has 8 degrees of freedom, thereforewe need 8 unique point pairs to compute it. Every pair definesequation, which solution contains the nine coefficients of F
I Algebraic minimization algorithm: based on the eight pointalgorithm, but tries to minimize the algebraic error caused bynoisy measurement.
I Gold standard algorithm: dealing with the problem ofGaussian noise. This approach uses statistical methods forsolving the triangulation puzzle, namely computing F byminimizing the Likelihood function. (proposed in the book:”Multiple View Geometry in Computer Vision” - RichardHartley and Andrew Zisserman)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
Algorithms for computing F
Having F computed gives us the possibility to estimate the scenepoints. There are some algorithms available:
I Eight point algorithm: F has 8 degrees of freedom, thereforewe need 8 unique point pairs to compute it. Every pair definesequation, which solution contains the nine coefficients of F
I Algebraic minimization algorithm: based on the eight pointalgorithm, but tries to minimize the algebraic error caused bynoisy measurement.
I Gold standard algorithm: dealing with the problem ofGaussian noise. This approach uses statistical methods forsolving the triangulation puzzle, namely computing F byminimizing the Likelihood function. (proposed in the book:”Multiple View Geometry in Computer Vision” - RichardHartley and Andrew Zisserman)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
3D point reconstructionComputation of the Fundamental matrix F
Algorithms for computing F
Having F computed gives us the possibility to estimate the scenepoints. There are some algorithms available:
I Eight point algorithm: F has 8 degrees of freedom, thereforewe need 8 unique point pairs to compute it. Every pair definesequation, which solution contains the nine coefficients of F
I Algebraic minimization algorithm: based on the eight pointalgorithm, but tries to minimize the algebraic error caused bynoisy measurement.
I Gold standard algorithm: dealing with the problem ofGaussian noise. This approach uses statistical methods forsolving the triangulation puzzle, namely computing F byminimizing the Likelihood function. (proposed in the book:”Multiple View Geometry in Computer Vision” - RichardHartley and Andrew Zisserman)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Active triangulation
Light spot technique
I Simple construction: laser ray, lens, detector (CCD or PSD)I Advantages: fast, accurate, independent from surface colorI Disadvantages: the surface should be no ideal mirror
h
ӨPSD
lase
r
p
measured object
p’
q
q’
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Active triangulation
Stripe projection
I The object’s surface manipulates the scan lineI Resulting displacement in the light stripe ∼ to obj. distance
reference surface
camera
laser
hӨ
d
measured object
(a) (b)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Active triangulation
Projection of encoded patterns
I Disadvantage of the stripe projection: too slowI Correspondence problem by static line pattern projectionI Solutions: Binary coding, Grey coding, Phase shifted pattern
projection, Colored pattern (the picture is taken from thebook ”Digitale Bildverarbeitung” - Bernd Jahne)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Appliance in the medical roboticsClosing words
Polaris R©, NDI
I Standard optical tracking system in medicine, produced byNorthern Digital Inc. (NDI)
I Offers passive, active and hybrid tracking.
I The triangulated points are fixed on the surgical instrument.
I http://www.ndigital.com/medical/polarisfamily.php
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Appliance in the medical roboticsClosing words
A.R.T. R© Systems
I Advanced Realtime Tracking GmbH (A.R.T. GmbH)I Multiple camera systems - 3, 4, 5 cameras for better resultsI Example system: smARTtrack - two ARTtrack2 cameras
mounted on a rigid bar, so that no calibration needed.I different configurations depending on focal length, angle
between both cameras, baselineI http://www.ar-tracking.de/smARTtrack.49.0.html
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Appliance in the medical roboticsClosing words
Da Vinci R© Surgical System
I a) A high-resolution 3D endoscope coupled with two 3-chipcameras take the surgeon ”inside” the patient
I b) The console helps by visualizing the camera records and byrepositioning the surgical camera inside the patient.
I www.intuitivesurgical.com/products/davinci surgicalsystem
(c) (d)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Appliance in the medical roboticsClosing words
Conclusion
Through the methods of triangulation the robots similar tohumans process the visual information.For triangulation the following prerequisites are needed:
I at least 2 points of view (implemented either with cameras ormixed with light sources)
I object point, placed on a comparably closer distance (not atinfinity)
I statistically stable algorithms for computing the pointcorrespondences, respectively the distance to the world point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods
OutlineIntroduction
Triangulation methodsPractical examples
Conclusion
Appliance in the medical roboticsClosing words
Questions time
Thank You for Your attention!
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods