vision-based registration for ar presented by diem vu nov 20, 2003
Post on 20-Dec-2015
219 views
TRANSCRIPT
![Page 1: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/1.jpg)
Vision-based Registration for AR
Presented by Diem Vu
Nov 20, 2003
![Page 2: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/2.jpg)
Markerless Tracking using Planar Structure in the Scene. G. Simon, A.W. Fitzgibbon and A. Zisserman, 2000.
Calibration-Free Augmented Reality. K.N Kutulakos and J.R. Vallino, 1998.
![Page 3: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/3.jpg)
Planar-surface tracking.
Camera position can be recovered from planar homography.
Planar structure is common in almost all scenarios.
![Page 4: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/4.jpg)
y
x
z
Hw
World to image homography
jiH
Image to image homography
![Page 5: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/5.jpg)
World to image homography
Consider our tracking plane is the plane Z=0
y
x
z
Hw
1
H
1w Y
X
y
x
![Page 6: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/6.jpg)
Projection matrix
trrr 321KP
![Page 7: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/7.jpg)
trrrr 2121 KP
Projection matrix
1
y
x
1
0
Y
X
y
x
z
P
1
0K
12
Y
X
y
x
trrr 31
![Page 8: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/8.jpg)
trrrr 2121 KP
Projection matrix
1
y
x
1
0
Y
X
y
x
z
P
1
K
1
Y
X
y
x
trr 21
![Page 9: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/9.jpg)
If K and Hw are known, then r1, r2 and t can be recovered, thus P.
Question: How to compute Hw?Direct.Indirect.
trr 21KHw
![Page 10: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/10.jpg)
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
(0,0)
(0,1) (1,1)
((1,0))
![Page 11: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/11.jpg)
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
s,1)(1,diagH H w
trr 21 s HK -1
s,1)(1,diagK H trr 21
Compute Hw=Hdiag(1,1/s,1)
(0,0)
(0,s) (1,s)
((1,0))
![Page 12: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/12.jpg)
Indirect measurement of Hw
iwH
jiH
? H jw
y
x
z
![Page 13: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/13.jpg)
Indirect measurement of Hw
iwH
jiH
iw
ji
jw HHH
y
x
z
![Page 14: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/14.jpg)
Algorithm summary
Compute (direct measure).For each frame i, compute frame to frame
homography (RANSAC)Compute by:
0wH
1-iw
i1-i
iw HHH
i1-iH
iwH
![Page 15: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/15.jpg)
Other …
Using only 2 points in direct method ??Matching the frame i with frame 0 in order
to reduce error.Estimate intrinsic parameters K Hand-off mechanism.
![Page 16: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/16.jpg)
![Page 17: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/17.jpg)
![Page 18: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/18.jpg)
![Page 19: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/19.jpg)
![Page 20: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/20.jpg)
Possible problems?
Homography is only up-to-scale?Plain surface (no texture) or moving
objects in the foreground ?Depth order, occlusion ?Speed ?
![Page 21: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/21.jpg)
Affine virtual object representation
Represent virtual objects so that their projection can be computed as a linear combination of the projection of the fiducial points.
![Page 22: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/22.jpg)
![Page 23: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/23.jpg)
Project a point from its affine coordinates
![Page 24: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/24.jpg)
Compute affine coordinates from projection along two viewing
direction
![Page 25: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/25.jpg)
Algorithm
Setup the affine basis
![Page 26: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/26.jpg)
Algorithm
Setup the affine basis Locate the object in 2 frames.
![Page 27: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/27.jpg)
Algorithm
Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point.
![Page 28: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/28.jpg)
Algorithm
Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point. Compute projection of the object
and render the object in each frame.
![Page 29: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/29.jpg)
![Page 30: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/30.jpg)
Camera viewing direction
and are the first and second row of 2x3.
The camera viewing direction expressed in the coordinate frame of the affine basis points: =
![Page 31: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/31.jpg)
Depth order
w is the z-value of point p (x,y,z).
![Page 32: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/32.jpg)
![Page 33: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/33.jpg)
Advantages
No need any metric information.Able to use with the existing hardware to
accelerate graphics operations.Can be used to improve tracking.
![Page 34: Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003](https://reader030.vdocument.in/reader030/viewer/2022020111/56649d4d5503460f94a2bafa/html5/thumbnails/34.jpg)
Limitation
Affine constraints.Lost of metric information.