structure from motion · 2019-06-06 · structure from motion “3d pointcloud + cameras” •...
TRANSCRIPT
![Page 1: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/1.jpg)
Structure from Motion
CS448V — Computational Video Manipulation
April 24th, 2019
(or 3D from 2D)
![Page 2: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/2.jpg)
Overview
•Structure from Motion
•Photo Tourism
![Page 3: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/3.jpg)
3D from 2D• 3D? Really? Why 3D?
• Didn’t I sign up for video manipulation?
• Yes, but our world is 3D
• Images are ‘just’ a 2D projection of our real world
• Many edits are easier to perform directly in 3D
![Page 4: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/4.jpg)
Video Stabilization in 3D
![Page 5: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/5.jpg)
![Page 6: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/6.jpg)
Structure from Motion (SfM)What does SfM recover?
1) Sparse 3D point cloud 𝐯𝐯𝑖𝑖 ∈ ℝ3
2) Camera Extrinsic Parameters
• 𝐭𝐭𝑗𝑗 ∈ ℝ3, position
• 𝐑𝐑𝑗𝑗 ∈ 𝐒𝐒𝐒𝐒 𝟑𝟑 , orientation
3) Camera Intrinsic Parameters
• focal length f, skew s, optical center 𝐜𝐜, lens distortion parameters, …
![Page 7: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/7.jpg)
Quiz: What is?• Structure from Motion
“3D pointcloud + cameras”
• Bundle Adjustment
“a way to perform SfM”
• Multi-view Stereo
“dense reconstruction step after SfM”
• Photogrammetry
“dense reconstruction step after SfM”
![Page 8: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/8.jpg)
Intrinsics of a Perspective Camera3D
2D
1)
2) 3)
“rule of equal triangles”
“math” “vectorize/homogeneous coordinates”
𝐩𝐩xf
=𝐯𝐯x𝐯𝐯z
𝐩𝐩x =f � 𝐯𝐯x𝐯𝐯z
𝐩𝐩y =f � 𝐯𝐯y𝐯𝐯z
𝐊𝐊 =f s 𝐜𝐜x0 f 𝐜𝐜y0 0 1
�𝐩𝐩x�𝐩𝐩y�𝐩𝐩z
=f 0 00 f 00 0 1
𝐯𝐯x𝐯𝐯𝒚𝒚𝐯𝐯z
�𝐩𝐩 = 𝐊𝐊𝐯𝐯
𝐩𝐩 = 𝐝𝐝𝐝𝐝𝐝𝐝𝐝𝐝𝐝𝐝(�𝐩𝐩)“more general”
![Page 9: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/9.jpg)
Epipolar Geometry & Triangulation
𝐝𝐝l 𝐝𝐝r
𝐟𝐟
𝐯𝐯𝟎𝟎𝐯𝐯𝟏𝟏
𝐯𝐯𝟐𝟐𝐯𝐯𝟑𝟑
Epipolar Line
Epipole
Epipolar Plane
![Page 10: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/10.jpg)
Essential/Fundamental Matrix
𝐝𝐝l 𝐝𝐝r
𝐜𝐜l ∈ ℝ3 𝐜𝐜r
𝐟𝐟𝐥𝐥 ∈ ℝ2 𝐟𝐟𝐫𝐫
𝐜𝐜rT𝐄𝐄𝐜𝐜l = 0Essential Matrix
“3D to 3D”
“Camera Space points” ̅𝐟𝐟lT𝐅𝐅 ̅𝐟𝐟r = 0Fundamental Matrix
“2D to 2D”
“homogeneous”
![Page 11: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/11.jpg)
Pipeline: Structure from Motion
Keypoint Detection
Detect features using SIFT[Lowe, IJCV 2004]
Images
![Page 12: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/12.jpg)
Feature detection
Detect features using SIFT [Lowe, IJCV 2004]
![Page 13: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/13.jpg)
Pipeline: Structure from Motion
Keypoint Detection Keypoint Matching
Detect features using SIFT[Lowe, IJCV 2004]
Refine matches using RANSAC
Images
![Page 14: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/14.jpg)
Match Features between Images
Refine matches using RANSAC [Fischler & Bolles 1987]
to estimate fundamental matrices between pairs
![Page 15: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/15.jpg)
Correspondence estimation
Link up pairwise matches to form connected components
Single 3D Point
Image 1 Image 2 Image 3 Image 4
![Page 16: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/16.jpg)
Pipeline: Structure from Motion
Keypoint Detection Bundle AdjustmentKeypoint Matching
Detect features using SIFT[Lowe, IJCV 2004]
Refine matches using RANSAC
Energy MinimizationImages
![Page 17: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/17.jpg)
Multi-view Geometry
Camera 1Θ1 = {𝐑𝐑1, 𝐭𝐭1, 𝐟𝐟1, 𝐜𝐜1, … }
Camera 2Θ2 = {𝐑𝐑2, 𝐭𝐭2, 𝐟𝐟2, 𝐜𝐜2, … }
Camera 3Θ3 = {𝐑𝐑3, 𝐭𝐭3, 𝐟𝐟3, 𝐜𝐜3, … }
𝐯𝐯1𝐯𝐯2𝐯𝐯4
𝐯𝐯3
𝐯𝐯5
𝐯𝐯6𝐯𝐯7
𝐕𝐕 = 𝐯𝐯1, … , 𝐯𝐯7
𝚯𝚯 = {Θ1, … ,Θ3}
![Page 18: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/18.jpg)
Bundle Adjustment
• Energy Minimization based Approach for SfM
Θ∗,𝑉𝑉∗ = argminΘ,𝑉𝑉
𝐸𝐸 Θ,𝑉𝑉𝐕𝐕 = 𝐯𝐯1, … , 𝐯𝐯M
𝚯𝚯 = {Θ1, … ,ΘN}
𝑁𝑁 cameras
𝑀𝑀 points
![Page 19: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/19.jpg)
Re-projection Error
𝐸𝐸 Θ,𝑉𝑉 = �𝑖𝑖=1
𝑁𝑁
�𝑗𝑗=1
𝑀𝑀
w𝑖𝑖,𝑗𝑗 𝐩𝐩𝑖𝑖,𝑗𝑗 − Π𝑖𝑖 𝐯𝐯𝑗𝑗 22
Πi 𝐯𝐯𝑗𝑗 projection of point 𝑗𝑗 in camera 𝑖𝑖
w𝑖𝑖,𝑗𝑗 = � 1, camera 𝑖𝑖 observes point 𝑗𝑗0, otherwise
Square?
Camera 1Θ1 = {𝐑𝐑1, 𝐭𝐭1, 𝐟𝐟1, 𝐜𝐜1, … }
Camera 2Θ2 = {𝐑𝐑2, 𝐭𝐭2, 𝐟𝐟2, 𝐜𝐜2, … }
Camera 3Θ3 = {𝐑𝐑3, 𝐭𝐭3, 𝐟𝐟3, 𝐜𝐜3, … }
𝐯𝐯1𝐯𝐯2𝐯𝐯4
𝐯𝐯3
𝐯𝐯5
𝐯𝐯6𝐯𝐯7
𝐩𝐩𝑖𝑖,𝑗𝑗𝐩𝐩𝑖𝑖,𝑗𝑗
𝐩𝐩𝑖𝑖,𝑗𝑗
![Page 20: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/20.jpg)
![Page 21: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/21.jpg)
Photo Tourism
2) Structure from Motion 3) Photo Explorer
Extrinsic and IntrinsicCamera Parameter
3D Point cloud
1) Images from the Web
![Page 22: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/22.jpg)
15,464
76,389
37,383
![Page 23: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/23.jpg)
Reproduced with permission of Yahoo! Inc. © 2005 by Yahoo! Inc.YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.
![Page 24: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/24.jpg)
Photo Tourism
2) Structure from Motion 3) Photo Explorer
Extrinsic and IntrinsicCamera Parameter
3D Point cloud
1) Images from the Web
![Page 25: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/25.jpg)
SfM / Non-Convex Optimization
Why non-linear?• Rotation parameterization• Projection (‘z’-devide)
𝚯𝚯0,𝐕𝐕0
We need a good initialization!𝚯𝚯0,𝐕𝐕0
𝐸𝐸 Θ,𝑃𝑃 = �𝑖𝑖=1
𝑁𝑁
�𝑗𝑗=1
𝑀𝑀
w𝑖𝑖,𝑗𝑗 𝐩𝐩𝑖𝑖,𝑗𝑗 − Π𝑖𝑖 𝐯𝐯𝑗𝑗 22
![Page 26: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/26.jpg)
Incremental SfM• Select two “suitable” images/cameras
• Find camera parameters using the 8-Point algorithm / RANSAC
• Iterate:• Add one camera using the Direct Linear Transform (DLT)
• Bundle Adjustment (Levenberg Marquardt)
• Prune “bad tracks” after each iteration
![Page 27: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/27.jpg)
Incremental Structure from Motion
![Page 28: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/28.jpg)
Scale, Rotation, Position Ambiguity
Scale
Rotation
Translate
![Page 29: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/29.jpg)
Geo-Registration
![Page 30: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/30.jpg)
Photo Tourism
2) Structure from Motion 3) Photo Explorer
Extrinsic and IntrinsicCamera Parameter
3D Point cloud
1) Images from the Web
• Browsing• Rendering• Annotation
![Page 31: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/31.jpg)
3D Browsing
![Page 32: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/32.jpg)
Object-based browsing
![Page 33: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/33.jpg)
Object-based browsing• Visibility
• Resolution
• Head-on view
![Page 34: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/34.jpg)
Relation-based browsing
Zoom in
Zoom out
Move left Move right
Find all details Find all zoom outs
Find all similar images
![Page 35: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/35.jpg)
Relation-based browsing
![Page 36: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/36.jpg)
Relation-based browsing
Image A
Image C
Image D
Image B
to the right of
is detail ofis zoom-out of is detail of
to the left of
is zoom-out of
is detail of
is detail of is zoom-out of
![Page 37: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/37.jpg)
Photo Tourism
2) Structure from Motion 3) Photo Explorer
Extrinsic and IntrinsicCamera Parameter
3D Point cloud
1) Images from the Web
• Browsing• Rendering• Annotation
![Page 38: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/38.jpg)
Rendering Transitions
![Page 39: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/39.jpg)
Rendering Transitions
Camera A Camera B
![Page 40: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/40.jpg)
Photo Tourism
2) Structure from Motion 3) Photo Explorer
Extrinsic and IntrinsicCamera Parameter
3D Point cloud
1) Images from the Web
• Browsing• Rendering• Annotation
![Page 41: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/41.jpg)
Annotations
![Page 42: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/42.jpg)
Annotations
Reproduced with permission of Yahoo! Inc. © 2005 by Yahoo! Inc.YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.
![Page 43: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/43.jpg)
Limitations1. Some photos can not be registered
• Reason: blur, noise, no overlap, dynamic objects, …
2. Runtime Speed (up to several weeks)
• Fix: Hierarchical approaches, Parallelization, …
![Page 44: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/44.jpg)
Building Rome in a Day
The old city of Dubrovnik, 4,619 images, 3,485,717 points
![Page 45: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/45.jpg)
Small Detour: Bundle Fusion
![Page 46: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/46.jpg)
Limitations1. Some photos can not be registered
• Reason: blur, noise, no overlap, dynamic objects, …
2. Runtime Speed (up to several weeks)
• Fix: Hierarchical approaches, Parallelization, …
3. Transitions do not look “real”
• Better Image-based or Neural Rendering?
![Page 47: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/47.jpg)
Enhancing Transitions
![Page 48: Structure from Motion · 2019-06-06 · Structure from Motion “3D pointcloud + cameras” • Bundle Adjustment “a way to perform SfM” • Multi-view Stereo “dense reconstruction](https://reader033.vdocument.in/reader033/viewer/2022042909/5f3c1eb4bb7aac09127b5287/html5/thumbnails/48.jpg)
Summary
• 3D is important
• Perspective Cameras
• Structure from Motion
• Photo Tourism