peter sturm inria grenoble – rhône-alpes (institut national de recherche en informatique et...
TRANSCRIPT
Peter Sturm
INRIA Grenoble – Rhône-Alpes(Institut National de Recherche en Informatique et Automatique)
An overview of multi-view stereoand other topics in
3D modeling from 2D images
Currently: sabbatical at TUM (Humboldt-fellow)
OverviewIngredients and factors of improvement:
• Geometry• Image matching• Statistics
• Better data• Multi-view stereo
Other approaches and related subjects:
• Other 3D modeling approaches• Camera and object tracking• Omnidirectional vision• Sensor design• ...
• Numerical optimization
Current research directions
Before about 1995
[Mohr et al.]
3D modeling from multiple images:
• image matching often done by hand• main focus: given the matching, how to estimate camera positions, camera calibration, 3D point positions?
• 1990’s: golden age of multi-view geometry
Exceptions, concerning manual matching:• binocular stereo, aerial imagery• close-range photogrammetry, mo-cap• 3D modeling from silhouettes
1996
[Beardsley et al.]
The first fully automatic approaches
• automatic matching• systematic use of robust statistics• going towards surface models (here, meshes)
1998
[Fitzgibbon et al.]
Handling longer image sequences
• better numerical optimization methods• system integration• better data: image quality, calibration
From 1998 onwardsGoing from “hacks” to mathematically founded surfacerepresentations and evolutions: level sets (now, increasedusage of meshes)
[Keriven et al.]
About 2003
[Strecha et al.]
• probabilistic occlusion handling
Towards truly dense 3D models:
• point-based surface representations
Quick Summary
• surface representation: point-based, patch-based, mesh, implicit surface, voxel sets, ...
Ingredients of multi-view stereo:
• quality measure: “photo-consistency”, invariance to lighting differences, ...• prior assumptions (regularization): smoothness, piecewise planarity, class-specific priors (e.g. faces)• optimization approach, resolution strategy
[Vetter et al.]
OverviewIngredients and factors of improvement:
• Geometry• Image matching• Statistics
• Better data• Multi-view stereo
• ...
• Numerical optimization
Current research directions
Other approaches and related subjects:
• Other 3D modeling approaches• Camera and object tracking• Omnidirectional vision• Sensor design
[Criminisi et al.]
Other approaches and subjectsInteractive 3D modeling based on user annotations
• single-image 3D modeling
Other approaches and subjectsInteractive 3D modeling based on user annotations
• multi-image 3D modeling
[Debevec et al.]
Other approaches and subjectsExploitation of other depth cues than stereo
• shading• focus / defocus
Active approaches using controlled illumination:
Other approaches and subjects
• Photometric stereo• Flash photography• Structured light• ...
3D Modeling from silhouettes (“visual hull”)
[Hernandez et al.]
Other approaches and subjects
• Possible in real-time
Other approaches and subjectsReal-time camera tracking:
• SLAM (simultaneous localization and mapping)• Model-based object tracking
[Lhuillier et al.]
VIDEO…
Other approaches and subjectsOmnidirectional vision:
• wide angle cameras
• surveillance• navigation
• panoramic imaging
Other approaches and subjects
• Filters for HDR mosaics
• Mirror design
[Nayar et al.]
Some works on sensor design:
• Lens arrays
Other approaches and subjectsSome works on sensor design:
• Filters for HDR mosaics• Lens arrays, e.g. for computational photography
[Georgiev et al.]
OverviewIngredients and factors of improvement:
• Geometry• Image matching• Statistics
• Better data• Multi-view stereo
• Global optimization methods for geometry computations• Use of community image collections• Modeling surface appearance• Modeling deformable / articulated objects• ...
• Numerical optimization
Current research directions:
Other approaches and related subjects
[Snavely et al.]
Current research directions
• PhotoSynth / PhotoTourism
Exploration of community image collections:
[Furukawa et al.]
Current research directions
• Interior of buildings, cities
Detailed 3D modeling of large scenes
VIDEO…
Current research directions
• Tanslucid, refractive, reflective surfaces, smoke, fire, ...
Modeling surface appearance: