1 image-based visual hulls paper by wojciech matusik, chris buehler, ramesh raskar, steven j....
Post on 20-Dec-2015
215 views
TRANSCRIPT
1
Image-Based Visual Hulls
Paper by Wojciech Matusik, Chris Buehler, Ramesh Raskar,
Steven J. Gortler and Leonard McMillan[http://graphics.lcs.mit.edu/~wojciech/vh/]
Vortrag von Simon DellenbachGDV Fachseminar 2001
2
Overview (1)
• Motivation
• Basics
– Viewpoint Model
– Visual Hull
– Epipolar Geometry
• Creating Image-Based Visual Hulls
3
Overview (2)
• Rendering IBVH
• System Implementation
• Summary & Results
• Future Work
• Personal Opinion
4
Motivation (1)
• Traditional computer graphics, rendering..– static synthetic scenes (CG Images)– dynamic synthetic scenes (CG Animations)– static acquired scenes (Image-Based Rendering)
• Acquire and render dynamic scenes in real-time:– appropriate representation– rendering system
5
Motivation (2)
6
Viewpoint Model - Basics (1)
7
Visual Hull - Basics (2)
• Geometric shape obtained using silhouettes of object seen from number of views:– extruded silhouette = cone-like volume limiting
the extent of object– intersection of volumes results in a visual hull– more views better approximation of object– limitation: concavities can’t be captured
(e.g. an open box looks like a solid cube)
8
Visual Hull - Basics (3)
9
Epipolar Geometry - Basics (4)
• The tree points [COP1,COP2,P] form an epipolar plane
• Intersection of this plane with image planes results in epipolar lines
• The line connecting the two centers of projection [COP1,COP2] intersects the image planes at the conjugate points e1 and e2 which are called epipoles
10
Epipolar Geometry - Basics (5)
11
Creating Image-Based Visual Hulls (1)
• Algorithm input:– set of k silhouettes (binary images) with
associated viewpoints– desired viewpoint (in this case, constructed
visual hull is viewpoint-dependent)
• Algorithm output:– sampled image of the visual hull, each pixel
containing a list of occupied intervals of space
12
Creating Image-Based Visual Hulls (2)
• The Basic Algorithm:– cast ray into space for each pixel in the
desired view of the visual hull– intersect ray with the k silhouette cones
k lists of intervals; intersect together single list of intersections of the viewing ray with the visual hull
13
Creating Image-Based Visual Hulls (3)
14
Creating Image-Based Visual Hulls (4)
• Trick: due to Epipolar Geometry interval calculation can be done in image space of reference images:– 3D: intersecting silhouette cone with viewing
ray– 2D: intersecting projected viewing ray with
silhouette
15
Creating Image-Based Visual Hulls (5)
16
Rendering IBVH (1)
• Reference images are used as textures• For each pixel:
– rank reference-image texture from “best” to “worst” according to angle, take reference with lowest
– avoid texturing surface points with an image whose line-of-sight is blocked by some other point of the visual hull
– consider visibility during shading based on visual hull (not actual geometry)
17
Rendering IBVH (2)
18
Rendering IBVH (3)
19
System Implementation (1)
• Four calibrated and triggered digital cameras
• One desktop PC per camera for capturing and pre-processing video frames (image segmentation)
• Silhouette and texture information sent to central server for IBVH processing
20
System Implementation (2)
• Server runs IBVH intersection and shading algorithms
• IBVH objects can be combined with OpenGL background
• System runs in ‘real time’ with heavy optimization (like caching strategies for silhouette intersection)
21
System Implementation (3)
22
Summary & Results
• Use visual hull as object shape approximation• Using silhouette information from reference
views to generate view dependent visual hull• Reference images are used as ‘textures’
• Results:Videoclips
23
Future Work
• Find Techniques for blending between textures to produce smoother transitions
• Scale up system by using larger number of cameras
• Split workload on multiple servers, as algorithm parallelizes fairly much
• Speed up viewing ray silhouette intersections (most expensive part of the computation)
24
Personal Opinion (1)
• Pros:– simple technique / low-cost hardware– image-based representation partially
compensates simplification problems– epipolar geometry reduces 3D-intersection
problems to 2D-intersections
25
Personal Opinion (2)
• Cons:– texture flipping during viewpoint transitions
produces ugly results– shadows are considered as part of the object– preprocessing is really expensive
(85 ms for image foreground segmentation)
26
The End
“If there are no questions,there won’t be any answers.”
? ? ?
???