marc levoy ibm / ibr “the study of image-based modeling and rendering is the study of sampled...
Post on 22-Dec-2015
215 views
TRANSCRIPT
![Page 1: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/1.jpg)
Marc Levoy
IBM / IBR
“The study of image-based modeling
and rendering is the study of
sampled representations of geometry.”
![Page 2: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/2.jpg)
Marc Levoy
Shortcutting thevision/graphics pipeline
geometry
views
real world
graphicspipeline
image-basedrendering
visionpipeline
(from M. Cohen)
![Page 3: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/3.jpg)
Marc Levoy
Apple QuickTime VR[Chen, Siggraph ’95]
• outward-looking– panoramic views taken at regularly spaced points
• inward-looking– views taken at points on the surface of a sphere
![Page 4: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/4.jpg)
Marc Levoy
View interpolationfrom a single view
1. Render object
2. Convert Z-buffer to range image
3. Re-render from new viewpoint
4. Use depths to resolve overlaps
Q. How to fill in holes?
![Page 5: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/5.jpg)
Marc Levoy
View interpolationfrom multiple views
1. Render object from multiple viewpoints
2. Convert Z-buffers to range images
3. Re-render from new viewpoint
4. Use depths to resolve overlaps
5. Use multiple views to fill in holes
![Page 6: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/6.jpg)
Marc Levoy
Post-rendering 3D warping[Mark et al., I3D97]
• render at low frame rate
• interpolate to real-time frame rate– interpolate observer viewpoint using B-Spline
– convert reference images to polygon meshes
– warp meshes to interpolated viewpoint
– composite by Z-buffer comparison and conditional write
![Page 7: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/7.jpg)
Marc Levoy
Results
• rendered at 5 fps, interpolated to 30 fps
• live system requires reliable motion prediction– tradeoff between accuracy and latency
• fails on specular objects
![Page 8: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/8.jpg)
Marc Levoy
Light field rendering[Levoy & Hanrahan, Siggraph ’96]
• must stay outside convex hull of the object
• like rebinning in computed tomography
![Page 9: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/9.jpg)
Marc Levoy
A light field is an array of images
![Page 10: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/10.jpg)
Marc Levoy
Stanford multi-camera array(Horowitz, Levoy, Hanrahan)
• cheap imagers + cheap optics +fast networking + plentiful computation =
high-performance imagingusing an array of low-cost cameras
![Page 11: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/11.jpg)
Marc Levoy
Applications for the array
• key issue is the spacing andarrangement of the cameras
![Page 12: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/12.jpg)
Marc Levoy
Cameras tightly packed:high-X imaging
• high-resolution– by abutting the cameras’ fields of view
• high speed– by staggering their triggering times
• high dynamic range– mosaic of shutter speeds, apertures, density filters
• high depth of field– mosaic of differently focused lenses
![Page 13: Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”](https://reader035.vdocument.in/reader035/viewer/2022062715/56649d785503460f94a5afda/html5/thumbnails/13.jpg)
Marc Levoy
Cameras widely spaced:a video-rate light field camera
• compression of (video) light fields– key is estimation of rough geometry
• autostereoscopic display of light fields– lens arrays, bumpy mirrored sheets
• shape from light fields– Do more images help?