high-quality scanning using time-of-flight depth superresolution 17nd march 2008 sebastian schuon...
Post on 20-Dec-2015
218 Views
Preview:
TRANSCRIPT
High-quality Scanning usingTime-Of-Flight Depth Superresolution
17nd March 2008
Sebastian Schuonschuon@cs.stanford.edu
Prepared for: Final Presentation CS223B, Winter 2008
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 2
Problem StatementTime-of-Flight (TOF) Cameras Have a Low Resolution
ZCam by 3DV Systems (Israel)
Measurement principle: time of flight
Depth recording: 320x240, 8bit
► Less noisy depth images with higher resolution desired
Native resolution Superresolution (4x) Geometry renderingfrom superresolution
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 3
ApproachCombine Several Images to Increase Resolution
Use multiple, here N=15, recordings from different viewpoints by translating the camera
Estimating the high resolution image resembles to an optimization problem
Optimization is multi-objective: similarity and smoothness is enforced
►
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 4
ResultsSubtle Details Become Visible and Noise is Reduced
De
pth
ma
p3
D g
eo
me
try
ren
de
ring
Native resolution Superresolution (4x)
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 6
Hidden SlideDistribution of Work
Project solely undertaken by Sebastian Schuon (me)
Supervision / Collaboration:Christian TheobaltJames Davis (UCSC)
Additional imagery for paper provided byHylke Buisman
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 7
SuperresolutionGoals
Enhance resolution
Reduce noise
Recording Resolution
(320 x 240)
(Contrast enhanced)
(Contrast enhanced)
Super Resolution(4x upsample)
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 8
Depth CameraTheory
ZCam by 3DV Systems (Israel)
RGB and Depth camera in one housing (“RGBD”)
RGB: 640x320 @ 30fps, Depth: 320x240, 160x120 @ 30fps
Measurement principle: time of flight
Depth image: distance between camera and object (not Z-coordinate)
Unprojection to 3D coordinates necessary
Specification of tracking window required
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 9
Depth CameraResults
Recording scene with different layers of depth
Black, shiny surfaces tend to be problematic
One needs to know where to record (tracking window)
Unprojection of depth images leads to 3D representation
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 10
Depth CameraNoise Characteristic
Outer regions tend to be a lot more noisy
Noise can be approximated with Gaussian
Hypothesis: Noise increases quadratic with distance
Hypothesis : Noise is correlated with color of object recorded
Variance Plot Pixel Distribution over Time at Center
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 11
Depth CameraSystematic Bias
RBG Processing disabled
Variance Plot (RGB disabled)
RBG Processing enabled
Variance Plot (RGB enabled)
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 12
SuperresolutionTheory
Based on Shift-And-Add family of algorithms
Quite well studied for grayscale and color images, overview in [Farsiu04]
We used Bilateral Shift-And-Add [Farsiu03]
Formulation as inverse problem
Our approach: rotating camera
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 13
SuperresolutionResults Simple Approach
Depth Image - Superresolution(Contrast enhanced)
Depth Image - Recording Resolution(Contrast enhanced)
3D Rendering - Recording Resolution
3D Rendering - Superresolution
Sebastian Schuon schuon@cs.stanford.edu Project_26.ptt 14
Conclusion
Findings
Camera Interface and Software still beta / undocumented
Interesting effects can happen, that are not expected
Superresolution on depth images is feasible
Further steps
Reimplementation of known algorithms
Ideas for improvement:– Depth camera specific noise characteristic– Incorporating confidence map
top related