real-time depth up-sampling
DESCRIPTION
Real-time depth up-sampling. Hylke Buisman and Derek Chan Supervisor: Christian Theobalt 03-17-2008. Problem description. Primary goals Super resolution and noise removal Input: low-res depth map, high-res color image Output: 3D mesh of up-sampled and denoised depth - PowerPoint PPT PresentationTRANSCRIPT
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Real-time depth up-sampling
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
03-17-2008
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Problem description
• Primary goals– Super resolution and noise removal
• Input: low-res depth map, high-res color image• Output: 3D mesh of up-sampled and denoised depth
– Optimize for (near) real-time processing
• Data samples
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Approach
Bilateral filter affects only areas of similar color
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Approach
• Joint bilateral up-sampling (Kopf et al.):
• Implementations explored:– Bilateral grid (Paris & Durand 2007)– Separable kernel approximation
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Results
+ =
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Results
Naive Optimized Naïve (Color)
Bilateral grid (No color)
SeparableKernel (Color)
GPU
5 s 1 s 0.3 s 0.6 s 75 ms
Runtimes (800 x 600 image)
+
=
Input Result
Hylke Buisman and Derek ChanSupervisor: Christian Theobalt
Designing a 3D Video Camera
Hidden slide• Hylke - 50%
– Alignment:• Extrinsics calibration• Homography between depthmap and color image
– CPU implementation • JBU• Separable kernel
– Bilateral grid • Derek - 50%
– Alignment: • 3d point cloud tesselation• Reprojection into high-res camera
– Median filter– GPU implementation of upsampling– 3D results with texture