Structure Light

Structured Light--- some recent progress

Bo FuUniversity of Kentucky1Paper List1. Structured Light 3D Scanning in the Presence of Global Illumination. (CVPR 11) [systematic error introduced by ill lighting condition] Extension: A Practical Approach to 3D Scanning in the Presence of Inter-reflections, Subsurface Scattering and Defocus. (IJCV 12)

2. Implementing High Resolution Structured Light by Exploiting Projector Blur. (WACV 12) [increase depth resolution]

3. Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light. (IEEE Trans. on Image Processing 2008) [reduce acquisition time]2Structured Light 3D Scanning in the Presence of Global Illumination

Mohit Gupta, etc

Carnegie Mellon University 3IntroductionMotivation One important assumption of most structured light techniques does not always hold: scene points receive illumination only directly from the light source.

4IntroductionMain idea Design patterns that modulate global illumination and prevent the errors at capture time itself.

Gray codeMin-SWXOR-025Errors due to Global IlluminationShort range effect: sub-surface scattering, defocus

Long range effect: inter-reflection, diffusion

61. inter-reflection error

71. inter-reflection errorsolution

82. sub-surface scattering error

92. sub-surface scattering errorsolution

10Error formulation

11Error formulation

: projector defocus fraction12Error formulationCorrect binarization:

Note: without global illumination (Lg =0), defocus ( = 1), this condition automatically holds

13Error formulationLong range effect diffuse and specular inter-reflectionConsequence: low frequency decode error. Since the low frequency pattern correspond to the higher-order bits, this results in a large error in the recovered shape.

14Error formulationShort-range effect sub-surface scattering and defocus

Consequence: loss of depth resolution15Pattern for error preventionHow to design pattern to prevent both long range effect and short range effect? pattern with only high frequencies can prevent long range effect. pattern with only low frequencies can prevent short range effect. It is possible to design code with only high frequency patterns, while for short range effect, patterns with large minimum stripe-width can be designed.16Pattern for error prevention

17Pattern for error prevention1. For long range effect

Base patternGray codeXOR18Pattern for error prevention1. For long range effect

19Pattern for error prevention2. For short range effect Design codes with large minimum stripe-width (min-SW) well studied in combinatorial mathematics [1].

[1] binary gray codes with long bit runs. The electronic journal of combinatorics. 200320Pattern for error prevention2. Ensemble of codes for general scenes using four codes optimized for different effects: XOR-04 and XOR-02 for long range effect Gray code with maximum min-SW and Gray code for short range effect Rule: if any two agree within a small threshold, that value is returned as true depth,. 21ExperimentPlease refer to the paper (IJCV preferred)22Implementing High Resolution Structured Light by Exploiting Projector Blur

CamilloTaylor

University of Pennsylvania 23IntroductionMotivation 1. With standard structured light decoding schemes, one is limited by the resolution of the projector. The quantization of the projector ultimately limits the accuracy of the reconstruction.

24IntroductionMotivation 2. Growing disparity between the resolution of the image sensor and the resolution of the projector systems.

1600*1200640*480VS25IntroductionMain idea By exploit the blur induced by the optics of the projector, subpixel correspondences between camera frame and projector frame can be established

Major comparison: Li Zhang, Shree Nayar. Projection Defocus Analysis for Scene Capture and Image Display. (TOG 06)

26Approach

The effective irradiance that a pixel in the projector contributes to a point in the scene is related to the displacement between the projection of that scene point on the projector frame and the center of the pixel. This falloff is modeled as a Gaussian

I: observed scene radiance measured at a pixel in the cameraf: BRDF at the scene pointE: irradiance supplied to the corresponding scene point by the projector27Approach

k: stripe index: width of the blur kernel at point in the scene: projection of scene point in the projector frame (with subpixel precision)

k

28Approach

I0: scene irradiance due to ambient illumination (known)

: floating point offset between -0.5 and 7.5

29Experiment

For more result, please refer to original paper30Vision Processing for Real-time 3D Data Acquisition Based on Coded Structured Light

S. Y. Chen, etc

City University of Hong Kong31MotivationConventional structured light system can not be applied to moving surfaces since multiple patterns must be projected.32Main ideaGrid solid pattern for 3d reconstruction with fast matching strategies.

33Thank you34