Fastest large volume3D inspection in motion MotionCam-3D

Control 2019 Marcel Svec

#MotionCam-3DFastest large volume

3D inspection in motion

Goal: 3D scan of a large volume in high quality and in motion

3D Sensing Trade-offs● Accuracy

● Resolution

● Speed (static vs. dynamic scene)

● Robustness to light conditions

● Large depth of field

● Large field of view

● Form-factor

● Ease of use

Static scenes - structured light

1- https://azttm.wordpress.com/2011/04/03/kinect-pattern-uncovered/2- J. Geng, "Structured-light 3D surface imaging: a tutorial," Adv. Opt. Photon. 3, 128-160 (2011).

Multiple line patterns (sequence of frames)+ High resolution+ High quality+ Reasonable scanning volume and robustness to ambient light- Can’t be used for scanning in motion

https://www.researchgate.net/figure/Binary-structured-light-patterns-The-codeword-of-a-point-p-is-created-by-the-successive_fig8_286924257

3D/DepthMap Texture Normals

Dynamic scenes - parallel structured light

Time-of-flight Active stereo Structured patterns / dots

Parallelstructured light

Low resolutionLow accuracy

Trade-off reliability vs resolution

Low resolutionLow quality features

High resolutionHigh accuracyHigh quality

MicrosoftSonyPMD

AMS/HeptagonPanasonic

Odos/RockwellEspros

AppleQualcomm/Himax

Mantis VisionOrbbec

IntelZED

Ensenso/IDSPhotoneo

Existing methods Photoneo

We parallelize sequential structured light

World’s best 3D Camera

● Scanning in motion● Novel technology - Parallel Structured Light

○ Custom CMOS image sensor○ Rolling => Global => Mosaic Shutter

● 10x Higher Resolution & Accuracy● No Motion Blur● Works Outdoors● Multiple Devices Simultaneously● Optimal Light Efficiency● Rapid Acquisition - up to 60 FPS● Patented

Photoneo CMOS image sensor

Photoneo 3D Camerais superset of Photoneo 3D Scanner

One shot mode (default)

● Resolution: 500K points● 1 frame

○ In motion

High resolution mode

● Resolution: 2M points● 4 frames

○ Static scene

Software on-the-flyswitch

Every bit of every pixel contributes to the final quality of 3D data. Each 3D point is independent measurement.

Active StereoStructured PatternsTime of Flight

Passive Stereo

3D Camera - Comparison

Mantis Vision

ZED

Intel

Panasonic

Apple

Ensenso

True

res

olut

ion

Precision

Photoneo500K points

30K points

100K points

Axes

are

not

line

ar.

10mm 5mm 1mm 0.1mm

Sony

Parallel structured light

Output / Sensor resolution True resolution Z noise

@ 1m, 50% albedo

Consumer ??? Active stereoCorrelation 1280 × 800 ~5 000 projector dots

(+features from passive stereo) 3mm

Industrial ??? Active stereoCorrelation 1280 x 1024 512 x 410 = ~210 000

(Nyquist-Shannon sampling theorem) 3mm

Consumer ??? Structured dotsRecognition 640 × 480 ~70 000 projector dots 3mm

Consumer ??? Time-of-Flight 512 × 424 256 × 212 = ~54 000(0.5x penalty - heavy spatial smoothing) 3mm

Consumer ??? Time-of-Flight 160 × 120 160 × 120 = ~19 000 3mm

Industrial ??? Time-of-Flight 640 × 480 320 × 240 = ~77 000(0.5x penalty - heavy spatial smoothing) 6mm

Consumer ??? Time-of-Flight 640 × 480 640 × 480 = ~300 000 6mm

Industrial ??? Time-of-Flight 640 x 480 640 × 480 = ~300 000 10mm

Photoneo Parallel Structured Light 1068 × 800(One shot mode) ~500 000 0.5mm

Computational Image Sensor● Photoneo developed and patented a novel, ultimate IS for Computational Photography● It's not just another image sensor with "epsilon improvement" (like lower noise or higher

resolution). It fundamentally changes the way how the image is captured and thus reveals a novel, valuable information.

● Broad applicability:3D Scanning, HDR video, Motion Tracking, Image Deblurring, Compressive sensing

Rolling Shutter Global Shutter Mosaic/Matrix Shutter

Bin Picking / Belt Picking

Automotive, general manufacturing, Logistics, e-Commerce

Palletizing, de-palletizing, machine tending

Online Quality Control and Metrology

People counting, behaviour monitoringVR, AR and ergonomics

www.channelfutures.com

Food processing / handling

Picking, Sorting,

Harvesting

Festo

Thank you

● Effective resolution is limited because:○ Surface is sampled by a projected pattern○ Projected pattern is sampled by camera

=> A surface detail should be at least ~7 camera pixels wide to be resolved by the camera

● Structured dots case● “no dot” area = no information

● Stereo case● “no dot” area = passive stereo

(= poor performance)

Active Stereo / Kinect like devices

Projector dots Camerapixel

No dot means no 3D information