continuous 360 degree real-time 3d laser...
TRANSCRIPT
![Page 1: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/1.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 1
Continuous 360 degree real-time 3D laser scanner
Marko Reimer, Oliver Wulf, Bernardo Wagner
Research Centre L3S andInstitute for Systems Engineering
University of Hannover
![Page 2: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/2.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 2
Motivation
Localization for mobile robots
Path planning for mobile robots
Environment modeling for mobile robots
Goals: Unrestricted autonomous navigation and 3D map building in real-time
3D visualization for human operators
![Page 3: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/3.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 3
Why use 3D data for navigation?
Real indoor environments are often cluttered
Good landmarks like walls are often partially occluded and thus invisible for 2D sensors
Obstacles not within the 2D scanline may exist
Negative obstacles can not be detected in 2D scans
![Page 4: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/4.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 4
Ways to reach the 3.rd dimension
4 Alternative scanning methods
![Page 5: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/5.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 5
Main components of the used hardware
ScanDrive (rotating platform)Capable of continuous rotationNo accelerations
- Less mechanical load- Less power consumption
Data- and Power-Transfer via slip rings
Scalable Processing Box (SPB)Embedded PC running “LiRE” (Linux Realtime Environment) as real-time operating system
![Page 6: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/6.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 6
Result of real-time synchronisation
3D-laser scans consist of data fusion from multiple sensors
(2D laser scanner data, laser scanner orientation and robot position)
Systematic errors can be avoided by exact synchronization
Poor adjustment Good adjustment
![Page 7: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/7.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 7
Software / Sensor architecture
3D Scanner
2D Scanner Rotating plate Odometrie
Chassis3D Gyro
![Page 8: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/8.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 8
Real-time data~32.000 points in ~2.4 sec
Two main aspects encourage the use of a real-time operating system
All gathered data is marked with a synchronized timestamp
Later and offline processing is possible while keeping the data synchronized
Data can be gathered during movement
The data processing is always done within a specified time
The system acts like a sensor ->no higher-level system computations needed
Resulting data can be used for time critical operations
![Page 9: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/9.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 9
Visualization of the 3d data I
As depth image (gray values representing distance)
![Page 10: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/10.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 10
Visualization of the 3d data II
As 3D point cloud (already with colored objects)
![Page 11: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/11.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 11
3D Perception and 2D Maps
3D map representations
Registered 3D point clouds
Digital elevation maps (2½D)
Computational expensive
Combination of 3D perception and 2D map representation by use of Virtual 2D Scans
Full 3D information is not needed for robot navigation
Existing 2D navigation algorithms work more efficient
![Page 12: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/12.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 12
Complexity reduction
Map building requires landmarks (2D)
Path planning requires obstacle information. (2D)
=> Two “different maps” needed
Red points are detected landmarks
(virtual landmark scan)
Blue points are detected obstacles
(virtual obstacle scan)
![Page 13: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/13.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 13
Virtual 2D Scan Real Environment
Virtual 2D Scan3D-point cloud
![Page 14: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/14.jpg)
09 Sep. 2005Marko Reimer 1. Range Imaging Research Day 2005 14
Designed a 3D laserscanner
For use on a mobile platform
Delivering 3D data in real-time
Selectable point density
Developed an
Fast
Efficient
Robust
algorithm for navigation of a mobile system.
Conclusion
![Page 15: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/15.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Robust 3D Measurement with PMD Sensors
“CCD goes 3D !”1st range imaging research day
8 September 2005, Zurich
CMOS
PMDTechnologies GmbHAm Eichenhang 50D-57076 Siegen
phone + 49 271 238538 -800fax + 49 271 238538 [email protected]
![Page 16: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/16.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
TheThe CompanyCompany
THE COMPANYTHE COMPANY
![Page 17: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/17.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Company Profile: PMD Technologies GmbHCompany Profile: PMD Technologies GmbH
- Founded in 2002
- Employees: 20+
- Shareholders: - Audi AG (50%) - ifm electronics (50%)
- Worldwide patents
- Markets: Automotive, Industrial and consumer.
- 1st all-solid-state-3D series-product worldwide
- Camera and chip prototypes available for evaluation of 3D technology
- Automotive: camera systems in field test with several OEMs.
- Awards:- „German future award“ 2002 nominee- Hermes award 2005 for “efector PMD”
![Page 18: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/18.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
BASIC PRINCIPLEBASIC PRINCIPLE
![Page 19: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/19.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
What makes 3DWhat makes 3D--PMD so smart?PMD so smart?
CCD or CMOS 2D-Imagers“Each pixel is a photodiode”
PMD“Each pixel is a complete distance
measurement system”
![Page 20: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/20.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
TTimeime--OOff--FFlight Basic Principlelight Basic Principle
SENDER
RECEIVER
00:00:00
Start
Stop
Distance is measured by determining the turn- around time a light pulse needs to travel from the sender to the target and back to the receiver. With knowledge of speed of light the distance can easily be calculated.
Distance is measured by determining the turnDistance is measured by determining the turn-- around time a light pulse needs to around time a light pulse needs to travel from the sender to the target and back to the receiver. Wtravel from the sender to the target and back to the receiver. With knowledge of ith knowledge of speed of light the distance can easily be calculated.speed of light the distance can easily be calculated.
![Page 21: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/21.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
3D TOF3D TOF-- CameraCamera
Chip measures both intensity and distance (arrival time of light) in each pixel 3D-camera
Chip measures both intensity and distance Chip measures both intensity and distance (arrival time of light) in each pixel (arrival time of light) in each pixel 3D3D--cameracamera
![Page 22: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/22.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
PMDPMD-- Principle: Basic Cross SectionPrinciple: Basic Cross Section
![Page 23: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/23.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
PMDPMD--operationoperation
d=0mPopt_sent
Popt_received
mod_A mod_B
Q_a Q_b
mod_A mod_B
Q_a Q_b
Mod_A, Mod_B A B
Q_A, Q_BA
B
d=2mPopt_sent
Popt_received
mod_A mod_B
Q_a Q_b
mod_A mod_B
Q_a Q_b
Mod_A, Mod_B A B
Q_A, Q_BA
B
d=3.75mPopt_sent
Popt_received
mod_A mod_B
Q_a Q_b
mod_A mod_B
Q_a Q_b
Mod_A, Mod_B A B
Q_A, Q_BA
B
d=5mPopt_sent
Popt_received
mod_A mod_B
Q_a Q_b
mod_A mod_B
Q_a Q_b
Mod_A, Mod_B A B
Q_A, Q_B
AB
d=7.5mPopt_sent
Popt_received
mod_A mod_B
Q_a Q_b
mod_A mod_B
Q_a Q_b
Mod_A, Mod_B A B
Q_A, Q_B
A
B
![Page 24: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/24.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Analysis of AKF (autocorrelation function)Analysis of AKF (autocorrelation function)
λ = 15 m fmod = 20MHz
τ ~ ϕ
Α1 Α2 Α3 Α4Α4
b
a
ϕ
AKF
Signal Phase ϕ: distanceSignal Phase Signal Phase ϕ: ϕ: distancedistance
Amplitude a: signal strength of active illumination: quality of 3D measurement.
Amplitude a: Amplitude a: signal strength of active illumination: signal strength of active illumination: quality quality of 3D measurement.of 3D measurement.
Offset b: gray scale imageOffset b: Offset b: gray scale imagegray scale image
![Page 25: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/25.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Key Innovation of PMDKey Innovation of PMD
1. Non-scanning 3D Arrays.Highly integrated Arrays of individual Range Finding Pixels possible. Low-cost, Robust
2. Classical TOF Range Finders drastically simplified.Complete Range Finder realized in quasi-optical Domain.No HF- Signal Processing.
![Page 26: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/26.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
CHALLENGESCHALLENGES
![Page 27: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/27.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
CHALLENGESCHALLENGES
Need to measure picoseconds. (What is a picosecond?)How much light do I need for a certain resolution?What impact has background light (sun)?
How bright is the sun?What are the problems?
![Page 28: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/28.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
What is a What is a picosecondpicosecond??
speed of light is 3E8 m/s = 2mm/6.67psTOF: 6.67ps resolution required for 1mm precisionspeed of light is 3E8 m/s = 2mm/6.67psTOF: 6.67ps resolution required for 1mm precision
1s 1E-3s 1E-6s 1E-9s 1E-12s
1s 1/1000s 300ps 6.67ps(3GHz) 1mm
1000 1000 1000 1000
![Page 29: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/29.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Predictable measurement precisionPredictable measurement precisionk: contrastS: # signal electronsk*S: # active (demodulated) electronsN: # noise electronsλmod: “length of modulation wave” (=c/fmod)
π⋅λ
⋅⋅
⋅=8N/Sk
1N
1dR mod
phase
What do we need for a high distance accuracy (low dR) ?
k: high high contrast PMD-pixels (+bandwidth), high contrast light source
S: high strong light source, large lens aperture, high pixel sensitivity (good fill factor, large pixels)good reflecting targets, close targets.
N: low low shot noise, low background light, low dark current low camera noise (system noise)
λmod: low (fmod high) high bandwidth.
![Page 30: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/30.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Predictable measurement precisionPredictable measurement precision
π⋅λ
⋅⋅
⋅=8N/Sk
1N
1dR mod
phase
0
0.5
1
1.5
2
2.5
3
3.5
4
50 100 150 200 250 300 350 400distance (cm)
stan
dard
dev
iatio
n (c
m)
PMD[vision] 19k MeasurementPMD[vision] 19k Theory
0
0.5
1
1.5
2
2.5
3
3.5
4
50 100 150 200 250 300 350 400distance (cm)
stan
dard
dev
iatio
n (c
m)
PMD[vision] 19k MeasurementPMD[vision] 19k Theory
![Page 31: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/31.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Influence of sunlightInfluence of sunlight
LED illumination
For most outdoor applications sunlight is much brighter than theactive illumination. This has two bad impacts on the TOF-system:
(1) Shot noise is increased ( distance noise)
(2) The dynamics of the sensor can not handle the sunlight (saturation).
For most outdoor applications sunlight is much brighter than theactive illumination. This has two bad impacts on the TOF-system:
(1) Shot noise is increased ( distance noise)
(2) The dynamics of the sensor can not handle the sunlight (saturation).
![Page 32: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/32.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Influence of sunlight Influence of sunlight –– simple workaroundssimple workarounds
LED illumination
Filter
Filter
LED illumination
LED illumination
Sunlight much brighter than LED illumination. Very poor SNR (active light / background light).
Sunlight much brighter than LED illumination. Very poor SNR (active light / background light).
Optical Spectral Filter:
Transmission only of LED- spectrum. Improvement of SNR by reduction (filtering) of background light.
Optical Spectral Filter:
Transmission only of LED- spectrum. Improvement of SNR by reduction (filtering) of background light.
Burst operation of LEDs:
Improvement of SNR by increase of active peak power.
Burst operation of LEDs:
Improvement of SNR by increase of active peak power.
![Page 33: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/33.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Calculation: Sunlight vs. Active lightCalculation: Sunlight vs. Active light
LED SUNLED 0.3 nm/KTemp. Automotive min -40 °CTemp. Automotive max 120 °CTemp. Automotive range 160 K
48 nm
LED 50 nmFilter 98 nm
Viewing angle 20 deg (diagonal)Range 40 mImage Format 1:1edge of image (object space) 10.0 marea of image (object space) 99 m^2#LEDS 100 LEDsoptical power per LED 0.04 W/LEDtotal optical power of LEDs (CW) 4 Wpower density in object space (CW) 0.040 W/m^2 726 W/m^2 power density of sunpower density in object space (CW, filtered) 0.039 W/m^2 83 W/m^2 power density of sun (filtered)Burst-Operation 5 Factor
20 Wpower desnisty in object space (Burst) 0.20 W/m^2power desnisty in object space (Burst, filtered) 0.19 W/m^2
![Page 34: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/34.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Situation 1: No Filter, No BurstSituation 1: No Filter, No Burst
0.01
0.1
1
10
100
1000
10000
400 500 600 700 800 900 1000 1100
wavelength in nm
optic
al p
ower
den
sity
in m
W/m
2/nm
sunLEDs (4W, 20deg FOV, 40m)
0.04W/m2
750W/m2
ratio sun/active = 18‘000 = 85dBratio sun/active = 18‘000 = 85dB
![Page 35: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/35.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Situation2: Spectral FilteringSituation2: Spectral Filtering
0.01
0.1
1
10
100
1000
10000
400 500 600 700 800 900 1000 1100wavelength in nm
optic
al p
ower
den
sity
in m
W/m
2/nm
0
10
20
30
40
50
60
70
80
90
100
%
sunLEDs (4W, 20deg FOV, 40m)sun, filterednormalized spectral response of PMDspectral filter
0.04W/m2
80W/m2 (improvement of factor 9)
ratio sun/active = 2‘000 = 66dBratio sun/active = 2‘000 = 66dB
750W/m2
![Page 36: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/36.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Situation 3: Spectral Filtering and Burst Operation of Situation 3: Spectral Filtering and Burst Operation of LEDsLEDs
0.01
0.1
1
10
100
1000
10000
400 500 600 700 800 900 1000 1100
wavelength in nm
optic
al p
ower
den
sity
in m
W/m
2/nm
0
10
20
30
40
50
60
70
80
90
100
%
sunLEDs (4W, 20deg FOV, 40m)sun, filteredLEDs (bursted, filtered)spectral filter
0.2W/m2 (factor 5)
83W/m2 (improvement of factor 9)
ratio sun/active = 400 = 52dBratio sun/active = 400 = 52dB
750W/m2
0.04W/m2
![Page 37: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/37.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Influence of sunlight Influence of sunlight -- SummarySummary
LED illumination
Filter
Filter
LED illumination
LED illumination
ratio of sunlight / active
18‘000 = 85dB
2‘000 = 66dB
Filter:-19dB
-33dB
Burst Operation-14dB
400 = 52dB
![Page 38: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/38.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Influence of sunlight Influence of sunlight -- SummarySummary
• Spectral Filtering and burst operation of light source are effective methods to improve „optical“ SNR.
• Reasonable sunlight rejection of these methods: -33dB.
• The remaining ratio of sunlight / active signal is still poor:
• Even the filtered sun is still a factor of 400 brighter than thepulsed LEDs.
Additional sunlight rejection methods have to be found.
• Spectral Filtering and burst operation of light source are effective methods to improve „optical“ SNR.
• Reasonable sunlight rejection of these methods: -33dB.
• The remaining ratio of sunlight / active signal is still poor:
• Even the filtered sun is still a factor of 400 brighter than thepulsed LEDs.
Additional sunlight rejection methods have to be found.
![Page 39: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/39.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Sum up: ChallengesSum up: Challenges
BandwidthSensitivityDynamics and sunlight robustness
… are the key challenges for optical TOF measurement.
![Page 40: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/40.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
TECHNICAL SOLUTIONSTECHNICAL SOLUTIONS
![Page 41: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/41.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Contrast Contrast –– measurementmeasurement
Bandwidth: beyond 100MHzBandwidth: beyond 100MHz
![Page 42: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/42.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Size of PMD Pixels is scalable.Size of PMD Pixels is scalable.
4040µµmm 118118µµmm 180180µµmm 280280µµmm
Due to the scalable size of PMD pixels, good fill factors (>50%) can be achieved, even when integrating additional pixel- functionality like SBI, A2I, Multi- sampling, …
Due to the scalable size of PMD pixels, good fill factors Due to the scalable size of PMD pixels, good fill factors (>50%) can be achieved, even when integrating additional (>50%) can be achieved, even when integrating additional pixelpixel-- functionality like SBI, A2I, Multifunctionality like SBI, A2I, Multi-- sampling, sampling, ……
![Page 43: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/43.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
SBI: Suppression of background illuminationSBI: Suppression of background illumination
SBI
Working Principle of SBI:
The SBI circuitry removes the charge packets of background illumination and dark current from the readout nodes. The complete dynamic range of the readout chain can be used for the active signal.
Working Principle of SBI:
The SBI circuitry removes the charge packets of background illumination and dark current from the readout nodes. The complete dynamic range of the readout chain can be used for the active signal.
no SBI SBI
![Page 44: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/44.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
active SBI (3) active SBI (3) –– resultsresults
no SBI SBIZ10_12P1 vs
Z9_2P2 vs, sbi
01
10100
100015pA
150pA1500pA
0
10
20
30
40
50
60
70
Sta
ndar
dabw
eich
ung
[cm
]
iphot o :iH GL 1:xiphot o [pA]
Standardabweichung Tint=2ms
15pA150pA1500pA
01
10100
100015pA
150pA1500pA
0
10
20
30
40
50
60
70
Sta
ndar
dabw
eich
ung
[cm
]
iphot o :iHGL 1:xiphot o [pA]
Standardabweichung Tint=2ms
15pA150pA1500pA
![Page 45: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/45.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Dynamics Measurement (no SBI)Dynamics Measurement (no SBI)
10-3
10-2
10-1
100
101
102
10-3 10-2 10-1 100 101 102 103
Phasedeviation [°] vs Signal and Backlight
Backlightillumination [W/m²]
Sig
nalil
lum
inat
ion
[W/m
²]
0
2
4
6
8
10
12
14
16
18
20
saturation boundaries
![Page 46: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/46.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Dynamics Measurement (SBI)Dynamics Measurement (SBI)
10-3
10-2
10-1
100
101
102
10-3 10-2 10-1 100 101 102 103
Phasedeviation [°] vs Signal and Backlight
Backlightillumination [W/m²]
Sig
nalil
lum
inat
ion
[W/m
²]
0
2
4
6
8
10
12
14
16
18
20saturation boundaries
SBI
SBI
Factor of 300 = 50 dB !!!
Factor of 300 = 50 dB !!!
![Page 47: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/47.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
SBI SBI -- SummarySummary
LED illumination
Filter
Filter
LED illumination
LED illumination
18‘000 = 85dB
2‘000 = 66dB
400 = 52dB
Filter:-19dB
Burst Operation-14dB
-33dB
SBI: 50dB
LED illumination
Only SBI circuitry allows robust 3d measurement under full sunlight.
Only SBI circuitry allows robust 3d measurement under full sunlight.
Filter + SBI
![Page 48: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/48.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Technical challenges and solutionsTechnical challenges and solutions
Technical challenge
high bandwidth
PMD solution
finger structures,short modulation channels
1cm 67ps >100 MHz
high dynamics (sunlight) active SBI, (spectral filter, LED burst)
0.5-5m, r=0.05..1, 10levels 86dBplus sunlight: up to >> Factor 100
01
10100
100015pA
150pA1500pA
0
10
20
30
40
50
60
70
Stan
dard
abw
eich
ung
[cm
]
iphot o :iHGL 1:xiph ot o [pA]
Standardabweichung Tint=2ms
15pA150pA1500pA
01
10100
100015pA
150pA1500pA
0
10
20
30
40
50
60
70
Stan
dard
abw
eich
ung
[cm
]
iphot o :iHGL 1:xip hot o [pA]
Standardabweichung Tint=2ms
15pA150pA1500pA
50 dB
high sensitivity scalable pixel size
typical photo currents down to few 100 fA50% FF
![Page 49: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/49.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
ADVANTAGES and USPs
ADVANTAGES and USPs
![Page 50: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/50.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Comparison to existing solutionsComparison to existing solutions
Stereo Vision• Shadowing effects• Problems with low contrast scenes• Elaborate computing required• Significant delay times for 3D
calculation• 2 complete cameras PLUS
powerful processor
Laser Scanners• Moving parts• Lots of components• Elaborate mechanics• Expensive• vibration sensitive• slow
PMD is cheaper.PMD has better performance.PMD is smaller in size.
![Page 51: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/51.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
AdvantagesAdvantages
Suppression of background illumination (SBI)Various sensor resolutions (pixels scalable in size/sensitivity)
16x16, 32x24, 64x16, 64x48,160x120Highest sensitivityHighest bandwidthHighest contrast
![Page 52: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/52.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
PRODUCTSPRODUCTS
![Page 53: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/53.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
efectorefector PMDPMD
1D Distance sensor based on PMD principle1D Distance sensor based on PMD principleSensor introduced on HMI 2005Sensor introduced on HMI 2005current pricing (2005): 248,current pricing (2005): 248,-- €€
![Page 54: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/54.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Cameras: PMD[vision]®Cameras: Cameras: PMD[visionPMD[vision]]®®
Camera platform: FireWire; Ethernet; SC520 133 MHz CPU, FPGA Spartan II, eLinOS operating systemAvailable Sensors: 64x48 SBI, 64x16 SBI, 160x120 (SBI: Suppression of background illumination)
PMD [vision]®
![Page 55: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/55.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
SENSORS: SENSORS: PhotonICsPhotonICs®® PMD PMD
PhotonICs® PMD 1k-S23D Video Sensor Array with Active SBI64x16 Pixels
PhotonICs® PMD 19k3D Video Sensor Array160x120 Pixels, QQVGA
PhotonICs® PMD 3k-S3D Video Sensor Array with Active SBI64x48 Pixels
![Page 56: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/56.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
ApplicationsApplications
APPLICATIONSAPPLICATIONS
![Page 57: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/57.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Markets and ApplicationsMarkets and Applications
Industrialsafetysurveillanceobject measurementobject recognition…
Industrialsafetysurveillanceobject measurementobject recognition…
Automotivepedestrian safetyACC stop & go (ACC)collision avoidanceparking aidemergency breakman-machine interface (gesture recognition)smart airbag...
Automotivepedestrian safetyACC stop & go (ACC)collision avoidanceparking aidemergency breakman-machine interface (gesture recognition)smart airbag...
RoboticsnavigationAGV ‘scollision avoidance…
RoboticsnavigationAGV ‘scollision avoidance…
Multimediavirtual realityuser interfacesgames video conferencingfilm & televisionproduct presentation
» ...
Multimediavirtual realityuser interfacesgames video conferencingfilm & televisionproduct presentation
» ...
![Page 58: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/58.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Background replacementBackground replacement
original data background replaced
![Page 59: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/59.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Parking AssistantParking Assistant
![Page 60: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/60.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Rear View CameraRear View Camera
![Page 61: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/61.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Pedestrian SafetyPedestrian Safety
![Page 62: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/62.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Gesture ControlGesture Control
![Page 63: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/63.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Keyless DialingKeyless Dialing
![Page 64: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/64.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
Advantages of 3D for Gesture RecognitionAdvantages of 3D for Gesture Recognition
Classical 2D-camera, no depth information
Additional information of a 3D-camera
![Page 65: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/65.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
3D Siam3D Siam-- ProjektProjekt
3D-Siam 3D-Sensorik für vorausschauende Sicherheitssysteme im Automobil.
BMBF- Projekt (08/2001 bis 07/2005)
Fachhochschule Trier
![Page 66: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/66.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
3D3D--Siam Test CarSiam Test Car
IR-Sender
PMD-Kamera
![Page 67: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/67.jpg)
……discovering new dimensionsdiscovering new dimensions© PMDTec GmbH 2005 Dr. R. Lange
THANKS FOR YOUR ATTENTIONTHANKS FOR YOUR ATTENTION
![Page 68: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/68.jpg)
Systematic Investigation of Properties of PMD-Sensors
![Page 69: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/69.jpg)
Systematic Investigation of Properties of PMD-Sensors
1st Range Imaging Research Day, Zürich
Jens Pannekamp, Alexander Steitz
![Page 70: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/70.jpg)
Motivation
• Tremendous success of machine vision(2005: 1 billion € for machine vision in Germany)
• Many real world interactions need 3D!
• Abundant applications in automation & robotics, mobile systems and safety & security
• Compact range sensors become available(PMD Technologies, CSEM, Canesta, 3DV Systems, Matsushita)
Guide for evaluation and selection of 3D-sensors
![Page 71: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/71.jpg)
Applications for 3D-Sensors
• Driver assistence (PMD Technologies)
• Guidance (Claas)
• Collision avoidance (Götting)
• Consumer tracking (Vitracom)
• Bin picking (3 x IPA)
![Page 72: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/72.jpg)
Investigated Range Sensor
• PMD[vision] 1k-S of PMD Technologies
• Sensor array with 64 x 16 pixels
• Suppression of background illumination
• Visualization software CamVis Pro
Data sheet
• Max. range: 7.5 m
• Z-resolution: > 6 mm
• Field of view: 34° (h) x 12° (v)
• Framerate: up to 50 fpsSource: PMD Technologies, Siegen
![Page 73: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/73.jpg)
Example Images
• Upper image:Intensity (grayscale) image of hand in front of chess board
• Lower image:Distance image of same scene – segmentation not disturbed by surface properties
• Problems in practice:Distance values not independent of surface (material, color, texture)
![Page 74: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/74.jpg)
Experimental Setup
• Camera on vertical stage
• 11 materials on plates of size 35 x 35 cm2 :paperaluminium (smooth and sand blasted)steel (smooth and sand blasted)plastics (smooth and rough)ceramic tile (smooth and rough) woodcloth
• Tiltable mount for exchangeable plates
![Page 75: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/75.jpg)
Calibration and Image Evaluation Procedure
• Usage of calibration tool „CamVis Pro“
• Calibration distance: 50 cm
• Calibration material: white paper
• Manual optimization of shutter and integration time for each series of 50 images
• Optimization criterion: Reduction of visible noise in image
• Typical integration time: 10 000 µsec = 0.01 sec
• Manual selection of ROI for relevant pixels, ie. pixels measuring plate
![Page 76: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/76.jpg)
Series of Experiments
a. Influence of distance and materialDistances = {50, 100, 200, 400} cmMaterials = {all materials}50 images for each combination with optimized shutter and integration time
b. Linearity of distance measurementDistances = {50, 100, 200, 400, 550} cmMaterials = {paper, sand blasted aluminium, smooth steel}50 images for each combination with optimized shutter and integration time
c. Estimation of tilt angleTilt anlge = {0°, 5°, 10°, 15°, 30°, 45°, 60°}Distances = {200 cm}Materials = {paper, sand blasted aluminium, smooth steel}50 images for each combination with optimized shutter and integration time
![Page 77: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/77.jpg)
Camera Properties
1. Systematic error for single pixel”Mean that would result from an infinite number of measurements of the same measurand carried out under repeatability conditions minus a true value of the measurand.”
2. Standard uncertainty for single pixel”Uncertainty of the result of a measurement expressed as a standard deviation.”
3. LinearityThe maximum deviation of any points from a straight line drawn as a "best fit"through the point cloud.
4. Estimation of tilt angleTilt angle is extracted from ROI by means of a best fit algorithm and compared to the true value.
![Page 78: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/78.jpg)
Systematic Error: Derivation
For each distance/material combination of measurement series A, distance values of all 50 images and all relevant sensor pixels are averaged. The difference between this averaged value and the true distances is considered as systematic error.
![Page 79: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/79.jpg)
Systematic Error
![Page 80: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/80.jpg)
Systematic Error: Data
Systematic Error [mm] Distance [mm]Material 500,00 1000,00 2000,00 4000,00sand blasted steel 44,87 43,99 169,98 94,76rough ceramic tile 31,80 54,74 187,29 143,75smooth ceramic tile 19,58 42,47 163,66 103,83rough plastics 40,71 41,26 172,95 98,93smooth plastics 28,78 46,95 186,05 92,50smooth steel 9,83 10,48 161,81 117,77smooth aluminium 57,30 62,02 163,30 71,91sand blasted aluminium 7,07 1,36 172,09 194,10wood 36,07 57,91 203,53 142,26paper 3,58 5,25 190,35 196,24cloth 30,59 50,76 188,56 145,99
![Page 81: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/81.jpg)
Systematic Error: Interpretation
• Short distances (50 cm – 100 cm):0.5 cm .. 6 cmDiffuse scattering materials (paper, sand blasted aluminium) perform best
• Longer distances (200 cm – 400 cm):7 cm .. 20 cmSmooth surfaces (smooth aluminium, smooth plastics) perform bestDiffuse scattering surfaces probably do not return enough light (?)
• Material and surface roughness determine quality of distance measurement
• High errors – especially for longer distances – can be attributed partially to calibrationat 50 cm
• Potential for improving (one-point) calibration
![Page 82: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/82.jpg)
Standard Uncertainty: Derivation
For each distance/material combination of measurement series A, the standard deviation is calculated over all 50 images and all relevant sensor pixels.
(Different approach in paper:First, for each distance/material combination of measurement series A an average range image is calculated from the 50 images. Then, the standard deviation is calculated over all relevant sensor pixels.)
![Page 83: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/83.jpg)
Standard Uncertainty
![Page 84: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/84.jpg)
Standard Uncertainty: Data
Standard Uncertainty [mm]Material 500,00 1000,00 2000,00 4000,00sand blasted steel 22,81 19,88 12,82 21,65rough ceramic tile 19,21 18,14 34,24 95,62smooth ceramic tile 27,34 22,49 16,69 31,43rough plastics 21,11 24,94 46,86 128,36smooth plastics 38,26 42,81 74,38 170,09smooth steel 32,46 25,50 22,32 35,00smooth aluminium 115,33 68,32 56,56 30,64sand blasted aluminium 19,75 18,42 23,23 57,32wood 18,33 19,26 21,25 50,36paper 18,72 22,47 26,66 85,33cloth 18,51 20,28 23,45 65,84
Distance [mm]
![Page 85: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/85.jpg)
Standard Uncertainty: Interpretation
• Standard uncertainty for single pixel: 2 cm .. 12 cm
• Plastics performs badly for longer distances (13 cm .. 17 cm)
• For diffuse scatterers (wood, paper, cloth) uncertainty rises from low levels
• Materials with good and near constant performance: sand blasted steel, smooth ceramic tile, smooth steel
• Smooth aluminium with inverse behaviour
• For deeper understanding: measurement of roughness and optical properties
• Standard uncertainty can be reduced by ost-processing (smoothing, median filtering)
![Page 86: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/86.jpg)
Linearity: Derivation
The linearity is derived from plots of measurement series B depicting measured distance values versus true distances. The maximum deviation between any measured distance value and the regression line obtained by a least squares fit is taken as linearity. The distance values are obtained by averaging over all 50 images and all relevant sensor pixels.
![Page 87: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/87.jpg)
Linearity
![Page 88: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/88.jpg)
Linearity: Data
Linearity as Deviationfrom Regression Line [mm]Material 500,00 1000,00 2000,00 4000,00 5500,00sand blasted aluminium 17,30 42,28 77,42 3,86 21,70smooth steel 30,03 24,62 91,47 39,47 2,65paper 10,32 46,89 84,95 23,14 4,60
Distance [mm]
![Page 89: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/89.jpg)
Linearity: Interpretation
• Data shows only little deviations from linear behaviour
• Suggestion for calibration: Removal of systematic errors by means of linear regression
• Remaining non-linear errors 0.3 cm .. 9 cm for measuring range 50 cm .. 550 cm
• Remaining relative errors 1% .. 5%But: This calibration depends on material!
![Page 90: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/90.jpg)
Estimation of Tilt Angle: Derivation
The tilt angle of the plate is estimated from the range images of measurement series C. First, all 50 images of one tilt angle/material combination are averaged. Then, a plane is fitted to the data by means of a least squares fit. The derived orientation of this plane is compared to the true tilt angle.
![Page 91: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/91.jpg)
Estimation of Tilt Angle
![Page 92: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/92.jpg)
Estimation of Tilt Angle: Data
Tilt Angle [°] Reference Angle [°]Material 0,00 5,00 10,00 15,00 30,00 45,00 60,00sand bl.alum. -4,27 1,00 8,22 16,53 40,73 58,19 60,53paper -3,40 1,61 5,88 13,72 32,01 50,32 67,90smooth steel -5,12 4,07 8,33 15,61 42,14 57,39
![Page 93: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/93.jpg)
Estimation of Tilt Angle: Interpretation
• Tilted surfaces can be measured up to at least 60°
• Paper shows a near linear behaviour
• Number of available pixels for plane fit decrease with tilt angle
![Page 94: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/94.jpg)
ConclusionSystematic error and standard uncertaintySystematic error and standard uncertaintySystematic error and standard uncertaintySystematic error and standard uncertainty•••• For short distances (50 cm, 100 cm) diffuse scattering materials such as paper or sand
blasted aluminium perform best (systematic error 0,4 cm resp. 0,7 cm).•••• For longer distances (200 cm, 400 cm) smooth surfaces such as smooth aluminium or
smooth plastics yield best results (systematic error 7 cm resp. 9 cm).•••• Standard uncertainty ranges from 1 cm to 17 cm and depends strongly on material and
distance
LinearityLinearityLinearityLinearity•••• If systematic errors are removed by means of a linear regression, the remaining non-
linear errors are in the range of 0,3 cm to 9 cm for a measuring range of 50 cm to 550 cm. This corresponds to relative errors ranging from less than 1% up to 5%.
Tilted surfacesTilted surfacesTilted surfacesTilted surfaces•••• Tilted surfaces can be measured up to at least 60°.
![Page 95: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/95.jpg)
Open Questions• Investigation how systematic error and uncertainty depend on surface parameters
(roughness, optical properties, color etc.)
• Comparison of cameras of different manufacturers
Acknowledgement
The technical support of PMD Technologies is strongly appreciated.
![Page 96: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/96.jpg)
Research Project „SmartVision“
• Project in German InnoNet-Program
• Start: January 2006
Objectives
• Development of algorithms for object tracking and object recognition(IWR, IPA, 3Soft)
• Applications in the fields of automation & robotics, mobile systems and safety & security
![Page 97: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/97.jpg)
Unified analysis of the performanceand physical limitations of opticalrange-imaging techniques
Peter SeitzVice President, CSEM SA, ZurichProfessor of Optoelectronics, University of Neuchâtel
Range Imaging Days, ETH Zurich, September 8-9, 2005
![Page 98: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/98.jpg)
Peter Seitz :: 05.10.2005 :: Page 2
Content
§ Introduction and motivation
§ Taxonomy of optical range imaging techniques
§ Limitations of the generation and detection of light
§ Linear shift-invariant (LSI) systems
§ A unified framework for optical RIM system analysis
§ Quantum noise limited phase measurement
§ Comparison of RIM system performance limitations
§ Conclusions
§ Summary
![Page 99: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/99.jpg)
Peter Seitz :: 05.10.2005 :: Page 3
![Page 100: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/100.jpg)
Peter Seitz :: 05.10.2005 :: Page 4
![Page 101: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/101.jpg)
Peter Seitz :: 05.10.2005 :: Page 5
Taxonomy of optical range imaging techniques
Interferometry Thomas Young (1801)
lens
light source
rotatingcogwheel
observer
beam splitter planemirror
lens
lens
lens
lens
light source
rotatingcogwheel
observer
beam splitter planemirror
lens
lens
lens
Time-of-flight ranging Armand Fizeau (1849)
Triangulation Egyptian surveyors (1400 B.C.)
![Page 102: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/102.jpg)
Peter Seitz :: 05.10.2005 :: Page 6
Optical setups of the basic ranging methods
x
α
z
Triangulation
z
Interferometry
z
Time-of-flight
![Page 103: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/103.jpg)
Peter Seitz :: 05.10.2005 :: Page 7
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0 5 10 15 20 25
Photon number
Pro
babi
lity
Poisson
Bose-Einstein
n=10
Limitations of light generation and detection (I)
Generation and detection of light are statistical, noisy processes: - Coherent light (independent random events): Poisson statistics - Thermal light (correlated events): Bose-Einstein statistics
NPoisson =σ
2NN
EinsteinBose
+
=−σ
![Page 104: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/104.jpg)
Peter Seitz :: 05.10.2005 :: Page 8
1.E+00
1.E+02
1.E+04
1.E+06
1.E+08
1.E+10
1.E+00 1.E+02 1.E+04 1.E+06 1.E+08 1.E+10
Number of photons
Sta
nd
ard
dev
iati
on
Poisson Bose-Einstein
Limitations of light generation and detection (II)
![Page 105: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/105.jpg)
Peter Seitz :: 05.10.2005 :: Page 9
Linear shift invariant (LSI) systems
LSI
In the complex function space, harmonic functions are eigenfunctions of all LSI systems. The function system of the harmonic functions is complete in the linear space of square integrable functions
![Page 106: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/106.jpg)
Peter Seitz :: 05.10.2005 :: Page 10
Unified framework for optical RIM system analysis
x
z
x
P
z
P
t
Pλ/2 1/fΛ
α
z
z
πϕ
α 2tanTRIz
Λ=
πϕλ22
INTz =π
ϕ22TOF
fc
z =
![Page 107: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/107.jpg)
Peter Seitz :: 05.10.2005 :: Page 11
Quantum noise limited phase measurement (I)
s
F(s)
B
A
ϕ
S=1/f
∆s
Fi
−−
=
+++=
−+−=
20
13
3210
220
213
arctan
4
2)()(
FFFF
FFFFb
FFFFa
ϕ
Ss
sb
Bs
aA
∆=
∆=
∆=
πδ
δδ
sin
![Page 108: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/108.jpg)
Peter Seitz :: 05.10.2005 :: Page 12
s
F(s)
B
A
ϕ
S=1/f
∆s
Fi
Quantum noise limited phase measurement (II)
ab
sF k
k k 2
23
0
=
∂∂
= ∑=
ϕσϕ
variances si = Fi
![Page 109: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/109.jpg)
Peter Seitz :: 05.10.2005 :: Page 13
Comparison of RIM system performance limitations
πσ
ασ ϕ
2tan,Λ
=TRIz π
σλσ ϕ
22, =INTz π
σσ ϕ
22, fc
TOFz =
l
b
αΛ
z
xx
bbz
l==
αtan
Triangulation
z
Interferometry Time-of-flight
![Page 110: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/110.jpg)
Peter Seitz :: 05.10.2005 :: Page 14
Extending the non-ambiguity range
21
2112 Λ−Λ
ΛΛ=Λ
The non-ambiguity range is not restricted to half of the wavelength: By using two different wavelengths Λ1 and Λ2 a synthetic wavelength Λ12 is produced, leading to a much larger non-ambiguity range
![Page 111: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/111.jpg)
Peter Seitz :: 05.10.2005 :: Page 15
Conclusions and Summary
§ Photon quantum noise is one of the key factors limiting the performance
of optical range imaging methods (diffraction, speckle, etc.)
§ Measurement = Comparison with known common standard. Conclusion:
Three fundamental methods: Triangulation, interferometry, time-of-flight
§ Unified analysis of all optical range imaging methods in the framework of
LSI systems: Identical functional relationship – phase measurement.
§ Phase measurement precision limited by photon quantum noise
§ Largely unexplored notion: Combination of various methods
§ The different methods have more in common than one might think.
Conclusion: Share approaches, insights and ideas – work together !
![Page 112: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/112.jpg)
T h a n k y o u f o r y o u r a t t e n t i o n.
![Page 113: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/113.jpg)
CCD/CMOS Lock-In Pixel for Range ImagingChallenges, Limitations and State-of-the-Art
Bernhard BuettgenCSEM, Centre Suisse d’Electronique et de Microtechnique SA
September 8, 2005
![Page 114: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/114.jpg)
Overview
Phase measurement for range imaging
Shot-noise limitation
Pixel concepts
Distance accuracy
Dynamic range
Background-light suppression
Enhanced pixel structures
Summary
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 1
![Page 115: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/115.jpg)
Homodyne phase measurementIntensity Modulation:
Pulse: direct ‘’time-of-flight’’
Continuous Wave: phase delay between emitted and detected signal
Simultaneous acquisition of depth information and brightness
pixelmatrixL
objects
Popt
illumination
sensor readout andimage display
start
stop
ϕπ
⋅⋅
=modf
cL4
c: speed of light
fmod: modulation frequency
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 2
![Page 116: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/116.jpg)
Pixel-Inherent Sampling
Emitted signal
Detected signal
Measurements: four sampling points A0 ... A3
)2sin()( 00 tfPPtP modopt π⋅+=
)2sin()( ϕπ +⋅+= tfABtN modmeasmeasel
t
Nel
A0
A1
A2
A3
Popt
emitted signal
detected signal
ϕ Bmeas
Ameas
P0
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=31
20arctanAAAA
ϕ
Brightness information
Accuracy information
Distance information
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 3
![Page 117: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/117.jpg)
Figure of Merit: Demodulation Contrast
sig
meas
AA
c =demod
Measured signal amplitude [#e]
Signal offset [#e]
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 4
![Page 118: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/118.jpg)
Physical Limitation by Photon Shot Noise
sigL Ac
Bfc
⋅⋅
⋅⋅=
demodmod 24πσ
t
Nel
A0
A1
A2
A3
Popt
emitted signal
detected signal
ϕ Bmeas
Ameas
P0
c: light velocity [m/s]
fmod: modulation frequency [Hz]
cdemod: demodulation contrast [-]
B: intensity [#e]
A: amplitude [#e]
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 5
![Page 119: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/119.jpg)
Pixel Concepts
1-Tap 2-Tap 4-Tap N-Tap
Exploitation of optical energy
50% lost 100% used 100% used 100% used
Mismatch problems No Yes Yes Yes
Fill-factor +++ ++ - ---
Number of acquisitions 4 2 1 1
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 6
![Page 120: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/120.jpg)
2-Tap Pixel
Two samples per
acquisition
CCD-arrangement for
sampling the light
CMOS read-out amplifiers
Charge transport mainly
based on diffusion
processes
Buried channel enhances
the performance
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 7
![Page 121: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/121.jpg)
Challenge I: Distance Accuracy
Precise 3D measurements in real-time without …increasing the total emitted optical power (keeping eye safety)increasing the optics aperture (keeping the systems small)using electronics with better noise performance(keeping the cost level low)
Goal: Std.=1mm or even below!
Enabling new applicationsBiometricsIndustrial packaging
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 8
![Page 122: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/122.jpg)
Accuracy – Limitationssig
L AcB
fc
⋅⋅
⋅⋅=
demodmod 24πσ
104 105 1060
0.5
1
1.5
2
2.5
3
mean number of photo-generated electrons per sample [-]
accu
racy
[cm
]
fmodsweep from 10MHzto 80MHz
Assumptions:
- samples shifted by 90 degrees
- demodulation contrast of 64%
- no delays in the charge transport
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 9
![Page 123: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/123.jpg)
Accuracy – Modulation Frequency
Charge transport dominated by …
… drift processes … diffusion processes
Ultdrift µ
2
=Dltdiff
2
=
110 3V U:Exp. ≈⇒=drift
diff
tt
Higher cut-off frequencies using drift fields!
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 10
![Page 124: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/124.jpg)
Challenge II: Dynamic Range
3D-images are composed of objects at
short distances with low and high reflection coefficients
long distances with low and high reflection coefficients
Pixel capability of measuring all distance between the longest and shortest
distances
No saturation!
Minimum standard deviation of the distance measurement!
sigL Ac
Bfc
⋅⋅
⋅⋅=
demodmod 24πσ
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 11
![Page 125: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/125.jpg)
Dynamic Range – Limitations
Dynamic range sets the minimal and maximal storable amount of
charges into relation!
Minimum amount of charges
defines the smallest uncertainty of the distance measurement
Maximum amount of charges
defined by the brightest points in the scenery
[ ]⎥⎥⎦
⎤
⎢⎢⎣
⎡⋅⎟⎟
⎠
⎞⎜⎜⎝
⎛⋅=
min
max
2
min
max10log20δδ
RRdBDR
sigL Ac
Bfc
⋅⋅
⋅⋅=
demodmod 24πσ
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 12
![Page 126: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/126.jpg)
Dynamic Range – Example
Distance range: Rmin = 20cm, Rmax = 5m
Reflection coefficients: δ = 5%, δ = 100%
DR = 82dB (Factor of 12500)
Minimum accuracy of 1cm requires 60.000 electrons
Also 750.000.000 electrons must be storable!
sigL Ac
Bfc
⋅⋅
⋅⋅=
demodmod 24πσ
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 13
![Page 127: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/127.jpg)
Challenge III: Background Light
400 500 600 700 800 900 1000 11000
0.5
1
1.5
wavelength [nm]
optic
al p
ower
den
sity
[kW
/m2 / µ
m]
bandwidth
~800kW/m2/µm
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 14
![Page 128: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/128.jpg)
Background-Light Suppression – Limitations
0 5 10 15 200
2
4
6
8
background to signal ratio [-]
accu
racy
[cm
]
signal electrons:1) 100002) 316003) 1000004) 3160005) 1000000
signal power 1
2
34
5
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 15
![Page 129: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/129.jpg)
Enhanced Pixel for High Dynamic Range
Pixel-wise integration (PWI)
Functionality verified
1 1.5 2 2.5 3 3.50
2
4
6
8
real distance [m]
mea
sure
d di
stan
ce [m
] normal modepwi mode
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 16
![Page 130: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/130.jpg)
Enhanced Pixel for High Background Light
Difference of charge packets is processed in-pixel
0 20 40 60 80 100 120 1400
2
4
6
8
10
12
14
ratio background to signal [-]
accu
racy
[cm
]
theorymeasurement
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 17
![Page 131: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/131.jpg)
Summary and Outlook
Smart pixels for fast optical distance measurement based on TOFChallenges for future designs
High accuracy high modulation frequenciesHigh dynamic range adaptive integration timeBackground-light suppression subtraction of charge packets
State-of-the-artEnhanced pixel structures for extended dynamic range and background light suppressionAspects of high modulation frequencies to be investigated!
New pixel architectures are required that enable the charge transport by electric drift fields.
Bernhard Buettgen :: RIM Day 2005 :: ETH Zurich 18
![Page 132: Continuous 360 degree real-time 3D laser scannerwebarchiv.ethz.ch/geometh-data/downloads/RIM_day/Presentations_… · Main components of the used hardware ScanDrive (rotating platform)](https://reader030.vdocument.in/reader030/viewer/2022041112/5f1a942558b5ff2a257bad9b/html5/thumbnails/132.jpg)
T h a n k y o u f o r y o u r a t t e n t i o n.