signal processing and analysis - miun.se

W

V

Benny Thörnberg

Associate professor

in electronics

Signal Processing and Analysis3D Image Imaging

Copyright (c) Benny Thörnberg 1:44

Outline

•Scanning shadow shape curves

•Coded light

•Laser triangulation

•Stereo Imaging

•Time Of Flight Imaging


Acquiring shadow shapes using area

scan imaging Illumination lamp

Hi-Light Screen Camera

Silhouette image of frog

1.S. Pansang and C. Kimpan, “3-D object recognition from shadow shape curves”,

Proceedings from 2000 IEEE Asia-Pacific Conference on Circuits and Systems.

Electronic Communication Systems, pp. 437-40, Tianjin China, 2000


3D image acquisition using shadows

Camera

Illumination

Shadow


3D image acquisition using coded light

Projector

Camera

Stripe

patterns

Every point on the surface will be illuminated by a sequence

of light-no-light that is unique for a certain angle of the

projected light.

1.B. Achermann, X. Jiang and H. Bunke, “Face recognition using range

images”, Proceedings. International Conference on Virtual Systems and

MultiMedia VSMM '97, pp. 129-36, Geneva Switzerland, 1997.


2D image acquisition using line scan

Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications

A 1D image sensor can be set up to acquire

a 2D projection of a 3D scene. Motion is then used to capture the second dimension

of the projection.


3D image acquisition using laser

triangulationA 3D representation of the 3D

scene is acquired from using

triangulation with structured

light.

R

X

Y

Sheet of light laser 2-D image sensor

d Scanning in

X-dimension An area scan sensor is used

in combination with motion to

capture the third image

dimension.

Light intensity information can be simultaneously acquired.


Laser triangulation

Laser

H r

Object

Range reference level β

F

-D

D

d

α

rMAX

Projection center of lens

Principal distance

of lens

Φ

S


Laser triangulation

Laser

H r

Object


F

-D

D

d

α

rMAX

Projection center point of lens

Principle distance

of lens

Φ

S

20

)tan()tan(

)tan()tan()tan( πα

α

αα<<∧

+Φ

⋅Φ−=

SHr

20

)tan(

)tan()tan(

πβ

β

β<<∧

+

−=Φ

dF

dF

20

)tan(

))tan(( πβ

β

β<<∧

+

−=

DF

DFHS


Laser triangulation

Laser

H r

Object


F

-D

D

d

α

rMAX

Focal point of lens

Principle distance

of lens

Φ

S

If both the laser and the camera

angle α and β equal 45 degrees, a perfect linearity and

highest SNR will be achieved

while minimizing the risk of

occlusion.

F

SHdSHF

F

dFSdFHr

2

)()(

2

)()( ++−=

−−+=


Laser triangulation


Laser triangulation

Laser

H

r Object


FZD

-D

D d

rMAX

Focal point of lens

Principle distance of

lens

C

Using two imaging

elements and one

orthogonally oriented laser

will reduce the risk of

optical occlusion.

+⋅+

−⋅−−⋅=

))tan(())tan((

))tan(())tan((1

ββ

ββ

dFDF

dFDFHr

ZDZD

ZDZD


Laser triangulation

Laser

H

r Object


FZD

-D

D d

rMAX

Focal point of lens

Principle

distance of

lens

C

+⋅+

−⋅−−⋅=

))tan(())tan((

))tan(())tan((1

ββ

ββ

dFDF

dFDFHr

ZDZD

ZDZD

+⋅+

−⋅−−⋅=

))tan(())tan((

))tan(())tan((1

ββ

ββ

DFDF

DFDFHr

ZDZD

ZDZDMAX

))tan(())tan((

))(tan1( 2

ββ

β

DFDFFH

d

r

ZDZD

ZD−⋅+

+⋅⋅≤

∂

∂

The largest derivate of r with respect to

d will occur when d=-D,

))tan()()tan((

))(tan1( 2

ββ

β

DFDFkdHFr

ZDZD

ZD−+

+⋅⋅∂⋅≤∂


Center Of Gravity - COG

sNoOfColumnc

crI

crIr

cdNoOfRows

r

NoOfRows

r ≤≤

⋅

=

∑

∑

=

= 1,

),(

),(

)(

1

1


Sub-pixel precision analysis

0 0.005 0.01 0.015 0.02 0.02516

18

20

22

24

26

28

30

Exposure time [s]

SN

R [d

B]

SNR vs Exposure time

SNR for a laser source



Speckle noise (multiplicative!)

F-number=5.6 F-number=40


Sub-pixel precision vs Noise

H.C. van Assen, M. Egmont-Petersen and J.H.C. Reiber, “Accurate Object Localization in Gray

Level Images Using the Center of Gravity Measure: Accuracy Versus Precision”, IEEE Transactions

on Image Processing, Vol. 11, No. 12, Dec. 2002


Thresholding


Systematic error from thresholding

0.05 0.1 0.15 0.2 0.25 0.30

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

Threshold

Sta

nd

ard

de

via

tio

n o

f su

b-p

ixe

l e

rro

r

Simulated errorEstimated error

B.F. Alexander and Ng Kim Chew, ”Elimination of systematic error in subpixel accuracy

centroid estimation”, Optical Engineering, Vol. 30, No. 9, Sept. 1991



How to select threshold?

0 0.2 0.4 0.6 0.8 10.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Threshold [Rate of max signal intensity]

Std

.de

v. o

f C

OG

ou

tpu

t

Subpixel precision versus threshold



Conclusions

•Higher values for the intensity thresholding results in lower

standard deviation for computed positions where the uncertainty of COG is stemming mainly from optical noise (noisy image)

•COG computation can benefit from choosing lower values for intensity thresholding if the analysed image has very little of noise.

This is because systematic error can become dominant.

•Threshold level must be selected based on a trade of between

systematic error and image noise


Example – Wood Chip Analyzer

Range Imager

Collector

Chip feeder

W

V

Principles of a Wood Chip Analyzer


A real world wood chip analyzer

Example – Wood Chip Analyzer


Stereo imaging

x2,X2

y2,Y2

z2,Z2

(X2,Y2,Z2)

(x2,y2)λ

λ

(x1,y1)

y1,Y1

x1,X1

z1,Z1

(X1,Y1,Z1)

B

BXX

Zx

X

Zx

X

+=

−=

−=

12

22

2

11

1

)(

)(

λλ

λλ

World coordinate

Acquiring a 3D representation of a 3D

scene is a one to one mapping and can be done using two 2D imaging

elements.


Stereo imaging

ZZZ == 21

Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley

)(

)(

21

11

Zx

BX

Zx

X

−=+

−=

λλ

λλ

12 xx

BZ

−−=

λλ


Stereo imaging

12 xx

BZ

−−=

λλ

This equation implies that two corresponding points x1 and x2 must be found in

two different images of the same 3D scene. In literature, this operation is referred to as correspondence matching. This matching is preferable done

with correlation techniques (compare with Template matching from previous

lecture, SAD, SSD, NCC).

Distance between corresponding interest points x2–x1 give us the Disparity

map. Interest point = spatially local point with high gradient intensity.

Compute interest points � scan for pairs of similar interest points � compute

distance between pairs

Reference: C. Georgoulas et.al, “Real-time disparity map computation module”, Microprocessors and Microsystems,

32 (2008), Elsevier


Stereo imaging

Left side camera Right side camera

Disparity map

Reference: image data set at Stanford Vision Lab

http://vision.stanford.edu/~birch/p2p/

12 xx

BZ

−−=

λλ

Further

computation of Z will give depth map


Stereo imaging

�� = �� · �� = �� · �� , � are the pixel spacing

�� , �� are normalized pixel indexes

� = � − �� − ��

� = � − � · �� = � − � · ��

�� = �, �, � is the metric

coordinate for a world point.�, � is the metric coordinate

of a pixel.

Point cloud data � is a matrix of world coordinates,� = �� , �� , … , �� ,

where � is the number of measured world coordinates.

A depth map is a matrix of computed depths, D= ��,�


Stereo imaging

Left camera Right camera

Disparity Map

Point cloud data

Example of point cloud data


Stereo imagingDepth resolution

From earlier slide, we have the formula for

computing depth, � = � − ��

Let us define disparity as �� − �� = �� ⋅ " ,

such that � = � − ��#$⋅%

Let us now derivate Z with respect to s, &'&% = ��

#$%� , giving an expression for how

much Z will change due to a small change on s, (� = (" ��#$%�

Consider the following example,� = 8*+�� = 3.5/+� = 5++


Stereo imagingDepth resolution

0 5 10 15 20 250

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Depth Z

Depth

resolu

tion d

Z

Depth resolution for different sub-pixel precisions

0.1 pix

0.5 pix

1.0 pix

Great improvements of depth

resolution can be observed for

sub-pixel resolutions of the

disparity computation


Stereo imagingAvoid correspondence matching at pixel level

Think of a stereo camera pair used for surveillance of an air space close to a wind turbine.

The goal is to be able to position any birds coming close to turbine.

We can first segment both right and left image to identify the bird as an image object.

Using the pixel position of the bird in left and right allows for triangulation of only the

image object. Hence, more computing spent on segmentation while less spent on stereo

matching.


LED

driver

Time Of Flight Imaging

I-to-V

Transmitted signal

Received signal

Time of Flight imaging can be divided into two classes based on the method

used for modulation of light:

• Pulsed-light

• Light Detection and Ranging - LIDAR range scanners

• Flash LIDAR cameras

• Continuous Wave Modulation

• Phase-shift between emitted and received signal is used to indirectly measure time of flight



By Driving_Google_Self-

Driving_Car.jpg: Steve

Jurvetsonderivative work:

Mariordo - This file was

derived from: Driving

Google Self-Driving

Car.jpg:, CC BY 2.0,

LIDAR

A set of laser light rays is used for

spatial sampling of range data in the

vertical direction. Continues

rotations are used for scanning a 360 degree view.

https://youtu.be/nXlqv_k4P8Q https://youtu.be/kIrMaPTENB0


LED

driver


I-to-V

0 10 20 30 40 50 60-1

-0.5

0

0.5

1

1.5

2

Time [ ns ]

Voltage [

V ]

Transmitted signal

Received signal

Transmitted signal

Received signal

0

12�+32"452673�8"92:*3 = �2 = *

2<

A phase shift 0 of up to 360 degrees can be

measured uniquely. Correspondingly, the traveled distance of light is max �.

Longer distances will result in ambiguous data

Continuous wave modulation



= > = 22 *?" 0 @ A>

Phase and amplitude measurements



= > = 22 *?" 0 @ A>

Instead of expressing a time shift > for the cross correlation between two harmonics having the same angular frequency A, we express an angular shift B = A>= B = 2

2 *?" 0 @ BNext step is to sample the correlation function = B at four points of B equally spaced

over one period,

= 0D = 22 *?" 0

= 90D = − 22 "8: 0

= 180D = − 22 *?" 0

= 270H = 22 "8: 0




Let us have a look at the following expression,

= 270D − = 90D= 0D − = 180H =

22 "8: 0 @ 22 "8: 022 *?" 0 @ 22 *?" 0 = 2 · "8: 0

2 · *?" 0 = tan 0

Thus, the phase shift 0 of the reflected harmonic signal can be computed as,

0 = 92:�� = 270D − = 90D= 0D − = 180H




Let us have a look at a second expression,

= 270D − = 90D � @ = 0D − = 180H � = 2�"8:� 0 @ 2�*?"� 0 = 2�

Thus, the amplitude 2 of the reflected harmonic signal can be computed as,

2 = = 270D − = 90D � @ = 0D − = 180H �L




Let us have another look at the expression for cross correlation between reflected signal and demodulation signal,

= > = lim�⇢Q R 1 @ 2 · *?" A9 − 0� �S

�� S· *?" A9 @ A> �9

5 9 d 9 @ >Sampling the cross correlation signal = > at four points α = A> = 0D , 90D , 180D , 270Hmeans that we should demodulate the reflected signal with a demodulation signal having four different phase shifts

5(9)*?" A9 @ B

Integrator

R L

LReset

C(B)



LED

driver


I-to-V

Phase

shiftB = 0D , 90D , 180D , 270D

BP

filter

LP

filterC(B)

Lock-in amplifiers for all pixels

Pixel

Four images = B having four different phase shifts B at demodulation are used to

compute an Amplitude image and a Phase image.


Time Of Flight ImagingRange image to Point cloud

x,X

y,Y

z,Z

�� = �, �, �

(x,y)λ

� = 5 · �� @ �� @ ��L

� = � · ��

� = � · ��

5

�, �, � is the metric

coordinate for a world point.�, � is the metric coordinate

of a pixel.5 is the time-of-flight range

from pixel to world coordinate

Point cloud data � is a matrix of world coordinates,� = �� , �� , … , �� ,

where � is the number of measured world coordinates.

A range image is a matrix of range values, W = 5��,�

�� = �� · �� = �� · �� , � are the pixel spacing

�� , �� are normalized pixel indexes


Time Of Flight ImagingExample: ToF camera from Melexis


signal processing and analysis - miun.se

Documents