ipkpaper
TRANSCRIPT
-
7/29/2019 IPKpaper
1/5
DATA EVALUATION FOR DEPTH CALIBRATION OF A
CUSTOMARY PMD RANGE IMAGING SENSOR CONSIDERING
OBJECTS WITH DIFFERENT ALBEDO
Jochen RadmerInstitut f ur Werkzeugmaschinen und Fabrikbetrieb (IWF), Technische Universit at Berlin, Berlin, Germany
Alexander Sabov, Jorg KrugerInstitut f ur Produktionsanlagen und Konstruktionstechnik (IPK), Fraunhofer Gesellschaft, Berlin, Germany
{alexander.sabov, joerg.krueger}@ipk.fraunhofer.de
Keywords: Range Imaging Camera, Photo Mixer Device (PMD), Depth Calibration, Albedo, Reflectance Properties
Abstract: For various applications, such as object recognition or tracking and especially when the object is partly oc-
cluded or articulated, 3D information is crucial for the robustness of the application. A recently developed
sensor to aquire distance information is based on the Photo Mixer Device (PMD) technique. This article
presents an easy but accurate data acquisition method for data evaluation of a customary sensor. Data evalu-
ation focuses on the detection of the over- and underexposured data under consideration of objects with two
different albedos.
1 INTRODUCTION
Since we are living in a three-dimensional world, for
various applications, such as object recognition ortracking, 3D information is crucial for the robust-
ness of the application, especially when the object
is partly occluded or articulated. Due to its signifi-
cance, the field of depth data acquisition has attractedmany researchers working on sensors or sensor sys-
tems respectively. A recently developed sensor is
based on the Photo Mixer Device (PMD) technique
which works on modulated, coherent infrared light
using the Time of Flight (ToF) approach and rely ona technology that correlates reference and received
signals directly on the chip. It was first mentioned
by (Lange, 2000) and (Schwarte, 1999). This sensor
provides direct depth information for a whole image,for which reason such sensors are called range imag-ing sensors. Additionally to the depth data the cam-
era provides two other channels. One channel corre-
sponds to the luminance of the scene and the other
channel is the amplitude as it is described in section
(Lange, 2000) and corresponds to the signal strengthof the incoming signal. The PMD range imaging cam-
era is shown in figure 1 with infrared LED matrices on
both sides of the camera.
Since this sensor is based on a rather new technol-
ogy, data evaluation and depth calibration has to be
Figure 1: This figure shows the examined PMD RangeImaging Camera of model PMD [vision] 19k with the in-frared LED matrices on both sides of the camera.
carried out for it. (Lindner and Kolb, 2006) did a lat-eral and depth calibration on a modified research sen-
sor. In this work the non-linearity of the depth datawas coped by a distance deviation plot. (Kahlmann
et al., 2006) additionally considered the dependency
on the integration time. But since the sensor emitslight for ToF and is dependent on the amount of inci-
dent light, the integration time is not the only param-
eter affecting the depth measurement. It is just one
parameter influencing the amount of incident light.
The incident light can also vary with the infrared re-flectance properties of the looked at object, the dis-
tance to it and the LEDs properties. Therefore the
data evaluation for the depth calibration has to be car-
-
7/29/2019 IPKpaper
2/5
ried out considering the amount of incident light. In
addition, in order to guarantee a certain exactitude
of the data, over and underexposed data has to be
identified. For this reason this paper focuses on data
evaluation and improvement considering different re-flectance properties in combination with the distance
and the integration time for a customary sensor. On
account of the fact that the reflectance properties of
an object vary greatly with its material and orienta-
tion, the measurement will be simplified. Instead ofthe reflectivity we will rather consider different albe-
dos. Albedo is a unitless measure indicative of an ob-
jects diffuse reflectance properties. Nevertheless, a
comprehensive investigation has to be done for many
reflectance properties so that this work can be seenas some starting point. Thus, the main contribution
of this paper is the analysis of the data provided by
the PMD range camera under consideration of objectswith different albedos and to give method for distance
measuring improvement.
The distance measuring methods used be the PMDrange imaging camera is the Time of Flight (ToF)method. The article of (Lange, 2000) offers a de-
tailed description of the functional way of the PMD
range imaging camera. A short overview of the dis-
tance calculation can be found in (Sabov and Krger,2007).
The paper is structured as follows. First the data
acquisition and data evaluation is described. The ba-
sic methods for the improvement of the data are given.
Within section three results are presented and dis-cussed and finally this article is concluded in sectionfive.
2 DATA EVALUATION
2.1 DATA ACQUISITION METHOD
For the data evaluation fixed targets were considered,a black and a white wall, with diffuse, Lambertian re-
flectance model like characteristics. We have selected
a white and a black wall as objects because of theircontrariness in their albedo.
The measuring setup for the data acquisition ex-
isted of the PMD range camera, two lasers to distancemeasuring instruments (range finder), a camera con-
struction to arm the distance measuring instruments
on the PMD range camera, two markers, a tripod and
the object, the wall. The measurement range of the
distance measuring instrument was 0.2m to 30m me-ter with an accuracy of2.0mm. Those range finderswere aimed at the ends of the camera construction in
parallel in the direction of the camera. With every
Figure 2: This figure shows the measurement setup as de-scribed in the text.
measurement the camera was aimed with both dis-
tance measuring instruments showing the same dis-
tance. The distance measured by both instrumentswas taken as the nominal distance to the object. In
addition, the laser points of the distance measuring in-
struments were directed upon two marks on the wallto verify the parallelism of the distance measuring in-
struments. The base distance b between the marks onthe wall as well as the distance between both range
finders is identical. To guarantee the orientation of
the camera also in vertical direction, the marks were
appropriated by the same height like the range finders.
The base distance b can be seen in the figure 2, whichshows the whole measurement setup. The parallelism
and the same distances measured by the instruments
guaranteed the horizontal orientation of the cam-
era towards the wall to be perpendicular. This is done
to reduce perspective influence on the measurement.
Based on this measuring setup and with the base dis-tance b = 0.4m the maximum angle error max of
the adjustment of the camera arises to max 0.6
using the following equation:
max = arctan|d1|+ |d2|
b= arctan
0.004m
0.4m(1)
This assumes that the wall is plane and arming of the
range finders was performed in a precise way. The
maximum distance error dmax can be calculated re-
garding the direction of the distances with equation 2.The maximum dmax then arises to 2.0mm.
dmax =
d1+d2
2 (2)
The measurement was carried out in different dis-
tances of the camera to the wall in intervals betweenthe single measurements of 0.1m. The measurement
was applied about the whole measuring space of the
camera. Ten measurements were carried out in every
measuring distance and in each case the local neigh-
borhood of the centre was captured to get more sta-ble data. The centre was determined before by the
determination of the intrinsic parameters. This lat-
eral calibration was done using the method described
-
7/29/2019 IPKpaper
3/5
in (Bouguet, 1999). The measurements per measur-
ing distance were performed for different integration
times. The span of the used integration times reached
from 500s to 20000s which were gone through in
500s steps.(Kahlmann et al., 2006) showed that the sensor
SwissRanger heats up self-induced and that this af-
fects the captured data. This process stabilizes a few
minutes after having started the data acquisition. To
avoid these effects of self-induced heating, the dataacquisition for the data evaluation was started after
20 minutes of sensor run-time.
Although the distance measurement accuracy vary
over the sensor and every measuring point must beconsidered individually for improvement (Kahlmann
et al., 2006), in this work only the central measuring
points are considered, because the focus on the dis-
tance data improvement lies in relation for the inte-gration time, the nominal distance and the albedo ofthe objects.
2.2 DISTANCE MEASUREMENT,
AMPLITUDE AND OBJECT
ALBEDO
In this section distance measuring results regarding
integration time and nominal distance are considered.First it is looked at the measuring results without vary-
ing the albedo of the object. In figure 3 the deviation
of the measured to the nominal distance for a whitewall can be seen. The deviations are applied for in-
tegration time and nominal distance. As can be seenclearly, low integration times and high nominal dis-
tances result in erroneous distance measurements. In
this connection must be mentioned that the low inte-
gration times which yield erroneous distance measur-
ing values play only one very low role in the practi-cal use of the sensor. The same can be ascertained
for long integration times in combination with low
nominal distances. Since distance measurements with
under- and overexposure do not include any useful in-
formation, these distances have to be detected first be-
fore improving the distance data.In the area in which the measured distances de-
viate only slightly from the nominal ones a period-
ical dependence of the distance offset can be ascer-tained to the nominal distance. This periodic error
in the depth measurement agrees with mathematical
predictions due to the shape of the correlation func-
tion of the reference and the received signals. Be-
side this periodical dependence the distance measur-ing error increases with an enlargement of the integra-
tion time. The error measured for different integration
times tending downwards has a range of about 50mm
Figure 3: This figure shows the absolute deviation of nomi-nal to measured distances in meter regarding the integrationtimes in s104 and nominal distance in meter. A white wallwith diffuse, Lambertian reflectance model like characteris-tics had been considered as object.
Figure 4: This figure shows the amplitude of the signal usedfor distance determination regarding the integration times ins and nominal distance in meter. A white wall was used asobject.
up from 2m nominal distances. For closer distancesthe error introduced by the near field effect caused
by different ToF from the left and the right LED ar-
rays dominates over the error introduced by the anhar-
monic signals. For these areas which can be described
about integration time and nominal distance a correc-tion can be achieved about Look-up-tables (LUT). Ac-
cording to (Kahlmann et al., 2006) these LUTs have
to contain an individual offset for each data point. A
better knowledge of the received LED and the refer-
ence signal likely would provide a more robust ba-
sis for depth calibration. However, with the practi-cal application of the sensor the nominal distance is
unknown. Hence, we take into consideration for the
detection of the useless distance values, in addition,
the amplitude Ampl. In figure 4 the accompanying
Ampl is shown. Likewise with regard to the integra-
tion time and the nominal distance. A connection can
be noticed between distance measuring error and the
accompanying Ampl. Through a simple threshold the
underexposed data can be identified, when Ampl fallsbelow a certain threshold.
The area in which on account of low nominal dis-
-
7/29/2019 IPKpaper
4/5
Figure 5: This figure shows the standard deviation in meterregarding the integration times in s104. A white wall wasconsidered as object with a nominal distance of 0 .9m. Itcan be noticed that the standard deviation can be used forthe detection of overexposure.
tance, high reflectance property and long integration
time it comes to an overexposure cannot be detected
simply about a threshold. In this case the standard
deviation of the distance is considered giving a direct
quality measurement for the measured distances. Ex-emplary the standard deviation for the distance mea-
surement at a nominal distance of 0.9m is shown in
figure 5. It was computed out of ten measured dis-
tances. The nominal distance of 0.9m was selectedbecause it yields correctable distance values at short
integration times and arbitrary distance values at long
ones, which are not correctable (cp. figure 3). Fig-
ure 5 shows that the standard deviation is high when
an overexposure occurs and low instead. Thereforethe standard deviation of measured distances can be
used for the detection of overexposure. It can be seen
that the standard deviation in spite of the very small
number of ten samples being used for the calculationachieves a high discriminatory power. Comparableresults can be achieved by calculating the standard
deviation of the local neighborhood and therefore us-
ing a spatial instead of a temporal window. A spa-
tiotemporal consideration is not necessary because of
the high discriminatory power of one approach.By the detection of over- and underexposure useless
measured distances can be neglected. The remaining
measured distances then can be corrected by a LUT,
which is based upon the nominal to measured distance
measurements.In the following the measuring results are exam-
ined under consideration of a different albedo of theobject. For measurements with a black wall as object
the distance offset can be seen in figure 6. Of course
these measurements also show areas with erroneousmeasured distances due to underexposure, since the
albedo of a black wall is lower than the one of a white
wall. The recently described method for the detection
of measured distance in case of over- or underexpo-
sure show good results also with this object, whenusing the same global thresholds. The periodicity
has the same period length over the nominal distance.
This is due to the shape of the correlation function
Figure 6: This figure shows the absolute deviation of nomi-nal to measured distances in meter regarding the integrationtimes in s104 and nominal distance in meter. A black wallwas used as object.
Figure 7: This figure shows the amplitude of the signal usedfor distance determination regarding the integration times ins and nominal distance in meter. A black wall was used asobject.
of the reference and the received signals, which does
not change. But in comparison to the measurements
with the white wall, a bigger distance offset can be
noticed. Therefore the albedo must be identified forthe improvement of the distance measuring data.
Since the range camera works with a globally set in-
tegration time, the method for the improvement of the
data needs to take into consideration the amount of
incident light. That can be achieved by directly con-sidering Ampl. Figure 7 shows Ampl of the measure-
ments with the black wall. Results for the identifi-
cation of the albedo with a case-based measured dis-
tance improvement are presented in the next section.
3 RESULTS
In this section the results of the work are pre-
sented, meaning that the identification of the albedowith a case-based distance value improvement are
presented for a black and white wall. In figure 8 is
shown on the left side the distance offset in meter
-
7/29/2019 IPKpaper
5/5
and on the right side the according amplitude Amplat central data points along a central column. The
left side shows the black wall and the right side the
white wall showing a lower distance measuring er-
ror. The crossover between black and white can beseen clearly. The result after applying the LUTs for
black and white on the distance measurement data is
shown in figure 9 showing the distance offset in me-
ter. It can be noticed that the distance offsets were de-
creased significantly from up to 0.3m down to about0.02m. The difference of the distance offset between
the white and the black wall was nearly completely
eliminated. At data points where the different albedo
contact a bigger distance offset can be noticed. This is
caused by the intermediate values for which no LUTexist.
Figure 8: In the left figure the distance offset in meter andon the right side the according amplitude Ampl at centraldata points along a central column is shown. The crossoverbetween the two albedos can be seen clearly.
4 CONCLUSION
This article presents an easy but accurate data ac-quisition method for data evaluation of a customary
Figure 9: This figure shows the distance offset in meter af-ter applying the LUTs for black and white on the distancemeasurement data. The distance offsets were decreasedfrom up to 0.3m down to about 0.02m. The difference be-tween the offset was nearly completely eliminated. At thecrossover a bigger distance offset can be noticed. This iscaused by the intermediate values for which no LUT exist.
sensor. Data evaluation focuses on the detection of
the over- and underexposed data under consideration
of objects with two different albedos. This improves
the exactitude of the data for the study of the sys-
tematic errors of this sensor. Therefore the work ofthis article can be seen as an intermediate result on
the way to a PMD range imaging sensor calibration.
We showed that the objects reflectance properties af-
fects the distance measuring. Therefore we proposed
a basic method for the identification of the albedoof the considered objects so that an albedo accord-
ing LUT can be applied for the correction. These
LUTs have to be created for a set of reference ob-
jects with different albedos. The example in the pre-
vious section shows that a correction can be achievedabout LUTs. Because of their contrariness in their
albedos, we selected a white and a black wall as ob-
jects. A statement about the behaviour of the sen-sor can thereby be done only for those extreme albe-
dos. Apart from these albedos a statement cannot oc-cur for general reflectance properties. Further work
has to be done in the field of interpolation between
LUTs when unknown albedos are detected and how
to handle crossovers between different albedo. Addi-
tionally, objects which reflectance properties cannotbe described by a Lambertian reflectance model need
to be investigated.
REFERENCES
Bouguet, J. (1999). Visual methods for three-dimensionalmodeling. PhD thesis, California Institute of Technol-ogy.
Kahlmann, T., Remondino, F., and Ingensand, H. (2006).Calibration for increased accuracy of the range imag-ing camera swissranger. In International Archives ofPhotogrammetry, Remote Sensing and Spatial Infor-mation Sciences, ISPRS Commission V Symposium,volume XXXVI, pages 136141.
Lange, R. (2000). 3D Time-of-Flight Distance Mea-surement with Custom Solid-State Image Sensors inCMOS/CCD-Technology . PhD thesis, UniversitySiegen.
Lindner, M. and Kolb, A. (2006). Lateral and depth calibra-tion of pmd-distance sensors. In ISVC (2)- Advancesin Visual Computing, Second International Sympo-sium, volume 4292 ofLecture Notes in Computer Sci-ence, pages 524533. Springer Verlag.
Sabov, A. and Krger, J. (2007). Improving the data qualityof pmd-based 3d cameras. In Proc. Vision, Modelingand Visualization (VMV).
Schwarte, R. (1999). Handbook of Computer Vision andApplications, chapter Principles of 3-D Imaging Tech-niques. Academic Press.