a robust-coded pattern projection for dynamic 3d scene …eia.udg.es/~qsalvi/papers/1998-prl.pdf ·...

11
A robust-coded pattern projection for dynamic 3D scene measurement J. Salvi a, * , J. Batlle a,1 , E. Mouaddib b,2 a Computer Vision and Robotics Group, Institute of Informatics and Applications, University of Girona, Avda. Lluis Santal o s/n, 17071 Girona, Spain b Automatic Systems Laboratory, University of Picardie–Jules Verne, 7, Rue du Moulin Neuf, 80000 Amiens, France Received 11 September 1997; received in revised form 29 May 1998 Abstract This paper presents a new coded structured light pattern which permits to solve the correspondence problem by a single shot and without using geometrical constraints. The pattern is composed by the projection of a grid made by coloured slits in such a way that each slit with its two neighbours appears only once in the pattern. The technique proposed permits a rapid and robust 3D scene measurement, even with moving objects. Ó 1998 Elsevier Science B.V. All rights reserved. Keywords: Coded pattern projection; Correspondence problem; Structured light; 3D scene measurement 1. Introduction There are some essential problems in stereo vi- sion which make it dicult to solve the corre- spondence problem between both image planes. Some authors consider to solve the problem of matching from singular points, i.e. finding points in both image planes with more or less the same neighbourhood characteristics, classifying them as corners and vertexes. Since both 2D images are projections of the same scene, images taken from dierent positions may be quite dierent one from the other. Consequently, this hypothesis is rather poor and becomes even poorer if we take into account that the precision of the 3D measurement depends on the distance between both cameras. The more this distance increases, the more the measurement will be precise. However, when one increases the distance between both cameras, the projective images of the 3D scenes become more dierent, constraining considerably their match using similarity. Of course, we must also consider that in the same 3D scene more than one object with approximately the same shape and size could be present. In this case it becomes rather dicult to match the images from the mere concept of singular points. However, some geometrical con- straints can be used to reduce the problem of matching. One of the most popular constraints is known as the epipolar constraint imposed by the geometrical relationship between both cameras (Faugeras, 1993, p. 169). The epipolar constraint allows us to reduce the searching of the Pattern Recognition Letters 19 (1998) 1055–1065 * Corresponding author. Tel.: +34 72 41 8474; fax: +34 72 41 8098; e-mail: [email protected]. 1 E-mail: [email protected]. 2 E-mail: [email protected]. 0167-8655/98/$ – see front matter Ó 1998 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 8 6 5 5 ( 9 8 ) 0 0 0 8 5 - 3

Upload: others

Post on 26-Jan-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

A robust-coded pattern projection for dynamic 3D scenemeasurement

J. Salvi a,*, J. Batlle a,1, E. Mouaddib b,2

a Computer Vision and Robotics Group, Institute of Informatics and Applications, University of Girona, Avda. Lluis Santal�o s/n, 17071

Girona, Spainb Automatic Systems Laboratory, University of Picardie±Jules Verne, 7, Rue du Moulin Neuf, 80000 Amiens, France

Received 11 September 1997; received in revised form 29 May 1998

Abstract

This paper presents a new coded structured light pattern which permits to solve the correspondence problem by a

single shot and without using geometrical constraints. The pattern is composed by the projection of a grid made by

coloured slits in such a way that each slit with its two neighbours appears only once in the pattern. The technique

proposed permits a rapid and robust 3D scene measurement, even with moving objects. Ó 1998 Elsevier Science B.V.

All rights reserved.

Keywords: Coded pattern projection; Correspondence problem; Structured light; 3D scene measurement

1. Introduction

There are some essential problems in stereo vi-sion which make it di�cult to solve the corre-spondence problem between both image planes.Some authors consider to solve the problem ofmatching from singular points, i.e. ®nding pointsin both image planes with more or less the sameneighbourhood characteristics, classifying them ascorners and vertexes. Since both 2D images areprojections of the same scene, images taken fromdi�erent positions may be quite di�erent one fromthe other. Consequently, this hypothesis is ratherpoor and becomes even poorer if we take into

account that the precision of the 3D measurementdepends on the distance between both cameras.The more this distance increases, the more themeasurement will be precise. However, when oneincreases the distance between both cameras, theprojective images of the 3D scenes become moredi�erent, constraining considerably their matchusing similarity. Of course, we must also considerthat in the same 3D scene more than one objectwith approximately the same shape and size couldbe present. In this case it becomes rather di�cultto match the images from the mere concept ofsingular points. However, some geometrical con-straints can be used to reduce the problem ofmatching. One of the most popular constraints isknown as the epipolar constraint imposed by thegeometrical relationship between both cameras(Faugeras, 1993, p. 169). The epipolar constraintallows us to reduce the searching of the

Pattern Recognition Letters 19 (1998) 1055±1065

* Corresponding author. Tel.: +34 72 41 8474; fax: +34 72 41

8098; e-mail: [email protected] E-mail: [email protected] E-mail: [email protected].

0167-8655/98/$ ± see front matter Ó 1998 Elsevier Science B.V. All rights reserved.

PII: S 0 1 6 7 - 8 6 5 5 ( 9 8 ) 0 0 0 8 5 - 3

Page 2: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

correspondence from the two-dimensional space ofthe image plane into the one-dimensional one ofthe epipolar line. However, we must be aware thatsome points in an image plane might not have acorrespondence on the other one due to a surfaceocclusion or simply because it has been projectedout of the scope of the camera. Note that we areunable to know a priori whether a point has acorrespondence or not, which makes the matchingenormously di�cult and imposes the use of aposterior step to remove false matching mostly byusing a region growing algorithm. At this pointanother constraint which arises from the structureof the objects of the scene can be used. It could beassumed that the scene surfaces vary in depthsmoothly almost everywhere. This constraint isknown as the disparity gradient. Although thisconstraint can be used to reduce false matchesobtained from the geometrical properties of theepipolar line, the reader must know that it cannotbe used at depth discontinuities. Note that depthdiscontinuities principally produce the edges andvertexes that will be used as tokens to the matchingprocess. We must also note that almost all stereovision systems restrict the obtaining of 3D infor-mation from the vertexes and corners of the ob-jects. Hence the disparity gradient could not beused except if we obtain 3D information followingthe edges which de®nitely complicates the match-ing process increasing the computing time.

It is known that the correspondence problemcan be alleviated leaving o� stereo vision, andgoing to the structured light concept. Here, thesecond stereo camera is replaced by a light source,which projects a known pattern of light on themeasuring scene. The ®rst stereo camera imagesthe illuminated scene and, analysing the deforma-tions of the imaged pattern with respect to theprojected one, can obtain the desired 3D infor-mation. Of course, depending on the chosen pat-tern, some correspondences between the projectedpattern and the imaged one should be solved.Most of the proposed structured light techniquesare based on the projection of regular patterns onthe measuring scene (e.g. Will and Pennington,1971; Hu and Stockman, 1989; Wang and Pandey,1991). All these methods obtain 3D informationfrom the geometric constraint propagation, espe-

cially from the epipolar constraint, and some ofthem are rather limited to measure surfaces withdepth discontinuities.

In recent years a new structured light techniquehas increased in importance. This technique isbased on a unique codi®cation of each token oflight projected on the scene. When the token isimaged by the camera, this codi®cation allows usto obtain the correspondence, i.e. to know where itcomes from. Then 3D measurements are directlyobtained as we are not to use hard computationalgeometric constraints. This technique is basicallyknown as coded structured light. Several codedstructured light techniques have been proposed inthe past, which have been discussed and comparedin a quite recent survey (Batlle et al., 1997). Thetechniques are mostly based on coded dot projec-tion (e.g. Vuylsteke and Oosterlinck, 1990; Yeeand Gri�n, 1994; Ito and Ishii, 1995) and codedslit projection (e.g. Boyer and Kak, 1987; Tajimaand Iwakawa, 1990; Maruyama and Abe, 1993).

Our goal is to propose a new coded structuredlight pattern. The technique is based on obtaining3D scene information from a single pattern shotprojection, so that it could be used to measurestatic and dynamic scenes. The projecting patternhas been designed such that it could be easily androbustly segmented, allowing us to e�ciently solvethe correspondence problem without spending alot of computing time.

The paper is divided as follows. Firstly, the newpattern is presented. Secondly, the system model-ling and the iterative calibration method are ex-plained. Then, the segmentation process and someexperimental results are shown and discussed. Fi-nally, conclusions are presented.

2. Pattern design

Our intention is to perform the measurement ofany static or dynamic scene by the projection of acoded structured light pattern on the measuringobjects. Therefore, a single pattern shot projectionis allowed as the scene could be composed bymoving objects. Furthermore, the pattern must besegmented without a signi®cant amount of com-puting time. If we project a straight line on planar

1056 J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065

Page 3: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

surfaces, straight segments of such line will beimaged leading to an easy segmentation. That iswhy we propose to use a pattern based on theprojection of straight lines. Several authors haveproposed striped pattern projection coded along asingle axe (e.g. Boyer and Kak, 1987; Tajima andIwakawa, 1990), because a single axe codi®cationof the projected pattern is needed to infer the 3Dinformation from the 2D points grabbed by thecamera (to demonstrate this deduction the readeris referred to (Hall et al., 1982)). Nevertheless, wesuggest to improve the robustness of the matchingusing a grid pattern projection coded along bothaxes. 3D information is inferred from the least-squares method. Quite a few techniques based onboth axes codi®cation have been proposed. Forinstance, Gri�n et al. (1992) proposed to projectuniquely encoded dots. The code of a given dot ismade by the colour of that dot and of its fourneighbours. However, due to an absence of linksamong dot neighbours, the identi®cation of theneighbours of a given dot is highly determined bythe measuring scene. Two years later, Yee andGri�n (1994) used the same principle of codi®ca-tion to project uniquely encoded binary dots. Thedots were linked, making easy the identi®cation ofthe neighbours. However, the segmentation of thepattern and further dot identi®cation becomeharder as dots are coded by using di�erent shapes.That is the reason why we propose to use a gridpattern which may be easily segmented.

We propose to code the grid by colouring itshorizontal and vertical slits. Although it is knownthat high saturated colour objects may producefurther segmentation errors, we assume that theyare not abundant in the scene. We must also notethat a system based on light projection is mostly tobe used under scene light control. Whereas onlythe light coming from the pattern is allowed if agood segmentation is required, slender light ispermitted if a high power projector lamp is used.Note that slender light will not produce the albedoe�ect which could make di�cult the segmentationof the pattern.

We propose to project a grid made by colouredslits. Although the pattern has been printed ingreyscale in Fig. 1, it is a coloured one. Six dif-ferent colours have been used. As an example we

have chosen red, green and blue to code the hori-zontal slits; and magenta, cyan and yellow to codethe vertical ones. But, what is important is that wemust chose well-spaced colours in the HSI cone,through which they can be well segmented usingtheir hue and saturation components. We haveused a codi®cation similar to the one used byGri�n et al. (1992). More precisely, Gri�n et al.are interested in obtaining the uniquely codedmaximum matrix from a determined basis. We areonly interested in the maximum vector, that is, themaximum sequence of triplets that can be madewithout repetition. Given a basis p, the maximumvector is de®ned by a sequence of numbers in the1 . . . p range, as shown by Eq. (1).

The problem can be solved using the graphtheory. Given a p basis we can obtain a set ofnodes S where each node is de®ned by a triplet ofthree elements of P � f1; 2; . . . ; pg, that isS�VRp

3 obtaining up to p3 nodes. A node is de-®ned by a triplet ijkji; j; k 2 P . At this point, thegraph can be constructed. Each node N� ijk has pinputs and p outputs, but the nodes where i� j� kas they have p ) 1 inputs and p ) 1 outputs. Then,the problem is reduced to construct the sequenceof triplets by visiting all the nodes of the graphonly once, which can be easily solved by graph

Fig. 1. The coloured pattern.

J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065 1057

Page 4: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

theory. Note that we can go from the node N� ijkto the node N0 � i0j0k0 only if j� i0 and k� j0. (For amore detailed explanation the reader is referred toGri�n et al., 1992, p. 611.)

In our example, we have chosen a basis equal to3, then we obtain a sequence of 29 numbers in therange 1 . . . 3. This sequence de®nes a coloured slitsequence. Horizontal slits are obtained substitut-ing 1, 2 and 3 by red, green and blue, respectively;and the vertical slits substituting 1, 2 and 3 bymagenta, cyan and yellow. What is important isthat each slit colour with its two neighbour slitscolours forms a triplet that exists only once in thewhole pattern. Slits are constantly spaced forminga regular grid of 29 ´ 29 cross-points. The reso-lution of the pattern may be increased by simplychanging the p basis, i.e. using more colour prim-itives.

In the aim of testing the proposed pattern, a labscenario has been set (see Fig. 2). The system iscomposed by an RGB camera, a computer and anelectronic slide projector. In Fig. 2, the cameracalibrating pattern is also shown. The colouredpattern is shaped in a 512 ´ 512 RGB image whichis projected on the measuring scene using theelectronic slide projector, and the scene is thencaptured by the camera into the computer mem-ory.

3. Calibration

The hard calibration method is based on ob-taining the intrinsic and extrinsic parameters of thecamera model and the projecting system modelknowing the coordinates of the 3D object points.In fact, the problem is reduced to the computationof the transformation equations which models therelation between the 3D object points (xr, yr, zr)and their 2D observable correspondence point (uk,vk) in the image plane of the camera (or their 2Dprojecting points of the projector system). Thelinear relation is modelled by a 3 ´ 4 transforma-tion matrix. This matrix contains the 6 extrinsicparameters de®ned by the three rotation angles (a,b, c), expressed as a 3 ´ 3 rotation matrix R, andthe translation vector t � �tx; ty ; tz�; and the 4 in-trinsic parameters de®ned by the projection of theoptical centre in the image plane (or projectorframe) (uo, vo), and the perspective parameters�au; av� (the reader is referred to (Toscani, 1987)for a wide explanation). However, as a result ofsome types of imperfections in the design and as-sembly of the lens composing the optical system, alinear relation does not hold true (see Tsai, 1987).These kinds of imperfections, known as lens dis-tortion, can be modelled by a radial and tangen-tial approximation. Radial distortion causes an

Fig. 2. The scenario of the 3D measuring system.

1058 J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065

Page 5: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

inward or outward displacement of a given imagepoint from its ideal location, and it has beendemonstrated to be quite important in cameramodelling (see (Weng and Cohen, 1992) for a de-tailed explanation). Radial lens distortion can beapproximated by nonlinear equations governed bythe ®rst coe�cient k1. The relation between a 3Dobject point and its observable image point (orprojective point) is then modelled by Eqs. (1) and(2).

U � Xu ÿ Xd ÿ k1r2Xd ; �1�

V � Yu ÿ Yd ÿ k1r2Yd ; �2�where

X

Y

Z

0B@1CA � �R t�

xr

yr

zr

1

0BBB@1CCCA; r2 � X 2

d � Y 2d ;

Xu � fXZ; Yu � f

YZ;

Xd � �uk ÿ u0�ku

; Yd � �vk ÿ v0�kv

;

we have expanded the au and av parameters asfollows:

au � f ku; av � f kv:

Let us consider Eqs. (1) and (2) like two func-tions G which depend on 11 unknowns. These 11unknowns are arranged in a vector called X, that isX � fu0; u1; . . . ; u10g. The obtaining of the X vec-tor is the main objective of the calibrating method.As a result of the inclusion of the lens distortion inthe camera model, G has become a nonlinearequation. That is why we have to use an iterativecalibrating method to minimise G in order to ob-tain X.

We want to calculate the values of X whichsolve the equation G(X)� 0 iterating from an ini-tial solution X0. The initial solution is computedusing the method of Toscani assuming no lensdistortion (k1� 0). We have used the widelyknown Newton±Raphson iterative method offunction minimisation. Then, from a determinediteration k, we can approximate the next values ofX from the last ones, using Eq. (3). Note that the

solution we want to ®nd is vector X of the un-knowns, which solve Eq. (4).

G�Xk� � G�Xkÿ1� � J�Xkÿ1�DXk; �3�

G�Xk� � 0: �4�Then Eq. (5) is derived, which may be solved

from the least-squares method expressed inEq. (6).

DXk � ÿJÿ1�Xkÿ1�G�Xkÿ1�; �5�

DXk � ÿ JT�Xkÿ1�J�Xkÿ1�ÿ �ÿ1

JT�Xkÿ1�G�Xkÿ1�:�6�

If we consider that n di�erent correspondencematchings are obtained, then G(Xkÿ1) is an n-di-mensional vector. J(Xkÿ1) is an n ´ 11 matrix inwhich each column is a partial derivative of Gfrom each unknown and each row and evaluationof these derivatives from each correspondencecouple points. DXk is an 11-vector which containsthe estimated error of the 11 unknown parametersto be determined. Eq. (6) is computed by the it-erative algorithm. After each iteration, the errorfound in each parameter is added to its value ob-taining the new value for the next iteration. Wemust iterate until Eq. (7) is reached.

DXk < e: �7�We have computed intrinsic and extrinsic pa-

rameters using the method proposed by Toscani.However, the maximum discrepancy between themodelled projections and the observable points israther large as it was around 2.8 pixels. The mainproblem is due to the fact that Toscani does nottake into account the lens deformation and that itis a non-iterative method. Then an iterative algo-rithm is quite interesting to decrease this discrep-ancy, readjusting the intrinsic and extrinsicparameters of the system. We have used the iter-ative method without modelling the lens defor-mation and we obtain a maximum error around1.2 pixels, which improves considerably Toscani'smethod. Then, we have modelled the lens defor-mation. We start the iterative algorithm from theparameters obtained by Toscani and a null radialdistortion. The focal distance f has been ®xed,otherwise the system is unstable because several

J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065 1059

Page 6: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

combinations of (f, ku, kv) can be obtained withoutchanging the (au, av) values. That is why we have®xed f� 10 mm. We have compared the initialsolution with the ®nal parameters. The maximumdiscrepancy between the modelled projections andthe observable points with the obtained parame-ters was around 0.0053 and 0.0069 pixels for thecamera and the projector system, respectively. In-creasing the number of iterations, which is aboutthe order of 20, does not improve the accuracy ofthe system. Table 1 shows the initial solution andthe resulting intrinsic and extrinsic parametersobtained by the iterative algorithm.

A calibrating pattern made by two orthogonalplanes have been set to calibrate the system (seeFig. 2). In order to calibrate the camera each planeis composed by a set of equidistant squares, ob-taining a pattern like the one shown in Tsai (1987).The square vertexes are merely used to calibratethe camera. On the other hand, a millimetric sheetof paper is ®xed on each calibrating plane in orderto calibrate the projector. The coloured grid pat-tern is projected on both millimetric sheets of paperobtaining the 3D position of the slits intersectionswhich are used to calibrate the projector.

4. Pattern segmentation

An RGB camera snaps an image of the patternprojected on the scene. The ®rst step is the trans-

formation of the image from the RGB model to theHSI perceptual model, close to human perception.In this way, the slits can be well segmented as weknow the projected colour, and we have assumedthat the scene is composed by pale and neutralcolour objects. With this goal, we have used a real-time pre-processor composed by three (256 Kbytes)LUTs which are addressed by an 18 bits bus, 6 bitsper each RGB channel, at video rate, obtaining thetransformation from the three basic colours to theHSI model. Actually, LUTs are loaded with thehue, saturation and intensity values of the selectedcolours. The rest of the memory cells are all loadedwith zeros. In fact, the pre-processor allows to dealwith any mathematical colour conversion (thereader is referred to (Garcia-Campos et al., 1996)for a wide explanation of this card). Then, the HSIimage containing the segmented scene is capturedby a Matrox Meteor acquisition card into thememory of a PC Pentium.

At this point, we have three images corre-sponding to the previous chosen hue, saturationand intensity values into the memory of the per-sonal computer. The six colours have been codedby six di�erent grey levels in a single image. Then,this image has to be processed to detect straightsegments for each grey level. We have used an easymethodology which involves image ®ltering, thin-ning and edge reconstruction. Fig. 3 shows thesegmentation of the blue colour straight segmentsobtained after edge reconstruction of the ®rst ex-

Table 1

Intrinsic and extrinsic parameters obtained by the process of calibration

Camera parameters Projector parameters

Initial solution Iterative solution Initial solution Iterative solution

a (rad) 1.6605 1.7294 1.8091 1.7511

b (rad) )1.2051 )1.2537 )0.7090 )0.7042

c (rad) )3.0980 )3.0325 )2.9961 )3.0324

tx (mm) 19.7 )84.9 )79.8 )61.8

ty (mm) 144.9 106.3 )82.7 )14.3

tz (mm) 1829.4 2432.1 1772.1 1910.8

f (mm) 10 10 10 10

ku (px/mm) 316.05 421.07 251.93 272.80

kv (px/mm) 332.57 442.43 261.31 282.66

uo (px) 223.43 405.19 376.22 350.82

vo (px) 227.25 297.80 560.48 460.09

k1 (ad) 0 )0.03505 0 )0.00309

rad: radians; mm: millimetres; px: pixels; ad: adimensional

1060 J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065

Page 7: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

perimental example. Straight segments will beuseful to solve the correspondence problem.However, segments can not be used to detect thepattern cross-points due to their high discontinu-ity. Note that it is essential to obtain well seg-mented cross-points as they are directly used toinfer 3D information.

The main idea is that cross-points have beendetected by searching for local maximum in theintensity image. Actually, cross-points are made bythe intersection of two slits with two di�erent in-tensity levels. Then, the cross-point must have anintensity of approximately the sum of both which

value de®nes a dynamic threshold. This thresholdhas been used to segment a region containing thecross-point which is further detected by computingthe gravity centre. Fig. 4 shows the cross-pointdetection of an image portion of the ®rst example.

Using the coloured codi®cation of the segmentswhich forms the cross-point in the image plane, thematching, i.e. the determination of the position ofthe same point in the projector image plane, iscarried out. Then, 3D object point coordinates areobtained from both 2D positions of their projec-tive points in the image plane and in the projectedpattern.

Fig. 3. Segmentation and edge reconstruction of blue colour.

Fig. 4. A sample showing two cross-points detection. (a) A portion of the Intensity image showing three slits. (b) The segmented

regions of both cross-points obtained by local thresholding.

J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065 1061

Page 8: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

5. Experimental results

Hereafter, two examples are shown. The ®rstexample is composed by a scene with three geo-metric and achromatic objects illuminated by thecoloured pattern, as shown in Fig. 5. Fig. 6 showsthe measured correspondence points in the pro-jector and camera frame, and Fig. 7 shows the 3Dreconstruction. The second example is based onthe 3D reconstruction of the mask shown in Fig. 8.Fig. 9 shows the 3D reconstruction.

In order to compare 3D geometric informationinferred by the proposed coded structured lightsystem and the real 3D information of the scene,

we have projected the coloured pattern on a singleplane. Several images have been taken whilemoving the plane. We have measured the 3D realpoint, and we have inferred the 3D object pointfrom both projective 2D points. The (X, Y, Z)error measured is the discrepancy between realcoordinates and the inferred ones. We have mea-sured the error in approximately 100 points. Theaverage values of the error deviate 0.728 mm(0.31%) from the X axes, 0.624 mm (0.27%) fromthe Y axes, and 0.465 mm (0.20%) from the Z axes.The results are quite interesting if we take intoaccount that the deviation degree is highly in¯u-enced by the segmentation process, i.e. by the

Fig. 6. The matching obtained from Fig. 5. (a) The Matching points from the 576 ´ 768 camera image plane. (b) The Matching points

from the 512 ´ 512 projected pattern frame.

Fig. 5. (a) A real scene composed by three geometric objects (not used to infer 3D information). (b) The scene illuminated by the

coloured pattern.

1062 J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065

Page 9: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

Fig. 8. A real scene composed by an illuminated mask.

Fig. 7. The reconstruction of the 3D objects shown in Fig. 5. (a) A perspective view from the right side. (b) A top view.

J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065 1063

Page 10: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

image noise and the problem of grabbing theprojected lines of the pattern with di�erent thick-ness.

On the other hand, the percentage of correctlyidenti®ed intersections reaches the 100% of cross-point identi®cation, measuring a planar surfaceunder scene light control. However, the identi®-cation is completely constrained by the kind ofscene to be measured and the light conditions.Then, when the light conditions decrease, theprojection of colour weaks considerably the use ofsuch a kind of pattern. Of course, the utilisation ofa digital light projector with a higher resolution, oreven a more powerful lamp, will increase the av-erage of correctly identi®ed intersections.

6. Conclusions

In order to obtain a 3D measurement of eitherstatic or dynamic scenes, we propose to project anew coloured grid pattern on the measuring ob-jects. The columns and rows of the grid are codi-®ed using six well-de®ned colours of the HSI cone.In the presented example, the pattern uses thethree primary colours (red, green and blue) to codethe rows. The magenta, cyan and yellow coloursare used to code the columns. The slits have beencoloured with the aim that each slit, with its twoneighbours, forms a unique triplet in the wholepattern. Such a codi®cation permits that the pat-tern could be used to obtain 3D information of theilluminated scene from a single shot projection.

Then, an easy and quick decoding step allows thepattern to be used in dynamic 3D measurements.Experimental results of object measurement and3D reconstruction have been presented in the pa-per. Some applications of the sensor could, inparticular, be used within the scope of scene in-terpretation in mobile robot navigation andtracking of moving objects, and, in general, in any3D scene measurement where a single shot is al-lowed and a quick response of the system is re-quired.

As a further work, we are interested in the fre-quency shifting of the imaging colour with respectto the projecting one as a result of the intrinsiccolour of the measuring objects. If the patterncolours are well chosen, this discrepancy may allowus to obtain the intrinsic colour of the scene ob-jects. Then the proposed system might be used toobtain 3D information and coloured informationof any scene by a single pattern shot projection.

References

Batlle, J., Mouaddib, E., Salvi, J., 1997. A survey: Recent

progress in coded structured light as a technique to solve the

correspondence problem. To be published in Pattern Rec-

ognition Journal.

Boyer, K.L., Kak, A.C., 1987. Color-encoded structured light

for rapid active ranging. Pattern Analysis and Machine

Intelligence 9 (1), 14±28.

Faugeras, O., 1993. Three-Dimensional Computer Vision: A

Geometric Viewpoint. MIT Press, London, England.

Garcia-Campos, R., Batlle, J., Bischo�, R., 1996. Architecture

of an object-based tracking system using colour segmenta-

Fig. 9. The reconstruction of the mask shown in Fig. 8.

1064 J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065

Page 11: A robust-coded pattern projection for dynamic 3D scene …eia.udg.es/~qsalvi/papers/1998-PRL.pdf · 2008-01-16 · A robust-coded pattern projection for dynamic 3D scene measurement

tion. In: Proceedings of 3rd Internat. Workshop on Image

and Signal Process., pp. 299±302.

Gri�n, P.M., Narasimhan, L.S., Yee, S.R., 1992. Generation of

uniquely encoded light patterns for range data acquisition.

Pattern Recognition 25 (6), 609±616.

Hall, E.L., Tio, J.B.K., McPherson, C.A., Draper, C.S.,

Sadjadi, F.A., 1982. Measuring curved surfaces for robot

vision. Computer (December), 42±54.

Hu, G., Stockman, G., 1989. 3-D surface solution using

structured light and constraint propagation. Pattern Anal-

ysis and Machine Intelligence 11 (4), 390±402.

Ito, M., Ishii, A., 1995. A three-level checkerboard pattern

(TCP) projection method for curved surface measurement.

Pattern Recognition 28 (1), 27±40.

Maruyama, M., Abe, S., 1993. Range sensing by projecting

multiple slits with random cuts. Pattern Analysis and

Machine Intelligence 15 (6), 647±651.

Tajima, J., Iwakawa, M., 1990. 3-D data acquisition by

rainbow range ®nder. In: Proceedings of IEEE Internat.

Conf. on Pattern Recognition, pp. 309±313.

Toscani, G., 1987. Syst�emes de calibration et perception du

mouvement en vision arti®cielle. Th�ese Universit�e Paris Sud.

Tsai, R.Y., 1987. A versatile camera calibration technique for

high-accuracy 3D machine vision metrology using o�-the-

shelf TV cameras and lenses. Journal of Robotics and

Automation RA-3 (4), 323±344.

Vuylsteke, P., Oosterlinck, A., 1990. Range image acquisition

with a single binary-encoded light pattern. Pattern Analysis

and Machine Intelligence 12 (2), 148±164.

Wang, Y.F., Pandey, A., 1991. A study on using structured

lighting to analyze time varying image seguences. Pattern

Recognition 24 (8), 723±738.

Weng, J., Cohen, P., 1992. Camera calibration with distortion

models and accuracy evaluation. Pattern Analysis and

Machine Intelligence 14 (10), 965±980.

Will, P.M., Pennington, K.S., 1971. Grid coding: A prepro-

cessing technique for robot and machine vision. Computers

and Arti®cial Intelligence, pp. 66±70.

Yee, S.R., Gri�n, P.M., 1994. Three-dimensional imaging

system. Optical Engineering 33 (6), 2070±2075.

J. Salvi et al. / Pattern Recognition Letters 19 (1998) 1055±1065 1065