enhancing the 3d laser scanning pipeline using...

EUROGRAPHICS ITALIAN CHAPTER

Enhancing the 3D Laser Scanning Pipeline using Mirrors

Andrea Fasano, Marco Callieri, Paolo Cignoni, Roberto Scopigno

Istituto Scienza e Tecnologie dell’InformazioneU Consiglio Nazionale delle Ricerche

AbstractThis paper proposes and evaluates the use of a mirror for improving the scanning process when using laser-stripeacquisition devices. Starting from the idea that a triangulation laser scanner is able to acquire both real geometryas well as objects parts reflected in a mirror, our work aims to find a way to integrate the use of mirrors insidethe 3D scanning pipeline without changing the scanning hardware and with small intervention on software side.After analyzing problems related to the acquisition of reflected surfaces, we propose two scenarios where using amirror can greatly decrease the scanning time for small but complex objects using a mirror placed on a rotary tableor increase the completeness of the model allowing the capture of hard-to-reach parts using a small, hand-heldmirror.

Categories and Subject Descriptors(according to ACM CCS): I.3.1 [Computer Graphics]: Input devices

1. Introduction

Nowadays 3d scanning technologies are becoming verycommon in different contexts and with various objectivesand purposes. In the cultural heritage field 3D scanning hasproven as a very accurate medium for the truthful documen-tation of the shape and status of three dimensional artifactslike sculptures or other works of art. Notable examples inthis field of application of 3D scanning technologies where,just to cite some, the Digital Michelangelo project7, the ac-quisition of the Michelangelo’s Florentine Pietà1 and themonitoring of the restoration of the Minerva of Arezzo10.

On other hand, industrial applications of 3D scanning aremainly focused to quality control during manufacturing andreverse engineering.

In fact, two are the main requirement for this kind of tech-nology: in the cultural heritage field is required complete-ness, since only original data is important to document onlythe real status of an artifact without including any possiblesubjective intervention. Industrial applications, to the con-trary, are much interested in having faster methods to acquireobjects. A common wording in this field reports that to scana complex object you get 90% of the surface in the first 10%of the time and use the other 90% trying to scan the last 10%of the surface.

To face these problems we have found that mirrors can

greatly help to laser-scan: using a mirror combined with arotary table it is possible to reduce acquisition time whenperforming automatic scanning. With a small and adjustablemirror, it is possible to easily acquire parts that are hard to bedirectly reached. In section2 we report some previous usesof mirrors in the 3D scanning field while in section3 wegive the main details on our approach describing the issuesrelated with the placement of a mirror in a scanning environ-ment and explaining the nature of error and noise resultingin the generated surfaces. Sections5 and4 describe the twoproposed applications of mirror-enhanced scanning: reduc-ing time of automatic unattended 3d scanning and acquiringhard to be reached parts.

2. Related Works

Range scanning technology has evolved in a considerablemanner in the last few years. An overview of the field, cover-ing both hardware and software issues, is available in a cou-ple of recent papers4, 2. Many different systems have beenproposed; a common characterization subdivides them intocontact and non-contact devices. An important subclass ofthe latter is the one based on optical technology, and it canbe further subdivided into active and passive approaches. Wegive in the following a brief characterization of active opticaldevices.

c© The Eurographics Association 200x.

Fasano et al / Mirror 3d scans

Figure 1: With the use of a mirror during 3D acquisition, itis possible to acquire both directly visible parts as well asparts visible only in the reflection.

Figure 2: Placing a mirror over a rotary table allows to re-cover a more complete representation of a 3d object in atotally unattended and automatic manner.

Active optical devices are based on an emitter, which pro-duces some sort of structured illumination on the object to bescanned, and a sensor (typically a CCD camera), which ac-quires images of the distorted pattern reflected by the objectsurface. In most cases the depth information is reconstructedby triangulation, given the known relative positions of theemitter-sensor pair. The emitter can produce coherent light(e.g. a laser beam or stripe) or incoherent light; in both cases,a given light pattern (point-wise, stripewise or a more com-plex pattern) is projected on the object surface. In this paperwe focus on the use of scanning devices based on the auto-matic sweep of a laser stripe over an object. This approach

Figure 3: Many statues present some parts that are quitedifficult, or even impossible to be directly acquired. On left:holes in the lower part of the DavidŠs chin. On right: Thesmall mirror used to acquire hard to be reached parts.

represent one of the well established scanning technologiesand various commercial products: based on this technolo-gies, are present on the market like, just to cite a few, Cy-berware, Minolta, etc. In our experiment we used a MinoltaVivid 900.

The idea of exploiting mirrors for improving the 3d scan-ning process is not completely new: it has been already usedbut, to our knowledge, only in the field of passive systems.In 8 the authors use two mirrors placed vertically nearbya talking human face to improve the tracking of markersplaced on head; in this way they can estimate, with a sin-gle camera (simulating three stereo camera), the 3D positionof the markers. Previously, but still regarding passive shapeacquisition, Huynh6 work focused on the problem of solv-ing of 3D position estimation in the situation of a symmetryplane and discussed the advantage of non-linear computa-tion that can be exploited when the image to be processedcontains a mirrored image of the shape that we want to de-tect. Moreover, mirrors are used as internal components of3D scanner hardware to control orientation of the laser (likein the Minolta Vivid 900), of the acquisition sensor, or both(synchronized scanners9). We propose to use mirrors as ex-ternal parts, coupled with existing scanning device, slightlymodifying the scanning pipeline. One of the most impor-tant concepts in our approach is that the scanner hardwareis completely unaware of the mirror presence and only mi-nor changes are to be applied to the usual 3D acquisitionpipeline. The main problems that we have to face are: under-standing the location of the duplicated geometry and copingwith the errors due to the interaction of the scanning devicewith the real and the virtual object at the same time.

3. Active Optical Scanning with a Mirror

For a better understanding of the advantages and the prob-lems when simultaneously using a laser stripe scanning de-vice and a mirror, it is useful to recall the basic principleof this kind of technology. As shown in Fig.1, the laserprojects a horizontal line onto the object and the cameraCCD captures the distorted shape of the line onto the surfaceof the object. Knowing the relative positions of the laser and



Figure 4: A double laser stripe onto the object caused by themirrored laser emitter and the wrong sample points gener-ated during scanning.

the camera, the 3d position of each point of the laser stripeis recovered by triangulation. Adding a mirror to the scenewe virtually duplicate the object as shown in figure1.

3.1. The mirrors

We performed the scans using two different sized front-surface mirrors, one of 400x400mm and a smaller one of250x100mm. A front-surface mirror has the specularly re-flective layer on its front side. Front-surface mirrors repre-sent a fundamental element in order to achieve a good mir-rored scan. In fact, our very first experiments done with stan-dard back-surface mirrors showed clearly that these mirrorsproduce too much noise and errors: the laser beam emittedby the scanner must travel two times through a thick glasslayer before hitting the reflective (back) surface and bounc-ing out. This behavior introduces many artifacts in the finalmesh obtained.

3.2. Noise Geometry

The main problem when using a laser stripe scanner with amirror in the scene it can happen that the CCD see two dif-ferent laser stripes: the real one and the one generated by the

mirror (you can think the laser emitter has been duplicatedtoo). Figure 4 shows a “worst case” example where a ver-tical surface placed orthogonally onto an horizontal mirrorand the scanner sees two distinct laser stripes; this doubleline usually causes a failure of the surface recovering algo-rithm, since the scanner CCD is expecting a single spot foreach vertical line. In this situation the laser stripe (shown inFig. 4 as a red star-shaped dot) is seen twice: as a pointP onto the real object and as a pointS on the mirrored ob-ject. If the reconstruction software chooses the wrong points the reconstructed surface will pass through a pointP′ thatis wrongly placed; in fact the fake pointP′, will be placed onthe intersection between the laser line and the line connect-ing the center of projection of the camera with the wrongspot S. It is interesting to note that, once we have chosena particular scanner/mirror configuration, we can, in somemeasure, predict the noise and errors due to the double laserstripe. For example, in the case shown in figure4 we cananalytically calculate the locus of the points where the fakesurface is created; this locus is shown as the blue curved linethat crosses the contact point between the mirror and the ob-ject (where the real and mirrored laser spot coincide). It canbe noted that a double laser stripe appears every time thesame portion of the surface is visible twice†: directly andmirrored. There are various solutions to this problem. Thesimpler one is to try to place the mirror such that it cap-tures only unseen parts, for example behind the object withrespect to the scanner. Another solution, when the object isplaced over a mirror, is placing the object to be scanned overa pedestal that keeps it away from the mirror surface (if pos-sible, using pedestal colored such it is not acquired by thescanner). This technique works because the zone of contactbetween object and the mirror is where the multiple laserstripes are more frequent Moreover, the use of a pedestal isalso useful because lifting the object allows to see better itsbottom part.

4. Rotary platform

The use of a rotary platform together with a 3D scan-ner is a common technique to automatize the acquisitionof small/medium sized objects. Usually, after a calibrationstep which allows to reconstruct the relative position of theturntable with respect to the scanner, a set of 8-16 rangemaps are automatically taken with the object rotated in dif-ferent positions. A subsequent precise alignment done usingICP3, 11 is usually done to take off possible turntable calibra-tion errors. The well registered range maps are then mergedtogether in a single mesh using, in our case, a volumetricapproach5 While this approach works well for some kindof object, like for example human heads that have a roughlycylindrical shape, only objects with a very simple shape canbe completely acquired with just a single scanning turn. For

† from both the CCD and the laser emitter



Figure 5: Acquisition of a pig-shaped coin holder with ro-tary table and the mirror. On left side the , on right side: thelower part has been flipped and re-aligned to the upper partusing mirror position.

most real object it is necessary to complete the scanning pro-cess by adding some range maps or by placing the object ina different position onto the rotary table and then aligningand merging by hand the new data.

For this reason we have found that a very interesting useof mirrored scanning is its combination with a rotary table inorder to make the scanning process of small objects really acompletely unattended and automatic process. For this pur-pose we have chosen to place the mirror under the object asshown in figure2. As we can see in left side of figure5, theresult of the round scanning is composed by two parts: anupper part that corresponds to the portion of surface directlyscanned, while the lower is the one that is acquired throughthe reflection on mirror. Assuming that the rotary table hasbeen calibrated, we know the position of the mirror plane:it is then very easy to separate the upper (real object) fromthe lower (reflection) part, and flip the lower part to bring itback in the right coordinate system (right side of figure5).In this way we acquired an almost complete model in halfof the normal time. In table 1 we show some numerical re-sults on the use of the mirrored rotary table. The first columnreports the angle of inclination of the scanner (0 horizontal,90 looking vertically down), the second and third column re-port, respectively, the area of the upper surface (directly seenby the scanner) and lower surface (acquired exploiting thereflection). The third and fourth column reports the surfacearea (absolute and relative to the complete object) that itsobtained after the merging of both parts. The last column re-ports the total surface of the object resulting from a standardscanning of the object obtained taking 2-3 turns in differentpositions plus some range maps to cover difficult parts. Itcan be observed that the incidence angle of the scanner af-

Angle Upper Lower Merged Perc. Total

Head 30◦ 553.82 276.14 591.25 96.3% 613.3537◦ 538.41 314.74 588.63 95.9%45◦ 512.25 416.40 592.16 96.5%53◦ 471.57 109.59 555.90 90.6%

Pig 30◦ 342.05 238.71 435.02 98.8% 440.0937◦ 328.15 294.83 438.05 99.5%45◦ 310.61 267.49 434.89 98.8%53◦ 291.14 226.09 422.65 96.0%

Gargoyle 30◦ 324.90 214.43 394.05 90.7% 434.3737◦ 302.34 193.45 385.18 88.6%45◦ 288.48 209.31 382.83 88.1%53◦ 241.38 124.53 330.58 76.1%

Table 1: Measures of the area of the surfaces that are ac-quired directly, through the mirror and the combination ofboth. Varying the incidence angle of the scanner affects thecompleteness of the final surface.

fects the completeness of the surface: optimal angle dependsheavily on object geometry and is between 30 and 45 degree.

5. Hand held mirror

The most interesting use of mirror is to help the scanningof hard to be reached parts. A typical example of missingdata is the chin of the Michelangelo’ s David (Figure3) thatcould not be acquired in a complete manner because it wasnot possible to put the scanner in the right position (with-out touching the statue). of the statue. This kind of problemis quite frequent in the context of scanning cultural heritageobjects where it is not often possible to freely move the ar-tifacts to be scanned. With the purpose of helping the scanprocess we have adopted a small mirror of 100x250 mm thatcan be either placed by hand or mounted on a photographictripod support for a more stable setup. Fig. 3 shows our handheld mirror fixed to the tripod.

5.1. Locating the mirror and flipping the geometry

Clearly, to use the added geometry that is acquired throughthe mirror, we need to know the mirror position with respectto the scanner. For this purpose we have placed six opticalmarkers near to the edges of the mirror. The markers aredrawn in colors that are easily acquired by the 3d scanner:white and red (Fig.3). The position in image space of themarkers is detected by analyzing the color CCD image re-turned by the Minolta 900 scanner. Then we can easily findon the scanned geometry the position of the markers andtherefore recovering the position of the mirror. Once youhave located the mirror you have to identify the mirroredgeometry and flip it with respect to the mirror. To identifythe mirrored geometry it is sufficient to intersect the whole



Figure 6: Recovering the lower part of hair of a statue. Onthe left the result of the scanning before the processing, onthe right the result obtained after the mirror plane detectionand geometry reflection and merging.

Figure 7: When scanning hidden parts with a small mirrorpart of the scanned geometry (in blue) has to be clipped andflipped in the correct position.

scanned geometry with, in sequence, the frustum pyramiddefined by the center of projection of the camera and the bor-der of the mirror, the frustum pyramid defined by the laseremitter and the mirror border and then the half plane behindthe mirror. Fig.7 illustrates this configuration in 2D. On theleft a concave object that cannot be entirely scanned, the ac-quired geometry is shown as a thick blue line. On the rightthe insertion of a mirror (the black thick line) allows the ac-quisition of a newer portion that appears behind the mirror(in blue). Once clipped against the camera and laser frustumand flipped with respect to the mirror plane it is placed in thecorrect position (in red).

5.1.0.1. Results To evaluate the gain that can be obtainedby exploiting mirrors during scanning we fixed a small gar-goyle onto a small flat pedestal in order to introduce a con-strain in the scanning process and then we tried to acquire thestatue without moving it. Figure8 shows the result of thisexperiments, on the left the result of the scanning without themirror; many small parts of the statue where impossible to

Figure 8: Scanning a constrained object, a small statue fixedon a pedestal (bottom). Left and center image show, respec-tively, the results of the acquisition without and with the helpof a small mirror.

be scanned without removing it from the pedestal. With thehelp of a small mirror, almost the whole surface was recov-ered, yielding a approximative gain in the acquired surfaceof 4% (it is not a very high gain in terms of percentage, butare added surface otherwise unreachable).

6. Conclusion

We have presented and discussed the use of mirrors to im-prove the 3d scanning process when using a laser stripe scan-ning device. We have shown how the combined use of a mir-ror and a rotary table allows to make almost complete objectreconstructions using just a single pass round scan. More-over, we have investigated the use of freehand mirrors forthe acquisition of hard to be reached parts showing how awell placed small mirror can help a lot the scanning of hid-den parts. Both scenarios presents clear benefits and can beeasily integrated inside the usual 3D acquisition pipeline.

Acknowledgements

We acknowledge the financial support of the EU IST-2001-32641 “ViHAP3D” project and of the FIRB - “MACRO-Geo” project. NOTE: this is a revised and shortened versionof a paper that will appear in 3DIM 2003 international con-ference.

References

1. F. Bernardini, J. Mittleman, H. Rushmeier, andG. Taubin,Case study: Scanning Michelangelo’s Flo-rentine Pieta’, ACM SIGGRAPH 99 Course Notes,Course 8, August 1999.1



Figure 9: Automatic, unattended scanning of the gargoyle and of the statue head with a rotary table and a mirror; columnsfrom left to right: the upper part acquired directly by the scanner, the lower part acquired through reflection (already flipped)and the merged meshes.

2. F. Bernardini and H. E. Rushmeier,The 3D ModelAcquisition Pipeline, Computer Graphics Forum21(2002), no. 2, 149–172.1

3. P. J. Besl and N. D. McKay,A method for registrationof 3-D shapes, IEEE Transactions on Pattern Analysisand machine Intelligence14 (1992), no. 2, 239–258.3

4. F. Chen, G.M. Brown, and M. Song,Overview of three-dimensional shape measurement using optical methods,Optical Engineering39 (2000), no. 1, 10–22.1

5. B. Curless and M. Levoy,A volumetric method forbuilding complex models from range images, Comp.Graph. Proc., Annual Conf. Series (SIGGRAPH 96),ACM Press, 1996, pp. 303–312.3

6. D.Q. Huynh,Affine reconstruction from monocular vi-sion in the presence of a symmetry plane, Proc. ICCV1999, 1999, pp. 476–482.2

7. M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz,D. Koller, L. Pereira, M. Ginzton, S. Anderson,J. Davis, J. Ginsberg, J. Shade, and D. Fulk,The Dig-ital Michelangelo Project: 3D scanning of large stat-ues, SIGGRAPH 2000, Computer Graphics Proceed-

ings, Annual Conference Series, Addison Wesley, July24-28 2000, pp. 131–144.1

8. I-Chen Lin, Jeng-Sheng Yeng, and Ming Ouhyoung,Extracting 3d facial animation parameters from mul-tiview video clips, IEEE Computer Graphics & Appli-cations22 (2002), no. 6, 72–80.2

9. Marc Rioux,Laser range finder based on synchronizedscanners, Applied Optics23 (1984), no. 21, 3837U44.2

10. C. Rocchini, P. Cignoni, C. Montani, P. Pingi,R. Scopigno, R. Fontana, L. Pezzati, M. Cygielman,R. Giachetti, and G. Gori,3D scanning the Minerva ofArezzo, ICHIM’2001 Conf. Proc., Vol.2, Politecnico diMilano, 2001, pp. 265–272.1

11. S. Rusinkiewicz, O. Hall-Holt, and M. Levoy,Real-time 3d model acquisition, Comp. Graph. Proc., AnnualConf. Series (SIGGRAPH 02), ACM Press, July 22-262002, pp. ??–??3


enhancing the 3d laser scanning pipeline using...

Documents