first sea trials of a laser aided three dimensional...

7
First sea trials of a laser aided three dimensional underwater image mosaicing technique L. Brignone, M. Munaro, AG. Allais, J. Opderbecke IFREMER, Centre de Méditerranée La Seyne sur Mer, France Abstract- Georeferenced optical surveys of the seabed are obtained by composing mosaics of underwater images to allow scientists to study marine habitats, classify their features, quantify the population and measure the evolution over time. These are sought after tools but extremely complex to produce given the extent of the surfaces to map and the limited swath and range obtainable with standard diver or ROV operated image surveys. In this article we describe the development of an original technique based on the integration of laser light projection and digital image processing for the synthesis of georeferenced 3D optical maps of underwater scenes. For testing purposes a full scale payload prototype has been designed and integrated to IFREMER’s experimental AUV (Autonomous Underwater Vehicle) Vortex. Promising results are shown using data collected during dedicated sea trials performed in the Mediterranean Sea. I. INTRODUCTION AND STATE OF THE ART The technique presented in this article aims at improving the quality of underwater photo mosaics by estimating a three dimensional morphological model of the seabed to be textured with the acquired image data. A great emphasis is put on seabed image mosaicing as an invaluable tool for the assessment of the marine ecosystem and its evolution. Recent European directives have set tight requirements for both scale and quality of the seabed surveys in marine protected areas, which call for ever improving tools and techniques to be introduced. Image mosaics of underwater planar scenes can be composed by applying perspective transformations to each individual frames, using the homography matrix estimated from matching local features extracted automatically from the images [1]. Best results are obtained using offline computation to optimise a cost function defined for all collected images by exploiting detected overlaps from loops in the actual trajectory. The relative homography for each pair of overlapping images is computed and a global alignment method (bundle adjustment) is used to adjust the transformation to be applied to each image [2]. If navigation data of the underwater vehicle carrying the imaging sensor is available, it can be used as initial estimation of the trajectory and to bind the position of each image in the mosaic. Navigation information can also be included into the cost function to restrict positional errors and drift which often occurs in lack of enough loop closures [3]. When however the survey is conducted over rough terrain, local variation of elevation is no longer negligible when compared to the altitude of the vehicle. The planar scene hypothesis fails and a perspective model is no longer applicable for composing the images. In addition to that a 3D reconstruction of the environment is often needed by the scientists to better understand and contextualise their observations. Both iterative [4] and globally optimised [5] solutions have been proposed for this task. These approaches rely on feature extraction, matching and tracking from several images and require that each feature of the seabed appear in at least three images to allow estimation of a reliable 3D position. While this is generally feasible by using a video camera collecting several frames per second and by performing multiple passes over the area of interest, the approach is only suitable for small scale surveys carried out by ROVs. If larger areas are to be covered, AUVs provide the ideal platforms from which to acquire images, however their limited energy resources are unfortunately not compatible with the illumination requirements of video surveys. Finally, the ability to extract distinctive features for 3D structure reconstruction relies heavily on the nature of the observed scene and the overall quality of the result cannot therefore be guaranteed. In this paper we propose a different approach based on a laser projector and calibrated camera setup, together with the vehicle’s navigation sensors, as a mean to improve the continuity of the 3D model in a feature-based bundle adjustment framework or as a complete system for building textured 3D models of areas where the feature tracking process cannot be performed. II. PROPOSED TECHNIQUE Laser-camera systems have been used for years as inexpensive tools for metrology purposes and surface reconstruction [6]. In [7] and [8] a coded structured light is projected on the object, in order to identify a greater number of 3D points from a single image. A synthesis of several algorithms for peak detection and stripe indexing is presented and analysed in [9] where the surfaces estimated from different points of view are also fused with surface registration methods. Examples of the use of laser projectors in the underwater domain are only recently starting to appear in different automatic applications. In [10] and [11] a laser-camera setup is mounted on an AUV for estimating the distance and the angle with respect to a dam wall in order to provide distance feedback to the vehicle’s control system. Four low range laser pointers are used at close proximity [12][13] to improve dead-reckoning navigation and motion estimation at slow speed. The use of a sheet laser, is described in [14] to enable to identify a flat area of the seabed suitable for the controlled “landing” of an AUV. A similar system is used

Upload: others

Post on 25-Mar-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

First sea trials of a laser aided three dimensional underwater image mosaicing technique

L. Brignone, M. Munaro, AG. Allais, J. Opderbecke IFREMER, Centre de Méditerranée

La Seyne sur Mer, France

Abstract- Georeferenced optical surveys of the seabed are

obtained by composing mosaics of underwater images to allow scientists to study marine habitats, classify their features, quantify the population and measure the evolution over time. These are sought after tools but extremely complex to produce given the extent of the surfaces to map and the limited swath and range obtainable with standard diver or ROV operated image surveys. In this article we describe the development of an original technique based on the integration of laser light projection and digital image processing for the synthesis of georeferenced 3D optical maps of underwater scenes. For testing purposes a full scale payload prototype has been designed and integrated to IFREMER’s experimental AUV (Autonomous Underwater Vehicle) Vortex. Promising results are shown using data collected during dedicated sea trials performed in the Mediterranean Sea.

I. INTRODUCTION AND STATE OF THE ART

The technique presented in this article aims at improving the quality of underwater photo mosaics by estimating a three dimensional morphological model of the seabed to be textured with the acquired image data.

A great emphasis is put on seabed image mosaicing as an invaluable tool for the assessment of the marine ecosystem and its evolution. Recent European directives have set tight requirements for both scale and quality of the seabed surveys in marine protected areas, which call for ever improving tools and techniques to be introduced.

Image mosaics of underwater planar scenes can be composed by applying perspective transformations to each individual frames, using the homography matrix estimated from matching local features extracted automatically from the images [1]. Best results are obtained using offline computation to optimise a cost function defined for all collected images by exploiting detected overlaps from loops in the actual trajectory. The relative homography for each pair of overlapping images is computed and a global alignment method (bundle adjustment) is used to adjust the transformation to be applied to each image [2]. If navigation data of the underwater vehicle carrying the imaging sensor is available, it can be used as initial estimation of the trajectory and to bind the position of each image in the mosaic. Navigation information can also be included into the cost function to restrict positional errors and drift which often occurs in lack of enough loop closures [3].

When however the survey is conducted over rough terrain, local variation of elevation is no longer negligible when compared to the altitude of the vehicle. The planar scene hypothesis fails and a perspective model is no longer applicable for composing the images. In addition to that a

3D reconstruction of the environment is often needed by the scientists to better understand and contextualise their observations. Both iterative [4] and globally optimised [5] solutions have been proposed for this task. These approaches rely on feature extraction, matching and tracking from several images and require that each feature of the seabed appear in at least three images to allow estimation of a reliable 3D position. While this is generally feasible by using a video camera collecting several frames per second and by performing multiple passes over the area of interest, the approach is only suitable for small scale surveys carried out by ROVs. If larger areas are to be covered, AUVs provide the ideal platforms from which to acquire images, however their limited energy resources are unfortunately not compatible with the illumination requirements of video surveys.

Finally, the ability to extract distinctive features for 3D structure reconstruction relies heavily on the nature of the observed scene and the overall quality of the result cannot therefore be guaranteed.

In this paper we propose a different approach based on a laser projector and calibrated camera setup, together with the vehicle’s navigation sensors, as a mean to improve the continuity of the 3D model in a feature-based bundle adjustment framework or as a complete system for building textured 3D models of areas where the feature tracking process cannot be performed.

II. PROPOSED TECHNIQUE

Laser-camera systems have been used for years as inexpensive tools for metrology purposes and surface reconstruction [6]. In [7] and [8] a coded structured light is projected on the object, in order to identify a greater number of 3D points from a single image. A synthesis of several algorithms for peak detection and stripe indexing is presented and analysed in [9] where the surfaces estimated from different points of view are also fused with surface registration methods.

Examples of the use of laser projectors in the underwater domain are only recently starting to appear in different automatic applications. In [10] and [11] a laser-camera setup is mounted on an AUV for estimating the distance and the angle with respect to a dam wall in order to provide distance feedback to the vehicle’s control system. Four low range laser pointers are used at close proximity [12][13] to improve dead-reckoning navigation and motion estimation at slow speed. The use of a sheet laser, is described in [14] to enable to identify a flat area of the seabed suitable for the controlled “landing” of an AUV. A similar system is used

Page 2: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

for high resolution micro-bathymetrunderwater archaeological sites [15] bcontour of the sheet laser profile in imageacquired images only feature the laser linimage data of the seabed) is available to estimated bathymetry.

The work presented in this article diffsolutions as it uses a 532nm (green) cinstead of a single line laser, and aims tolaser points and the image texture. In camera and laser projector are setup to whigher altitude (2m to 4m) in order to inswath compared to other approaches.

The choice of projecting a cross insteaallows to triangulate a higher number oeach individual photo, but requires a segmentation technique, capable of sbelonging to the two sections of the procedure is central to the overall origienables the correct triangulation of the dete

Care has been taken to demonstrateconditions compatible with the energy AUV both the laser cross and the features scene can be extracted from the collectedturn implies that the 3D structure and tscene can be acquired in a concurrent mann

III. EXPERIMENTAL IMPLEME

A. Laser – camera configuration The configuration of the AUV mo

considered in this work features a verticacamera and a laser projector on the side wis offset by α =25° from the camera axiThe angle derives from consideratiocompromise for 3D triangulation and the ocontain the baseline between camera andangle is fixed, the baseline is given as mean altitude considered for the survey, inlaser cross aimed at the center of the opticavalue used in the experiments is d = 1.2 m.

The laser emitter is oriented so that thprojected cross lie at a 45° angle witcamera’s reference frame, as shown incalibration of the described AUV opparamount for the overall performance of must be done prior to deployment over a fl

Figure 1a (left) – camera-laser setup Figure 1b – (rig

ric mapping of by extracting the e space. Since the ne, no texture (i.e.

be applied to the

fers from previous ross pattern laser o extract both the addition to that,

work at a relatively ncrease the actual

ad of a single line f 3D points from more elaborated

separating points laser cross. This

inal approach and ected points.

e that in lighting constraints of an of the underwater

d images. This in the texture of the ner.

ENTATION

ounted equipment ally down-looking whose optical axis s (see Figure 1a).

ons on a valid operational need to d laser. Once the a function of the

n order to have the al scene. A typical . e segments of the th respect to the n Figure 1b. The ptical payload is the technique and

lat surface.

ght) Real image example

A prototype implementationbeen developed, integrated anexperimental AUV Vortex as d

B. Laser point detection In order to identify points be

cross in image space, the technseeks for points of correspondgreen channel and its seimplemented by horizontallyapplying the following steps:

- the green channel valunormalised between 0 an

- for every line its secoconvolving it twice [-1 -1 -1 0 1 1 1].

- separate thresholds are and to its second derivaboth threshold tests sicandidates.

In the underwater scenes coexperimental work it was foungreen channel intensity thresthree and six standard deviatioimage, depending on the lightinthe case of the second derivatiinferior to -5σ are kept, as thgreen channel are sought for.

A successful candidate is positive and matching negaderivative as the image is scanbottom. Orphan peaks in a rowbe correctly associated with onlaser cross especially when thterrain’s characteristics. Thiindividual parameter, i.e. thdeviation of the image used to green channel. High values (5σtaken with poor illumination, ware found to be more suitconditions, when images are extract and exploit both lasefeatures at a time.

A typical result is shown points on an individual scan liare highlighted (right). Lone hidiscarded as orphans (blue poin

Figure 2a (left) – line scanning F

n of the optical payload has nd fully tested on IFREMER’s discussed further.

elonging to the projected laser nique developed in this work

ding peaks of intensity in the econd derivative. This is y scanning the image and

ues of the current image are nd 255 to enhance contrast. ond derivative is computed with a kernel defined by

applied to the green intensity ative and the points that pass multaneously are elected as

ollected in the course of this nd that typical values for the shold usually range between ons (σ) above the mean of the ng conditions of the scene. In ive all the points with a value e maxima of intensity of the

identified by a sequence of ative peak in the second nned line by line from top to

w are discarded as they cannot ne or the other branch of the his is fragmented due to the s method depends on an e multiple of the standard compute the threshold for the

σ to 7σ) work well for images while lower values (3σ to 4σ) ed to intermediate lighting

taken with the purpose to er points and seabed texture

in Figure 2 where detected ine (left) and the whole cross its on individual scanlines are nts in rightmost image) .

Figure 2b(right) – Point detection

Page 3: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

C. Cross segmentation In order to successfully compute the tcoordinates of the points highlighted bnecessary to single out each branch of the cExperiments show how the laser cross bacimage acquired by the camera appears defoshortened or broken into smaller segmentsvariations (local elevation) in the scecamera/laser introduces a strong hypdisplacement of laser points in the image.case of a planar scene, a three dimensionthe laser points to be displaced solely aloaxis of the image reference frame. This phexplained with projective geometry and athat if two laser point candidates are idenrow of the image, they ought to belong to of the cross. Considerations on the verticalrow in question with respect to the centerallow to discriminate points belonging tobranch. This implies that effective cross seon the ability to accurately pinpoint the ccross in image space. A simple soimplemented and shown to perform well odataset collected to support this developprinciple is described as follows.

First the double threshold technique deIII.B is applied to the green channel oferosion operators are subsequently applieimage, featuring a -45° and 45° segrespectively. The intersection of the pointhe process of double erosion are likely to in the central part of the cross and the ithese are considered as successful candidacentre. A simple geometric test is applied by assuming it as the center of a circlintersections with the thresholded grecandidate featuring the most uniformly spis chosen as the centre point. The operationsuccessful candidate is found or else the op

The overall robustness of the proposhown in the results in Figure 3 depictinexperiences. The method allows to single branch of the cross (blue and red poipresence of severe and discontinuo(fragmentation) of the laser cross due tsharp contours. This is considered a validlife scenarios encountered in underwater nD. Triangulation and outlier detection

The estimation of the 3D position of thpoints in a given scene relies upon kgeometrical and optical configuration of timage acquisition payload. These incluparameters of the camera, especially the pfocal length, as well as the extrinsic paramnamely the laser-camera baseline and the optical axes. Given the extrinsic parameterequations of the two planes that describe

three dimensional by the cross it is cross individually. ck-projected in the formed, lengthened, in presence of 3D

ene. The specific pothesis on the . Compared to the nal contour causes ong the horizontal henomenon can be allows to conclude ntified on a given different branches l coordinate of the

r of the laser cross o the left or right egmentation relies centre of the laser olution has been over the extensive pment work. The

scribed in Section f the image. Two ed to the resulting gment as kernel nts obtained from include the pixels nevitable outliers; ates for the cross’ to each candidate

le to identify the en channel. The

paced intersections n is iterated until a peration fails. osed technique is ng laboratory test out correctly each ints) even in the ous deformation to projection over d step towards true atural scenes.

he identified laser knowledge of the the AUV mounted ude the intrinsic

principal point and meters of the setup,

angle between the rs of the setup, the

e the projection of

Figure 3 – Cross detection, center

laboratory tests ov

the laser cross can be easily desystem. The 3D positions ocomputed following a sequencThe 2D position of every detecis translated into camera parameters. The estimated 3D found at the intersection betwepoint lies and the line joiningoptical center. It is therefore nprocess to work to differentiatlaser planes they belong to. Thprocess are improved by erejection scheme. The z compoin an histogram with bins spbelonging to bins whose pothreshold are considered outlierIn Figure 4a (left) a representaresult is shown where the axes(L) reference frame are indicategreen mesh is used to represenand blue points indicate the resprocess for the photo shown infinally indicate the three dimepoint as estimated by the triangThe original image was taken fresh water pool, where the awas provided by a Tritech PAdimensional optical triangulatiof 2306mm, which in turn inground truth; the accuracy of confirmed by the limited stcomponents of the laser poin4.5mm. This performance was comprehensive dataset featurcollected in the fresh water algorithm over synthetic scetypical example is shown in Fcardboard box was illuminatedphotographed. The measured camera approach in this case is

r identification and segmentation in er sharp contours

erived in the camera reference f the laser points are then

ce of conventional operations. cted laser point in image space

frame using the intrinsic position of each laser point is een the laser plane where the g the point and the camera’s necessary for the triangulation te the points according to the he results of the triangulation employing a simple outlier onent of the points is collected paced of 100mm. The points opulation is below a given rs and discarded.

ation of a typical triangulation s of the camera (C) and laser ed by coloured segments. The

nt the image plane, and the red sults of the cross segmentation n Figure 1b. The green points ensional position of the laser gulation process.

over a planar surface of the altitude measure of 2236mm A500 echosounder. The three ion process yields a measure ndicates a 3% error over the the obtained result is further tandard deviation of the z nts which was evaluated at found to be consistent over a

ring 50 underwater photos pool. The results using the

enes were also analysed; a igure 4b where a 225mm tall d with the laser projector and

height obtained with laser-s 221mm.

Page 4: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

Figure 4a (left) – Cross 3D reconstruction on planar scene Figure 4b (right) Cross reconstruction over cardboard box

E. Camera motion estimation The global approach presented in this article is based on

the acquisition of underwater still images from an AUV following a given trajectory. The 3D model of the seabed corresponding to each image is reconstructed using the laser cross procedure described in the previous sections. In order to build a complete and continuous model of the observed seabed, it is necessary to place accurately each image with respect to a georeferenced coordinate frame. For this purpose, dead reckoning navigation data can be used as standard solution [15], however this may be flawed by the inevitable positional errors due to drift.

At the current stage of this work we have on the other hand opted to focus on an image processing based camera pose computation for motion estimation, with the aim to merge it with navigational data in the future.

Given a pair of overlapping images, the feature-based technique devised for 6 DOF camera motion estimation is implemented using SIFT features extracted from each images. Positive matches are selected using known techniques [1].

The series of matching points allow to estimate the homography matrix (H) using the direct linear transformation method [16] and the essential matrix (E) using a RANSAC-based least squares method [17].

From these matrices it is possible to obtain the motion parameters (rotation R and translation T) up to a scale factor k by singular value decomposition (SVD), as proposed in [18] for H and in [19] for E.

In order to solve for the scale factor k hat affects the translation T a third image that overlaps the other ones can be used if available, but in this work we rather opt to exploit the triangulated laser points to estimate the scale factor from only two images. For this purpose the couple (f,p) of image feature and image laser point whose distance is minimum is chosen. It can be safely supposed that f and p refer to 3D points with similar distances to camera. Thus, the scale factor k is estimated by dividing the 3D distance of the laser point from the camera optical centre by the 3D distance of the feature computed with the motion parameters (R,T/k).

The essential matrix must be used for motion estimation when the scene induces significant parallax, but, when the scene is planar or the parallax effect is small, the homography matrix gives a more robust solution [4]. One of the novel aspects proposed here lies in the exploitation of

the 3D information gathered with the triangulation of laser points to help decide upon the use of one or the other technique for motion estimation. Camera motion is estimated using the essential matrix E when the laser points referred to the current pair of images fail to successfully fit a plane according to a RANSAC-based estimation [20]; the homography transformation described by H is considered in the opposite case.

IV. TESTING AND PRELIMINARY RESULTS

An initial set of experiments was performed in a fresh water pool whose particular geometry features surfaces of varying orientation (slope). The pool’s bottom is artificially textured with a poster image representing a typical underwater scene. A dataset consisting of fifty 10 Megapixel images was collected over a lawnmower trajectory at a constant altitude of about two metres from the bottom and aiming for subsequent image overlap between 50% and 75%.

The camera used for acquiring the photos was a remotely controlled Canon Powershot G9 and the laser projector is a 40mW Z-Laser ZM18 emitting a 532nm (green light) cross. The camera-laser baseline was set with d=104 cm and α=25° for a target altitude of 2.5m. The lighting conditions were adjusted in order to acquire photos with good visibility of both the laser cross and the poster texture, as shown in the example in Figure 1b belonging to the acquired dataset.

From these photos the laser cross was extracted, segmented and the resulting points triangulated with the techniques described earlier, obtaining the 3D position of the laser points with respect to each camera reference system.

The 6-DOF displacement of the camera between each pair of consecutive images is estimated with the feature-based technique presented in Section III.E because no navigation data could be provided during the tests in the pool. The displacement of each camera position with respect to the world reference system (that in this case coincide with the first image reference frame) is obtained by iteratively multiplying its transformation (translation and rotation) matrix with the matrix of the previous image, as in:

RTiW = RTi

i-1 ⋅ RTi-1W

where RTij indicates the transformation matrix containing

the motion information (R,T) between camera j and i, and W indicates the world reference system.

The computed transformation matrices allow to associate all the triangulated laser points to the same reference system.

The resulting model well represented the actual geometry of the pool which features a negatively sloping (about -45°) section followed by a flat section. Small imperfections at the interface of the two differently oriented surfaces are found in the reconstructed model due to errors in motion estimation. The laser points are well extracted and triangulated from all the images and no outliers hinder the perceived quality of the reconstruction, despite the limited lighting conditions. The dense series of three dimensional points is joined in a surface obtained by applying Delaunay

-1000-500

0500

-400-200

0200

-2000

-1500

-1000

-500

0L

C

z [m

m]

-200

0

200

-200 -100 0 100 200 300

-1900

-1850

-1800

-1750

-1700

y [mm]x [mm]

z [m

m]

Page 5: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

Figure 6 – 3D view of the textured model of the fres

from joint image mosaicing and laser tria triangulation on the (x,y) coordinates ofcollected images are then mapped to the texture from different photos accordinmethod, i.e. every pixel of the surface image data from the photo where that pixcentre. In order to improve the qualitysurface, the laser cross is erased from the osimply cutting the corresponding pixels aslaser detection procedure.

The obtained textured model is sho(perspective view) and Figure 7a (top viewresult obtained with the presented techniqby comparing the top view of the model wbi-dimensional mosaic of the scene (Figusing a feature-based bundle adjustment aa planar scene hypothesis [2].

A detailed analysis shows how a dimensional mosaic technique fails whereof the scene changes and the planar hypstands. This causes the mismatches inportion of the 2D mosaic which are locatchange section of the pool. On the contrarthe 3D textured model shows how thepresented here is robust over slope chexploits the three dimensional reconstrucOnly minor artefacts are introduced in cthe model as an effect of poor motionestimation of the camera. Introducing thnavigation sensors will considerably reduproblem.

V. SEA TRIALS AND LEARNT LE

The described techniques were thorouseries of sea trials organised by IFMediterranean coast close to Toulon.

The experimental AUV Vortex wapurposely developed optical survey payloa

sh water pool obtained angulation

f the model. The surface creating a

ng to the closest is textured using el is closest to the y of the textured original images by s identified by the

own in Figure 6 w). A fundamental que can be shown

with a conventional gure 7b) obtained algorithm based on

conventional bi-e the overall slope pothesis no longer n the highlighted ted after the slope ry, the top view of e global approach hanges as it fully ction of the scene. certain portions of n and orientation he feedback from uce this particular

ESSONS

ughly tested in a FREMER in the

s equipped with ad consisting of a

Figure 7a (left) – Top view of the 3D

2D mosaic obtained enforcin

Canon G9 digital still camera aFigure 8). Several autonomouplanned over shallow water area mixed compound of sand, grpath featured multiple passes areas spanned approximately 2Typical velocity setpoint was 10 Megapixel images were cinterval to allow sufficient oshots. Ambient light at such shallowsignificant problems to allowover the texture of the seabeemission power of the projectoimages captured at night coulddue to hardware issue with thethe vehicle. This experience yiresults: the validation of thtechnique using the feature extcaptured from a moving Aextraction from the detection across. This latter result is particimages collected at night and vehicle a 3D bathymetric maptriangulated laser points. A smthe series of discrete 3D preconstruct a high resolutiocompares well with the knownarea (Figure 9).

The experience from this fithat improvements to imagimplemented in order to allowthe laser cross to be simultaneowill be explored next is to relycollect two images in rapid slight to illuminate the scene aone in complete darkness to alThe spatial offset between thcompensated using dead

D textured model; Figure 7b (right) ng planar scene hypothesis

and a laser cross projector (see us mission trajectories were eas where the seabed featured ravel and rocks. The mission at 3m altitude and the target

250m x 250m per each profile. 0.3m/s. Several hundreds of

collected at 4 to 6 seconds overlap between consecutive

w depths (10 to 20 m) posed visibility of the laser cross d despite the relatively high r (40 mW). On the other hand, d only feature the laser cross

e dedicated lighting system on ielded however two important he overall image mosaicing traction approach over images AUV, and the 3D contour and segmentation of the laser cularly interesting as using the the navigation data from the

p was reconstructed from the moothed meshed surface from points is then processed to n bathymetric chart which n bathymetry of the surveyed

irst set of sea trials indicates ge acquisition need to be w both the seabed texture and ously acquired. A solution that y on ambient darkness and to sequence: one using a strobe and reveal the seabed texture, llow capturing the laser cross. he two images can safely be d reckoning navigation

Page 6: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

Figure 8 - The experimental AUV Vortex on ship’s deck featuring the

installed digital camera (red marker) and laser projector (green marker) information given the limited time lag (as little as 2 seconds as early experiments show) between consecutive shots.

VI. DISCUSSION AND CONCLUSIONS

This paper has presented an original method to perform accurate reconstruction of three dimensional underwater scenes from collected images. As part of the experimental validation, a self contained optical mapping payload has been prototyped, integrated and successfully tested on the AUV Vortex.

The results collected over a series of experiments ranging from laboratory setup scenes to underwater images show the accuracy of the three dimensional model estimation technique using the projected laser cross. The robustness of the method in varying lighting condition has been shown as outliers are successfully rejected to ensure that the resulting model matches well the measured ground truth. In this respect, the measured accuracy has been evaluated in the millimeter range by comparing the obtained models to measurements from a single beam echo sounder (pool tests) or the known geometry of the actual objects (laboratory scenes).

The results from experiments in the fresh water pool where both the texture and the laser points were simultaneously acquired show the overall accuracy of the 3D reconstruction of the geometry of the scene. Having introduced a method to select the image matching technique that best suits the detected 3D characteristics of the scene, an improvement over the accuracy of conventional bi-dimensional mosaic is also shown on the same dataset.

Future efforts will focus on developing a fully three dimensional bundle adjustment method for globally estimating the camera motion and the scene structure by considering all the acquired images. Thus, the bundle adjustment technique will provide an optimal solution to building and texturing a 3D model of the seabed if a

Figure 9 – High resolution bathymetry from reconstruction of triangulated

laser cross points

sufficient number of features are detected and the images overlap. In the areas where these hypotheses no longer stand, the structure estimation with laser points triangulation will allow to continue to add points to the model, as motion estimation would mostly relay on dead reckoning navigation due to the lack of characteristic features.

ACKNOWLEDGEMENTS

This research has been partly sponsored by the Marie Curie Research Training Network FREEsubNET, Contract n. MRTN-CT-2006-036186; http://www.freesubnet.eu.

The ongoing cooperation and fruitful exchange with colleagues at the Underwater Robotics Research Center (CIRS) of the University of Girona is also acknowledged as well as their support and hospitality during the fresh water pool tests.

REFERENCES

[1] Brown, M. , Lowe, D., 2007. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, vol. 74, pp. 59-73.

[2] Gracias, N., 2002. Mosaic-based visual navigation for autonomous underwater vehicles. PhD thesis, University of Girona.

[3] Ferrer, J., Elibol, A., Delaunoy, O., Gracias, N. , Garcia, R., 2007. Large-area photo-mosaics using global alignment and navigation data. Proceedings of the OCEANS '07, Vancouver, Canada.

[4] Nicosevici, T., Gracias, N., Negahdaripour, S. , Garcia, R., 2009. Efficient 3D modeling and mosaicing. Journal of Field Robotics, vol. 26, no. 10, pp. 759-788.

[5] Pizarro, O., 2004. Large scale structure from motion for autonomous underwater vehicles surveys. PhD thesis, Massachusetts Institute of Technology.

[6] Forest, J., 2004. New methods for triangulation-based shape acquisition using laser scanners. PhD thesis, University of Girona.

Page 7: First sea trials of a laser aided three dimensional ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro_Oceans11.pdf · First sea trials of a laser aided three dimensional underwater

[7] Salvi, J., Pagès, J. , Batlle, J., 2004. Pattern codification strategies in structured light systems. Pattern Recognition, vol. 37, pp. 827-849.

[8] Matabosch, C., Fofi, D., Salvi, J. , Forest, J., 2005. Registration of moving surfaces by means of one-shot laser projection. Pattern recognition and image analysis, vol. 3522/2005, pp. 145-152.

[9] Matabosch, C., 2007. Hand-held 3D-scanner for large surface registration. PhD thesis, University of Girona.

[10] Sakai, H., Tanaka, T., Ohata, S., Ishitsuka, M., Ishii, K. , Ura, T., 2004. Applicability and improvement of underwater video mosaic system using AUV. OCEANS '04. MTTS/IEEE Techno-Ocean'04.

[11] Ohata, S., Ishii, K., Sakai, H., Tanaka, T. , Ura, T., 2006. An autonomous underwater vehicle for observation of underwater structure. International Congress Series, vol. 1291, pp. 277-280

[12] Caccia, M, 2006. Laser-triangulation optical-correlation sensor for ROV slow motion estimation. Journal of Oceanic Engineering, vol. 31, no.3: pp. 711-727.

[13] Caccia, M., Bruzzone, G., Ferreira, F. , Veruggio, G., 2009. Online video mosaicing through SLAM for ROVs. OCEANS '09, Bremen, 11-14 May 2009, pp. 1-6.

[14] Sangekar, M. N., Thornton, B., Nakatani, T. , Ura, T., 2010. Development of a landing algorithm for autonomous underwater vehicles using laser profiling. OCEANS '10, Sidney.

[15] Roman, C., Inglis, G. , Rutter, J., 2010. Application of structured light imaging for high resolution mapping of underwater archaeological sites. OCEANS '10, Sidney.

[16] Cambridge University Press, 2004. Multiple View Geometry in Computer Vision. Hartley, R. , Zisserman, A., second edition.

[17] Armangue, X., , Salvi, J., 2003. Overall view regarding fundamental matrix estimation. Image and Vision Computing, vol. 21, pp. 205-220.

[18] Faugeras, O. , Lustman, F., 1988. Motion and structure from motion in a piecewise planar environment. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 2, pp. 485-508.

[19] Lourakis, M. , Deriche, R, 1999. Camera self-calibration using the singular value decomposition of the fundamental matrix. Asian Conference on Computer Vision, Taipei, Taiwan, pp. 403-408.

[20] Fischler, M. , Bolles, R., 1981. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM, vol. 24, pp. 381-395.