nrc-iit publications iti-cnrc | view planning for

National ResearchCouncil Canada

Institute forInformation Technology

Conseil nationalde recherches Canada

Institut de technologiede l'information

View Planning for Automated 3D Object Reconstruction and Inspection * Scott, W., Roth, G., and Rivest, J.F. March 2003 * published in The ACM Computing Surveys, 35(1): 64-96. March 2003. NRC 45866. Copyright 2003 by National Research Council of Canada

Permission is granted to quote short excerpts and to reproduce figures and tables from this report, provided that the source of such material is fully acknowledged.

View Planning for Automated Three-Dimensional ObjectReconstruction and Inspection

WILLIAM R. SCOTT and GERHARD ROTH

National Research Council of Canada

AND

JEAN-FRANCOIS RIVEST

University of Ottawa

Laser scanning range sensors are widely used for high-precision, high-densitythree-dimensional (3D) reconstruction and inspection of the surface of physical objects.The process typically involves planning a set of views, physically altering the relativeobject-sensor pose, taking scans, registering the acquired geometric data in a commoncoordinate frame of reference, and finally integrating range images into anonredundant model. Efficiencies could be achieved by automating or semiautomatingthis process. While challenges remain, there are adequate solutions to semiautomatethe scan-register-integrate tasks. On the other hand, view planning remains an openproblem—that is, the task of finding a suitably small set of sensor poses andconfigurations for specified reconstruction or inspection goals. This paper surveys andcompares view planning techniques for automated 3D object reconstruction andinspection by means of active, triangulation-based range sensors.

Categories and Subject Descriptors: I.2.9 [Artificial Intelligence]: Robotics—sensors;I.2.10 [Artificial Intelligence]: Vision and Scene Understanding—Modeling andrecovery of physical attributes; I.4.1 [Image Processing and Computer Vision]:Digitization and Image Capture—Scanning; I.5.4 [Pattern Recognition]:Applications—Computer vision

General Terms: Algorithms, Design, Measurement, Performance

Additional Key Words and Phrases: View planning, range images, object reconstruction,object inspection

1. INTRODUCTIONThe demand for high—quality three di-mensional (3D) virtual models of complexphysical objects is growing in a wide range

This work was partially sponsored by scholarships from the National Science and Engineering ResearchCouncil (Canada), the Ontario Graduate Student Scholarship program, and the University of Ottawa grad-uate scholarship program. The research was conducted while the principal author was a guest researcher atthe National Research Council of Canada’s Institute for Information Technology.Authors’ addresses: W. R. Scott and G. Roth, Computational Video Group, Institute for Information Tech-nology, National Research Council, M-50, 1200 Montreal Rd., Ottawa, Ont., Canada, K1A 0R6; email:{william.scott, [email protected]}; J.-F. Rivest, Defence Research Establishment Ottawa, 3701Carling Ave., Ottawa, Ont., Canada, K1A 0Z4.Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted with-out fee provided that the copies are not made or distributed for profit or commercial advantage, the copyrightnotice, the title of the publication, and its date appear, and notice is given that copying is by permission ofACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specificpermission and/or a fee.c©2003 ACM 0360-0300/03/0300-0064 $5.00

of applications (e.g., industrial, training,medical, entertainment, cultural, archi-tectural). Computer graphics can pro-duce synthetic models for some of these

ACM Computing Surveys, Vol. 35, No. 1, March 2003, pp. 64–96.

View Planning for Automated 3D Object Reconstruction and Inspection 65

applications by a combination of artis-tic and technical design processes. In thecomputer vision field, on the other hand,model acquisition is achieved by measur-ing the shape and visual properties of realphysical objects. Numerous applicationsrequire the computer vision approach toobject surface reconstruction.

Automating or semiautomating the cur-rent manual, time-consuming reconstruc-tion process would improve productiv-ity and quality, reduce operator skilllevel and rework requirements, and lowercost. While a limited number of commer-cial systems (Hymarc Ltd., Rapidscan,www.hymarc.com; Digibotics Ltd., Adap-tive Scan, www.digibotics.com) offer semi-automated tools to aid scanning, thereis presently no general purpose com-mercial system for automated objectreconstruction.

Both mechanical and optical geomet-ric measurement sensors are used for3D object reconstruction and inspec-tion. Mechanical Coordinate Measur-ing Machines (CMMs) use precise me-chanical movement stages and touchprobes to achieve very high measurementprecision—around 1 µm for a suitably cal-ibrated high-end machine operated by askilled technician. However, the acquisi-tion and operating costs for such precisionare high. With a relatively low data ac-quisition rate, CMMs are generally suit-able for sparse sampling applications only.While their measurement precision is in-ferior to CMMs, optical sensors have lowercapital and operating costs and are capa-ble of dense, noncontact sampling at highthroughput rates.

This survey deals with view planningfor automated high-quality object recon-struction or inspection by means of ac-tive, triangulation-based range cameras.In this context, view planning is the pro-cess of determining a suitable set of view-points and associated imaging parametersfor a specified object reconstruction or in-spection task with a range camera and po-sitioning system. By object reconstructionwe mean acquisition of a virtual computermodel of the surface of a physical object.This is normally a triangulated polygo-

nal mesh representation of surface geom-etry. Alternative surface representationssuch as splines, swept cylinders, and su-per quadratics are advantageous in spe-cial cases but are neither as general pur-pose nor as flexible as a triangular mesh.

While dealing mainly with object recon-struction, the techniques examined herealso apply to inspection. The later is aneasier problem, having the advantage of apreexisting model. Essential informationcan be collected off-line in a suitable datastructure, allowing different tasks to beplanned quickly online.

Many applications are concerned withshape only. A growing number also seekcoregistered surface geometry and surfacevisual texture or reflectance properties.Capturing high-quality surface geome-try (the focus of this survey) is a pre-requisite for the accurate measurementof reflectance properties which may beachieved simultaneously with some rangesensors [Baribeau et al. 1991].

The paper is structured as follows. Af-ter an overview of the problem (Section 2),view planning requirements are defined indetail (Section 3). We then reference re-lated surveys in the field (Section 4) be-fore presenting a survey of view planningmethods in two broad categories: model-based (Section 5) and non-model-based(Section 6). Finally, we compare (Section 7)and critique (Section 8) existing methodswith respect to the defined requirements,examine related issues (Section 9), andconclude with a discussion of the remain-ing open problems (Section 10).

2. PROBLEM OVERVIEW

2.1. Imaging Environment

The imaging environment (Figure 1) forobject reconstruction consists of a rangecamera, positioning system, various fix-tures, and the target object.

Range camera. The principal compo-nent of the imaging environment is arange camera—a sensor for 3D shape mea-surement. A wide variety of technologiesare used for measuring object shape [Besl1989]. Range cameras can be categorized

ACM Computing Surveys, Vol. 35, No. 1, March 2003.

66 Scott et al.

Fig. 1 . Imaging environment.

into active and passive devices. The mostcommon passive range-finding technol-ogy, stereo vision, can provide accuratemeasurement at short stand-off distances.However, passive stereo depends on vi-sual texture on the target object, is sub-ject to illumination constraints and in-terference, and can provide only sparsedepth measurement. With an integral il-lumination source, active sensors are ca-pable of dense range measurement andare less susceptible to external interfer-ence. However, the illumination source(frequently a laser) may impose systemsafety constraints. Active range sensorscan be divided into two subcategories,time-of-flight and triangulation. Time-of-flight systems require very accuratetiming sources and typically providemodest resolution (generally centimeterbut occasionally millimeter accuracy) forlonger range applications, that is, tensto thousands of meters [Amann et al.2001]. They are best suited to distancemeasurement and environment model-ing at medium to long ranges. Manydifferent types of triangulation sensorshave been developed and are in wideusage. All are based on the principleof triangulating a measurement spot onthe object from a physically separatecamera optical source and detector. Bysimple geometry, the x, z coordinates ofthe illuminated spot on the object arecalculated (Figure 2). In general, activetriangulation-based range cameras are ca-pable of very precise (≤100 µm), densedepth measurements (many samples persquare millimeter) over relatively small

Fig. 2 . Conventional active triangulation. (From El-Hakim and Beraldin[1994]; c©IEEE 1994.)

sensor frustums up to about a meter instandoff distance [El-Hakim and Beraldin1994]. In the following, z is range, x is dis-tance across the scan, f0 is the sensor focallength, p is the position of the imaged spoton the sensor detector, and θ is the laserscan angle:

z = d f0

p+ f0 tan θ, (1)

x = z tan θ. (2)

This survey focuses on triangulation-based active laser range scanners as theyare widely used for precise measurementfor both object reconstruction and in-spection. Several sensor attributes impactview planning. The optical baseline, thedistance between the laser and the opti-cal receiver, is significant with respect tothe measurement stand-off distance. Con-sequently, coverage shadow zones arisewith respect to visibility by the laser,the optical receiver, or both. The sensorfield of view, depth of field, and there-fore frustum volume are limited. Mea-surement precision and sampling densityare highly nonuniform within the frus-tum. Additionally, measurement is subjectto random nonisotropic geometric noise[Beraldin et al. 1993] and several artifactphenomena [Curless and Levoy 1996].

Positioning system. Imaging all sides ofan object requires a variety of viewing per-spectives. Thus, a positioning system is



required to move the sensor, the object,or both in some combination. An imper-fect positioning system introduces pose er-ror, which has two consequences. First, aplanned view is not the one actually ex-ecuted. Sufficiently severe pose error canrender view planning futile. Second, theresulting pose uncertainty necessitates animage registration stage.

Viewpoint space. Viewpoints should betreated as generalized viewpoints (v, λs)consisting of sensor pose v and a set ofcontrollable sensor parameters λs [Tara-banis et al. 1995b; Roberts and Marshall1997; Pito 1997a]. Thus each viewpointhas an associated sensor configuration.Sensor pose is limited by the degrees offreedom and range of motion of the posi-tioning system. Viewpoint space V is theset of generalized viewpoints defined bythe range and sampling of these param-eters.

Object. The object is the feature tobe modeled. Specifically, we wish to cap-ture its 3D shape. Object surface space Sis the set of 3D surface points sampled onthe object. These can be considered as ver-tices in the resulting object mesh model.

Fixtures. The object must be supportedby a fixture of some kind. Consequently,acquisition of all-aspect views will requirethe object to be repositioned on the fix-ture at least once and more likely sev-eral times. After each repositioning, thenew relative pose must be determined tobring the system into a single coordinateframe. Fixtures, the positioning system,and any other structures in the imagingwork space I introduce occlusion and col-lision avoidance considerations.

2.2. Reconstruction Cycle

The classical model building cycle(Figure 3) consists of four main phases—plan, scan, register, and integrate. Asequence of views, the view plan ornext-best-view (NBV) list N , must becomputed. The sensor must be moved tothe appropriate pose and configured withappropriate settings, after which scans

Fig. 3 . Object reconstruction cycle.

are taken. Unless the positioning systemis error-free, acquired range images mustbe registered by an image-based regis-tration technique such as the standardIterative Closest Point (ICP) method[Besl and McKay 1992]. Subsequently,registered images must be combined intoa single, nonredundant model—a processcommonly known as integration. Theprocess continues until some stoppingcriteria are satisfied.

These are the basic reconstructionsteps. Other related activities are brieflyas follows. Calibration of both sensor andpositioning system is an essential prereq-uisite and may need to be repeated aftercertain positioning system reconfigura-tions. Filtering of noise and artifacts is re-quired at several stages. Depending on theapplication, there may also be a require-ment for model compression or decima-tion and for texture mapping of reflectancedata onto the geometry [Baribeau et al.1991].

While challenges remain, adequate so-lutions exist to automate the scan-register-integrate functions.1 However,automated view planning remains an openproblem despite two decades of research.For large reconstruction tasks, the imageregistration and integration (model build-ing) functions can be very time consum-ing. By the time deficiencies in the partialmodel are discovered, the imaging teammay have left the site or access to the ob-ject may no longer be readily available.Hence, there is a need for a view planning

1 Research on ICP refinements remains an activefield of research. For example, see Rusinkiewicz andLevoy [2001] and Langis et al. [2001]. Research onmodel building also remains active, with a numberof open issues [Roth 2000].


68 Scott et al.

scheme capable of developing reliable viewplans in a timely manner for comprehen-sive object coverage in accordance withall technical criteria of the reconstructiontask.

A different approach is to automateonly the register-integrate functions andrely on human-based view planning andscanning. Huber [2001] used graph-basedmethods to increase the reliability ofmultiview registration and integration.Hebert [2001] used a hand-held scannerto sweep the object as one would usea paint sprayer. Both techniques employprogressive model building. Both requirenear real-time registration and integra-tion, difficult tasks whose computationalcomplexity is sensitive to model size andnumber of images taken. Registration ofsingle 1D scans faces reliability and sta-bility issues. Humans are relatively goodat high-level view planning for coverageof simple objects but even experienced op-erators will encounter considerable diffi-culty with topologically and geometricallycomplex shapes. A visualization feedbackloop can overcome some of these difficul-ties. Hand-held or even robotically sup-ported sensors will introduce pose errorwith consequences discussed later in thisreview. And, of course, heavy reliance on ahuman operator defeats the ultimate ob-jective of full automation. Notwithstand-ing these weaknesses, human-based viewplanning techniques are good candidatesas interim semiautomated measures forapplications with modest quality objec-tives. The balance of this review is re-stricted to machine-based view planning.

2.3. The View Planning Problem

From the foregoing, it is apparent thatthe view planning problem (VPP) involvesreasoning about the state of knowledgeof three spaces—object surface space S,viewpoint space V , and imaging workspace I . While tied to 3D surface geometry,S is amenable to 2D parameterization. Inthe unrestricted case, the pose componentof viewpoint space V is six dimensional—three position and three rotation. Config-urable sensor parameters such as laser

power and scan length can further raisethe dimensionality of generalized view-point space, which we indicate as 6D+.Imaging workspace I is the 3D region ca-pable of being viewed by the sensor overthe full range of the positioning system. Itis of concern primarily for visibility analy-sis and collision avoidance considerations.The complexity of the VPP is immediatelyapparent from the high dimensionality ofthe search spaces in S, V , and I .

Given this imaging environment and re-construction cycle, the view planning taskis deceptively simple, yet computationallycomplex. Expressed informally, the viewplanning problem is as follows:

For a given imaging environment and targetobject, find a suitably short view plan N sat-isfying the specified reconstruction goals andachieve this within an acceptable computationtime.

N will be a subset (preferably a verysmall subset) of viewpoint space V , thatis, N ⊂ V . As the View Planning Prob-lem has been shown to be NP-Complete([Tarbox and Gottschlich 1995]; [Scott2002]), in most realistic reconstruction orinspection tasks it is impractical to seekan absolute minimal length view plan.What is an “acceptably short” computa-tion time is application dependent. Inan industrial setting, object reconstruc-tion involves repetitive modeling of dif-ferent objects. In this case, throughputis critical, emphasizing time-efficient al-gorithms. Perhaps some aspects of qual-ity, such as the view plan length, may betraded off in the interests of computationalefficiency. In some reconstruction tasks,the primary objective is model quality andscanning efficiency is secondary. Inspec-tion applications involving repetitive ex-ecution of an inspection plan on a produc-tion line place a premium on view plan ef-ficiency over the time taken to create it.

Several other machine vision applica-tions also involve view planning, suchas environment modeling [Sequeira andGoncalves 2002], autonomous exploration[Tremblay and Ferrie 2000], and sensor-based robot path planning [Yu and Gupta2000]. View planning requirements,



techniques, and sensors appropriatefor these applications are somewhatdifferent than for high-quality objectreconstruction/inspection. These applica-tions typically have much lower accuracyand sampling density requirements butpotentially larger models, so differentsensors may be appropriate such astime-of-flight (optical, sonar or radar) aswell as passive optical technology.

2.4. Performance-Oriented Reconstruction

High-quality model building requires viewplanning for performance-oriented recon-struction [Scott et al. 2000], which is de-fined as model acquisition based on a set ofexplicit quality requirements expressed ina model specification. High-quality inspec-tion tasks are also generally specification-driven. In addition to all-aspect coverage,measurement quality may be specified interms of precision, sampling density, andperhaps other quality factors. Specifiedmeasurement quality may be fixed or vari-able over the object surface.

Performance-oriented view planning re-quires suitable models of both sensor andpositioning system performance. Thesecan be combined in an imaging environ-ment specification. Specifically, it requiresthe following:

—a sensor model with a description of thecamera geometry and frustum geome-try and a characterization of measure-ment performance within the calibratedregion; and

—a positioning system model describingthe degrees of freedom, range of motion,and positioning performance within themovement envelope.

There has been relatively little workon performance-oriented view planning.Cowan and Kovesi [1988] used a con-straint satisfaction approach for sensor lo-cation subject to task requirements whichincluded resolution, focus, field of view,visibility, view angle, and prohibited re-gions. The MVP system developed byTarabanis et al. [1995] automatically syn-thesizes views for robotic vision tasks withintensity cameras based on a task speci-

fication as well as a geometric model ofthe scene and optical models of the sen-sor and illumination sources. The MVPtask specification includes visibility, fieldof view, resolution, focus, and image con-trast. Soucy et al. [1998] described a sys-tem for automatically digitizing the sur-face of an object to a prescribed samplingdensity using a contour following scheme.Prieto et al. [1999, 2001] set CAD-basedinspection criteria based on range sensorperformance characterization. A numberof authors have considered grazing angleas a subjective quality measure. Other-wise, the objective of most work in the fieldhas been explicitly or implicitly limited tofull surface coverage. Recently, Scott et al.[2000, 2001b] used a model-based, mul-tistage approach to performance-orientedview planning based on an input specifica-tion for sampling precision and density.

3. VIEW PLANNING REQUIREMENTS

In this section, we define requirementsand performance measures for the viewplanning process in greater detail.2 Therequirements are drawn up from a re-search perspective. Additional develop-ment and system integration details willarise when moving prototype view plan-ning algorithms to a production configura-tion of an automated object reconstructionor inspection system.

3.1. Assumptions

To begin, our definition of the viewplanning problem for object reconstruc-tion and inspection is based on severalassumptions:

—Both the range camera and positioningsystem are calibrated and calibrationparameters are available to the viewplanning module.

—The imaging work space is compatiblewith object size and shape.

2 Pito [1997a] also provided a good description of theview planning problem and defined constraints onthe choice of the next-best-view and desirable prop-erties of view planning algorithms.


70 Scott et al.

—Range camera performance is compati-ble with the model specification for thereconstruction task.

—The sensor and positioning systemhave compatible performance. Specifi-cally, pose error is compatible with thevolume of the range camera frustum.

—Object shape is compatible with sensorcapabilities.

—Object material properties are compati-ble with sensor capabilities.

The last two items merit elaboration.Range cameras are inherently samplingdevices. Therefore, the reconstruction pro-cess is subject to the limitations of a sam-pled representation of continuous surfaceshape. The familiar sampling theorem for1D and 2D signals does not extend to threedimensions. As there is no natural low-pass filter for 3D physical shape, alias-ing effects can be anticipated. Convolutedshapes can easily be conceived which can-not be fully imaged by a range camera,for example, a gourd or a sea urchin-likeshape. Deep cavities, deep holes, or mul-tiple protrusions will be troublesome orcompletely impractical to image. There-fore, practicality requires that object ge-ometry and topology be “reasonable.”

Laser scanners have difficulty with cer-tain materials. They perform best withmaterial having surface scattering proper-ties and a relatively uniform reflectance,neither overly absorbent nor overly re-flective. Natural or humanly made ma-terial with volume scattering propertiessuch as hair and glass produce multi-ple reflections over a surface depth zone,degrading or prohibiting accurate rangemeasurement.

The dynamic range of the current gen-eration of range sensors is limited. Exces-sively absorbent material scatters insuf-ficient light, causing “drop outs,” that is,missing data. Shiny surfaces are charac-terized by specular reflection. Dependingon illumination and observation geome-try, specular surfaces may result in dropouts due to insufficient scattered energyor outliers or drop outs (depending on cam-era design) due to receiver saturation from

Table I. View Planning Requirements

Category Requirement

Model quality specification

Generalizable algorithm

Generalized viewpoints

General View overlap

Robust

Efficient

Self-terminating

Minimal a priori knowledge

Object Shape constraints

Material constraints

Frustum

Sensor Shadow effect

Measurement performance

6D posePositioning

Pose constraintsSystemPositioning performance

excessive received energy. Multiple reflec-tions on a shiny surface in corner regionscan also produce wild measurements. Fi-nally, abrupt reflectance changes will pro-duce edge effects in the form of measure-ment biases and artifacts [Curless andLevoy 1996].

Consequently, the current state-of-the-art is limited to view planning for ob-jects with “reasonable” shapes and benignreflectance characteristics. Some viewplanning techniques are applicable to re-stricted shape classes, such as manufac-tured objects constructed of a limited setof shape primitives. The issue of extend-ing view planning to objects with moredifficult reflectance characteristics is ad-dressed in Section 10.1.

3.2. Constraints and Requirements

The following constraints and require-ments apply to the view planning taskfor object reconstruction. They are sum-marized in Table I under the categories ofgeneral, object, sensor, or positioning sys-tem. The following amounts to a “researchspecification” for a view planning system.Where objectives are quantified, it is pre-sumed that research-quality algorithms



are subject to refinement, development,and system integration on suitable pro-duction hardware and software platforms.

3.2.1. Model Quality Specification. By def-inition, performance-oriented view plan-ning is based on a model specification con-taining explicit, quantified, model qualityrequirements such as measurement pre-cision and sampling density. For object re-construction, there is usually an implicitrequirement for 100% surface coverage. Inother applications such as inspection, itmay be appropriate to limit coverage tospecified object regions.

3.2.2. Generalizable Algorithm. For wideapplicability, the technique should applyto a broad class of range sensors, position-ing systems, and objects.

3.2.3. Generalized Viewpoints. In additionto sensor pose, the view planning algo-rithm should plan reconfigurable sensorparameters such as laser power and scanlength through the use of generalizedviewpoints.

3.2.4. View Overlap. The algorithmshould provide the degree of imageoverlap necessary for integration andregistration. Integration requires imageoverlap along the boundaries of adjacentimages in the view plan. If pose erroris such that image-based registrationis required, the image set should havesufficient shape complexity in overlappingregions for registration of the image setwithin the specified precision.

3.2.5. Robust. The view planning algo-rithm should be immune to catastrophicfailure, handle sensor and positioning sys-tem noise and artifacts, and require mini-mal operator intervention.

3.2.6. Efficient. The view planning al-gorithm should be sufficiently efficientto be competitive with skilled humanoperators. As an initial goal, the algo-rithm should be capable, with production-quality hardware and software, of produc-

ing a specification-compliant view plan fora moderately complex object within 1 h.

3.2.7. Self-Terminating. The view plan-ning algorithm should recognize when thegoal has been achieved or when progresstoward it has stalled.

3.2.8. Limited a priori Knowledge. Theview planning algorithm should be effec-tive with minimal a priori object knowl-edge, specifically no more than approxi-mate bounding dimensions and centroid.

3.2.9. Shape Constraints. The view plan-ning algorithm should be effective with allreasonable object geometry and topologythat is compatible with sensor measure-ment capabilities.

3.2.10. Material Constraints. The algo-rithm should be effective with all objectmaterial properties compatible withsensor measurement capabilities.

3.2.11. Frustum. Sensor frustum shapeshould be modeled. This includes sensordepth of field, field of view, and scan lengthor scan arc.

3.2.12. Shadow Effect. The bistatic na-ture of the sensor should be modeled—thatis, the physical separation of the laser anddetector.

3.2.13. Measurement Performance. Sen-sor measurement performance shouldbe modeled, including variation of mea-surement precision and sampling densitywithin the frustum and surface inclina-tion effects. The following range cameraartifacts should be modeled: geometricand reflectance step edges and multiplereflections.

3.2.14. 6D Pose. The view planning al-gorithm should handle a positioning sys-tem with an unconstrained pose space-three position and three rotation.

3.2.15. Pose Constraints. The view plan-ning algorithm should model constraints


72 Scott et al.

on the degrees of freedom and range of mo-tion of the positioning system.

3.2.16. Positioning System Performance.Positioning system performance shouldbe modeled to include pose error andrepositioning/reconfiguration time.

3.3. Performance Measures

To date, adequate standards for quantify-ing view planning performance have notexisted. This complicates the comparativeassessment of view planning algorithms.We propose the following measures forevaluating view planning algorithm per-formance for a given reconstruction taskwith a given imaging environment:

—View plan quality The quality of a viewplan is determined by the quality ofthe reconstruction it generates. Thiscan be expressed as the overall verifiedmeasurability3 mv of the reconstructionwith respect to the model specification.mv is an indicator of the fidelity and ro-bustness of the view planning processplus the adequacy of the discretizationschemes for viewpoint space and surfacespace. The goal is mv = 1.0.

—View plan efficiency A measure of viewplan efficiency is the length of the gen-erated view plan relative to the opti-mum achievable for that task, that is,ev = nOpt/n, where n = |N |. As it may beimpractical to determine with certaintythe optimum view plan length nOpt forcomplex reconstruction tasks, a suitablesurrogate is the length of the best solu-tion nBest found thus far among all viewplanning techniques examined for thesame task, that is, nOpt ≈ nBest . ev isan indicator of the efficiency and com-pleteness of discretization schemes forviewpoint and surface space and the ef-ficiency of the set covering process. Thegoal is ev = 1.0.

3 Verified measurability of a reconstruction can beseen as the ratio of reconstructed surface area com-pliant with the specified model criteria to the overallobject surface area. For more precise definitions, re-fer to Scott et al. [2001b, 2002].

—View plan computational efficiency Mea-sures of the computational cost of gen-erating the view plan are computa-tional complexity and execution timeon a defined platform. While each hasweaknesses, together they provide areasonable appreciation of algorithmperformance. Computational efficiencyindicates the efficiency of discretiza-tion schemes for viewpoint and surfacespace, visibility analysis, and the setcovering process. Computational effi-ciency goals are application-dependent.

4. RELATED SURVEYS

Surveys relevant to view planning for ob-ject reconstruction can be found primarilyin doctorate and masters theses publishedover the last decade (Table II).

In a brief and clearly written tech-nical report addressing only seven pa-pers (Banta et al. [1995], Connolly [1985],Hutchinson and Kak [1989], Maver andBajcsy [1993], Pito [1996b], Tarabaniset al. [1995a], and Whaite and Ferrie[1990]), Hall-Holt [1998] quickly zeroedin on two key issues—representation ofavailable geometric information and howto deal with the potentially vast searchspace. While he did not answer the ques-tions he posed, Hall-Holt succinctly exam-ined from several perspectives the viewplanning approaches taken by some ofthe most notable authors. The survey con-cluded with a discussion of open issues:algorithm efficiency in the presence oflarge amounts of data, appropriate geo-metrical representations, hybrid models,multiresolution techniques, ray tracingavoidance, quantitative performance as-sessment, performance benchmarks, andimproved scanner models.

The classic survey by Tarabaniset al. [1995a], considered three visiontasks—inspection, recognition, andreconstruction—but focused almost exclu-sively on the first of these. The underlyingperspective was that of conventionalintensity imaging, although a few refer-ences were made to range imaging. Viewplanning was categorized as following oneof three paradigms—synthesis, generate



Table II. View Planning Surveys

Year Author(s)

2002 William Scott [2002] Ph.D. thesis

1999 Flavio Prieto [1999] Ph.D. thesis

1998 Michael Reed [1998] Ph.D. thesis

1998 Olaf Hall-Holt [1998] Technical Report

1997 Steven Abrams [1997] Ph.D. thesis

1997 Brian Curless [1997] Ph.D. thesis

1997 Nikos Massios [1997] Masters thesis

1997 Dimitri Papadopoulos-Orfanos [1997] Ph.D. thesis

1997 Richard Pito [1997a] Ph.D. thesis

1997 Roberts and Marshall [1997] technical report

1997 Yiming Ye [1997] Ph.D. thesis

1996 Joseph Banta [1996] Masters thesis

1996 Leland Best [1996] Ph.D. thesis

1996 Vitor Sequeira [1996] Ph.D. thesis

1996 Mark Wheeler [1996] Ph.D. thesis

1995 Jasna Maver [1995] Ph.D. thesis

1995 Tarabanis et al. [1995] survey paper

1994 Stephane Aubry [1994] Ph.D. thesis

1994 Kiriakos Kutulakos [1994] Ph.D. thesis

1994 Chris Pudney [1994] Ph.D. thesis

1994 Lambert Wixson [1994] Ph.D. thesis

1993 Sergio Sedas-Gersey [1993] Ph.D. thesis

1993 Glen Tarbox [1993] Ph.D. thesis

1993 Xiaobu Yuan [1993] Ph.D. thesis

1992 Konstantinos Tarabanis [1992] Ph.D. thesis

1992 Besma Roui-Abidi [1992] Ph.D. thesis

1990 Michael Buzinski [1990] Masters thesis

1990 Seungku Yi [1990] Ph.D. thesis

and test, or expert system. The openproblems in view planning circa 1995 asseen by Tarabanis et al. [1995a] were thefollowing:

—modeling and incorporating other con-straints such as integrating collisionavoidance with view planning, dynamicsensor planning, and modeling the oper-ating range of the employed sensor;

—modeling and incorporating other sen-sors such as tactile, range, force-torque,and acoustic sensors;

—relaxing some of the assumptions madein current approaches such as featureuncertainty and error characterization;

—illumination planning to include higher-order lighting and reflectance models,multiple sources, specularity, and inter-reflections; and

—sensor and illumination modeling to in-clude mapping model parameters to con-trollable sensor settings and accuratesensor noise models.

A comprehensive survey of automatedvisual inspection systems by Newman andJain [1995] considered binary, intensity,colored and range image sensors for a widerange of inspection applications.

These surveys categorized view plan-ning methods in several different ways.


74 Scott et al.

Fig. 4 . Model-based view planning algorithms.

From our perspective, we begin with twotop level categories—model-based or non-model-based. It is convenient to thensub-categorize the two classes somewhatdifferently.

5. MODEL-BASED VIEW PLANNING

Model-based view planning methods canbe categorized by the representation usedfor the knowledge embedded in the objectmodel (Figure 4). “Model-based” methodsbase planning on an a priori object modelat some level of fidelity.

5.1. Set Theory Methods

Visibility matrices. At the heart of theset theoretic approach is a visibility ma-trix, whose elements encode in a singledata structure the visibility of discrete ob-ject surface points from each pose in quan-tized viewpoint space. A more sophisti-cated enhancement replaces visibility bymeasurability, in which the data elementsencode estimated measurement quality.

Tarbox and Gottschlich. Tarbox andGottschlich [1995] incorporated elementsof set theory, graph theory, computationalgeometry, mathematical morphology, andstatistical nonlinear optimization in theirthorough examination of view planning forautomated inspection. They discretizedviewpoint space by recursive subdivisionof an icosahedron and constrained solu-tions to lie on a viewing sphere completelycontaining the object with the viewing di-rection oriented toward the center of thesphere. They further assumed that theobject lies completely within the sensor’sfrustum. An octree-encoded voxel occu-

pancy model represented the sensing vol-ume. Their view planning approach wastailored to a specific long-baseline, activetriangulation-based range sensor so thatthe view planning problem becomes oneof examining the set of all ordered pairsof points on the view sphere separated bythe specified sensor baseline. They brieflydiscussed culling inadmissible points fromthe viewpoint set by ray tracing opera-tions, such as those occluded by a mount-ing fixture and supporting plane. The prin-cipal consideration in viewpoint selectionis taken to be the surface area that a givenviewpoint is capable of sensing.

Consideration of grazing angle effectsfigured prominently in their approach.Different grazing angle thresholds wereset for the light source and the cam-era. The only grazing angle effects con-sidered were the impact on dilution ofsurface sampling density and viewabilityat acute angles in the presence of sur-face microstructure. This masked the mostimportant impact of grazing angle whichis on measurement error. Other aspectsof sensor noise and artifacts were notconsidered.

The authors noted that it would be desir-able to find the shortest possible view plan.However, as this problem is known to beNP-complete, the authors concluded thatit would be necessary to employ a viewplanning algorithm that can find a satis-factory but not necessarily optimal viewsequence.

Tarbox and Gottschlich [1995] devel-oped and examined the performance ofthree algorithms. The first two employedan irrevocable selection strategy and dif-fered by the manner in which grazing



angle constraints were treated. The thirdalgorithm was novel in that it employed arevocable selection strategy for viewpointset minimization based on a randomizedsearch with simulated annealing.

To account for slight pose errors, theyremoved viewpoints which were not ro-bust to pose variation. This was accom-plished by morphological operations onthe viewpoint set associated with each sur-face point.

Their approach was based on a mea-surability matrix M(i, j ) computed by acomplete visibility analysis for the lasersource and camera over the set of all sur-face points and all admissible viewpoints.Given the span of the discretized vari-ables, the computational complexity of theapproach was prohibitive. This was a fun-damental limitation regarding direct ap-plicability of the work to object reconstruc-tion. Nevertheless, the authors’ thoroughand well-written treatment of the subjectprovided useful insights into several as-pects of the view planning problem.

5.2. Graph Theory Methods

Aspect graphs. An aspect graph of an ob-ject has a node representing every aspectof that object and arcs connecting all ad-jacent aspects on an object [Tarbox andGottschlich 1995; Bowyer and Dyer 1990].An aspect is loosely defined as the set ofviewpoints of the object such that the un-occluded portion of the object seen from allthose viewpoints is qualitatively the same.In other words, viewpoint space is parti-tioned into regions providing equivalentviews. Each node represents one such re-gion while arcs represent their adjacencyin viewpoint space.

While aspect graphs are an intrigu-ing theoretical possibility, there are prac-tical difficulties. When dealing with ob-jects as simple polyhedra and consideringa viewpoint purely in terms of the visi-bility of object features from that point,the criterion “qualitatively the same” sim-ply means that the same vertices, edges,and faces are visible from that viewpoint.When the object is changed from discreteto continuous, the quantization of view-

point space inherent to an aspect graphrepresentation strictly fails. For some ap-plications, one may redefine an aspectgraph in terms of the visible topology ofthe object’s silhouette, but this is not use-ful for object reconstruction. Additionally,as we are dealing with a sensing problemwhich involves not only visibility but men-suration, viewpoints with the same visibil-ity (i.e., the same aspect) are by no meansequivalent for sensing purposes. Second,aspect graphs for even moderately com-plex objects quickly become huge and un-wieldy, as does the computational com-plexity of handling them. Consequently,aspect graph theoretical development isnot sufficiently mature [Faugeras et al.1992] to be a suitable representation basisfor performance-oriented view planning.

5.3. Computational Geometry Methods

Art gallery problem. A classic 2D com-putational geometry problem concerns thevisibility of edges in a polygon, the so-called “art gallery problem” [Urrutia 2000;Xie et al. 1986]. Given the floor plan ofan art gallery as a polygon P, the prob-lem is to find an upper bound on the num-ber of “guards” represented by points suchthat the interior walls of P are completelyvisible [Kahn et al. 1980]. The task isto determine the minimum number andplacement of guards who collectively cansee all walls in the gallery. The objectreconstruction problem is somewhat re-lated to this classic computational geome-try problem. However, there are additionalcomplexities—three dimensions, bistaticvisibility (source and receiver), plus men-suration in lieu of visibility.

6. NON-MODEL-BASED VIEW PLANNING

It is convenient to classify existing viewplanning methods (most of which are non-model-based) by the domain of reason-ing about viewpoints—that is, volumetric,surface-based, or global (Figure 5). Somemethods combine several techniques. Themajority fall into two subcategories—voxel occupancy or occlusion edges.


76 Scott et al.

Fig. 5 . Traditional non-model-based view planning methods.

6.1. Surface-Based Methods

Surface-based algorithms reason aboutknowledge of object surface space S.

6.1.1. Occlusion Edge Methods

Occlusion edges. The most commonlyused traditional method exploits geomet-ric jump edges. Illustrated in Figure 6,this approach is based on the premisethat occlusion edges internal to the im-age indicate surface areas not yet sam-pled, while boundary jump edges rep-resent the boundary of the unobservedvolume. In both cases, occlusion edgesprovide cues for the next-best-viewingdirection.

Maver and Bajcsy. Some of the earliestpapers using the occlusion edge methodwere by Maver and Bajcsy [1990, 1993].Their approach was tailored to a longbaseline laser profile scanner with a po-sitioning system limited to a single rota-tional degree of freedom. The two-stagemethod separately considered source andreceiver occlusions. The focus was onfinding and labeling boundary occlusionedges. View planning was done with ref-erence to a plane defined by the supportfixture. The initial stage planned viewsas a rotation about the range cameraboresight. The projection of occluded re-gions on the support plane was approx-imated by polygons. Occlusion-free arcs

were computed by a visibility analysismaking assumptions about the height ofeach “shadow zone pixel.” A histogramwas formed by summing the viewing arcsfor each pixel and a next-best-view deter-mined from histogram maxima. The algo-rithm was computationally intensive dueto the many-on-many visibility calcula-tions. Performance was sensitive to the un-derlying assumptions.

In a second view planning stage, de-scribed analytically but not implemented,occlusion edges of source shadow zoneswere used in a similar manner to deter-mine a new orientation for the camera ill-umination plane. The authors restrictedthe search for new scanning planes tothe set perpendicular to the originalscanning set up. Simulated results wereshown.

The approach was limited by its formu-lation as a 21/2 D scene view rather thana true 3D perspective. The scene was de-fined as a horizontal x- y array of pixelswith a height value, either measured or as-sumed. Coupled with sensor and position-ing system limitations, the method was in-capable of acquiring an all-aspect model.Garcia et al. [1998a, 1998b] also utilizedan occlusion edge method.

6.1.2. Contour Following. Once having lo-cated a portion of the object, the con-tour following technique involves “paint-ing” the object with the sensor by keeping



it in close proximity to the surface at alltimes. The technique has been applied toa class of range sensors with a limitedsensing volume. Collision avoidance is aprimary concern. Pudney [1994] describedthe application of contour following to arobot-mounted range sensor.

Conventional terminology labels rangesensors as range “cameras,” implying anoutput in the form of an “image.” Infact, most range cameras are profile scan-ners which only acquire data in an imageformat through linear or rotational sen-sor motion by a fixed or programmableamount. Contour following usually in-volves acquiring long strips of profiles ofan arbitrary length rather than a rangeimage in the conventional sense.4 Becausethe positioning systems used typically areaccurate in pure translation but have lim-ited mobility or may require recalibrationin rotation, scanning is frequently donealong nonlinear curves with a fixed ori-entation. The technique is widely usedcommercially with mechanical CMM de-vices. Contour following works best withobjects large in size relative to sensorcoverage and for relatively simple shapeswith smoothly flowing lines. It is moreproblematic with smaller objects and morecomplex shapes. Other contour followingwork with range sensors includes Soucyet al. [1998], Lamb et al. [1999], andMilroy et al. [1996]. The latter used a com-bined region growing and contour follow-ing strategy.

6.1.3. Parametric Surface Representations.Superquadric models have been widelyused in machine vision as flexible, com-pact representations. However, they arelimited to simple scenes unless the scene issegmented and piecewise fitted with sep-arate models. Additionally, superquadricsare highly nonlinear.

Whaite and Ferrie. Related more torobotic exploration than object recon-struction, the autonomous exploration ap-proach of Whaite and Ferrie [1990, 1991,1992, 1997] nevertheless provides an in-

4 See also the discussion in Section 9.2.1.

teresting high-level perspective on charac-terization and exploitation of uncertaintyin view planning. Their gaze-planningstrategy used model uncertainty as a basisfor selecting viewpoints. After the data issegmented, the task is to find superellip-soid parameters best describing each seg-mented part. They observed that the bestsensor locations are those where the abil-ity to predict is worst—that is, where vari-ance in the fit of the data to the currentmodel is greatest.

Because of concerns about the validity ofthe linearized theory, they took a conser-vative approach in which the sensor wasalways moved in small steps. Using a nu-merical analysis technique, they searchedfor and moved the sensor in the direction ofmaximum uncertainty. Movements wereconstrained to a view sphere at a fixedgeodesic distance from the current scan-ner location. Experiments showed the al-gorithm to be attracted to regions of highobject curvature.

The authors used a multilayered sys-tem design. A general-purpose, sensor-in-dependent, and environment-independentlower-layer navigator module handledview planning. A higher-level explorermodule contained application-specificknowledge and handled executive-levelfunctions such as delegating tasks, mon-itoring progress, making decisions, andresolving conflicts. While the layereddesign is attractive, it is difficult to seehow detailed sensing and environmentalfactors can be separated from the act ofsensor view planning.

The method directly incorporates mea-surements of sensor noise, although it isunclear exactly which noise sources aremodeled or what the fidelity of the corre-sponding models is. View overlap is a con-sequence of the conservative nature of thesearch strategy rather than application ofan explicit constraint.

While innovative and providing usefulinsights into view planning at a high-levelof abstraction, the technique was designedmore for approximate volumetric model-ing of simple scenes for robotic manip-ulation than for high-definition surfacemodeling.


78 Scott et al.

6.2. Volumetric Methods

Volumetric methods select viewpoints byreasoning about the state of knowledge ofimaging work space I . Each scan labels aportion of I . The NBV is the viewpoint of-fering the greatest prospective reductionin uncertainty about I . Volumetric meth-ods focus particularly on the solid shadowscast by the scanned object.

6.2.1. Voxel Occupancy Methods. Com-mon volumetric methods involve encodingspace occupancy by a voxel occupancygrid or an octree.

Voxel occupancy grids. Voxelization is awidely used, compact means of encod-ing spatial occupancy. Voxel grids canalso be used for a coarse surface repre-sentation, although they are clearly notsuited for high-precision modeling. Theirprincipal disadvantage is the large mem-ory requirement for even moderate vol-umetric quantization levels. In a typicalimaging environment, the object occupiesonly a small portion of the imaging workspace and its surface intersects an evensmaller portion of the voxelized space.Most authors have ignored the impact ofmisalignment of spatial quantization in-tervals [Greespan 2002] between views.The phenomenon is similar to timing jit-ter in conventional time domain signalprocessing.

Banta et al. Banta and Abidi [1996] andBanta et al. [1995] defined the NBV as“the next camera pose which will extractthe greatest amount of unknown scene in-formation.” Their objective is to minimizethe number of views required. The workwas similar to that of Connolly [1985] al-though they constrained the NBV searchto a smaller search space. Both the imag-ing work space and object surface are rep-resented by voxel occupancy grids in whichvoxels are labeled occupied or unoccupied.Rather than allowing an “unknown” state,this binary approach labels all points notcurrently visible as occupied and mergesviews by a voxel-wise logical “AND” oper-ation. The papers explored several cuingmechanisms for suggesting feasible views,

including orienting the sensor in the di-rection of the viewpoint with the greatestnumber of potentially visible voxels basedon local surface exposure,5 the viewpointrevealing the greatest number of hiddenvoxels based on a ray tracing visibilityanalysis, the mean of the three largestjump edges in the most recently acquiredimage, or the centroid of the cluster con-taining the largest number of unknownvoxel faces.

The notion of applying several viewplanning concepts in combination underintelligent control is a useful contribution.The authors also proposed tests to ensureselection of a “good” NBV such as validat-ing by ray tracing that the feature of inter-est is actually visible and enforcing a min-imum angular separation between viewson the view sphere. Additionally, they ex-amined several termination criteria, thatis, terminating when the size of eitherthe surface model or the occluded modelceases to change by a significant amountor when the ratio of the size of the sur-face model to that of the occluded modelis “large.” However, the proposed termi-nation criteria do not relate to objectiveperformance requirements for the targetmodel.

The various algorithms attempted wereeffective in what could be called ex-ploratory, approximate view planning fortopologically complex objects with purelysynthetic data but were less effective inacquiring smaller geometric detail. Whileinnovative, the work was subject to anumber of simplifications and limitations.The range camera was treated as anerror-free monostatic sensor with uniformperformance within the imaging workspace. Pose error was ignored. Viewpointswere restricted to a fixed-radius viewsphere.

Massios and Fisher. A paper [Massiosand Fisher 1998] summarizing Massios’[1997] Master’s thesis built on previousvoxel occupancy methods by using theweighted sum of visibility and “quality”

5 This is similar to Connolly’s “normal” algorithmwhich fails in concave regions.



factors as an NBV objective function:

ftotal(Ev) = wv fvisiblity(Ev)+wq fquality(Ev). (3)

The visibility cue was taken from anocclusion edge analysis finding occlusionplane voxels which are defined as unseenvoxels with an empty neighbour. Visibil-ity factor fvisiblity is set to the number ofocclusion plane voxels visible from view-ing direction Ev, as determined by ray trac-ing. The absolute value of the dot prod-uct of the estimated local surface normaland viewing direction vector is taken asa local quality measure for occupied vox-els. A region quality estimate is also for-mulated for each voxel due to unspecifiedsystem inaccuracies. Quality factor fqualityis formulated to maximize the numberof low-quality voxels visible from a givenviewpoint.

Viewpoint space is taken to be aconstant radius tesselated sphere, ob-tained by recursive subdivision of anicosahedron, fitted over the volumetricrepresentation for the object. Using anexperimental configuration whose posi-tioning system was limited to a single de-gree of freedom, the authors showed sep-arate and combined plots of the objectivefunction as a function of angle and the cu-mulative number of views taken, a presen-tation format usefully illustrating the im-pact of the quality measure.

While the introduction of a quality termin the objective function was a valuablecontribution, the chosen measure was sub-jective and did not relate to objective mea-surement constraints on the reconstructedmodel. Further, the model was determin-istic and considered only one error mecha-nism among several. Viewpoint space wasconstrained to two dimensions.

6.2.2. Octree Methods. Octree methodsencode voxel occupancy more efficiently.

Connolly. One of the earliest papers onview planning was by Connolly [1985].He appears to have first coined theterm “next-best-view” (NBV). Connollypresented two algorithms, planetariumand normal, differing in the cue used to

suggest feasible views and the selectionmechanism used to find the next-best-viewfrom the set of feasible candidates. Theimaging work space is voxelized and la-beled as empty, occupied, or unseen. Thisinformation is encoded in an octree whichis also used to represent the object surface.All viewing vectors are assumed to pointto the origin. Viewpoints are constrainedto evenly spaced points on a sphere aroundthe object.

The planetarium algorithm applies avisibility analysis of the octree for eachcandidate viewpoint on the sphere. Thearea of unseen voxels projected onto theimage plane for each candidate viewpointis taken as a measure of the solid angleof unseen space that will be swept by thatview. The viewpoint with the largest un-seen area is selected as the NBV. The algo-rithm suffers from time complexity prob-lems inherent with a complete visibilityanalysis for all candidate viewpoints. Thenormal algorithm simplifies the visibilityanalysis to the local rather than globallevel by examining faces in the octree com-mon to both unseen and empty voxels. It isfaster but does not deal as well with self-occluding scenes.

The method is subject to many sim-plifications and limitations. Neither sen-sor nor positioning system performance ischaracterized. The sensor is treated as anideal point source without constraints onfield-of-view or depth-of-field. Shadow ef-fects are ignored. Notwithstanding theselimitations, Connolly’s pioneering conceptof exploring and labeling imaging workspace is found in many later papers.

6.2.3. Space Carving. The space carvingtechnique6 applies primarily to a smallclass of range sensors with a limited sens-ing volume, particularly those designedas noncontact replacements for CMM me-chanical touch probes. The sensor is sweptthrough the imaging work space in apreplanned methodical manner, divertingaround obstacles, with the objective of

6 As used here, “space carving” is distinct fromthe shape from silhouette technique [Kutulakos andSeitz 2000].


80 Scott et al.

reliably labeling work space occupancy. Aswith contour following, collision avoidanceis a primary concern.

Papadopoulos-Orfanos. A paper by Papa-dopoulos-Orfanos and Schmitt [1997] su-mmarizing Papadopoulos-Orfanos’ [1997]Ph.D. research applied space carvingto a specific shallow depth-of-field sen-sor for automated object reconstruction.Emphasis was placed on updating thevoxel-based scene representation—in par-ticular, robust labeling of empty spaceto avoid collisions. Voxel labeling wasachieved by a complex analysis of occlu-sion zones followed by ray tracing op-erations. Most of the work was concen-trated on collision avoidance and pathplanning.

Papadopoulos-Orfanos described butdid not fully implement a two-stage 3Dimaging strategy of scene exploration fol-lowed by surface data acquisition. The ex-ploration stage, which was implemented,employed an exhaustive search of un-known portions of the work space. In thevicinity of the object, the goal was to getthe sensor as close as possible to the sur-face while avoiding collisions. During thisphase, sensor orientation was fixed andonly translations were employed to searchthe work space layer by layer in a zigzagpattern. As a consequence of the fixed ori-entation, occluded surfaces were not im-aged and surface occupancy in these re-gions was poorly defined. In a second stageof surface data acquisition (described butnot implemented), sensor orientation viewplanning was proposed. Some existingNBV techniques were briefly discussed ascandidate methods.

In its current form, the work does notaddress view planning per se. The explo-ration stage involves an exhaustive searchof the imaging work space following a pre-planned trajectory modified by collisionavoidance. No new view planning tech-nique is proposed. If more fully devel-oped, space carving has potential for high-precision scanning. However, it will likelyremain slow as a consequence of the smallsensor frustum and the exhaustive searchtechnique.

Lamb et al. [1999] also utilized spacecarving at a coarse level of resolution inthe first phase of a multistage approach tosemiautomated model acquisition.

6.2.4. Solid Geometry Methods. Thismethod utilizes standard solid geometryalgorithms available with most CADpackages to model the current state ofobject knowledge. The method can berobust with respect to complex topology.A generic problem with the techniquearises from solid geometry intersectionoperations which, by definition, subtractand cannot add volume. Therefore, ifa range image used to extrude a solidvolume does not completely cover theobject, the resulting volume will excludea portion of the object which can never berecovered by subsequent intersection op-erations. Consequently, difficulties arisewhen a view fails to completely enclosethe object or along occlusion boundarieswhere data is often missing or erroneousdue to inclination and edge effects.

Bistatic shadow effects. Although ig-nored by many authors, a triangulation-based laser scanner is a bistatic sen-sor whose optical baseline is generallysignificant with respect to the measure-ment stand-off distance. Therefore, theshadow effect is nonnegligible and has twocomponents—source and receiver shad-ows. Bistatic shadow effects are illus-trated in Figure 6. We can observe thatan occlusion edge as seen by the receivermay lie inside the object. This poses diffi-culties for occlusion-edge-based view plan-ning techniques employing solid geometryintersection operations. Collision avoid-ance routines operating in conjunctionwith view planning also need to make al-lowance for this fact. Further, in general,the object will not lie entirely within thecamera frustum, so a given range imagemay or may not contain boundary occlu-sion edges. Some NBV algorithms can failin the absence of occlusion edges.

Reed. In a series of articles by Reed andAllen [1997, 1999] and Reed et al. [1997a,1997b] arising from Reed’s [1998] Ph.D.



Fig. 6 . Occluding edges in geometric images with bistaticsensor.

thesis, a solid geometry approach was usedto synthesize a continuous 3D viewing vol-ume for a selected occlusion edge, ratherthan discretizing viewpoint space. A meshsurface was created for each range imageand each mesh triangle was swept to forma solid model. Solid geometry union oper-ations on the swept volume resulted in acomposite model comprised of the surfaceand occluded volume, as known at thatstage of modeling. Surfaces on the compos-ite model were labeled imaged surface oroccluded surface.

Reed’s view planning approach thenproceeded with manual selection of a spe-cific occluded edge as a target. A “visibil-ity volume” was calculated for the target—that is, a 3D volume specifying the setof all sensor positions having an unoc-cluded view of the entire target for themodel as presently defined. This was com-puted by subtracting the occlusion volumefor each model surface component fromthe target’s unoccluded volume, a half-space whose defining plane was coinci-dent with the target’s face and oriented inthe appropriate direction. The work wasbased on previous sensor planning workin the same lab by Tarabanis et al. [1996].

The volumetric approach allowed sensorand positioning system constraints to beadded.

The 3D imaging workspace searchwas an improvement over methods con-strained to an arbitrary 2D surface aroundthe object. The authors claimed their ap-proach had the advantage of defining moreaccurate viewpoints due to the treatmentof viewpoint space as continuous ratherthan discrete, although no analytical orexperimental data were provided to quan-tify this assertion.

Part of the reported research [Reed et al.1997a] relied on manual selection of a suit-able target and hence the NBV. There wasno discussion of the sensitivity of the al-gorithm to target occlusion edge selection.A second paper [Reed et al. 1997b] dis-cussed an approach to automated view-point selection. In this case, the algorithmselected the NBV as the one imaging themost occluded surface elements in the cur-rent composite model. The NBV was de-termined by computing the visibility vol-ume for each occluded surface element,intersecting each visibility volume withthe sensor’s reachable space and search-ing the intersection for the point imaging


82 Scott et al.

the most surface area. A third paper [Reedand Allen 1999] briefly examined NBVplanning based on the use of multipletargets.

Benefits of solid geometry methodsinclude robustness to complex objecttopology and the guarantee of water-tight models. Viewpoint synthesis can becomputationally advantageous with re-spect to methods which discretize view-point space. However, as presently de-fined, the approach has limitations. Thesensor frustum is only partially modeled—field-of-view constraints are not consid-ered. Bistatic shadow effects on view plan-ning are not addressed directly, althoughmissing data in small shadow zones isdealt with by interpolation and along oc-clusion boundaries by surface extensionalong the direction of the sensor baseline.Sensor errors are only briefly considered.View overlap planning is not addressed.Set operations on extruded surface ele-ments introduce artifacts along integra-tion boundaries.

6.3. Global View Planning Methods

A few methods derive a view planning cuefrom global rather than local characteris-tics of the geometric data.

6.3.1. Mass Vector Chain. Yuan [1993,1995] described an interesting view plan-ning mechanism. He observed that “areconstruction system expects a self-controlled modeling mechanism for thesystem to check the spatial closure of ob-ject models and to estimate the direction ofunprocessed features” [Yuan 1995, p. 307].

Illustrated in Figure 7, his approachsegments the observed surface into a setof small patches and describes the objectwith a mass vector chain (MVC). By def-inition, an MVC is a series of weightedvectors. A mass vector EVi is assigned toeach surface patch Si of the object. Thus,EVi = Eni Ri, where Eni is the average vis-

ible direction and Ri is the surface sizewhen viewed from that direction. It is eas-ily shown that the boundary surfaces ofan object compose a closed surface bound-ary only when their mass vectors form

Fig. 7 . Mass vector chain.

a closed chain. Thus, the next-best-viewcan be set to the negative of the cumu-lative MVC, which should define a view-ing direction whose mass vector will closethe chain or at least shorten the gap.Special rules are required for holes andcavities.

The algorithm was shown to work withsynthetic geometric data for a simpleobject. However, the idealized imagingenvironment does not address the com-plications of modeling automation withreal sensors and complex humanly madeor natural objects. The method is capa-ble only of estimating viewing direction,not position. It has no reliable measureof scale, range, or imaging volume. Themethod is able to deal with moderatelycomplex topology and may have poten-tial for machine vision tasks involving hu-manly made parts with simple shape or forthe initial global view planning phase of atwo-step coarse-fine planning process.

6.3.2. Intermediate Space Representations.The essence of view planning is capturing,representing, and optimizing measures ofvisibility of the object surface from sensorposes in viewpoint space. This mappingbetween the object surface and points in



the workspace of the sensor and position-ing system involves a large amount of in-formation and therefore a high degree ofcomputational complexity for its acquisi-tion, storage, and manipulation. It is de-sirable to find a more compact and easilysearched representation of the visibilityinformation with a minimum loss of in-formation. One approach following thisline of reasoning involves encoding visibil-ity information on a virtual surface posi-tioned between the object and the sensorworkspace—that is, an intermediate spacerepresentation.

Positional space—Pito. Pito’s [1997a]thesis is significant for its contributionsto all four aspects of geometric modelingautomation (scanning, registration, in-tegration, and view planning). See alsothe summary paper [Pito 1999] as well asPito [1996a, 1996b, 1997b] and Pito andBajcsy [1995].

Pito’s work is unusual in that he tookpains to characterize performance of therange scanner used for experiments andthen incorporated sensor models in the au-tomation algorithms. In addition to treat-ing the grazing angle effect (fairly com-mon in the literature), he also treatededge effect anomalies during the integra-tion phase. The view overlap requirementwas incorporated. The shadow effect wasexplicitly treated. However, nonstation-ary measurement error within the mea-surement volume was not addressed. Theneed for generalized viewpoints was dis-cussed but apparently not implemented.The technique is amenable to using gen-eralized viewpoints, at the expense of in-creasing the dimensionality of intermedi-ate space.

Like many others before him, Pito choseocclusion edges as a cuing mechanism forthe NBV search. He observed that thevoid volume can be economically repre-sented by defining only the void surfacenear edges of the current model, which herepresented by small rectangular patchesattached to occluding edges of the seensurface. He argued that it is unnecessaryto represent the complete boundary of theumbra cast by the object. In general, the

portion of the umbra nearest the occlusionboundary will be nearest to the real objectsurface and therefore be the best region tosearch next. Furthermore, such regions fitthe overlap constraint.

Illustrated in Figure 8, Pito’s NBV al-gorithm used an intermediate space rep-resentation, “positional space” (PS), asa repository for two types of visibilityinformation—object surface visibility andsensor scanning potential. A virtual posi-tional space surface (PSS), with a shapeappropriate to the sensor-positioning sys-tem combination, was placed betweenthe object and the sensor workspace.Object surface space (represented as amesh), PSS, and viewpoint space were dis-cretized.

The visibility of each triangular meshelement on the object surface can be en-coded in positional space by tracing an“observation ray” from the mesh elementto a PSS cell. The direction of an unoc-cluded observation ray relative to a lo-cal frame of reference on the PSS can bespecified by two angles, termed the posi-tional space direction (PSD). Pito chose toencode surface visibility as an analoguevalue equal to the area of the surfaceelement visible by a given ray weightedby a confidence measure tied to the mea-surement grazing angle. Thus, PS was ascalar field in four dimensions P (u, v, θ , φ)where u, v were coordinates in PSS andθ , φ were the components of PSD. Encod-ing the image of both the seen surface andvoid patches in PS provided a means to ap-ply an overlap constraint between viewsto meet registration and integration re-quirements. The range camera’s scanningpotential at a given viewpoint can be simi-larly encoded in positional space by deter-mining the intersection of “ranging rays”from the optical transmitter and receiverwith the PSS. A separate image was cal-culated for each viewpoint. Using PS asa placeholder for ranging and observationrays facilitated determining which of themwere collinear, as well as aiding the appli-cation of NBV constraints.

Without detracting from the compre-hensive treatment and contribution of thework, there are some unresolved issues


84 Scott et al.

Fig. 8 . Positional space.

and weaknesses with the approach:

—Overall computational complexity is-sues are not addressed; nor are trade-offs between discretization levels in ob-ject surface space S, viewpoint spaceV , or positional space PSS. In theend, visibility analysis relates S to V .The impact of quantization errors aris-ing from imposition of an intermediatespace between S and V is not addressed.Efficiency is a concern. The experimen-tal setup used to demonstrate the algo-rithm was limited to a single rotationaldegree of freedom.

—One of the major claims of the work wasthat the computational burden of visibil-ity analysis is decoupled from the size ofviewpoint space. Nevertheless, ray trac-ing figures heavily in the approach andit has not been demonstrated that thenet visibility analysis burden is loweredfor a given performance level. However,the algorithm appears to be readily par-allelizable.

—PS representation is a lossy compres-sion of pose information in which range-related information is lost. Stand-offrange critically impacts sensor perfor-mance.

—A PS image must be cast for eachviewpoint, a potentially large number.The algorithm does not provide guid-ance on efficiently sampling viewpoint

space. On the other hand, PS sensor im-ages need be calculated only once andstored offline. Memory issues are notaddressed.

—Reference is made to the use of multi-ple PS surfaces which would capture ad-ditional information in viewpoint spacebut this would appear to reduce theclaimed computational advantages ofthe representation.

6.3.3. Expert System. Artificial intelli-gence and expert system approaches toview planning have been few and limitedmainly to illumination issues [Batchelor1989; Novini 1988; Kitamura et al. 1990].

7. COMPARISON OF EXISTING METHODS

Recall the following (for papers being ref-erenced, see discussion in Section 6):

—Maver and Bajscy selected an NBVby reasoning about the angular arcthrough which occluded areas of theobject surface can be viewed, basedon assumptions about the unseentopography.

—Connolly selected an NBV as the oneable to acquire the most informa-tion about the unseen imaging volume,based on either a global or local visibil-ity analysis.



—Yuan selected an next-best-viewing di-rection based on a global analysis of themass vector chain for the object surface.

—Banta et al. discussed several view plan-ning mechanisms, the most recent beingthe NBV oriented to the centroid of thecluster containing the largest number ofunknown voxel faces.

—Massios and Fisher used an objectivefunction incorporating both visibilityand “quality” terms.

—Tarbox and Gottschlich computed ameasurability matrix encoding a com-plete visibility analysis over the set of allviewpoints and surface elements, andselected a viewpoint maximizing thevoid surface visible.

—Reed et al. first synthesized a visibilityvolume for a selected occluded edge us-ing solid geometry techniques and thenselected an NBV imaging the most oc-cluded surface elements.

—Whaite and Ferrie used uncertainty infitting parameterized superquadrics tosegmented range data as a cue to view-point selection and then selected a view-point at a fixed offset in the direction inwhich the variance of the fit of the datato the current model is the greatest.

—Pito separated the visibility analysisof object and sensor by casting visibil-ity images onto an intermediate spacerepresentation, positional space, andoptimized viewing direction selectionwithin that domain subject to variousconstraints.

—Papadopoulos-Orfanos defined a spacecarving technique for a shallow depth-of-field range sensor based on a pre-planned search pattern plus collisionavoidance.

Table III summarizes attributes of viewplanning techniques with respect to re-quirements specified in the introduction.The legend for the comparison table is asfollows: requirement satisfied (Y), not sat-isfied (N), partially satisfied (P), uncertain(?), or not applicable (−). The objective isa column of all Y’s.

8. CRITIQUE OF EXISTING METHODS

8.1. General

8.1.1. Model Quality Specification. Nomethod reviewed incorporates explicitquality goals for the reconstructed objectmodel. Massios used a quality term inthe view planning objective function butthis term was not tied to explicit pass/failcriteria. Several authors included a “mea-surement quality” term tied to grazingangle but did not relate the factor toexplicit model performance requirements.

8.1.2. Generalizable Algorithm. Severalmethods are generalizable to a broad classof range sensors, positioning systems andobjects. Others are not.

8.1.3. Generalized Viewpoint. Several au-thors described a generalized viewpoint intheir opening theoretical problem formu-lation. However, none retained the formu-lation during algorithm development. Inmost cases, pose space was constrained toone or two dimensions versus the gener-alized case which is 6D+. No author con-sidered a programmable sensor parame-ter other than pose. Some methods (Yuan,Pito) are capable of providing only incom-plete viewpoint information, specificallyviewing direction but not sensor positionor orientation about the sensor boresight.

8.1.4. View Overlap. Almost all authorsassumed error-free positioning and ig-nored view planning constraints on viewoverlap for image-based registration.Only Pito applied a view overlap con-straint, although shape complexity wasnot addressed.

8.1.5. Robust. None of the methods in-corporated both sensor and positioningsystem error models. Several methods(Massios, Tarbox, Pito) considered graz-ing angle effects as a subjective qualityfactor. No method considered geometricstep edges, reflectance step edges, or mul-tiple reflection effects. The robustness ofthe proposed methods in realistic sensingand view planning environments is either


86 Scott et al.

Table III. Comparison of View Planning Algorithms

Category Requirement Mav

er&

Baj

scy

Con

nol

ly

Yu

an

Ban

taet

al.

Mas

sios

&F

ish

er

Tar

box

&G

otts

chli

ch

Ree

det

al.

Wh

aite

&F

erri

e

Pit

o

Pap

adop

oulo

s-O

rfan

os

Model quality specification N N N N N N N N N N

Generalizable algorithm N Y N Y Y Y Y N Y N

Generalized viewpoints N N N N N N N N N N

General View overlap N N N N N N N P P N

Robust N N N N ? ? ? N ? ?

Efficient N ? ? ? ? N ? N ? N

Self-terminating N Y Y Y Y Y Y Y Y Y

Minimal a priori knowledge Y Y Y Y Y — Y N Y Y

Object Shape constraints N Y N Y Y Y Y N Y Y

Material constraints N N N N N N N N N N

Frustum modeled N N N N N N P N P Y

Sensor Shadow effect Y N N N N Y N N Y Y

Measurement performance N N N N P P N P P N

6D pose N N N N N N N N N YPositioning

Pose constraints N N N N Y Y Y N Y YSystemPositioning performance N N N N N P N N N N

Note: For papers being referenced, see discussion in Section 6.

weak or is undetermined—that is, the en-vironment of real sensors with real errorsand artifacts, real objects with representa-tive topological and geometrical complex-ity, and real positioning systems subjectto pose error and restricted movementenvelopes.

8.1.6. Efficiency. Given the lack of stan-dardized performance benchmarks, it isdifficult to make quantitative perfor-mance comparisons between proposedview planning methods. However, someauthors (Banta, Tarbox, Connolly) de-scribed multiple algorithms and pro-vided qualitative performance compar-isons. Few authors explicitly evaluated thecomputational complexity of their algo-rithms. Computational complexity is anissue with all methods. However, manyof the techniques are parallelizable and

computational power and memory are be-coming less of a concern with ongoing ad-vances in computer technology.

8.1.7. Self-Terminating. Various termina-tion criteria were used, but none relatedto explicit requirements to be met by thereconstructed object model.

8.2. Object Characterization

8.2.1. Minimal a priori Knowledge. Mostmethods imposed no a priori knowledgeconstraints on the objects to be mod-eled. Tarbox’s work focused on inspection.Whaite and Ferrie’s approach was aimedat robotic environment exploration.

8.2.2. Shape Constraints. Most methodsimposed no object shape constraints. How-ever, some would clearly be unable to han-dle complex shapes due the coarseness of



the underlying approximations, for exam-ple, the 21/2 D scene view of Maver andBajcsy [1990], the mass vector chain ap-proach [Yuan 1993], or parametric surfacerepresentation [Whaite and Ferrie 1997].Performance was not characterized withrespect to increasing object topological orgeometrical complexity.

8.2.3. Material Constraints. No methoddealt with object material effects on sensorperformance.

8.3. Sensor Characterization

8.3.1. Frustum Modeled. Most methodsdid not incorporate a realistic model of thesensor frustum, that is, the depth-of-field,angular field-of-view, and scan length/arcof real range cameras.

8.3.2. Shadow Effect. Few methods mod-eled shadow effects. This core attribute oftriangulation-based range sensors limitsobservation in cavities or around obsta-cles. A great deal of manual view plan-ning effort is spent addressing shadowzones. High-performance triangulation-based range sensors can be designedwith small optical baselines, for example,“BIRIS” and autosynchronous scanners(www.vit.iit.nrc.ca). Small bistatic shadoweffects benefit some view planning tech-niques, notably solid geometry methods aswell as all occlusion edge methods.

8.3.3. Measurement Performance Characteri-zation. With some exceptions [Pito 1999],the view planning literature generally as-sumes idealized, error-free sensors. Manymodel the camera as an ideal monos-tatic sensor. Yet geometric measurementtechniques are highly nonlinear and errorprone. A substantial portion of the effortin manual object reconstruction is spentminimizing errors in data acquisition andmodel building. The sensing error mostcommonly considered in the literature isinclination bias. No author addressed thenonisotropic and nonstationary nature ofresidual geometric noise within the sensorfrustum of a calibrated range sensor. Tra-ditional view planning approaches have

attempted to obtain complete coverage ofthe object surface with limited attentionto the quality of the geometric data.

8.4. Positioning System Characterization

8.4.1. 6D Pose. Most methods limitviewpoint space to one or two dimen-sions. The most common configurationconstrains the sensor to a “viewing sphere”with the camera boresight fixed on anobject-centered origin. It is not possibleto optimize sensor performance with suchlimitations.

8.4.2. Pose Constraints. Several of themore advanced approaches to view plan-ning incorporated positioning systemconstraints.

8.4.3. Positioning Performance. ExceptingTarbox, no author addressed the impactof pose error on view planning. Tarbox re-moved viewpoints not robust to pose vari-ation from the viewpoint set associatedwith each surface point. Positioning sys-tem performance was not modeled.

8.5. Critique Summary

8.5.1. Techniques and Representations. Byfar the most common NBV cuing mecha-nism is either the surface or the solid vol-ume of the umbra cast by a range imageview of the scene—that is, occlusion edgesor the shadow zone. The greatest variabil-ity between view planning approaches liesin the mechanism used to select the NBV.Common surface representations are vol-umetric occupancy and surface meshes.The most common representation of imag-ing work space is volumetric—voxel occu-pancy, octree, or solid geometry.

8.5.2. Object Size Assumption. Manytechniques assume that the object fallscompletely within the camera field ofview and would fail if this condition werenot met. For example, if a range imagecontains no jump edges, what should thealgorithm do next?


88 Scott et al.

8.5.3. Strategies. Most approaches aremonolithic in the sense that a single tech-nique was used throughout. It would ap-pear advantageous to divide the over-all view planning problem into stages asHall-Holt [1998] suggested and considerdifferent techniques at different stages—in particular, restricting computationallyintensive techniques to limited subsetsof the overall problem. All of the meth-ods examined are deterministic. Theremay be opportunities for random selectionmethodologies.

8.5.4. Performance Objectives and Evalua-tion. No current method includes explicitpass/fail quality criteria on the outcome.Generally, there is a lack of clarity withrespect to the objective. Is view plan-ning a search for global optimality ormerely acceptable viewpoints? What de-termines success—a minimum number ofviewpoints; time to complete the job; someother criteria?

8.5.5. Suitability. As presently devel-oped, no traditional view planning methodis directly suitable for performance-oriented automated object reconstructionfor the following principle reasons: overlyconstrained viewpoint space, inadequatesensor and positioning system perfor-mance characterization, and excessivecomputational complexity.

9. OTHER VIEW PLANNING ISSUES

9.1. Theoretical Framework

Theoretical treatment of the view plan-ning problem has been limited. Tarbox andGottschlich [1995] introduced the measur-ability matrix concept in a model-basedapproach to inspection. They showed theVPP to be isomorphic to the set cover-ing problem which is known to be NP-complete. Yuan [1995] used mass vec-tor chains in a global approach to viewplanning. Whaite and Ferrie [1997] pre-sented an autonomous exploration the-ory based on minimization of model un-certainty. Soucy et al. [1998] examinedview planning from the perspective of

an infinitely small surface explorer andlocal topological operations on a voxelizedsurface. They also took the capabilitiesand limitations of the positioning systeminto account. Arbel and Ferrie [1999] pre-sented an entropy-based gaze planningstrategy in the somewhat related field ofobject recognition. Scott et al. [2000] ex-tended Tarbox’s work to object reconstruc-tion with a set theory-based formulationof the VPP in terms of a measurabilitymapping between viewpoint space and ob-ject surface space. Scott et al. [2001a]later showed that the view planning prob-lem can be expressed as the problem ofcovering the rows of a binary measura-bility matrix by a minimal subset of itscolumns and provided an integer program-ming formulation of the problem with aregistration constraint. Most other workin the field has relied on a variety ofheuristic techniques without a theoreticalframework.

The referenced work has contributed toa better understanding of the view plan-ning problem, yet the field lacks a com-prehensive theoretical foundation. A trulycomprehensive theory would encompassall problem constraints and error mech-anisms and provide a theoretical basisfor sampling surface space and viewpointspace.

9.2. Viewpoint Generation

The view planning process computes a setof viewpoints to satisfy the specified recon-struction or inspection goal. This is usu-ally done by synthesis or generate and testmethods. Synthesis methods (Tarabaniset al. [1995b]; Reed et al. [1997a]) are inthe minority. In principle, synthesis meth-ods have the advantage of simultaneouslyoptimizing competing factors over multi-dimensional viewpoint space but, in prac-tice, computational difficulties of nonlin-earity and convergence usually arise. Thegreat majority of view planning methodsfollow the generate and test paradigm—that is, viewpoint space is discretized bysome method and the view planning algo-rithm selects a set of these by some opti-mization algorithm.



9.2.1. The Concept of a “Viewpoint.” Somesubtleties in use of the term viewpointmerit elaboration. In this survey, we con-sider a viewpoint to define a 3D frus-tum. However, most triangulation rangecameras are actually 2D sensors. Theymeasure depth z over a 2D fan-shapedregion swept by the laser in the sensorx-z plane. The third measurement dimen-sion is achieved by physically moving thesensor perpendicular to the plane of light.This can be achieved by a sweeping mo-tion along the camera y-axis (line-scanmode) or rotation about an axis parallel tothe camera x-axis (cylindrical-scan mode).Range data gathered in such a mannerwill form a “range image” in the conven-tional sense—that is, a u-v grid of depthpixels.

A different data acquisition approachis to move the sensor through an arbi-trary arc in space to collect a long swathof laser scan profiles. The motion veloc-ity vector need not have constant speedor direction. Sensor orientation may beconstant or variable during the sweep.Lamb et al. [1999] and Soucy et al. [1998]employed this mode in contour follow-ing strategies using a CMM as the po-sitioning system. In their setup, sensorposition could be changed readily but ori-entation changes were costly in recon-figuration and recalibration time. Spacecarving techniques such as those used byPapadopoulos-Orfanos and Schmitt [1997]acquire range data in a similar manner.

Our use of the term viewpoint followsthe first of these conventions. A viewpointdefines a single discrete position and ori-entation in space, with an associated con-stant 3D frustum shape. In addition topose, a generalized viewpoint also has aset of configurable sensor parameters. Weassociate “range images” with viewpoints.For most cameras, range image size (num-ber of range pixels) is either fixed or is pro-grammable over a narrow range of values.

A more appropriate term for the sens-ing operation commonly used for contourfollowing and space carving would be aview swath. A view swath has a constant2D frustum defined in the scanning plane.A view swath defines a continuous set of

viewing positions and orientations oversome arc in space, in effect a collectionof 2D laser profiles. Neither speed nor di-rection of the velocity vector defining thesweep arc need be constant. View swathesare associated with either a set of 2D rangeprofiles of arbitrary length or a 3D pointcloud of arbitrary size.

9.2.2. Generalized Viewpoints. Most viewplanning methods sharply constrain thedimensionality of viewpoint space tolimit computational complexity. Few au-thors have recognized the importanceof sensor parameters in addition topose. Tarabanis [1995b] and Pito [1999]were exceptions. View planning for high-performance sensors and difficult recon-struction tasks cannot ignore configurablesensor parameters.

9.2.3. View Sphere. The most commonstratagem for constraining viewpointspace V uses a virtual, object-centered“view sphere” enclosing the object ([Bantaand Abidi 1996a; Connolly 1985a; Garciaet al. 1998a; Massios and Fisher 1998;Morooka et al. 1999; Reed et al. 1997b;Tarbox and Gottschlich 1995; Yi et al.1995]). This reduces the view planningproblem to the task of finding the bestset of viewing directions. The viewing axisof any viewpoint on the sphere is ori-ented to the center of the reference frame.Variable orientations about the viewingaxis have rarely been considered. Thisstratagem reduces the dimensionality ofV from 6D+ to 2D. A number of authorshave gone further, reducing V to a circu-lar 1D domain, a simplification trivializ-ing the problem and masking the underly-ing complexity. Many authors also appearto have treated triangulation-based rangecameras as monostatic devices, thus dis-missing shadow-effects, an intrinsic sen-sor characteristic and key aspect of realis-tic view planning.

A few synthesis methods ([Reed et al.1997b; Tarabanis et al. 1995b]) havetreated the view sphere as continuous.The majority, however, have used dis-cretization. This includes parallels and


90 Scott et al.

meridians [Connolly 1985; Zha et al.1997], spherical distribution maps (SDMs)([Garcia et al. 1998a; Ye and Tsotsos 1999;Yi et al. 1995]) or, the most popular, a tes-sellated icosahedron ([Massios and Fisher1998; Morooka et al. 1999; Sakane et al.1992; Tarbox and Gottschlich 1995; Truccoet al. 1997; Zha et al. 1998]). While it iswell known that a uniform tessellation ofa sphere does not exist, a tessellated icosa-hedron provides a close approximation.Morooka et al. [1999] went further, trans-forming the geodesic tessellation into aflattened spherical array to improve effi-ciency in mapping orientation to geodesiccells.

Discretization by parallels and merid-ians offers simplicity at the expense ofnonuniformity. A tessellated icosahedronprovides an almost uniform subdivisionbut may require complex computationsregarding cell occupancy. The SDM ap-proach offers intermediate performance—fast mapping of orientations to cells withreasonable cell uniformity, although infe-rior to the tessellated icosahedron.

Visibility analysis is an essential butcomputationally costly component ofmany view planning algorithms. The ap-proach of Tarbox and Gottschlich [1995]is unique in that viewpoint space wasdefined as the set of all points on the viewsphere separated by a distance equal tothe sensor optical baseline. For a tessel-lated icosahedron, this provides five or sixorientations about the sensor boresightper viewing axis. In Tarbox’s algorithm,view sphere vertices serve as locationsfor either the optical source or receiver.Ray tracing is conducted once per vertexrather than twice per viewpoint, thusreducing the cost of visibility analysis.

Almost all users of the view spherestrategy select a fixed stand-off distancesuch that the sensor frustum enclosesthe whole object. This may be necessaryfor some conventional surface-based tech-niques relying on occlusion edges to cuethe next-best-view selection scheme. How-ever, the strategy fails to take into accountthe important dependence of sensor per-formance on stand-off distance, a key fea-ture of all triangulation-based range cam-

eras. Also, it fails completely for imagingtasks for which object size exceeds frustumdimensions.

The principal advantages of the viewsphere technique are twofold: a reductionin the dimensionality of the pose compo-nent of viewpoint space from six to two,and relatively uniform sampling of theaxis component of orientation (except fordiscretization by parallels and meridians).Its main disadvantages are as follows.To the degree that object shape is notspherical, viewpoint position is nonopti-mal. Measurement performance is furtherdegraded if the view sphere radius must beset to view the entire object. Additionally,the view sphere ties orientation to posi-tion whereas there is more to achievinggood viewing directions than merely se-lecting the orientation of the sensor bore-sight. The position from which the imageis taken is also important. Further, theview sphere approach says nothing aboutthe orientation of the camera around itsboresight (the twist angle). For mid- tohigh-performance range cameras, the op-tical baseline typically subtends an anglein the range (50◦–35◦) at the sensor’s op-timal stand-off range. Consequently, theshadow effect is sensitive to rotation aboutthe sensor boresight.

9.2.4. Intermediate Space. Pito [1999]separated visibility analysis of the objectsurface from that of the sensor by castingvisibility images onto a virtual inter-mediate space representation, positionalspace (PS), positioned between the object’sconvex hull and the sensor. Visibility isverified by comparing the orientation ofsensor ranging rays and object obser-vation rays as encoded with respect tothe local reference frame in positionalspace. However, PS representation is alossy compression of pose information inwhich range-related information is lost.Stand-off range critically impacts sensorperformance.

9.2.5. Viewpoint Generation Summary. Insummary, most techniques constrainviewpoint space to two dimensions, aview sphere or view cylinder, and in some



cases to a one-dimensional viewing cir-cle. Viewpoint positions are constrainedto this surface or arc and the sensorboresight is fixed on the object center.Orientation about the sensor boresight isfixed (excepting the approach by Tarbox).The view sphere radius is also fixed, mostcommonly to a distance ensuring visibilityof the complete object. The requirement toconsider generalized viewpoints has notbeen widely recognized. The view sphereapproach is mathematically attractiveand provides reasonably uniform sam-pling in orientation but fails to optimizeposition or some aspects of orientation.Finally, the practicality of matchingvirtual spherical viewpoint domains withthe capabilities and limitations of realpositioning systems, real sensors, andreal objects is rarely considered.

9.3. Pose Error

View planning is a computationally inten-sive task with the objective of arriving at asmall set of optimal or near-optimal view-points. When the NBV list is sent to a posi-tioning system whose position and orien-tation accuracy is inferior to that of thesensor, the coverage of individual view-points and of the NBV list as a whole iscompromised. Individual viewpoint posi-tions and orientations are corrupted. Ori-entation error is particularly troublesomeas effects are amplified by range. As illus-trated in Figure 9, image coverage (frus-tum occupancy), measurement precision,sampling density, and visibility will all beeffected.

We can recover a refined pose estimatepost facto by employing suitable registra-tion techniques and subsequently reesti-mate measurement quality within the ac-quired image. However, we are still leftwith data acquisition differing from thatwhich had been planned. As pose errordeteriorates, the computationally inten-sive view planning phase is progressivelycompromised—ultimately to be renderedfutile. Consequently, there is a need tomake the view planning process robustwith respect to pose uncertainty resultingfrom positioning system errors.

Fig. 9 . View planning with poseuncertainty.

The problem of pose error has receivedlittle attention in the view planning lit-erature. Tarabanis et al. [1995b] used asynthesis approach for generalized view-points which seeks to centralize view-points in the admissible domain. Tarboxand Gottschlich [1995] used morphologi-cal erosion of viewpoints on the periph-ery of viewpoint sets to reduce vulner-ability to pose error. Recently, Scott etal. [2002] examined pose error effects onrange sensing.

10. CONCLUSION

10.1. Open Problems

This paper has surveyed and comparedview planning techniques for automated3D object reconstruction and inspectionby means of active, triangulation-basedrange sensors. Subject to previously notedstrengths and weaknesses, some currentmethods are usable in a productionsetting if suitably integrated and if recon-struction/inspection quality requirementsare relaxed, particularly for a sceneexploration stage. This would includevoxel occupancy, occlusion edge, spacecarving, solid geometry, contour following,and intermediate space techniques. Thesecomments apply to objects with relativelysimple shape and benign reflectanceproperties. As yet, view planning for high-quality object reconstruction/inspectionhas no general-purpose solution. There


92 Scott et al.

are no known general-purpose automatedgeometric modeling environments orperformance-oriented view planning sys-tems in public usage. The remaining openproblems are simultaneously both intrigu-ingly simple and exceptionally complex—efficiency, accuracy, and robustness.

The efficiency issue relates to the com-putational complexity of the view plan-ning algorithm in terms of both time andmemory, although timeliness is probablythe more important factor. No method pro-posed to date is able to provide both per-formance and efficiency. In fact, even withunrealistic constraints on the problem do-main, most view planning techniques areunacceptably slow. However, ongoing com-puter technology developments are mak-ing this less of a concern.

Accuracy and robustness considerationsgo hand-in-hand. Both demand that theview planning module incorporate rea-sonably complete sensor noise and arti-fact models based on performance char-acterization of the sensor and positioningsystem. Additionally, high-accuracy objectreconstruction requires a highly mobilepositioning system. This means desirablya full six degrees of freedom over an ade-quate imaging work space. View planningalgorithms and imaging environmentsrestricted to a one- or two- dimensionalviewpoint space can at best providelimited-quality, limited-coverage modelsfor a restricted class of objects.

Finally, there is a need to measure andmodel object reflectance to handle shinysurfaces and compensate for limited rangecamera dynamic range. View planningtechniques are also required to avoid orcompensate for the effects of geometricstep edges, reflectance step edges, andmultiple reflections.

10.2. Future Research Directions

This survey of current view planningtechniques suggests more flexible, multi-faceted methods:

1. a multi-phase approach such as sceneexploration, rough modeling, fine mod-eling, problem resolution;

2. use of multiple techniques in combina-tion instead of a monolithic approachusing a single technique;

3. heuristic and expert system designsemulating human operators; and

4. multisensor fusion approaches—perhaps using a fast, wide field-of-view,low-precision sensor for scene explo-ration and collision avoidance in combi-nation with a high-quality range sensorfor precise surface measurements.

Overcoming dropouts and outliers dueto limited sensor dynamic range remainsan important open issue. Part of thesolution will be found in sensor im-provements, in particular digital signalprocessor-based advanced signal process-ing on the raw data. Further improve-ments should be possible by incorporatingreflectance measurement and modeling ina scene exploration stage, which couldalso address two other important errorsources—reflectance step edges and mul-tiple reflections.

Additionally, it is evident that the fieldcould benefit from standardized perfor-mance benchmarks and quantitative al-gorithm performance analysis. Finally,to date there has been only limitedtreatment of theoretical aspects of theproblem.

Predicting future developments is haz-ardous but we will make a few projec-tions. The growing capability of low-costcomputers will mitigate the previouslyprohibitive computational expense of vis-ibility analysis, a core element of manyview planning techniques. Practical viewplanning solutions are unlikely to bemonolithic but rather to involve multi-ple stages, techniques, and sensor types.Model-based approaches are likely to befruitful, particularly for high-quality re-construction/inspection applications. Posi-tioning system performance and cost willcontinue to be a problem. This is an under-studied area. Technology and applicationsfor 3D sensors will continue to proliferaterapidly, mainly in the directions of speedand lower cost, and more slowly in per-formance. Cost, time, and user skill levelissues will grow, increasing pressure for



automation solutions. Dynamic range lim-itations of optical sensors will continue topresent challenges for the foreseeable fu-ture. Fully automated object reconstruc-tion/inspection is some distance off andwill involve not just research but complexsystem engineering and integration chal-lenges. Nearer-term solutions will providesemiautomated aids to skilled operators.

REFERENCES

ABRAMS, S. 1997. Sensor planning in an activerobot work-cell. Ph.D. dissertation, ColumbiaUniversity, New York, NY.

AMANN, M.-C., BOSCH, T., MYLLYLA, R., AND RIOUX,M. 2001. Laser ranging: a critical review ofusual techniques for distance measurement.Opt. Eng. 40, 1, 10–19.

ARBEL, T. AND FERRIE, F. 1999. Entropy-based gazeplanning. In Proceedings of the 2nd IEEE Work-shop on Perception for Mobile Agents, in associ-ation with 1999 IEEE Computer Society Confer-rence on Computer Vision and Pattern Recogni-tion (Fort Collins, CO.). 87–94.

AUBRY, S. 1994. 3D model construction from multi-ple sensor viewpoints. Ph.D. dissertation, McGillUniversity, Montreal, P.Q., Canada.

BANTA, J. AND ABIDI, M. 1996. Autonomous place-ment of a range sensor for acquisition of optimal3D models. In Proceedings of the IEEE 22nd In-ternational Conference on Industrial Electronics,Control and Instrumentation (Taipei, Taiwan).1583–1588.

BANTA, J., ZHIEN, Y., WANG, X., ZHANG, G., SMITH,M., AND ABIDI, M. 1995. A best-next-view al-gorithm for three-dimensional scene reconstruc-tion using range images. SPIE 2588, 418–429.

BANTA, J. E. 1996. Optimal range sensor position-ing for three-dimensional model reconstruction.M.S. thesis, University of Tennessee, Knoxville,TN.

BARIBEAU, R., RIOUX, M., AND GODIN, G. 1991. Colorreflectance modeling using a polychromatic laserrange sensor. IEEE Trans. PAMI 14, 2 (Feb.),263–269.

BATCHELOR, B. 1989. A prolog lighting advisor. InProceedings of the SPIE Conference on Intelli-gent Robots and Computer Vision VIII: Systemsand Applications. Vol. 1193. 295–302.

BERALDIN, J.-A., EL-HAKIM, S., AND COURNOYER, L.1993. Practical range camera calibration. InProceedings of the SPIE Conference on Videomet-rics II (Boston, MA). Vol. 2067. 21–30.

BESL, P. 1989. Range image sensors. In Advancesin Machine Vision, J. Sanz, Ed. Springer-Verlag,New York, NY.

BESL, P. AND MCKAY, N. 1992. A method for regis-tration of 3D shapes. IEEE Trans. PAMI 14, 2(Feb.), 239–256.

BEST, L. C. 1996. Autonomous construction ofthree-dimensional models from range data.Ph.D. dissertation, University of Wyoming,Laramie, WY.

BOWYER, K. AND DYER, C. 1990. Aspect graphs: Anintroduction and survey of recent results. InProceedings of the SPIE Conference on Close-Range Photogrammetry Meets Machine Vision.Vol. 1395. 200–208.

BUZINSKI, M. J. 1990. A comparison of touch probeand laser triangulation sensors for verifying thedimensional accuracy of manufactured parts.M.S. thesis, Purdue University, Lafayette, IN.

CONNOLLY, C. 1985. The determination of next bestviews. In Proceedings of the IEEE InternationalConference on Robotics and Automation. 432–435.

COWAN, C. AND KOVESI, P. 1988. Automatic sensorplacement from vision task requirements. IEEETrans. PAMI 10, 3 (May), 407–416.

CURLESS, B. AND LEVOY, M. 1996. Better optical tri-angulation through space-time analysis. In SIG-GRAPH ’96. 1–10.

CURLESS, B. L. 1997. New methods for surface re-construction from range images. Ph.D. disserta-tion, Stanford University, Stanford, CA.

EL-HAKIM, S. AND BERALDIN, J.-A. 1994. On the in-tegration of range and intensity data to improvevision-based three-dimensional measurements.In Proceedings of the SPIE Conference on Video-metrics III (Boston, MA). Vol. 2350. 306–321.

FAUGERAS, O., MUNDY, J., AHUJA, N., DYER, C.,PENTLAND, A., JAIN, R., AND IKEUCHI, K. 1992.Why aspect graphs are not (yet) practicalfor computer vision. CVGIP: Image Under-stand. 55, 2, 212–218.

GARCIA, M., VELAZQUEZ, S., AND SAPPA, A. 1998a. Atwo-stage algorithm for planning the next viewfrom range images. In British Machine VisionConference 1998. 720–729.

GARCIA, M., VELAZQUEZ, S., SAPPA, A., AND BASANEZ, L.1998b. Autonomous sensor planning for 3Dreconstruction of complex objects from rangeimages. In Proceedings of the IEEE Interna-tional Conference on Robotics and Automation(Leuven, Belgium). 3085–3090.

GREESPAN, M. 2002. Geometric probing of denserange data. IEEE Trans. PAMI 24, 4 (April),495–508.

HALL-HOLT, O. 1998. Technical report, StanfordUniversity, Stanford, CA. Available online atwww-graphics.stanford.edu/∼olaf/nbv.html.

HEBERT, P. 2001. A self-referenced hand-heldrange sensor. In 3rd International Conferenceon 3D Digital Imaging and Modeling (QuebecCity, P.Q., Canada). 5–12.

HUBER, D. 2001. Automatic 3D modeling usingrange images obtained from unknown view-points. In 3rd International Conference on 3DDigital Imaging and Modeling (Quebec City,P.Q., Canada). 153–160.


94 Scott et al.

HUTCHINSON, S. AND KAK, A. 1989. Planning sens-ing strategies in a robot work cell with multi-sensor capabilities. IEEE Trans. Robot. Au-tomat. 5, 6 (Dec.), 765–783.

KAHN, J., KLAWE, M., AND KLEITMAN, D. 1980. Tra-ditional galleries require fewer watchmen. Tech.rep. RJ3021, IBM Watson Research Center,Yorktown Heights, NY.

KITAMURA, Y., SATO, H., AND TAMURA, H. 1990. Anexpert system for industrial machine vision. InProceedings of the 10th International Conferenceon Pattern Recognition. 771–773.

KUTULAKOS, K. AND SEITZ, S. 2000. A theory ofshape by space carving. Intl. J. Comp. Vis. 38, 3,1999–218.

KUTULAKOS, K. N. 1994. Exploring three-dimensional objects by controlling the point ofobservation. Ph.D. dissertation, University ofWisconsin-Madison, Madison, WI.

LAMB, D., BAIRD, D., AND GREENSPAN, M. 1999. Anautomation system for industrial 3D laser dig-itizing. In Proceedings of the 2nd InternationalConference on 3D Digital Imaging and Modeling(Ottawa, Ont., Canada). 148–157.

LANGIS, C., GREENSPAN, M., AND GODIN, G. 2001. Theparallel iterative closest point algorithm. In Pro-ceedings of the 3rd International Conference on3D Digital Imaging and Modeling (Quebec City,P.Q., Canada). 195–202.

MASSIOS, N. 1997. Predicting the best next view fora laser range striper system. M.S. thesis, Univer-sity of Edinburgh, Edinburgh, Scottland.

MASSIOS, N. AND FISHER, R. 1998. A best next viewselection algorithm incorporating a quality crite-rion. In British Machine Vision Conference (Sept.1998). 780–789.

MAVER, J. 1995. Collecting visual information us-ing an active sensor system. Ph.D. dissertation,University of Ljubljana, Ljubljana, Slovenia.

MAVER, J. AND BAJCSY, R. 1990. How to decide formthe first view where to look next. In Proceedingsof the DARPA Image Understanding Workshop(Pittsburgh, PA). 482–496.

MAVER, J. AND BAJCSY, R. 1993. Occlusions as aguide for planning the next view. IEEE Trans.PAMI 17, 5 (May), 417–433.

MILROY, M., BRADLEY, C., AND VICKERS, G. 1996. Au-tomated laser scanning based on orthogonalcross sections. Machine Vis. Appl. 9, 106–118.

MOROOKA, K., ZHA, H., AND HASEGAWA, T. 1999.Computations on a spherical view space for effi-cient planning of viewpoints in 3D object model-ing. In Proceedings of the 2nd International Con-ference on 3D Digital Imaging and Modeling (Ot-tawa, Ont., Canada). 138–147.

NEWMAN, T. AND JAIN, A. 1995. A survey of auto-mated visual inspection. Comput. Vis. Image Un-derstand. 61, 2 (March), 231–262.

NOVINI, A. 1988. Lighting and optics expert sys-tem for machine vision. In Proceedings of theSPIE Conference on Optics, Illumination, and

Image Sensing for Machine Vision III. Vol. 1005.131–136.

PAPADOPOULOS-ORFANOS, D. 1997. Numerisationgeometrique automatique a l’aide d’une cam-era 3D de precision a profondeur de champreduite. Ph.D. dissertation, Ecole nationalesuperieure des telecommunications, Paris,France.

PAPADOPOULOS-ORFANOS, D. AND SCHMITT, F. 1997.Automatic 3D digitization using a laserrangefinder with a small field of view. In Pro-ceedings of the International Conference onRecent Advances in 3D Digital Imaging andModeling (Ottawa, Ont., Canada). 60–67.

PITO, R. 1996a. Mesh integration based on co-measurements. In Proceedings of the Interna-tional Conference on Image Processing (Lau-sanne, Switzerland). Vol. 2. 397–400.

PITO, R. 1996b. A sensor based solution to the nextbest view problem. In Proceedings of the Interna-tional Conference on Pattern Recognition. 941–945.

PITO, R. 1997a. Automated surface acquisition us-ing range cameras. Ph.D. dissertation, Univer-sity of Pennsylvania, Philadelphia, PA.

PITO, R. 1997b. A registration aid. In Proceedingsof the International Conference on Recent Ad-vances in 3D Digital Imaging and Modeling(Ottawa, Ont., Canada). 85–92.

PITO, R. 1999. A solution to the next best viewproblem for automated surface acquisition.IEEE Trans. PAMI 21, 10 (Oct.), 1016–1030.

PITO, R. AND BAJCSY, R. 1995. A solution to the next-best-view problem for automated cad model ac-quisition of free-form objects using range cam-eras. In Proceedings of the SPIE Symposium onModeling, Simulation and Control Technologiesfor Manufacturing (Philadelphia, PA.) Vol. 2596.78–89.

PRIETO, F. 1999. Metrologie assistee par ordina-teur: Apport des capteurs 3D sans contact. Ph.D.dissertation, L’Institut National des SciencesAppliquees de Lyon, Lyon, France.

PRIETO, F., REDARCE, T., BOULANGER, P., AND LEPAGE, R.1999. CAD-based range sensor placement foroptimum 3D data acquisition. In Proceedings ofthe International Conference on 3D Digital Imag-ing and Modeling (Ottawa, Ont., Canada). 128–137.

PRIETO, F., REDARCE, T., BOULANGER, P., AND LEPAGE, R.2001. Tolerance control with high resolution3D measurements. In 3rd International Con-ference on 3D Digital Imaging and Modeling(Quebec City, P.Q., Canada). 339–346.

PUDNEY, C. J. 1994. Surface modeling and surfacefollowing for robots equipped with range sen-sors. Ph.D. dissertation, University of WesternAustralia, Perth, Australia.

REED, M. AND ALLEN, P. 1997. A robotic sys-tem for 3D model acquisition from mul-tiple range images. In Proceedings of theIEEE International Conference on Robotics



and Automation (Albuquerque, NM.). 2509–2514.

REED, M. AND ALLEN, P. 1999. 3D modeling fromrange imagery. Image Vis. Comput. 17, 1 (Feb.),99–111.

REED, M., ALLEN, P., AND STAMOS, I. 1997a. 3Dmodeling from range imagery: An incrementalmethod with a planning component. In Proceed-ings of the International Conference on RecentAdvances in 3D Digital Imaging and Modeling(Ottawa, Ont., Canada). 76–83.

REED, M., ALLEN, P., AND STAMOS, I. 1997b. Auto-mated model acquisition from range images withview planning. In Proceedings of the IEEE Con-ference Visual Pattern Recognition (San Juan,Puerto Rico). 72–77.

REED, M. K. 1998. Solid model acquisition formrange imagery. Ph.D. dissertation, ColumbiaUniversity, New York, NY.

ROBERTS, D. AND MARSHALL, A. 1997. A review ofviewpoint planning. Tech. Rep. 97007, Uni-versity of Wales College of Cardiff, Cardiff,U.K.

ROTH, G. 2000. Building models from sensor data:an application shared by the computer vi-sion and the computer graphics community. InConfluence of Computer Vision and ComputerGraphics, A. Leonardis et al., Eds. Kluwer TheHague, The Netherlands.

ROUI-ABIDI, B. 1992. Automatic sensor placementfor volumetric object characterization. Ph.D. dis-sertation, University of Tennessee, Knoxville,TN.

RUSINKIEWICZ, S. AND LEVOY, M. 2001. Efficient vari-ants of the ICP algorithm. In Proceedings ofthe 3rd International Conference on 3D Digi-tal Imaging and Modeling (Quebec City, P.Q.,Canada). 145–152.

SAKANE, S., NIEPOLD, R., SATO, T., AND SHIRAI, Y. 1992.Illumination setup planning for a hand-eye sys-tem based on an environmental model. Advan.Robot. 6, 4, 461–482.

SCOTT, W., ROTH, G., AND RIVEST, J.-F. 2000.Performance-oriented view planning for au-tomatic model acquisition. In Proceedings ofthe 31st International Symposium on Robotics(Montreal, P.Q., Canada). 314–319.

SCOTT, W., ROTH, G., AND RIVEST, J.-F. 2001a. Viewplanning as a set covering problem. Tech.Rep. NRC-44892. National Research Council ofCanada, Institute for Information Technology,Ottawa, Ont., Canada.

SCOTT, W., ROTH, G., AND RIVEST, J.-F. 2001b. Viewplanning for multi-stage object reconstruction.In Proceedings of the 14th International Confer-ence on Vision Interface (Ottawa, Ont., Canada).64–71.

SCOTT, W., ROTH, G., AND RIVEST, J.-F. 2002. Poseerror effects on range sensing. In Proceedingsof the 15th International Conference on Vi-sion Interface (Calgary, Alta., Canada). 331–338.

SCOTT, W. R. 2002. Performance-oriented viewplanning for automated object reconstruc-tion. Ph.D. dissertation, University of Ottawa,Ottawa, Ont., Canada.

SEDAS-GERSEY, S. W. 1993. Algorithms for auto-matic sensor placement to acquire completeand accurate information. Ph.D. dissertation,Carnegie Mellon University, Pittsburgh, PA.

SEQUEIRA, V. 1996. Active range sensing for three-dimensional environment reconstruction. Ph.D.dissertation, IST-Technical University of Lisbon,Lisbon, Portugal.

SEQUEIRA, V. AND GONCALVES, J. 2002. 3D realitymodeling: Photo-realistic 3D models of realworld scenes. Proceedings of the First Interna-tional Symposium on 3D Data Processing Visual-ization and Transmission (3DPVT02). 776–783.

SOUCY, G., CALLARI, F., AND FERRIE, F. 1998. Uni-form and complete surface coverage with arobot-mounted laser rangefinder. Proceedingsof the IEEE/RSJ International Conference onIntelligent Robots and Systems (Victoria, B.C.,Canada). 1682–1688.

TARABANIS, K. 1992. Sensor planning and model-ing for machine vision tasks. Ph.D. dissertation,Columbia University, New York, NY.

TARABANIS, K., ALLEN, P., AND TSAI, R. 1995a. A sur-vey of sensor planning in computer vision. IEEETrans. Robot. Automat. 11, 1 (Feb.), 86–104.

TARABANIS, K., TSAI, R., AND ALLEN, P. 1995b. TheMVP sensor planning system for robotic visiontasks. IEEE Trans. Robot. Automat. 11, 1 (Feb.),72–85.

TARABANIS, K., TSAI, R., AND KAUL, A. 1996. Com-puting occlusion-free viewpoints. IEEE Trans.PAMI 18, 3 (March), 279–292.

TARBOX, G. AND GOTTSCHLICH, S. 1995. Planning forcomplete sensor coverage in inspection. Comput.Vis. Image Understand. 61, 1 (Jan.), 84–111.

TARBOX, G. H. 1993. A volumetric approach toautomated 3D inspection. Ph.D. dissertation.Rensselaer Polytechnic Institute, Troy, NY.

TREMBLAY, P.-J. AND FERRIE, F. 2000. The skepticalexplorer: A multi-hypotheses approach to visualmodeling and exploration. Auton. Robot. 8, 2,193–201.

TRUCCO, E., UMASUTHAN, M., WALLACE, A., AND ROBERTO,V. 1997. Model-based planning of optimalsensor placements for inspection. IEEE Trans.Robot. Automat. 13, 2 (April), 182–194.

URRUTIA, J. 2000. Art gallery and illuminationproblems. In Handbook of Computational Ge-ometry, J.-R. Sack and J. Urrutia, Eds. Elsevier,New York, NY.

WHAITE, P. AND FERRIE, F. 1990. From uncertaintyto visual exploration. In Proceedings of the 3rdInternational Conference on Computer Vision(Osaka, Japan). 690–697.

WHAITE, P. AND FERRIE, F. 1991. From uncertaintyto visual exploration. IEEE Trans. PAMI 13, 10(Oct.), 1038–1049.


96 Scott et al.

WHAITE, P. AND FERRIE, F. 1992. Uncertain views.In Proceedings of the IEEE Conference on VisualPattern Recognition. 3–9.

WHAITE, P. AND FERRIE, F. 1997. Autonomous ex-ploration: Driven by uncertainty. IEEE Trans.PAMI 19, 3 (March), 193–205.

WHEELER, M. D. 1996. Automatic modeling and lo-calization for object recognition. Ph.D. disser-tation, Carnegie Mellon University, Pittsburgh,PA.

WIXSON, L. E. 1994. Gaze selection for visualsearch. Ph.D. thesis, University of Rochester,Rochester, NY.

XIE, S., CALVERT, T., AND BHATTACHARYA, B. 1986.Planning views for the incremental constructionof model bodies. In Proceedings of the 8th In-ternational Conference on Pattern Recognition(Paris, France). 154–157.

YE, Y. 1997. Sensor planning for object search.Ph.D. dissertation, University of Toronto,Toronto, Ont., Canada.

YE, Y. AND TSOTSOS, J. 1999. Sensor planning for3D object search. Comput. Vis. Image Under-stand. 73, 2 (Feb.), 145–168.

YI, S. 1990. Illumination control expert for ma-chine vision: A goal driven approach. Ph.D. dis-sertation, University of Washington, Seattle,WA.

YI, S., HARALICK, R., AND SHAPIRO, L. 1995. Optimalsensor and light source positioning for machinevision. Comput. Vis. Image Understand. 61, 1(Jan.), 122–137.

YU, Y. AND GUPTA, K. 2000. An information theo-retic approach to view point planning for motionplanning of eye-in-hand systems. In Proceedingsof the 31st International Symposium on Robotics.306–311.

YUAN, X. 1993. 3D reconstruction as an automaticmodeling system. Ph.D. dissertation, Universityof Alberta, Edmonton, Alta., Canada.

YUAN, X. 1995. A mechanism of automatic 3D ob-ject modeling. IEEE Trans. PAMI 17, 3 (March),307–311.

ZHA, H., MOROOKA, K., AND HASEGAWA, T. 1998. Nextbest viewpoint (NBV) planning for active ob-ject modeling based on a learning-by-showingapproach. In Proceedings of the 3rd AsianConference on Computer Vision (Hong Kong).185–192.

ZHA, H., MOROOKA, K., HASEGAWA, T., AND NAGATA, T.1997. Active modeling of 3D objects: Planningon the next best pose (NBP) for acquiring rangeimages. In Proceedings of the International Con-ference on Recent Advances in 3D Digital Imag-ing and Modeling (Ottawa, Ont., Canada). 68–75.

Received March 2002; revised September 2002; accepted November 2002


nrc-iit publications iti-cnrc | view planning for

Documents