3D Scanning3D Scanning

Acknowledgement: some content and figures by Brian Curless

Data TypesData Types

• Volumetric DataVolumetric Data– Voxel gridsVoxel grids– OccupancyOccupancy– DensityDensity

• Surface DataSurface Data– Point cloudsPoint clouds– Range images (range maps)Range images (range maps)

Related FieldsRelated Fields

• Computer VisionComputer Vision– Passive range sensingPassive range sensing– Rarely construct complete, accurate Rarely construct complete, accurate

modelsmodels– Application: recognitionApplication: recognition

• MetrologyMetrology– Main goal: absolute accuracyMain goal: absolute accuracy– High precision, provable errors more High precision, provable errors more

important than scanning speed, complete important than scanning speed, complete coveragecoverage

– Applications: industrial inspection, quality Applications: industrial inspection, quality control, as-built modelscontrol, as-built models

Related FieldsRelated Fields

• Computer GraphicsComputer Graphics– Often want complete modelOften want complete model– Low noise, geometrically consistent Low noise, geometrically consistent

model more important than absolute model more important than absolute accuracyaccuracy

– Application: animated CG charactersApplication: animated CG characters

TerminologyTerminology

• Range acquisition, shape acquisition, Range acquisition, shape acquisition, rangefinding, range scanning, 3D rangefinding, range scanning, 3D scanningscanning

• Alignment, registrationAlignment, registration

• Surface reconstruction, 3D scan Surface reconstruction, 3D scan merging, scan integration, surface merging, scan integration, surface extractionextraction

• 3D model acquisition3D model acquisition

Range Acquisition TaxonomyRange Acquisition Taxonomy

RangeRangeacquisitionacquisition

ContactContact

TransmissiveTransmissive

ReflectiveReflectiveNon-opticalNon-optical

OpticalOptical

Industrial CTIndustrial CT

Mechanical Mechanical (CMM, jointed arm)(CMM, jointed arm)

RadarRadar

SonarSonar

UltrasoundUltrasound

MRIMRI

Ultrasonic trackersUltrasonic trackersMagnetic trackersMagnetic trackers

Inertial Inertial (gyroscope, accelerometer)(gyroscope, accelerometer)

Range Acquisition TaxonomyRange Acquisition Taxonomy

Opticalmethods

Passive

Active

Shape from X:stereomotionshadingtexturefocusdefocus

Active variants of passive methodsStereo w. projected textureActive depth from defocusPhotometric stereo

Time of flight

Triangulation

Optical Range Scanning Optical Range Scanning MethodsMethods

• Advantages:Advantages:– Non-contactNon-contact– SafeSafe– Usually inexpensiveUsually inexpensive– Usually fastUsually fast

• Disadvantages:Disadvantages:– Sensitive to transparencySensitive to transparency– Confused by specularity and interreflectionConfused by specularity and interreflection– Texture (helps some methods, hurts Texture (helps some methods, hurts

others)others)

StereoStereo

• Find feature in one image, search Find feature in one image, search along epipole in other image for along epipole in other image for correspondencecorrespondence

StereoStereo

• Advantages:Advantages:– PassivePassive– Cheap hardware (2 cameras)Cheap hardware (2 cameras)– Easy to accommodate motionEasy to accommodate motion– Intuitive analogue to human visionIntuitive analogue to human vision

• Disadvantages:Disadvantages:– Only acquire good data at “features”Only acquire good data at “features”– Sparse, relatively noisy data (correspondence is Sparse, relatively noisy data (correspondence is

hard)hard)– Bad around silhouettesBad around silhouettes– Confused by non-diffuse surfacesConfused by non-diffuse surfaces

• Variant: multibaseline stereo to reduce Variant: multibaseline stereo to reduce ambiguityambiguity

Shape from MotionShape from Motion

• ““Limiting case” of multibaseline Limiting case” of multibaseline stereostereo

• Track a feature in a video sequenceTrack a feature in a video sequence

• For For nn frames and frames and ff features, have features, have22nnff knowns, 6 knowns, 6nn+3+3ff unknowns unknowns

Shape from MotionShape from Motion

• Advantages:Advantages:– Feature tracking easier than Feature tracking easier than

correspondence in far-away viewscorrespondence in far-away views– Mathematically more stable (large Mathematically more stable (large

baseline)baseline)

• Disadvantages:Disadvantages:– Does not accommodate object motionDoes not accommodate object motion– Still problems in areas of low texture, in Still problems in areas of low texture, in

non-diffuse regions, and around silhouettesnon-diffuse regions, and around silhouettes

Shape from ShadingShape from Shading

• Given: image of surface with known, Given: image of surface with known, constant reflectance under known constant reflectance under known point lightpoint light

• Estimate normals, integrate to find Estimate normals, integrate to find surfacesurface

• Problem: ambiguityProblem: ambiguity

Shape from ShadingShape from Shading

• Advantages:Advantages:– Single imageSingle image– No correspondencesNo correspondences– Analogue in human visionAnalogue in human vision

• Disadvantages:Disadvantages:– Mathematically unstableMathematically unstable– Can’t have textureCan’t have texture

• Not really practicalNot really practical– But see photometric stereoBut see photometric stereo

Shape from TextureShape from Texture

• Mathematically similar to shape from Mathematically similar to shape from shading, but uses stretch and shrink of a shading, but uses stretch and shrink of a (regular) texture(regular) texture

Shape from TextureShape from Texture

• Analogue to human visionAnalogue to human vision

• Same disadvantages as shape from Same disadvantages as shape from shadingshading

Shape from Focus and DefocusShape from Focus and Defocus

• Shape from focus: at which focus Shape from focus: at which focus setting is a given image region setting is a given image region sharpest?sharpest?

• Shape from defocus: how out-of-Shape from defocus: how out-of-focus is each image region?focus is each image region?

• Passive versions rarely usedPassive versions rarely used

• Active depth from defocus can beActive depth from defocus can bemade practicalmade practical

Active Optical MethodsActive Optical Methods

• Advantages:Advantages:– Usually can get dense dataUsually can get dense data– Usually much more robust and Usually much more robust and

accurate than passive techniquesaccurate than passive techniques

• Disadvantages:Disadvantages:– Introduces light into scene (distracting, Introduces light into scene (distracting,

etc.)etc.)– Not motivated by human visionNot motivated by human vision

Active Variants of Passive Active Variants of Passive TechniquesTechniques

• Regular stereo with projected textureRegular stereo with projected texture– Provides features for correspondenceProvides features for correspondence

• Active depth from defocusActive depth from defocus– Known pattern helps to estimate Known pattern helps to estimate

defocusdefocus

• Photometric stereoPhotometric stereo– Shape from shading with multiple Shape from shading with multiple

known lightsknown lights

Pulsed Time of FlightPulsed Time of Flight

• Basic idea: send out pulse of light Basic idea: send out pulse of light (usually laser), time how long it takes (usually laser), time how long it takes to returnto return tcr

2

1tcr

2

1

Pulsed Time of FlightPulsed Time of Flight

• Advantages:Advantages:– Large working volume (up to 100 m.)Large working volume (up to 100 m.)

• Disadvantages:Disadvantages:– Not-so-great accuracy (at best ~5 mm.)Not-so-great accuracy (at best ~5 mm.)

• Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds• Does not scale with working volumeDoes not scale with working volume

• Often used for scanning buildings, Often used for scanning buildings, rooms, archeological sites, etc.rooms, archeological sites, etc.

AM Modulation Time of FlightAM Modulation Time of Flight

• Modulate a laser at frequencyModulate a laser at frequencym m ,, it it

returns with a phase shift returns with a phase shift

• Note the ambiguity in the measured Note the ambiguity in the measured phase!phase! Range ambiguity of Range ambiguity of 11//22mmnn

2

2

2

1 n

ν

cr

m

2

2

2

1 n

ν

cr

m

AM Modulation Time of FlightAM Modulation Time of Flight

• Accuracy / working volume tradeoffAccuracy / working volume tradeoff(e.g., noise ~ (e.g., noise ~ 11//500 500 working volume)working volume)

• In practice, often used for room-sized In practice, often used for room-sized environments (cheaper, more environments (cheaper, more accurate than pulsed time of flight)accurate than pulsed time of flight)

TriangulationTriangulation

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

• Moving independently leads to Moving independently leads to problems with focus, resolutionproblems with focus, resolution

• Most scanners mount camera and Most scanners mount camera and light source rigidly, move them as a light source rigidly, move them as a unitunit

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

Triangulation: Moving theTriangulation: Moving theCamera and IlluminationCamera and Illumination

Triangulation: Extending to 3DTriangulation: Extending to 3D

• Possibility #1: add another mirror (flying Possibility #1: add another mirror (flying spot)spot)

• Possibility #2: project a stripe, not a dotPossibility #2: project a stripe, not a dot

ObjectObject

LaserLaser

CameraCameraCameraCamera

Triangulation Scanner IssuesTriangulation Scanner Issues

• Accuracy proportional to working volume Accuracy proportional to working volume (typical is ~1000:1)(typical is ~1000:1)

• Scales down to small working vol. (e.g. 5 Scales down to small working vol. (e.g. 5 cm. working volume, 50 cm. working volume, 50 m. accuracy)m. accuracy)

• Does not scale up (baseline too large…)Does not scale up (baseline too large…)

• Two-line-of-sight problem (shadowing from Two-line-of-sight problem (shadowing from either camera or laser)either camera or laser)

• Triangulation angle: non-uniform resolution Triangulation angle: non-uniform resolution if too small, shadowing if too big (useful if too small, shadowing if too big (useful range: 15range: 15-30-30))

Triangulation Scanner IssuesTriangulation Scanner Issues

• Material properties (dark, specular)Material properties (dark, specular)

• Subsurface scatteringSubsurface scattering

• Laser speckleLaser speckle

• Edge curlEdge curl

• Texture embossingTexture embossing

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #1: assume surface Answer #1: assume surface continuitycontinuity

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #2: colored stripes (or dots)Answer #2: colored stripes (or dots)

Multi-Stripe TriangulationMulti-Stripe Triangulation

• To go faster, project multiple stripesTo go faster, project multiple stripes

• But which stripe is which?But which stripe is which?

• Answer #3: time-coded stripesAnswer #3: time-coded stripes

Time-Coded Light PatternsTime-Coded Light Patterns

• Assign each stripe a unique illumination Assign each stripe a unique illumination codecodeover time [Posdamer 82]over time [Posdamer 82]

SpaceSpace

TimeTime

Gray-Code PatternsGray-Code Patterns

• To minimize effects of quantization error:To minimize effects of quantization error:each point may be a boundary only onceeach point may be a boundary only once

SpaceSpace

TimeTime