3D Photography3D Photography

Szymon RusinkiewiczSzymon Rusinkiewicz

Fall 2002Fall 2002

3D Photography3D Photography

• Obtaining 3D shape (and sometimes Obtaining 3D shape (and sometimes color)color)of real-world objectsof real-world objects

Industrial InspectionIndustrial Inspection

• Determine whether manufactured partsDetermine whether manufactured partsare within tolerancesare within tolerances

MedicineMedicine

• Plan surgery on computer model,Plan surgery on computer model,visualize in real timevisualize in real time

MedicineMedicine

• Plan surgery on computer model,Plan surgery on computer model,visualize in real timevisualize in real time

MedicineMedicine

• Plan surgery on computer model,Plan surgery on computer model,visualize in real timevisualize in real time

MedicineMedicine

• Plan surgery on computer model,Plan surgery on computer model,visualize in real timevisualize in real time

Scanning BuildingsScanning Buildings

• Quality control during buildingQuality control during building

• As-build modelsAs-build models

Scanning BuildingsScanning Buildings

• Quality control during buildingQuality control during building

• As-build modelsAs-build models

ClothingClothing

• Scan a person, custom-fit clothingScan a person, custom-fit clothing

• U.S. Army; booths in mallsU.S. Army; booths in malls

Graphics ResearchGraphics Research

• Availability of complex datasets drives Availability of complex datasets drives researchresearch(you wouldn’t (you wouldn’t believebelieve how the poor how the poor bunny has been treated…)bunny has been treated…)

Sculpture ScanningSculpture Scanning

• The Pietà ProjectThe Pietà ProjectIBM ResearchIBM Research

• The Digital Michelangelo ProjectThe Digital Michelangelo ProjectStanford UniversityStanford University

• The Great Buddha ProjectThe Great Buddha ProjectUniversity of TokyoUniversity of Tokyo

Why Scan Sculptures?Why Scan Sculptures?

• Interesting geometryInteresting geometry

• Introduce scanning to new disciplinesIntroduce scanning to new disciplines– Art: studying working techniquesArt: studying working techniques

– Art historyArt history

– Cultural heritage preservationCultural heritage preservation

– ArcheologyArcheology

• High-visibility projectsHigh-visibility projects

Why Scan Sculptures?Why Scan Sculptures?

• ChallengingChallenging– High detail, large areasHigh detail, large areas

– Large data setsLarge data sets

– Field conditionsField conditions

– Pushing hardware, software technologyPushing hardware, software technology

• But not too challengingBut not too challenging– Simple topologySimple topology

– Possible to scan most of surfacePossible to scan most of surface

Issues AddressedIssues Addressed

• ResolutionResolution

• CoverageCoverage– Theoretical: limits of scanning technologiesTheoretical: limits of scanning technologies

– Practical: physical access, timePractical: physical access, time

• Type of dataType of data– High-res 3D data vs. coarse 3D + normal High-res 3D data vs. coarse 3D + normal

mapsmaps

– Influenced by eventual applicationInfluenced by eventual application

• Intellectual PropertyIntellectual Property

The Digital Michelangelo ProjectThe Digital Michelangelo Project

GoalsGoals

• Scan 10 sculptures by MichelangeloScan 10 sculptures by Michelangelo

• High-resolution (“quarter-millimeter”) High-resolution (“quarter-millimeter”) geometrygeometry

• Side projects: architectural scanning Side projects: architectural scanning (Accademia and Medici chapel), (Accademia and Medici chapel), scanning fragments of Forma Urbis scanning fragments of Forma Urbis RomaeRomae

Why Capture Chisel Marks?Why Capture Chisel Marks?

Atlas Atlas (Accademia)(Accademia)

ugnettougnettougnettougnetto

??

Why Capture Chisel Marks?Why Capture Chisel Marks?

Day (Medici Day (Medici Chapel)Chapel)

2 mm2 mm

Who?Who?

Faculty and staffFaculty and staffProf. Brian CurlessProf. Brian Curless John GerthJohn Gerth

Jelena JovanovicJelena Jovanovic Prof. Marc LevoyProf. Marc Levoy

Lisa PacelleLisa Pacelle Domi Pitturo Domi Pitturo

Dr. Kari PulliDr. Kari Pulli

Graduate studentsGraduate studentsSean AndersonSean Anderson Barbara CaputoBarbara Caputo

James DavisJames Davis Dave KollerDave Koller

Lucas PereiraLucas Pereira Szymon RusinkiewiczSzymon Rusinkiewicz

Jonathan ShadeJonathan Shade Marco TariniMarco Tarini

Daniel WoodDaniel Wood

UndergraduatesUndergraduatesAlana ChanAlana Chan Kathryn ChinnKathryn Chinn

Jeremy GinsbergJeremy Ginsberg Matt GinztonMatt Ginzton

Unnur GretarsdottirUnnur Gretarsdottir Rahul GuptaRahul Gupta

Wallace HuangWallace Huang Dana KatterDana Katter

Ephraim LuftEphraim Luft Dan PerkelDan Perkel

Semira RahemtullaSemira Rahemtulla Alex RoetterAlex Roetter

Joshua SchroederJoshua Schroeder Maisie TsuiMaisie Tsui

David WeeklyDavid Weekly

In FlorenceIn FlorenceDottssa Cristina Acidini Dottssa Cristina Acidini Dottssa Franca FallettiDottssa Franca Falletti

Dottssa Licia BertaniDottssa Licia Bertani Alessandra MarinoAlessandra Marino

Matti AuvinenMatti Auvinen

In RomeIn RomeProf. Eugenio La RoccaProf. Eugenio La Rocca Dottssa Susanna Le PeraDottssa Susanna Le Pera

Dottssa Anna SomellaDottssa Anna Somella Dottssa Laura FerreaDottssa Laura Ferrea

In PisaIn PisaRoberto ScopignoRoberto Scopigno

SponsorsSponsorsInterval ResearchInterval Research Paul G. Allen Foundation for the ArtsPaul G. Allen Foundation for the Arts

Stanford UniversityStanford University

Equipment donorsEquipment donorsCyberwareCyberware Cyra TechnologiesCyra Technologies

Faro TechnologiesFaro Technologies IntelIntel

Silicon GraphicsSilicon Graphics SonySony

3D Scanners3D Scanners

Scanner DesignScanner Design

4 motorized axes4 motorized axes

laser, range camera,laser, range camera,white light, and color camerawhite light, and color camera

TriangulationTriangulation

• Project laser stripe onto objectProject laser stripe onto object

ObjectObject

LaserLaser

CameraCameraCameraCamera

CameraCameraCameraCamera

TriangulationTriangulation

• Depth from ray-plane triangulationDepth from ray-plane triangulation

LaserLaser

(x,y(x,y))

ObjectObject

Scanning a Large ObjectScanning a Large Object

• Calibrated motionsCalibrated motions– pitch pitch (yellow)(yellow)

– pan pan (blue)(blue)

– horizontal translation horizontal translation (orange)(orange)

• Uncalibrated motionsUncalibrated motions– vertical translationvertical translation

– rolling the gantryrolling the gantry

– remounting the scan headremounting the scan head

Single Scan of St. MatthewSingle Scan of St. Matthew 1 mm1 mm1 mm1 mm

Statistics About the Scan of DavidStatistics About the Scan of David

• 480 individually aimed 480 individually aimed scansscans

• 0.3 mm sample spacing0.3 mm sample spacing

• 2 billion polygons2 billion polygons

• 7,000 color images7,000 color images

• 32 gigabytes32 gigabytes

• 30 nights of scanning30 nights of scanning

• 22 people22 people

Head of Michelangelo’s Head of Michelangelo’s DavidDavid

PhotographPhotograph 1.0 mm computer model1.0 mm computer model

Side project:Side project:The Forma Urbis RomaeThe Forma Urbis Romae

Forma Urbis Romae FragmentForma Urbis Romae Fragment

side face

IBM’s Pietà ProjectIBM’s Pietà Project

• Michelangelo’s Michelangelo’s “Florentine Pietà”“Florentine Pietà”

• Late work (1550s)Late work (1550s)

• Partially destroyed by Partially destroyed by Michelangelo, Michelangelo, recreated by his recreated by his studentstudent

• Currently in the Museo Currently in the Museo dell’Opera del Duomodell’Opera del Duomoin Florencein Florence

Who?Who?

• Dr. Jack Wasserman, professor emeritus Dr. Jack Wasserman, professor emeritus of art history at Temple Universityof art history at Temple University

• Visual and Geometric Computing groupVisual and Geometric Computing group@ IBM Research:@ IBM Research:

Fausto BernardiniHolly RushmeierIoana MartinJoshua Mittleman

Gabriel TaubinAndre GueziecClaudio Silva

ScannerScanner

• Visual Interface “Virtuoso”Visual Interface “Virtuoso”

• Active multibaseline stereoActive multibaseline stereo

• Projector (stripe pattern),Projector (stripe pattern),6 B&W cameras, 1 color camera6 B&W cameras, 1 color camera

• Augmented with 5 extraAugmented with 5 extra“point” light sources for“point” light sources forphotometric stereophotometric stereo(active shape from shading)(active shape from shading)

DataData

• Range data has 2 mm spacing, 0.1mm Range data has 2 mm spacing, 0.1mm noisenoise

• Each range image: 10,000 points, Each range image: 10,000 points, 202020 cm20 cm

• Color data: 5 images with controlled Color data: 5 images with controlled lighting, 1280lighting, 1280960, 0.5 mm resolution960, 0.5 mm resolution

• Total of 770 scans, 7.2 million pointsTotal of 770 scans, 7.2 million points

ScanningScanning

• Final scan June 1998, Final scan June 1998, completed July 1999completed July 1999

• Total scanning time: Total scanning time: 90 hours over 14 90 hours over 14 daysdays(includes equipment (includes equipment setup time)setup time)

ResultsResults

The Great Buddha ProjectThe Great Buddha Project

• Great Buddha of KamakuraGreat Buddha of Kamakura

• Original made of wood, completed 1243Original made of wood, completed 1243

• Covered in bronze and gold leaf, 1267Covered in bronze and gold leaf, 1267

• Approx. 15 m tallApprox. 15 m tall

• Goal: preservation ofGoal: preservation ofcultural heritagecultural heritage

Who?Who?

• Institute of Industrial Science,Institute of Industrial Science,University of TokyoUniversity of Tokyo

Daisuke MiyazakiDaisuke MiyazakiTakeshi OoishiTakeshi OoishiTaku NishikawaTaku NishikawaRyusuke SagawaRyusuke Sagawa

Ko NishinoKo NishinoTakashi TomomatsuTakashi TomomatsuYutaka TakaseYutaka TakaseKatsushi IkeuchiKatsushi Ikeuchi

ScannerScanner

• Cyrax range scanner by Cyra Cyrax range scanner by Cyra TechnologiesTechnologies

• Laser pulse time-of-flightLaser pulse time-of-flight

Pulsed Time of FlightPulsed Time of Flight

• Send out pulse of light (usually laser),Send out pulse of light (usually laser),time how long it takes to returntime how long it takes to return

tcd 2

1tcd

2

1

Pulsed Time of FlightPulsed Time of Flight

• Advantages:Advantages:– Large working volume (up to 100 m.)Large working volume (up to 100 m.)

• Disadvantages:Disadvantages:– Not-so-great accuracy (at best ~5 mm.)Not-so-great accuracy (at best ~5 mm.)

• Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds

• Does not scale with working volumeDoes not scale with working volume

• Often used for scanning buildings, Often used for scanning buildings, rooms, archeological sites, etc.rooms, archeological sites, etc.

DataData

• 20 range images (a few million points)20 range images (a few million points)

• 4 mm accuracy4 mm accuracy

• Several nights of scanningSeveral nights of scanning

ResultsResults

CourseCourse

• Meets Mondays 11-12, Wednesdays 11-Meets Mondays 11-12, Wednesdays 11-12:2012:20

• Class project – build a 3D photography Class project – build a 3D photography system from the ground upsystem from the ground up

• http://www.cs.princeton.edu/courses/http://www.cs.princeton.edu/courses/597B/597B/