capture of hair geometry from multiple images sylvain paris – hector m. briceño – françois x....

Capture of Hair GeometryCapture of Hair Geometryfrom Multiple Imagesfrom Multiple Images

Capture of Hair GeometryCapture of Hair Geometryfrom Multiple Imagesfrom Multiple Images

Sylvain Paris – Hector M. Briceño – François X. Sillion

ARTIS is a team of the GRAVIR - IMAG research lab(UMR 5527 between CNRS, INPG, INRIA and UJF), and a project of INRIA.

MotivationMotivationMotivationMotivation

Digital humans more and more common Movies, games…

Hairstyle is important Characteristic feature

Duplicating real hairstyle

Dusk demo - NVIDIA© 2004 NVIDIA Corporation. Dusk image is © 2004

by NVIDIA Corporation. All rights reserved.

MotivationMotivationMotivationMotivation

User-based duplication of hair Creation from scratch Edition at fine level

Image-based capture Automatic creation Copy of original features

Edition still possible [Kim02]

Our approachOur approachOur approachOur approach

Digital copy of real hairstyle

Only static geometry(animation and appearance as future work)

Dense set of 3D strandsfrom

multiple images

OutlineOutlineOutlineOutline

• Previous work

• Definitions and overview

• Details of the hair capture

• Results

• Conclusions

Previous workPrevious workShape reconstructionShape reconstructionPrevious workPrevious workShape reconstructionShape reconstruction

Computer Vision techniques– Shape from motion, shading, specularities

3D scanners

Difficulties with hair complexity

Only surfaces

Previous workPrevious workLight-field approachLight-field approachPrevious workPrevious workLight-field approachLight-field approach

Matusik et al. 2002

New images from:Different viewpoints + lightsAlpha mattes

Duplication of hairstyle

No 3D strands

Not editable[Matusik02]

Previous workPrevious workEditing packagesEditing packagesPrevious workPrevious workEditing packagesEditing packages

Hadap and Magnenat-Thalmann 2001

Kim et al. 2002

Dedicated tools to help the user

3D strands

Total control

Time-consuming

Duplication very hard

[Hadap01]MIRALab, University of Geneva

[Kim02]

Previous workPrevious workProcedural & Image-basedProcedural & Image-basedPrevious workPrevious workProcedural & Image-basedProcedural & Image-based

Kong et al. 1997

Hair volume from images

Procedural filling

3D strands

Duplication of hair volume

No duplication of hairstyle

New procedure for each hair type[Kong97]

Previous workPrevious workImage-basedImage-basedPrevious workPrevious workImage-basedImage-based

Grabli et al. 2002

Fixed camera, moving light

3D from shading

3D strands

Duplication of hairstyle

Partial reconstruction (holes)

We build upon their approach. [Grabli02]

Captured geometry

Sample input image

1. Dense and reliable 2D data Robust image analysis

2. From 2D to 3D Reflection variation analysis

• Light moves, camera is fixed. Several light sweeps for all hair orientations

3. Complete hairstyle Above process from several viewpoints

Our approachOur approachOur approachOur approach

OutlineOutlineOutlineOutline

Previous work

• Definitions and overview

Details of the hair capture

Results

Conclusions

DefinitionsDefinitionsDefinitionsDefinitions

Fiber

StrandVisible entity

SegmentProject on 1 pixel

Orientation

~1mm

Setup & inputSetup & inputSetup & inputSetup & input

Input summaryInput summaryInput summaryInput summary

We use:

4 viewpoints

2 sweeps per viewpoint

50 to 100 images per sweep

Camera and light positions known

Hair region known (binary mask)

Camera

s

one b

y one

All cam

eras

togeth

er

Main stepsMain stepsMain stepsMain steps

1. Image analysis 2D orientation

2. Highlight analysis 3D orientation

3. Segment chaining 3D strands

Camera

s

one b

y one

All cam

eras

togeth

er

Main stepsMain stepsMain stepsMain steps

1. Image analysis 2D orientation

2. Highlight analysis 3D orientation

3. Segment chaining 3D strands

Measure of 2D orientationMeasure of 2D orientationDifficult pointsDifficult pointsMeasure of 2D orientationMeasure of 2D orientationDifficult pointsDifficult points

Fiber smaller than pixel aliasing

Complex light interaction Scattering, self-shadowing…

Varying image properties

Selectmeasure method

per pixel

Measure of 2D orientationMeasure of 2D orientationUseful informationUseful informationMeasure of 2D orientationMeasure of 2D orientationUseful informationUseful information

Many images available

……

Selectlight positionper pixel

Our approachOur approachOur approachOur approach

Based on oriented filters

Try several options Use the ``best’’

= argmax |K I|

0° 180°

response

Most reliable most discriminantLowest variance

90°

Filter selectionFilter selectionFilter selectionFilter selection

ImplementationImplementationImplementationImplementation

1. Pre-processing: Filter images

2. For each pixel, test:

Filter profiles

Filter parameters

Light positions

Pick option with lowest variance

3. Post-processing: Smooth orientations (bilateral filter)

2 4 8

8

Per pixel selectionPer pixel selectionPer pixel selectionPer pixel selection

4

Canny

2

Gabor

4

2nd Gauss.

2D results2D results2D results2D results

Sobel [Grabli02] Our result

(More results in the paper)

8 filter profiles3 filter parameters

9 light positions

All cam

eras

togeth

er

Camera

s

one b

y one

Main stepsMain stepsMain stepsMain steps

1. Image analysis 2D orientation

2. Highlight analysis 3D orientation

3. Segment chaining 3D strands

Mirror reflectionMirror reflectionComputing segment normalComputing segment normalMirror reflectionMirror reflectionComputing segment normalComputing segment normal

~3° accuracy [Marschner03]

n

For each pixel:Light position?

Practical measurePractical measurePractical measurePractical measure

Orientation from 2 planesOrientation from 2 planesOrientation from 2 planesOrientation from 2 planes

Intersection2 planes

3Dorientation

n

(3D position determined later)

All cam

eras

togeth

er

Camera

s

one b

y one

Main stepsMain stepsMain stepsMain steps

1. Image analysis 2D orientation

2. Highlight analysis 3D orientation

3. Segment chaining 3D strands

Starting point Starting point of a strandof a strandStarting point Starting point of a strandof a strand

Headapproximation

3D ellipsoid

…

Chaining the segmentsChaining the segmentsChaining the segmentsChaining the segments

?

Blending weightsBlending weightsBlending weightsBlending weights

Ending criterionEnding criterionEnding criterionEnding criterion

Strand grows until:

Limit length (user setting)

Out of volume (visual hull)

OutlineOutlineOutlineOutline

Previous work

Definitions and overview

Details of the hair capture

• Results

Conclusions

ResultsResultsResultsResults

Result summaryResult summaryResult summaryResult summary

Similar reflection patterns

Duplication of hairstyle

Curly, wavy and tangled

Blond, auburn and black

Middle length, long and short

ConclusionsConclusionsConclusionsConclusions

General contributions– Dense 2D orientation (filter selection)– 3D from highlights on hair

System– Proof of concept– Sufficient to validate the approach

Capture of a whole head of hair Different hair types

LimitationsLimitationsLimitationsLimitations

• Image-based approach: only visible part Occlusions not handled (curls)

• Head: poor approximation

• Setup: makes the subject moveDuring light sweepBetween viewpoints

Future workFuture workFuture workFuture work

• Better setup and better head approximation

Short term• Data structures for editing and animation• Reflectance

Long term• Hair motion capture• Extended study of filter selection

Thanks…Thanks…Questions ?Questions ?

Thanks…Thanks…Questions ?Questions ?

The authors thank Stéphane Grabli, Steven Marschner, Laure Heïgéas, Stéphane Guy, Marc Lapierre, John Hughes, and Gilles Debunne.

The images in the previous work appear by courtesy by NVIDIA, Tae-Yong Kim,

Wojciech Matusik, Nadia Magnenat-Thalmann, Hiroki Takahashi, and Stéphane Grabli.

Rendering usingdeep shadow mapskindly provided by Florence Bertails.

QuestionsQuestionsQuestionsQuestions

Visual hull Grazing angle

2D validation

Pre-processing Post-processing

Comparisons