Relief: A Modeling by Drawing Tool

David Bourguignon1 Raphaëlle Chaine2

Marie-Paule Cani3 George Drettakis4

1Princeton University / INRIA Rocquencourt 2LIRIS / CNRS / UCBL3GRAVIR / INP Grenoble 4REVES / INRIA Sophia-Antipolis

Outline

• Motivation• Previous Work• Tool Workflow• Reconstruction• Adaptive Sampling & Depth Inference• Tool Interface• Results

On Users

• Most people draw– Writing alternative

• Few people sculpt– Play-Doh days long gone– Materials difficult to handle

Goals

• Use 2D tools to perform 3D operations

Goals

• Use 2D tools to perform 3D operations• Model global and local surface

Goals

• Use 2D tools to perform 3D operations• Model global and local surface• Input: just plain strokes

Goals

• Use 2D tools to perform 3D operations• Model global and local surface• Input: just plain strokes• Output: triangle mesh

Outline

• Motivations• Previous Work• Tool Workflow• Reconstruction• Adaptive Sampling & Depth Inference• Tool Interface• Results

Previous Work

• Depth painting [Williams, 1990]

+

Previous Work

• Gradient editing [van Overveld, 1996]

Previous Work

• Maya 6.0 Artisan [Alias, 2004]

Outline

• Motivations• Previous Work• Tool Workflow• Reconstruction• Adaptive Sampling & Depth Inference• Tool Interface• Results

Tool Workflow

• First step: drawing input– Displacement map

• mid-grey = 0• white > 0• black < 0

Model of 3D sphere

Pencil

Brush

Tool Workflow

• First step: drawing– Displacement map– 2D shape boundary

(in green)• defines drawing mask

Tool Workflow

• First step: drawing– Displacement map– 2D shape boundary– Displacement regions (from 2 maps)

Tool Workflow

• Second step: modeling– Displace existing vertices

Tool Workflow

• Second step: modeling– Displace existing vertices– Create new surface patch

Tool Workflow

• Changing viewpoint

Modeling by drawing

Changing viewpoint

Reconstruction

• Based on evolving pseudo-manifold [Chaine, 2003]

Reconstruction

• Based on evolving pseudo-manifold [Chaine, 2003]

• Satisfy our requirements– Arbitrary number of connected components

Reconstruction

• Based on evolving pseudo-manifold [Chaine, 2003]

• Satisfy our requirements– Arbitrary number of connected components– Handle points off shape boundary

Reconstruction

• Based on evolving pseudo-manifold [Chaine, 2003]

• Satisfy our requirements– Arbitrary number of connected components– Handle points off shape boundary– Interactive (5k points per second)

2D reconstruction

• Start: pseudo-curve lies on oriented edges of Delaunay triangulation

2D reconstruction

• During: pseudo-curve evolves as long as oriented Gabriel criterion is not met

2D reconstruction

• Stop: topologically consistent set of oriented edges

Sampling and Depth

• Adaptive sampling– Displacement map

• Pencil and brush datain color buffer

Color buffer

Sampling and Depth

• Adaptive sampling– Displacement map– Approximate disp. map

sampled at existing vertices

Sampling and Depth

• Adaptive sampling– Displacement map (D)– Vertex-Sampled disp.

map (V)– Error map

E = 1 – ABS(D – V)– Arbitrary error value

Sampling and Depth

• Adaptive sampling– Displacement map– Approximate disp. map– Error map– Sampling [Alliez, 2002]

Sampling and Depth

• Adaptive sampling• Depth inference

– Identify surface vertices

Vertices ID buffer

Sampling and Depth

• Adaptive sampling• Depth inference

– Identify surface vertices– Assign depth values

Depth buffer

Sampling and Depth

• Adaptive sampling• Depth inference

– Identify surface vertices– Assign depth values– Infer depth values

• from existing surface• by depth propagation

Outline

• Motivations• Previous Work• Tool Workflow• Reconstruction• Adaptive Sampling & Depth Inference• Tool Interface• Results

Tool Interface

• Hole marks– Comic books production

Hole marks

Stone #3 (Avalon Studios)

Tool Interface

• Hole marks– Comic books production– Our system

Hole mark

Tool Interface

• Video: Basic interface

Tool Interface

• Blobbing

Drawing White shadingDistance field Height field Surface

Tool Interface

• Depth modes (chosen by menu)

Modeling “at depth”Depth inference Frisket mode

Video

• Modeling a tree

Paper sketch 3D model obtained with Relief

Outline

• Motivations• Previous Work• Tool Workflow• Reconstruction• Adaptive Sampling & Depth Inference• Tool Interface• Results

Results

• Models (1k to 4k points)

Discussion

• Intuitive shading convention

Discussion

• Intuitive shading convention• Problems with drawing metaphor

– No continuous visual feedback• Provide two modes

Discussion

• Intuitive shading convention• Problems with drawing metaphor

– No continuous visual feedback– Difficult to obtain continuous shading

• Provide higher-level drawing tools

Conclusion

• Modeling by drawing, but imprecise

Conclusion

• Modeling by drawing, but imprecise• Future work

– Speedup with local 3D reconstruction

Conclusion

• Modeling by drawing, but imprecise• Future work

– Speedup with local 3D reconstruction– Improve depth inference

Conclusion

• Modeling by drawing, but imprecise• Future work

– Speedup with local 3D reconstruction– Improve depth inference– Image-space and object-space sampling

AcknowledgementsThis work has been performed while the first author was a visiting research fellow at Princeton University, supported by an INRIA post-doctoral fellowship.

Many people have indirectly contributed to it. We would like to thank: Adam Finkelstein, Szymon Rusinkiewicz, Jason Lawrence, Pierre Alliez, Mariette Yvinec, Laurence Boissieux, Laure Heïgéas, Laks Raghupathi, Olivier Cuisenaire, Bingfeng Zhou.

Reconstruction

• Input: shape from strokes– 2D reconstruction– 3D reconstruction

Problems to be solved

• Provide appropriate drawing rep.– Shape from strokes

Tool Workflow

• Second step: modeling– Adaptive sampling

Tool Workflow

• Second step: modeling– Adaptive sampling– Depth inference

Our Approach

• Provide appropriate drawing rep.– Shape from strokes– Displacement from shading

• White/black metaphor for displacement

Rafaello Sanzio

Our Approach

• Provide appropriate drawing rep.• Provide appropriate surface rep.

– Free-form appearance

Our Approach

• Provide appropriate drawing rep.– 2D Shape from strokes

Our Approach

• Provide appropriate drawing rep.• Provide appropriate surface rep.

– Free-form appearance– Arbitrary topology changes

Outline

• Motivations• Previous Work• Contributions• Results

Contributions Overview

• Curve and surface reconstruction

Contributions Overview

• Curve and surface reconstruction• Adaptive sampling and depth inference

Contributions Overview

• Curve and surface reconstruction• Adaptive sampling and depth inference• Modeling by drawing interface

On Users

• Most people draw– Writing alternative

On Users

• Most people draw– Writing alternative– Minimal tool set

On Users

• Most people draw– Writing alternative– Minimal tool set– Since kindergarten

On Users

• Most people draw• Few people sculpt

– Play-Doh days long gone

Previous Work

• Wireframe reconstruction [Lipson, 1996]

Previous Work

• Gesture-based interfaces

[Zeleznik, 1996] [Igarashi, 1999]

Conclusion

• Modeling by drawing, but imprecise• Future work

– Speedup with local 3D reconstruction– Modeling alternative for “thin parts”

Current Alternative“Thin parts”