atomic volumes for mesh completion joshua podolak szymon rusinkiewicz princeton university

Atomic Volumes for Atomic Volumes for Mesh CompletionMesh Completion

Joshua PodolakJoshua Podolak

Szymon RusinkiewiczSzymon Rusinkiewicz

Princeton UniversityPrinceton University

OutlineOutline

Problem DescriptionProblem Description

BackgroundBackground

AlgorithmAlgorithm

ResultsResults

MotivationMotivation

Scanners usually need two unobstructed lines of sight Scanners usually need two unobstructed lines of sight between the cameras and the modelbetween the cameras and the model

Even with multiple scans, there are areas containing no Even with multiple scans, there are areas containing no depth information that need to be filleddepth information that need to be filled

Holes bounded by ring of half-edges.Holes bounded by ring of half-edges.

Filling HolesFilling Holes

Holes bounded by ring of half-edges.Holes bounded by ring of half-edges.

In simple cases, it is sufficient to create a patch by In simple cases, it is sufficient to create a patch by triangulating the half-edge loops around the holes.triangulating the half-edge loops around the holes.

Triangulate?Triangulate?

ChallengeChallenge

Triangulation is not enough.Triangulation is not enough.

Triangulation is not enough.Triangulation is not enough.

ChallengeChallenge

Talk OverviewTalk Overview

Problem DescriptionProblem Description

BackgroundBackground

AlgorithmAlgorithm

ResultsResults

Related WorkRelated Work

Point cloud reconstructionPoint cloud reconstruction[[Amenta01Amenta01, Kolluri04], Kolluri04]

Half-edge boundary triangulationHalf-edge boundary triangulation[Berg97, Liepa03][Berg97, Liepa03]

Implicit hole fillingImplicit hole filling[Curless96, Davis02, Masuda04, Ju04, Sharf04][Curless96, Davis02, Masuda04, Ju04, Sharf04]

Volumetric hole fillingVolumetric hole filling[Murali97][Murali97]

Related WorkRelated Work

Point cloud reconstructionPoint cloud reconstruction[Amenta01, Kolluri04][Amenta01, Kolluri04]

Half-edge boundary triangulationHalf-edge boundary triangulation[Berg97,[Berg97, Liepa03 Liepa03]]

Implicit hole fillingImplicit hole filling[Curless96, Davis02, Masuda04, Ju04, Sharf04][Curless96, Davis02, Masuda04, Ju04, Sharf04]

Volumetric hole fillingVolumetric hole filling[Murali97][Murali97]

Related WorkRelated Work

Point cloud reconstructionPoint cloud reconstruction[Amenta01, Kolluri04][Amenta01, Kolluri04]

Half-edge boundary triangulationHalf-edge boundary triangulation[Berg97, Liepa03][Berg97, Liepa03]

Implicit hole fillingImplicit hole filling[Curless96, [Curless96, Davis02Davis02, Masuda04, Ju04, Sharf04], Masuda04, Ju04, Sharf04]

Volumetric hole fillingVolumetric hole filling[Murali97][Murali97]

Related WorkRelated Work

Point cloud reconstructionPoint cloud reconstruction[Amenta01, Kolluri04][Amenta01, Kolluri04]

Half-edge boundary triangulationHalf-edge boundary triangulation[Berg97, Liepa03][Berg97, Liepa03]

Implicit hole fillingImplicit hole filling[Curless96, Davis02, Masuda04, Ju04, Sharf04][Curless96, Davis02, Masuda04, Ju04, Sharf04]

Volumetric hole fillingVolumetric hole filling[[Murali97Murali97]]

Talk OverviewTalk Overview

Problem DescriptionProblem Description

BackgroundBackground

AlgorithmAlgorithm

ResultsResults

Definition: Atomic VolumeDefinition: Atomic Volume

A volume is A volume is atomicatomic if it cannot be intersected by the if it cannot be intersected by the surface of the model.surface of the model.

An atomic volume must be either completely inside or An atomic volume must be either completely inside or completely outside the model.completely outside the model.

2 atomic volumes

Overview of ApproachOverview of Approach

1.1. Divide all of space into Divide all of space into atomic volumesatomic volumes. . Regions of space that will either be wholly in or wholly out of Regions of space that will either be wholly in or wholly out of

the reconstructed solid.the reconstructed solid.

2.2. For each atomic volume, decide whether it is inside For each atomic volume, decide whether it is inside the mesh or outside.the mesh or outside.

3.3. The boundary between interior volumes and exterior The boundary between interior volumes and exterior volumes is the new surface.volumes is the new surface.

Step 1: Spatial PartitioningStep 1: Spatial Partitioning

Step 1: Spatial PartitioningStep 1: Spatial Partitioning

Blank Region

Blank cubes do not contain any part of the input model.Blank cubes do not contain any part of the input model.

Blank cubes

Step 1: Spatial PartitioningStep 1: Spatial Partitioning

In Region

Out Region

Blank Region

InsideInside//OutsideOutside ( (IIOO): Cubes containing elements of the surface): Cubes containing elements of the surfaceaway from the holes. away from the holes.

IO cubes

Step 1: Spatial PartitioningStep 1: Spatial Partitioning

In Region

Out Region

Hole Region

Blank Region

Hole Hole cubes are subdivided until they can be cubes are subdivided until they can be trivially triangulated trivially triangulated [Mitchell92][Mitchell92] ..

Hole cubes

Step 2: Label AssignmentStep 2: Label Assignment

Most of the volumes can be labeled as inside or outside based on Most of the volumes can be labeled as inside or outside based on the normals of the input mesh.the normals of the input mesh.

Filling the hole requires labeling the remaining volumes as inside Filling the hole requires labeling the remaining volumes as inside and outside.and outside.

Outside

Inside

Step 2: Label AssignmentStep 2: Label Assignment

Atomic volumes correspond to nodes in the graph with edges Atomic volumes correspond to nodes in the graph with edges between neighboring volumes.between neighboring volumes.

Step 2: Label AssignmentStep 2: Label Assignment

Atomic volumes correspond to nodes in the graph with edges Atomic volumes correspond to nodes in the graph with edges between neighboring volumes.between neighboring volumes.

Atomic volumes on either side of a surface are not connected.Atomic volumes on either side of a surface are not connected.

Step 2: Label AssignmentStep 2: Label Assignment

Atomic volumes corresponds nodes in the graph Atomic volumes corresponds nodes in the graph with edges with edges between neighboring volumes.between neighboring volumes.

Atomic volumes on either side of a surface are not connected.Atomic volumes on either side of a surface are not connected.

The mesh graph can be split into two disjoint sub graphs.The mesh graph can be split into two disjoint sub graphs.

Mesh GraphMesh Graph

In Region

Out Region

Hole Region

Blank Region

Blank cube

Mesh GraphMesh Graph

I/O cube

In Region

Out Region

Hole Region

Blank Region

Mesh GraphMesh Graph

Hole cube

In Region

Out Region

Hole Region

Blank Region

In a watertight surface, there can be noIn a watertight surface, there can be no path path between inside and outside nodes.between inside and outside nodes.

Mesh GraphMesh Graph

In Region

Out Region

Hole Region

Blank Region

Step 2: Label AssignmentStep 2: Label Assignment

Splitting the graph into two sub graphs is equivalent to Splitting the graph into two sub graphs is equivalent to labeling each of the atomic volumes. labeling each of the atomic volumes.

Initially labeled atomic volumes must retain original Initially labeled atomic volumes must retain original labeling. labeling.

Minimum Cut Minimum Cut – Add source and sink nodes.Add source and sink nodes.– Constraint edges have a weight of infinity.Constraint edges have a weight of infinity.

Labeling VolumesLabeling Volumes

Sink

(outside)

Source

(inside)Constraint edges

Creating PatchCreating Patch

Every edge cut in the graph corresponds to a surface Every edge cut in the graph corresponds to a surface added to the model.added to the model.

In Region

Out Region

Hole Region

Blank Region

Topological ControlTopological Control

A graph allows adding additional constraints based on A graph allows adding additional constraints based on other sources of information such as Space Carving, other sources of information such as Space Carving, Shadow Carving, or direct user input.Shadow Carving, or direct user input.

Adding an edge with weight infinity between an atomic Adding an edge with weight infinity between an atomic volume to the sink node constrains that volume to be volume to the sink node constrains that volume to be outside the model.outside the model.

Atomic volumes containing points with contradicting Atomic volumes containing points with contradicting constraints may be subdivided.constraints may be subdivided.

Topological ControlTopological Control

In the figure on the right, points are added manually to In the figure on the right, points are added manually to indicate that the space between the two toes is outside.indicate that the space between the two toes is outside.

Initial mesh Unconstrained solution

Constrained solution

SmoothingSmoothing

Big atomic volumes cause faceting of the surfaceBig atomic volumes cause faceting of the surface..

We introduce a smoothing step to address faceting.We introduce a smoothing step to address faceting.

In Region

Out Region

Hole Region

Blank Region

SmoothingSmoothing

Big atomic volumes cause faceting of the surfaceBig atomic volumes cause faceting of the surface..

We introduce a smoothing step to address faceting.We introduce a smoothing step to address faceting.

SmoothingSmoothing

Big atomic volumes cause faceting of the surfaceBig atomic volumes cause faceting of the surface..

We introduce a smoothing step to address faceting.We introduce a smoothing step to address faceting.

Unsmoothed Smoothed

SmoothingSmoothing

Coarse smoothing will changes the labeling of entire Coarse smoothing will changes the labeling of entire atomic volumes.atomic volumes.

Fine smoothing keeps the topology of the mesh graph Fine smoothing keeps the topology of the mesh graph constant, but allows the boundary between adjacent constant, but allows the boundary between adjacent atomic volumes to change. atomic volumes to change.

Unsmoothed Smoothed

SmoothingSmoothing

Constrained Laplacian Smoothing.

We do not move vertices that cause the mesh graph to change.

Result after five iterations

Talk OverviewTalk Overview

Problem DescriptionProblem Description

BackgroundBackground– Related WorkRelated Work– Volumetric RepresentationVolumetric Representation

AlgorithmAlgorithm

ResultsResults

ResultsResults

Cross section of spiral torus.Half-torus filled with two different patches.

ResultsResults

Bunny: 70k faces. 78 sec.

Angel: 350k faces. 15 min.

ConclusionsConclusions

Atomic Volumes is a discrete volumetric method for Atomic Volumes is a discrete volumetric method for mesh completion.mesh completion.

The use of an adaptive subdivision of space allows the The use of an adaptive subdivision of space allows the algorithm to focus on target areas. algorithm to focus on target areas.

Using a mesh-graph provides flexibility in determining Using a mesh-graph provides flexibility in determining the patch.the patch.

Future WorkFuture Work

Soft constraints.Soft constraints.

Advanced smoothing.Advanced smoothing.– Flexible atomic volumes.Flexible atomic volumes.– Thin plate energy.Thin plate energy.– Texture Synthesis.Texture Synthesis.

Signal processing.Signal processing.– Localized filtering.Localized filtering.– Deformation.Deformation.

Thank youThank you

Mesh GraphMesh Graph

A mesh graph contains a node for each atomic volume.A mesh graph contains a node for each atomic volume.

An edge between neighboring atomic volumes represents the An edge between neighboring atomic volumes represents the strength of the belief that they will be labeled similarly.strength of the belief that they will be labeled similarly.