robust surface reconstructionvision.ucla.edu/papers/estellerss16surface.pdf · robust surface...

26
Copyright © by SIAM. Unauthorized reproduction of this article is prohibited. SIAM J. IMAGING SCIENCES c 2016 Society for Industrial and Applied Mathematics Vol. 9, No. 4, pp. 2073–2098 Robust Surface Reconstruction * V. Estellers , M. A. Scott , and S. Soatto Abstract. We propose a method to reconstruct surfaces from oriented point clouds corrupted by errors arising from range imaging sensors. The core of this technique is the formulation of the problem as a convex minimization that reconstructs the indicator function of the surface’s interior and substitutes the usual least-squares fidelity terms by Huber penalties to be robust to outliers, recover sharp corners, and avoid the shrinking bias of least-squares models. To achieve both flexibility and accuracy, we couple an implicit parametrization that reconstructs surfaces of unknown topology with adaptive discretizations that avoid the high memory and computational cost of volumetric representations. The hierarchical structure of the discretizations speeds minimization through multiresolution, while the proposed splitting algorithm minimizes nondifferentiable functionals and is easy to parallelize. In experiments, our model improves reconstruction from synthetic and real data, while the choice of discretization affects both the accuracy of the reconstruction and its computational cost. Key words. surface reconstruction, variational models, range imaging, RGB-D reconstruction, topology esti- mation, hierarchical splines, convex optimization AMS subject classifications. 65018, 65007, 65010, 65017 DOI. 10.1137/15M1048380 1. Introduction. New challenges to surface reconstruction from measurements emerge as datasets grow in size but lose accuracy. The reduction in accuracy appears as sensors evolve from short to long range, low-cost commodity scanners become widely available, and computer vision is increasingly used to infer 3D geometry from point sets. As a result, surface reconstruction methods must be robust to noise and outliers, and scale favorably in terms of computation and memory use. This impacts the choice of parametrization for the surface and the inference model that reconstructs a surface from an oriented point cloud. Point positions are subject to nonuniform sampling, scan misregistration, and gaps that appear as accessibility constraints leave regions devoid of data. The normal vectors that describe the orientation of the surface also suffer from noise and artifacts as they are estimated from the point positions or direct measurements. Normals estimated from the points are unreliable for thick and noisy point clouds or surfaces with touching sheets, while normals measured by scanning devices—such as photometric stereo—are corrupted by illumination artifacts. Noise in the oriented normals is especially detrimental because normals locally * Received by the editors November 16, 2015; accepted for publication (in revised form) September 27, 2016; published electronically December 8, 2016. http://www.siam.org/journals/siims/9-4/M104838.html Funding: The work of the first and third author was supported by the SNSF grant P300P2 161038. The work of the second author was supported by the Air Force Office of Scientific Research under contract FA9550-14-1-0113. The work of the third author was supported by the National Science Foundation (NSF), IIS-1422669. This support is gratefully acknowledged. Computer Science, UCLA, Los Angeles, CA 90095 ([email protected], [email protected]). Civil and Environmental Engineering, BYU, Provo, UT 84602 ([email protected]). 2073 Downloaded 10/21/18 to 128.97.159.65. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Upload: others

Post on 29-Feb-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

SIAM J. IMAGING SCIENCES c© 2016 Society for Industrial and Applied MathematicsVol. 9, No. 4, pp. 2073–2098

Robust Surface Reconstruction∗

V. Estellers† , M. A. Scott‡ , and S. Soatto†

Abstract. We propose a method to reconstruct surfaces from oriented point clouds corrupted by errors arisingfrom range imaging sensors. The core of this technique is the formulation of the problem as a convexminimization that reconstructs the indicator function of the surface’s interior and substitutes theusual least-squares fidelity terms by Huber penalties to be robust to outliers, recover sharp corners,and avoid the shrinking bias of least-squares models. To achieve both flexibility and accuracy, wecouple an implicit parametrization that reconstructs surfaces of unknown topology with adaptivediscretizations that avoid the high memory and computational cost of volumetric representations.The hierarchical structure of the discretizations speeds minimization through multiresolution, whilethe proposed splitting algorithm minimizes nondifferentiable functionals and is easy to parallelize.In experiments, our model improves reconstruction from synthetic and real data, while the choice ofdiscretization affects both the accuracy of the reconstruction and its computational cost.

Key words. surface reconstruction, variational models, range imaging, RGB-D reconstruction, topology esti-mation, hierarchical splines, convex optimization

AMS subject classifications. 65018, 65007, 65010, 65017

DOI. 10.1137/15M1048380

1. Introduction. New challenges to surface reconstruction from measurements emergeas datasets grow in size but lose accuracy. The reduction in accuracy appears as sensorsevolve from short to long range, low-cost commodity scanners become widely available, andcomputer vision is increasingly used to infer 3D geometry from point sets. As a result, surfacereconstruction methods must be robust to noise and outliers, and scale favorably in terms ofcomputation and memory use. This impacts the choice of parametrization for the surface andthe inference model that reconstructs a surface from an oriented point cloud.

Point positions are subject to nonuniform sampling, scan misregistration, and gaps thatappear as accessibility constraints leave regions devoid of data. The normal vectors thatdescribe the orientation of the surface also suffer from noise and artifacts as they are estimatedfrom the point positions or direct measurements. Normals estimated from the points areunreliable for thick and noisy point clouds or surfaces with touching sheets, while normalsmeasured by scanning devices—such as photometric stereo—are corrupted by illuminationartifacts. Noise in the oriented normals is especially detrimental because normals locally

∗Received by the editors November 16, 2015; accepted for publication (in revised form) September 27, 2016;published electronically December 8, 2016.

http://www.siam.org/journals/siims/9-4/M104838.htmlFunding: The work of the first and third author was supported by the SNSF grant P300P2 161038. The work

of the second author was supported by the Air Force Office of Scientific Research under contract FA9550-14-1-0113.The work of the third author was supported by the National Science Foundation (NSF), IIS-1422669. This supportis gratefully acknowledged.†Computer Science, UCLA, Los Angeles, CA 90095 ([email protected], [email protected]).‡Civil and Environmental Engineering, BYU, Provo, UT 84602 ([email protected]).

2073

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 2: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2074 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

define the surface to first order and identify its topology. While correcting the normals in anoriented point cloud is relatively easy, correcting the topology of the reconstructed surface isexpensive and delicate. For this reason, it is critical that reconstruction methods estimatetopology correctly.

We propose a robust algorithm to reconstruct a watertight surface from an oriented pointcloud. We formulate the reconstruction as a convex optimization that recovers the indicatorfunction of the interior of the surface. Our objective function penalizes deviations in theorientation and location of the surface with a robust Huber loss function to estimate thetopology of the surface and allow for sharp corners; this makes our model robust to outliersand avoids the shrinking bias [53] of least-squares models [30, 31]. This is our first contribution,which is explained in section 3.

Our second contribution is an efficient algorithm that exploits the convexity of the objec-tive function to derive a first-order minimization algorithm that scales well with the size ofthe point cloud and is easy to parallelize. To this end, in section 5 we adapt the primal-dualalgorithm of [13] to our objective and derive closed-form solutions for each proximal update.

A final contribution is the investigation of three different discretizations to solve the mini-mization problem described in section 4. In each case, the solution is parametrized by a linearcombination of basis functions. The basis functions are all scaled and translated versionsof a piecewise polynomial and create a hierarchical space that adapts the resolution of therepresentation to the resolution of the point cloud. The three discretizations utilize differentpolynomial orders and hierarchical constructions: a quadratic hierarchical B-spline basis, alinear hierarchical B-spline basis equivalent to the discretization of [11], and a simpler dictio-nary of quadratic B-splines that do not form a basis and are equivalent to the discretizationof [30, 31]. Usually, reconstruction methods are evaluated as a whole, making it difficult todetermine whether improvements in accuracy or speed are due to the model, the algorithm,or the discretization. Investigating different discretizations is necessary to disambiguate theaccuracy of state-of-the-art variational models from the accuracy of their discretizations andto decouple the computational cost of the optimization algorithms from the computationalcomplexity of the discretizations.

Compared to our conference paper [22], we have improved the model and the discretizationin three ways. First, the new model does not require normal interpolation and is able toimprove accuracy because it leads to better fits of the normal samples. Second, the new modelincorporates a Huber penalty in the screening term, instead of the least-squares penalty of[22], to be robust to outliers in the point locations; this affects the optimization problembecause the derivation of closed-form solutions now requires an additional dual variable thatincreases the memory and computational costs of the reconstruction and makes the choice ofan efficient discretization more critical. Third, our new discretization is less memory intensiveand computationally lighter, and speeds up reconstruction by an order of magnitude.

2. Related methods: Choice of representation. Surface reconstruction methods areclassified into parametric, implicit, and techniques based on computational geometry. Ap-proaches based on computational geometry partition space into Voronoi cells from the inputsamples and exploit the idea that eliminating facets of Delaunay tetrahedra provides a trian-gulated parametrization of the surface [3, 18, 5, 43, 10, 50, 32] and describes a combinatorialD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 3: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2075

problem solved by local analysis of the cells [5, 43, 10], eigenvector decomposition [50], orgraph cuts [32]. The reconstructed surface thus interpolates most of the input samples andrequires postprocessing to smooth the surface and correct the topology. Parametric techniquesassume the topology of the surface to be known and represent the surface as a topologicalembedding of a 2D parameter domain into 3D space; this requires dedicated representations[26] to efficiently parametrize complex topologies with continuous embeddings and knowledgeof the surface topology. Assuming the topology to be known is a major limitation for generalsurface reconstruction, and parametric models [4] are usually confined to applications withknown topology [19, 20, 51, 26], while implicit methods dominate the reconstruction of surfaceswith unknown topology. We thus adopt an implicit representation that approximates—notinterpolates—the point samples to reconstruct surfaces of arbitrary topology from the largeand corrupted point clouds that motivate our work. An overview of surface reconstructiontechniques when higher quality point clouds or surface priors are available is provided in [8].

Implicit representations both reconstruct the surface and estimate its topology, but in-crease the dimension of the problem by representing the surface as the zero-level set of avolumetric function. This increases their computational cost and calls for discretizations withnonuniform grids [30, 39, 11, 31] to reconstruct surfaces at high resolution and process pointclouds nonuniformly distributed over the volume.

Implicit representations can be global or local. Local methods consider subsets of nearbypoints one at a time and handle large datasets efficiently. Earlier methods [29, 16] estimatetangent planes from the nearest neighbors of each sample and parametrize the surface bythe signed distance to the tangent plane of the closest point. Moving least-squares (MLS)techniques [2, 34, 49, 6] reconstruct surfaces locally by solving an optimization that findsa local reference plane and then fits a polynomial to the surface. The least-squares fit ofMLS, however, is sensitive to outliers and smooths out small features; for this reason variantsrobust to outliers [42, 24] and sharp features [37, 17] appeared. In [41] the authors constructimplicit functions locally but blend them together with partitions of unity. Common to thesemethods is their locality (i.e., partitioning into neighborhoods and merging local functions),which makes them highly scalable but sensitive to nonuniform sampling and point-cloud gaps.

Global methods define the implicit function as the sum of compactly supported basisfunctions (e.g., truncated radial basis functions [12], splines [30, 31], wavelets [39]) and considerall the data at once without heuristic partitioning. Kazhdan et al. [30, 31] solve a Poissonproblem that aligns the gradient of the indicator function to the normals of the point cloudwith a least-squares fit not robust to outliers. Manson, Petova, and Schaefer [39] similarlyapproximate the indicator function with wavelets designed to compute basis coefficients withlocal sums over an octree. Calakli and Taubin [11] use a signed distance function to representthe surface, but also rely on least squares to fit the normals and include a screening term thatimproves accuracy by fitting the input points to the zero-level set of the implicit function. Forthis reason, our model includes a screening term together with a robust Huber penalty to fit thenormals and allow for sharp edges. Existing methods account for sharp features by explicitrepresentations [1, 27, 37] or anisotropic smoothing [15, 52, 14]; they are fast but dependon local operators that do not seek a global optimum. The anisotropic fairing techniques[28, 33] are designed, as is our model, to preserve sharp corners and to avoid shrinking of thinstructures, but rely on different representations of the surface ill-suited to noisy point cloudsD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 4: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2076 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

with gaps and scanning artifacts. The anisotropic denoising of [28] smoothes a simplicialsurface while preserving sharp edges and thin structures, but requires an initial input surfacewith the correct topology that is not available from our point cloud. In [33] the authorstransfer the surface denoising of [28] to point sets to allow changes in the surface topology,but a large number of point samples is required to accurately represent a surface without holeswhen scanning artifacts or missing data corrupt the point cloud. This limitation is overcome inour level-set representation by the continuity of the level-set function with watertight surfacereconstructions.

Our reconstruction combines benefits of global and local schemes. It is global in the sensethat it does not involve heuristics on neighborhoods, while it preserves locality by requiring thebasis functions to be locally supported and adapt to the input point cloud, as in [30, 31, 39].Our discretizations adapt to the input point cloud by means of an octree and are motivatedby the octree representations of [11] and [30, 31], while the use of a hierarchical B-spline [54]is inspired by the spline forests [46] of isogeometric analysis.

3. Variational model. The reconstruction of a surface S from oriented points can be castas a minimization problem that estimates the indicator function χ of the interior of the surface.Let (xk,nk)nP

k=1 be the oriented point cloud, with xk ∈ R3 the point location and nk ∈ S2

its associated normal; our goal is to estimate a continuous function χ : Ω ⊂ R3 → R such thatS=x : χ(x)=0, and χ is negative in the interior enclosed by S and positive outside.

The key to most variational formulations [30, 31, 11, 39] is to observe that each orientedpoint (xk,nk) is a sample of the gradient of the indicator function, that is, ∇χ(xk) = nk. As aresult, we can reconstruct S by finding the scalar function whose gradient best matches thesesamples. To account for noise in the data, we formulate the reconstruction as a minimizationproblem, as opposed to an interpolation problem:

minχ

α

nP∑k=1

g(χ(xk)) + β

nP∑k=1

f(nk −∇χ(xk)),(3.1)

where α, β are positive model parameters and f and g are the Huber loss functions in R andR3.

g(u) =

1

2εx|u|2 |u| < εx

|u| − εx2|u| ≥ εx

, f(v) =

1

2εn|v|2 |v| < εn

|v| − εn2|v| ≥ εn

,(3.2)

where | · | is the Euclidean norm and εx, εn ≥ 0 are the model parameters. The Huber lossfunctions are convex and differentiable penalties that avoid two artifacts of the least-squaresmodels [30, 11, 31] that overpenalize outliers: shrinkage of thin structures and smoothing ofsharp edges. They overcome these limitations by using different penalties for small errorsand outliers but result in a minimization that is harder to solve than the linear systems ofleast-squares fits.

The first term in (3.1) sets the points as soft interpolation constraints and fixes the surfaceparametrization to the zero-level set of χ; it is a generalization of the screening term of [40]but defined over a sparse set of points as in [11] and with a robust Huber penalty. The secondD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 5: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2077

term penalizes errors in the sampled normals with a Huber loss function that makes our modelrobust to noise in the normals; this is particularly important in reconstruction because errorsin the normals cause errors in the surface topology that are difficult and expensive to correctonce the surface is extracted.

The minimization problem (3.1) is undetermined for reconstruction because the modelonly constrains the gradient of χ close to the sampled points, but lets χ vary freely away fromthem. From all the functions whose gradients approximate the samples, we are interestedin those that lead to implicit representations of the surface and only evaluate to zero on it.We achieve this with a regularizer that ensures that χ does not evaluate to zero far from thesamples and produce spurious surface sheets. In level-set parametrizations, this is usuallyaccomplished by enforcing an approximate distance function with constant ∇χ far from thesurface [35, 23]; in our formulation we adopt a simpler formulation with a regularizer thatcontrols the surface behavior far from the samples by penalizing changes in ∇χ with a penaltyon its Hessian Hχ. The reconstruction model reads

minχ

α

nP∑k=1

g(χ(xk)) + β

nP∑k=1

f(nk −∇χ(xk)) +1

2

∫Ωw|Hχ|F ,(3.3)

where | · |F is the Frobenius norm, and the weighting function w is a binary mask of value 0in a neighborhood of the samples and value 1 far from them, to ensure that the data termsdefine the cost function close to the input samples while the regularization term dominatesthe objective far from them. The size of the neighborhood at each point is determined by thesize of the discretization cell that contains that point and, as a result, the binary mask is onlyset to zero for grid cells that contain at least a point. This adapts the binary mask to thedensity of the point cloud and ensures that the regularizer is always active when there are nopoint samples to solve the indetermination associated with the lack of data terms over thatcell.

Using the Frobenius norm of the Hessian provides a better approximation to a signeddistance function because it leads to an L1 optimization problem whose minimizers are char-acterized by a large set of zero residuals and a small set of nonzero ones; that is, the solutionsatisfies |Hχ| = 0 over most of its domain, and ∇χ and its gradient are constant almosteverywhere. The squared Frobenius norm proposed by [11], on the other hand, leads to an L2

optimization problem whose minimizers are characterized by a large number of small residualswhere |Hχ| > 0 and where ∇χ varies smoothly over the function domain. Since a signed dis-tance function is characterized by a constant |∇χ| almost everywhere (Eikonal equation), theL1 penalty offers a better approximation because its minimizers satisfy the constant-gradientassumption over most of its domain. Moreover, when two opposite surface sheets touch, theproposed L1 penalty reconstructs piecewise normal that correctly represent the surface topol-ogy, while an L2 Hessian penalty averages normal samples from opposite sheets and mergesthem.

Our method is related to the Poisson reconstruction of [31] and the smooth signed distance(SSD) of [11], but our model, representation of the surface, and minimization techniquesare different. First, in terms of the model, we propose a robust Huber penalty on boththe normals and the screening term to be resilient to outliers, instead of the least-squaresD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 6: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2078 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

penalty of [31] and [11]. Compared to [31] and our conference paper [22], we also avoid theinterpolation of the sampled normals into a field by the inclusion of the Hessian regularizer;this requires the computation of additional second-order derivatives but avoids preprocessingthe normal samples, and in general leads to more accurate reconstruction of corners, wherenormal interpolation smears sharp changes in orientation. Compared to [11], our regularizeruses the Frobenius norm of the Hessian, instead of its square, to better approximate a signeddistance function. Second, we adopt multiple discretizations to represent χ to investigatethe effects of the discretization on the quality and speed of the reconstruction. Finally, ourminimization exploits the convexity of (3.3) to develop an efficient primal-dual algorithm,instead of classic finite-element methods that cannot handle the Huber loss functions.

4. Discretization. While variational models in image processing are solved with the voxellattice as a computational grid, the choice of discretizations is still an open question for surfacereconstruction because the input data—point clouds—are not uniformly distributed and thecomputational cost of volumetric grids grows cubicly with resolution. As a result, differentsolutions have been proposed, but there is no consensus on a single approach. Our experimentsare a first modest step in answering this question.

Determining the role of the discretization in the reconstruction performance also answers amodeling question that guides our design choice: How does the substitution of the continuousnormal interpolation term

∫Ω g(n−∇χ) by the discrete

∑nPk=1 g(nk−∇χ) affect the speed and

accuracy of the method? While it is possible to consider the Poisson reconstruction of [30, 31]as an example of a continuous least-squares normal penalty and the SSD reconstruction of[11] as an example of its discrete counterpart, a direct comparison of their performance isinconclusive in the model because these reconstruction methods are implemented with dis-cretizations of different accuracy and speed. The only way to answer such questions is toimplement the different models with the same discretization and only then compare the mod-els. This raises the question of which discretization to choose and is the motivation for thissection.

4.1. Hierarchical B-splines. An efficient discretization of (3.3) should exploit the fact thatits solution only needs to be accurate near the zero isolevel that parametrizes the surface, thatis, in the neighborhood of the point samples. This calls for representations over irregular gridswith high resolution around the surface and coarser resolutions far from it. To this end, wediscretize the space of functions over Ω with the finite-dimensional vector space spanned bythe basis functions φ1, . . . , φn with higher spatial resolution near the point samples.

In the following, a function in Ω is discretized by the linear combination

χ(x) =∑

A

cAφA(x),(4.1)

where A is the finite index that sorts the basis functions, and c = (c1, . . . , cn) are the coeffi-cients that represent χ and the variables in the optimization.

The accuracy of the representation is determined by the shape and resolution of the basisfunctions φA, while its computational cost is determined by their evaluation. These twocriteria guide our choice of discretization: first, the basis functions must have derivatives upto second order that are integrable, either analytically or well approximated by quadratureD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 7: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2079

Figure 1. Three-level quadratic HBS space in one dimension. Basis functions are plotted in solid coloredlines, functions linearly dependent on higher-level functions in dashed colored lines, and inactive functions ingrey dotted lines.

rules; second, they must be compactly supported so that only a few functions are nonzero ateach point and the linear combinations that define χ,∇χ,Hχ can be evaluated quickly. Toaccomplish this, we adopt a hierarchical B-spline (HBS) representation and investigate theimpact of different polynomial degrees and smoothness with three discretizations:

1. a basis of quadratics HBS (denoted HB2S basis),2. a hierarchical dictionary of quadratic B-splines (HB2S dictionary),3. a linear HBS basis that approximates derivatives with finite differences (HB1S basis).

Hierarchical B-splines [25] are constructed through tensor products of splines over a hier-archy of uniform grids of increasing spatial resolution; see Figure 1. The construction has twosteps: first, uniform B-spline functions of degree p at each level define a set of functions withvarying spatial resolution; second, linear dependencies between levels are eliminated to definea basis with the desired resolution at each point. Hierarchical bases are a natural choice forour parametrization because they are compact and efficient and adapt their resolution.

B-spline spaces. At each level of the hierarchy, a trivariate tensor-product B-spline spaceis defined by specifying the polynomial degree p and a knot vector that partitions the domainΩ into a regular hexahedral grid. For a unit volume Ω = [0, 1]3, the space is spanned by thetensor-product B-splines

Npi,j,k(x, y, z) = Np

i (x) Npj (y) Np

k (z), 0 ≤ i, j, k ≤ m,(4.2)

where the univariate B-spline functions are parametrized by knot vectors [ξ0, ξ1, . . . , ξm] withξ0 = 0, ξi+1 − ξi = s, and ξm = 1 in each dimension, and defined recursively as

N0i (x) =

1 ξi ≤ x < ξi+1,

0 otherwise,Npi (x) =

x− ξiξi+p − ξi

Np−1i (x) +

ξi+p+1 − xξi+p+1 − ξi+1

Np−1i (x),(4.3)

where fractions with zero denominator evaluate to zero. The B-spline space at each level isthus parametrized by the degree p of the polynomial and the scale parameter s.D

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 8: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2080 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

From the definition, we directly see that each B-spline function Npi,j,k is positive, with

continuous p− 1-derivatives at the knots, and has compact support suppNpi,j,k = [ξi, ξi+p+1]×

[ξj , ξj+p+1]× [ξk, ξk+p+1]. The set N ps = Np

i,j,k is linearly independent and forms a basis forthe space of piecewise polynomials of degree p with uniform knot intervals of scale s.

The tensor-product structure of multivariate B-spline spaces limits their use to small andmedium problems because defining a high-resolution basis function centered at (x0, y0, z0)leads to the definition of high-resolution basis functions along the three axes (x, y0, z0) ∀x,(x0, y, z0) ∀y, and (x0, y0, z) ∀z. This increases their computational cost and motivates theuse of hierarchical splines.

Hierarchical B-spline spaces. In a hierarchical B-spline space, rectangular B-spline patchesat different hierarchical levels are glued together to locally refine the basis functions withoutspreading the refinement along the axes by tensor product.

A basis Bp of a hierarchical B-spline space is constructed recursively from a set of nestedvolumetric patches Ω = Ω0 ⊇ Ω1 ⊇ · · · ⊇ ΩL of increasing spatial resolution, as follows:

1. initialization: Bp0 = N p1 ;

2. recursion: Bpl = φ ∈ Bpl−1 : suppφ * Ωl ∪ φ ∈ N p2−l : suppφ ⊆ Ωl;

3. at level L, Bp = BpL defines the basis functions φA in our discretization.We initialize Bp0 = N p

1 and at each recursive step l we replace the basis functions whose supportis entirely contained in the refined patch Ωl by the refined basis functions in N p

2−l over thispatch. With this construction, the basis functions of each Bp are linearly independent andform a basis for spanBp with nested spaces spanBpl ⊆ spanBpl+1 [54].

4.2. Discretization 1: Quadratic HBS basis (HB2S basis). Our first discretization isa quadratic hierarchical B-spline space where the basis functions φA = B2 define a basis.We choose p = 2 because it provides the twice-differentiable basis with minimal support andallows us to have an analytical expression for the Hessian. This is important because it allowsus to use interpolation to evaluate χ, ∇χ, and Hχ in our objective functional.

In particular, the linearity of the discretization lets us efficiently compute the value of χat any point x in the domain by evaluating the basis functions at this point. In other words,

χ(x) =∑

A

cAφA(x) = [φ1(x), . . . , φn(x)] c.(4.4)

The evaluation of this sum is efficient for two reasons: first, the number of basis functionsthat are active at each point is logarithmic in the basis resolution and the sum mostly involvesonly a few terms and, second, there are efficient algorithms to evaluate the basis functionssuch as the Cox–De Boor algorithm. Similarly, we compute

∇χ(x) =∑

A

cA∇φA(x) = [∇φ1(x), . . . ,∇φn(x)] c,(4.5)

Hχ(x) =∑

A

cAHφA(x) = [Hφ1(x), . . . ,Hφn(x)] c.(4.6)

Finally, we discretize the integral in our objective functional (3.3) as the weighted sumDow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 9: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2081

∫R3

w(x)|Hχ(x)|Fdx ≈nQ∑i=1

ρiw(qi)|Hχ(qi)|F(4.7)

with Gauss quadrature rules, where qi is a quadrature point where we evaluate the HessianHχ and weighting function w, and ρi is its associated quadrature weight. For efficiency, weuse the midpoint quadrature with a single quadrature point at the middle of each cell and theweight given by its volume.

4.3. Discretization 2: Linear HBS basis (HB1S basis). Our second discretization isa linear hierarchical B-spline space where the basis functions φA = B1 define a basis.Choosing p = 1 reduces the support of the basis functions and speeds up the evaluation ofB-spline functions and their derivatives, but reduces the smoothness of the reconstruction.

With p = 1, each basis is centered at a vertex of a hexahedral grid and supported in thecells that contain that vertex. As a result, the value of a B-spline function inside each cellis obtained by trilinear interpolation from its values at the cell corners. We combine thisfinite-element discretization of χ with a finite-difference approximation of its gradient andHessian to obtain a discretization comparable to the mixed finite-element and finite-differencediscretizations proposed for surface reconstruction by [11]. In particular, we discretize thegradient with a piecewise constant basis that is discontinuous at the cell boundaries andconstant within each cell, with the gradient computed by averaging the finite differences ofthe function at cell corners as follows:

∇χ(xk) =

0.25∆x

A0 0

0 0.25

∆yA

0

0 0 0.25∆z

A

−1 −1 −1 −1 1 1 1 1−1 −1 1 1 −1 −1 1 1−1 1 −1 1 −1 1 −1 1

χA0

· · ·χA7

,(4.8)

where CA is the ∆xA×∆y

A×∆zA cell containing xk and χA0 , . . . , χA7 is the value of χ at its

vertices.The discretization of the Hessian consistent with the discrete gradient is then a distribution

supported on the shared cell faces. To compute the regularizer, we then substitute the integralover Ω by a sum over interior cell faces as follows:∫

Ω|Hχ|2F =

∑(A,B)

wAB|HAB χ|2F =∑

(A,B)

wAB∆AB

|∇Aχ−∇Bχ|2F ,(4.9)

where ∆AB is the Euclidean distance between the centers of the cells, ∇Aχ is the constantvalue of the discrete gradient inside CA, and wAB is the area of the face shared by CA andCB. This naturally leads to a staggered grid where the function is discretized by its value atcell corners (that define the trilinear interpolation), its gradient by its value at cell centers,and its Hessian by its value at cell boundaries. Staggered grids are common in finite-elementand finite-volume discretizations in computational sciences; our particular choice is inspiredby the surface reconstruction technique of [11].

The HB1S basis offers an additional advantage for surface reconstruction because it leadsto piecewise linear implicit functions that are well suited to the final marching cubes algorithm[38], which is designed for iso-surfacing piecewise linear implicit functions and can generateringing artifacts with more complex functions.D

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 10: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2082 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

4.4. Discretization 3: Dictionary of quadratic HBS (HB2S dictionary). Our third dis-cretization is equivalent to the hierarchical spline space of [30, 31]. This representation isdesigned to be as light as possible while providing smooth second derivatives and the samespatial resolution as the input point cloud. To this end, the hierarchical structure of the spaceis defined by the octree of the input samples, and a quadratic B-spline basis is created at thecenter of each leaf cell A with its scale set to match the width of the cell

φA(x) = N2

(x− xA

sA

),(4.10)

where φ is the unit trivariate quadratic B-spline basis with support [−1.5, 1.5]3, xA is thecenter of the cell, and sA is its width. As a result, the basis functions are again translatedand scaled versions of a quadratic uniform B-spline and are hierarchically organized by theoctree containing the input samples. Unlike in a quadratic HBS space, each basis function isdefined independently of the others with no guarantees of linear independence.

Dropping the linear independence condition allows replacing basis functions at coarserlevels more aggressively because there is no constraint on the size of the patch to be refined.Indeed, in the hierarchical construction of section 4.1, only basis functions whose supportis completely contained in the finer patch can be refined to ensure linear independence overthis patch [25]. As a result, the size of the refined patch must exceed the support of thebasis functions or, equivalently, any region needing refinement smaller than the support ofthe active basis must be expanded to ensure linear independence. This condition is morerestrictive with quadratic than linear splines because their support is larger, motivating thedefinition of a quadratic B-spline dictionary adapted to the surface as [30, 31]. In particular,with the hierarchical structure of the octree and the quadratic spline (4.10), basis functionscentered at a finer patch are refined even when their support exceeds the patch. This allows usto refine smaller patches than the quadratic HBS basis and reduce the size of the hierarchicalrepresentation at the cost of representation.

Although the functions do not need to form a basis to solve the optimization problem onelevel at a time, the vectorial spaces spanned by the dictionary functions at each level must benested B0 ⊂ B1 ⊂ · · · ⊂ BL in order to inject the solution from the coarser space to the finerone. This is not generally satisfied with the proposed dictionary; nevertheless, it is possibleto approximately inject the solution at depth l into the solution at depth l + 1 by assigningthe coefficient values of the parent node to each basis function associated with a new leaf, anduse this approximate solution to initialize the optimization at a finer scale without affectingthe convergence of the algorithm to the global optimum by convexity.

4.5. Hierarchical B-spline spaces over octrees. We use octrees to define our computa-tional grid and the hierarchy of B-spline spaces. To this end we construct the minimal octreeof depth L that assigns each input sample to a leaf of the octree and use the parent-childrelation of the octree cells Ωchild ⊂ Ωparent to define the nesting relation of the volumetricpatches Ω ⊇ Ω1 ⊇ · · · ⊇ ΩL of section 4.1.

For the HB1S space at level l, there is a first-order basis function associated with each leafin the minimal octree of depth l that stores the samples. As we obtain the octree of depth l+1by expanding the tree at depth l with the children of nonempty leaves of depth l, we defineD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 11: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2083

new basis functions associated with these children and remove the basis function associatedwith their parent to define the HB1S at level l + 1 from the HB1S at level l.

The HB2S dictionary is directly defined by the octree and its refinement procedure isdirectly performed by analyzing the tree at different depths. In particular, the HB2S dictionaryat level l is defined by the quadratic B-spline basis centered at octree nodes of level l or octreeleaves at levels lower than l; that is, by the quadratic B-spline basis centered at all the leavesof an octree of depth l containing the point cloud.

Partitioning the volume with the cells of the octree containing the point cloud naturallyleads to a finer resolution near the surface and provides a mesh with efficient procedures forlocating point samples and finding neighboring cells.

5. Minimization algorithm. In our discretizations, the integral in (3.3) becomes the sum∫R3

w(x)|Hχ(x)|Fdx =

nQ∑i=1

wi|Hχ(qi)|F .(5.1)

With quadratic basis functions, this sum is the result of applying standard quadrature rules,where wi = ρiw(qi) is the product of the quadrature weight ρi and the value of the weightingfunction evaluated at the quadrature point w(qi). In the case of the linear HBS, the sumresults from the definition of the Hessian as a Dirac distribution supported on the grid faces,and qi, wqi are the position and area of the shared grid faces.

By restricting χ to the span of functions φAnA=1, we confine the minimization to thecoefficients c ∈ Rn:

minc

α

nP∑k=1

g(χ(xk)) + β

nP∑k=1

f(∇χ(xk)− nk) +1

2

nQ∑i=1

wi|Hχ(qi)|F ,(5.2)

s.t. χ(xk) =∑

A

cAφ(xk), 1 ≤ k ≤ nP ,(5.3)

∇χ(xk) =∑

A

cA∇φ(xk), 1 ≤ k ≤ nP ,(5.4)

Hχ(qi) =∑

A

cAHφ(qi), 1 ≤ i ≤ nQ.(5.5)

In this form, it is clear how changing the basis functions changes the constraints in theoptimization problem but preserves the objective functional. Different spline spaces will thusresult in different solutions with different data and regularization fits.

The constraints (5.3) can be written more compactly as χ(xk) = Pk c by defining arow vector Pk = [φ1(xk) . . . φn(xk)] for each sample xk and stacking them into a sparsematrix P ∈ RnP×n with Pij = φj(xi). Similarly, we can write the linear constraints on thegradient and Hessian (5.4)–(5.5) as multiplications with sparse matrices N ∈ R3nP×n andQ ∈ RmnQ×n by stacking the components of each ∇χ(xk) and Hχ(qi) into column vectorswith block components Nkj = ∇φj(xk) ∈ R3 and Qij = Hφj(qi) ∈ Rm.1 The minimizationproblem (5.2) then simplifies into the constrained minimization

1m = 9 for the spline discretizations, while m = 3 for the linear HBS discretization.Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 12: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2084 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

minc

nP∑k=1

αg(uk)︸ ︷︷ ︸G(u)

+

nP∑k=1

βf(Vk − n(xk))︸ ︷︷ ︸F (V )

+1

2

nQ∑i=1

wi|Wi|F︸ ︷︷ ︸H(W )

s.t.

u = Pc,V = Nc,W = Qc,

(5.6)

minc

G(Pc) + F (Nc) +H(Qc).(5.7)

The problem now has the standard form of many convex minimization problems that are solvedwith splitting techniques. Among them, we adopt a primal-dual formulation and rewrite (5.7)as the saddle-point problem

maxν, λ, µ

minc− F ∗(λ)−G∗(ν)−H∗(W ) + 〈ν, Pc〉+ 〈λ,Nc〉+ 〈µ,Qc〉,(5.8)

where the dual variables ν, λ, µ are associated with the constraints u = Pc, V = Nc, W = Qc,and ∗ denotes the convex conjugate.

The primal-dual algorithm [13] solves (5.8) efficiently by exploiting the structure of theproblem —the sum of convex functions that are easy to minimize independently—and solvingthe following sequence of proximal problems with speed parameters σ, τ > 0:

νn+1 ← minν

σG∗(ν) +1

2‖ν − νn+1 − σP cn‖2,(5.9)

λn+1 ← minλ

σF ∗(λ) +1

2‖λ− λn − σNcn‖2,(5.10)

µn+1 ← minµ

σH∗(µ) +1

2‖µ− µn − σQcn‖2,(5.11)

cn+1 = cn − τ(P ∗νn+1 +N∗λn+1 +Q∗µn+1),

cn = 2cn − cn−1.

We choose a first-order method because the size of the problem makes second-order methodsinfeasible. The efficiency of the proposed algorithm comes from the spatial separability ofF ∗, G∗, H∗ and from the ability to find closed-form solutions for minimization problems (5.9)–(5.11). The derivation of closed-form solutions is detailed next, and Algorithm 1 summarizesthe resulting updates, which are easy to parallelize.

5.1. Minimization in ν. Let ν = ζnν −σQcn. We solve the minimization in ν (5.9) throughMoreau’s identity [44]:

ν ← minν

σG∗(ν) +1

2‖ν − ν‖2 ⇐⇒ ν = ν − σu∗, u∗ ← min

uG(u) +

σ

2‖u− ν

σ‖2.

The minimization in u is decoupled in each ui ∈ R with a single term in the sum

minu

nP∑k=1

αg(uk) + 0.5σ(uk − σ−1νk)2.(5.12)

The minimization is thus solved by independently minimizing each term in the correspondinguk. Due to the convexity and differentiability of the Huber loss g, the optimality conditionsare obtained by differentiating the objective function with respect to uk.D

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 13: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2085

Algorithm 1. Primal-dual minimization algorithm.

Initialize variables to zero. Set τ, σ > 0 according to [13].while ‖cn+1 − cn‖ > δ do

νn+1k = αmin

(1,|νk|

α+ εxσ

)sign(νk), ν = ζnν + σP

(2cn − cn−1

),

λn+1k = β

λkmax(β + σεn, |λk|2)

, λ = ζnλ + σ(

2Ncn −Ncn−1 − nk),

µn+1i =

γwimax(γwi, |µ|2)

µ, µ = ζnµ + σQ(

2cn − cn−1),

cn+1 = cn − τ(P ∗νn+1 +N∗λn+1 +Q∗µn+1

).

end

If |u∗k|1 ≤ εx, the optimality conditions are

α

εxu∗k + σu∗k = νk ⇐⇒ u∗k =

εxα+ εxσ

νk(5.13)

and the condition |u∗k|1 ≤ εx becomes |νk|1 ≤ α+ εxσ.If |u∗k|1 > εx, the optimality conditions

α sign(u∗k) + σu∗k = νk with |u∗k|1 > εx(5.14)

are solved by

u∗k =

σ−1(νk − α), νk > α+ σεx,

σ−1(νk + α), νk < −α− σεx.(5.15)

Combining these two cases, the dual variable νn+1k = νk − σu∗k is updated by

νk = αmin

(1,|νk|1

α+ εxσ

)sign(νk) =

α, νk > α+ σεx,

α

α+ σεxνk, |νk|1 ≤ α+ σεx,

−α, νk < −α− σεx.

(5.16)

5.2. Minimization in λ. Let λ = ζnλ −σNcn. We solve the minimization in λ (5.10) again

through Moreau’s identity [44] and set λ = λ− σV ∗ with

V ∗ ← minV

F (V ) +σ

2‖V − λ

σ‖2.

The minimization in V is decoupled in each one of its block components Vi ∈ R3,

minV

np∑k=1

βf(Vk − nk) + 0.5σ(Vk − σ−1λk)2,(5.17)

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 14: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2086 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

and solved by independently minimizing each term in the sum. As the Huber loss functionis differentiable and convex, the minimizers are the zeros of the derivative of the objectivefunction with respect to each Vk, that is,

βV ∗k − nk

max(εn, |V ∗k − nk|2)+ σV ∗k − λk = 0.(5.18)

If |V ∗k − nk|2 ≤ εn, the optimality conditions are solved as follows:

β

εn(V ∗k − nk) + σV ∗i = λi ⇐⇒ V ∗k = nk +

εnβ + εnσ

(λk − σnk),(5.19)

and the condition |V ∗k − nk|2 ≤ εn becomes |λk − σnk|2 ≤ β + εnσ.If |V ∗k − nk|2 > εn, the optimality conditions can be rewritten as[

β

|V ∗k − nk|2+ σ

](V ∗k − nk) = λk − σnk,(5.20)

and vectors V ∗k − nk and λk − σnk are colinear. The problem is then reduced to finding the

scalar ρ such that V ∗k − nk = ρ(λk − σnk), that is,

β

|λk − σnk|2+ σρ = 1 ⇐⇒ ρ =

1

σ

[1− β

|λk − σnk|2

].(5.21)

As a result, V ∗k = nk + 1σ [1− β

|λk−σnk|2](λk−σnk) and the condition |V ∗k −nk|2 > εn becomes

|λk−σnk|2 > β+εnσ. Combining these two cases, we obtain the following closed-form updatefor the dual variable λn+1

k = λk − σV ∗k :

λk = βλk − σnk

max( β + εnσ, |λk − σnk|2 ).(5.22)

5.3. Minimization in µ. Let µ = ζnµ −σQcn. We solve the minimization in µ (5.11) againthrough Moreau’s identity [44] and set µ = µ− σW ∗ with

W ∗ ← minW

H(W ) +σ

2‖W − µ

σ‖2.

The minimization is decoupled in each block component Wi ∈ Rm with a different term in

minW

nQ∑i=1

γwi|Wi|F + 0.5σ(Wi − σ−1µi)2(5.23)

and is solved by independently minimizing each summand with respect to its Wi; this corre-sponds to the proximal operator of the Frobenius norm that results in the shrinkage operator[47, 21, 48, 9]:

W ∗i =1

γwi + σµ ⇒ µi =

γwimax( γwi, |µi|F )

µi.(5.24)

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 15: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2087

6. Multiresolution. Given the hierarchy of the proposed discretizations, it is natural toperform the optimization in a multiresolution fashion to accelerate convergence. To this end,the basis functions are partitioned according to their hierarchical level Bp = BpL∪B

pL−1∪· · ·∪B

p1

and the optimization problem (5.2) is solved with increasing resolution as follows.1. Initialize χ0

0 = 0.2. Repeat the following.

(a) Optimization: find χl ∈ spanBpl solving (5.2) with Algorithm 1 initialized at χl0.

(b) Compute χl+10 by injecting χl into spanBpl+1 with B-spline refinement [25, 30].

3. At level L, χ = χL.At each level, the optimization typically converges with only a few iterations because thealgorithm is initialized close to the optimum using the solution from the coarser resolution.This speeds up the optimization without changing the final solution because the convexity ofthe problem guarantees convergence to its unique minimum.

Multiresolution reduces the size of the discretization, but the size of the data terms in theminimization problem (3.3) remains fixed because the sums

α

nP∑k=1

f(χ(xk)) + β

nP∑k=1

g(nk −∇χ)(6.1)

loop over all the input samples. As suggested by [31], we can introduce multiresolution inthe model and accelerate reconstruction by defining approximate data terms that cluster thesamples to the resolution of the representation. To this end, we exploit the octree structureand substitute all the point samples inside an octree cell C by their centroid xC and weighttheir contribution by the number of samples in the cell wC . At intermediate resolutions wethen substitute

nP∑k=1

αf(χ(xk)) + βg(nk −∇χ) ≈∑CwC[αf(χ(xC)) + βg(nC −∇χ(xC))

],(6.2)

where nC is the average normal of the samples in cell nC . The complexity of the resultingminimization matches the complexity of the computational grid and improves speed at in-termediate resolutions without compromising the accuracy of the reconstruction at the finestresolution, because the convexity of the problem guarantees convergence to the global opti-mum at the finest scale irrespective of the initialization at this scale.

7. Experimental results. We perform experiments with two kinds of publicly availabledata: synthetic data with ground truth, and point clouds obtained with a Kinect camera [7]or a range sensor. Kinect and range data suffers from nonuniform noise, large scanning gaps,and artifacts. We use the synthetic point clouds for quantitative evaluation and the noisyones to test reconstruction on data with real noise and artifacts.

The reconstructed surfaces are obtained by extracting the zero isolevel of the implicitfunction with a marching cubes algorithm [38, 45]. We run all our experiments in a singleprocessor of an Intel Core i7 at 2.6 GHz and 16 GB RAM memory to fairly compare allthe reconstruction techniques, instead of their implemented parallelization. The comparisonof their parallel implementations is beyond the scope of this paper as it depends on theD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 16: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2088 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

algorithmic choices of memory allocation, control mechanisms to access to shared data, andparticular implementations of octree data structures [36].

The model parameters α, β are normalized by the number of points and γ is normalizedby the size of the bounding volume to make the model independent of size and range of thepoint cloud. After this normalization, they are manually set to values that provide goodreconstruction once averaged over all experiments. The parameters εx, εn are fixed estimatesof the noise in the location and orientation of point cloud that we set manually to 0 forclean data and 0.05 for noisy ones. Spatially varying parameters are possible, but we do notinvestigate them here. The stopping condition of Algorithm 1 is set to δ = 1e − 6 and themaximum number of iterations to 1000. We limit the depth of the octree in order to make thedegrees of freedom of our discretization (the number of octree cells or grid coners) comparableto the size of the input point cloud; this is a natural bound on the resolution of the surfacebeyond which the reconstructions tend to fit the regularizer but do not add surface detail.As a result, the experiments of sections 7.1 and 7.2 are performed with octrees of maximumdepth 7, while the octrees of the experiments with real scanned point clouds in section 7.3have maximum depths 8–11 to account for the larger size of the point clouds.

7.1. Effects of the robust model. Our first experiment compares our model to a least-squares data-fidelity fit with the same discretization (HB1S basis). To this end, we randomlysample two meshes (a non-axis-aligned cube and a cow with 10K and 30K samples) and perturbthe point samples with noise. In Figures 2(a)–2(d) the least-squares model rounds the cube’scorners as it blindly averages the normal samples from different sides of the cube, while ourmodel reconstructs sharper corners by considering the normal samples from different cube sidesas outliers. In Figures 2(f)–2(i), a least-squares penalty on the screening term shrinks thinstructures, like the ears of the cow, because point samples from opposite sides act as outliersand pull the reconstruction to the center of the thin structure, causing shrinkage. The normalterm in the objective can overcome this effect with reliable information on the orientation ofthe surface. Normal estimates, however, tend to be noisier close to thin structures and least-squares normal fits are corrupted by outliers, so they cannot reliably estimate the surfaceorientation or complement the screening term to avoid shrinkage. We conclude that therobust data terms improve reconstruction in both screening and normals.

7.2. Effects of discretization. Our second experiment investigates the effects of dis-cretization in the accuracy of the reconstruction. Results are shown in Figure 3 and Table 1.The HB2S basis is the discretization with more degrees of freedom, while the HB1S basisis a simplification with a lower polynomial and smoothness degree and the HB2S dictionarysimplifies the refinement process by relaxing the basis property. The additional degrees offreedom of the HB2S basis are only useful if there are enough clean data to fit the modeland the noise level is small enough to prevent overfitting. Table 1 shows that the HB2S basisgenerally has twice as many degrees of freedom as the HB2S dictionary or the HB1S basis,and the reconstruction model cannot fit all the parameters of the HB2S basis to the inputpoint cloud. As a result, whenever the input point cloud does not contain enough data tocharacterize the surface, the degrees of freedom of the HB2S basis fit to the regularizer andthe reconstructed surface suffers from oversmoothing. Increasing the regularization to avoidoverfitting does not lead to reconstructions of the same quality as the use of multiresolutionD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 17: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2089

(a) LS fit, clean samples.Hausdorff dist. 0.82%

(b) Huber fit, clean samp.Hausdorff dist. 0.69%

(c) LS fit, noisy samplesHausdorff dist. 0.74%

(d) Huber fit, noisy samp.Hausdorff dist. 0.88%

(e) Ground truth (f) LS fit clean.Hausdoff 0.037%

(g) Huber clean.Hausdoff 0.032%

(h) LS fit noisy.Hausdoff 0.075%

(i) Huber noisy.Hausdoff 0.069%

Figure 2. Reconstruction of a unit cube (top row) and a cow (bottom) with least-squares (LS) and Huberdata-fidelity models with the HB1S basis. The Huber fit reconstructs sharp corners (cube) and avoids theshrinking bias of thin structures (cow’s ears) that affects least-squares models with clean and noisy data, andleads to a smaller average Hausdorff (as % of bounding-box diagonal) distance to the ground truth.

discretizations with the correct complexity without adapting the regularization parameter wto the noise and sample density, which is beyond the scope of this paper. This problem is morecritical for point clouds with undersampled complex structures, like the cow of Table 1. Weconclude that we cannot exploit the additional flexibility of the HB2S basis with noisy pointclouds that undersampled complex regions; simpler representations with the HB2S dictionaryor the HB1S basis improve the reconstruction both in terms of time and accuracy.

The HB2S dictionary and the HB1S basis have similar complexity, but differ slightly inspeed and accuracy. Their comparable complexity is a direct result of the parametrization ofeach discretization and Euler’s formula. The quadratic dictionary assigns a dictionary functionto each leaf cell of the minimal octree that stores the samples, and the dictionary has as manydegrees of freedom as there are leaves in the octree. Similarly, any function that is a linearcombination of the HB1S basis is parametrized by its value at the corners of the grid cells usedfor trilinear interpolation, and the number of degrees of freedom corresponds to the number ofcorners of the grid formed by the octree. In terms of speed, quadratic splines are slightly slowerdespite their lower complexity because they require computing second-order derivatives, whilethe HB1S basis only needs to evaluate finite-differences of vertex values. In terms of accuracy,the HB1S basis produces smooth reconstructions, while the HB2S dictionary tends to createtextured areas in smooth regions because it lacks the linear independence of the basis thatensures a unique representation; see the dragon’s or cow’s torso in Figures 3(a) and 3(d). As aD

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 18: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2090 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

(a) HB2S dictionary (b) HB2S dictionary (c) HB2S dictionary (d) HB2S dictionary

(e) HB2S basis (f) HB2S basis (g) HB2S basis (h) HB2S basis

(i) HB1S basis (j) HB1S basis (k) HB1S basis (l) HB1S basis

Figure 3. Effects of discretization. The HB2S dictionary tends to reconstruct texturized smooth regionsbecause the lack of a basis property (nonuniqueness of representation) leads to indeterminations in those regions.The HB2S and HB1S bases produce smooth reconstructions, but the HB2S leans towards oversmoothing as aresult of its higher polynomial degree.

Table 1Degrees of freedom (dof), reconstruction time, and average Hausdorff distance (as % of the bounding-box

diagonal) between the ground truth and the reconstructed surface with different discretizations. The cube andcow point clouds are obtained by sampling a ground-truth mesh, while the dragon and bunny are obtained bysubsampling a ground-truth point cloud.

Average distance # variables (dof)/running time (s)Point clouds # samples HB2S basis HB2S dict. HB1S basis HB2S basis HB2S dict. HB1S basis

Dragon 43k 0.387 0.105 0.098 276K/672 71K/81 95K/58Bunny 12k 0.538 0.111 0.143 264K/403 61K/75 94K/53Cow 30k 0.804 0.90 0.068 98K/432 38K/46 53K/45Cube 10k 1.125 0.808 0.730 22K/317 11K/23 14K/6

result, there are multiple combinations of HB2S dictionary functions that represent the samefunction χ and multiple global minimizers that cause wiggles in smooth regions as the solveroscillates between equivalent close minima at convergence.

Finally, extracting the surface with the dual marching cubes algorithm requires samplingthe surface over a balanced octree to accurately interpolate the zero-crossings of χ from thecorner samples of the octree grid. The HB2S dictionary and the HB1S basis incorporate this

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 19: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2091

(a) [31] (b) [11] (c) [42] (d) Our model

(e) [31] (f) [11] (g) [42] (h) Our model

Figure 4. Comparison to the state of the art for the synthetic points cloud of Table 2. The models of[31, 11, 42] round the cube’s corners and shrink the cow’s ears because they rely on least-squares data terms,while our model overcomes these artifacts with a robust Huber penalty. The method of [42] is robust to outliersbut leads to holes in small or isolated structures because its local formulation cannot leverage global informationto produce a watertight surface.

constraint by balancing the tree at the finest resolution before solving the optimization at asmall computational cost, but balancing the grid of the HB2S basis more than doubles the costof its refinement process because its basis functions have a larger support. For this reason,we avoid balancing the tree for the HB2S basis and use the standard marching cubes of [38]to extract the surface; this leads to cracks at the interfaces of cells differing by more than twolevels; see the right ear of the cow or the lower left wing of the angel in Figures 3(h) and 3(f),and explains our adoption of the HB1S basis discretization for the rest of our experiments.

7.3. Comparison to the state of the art. A third set of experiments compares our modelto the state-of-the-art techniques [30, 31, 11, 42].2 Figure 4 compares them visually forsynthetic data, for which we have ground truth, and Table 2 compares then quantitatively.Although the least-squares models [31, 11] have average Hausdorff distances comparable toour model (see Table 2), they produce errors that are not captured by the statistics of thepoint cloud but by the geometry of the reconstruction. The models of [31, 11] round thecube’s corners and shrink the ears of the cow because they rely on least-squares data terms,while our model overcomes these artifacts with a robust Huber penalty that recovers thesharp corner in the cube and avoids the shrinking bias. The robust model of [42] is able toreconstruct sharp corners and is robust to isolated outliers, but it cannot recover surfacesfrom point clouds with large gaps because of its local formulation, which produces holes in

2[31] is a generalization of [30] and we only report the best of the two for each point cloud.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 20: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2092 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

Table 2Average reconstruction time and Hausdorff distance, as a percentage of the bounding-box diagonal, between

the ground-truth mesh or point cloud and the reconstructed surface. The cube and cow point clouds are obtainedby sampling the ground-truth mesh while the dragon and bunny are obtained by subsampling the original pointcloud as detailed in Table 1.

dH 10% subsampled point cloud Point cloud with noise of σ = 0.1 AverageDragon Bunny Cow Cube Dragon Bunny Cow Cube Time (s)

[11] 0.112 0.116 0.069 0.71 0.150 0.290 0.198 0.74 8[31] 0.097 0.095 0.042 0.76 0.112 0.170 0.188 0.77 9[42] 0.01 0.13 0.32 0.92 0.19 0.21 0.56 0.93 109Ours 0.098 0.143 0.068 0.73 0.112 0.172 0.171 0.78 51

sharp undersampled regions like the ears of the cow.3 These experiments with synthetic dataallow us to quantitatively evaluate the performance of our model in Table 2 by measuring theHausdorff distance between ground-truth data and the surfaces reconstructed from perturbedpoint clouds, obtained by either sub sampling the point cloud or adding white Gaussian noiseof standard deviation σ to the point locations before recomputing their normals.

We also perform experiments with noisy point clouds captured with a Kinect camera [7]and range data to investigate the reconstruction from corrupted point clouds with real outliers.In this case, the least-squares models [31] and [11] reconstruct surfaces that reproduce thescanning artifacts of the point clouds and lead to incorrect topology estimates, as shown inFigures 5–7, while the robust model of [42] cannot recover surfaces from point clouds with largescanning gaps because of its local formulation, and does not produce watertight surfaces. Ourtwo model contributions are featured in these experiments. First, the use of a robust Huberfunction allows us to estimate topology more accurately—the shoelace of Figure 6(o) or thehuman ear in Figure 5(e)—and eliminate spurious point clusters—the head of the cleaner inFigure 5(y) or the statue of the dancers in Figure 5(o). Robust estimation of topology isan important feature because it is expensive to recover the correct topology once an explicitparametrization, e.g., a triangulated surface, has been extracted. Second, our regularizerallows for sharp changes in orientation and avoids oversmoothing the normals in texturedareas, as shown in Figures 6(d)–6(e) and 6(n)–6(o). In our regularizer, we weight the normof the Hessian with a mask that is only active far from the samples to avoid smoothing thenormals close to the point cloud, as [11] does in Figure 6(n), and do not square the Frobeniusnorm to allow for sharp changes in surface orientation that appear whenever two surfacesheets with opposite orientations are spatially close. As a result, we are able to reconstructthe correct topology of the statue in Figure 6(e), while [11] merges the two surface sheets atthe statue’s leg because it overpenalizes sharp changes in orientation and oversmooths thenormals in Figure 6(d). This is an important feature for the reconstruction of surfaces withtouching sheets and highlights the importance of both robust data terms and regularizers forthe estimation of a continuous surface from a set of sparse points.

8. Conclusions. We reconstruct surfaces from corrupted point clouds by formulating theproblem as a convex minimization model that is robust to the noise and scanning artifacts

3It is possible to close these gaps by increasing the scale of [42]’s kernels, but the reconstructions then mergethe ears and horns of the cow in a similar way to the least-squares models of [31, 11].D

ownl

oade

d 10

/21/

18 to

128

.97.

159.

65. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Page 21: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2093

(a) 189K PC (b) [42] (c) [31] (d) [11] (e) Our model

(f) Zoom on 5(a) (g) Zoom on 5(b) (h) Zoom on 5(c) (i) Zoom on 5(d) (j) Zoom on 5(e)

(k) 254K PC (l) [42] (m) [31] (n) [11] (o) Our model

(p) Zoom on 5(k) (q) Zoom on 5(l) (r) Zoom on 5(m) (s) Zoom on 5(n) (t) Zoom on 5(o)

(u) 163K PC (v) [42] (w) [31] (x) [11] (y) Our model

Figure 5. Comparison to state-of-the-art reconstructions [31, 11, 42] at octree depth 7 with noisy pointclouds (PC). The models [31, 11] are not robust to outliers and reproduce the clusters of outliers caused byscanning artifacts in the ear of the human head, the nozzle of the cleaner bottle, and the statue of a dancinggroup. The method of [42] is robust to isolated outliers but leads to holes in undersampled areas due to self-occlusions because its local formulation cannot leverage global information to produce a watertight surface. Ourmodel reconstructs topology and geometry more accurately despite the outliers.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 22: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2094 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

(a) 174K PC (b) [42] (c) [31] (d) [11] (e) Our model

(f) Zoom on 6(a) (g) Zoom on 6(b) (h) Zoom on 6(c) (i) Zoom on 6(d) (j) Zoom on 6(e)

(k) 123K PC (l) [42] (m) [31] (n) [11] (o) Our model

(p) Zoom on 6(k) (q) Zoom on 6(l) (r) Zoom on 6(m) (s) Zoom on 6(n) (t) Zoom on 6(o)

Figure 6. Comparison to state-of-the-art reconstructions [31, 11, 42] at octree depth 7 with noisy pointclouds (PC). The models [31, 11] are not robust to outliers and lead to reconstructions with incorrect topologies:[11] merges the two surface sheets in the statue’s knee or the shoelace, while [31] breaks the shoelace. The methodof [42] is robust to isolated outliers but leads to holes in undersampled areas because its local formulation cannotleverage global information to produce a watertight surface. Our model estimates topology and geometry moreaccurately despite the outliers.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 23: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2095

(a) 796K PC (b) [42] (c) [31] (d) [11] (e) Our model

(f) 796K noisyPC

(g) [42] (h) [31] (i) [11] (j) Our model

(k) Clean PC (l) Zoom on 7(g) (m) Zoom on7(h)

(n) Zoom on 7(i) (o) Zoom on 7(j)

(p) 208K PC (q) [42] (r) [31] (s) [11] (t) Our model

(u) Zoom on 7(p) (v) Zoom on 7(q) (w) Zoom on 7(r) (x) Zoom on 7(s) (y) Zoom on 7(t)

Figure 7. Row 1: Reconstruction with octrees of depth 11 with clean point cloud (PC). Global models([31, 11] and our ) reconstruct the eagle with comparable accuracies while the local method [42] reconstructssurfaces with holes in undersampled areas. Rows 2–3: Adding Gaussian noise to the point cloud, the differencesbetween least-squares ([31, 11]) and robust models ([42] and our) are visible in the shrinkage of the eagle’s beakand the noise ripples in smooth regions. Rows 4–5: Reconstruction with octrees of depth 10. The point cloudartifacts at the statue’s hands lead to surfaces with erroneous topology (holes [42] or multiple components [11])with local and least-squares models; our method reconstructs a surface with the correct topology because of thegeometric regularizer.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 24: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2096 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

of point clouds obtained with low-commodity range sensors. To this end, we substitute theusual least-squares penalties with robust Huber loss functions that preserve sharp edges andthin surface structures, and include a regularizer that allows sharp changes in the surface’sorientation and is only active far from the samples to avoid oversmoothing textured regions.

For an efficient parametrization, we approximate the implicit function with different hier-archical basis functions to match the complexity of the surface and the resolution power of themodel. Our investigations on discretizations of different complexity and smoothness concludethat a linear hierarchical B-spline basis over an octree offers the best performance for ourdata and model resolutions, allowing us to represent both smooth regions and sharp cornersat minimal computational cost without overfitting the surface to the point cloud noise.

The proposed discretizations lend themselves to a multiresolution strategy for the miniza-tion. This is particularly advantageous because the use of robust Huber penalties in ourvariational model leads to a more complex minimization problem that we solve efficiently byexploiting problem structure with a splitting algorithm that is easy to parallelize.

REFERENCES

[1] A. Adamson and M. Alexa, Point-sampled cell complexes, ACM Trans. Graph., 1 (2006), pp. 671–680.[2] M. Alexa, J. Behr, D. Cohen-Or, S. Fleishman, D. Levin, and C. T. Silva, Computing and

rendering point set surfaces, IEEE Trans. Vis. Comput. Graph., 9 (2003), pp. 3–15.[3] N. Amenta, M. Bern, and M. Kamvysselis, A new Voronoi-based surface reconstruction algorithm,

in Proceedings of the 25th Annual Conference on Computer Graphics and Intenctive Techniques(SIGGRAPH ’98) 1998, pp. 415–421.

[4] N. Amenta, S. Choi, T. K. Dey, and N. Leekha, A simple algorithm for homeomorphic surfacereconstruction, Int. J. Comput. Geom. Appl., 12 (2002), pp. 125–141.

[5] N. Amenta, S. Choi, and R. K. Kolluri, The power crust, in Proceedings of the 6th ACM Symposiumon Solid Modeling and Applications, 2001, pp. 249–266.

[6] N. Amenta and Y. J. Kil, Defining point-set surfaces, ACM Trans. Graph., 23 (2004), pp. 264–270.[7] J. Balzer, M. Peters, and S. Soatto, Volumetric reconstruction applied to perceptual studies of size

and weight, in Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision (WACV),2013.

[8] M. Berger, A. Tagliasacchi, L. Seversky, J. Levine, A. Sharf, and C. Silva, State of the artin surface reconstruction from point clouds, in Proceedings of the 35th Annual Conference of theEuropean Association for Computer Graphics (Eurographics 2014), 2014, pp. 161–185.

[9] J. M. Bioucas-Dias and M. A. T. Figueiredo, A new TwIST: Two-step iterative shrinkage/ thresh-olding algorithms for image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2992–3004.

[10] J.-D. Boissonnat and S. Oudot, Provably good sampling and meshing of surfaces, Graph. Models, 67(2005), pp. 405–451.

[11] F. Calakli and G. Taubin, SSD: Smooth signed distance surface reconstruction, Comput. Graph. Fo-rum, 30 (2011), pp. 1993–2002.

[12] J. C. Carr, R. K. Beatson, and T. R. Evans, Reconstruction and representation of 3D objects withradial basis functions, in Proceedings of the 28th Annual Conference on Computer Graphics andInteractive Techniques (SIGGRAPH ’01), 2001, pp. 67–76.

[13] A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applicationsto imaging, J. Math. Imaging Vis., 40 (2010), pp. 120–145.

[14] M. Chuang and M. Kazhdan, Interactive and anisotropic geometry processing using the screened Pois-son equation, ACM Trans. Graph., 30 (2011), 57.

[15] U. Clarenz, U. Diewald, and M. Rumpf, Anisotropic geometric diffusion in surface processing, inProceedings of the 11th Annual IEEE Visualization Conference (VIS ’00), 2000, pp. 397–405.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 25: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

ROBUST SURFACE RECONSTRUCTION 2097

[16] B. Curless and M. Levoy, A volumetric method for building complex models from range images Volu-metric integration, in Proceedings of the 23th Annual Conference on Computer Graphics and Inter-active Techniques (SIGGRAPH ’96), 1996, pp. 303–312.

[17] J. Daniels II, L. K. Ha, T. Ochotta, and C. T. Silva, Robust smooth feature extraction frompoint clouds, in Proceedings of the 2007 IEEE International Conference on Shape Modeling andApplications, 2007, pp. 123–136.

[18] T. K. Dey and S. Goswami, Tight cocone: A water-tight surface reconstructor, in Proceedings of the8th ACM Symposium on Solid Modeling and Applications, 2003, pp. 127–134.

[19] V. Dobrev, J.-L. Guermond, and B. Popov, Surface reconstruction via -minimization, In NumericalAnalysis and its Applications, Lecture Notes in Comput. Sci. 5434, Springer, Berlin, 2009, pp. 32–43.

[20] V. Dobrev, J.-L. Guermond, and B. Popov, Surface reconstruction and image enhancement via L1

em -minimization, SIAM J. Sci. Comput., 32 (2010), pp. 1591–1616.[21] D. Donoho and I. Johnstone, Adapting to unknown smoothness via wavelet shrinkage, J. Amer. Statist.

Assoc., 90 (1995), pp. 1200–1224.[22] V. Estellers, M. A. Scott, K. Tew, and S. Soatto, Robust Poisson surface reconstruction, in Scale

Space and Variational Methods in Computer Vision, Lecture Notes in Comput. Sci. 9087, Springer,Berlin, 2015, pp. 525–537.

[23] V. Estellers, D. Zosso, R. Lai, S. Osher, J.-P. Thiran, and X. Bresson, Efficient algorithm forlevel set method preserving distance function, IEEE Trans. Image Process., 21 (2012), pp. 4722–4734.

[24] S. Fleishman, D. Cohen-Or, and C. T. Silva, Robust moving least-squares fitting with sharp features,ACM Trans. Graph., 44 (2005), pp. 544–552.

[25] D. Forsey and R. Bartels, Hierarchical B-spline refinement, in Proceedings of the 15th Annual Com-perence on Computer Graphics and Interactive Techniquaes (SIGGRAPH ’88), 1988, pp. 205–212.

[26] C. M. Grimm and J. F. Hughes, Modeling surfaces of arbitrary topology using manifolds, in Proceedingsof the 22th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPM ’95),1995, pp. 359–368.

[27] G. Guennebaud and M. Gross, Algebraic point set surfaces, ACM Trans. Graph., 26 (2007), pp. 1–10.[28] K. Hildebrandt and K. Polthier, Anisotropic filtering of non-linear surface features, Comput. Graph.

Forum, 23 (2004), pp. 391–400.[29] H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle, Surface reconstruction

from unorganized points, in Proceedings of the 19th Annual Conference on Computer Graphics andInteractive Techniques (SIGGRAPH ’92), 1992, pp. 71–78.

[30] M. Kazhdan, M. Bolitho, and H. Hoppe, Poisson surface reconstruction, in Proceedings of the 4thEurographics Symposium on Geometry Processing, 2006, pp. 61–70.

[31] M. Kazhdan and H. Hoppe, Screened Poisson surface reconstruction, ACM Trans. Graph., 32 (2013),pp. 1–13.

[32] P. Labatut, J.-P. Pons, and R. Keriven, Robust and efficient surface reconstruction from range data,Comput. Graph. Forum, 28 (2009), pp. 2275–2290.

[33] C. Lange and K. Polthier, Anisotropic smoothing of point sets, Comput. Aided Geom. Design, 22(2005), pp. 680–692.

[34] D. Levin, Mesh-independent surface interpolation, in Geometric Modeling for Scientific Visualization,Springer, Berlin, 2004, pp. 37–49.

[35] C. Li, C. Xu, C. Gui, and M. D. Fox, Distance regularized level set evolution and its application toimage segmentation, IEEE Trans. Image Process., 19 (2010), pp. 3243–3254.

[36] X. Li, D. Andersen, M. Kaminsky, and M. Freedman, Algorithmic improvements for fast concurrentCuckoo hashing, in Proceedings of the 9th European Conference on Computer Systems (EuroSys ’14),2014, 27.

[37] Y. Lipman, D. Cohen-Or, and D. Levin, Data-dependent MLS for faithful surface approximation, inProceedings of the 9th Eurographics Symposium on Geometry Processing (SGP ’07), 2007, pp. 59–67.

[38] W. E. Lorensen and H. E. Cline, Marching cubes: A high resolution 3D surface construction algorithm,in Proceeding of the 14th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’87), 1987, pp. 163–169.

[39] J. Manson, G. Petrova, and S. Schaefer, Streaming surface reconstruction using wavelets, Comput.Graph. Forum, 27 (2008), pp. 1411–1420.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Page 26: Robust Surface Reconstructionvision.ucla.edu/papers/estellersS16surface.pdf · ROBUST SURFACE RECONSTRUCTION 2075 problem solved by local analysis of the cells [5,43,10], eigenvector

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

2098 V. ESTELLERS, M. A. SCOTT, AND S. SOATTO

[40] D. Nehab, S. Rusinkiewicz, J. Davis, and R. Ramamoorthi, Efficiently combining positions andnormals for precise 3D geometry, ACM Trans. Graph., 24 (2005), pp. 536–543.

[41] Y. Ohtake, A. Belyaev, M. Alexa, G. Turk, and H.-P. Seidel, Multi-level partition of unityimplicits, ACM Trans. Graph., 22 (2003), pp. 463–470.

[42] A. C. Oztireli, G. Guennebaud, and M. Gross, Feature preserving point set surfaces based on non-linear kernel regression, Comput. Graph. Forum, 28 (2009), pp. 493–501.

[43] J. Podolak and S. Rusinkiewicz, Atomic volumes for mesh completion, in Proceedings of the 3rdEurographics Symposium on Geometry Processing, 2005, pp. 33–41.

[44] R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1997.[45] S. Schaefer and J. Warren, Dual marching cubes: Primal contouring of dual grids, Comput. Graph.

Forum, 24 (2005), pp. 195–201.[46] M. A. Scott, D. C. Thomas, and E. J. Evans, Isogeometric spline forests, Comput. Methods Appl.

Mech. Engrg., 269 (2014), pp. 222–264.[47] S. Setzer, Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage, in Scale Space and

Variationall Methods in Computer Vision, Lecture Notes in Comput. Sci. 5567, Springer, Berlin,2009, pp. 464–476.

[48] S. Setzer, Operator splittings, Bregman methods and frame shrinkage in image processing, Int. J. Com-put. Vis., 92 (2011), pp. 265–280.

[49] C. Shen, J. F. O’Brien, and J. R. Shewchuk, Interpolating and approximating implicit surfaces frompolygon soup, ACM Trans. Graph., 23 (2004), pp. 896–904.

[50] R. Kolluri, J. R. Shewchuk, and J. F. O’ Brien, Spectral surface reconstruction from noisy pointclouds, in Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Pro-cessing (SGP ’04), 2004, pp. 11–21.

[51] S. M. Smith, Fast robust automated brain extraction, Hum. Brain Mapp., 17 (2002), pp. 143–155.[52] T. Tasdizen, R. Whitaker, P. Burchard, and S. Osher, Geometric surface smoothing via anisotropic

diffusion of normals, in Proceedings of the 2002 IEEE Visualization Conference (VIS ’02), 2002, pp.125–132.

[53] G. Taubin, Curve and surface smoothing without shrinkage, in Proceedings of the 5th InternationalConference on Computer Vision (ICCV ’95), 1995, pp. 852–857.

[54] A.-V. Vuong, C. Giannelli, B. Juttler, and B. Simeon, A hierarchical approach to adaptive lo-cal refinement in isogeometric analysis, Comput. Methods Appl. Mech. Engrg., 200 (2011), pp.3554–3567.

Dow

nloa

ded

10/2

1/18

to 1

28.9

7.15

9.65

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php