a variational analysis of shape from specularities using sparse … · 2009-01-16 · a variational...

A Variational Analysis of Shape from Specularities using Sparse Data∗

Jan Erik Solem1 Henrik Aanæs2 Anders Heyden1

1School of Technology and Society 2Informatics and Mathematical ModellingMalmö University, Sweden Technical University of Denmark, Denmark

jes,[email protected] [email protected]

AbstractLooking around in our every day environment, many of theencountered objects are specular to some degree. Activelyusing this fact when reconstructing objects from image se-quences is the scope of the shape from specularities prob-lem. One reason why this problem is important is that stan-dard structure from motion techniques fail when the objectsurfaces are specular. Here this problem is addressed byestimating surface shape using information from the spec-ular reflections. A specular reflection gives constraints onthe surface normal. The approach taken in this paper dif-fers significantly from many earlier shape from specular-ities methods since the normal data used is sparse. Themain contribution of this paper is to give a solid foundationfor shape from specularities problems. Estimation of sur-face shape using reflections is formulated as a variationalproblem and the surface is represented implicitly using alevel set formulation. A functional incorporating all surfaceconstraints is proposed and the corresponding level set mo-tion PDE is explicitly derived. This motion is then provento minimize the functional. As a part of this functional avariational approach to normal alignment is proposed andanalyzed. Also novel methods for implicit surface interpola-tion to sparse point sets are presented together with a vari-ational analysis. Experiments on both real and syntheticdata support the proposed method.

1 IntroductionStructure and motion, the reconstruction of an object andthe motion of the camera from a sequence of images, isone of the most widely studied fields in computer vision.However, looking at most of the work done on this impor-tant subject, specular reflections has been ignored or at bestbeen viewed as outliers. The standard assumption when de-termining the structure of the scene and motion of the cam-era is that the surfaces in the scene are Lambertian, i.e. that

∗This work has been supported by the EU-project IST-2000-28459 IN-VIEW.

their apparent brightness is independent of viewing direc-tion. For the parts of the scene where the surfaces are roughenough to fulfill the Lambertian assumption, standard struc-ture from motion techniques usually give a good reconstruc-tion of the scene [7]. This is generally not the case for spec-ular surfaces. This paper proposes a method for augmentinga structure from motion scheme in such a way that the in-formation given by the specular reflections can be utilized.This allows reconstruction of objects where there is only in-formation from specularities, e.g. the window in Figure 2.The proposed scheme is such that following a structure frommotion reconstruction of tracked points [7], a surface is fit-ted to the estimated 3D points applying the constraints onthe normal given by specular reflections. This is done byderiving a functional that captures these constraints on thesurface. This functional induces a PDE, which can be usedto evolve the surface in a level set framework.

Previous work has been done in the area of reconstruct-ing or estimating surfaces from specularities. Early workinclude the study of specularity disparity observed from twoviewpoints [1] and an examination of the surface informa-tion available from the motion of specularities [24]. Zhenget al. [23] propose surface reconstruction of an object on aturn table from specularities, and in [22] the shape of thelight source is investigated. Schultz [14] investigates 2.5Dsurface reconstruction from specularities. Both the meth-ods of Zheng et al. and Schultz require specularities to beobserved almost everywhere on the surface. This is an im-practical assumption in the structure from motion setting,in that it will require a large amount of images, which arenot necessarily present in the structure from motion case.Thus the method proposed here differs from these methods,in that it handles very sparse data sets. Also other relatedmethods have been proposed for fitting a surface to data us-ing a variational approach, e.g. [19, 3, 4, 8].

In [2] surfaces are reconstructed from reflections of a cal-ibrated scene using a space carving approach. Space carv-ing is also used together with a photo-consistency measurethat can handle specular reflections in [20]. Perfect mirrorsurfaces are reconstructed in [13] using reflections of a cal-

Proceedings of the 2nd Intl Symp 3D Data Processing, Visualization, and Transmission (3DPVT 2004) 0-7695-2223-8/04 $20.00 © 2004 IEEE

ibrated scene. Also related is [18] where an algorithm forcomputing the mirror shape for a catadioptic imaging sys-tem is presented. A first attempt to use implicit surfaces forshape from specularities with sparse data was made in [17].Methods for determining the constraints on the surface nor-mal for general light source positions are proposed in [16].

The main contribution of this paper is to give a solidfoundation for shape from specularities problems usingsparse data. Novel functionals and their corresponding levelset motions are proposed for solving both normal alignmentand surface interpolation of sparse point sets. As a result,a functional incorporating both of these new componentsis explicitly derived to solve shape from specularities prob-lems with sparse normal data. Although work has been doneon this in the past, e.g. [19], the incorporation of sparse con-straints in the level set framework is useful for many ap-plications and for the level set community in general. Thisvariational approach makes it possible to perform a rigorousanalysis of shape from specularities for implicit surfaces.It should be noted that the method proposed in this papercan also be used for reflections of 3D points, not only lightsources.

2 Background

2.1 Notation and Structure from MotionThe following notation will be used: qi are the image co-ordinates for specularity i, ci is the focal point for the cor-responding image and ri is the ray from ci through qi. Anarbitrary surface is denoted Γ. The computational domainis a volume Ω ⊂ R3.

Throughout this paper it is assumed that there are enoughfeatures in the scene to determine the motion of the cameraand the structure of a limited number of feature points. Thisis done by extracting and tracking feature points throughthe image sequence. The Euclidean structure and cameramotion and calibration is then determined using standardtechniques [7].

Given a camera matrix P , the line of sight correspondingto the image point q is given by

r(q) = c + λP+q , (1)

where c ∈ N (P ) denote the focal point, λ the depth param-eter and P+ denote the pseudo-inverse of P , see [7]. Henceif a specularity is observed at point q in a given camera, (1)denotes the possible locations of the surface reflecting thelight.

2.2 Constraints from reflectionsThe condition for specular reflection is that the surface nor-mal bisects the viewing direction and the incident light di-

N

L

S

ci

ri

i

Figure 1: The relation between a specularity in an imageand the surface normal.

rection. This means that at the intersection of the ray ri (1)and the surface, the normal Ni is known. This relation isshown in Figure 1. Solving for Ni we get

Ni =L − ri

|L − ri|, (2)

where ri is the directional vector for each ray, normalizedso that |ri| = 1 and L is the direction to the light source. Itis important to note that the depths of the points where theseconstraints apply are unknown.

These normal constraints are not enough to determinethe surface, in that they contain no depth information. Re-quiring that the surface also interpolates the points obtainedfrom the previous structure from motion step resolves thisambiguity. These constraints are denoted point constraints.However, the normal and point constraints do not uniquelydefine the surface and regularization or interpolation isneeded. Integrating the proposed method with other mul-tiple view stereo algorithms is likely to address this issue.In the remainder of this paper regularization is achieved us-ing a smoothness prior. A more abstract formulation of theproblem is then to find the smoothest surface that interpo-lates a set of points and has a given gradient or normal atthe intersection with a set of lines.

2.3 Level Set Surface Representation

The level set method was introduced in [11] as a tool forcapturing moving interfaces. The time dependent surfaceΓ(t) is implicitly represented as the zero level set of a func-tion φ(x, t) : Ω × R+ → R as

Γ(t) = x ; φ(x, t) = 0 , (3)


where φ is defined such that

φ(x, t)

< 0 inside Γ= 0 on Γ> 0 outside Γ .

(4)

Using the definition above gives the outward unit normal nand the mean curvature κ as

n =∇φ

|∇φ|and κ = ∇ ·

∇φ

|∇φ|. (5)

These expressions will be used frequently throughout thispaper. The zero set of φ(x, t) represents Γ(t) at all timest. This means that φ(x(t), t) ≡ 0 for a curve x(t) ∈ Γ(t).Differentiating with respect to t gives

φt + v · ∇φ = 0 ⇔ φt + vn|∇φ| = 0 , (6)

where v = dx(t)/dt and vn is the velocity normal to thesurface. To move the surface according to some derivedvelocity, this PDE is solved on a fixed grid in the domainΩ. One of the advantages of this representation is that thetopology of the surface is allowed to change as the surfaceevolves, thus making it easy to represent complex surfacesthat can merge or split and also surfaces that contain holes.For a more thorough treatment of level set surfaces cf. [12,15].

3 Variational FormulationThe main purpose of this paper is to propose a functionalthat incorporates the deviation from all the normal con-straints and all the local point constraints in one single ex-pression. This section describes the components of thisfunctional. The corresponding level set motion is derivedas a gradient descent of the functional and proofs are giventhat this motion makes the energy decrease until equilib-rium. The desired surface is then found by solving the mo-tion PDE and minimizing this functional.

3.1 Normal ConstraintsGiven a set of normals N and a set of corresponding pointsX where each normal constraint Ni ∈ N is associated witha point in space xi ∈ X , given by the intersection of (1)and Γ. Introduce an energy functional as the Lp norm ofthe normal deviation

EN (Γ,N ,X ) =

(∫

Γ

(1 − N(x) · n(x))p

dS

)1/p

, (7)

where N(x) is the extension of the desired normals Ni atpoints xi to the whole domain Ω, Γ is an arbitrary surfaceand n(x) is the surface normal. Here N(x) is defined as

N(x) =

Ni if x ∈ Bε(xi)n(x) otherwise ,

(8)

where Bε(xi) is a ball with radius ε centered around xi. Thewidth of Bε(xi) is typically chosen to enclose the nearestgrid point. This will make the integrand equal to zero wherethere are no constraints on the normal. Representing Γ asthe zero set of φ, (7) becomes

EN (φ,N ,X ) =

=

(∫ (

1 − N(x) ·

(

∇φ

|∇φ|

))p

δ(φ(x))|∇φ(x)| dx

)1/p

,

(9)where δ(·) is the one dimensional delta function andδ(φ(x))|∇φ(x)| dx is the surface area element, as a specialcase of the co-area formula [5]. This functional is men-tioned in [12] but without any reference. The derivation ofthe corresponding motion is therefore given here.

Proposition 1: The functional (9) will give an evolu-tion equation as

φt =

(

∇ ·∇φ

|∇φ|− ∇ · N(x)

)

|∇φ| , (10)

for p = 1, independent of the choice of N(x).Proof: see Appendix.

Note that for p = 1, measuring the normal deviation us-ing (1−N(x) · ∇φ

|∇φ| ) is equivalent to using 1

2|N(x)− ∇φ

|∇φ| |

since |N(x)− ∇φ|∇φ| | = |N(x)|2−2N(x) · ∇φ

|∇φ| + | ∇φ|∇φ| |

2 =

2(1−N(x) · ∇φ|∇φ| ) due to the fact that the vectors are of unit

length.Also note that it is easy to change the definition of N(x)

in (8) to make N(x) differentiable without changing the re-sults. However, this change would not make any differencein practice since, at the implementation stage, we work in adiscrete setting.

The following property of this motion can also be shown.

Proposition 2: For the motion (10), the energyEN (φ,N ,X ) decreases until it reaches equilibrium.Proof: see Appendix.

3.2 Point ConstraintsThe surface should interpolate a given set of points S. It istempting to introduce an energy functional similar to [21],which penalizes the deviation of the surface from the pointsin S, as

EP (Γ,S) =

(∫

Γ

d(x)p dS

)1/p

, (11)

where Γ is an arbitrary surface and d(x) = dist(x,S) isthe distance from x to the set of points S where the pointconstraints apply.


With sparse data, this has the drawback that the surfacewill continue to deform in the direction of the negative gra-dient of the distance potential d(x) even after the points areon the surface. This will also interfere with the local orien-tation of the surface.

Instead, punishing the deviation of the points in S fromthe surface Γ can be expressed using the energy functional

EP (φ,S) =∑

y∈S

|φ(y)| =∑

y∈S

∫

|φ(x)|δ(x − y) dx ,

(12)which will only minimize the shortest distance from thepoints to the surface if |∇φ| = 1.

Since (12) is only evaluated on a discrete set of pointsin the domain Ω and all functions are only given values atthese points, the following regularization is used

EP (φ,S) =

∫

|φ(x)|δε(x,S) dx , (13)

where δε(x,S) =∑

y∈S δε(x − y) and δε(·) is a smoothversion of the Dirac delta function with some width ε. Inpractice one can use a uniform Gaussian with standard de-viation ε. Since there are no terms containing ∇φ(x) in theintegrand, the condition for a minimum is

sign(φ(x))δε(x,S) = 0 , (14)

according to the Euler-Lagrange equation [6]. The evolu-tion equation for φ as the gradient descent of (13) is then

φt = − sign(φ(x))δε(x,S) . (15)

If the right-hand-side of (15) is a descent direction then sois

−f(x) sign(φ(x))δε(x,S)

if f(x) ≥ 0, ∀x ∈ Ω. Typically one can choose f(x) =|φ(x)|. This gives

φt = −φ(x)δε(x,S) , (16)

which is what was proposed in [17] in a more ad hocfashion. This approach is also related to the point setattractors in [10].

Proposition 3: For the motion (16), the energy EP (φ,S)decreases until it reaches equilibrium.Proof: see Appendix.

3.3 Total EnergyDefine the total energy ETot to be minimized as

ETot = EN + EP + αES , (17)

where ES is a regularizing smoothness term and α ∈ R+ isa constant determining the amount of smoothing. The usualchoice is to use surface area

ES(φ) =

∫

Γ

1 dS =

∫

δ(φ(x))|∇φ(x)| dx , (18)

which leads to a mean curvature motion

φt = κ|∇φ| , (19)

where κ = ∇ · ∇φ|∇φ| is the mean curvature of the level sets.

Putting all this together gives

ETot =

∫

|φ(x)|δε(x,S) +

(

(1 + α) − N(x) ·∇φ

|∇φ|

)

δ(φ(x))|∇φ(x)|dx . (20)

The resulting PDE for the motion will then be

φt = |∇φ| (κ −∇ · N(x)) − φ(x)δε(x,S) + ακ|∇φ| ,(21)

as a combination of (10), (16) and (19).

Theorem 1: For the motion (21), if ε and α are small, thetotal energy ETot decreases until it reaches equilibrium.Proof: see Appendix.

What conclusions can be made concerning the existenceof a solution? All three components of ETot are non-negative, which means that ETot ≥ 0. A simply-connectedsurface with non-zero area can only be a minimum of ETotif the normal and point constraints are fulfilled. Let G =Γ ; EN (Γ) = 0, EP (Γ) = 0,Γ simply − connected bethe set of all such surfaces, where we have excluded sur-faces containing holes. Denote the smallest value of ETotby Emin, where

Emin = infΓ∈G

ETot(Γ) .

Since ETot is bounded from below, there exist at least onesurface that gives a stationary point. We will consider anylocal minimizer of (20) as a valid solution. The desired sur-face, as a local minimum of (20), is then found by choosingan appropriate initial surface.

3.4 Local Minima and InitializationAs is the case with many optimization problems in com-puter vision, our approach suffers from the fact that theremight be several local minima and stationary points to theenergy functional (20). This depends on the point con-straints, normal constraints and the amount of regularizationimposed. Minimizing (20) using the gradient descent (21)will result in finding one of these stationary points. Whichstationary point is found depend on the initial values used.


In practice, this leads to the problem of finding a goodinitial guess for the surface. Since the point constraints areonly effective near the points in the set S, an initial guessshould interpolate these points. To do this the method de-scribed in [21] can be used. The surface is then found as aminimizer of (11), as described above. The correspondinglevel set evolution equation for p = 1 is then

φt = (∇d(x) · n + d(x)κ) |∇φ| , (22)

where κ is the mean curvature and n is the normal. Denotethe minimizing steady state solution of (11) by φ0.

If another initial surface is desired that still interpolatesthe points in S, weighted convection of the surface can beused

φ = φ0 + c(x)d(x)|∇φ0| , (23)

where c(x) : Ω → R. The factor d(x) is, as above, thedistance from x to S and makes sure that the zero set stillinterpolates the points in S. We have found that by varyingc(x) we can change the initial surface such that the desiredlocal minimum is obtained.

The final algorithm can then be summarized as

1. Find initial guess for Γ by minimizing (11).

2. Use (23), if necessary, to avoid possible undesired lo-cal minima.

3. Minimize the functional corresponding to the total en-ergy (20) to obtain the surface.

4 ExperimentsIn order to demonstrate the soundness of the proposedmethod experiments were performed on real data. To per-form a more rigorous validation the method was also ap-plied to synthetic data, where the ground truth was known.

4.1 Real DataTwo real data sets were used for reconstruction, one of acar window and one containing a wine bottle. In both casesthe surfaces were reconstructed satisfactory validating theproposed approach, see Figures 2 and 3.

The car window sequence was taken with a camcorder.Sample images from the sequence are shown in Figure 2.The sequence consisted of 38 images containing specu-larities and one image of the camera shadow used to de-termine the light direction. The window has sharp cor-ners which make good features. Two of these were usedfor determining the surface depth and solving the depthambiguity. This resulted in a setup with 38 normal con-straints and 2 point constraints. The initial surface was aplane which was convected using (23) with c = 0.3 and

(a) (b) (c)

(d) (e) (f)

Figure 2: Samples from the image sequence of a car win-dow. (a-b) frames number 1 and 20. (c) image used to deter-mine the light direction. (d-f) three views of the estimatedsurface.

(a) (b) (c)

(d) (e) (f)

Figure 3: Samples from the image sequence of a wine bot-tle. (a-c) frames number 3, 8 and 11 with the area of interestmarked in the last image. (d-f) three views of the estimatedsurface.

d(x) = min(dist(x,S), 3). The result is shown in Figure 2,where the zero set is visualized using the marching cubes al-gorithm [9] with only the triangles inside the convex hull ofX and S shown.

The wine bottle sequence contained 11 images where aspecular reflection moved over an un-textured region (theglass between the labels) of the bottle. The light source, inthis case the camera flash, moves with the camera. Sampleimages are shown in Figure 3 together with the estimatedsurface structure.

4.2 Synthetic DataTo perform a rigorous validation of the proposed method, adata set consisting of a synthetic surface of 1/8 of a spherewas used together with 24 normal constraints and 3 pointconstraints. The initial surface, as minimizer of (11) was


(a) (b) (c)

(d) (e) (f)

Figure 4: Results for a synthetic surface. (a) Initial guessas minimizer of (11). (b) Initial guess after using (23). (c)True surface. (d-f) Three views of the final surface.

0 50 100 150 200 250 3000

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0 50 100 150 200 250 3000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

(a) (b)

Figure 5: Errors for first 300 iterations for the synthetic sur-face. (a) Normal constraints: average value of (1 − N(x) ·∇φ|∇φ| ). (b) Point constraints: average value of φ(S) shownfor the iterations where φ is recomputed to a signed distancefunction. Values are then distance to surface measured invoxel width.

then a plane. To avoid this local minimum the surface wasevolved using (23) with c = 0.5. As approximation to thedelta function a Gaussian with standard deviation of onegrid cell was used.

Firstly, the convergence properties of the algorithm wereinvestigated and evaluated visually and numerically. Thevalue of α was 0.01. The results are shown in Figure 4. Theaverage values of (1 − N(x) · ∇φ

|∇φ| ) and φ(S) for the first300 iterations are shown in Figure 5. Here the values forφ(S) are shown for the iterations where φ is recomputed toa signed distance function. The values therefore representthe true distance to the surface. From this it is clear that themethod converges to the desired result which is also illus-trated visually in Figure 4. From Figure 5 it is also seen thatthe rate of convergence is acceptable.

5 ConclusionsIn this paper a rigorous variational analysis of the shapefrom specularities problem is presented. A functional in-corporating all surface constraints is derived. The corre-sponding level set motion is explicitly given and shown tomake the energy decrease until equilibrium. Experimentson both real and synthetic data validate the correctness ofthis variational approach.

Future work will include developing techniques for auto-matic detection of specular reflections and integration withvariational stereo methods.

AcknowledgementThe authors would like to thank Dr Niels Chr. Overgaardfor proofreading and for useful comments on the text.

A Proofs

A.1 Proposition 1Proof: The energy is

EN (φ,N ,X ) =

∫ (

1 − N ·

(

∇φ

|∇φ|

))

δ(φ)|∇φ| dx .

Denote the integrand as

F (x, φ,∇φ) =

(

1 − N ·

(

∇φ

|∇φ|

))

δ(φ)|∇φ| .

Then the condition for a minimum, given by the first varia-tion is:

∂F

∂φ−

∂

∂x

(

∂F

∂φx

)

−∂

∂y

(

∂F

∂φy

)

−∂

∂z

(

∂F

∂φz

)

= 0 .

The first term is

∂F

∂φ=

(

1 − N ·

(

∇φ

|∇φ|

))

δ′(φ)|∇φ| ,

and the second term gives

−∂

∂x

(

∂F

∂φx

)

= −∂

∂x

φx√

φ2x + φ2

y + φ2z

− Nx

δ(φ)

=

= −∂

∂x

φx√

φ2x + φ2

y + φ2z

− Nx

δ(φ)−

−

φ2x

√

φ2x + φ2

y + φ2z

− φxNx

δ′(φ) ,


where Nx is the first component of N . The remaining termsare treated analogously. Combining the last three then gives

−∇·

(

∇φ

|∇φ|− N

)

δ(φ)−

(

1 − N ·

(

∇φ

|∇φ|

))

δ′(φ)|∇φ| .

The last part cancels against the first term above and thegradient flow for φ is

φt = δ(φ)

(

∇ ·∇φ

|∇φ|− ∇ · N(x)

)

.

Extending the motion for φ = 0 to all the level sets of φ byreplacing δ(φ) with |∇φ| gives the desired flow.

A.2 Proposition 2Proof: The time derivative is

d

dtEN (φ,N ,X ) =

∫

d

dt((|∇φ| − N · ∇φ) δ(φ)) dx =

=

∫

(|∇φ| − N · ∇φ) δ′(φ)φt dx+

+

∫

d

dt(|∇φ| − N · ∇φ) δ(φ) dx . (24)

The second integral can be written∫

d

dt(|∇φ| − N · ∇φ) δ(φ) dx =

=

∫

d

dt

(√

φ2x + φ2

y + φ2z

)

dx −

∫

d

dt(N · ∇φ) dx ,

which is the same as∫ (

φx

|∇φ|φxt +

φy

|∇φ|φyt +

φz

|∇φ|φzt−

−d

dtN · ∇φ − N ·

d

dt∇φ

)

δ(φ) dx (25)

Taking each of the first three terms and integrating byparts gives∫ (

φx

|∇φ|φxt +

φy

|∇φ|φyt +

φz

|∇φ|φzt

)

δ(φ) dx = . . . =

= −

∫ (

∇ ·∇φ

|∇φ|

)

φtδ(φ) dx −

∫

|∇φ|φtδ′(φ) dx ,

(26)with ∂Γ/∂n = 0 on the boundary of Ω, where n is theoutward direction at the boundary. The second term of (26)cancels against the first half of the first term in (24). Thefourth term in (25) is zero since d

dtN = 0 , and the last termis

−

∫ (

N ·d

dt∇φ

)

δ(φ) dx = . . . =

=

∫

(∇ · N) δ(φ)φt dx +

∫

(N · ∇φ) δ′(φ)φt dx ,

(27)where again, integration by parts is used. The second termabove cancels against he second half of the first term in (24).Collecting the remaining terms gives

d

dtEN (φ,N,X) =

∫

(∇ · N) δ(φ)φt dx −

−

∫ (

∇ ·∇φ

|∇φ|

)

φtδ(φ) dx =

=

∫ (

∇ · N −∇ ·∇φ

|∇φ|

)

δ(φ)

(

∇ ·∇φ

|∇φ|− ∇ · N

)

|∇φ| dx =

= −

∫ (

∇ ·∇φ

|∇φ|− ∇ · N

)2

δ(φ)|∇φ| dx ≤ 0 ,

where the expression for φt is from (10).

A.3 Proposition 3Proof: The time derivative is

d

dtEP (φ,S) =

∫

d

dt|φ(x)|δε(x,S) dx =

=

∫

sign(φ(x))φtδε(x,S) dx .

Substituting φt according to (16), this becomes

d

dtEP (φ,S) = −

∫

|φ(x)| δε(x,S)2 dx ≤ 0 .

A.4 Theorem 1Proof: For α sufficiently small it suffices to show that

d

dtETot =

d

dtEN +

d

dtEP ≤ 0

for the motion

φt =

(

∇ ·∇φ

|∇φ|− ∇ · N(x)

)

|∇φ| − φ(x)δε(x,S) .

From the proofs of Propositions 3.1 and 3.2 this expressioncan be written

d

dtETot = −

∫ (

∇ ·∇φ

|∇φ|− ∇ · N

)

δ(φ)φt dx

+

∫

sign(φ(x))φtδε(x,S) dx .


Substituting φt this becomes

d

dtETot = −

∫ (

∇ ·∇φ

|∇φ|− ∇ · N

)2

δ(φ)|∇φ| dx

−

∫

|φ(x)| δε(x,S)2 dx

+

∫

sign(φ(x))δε(x,S)|∇φ|

(

∇ ·∇φ

|∇φ|− ∇ · N(x)

)

dx

+

∫ (

∇ ·∇φ

|∇φ|− ∇ · N

)

δ(φ)φ(x)δε(x,S) dx .

The last term is zero since δ(φ)φ(x) ≡ 0 and the third termis equivalent to integrating sign(φ(x))|∇φ| on the set S∩Xif ε is sufficiently small. Since S and X are disjoint in oursetting, this term is also zero.

References[1] A. Blake and G. Brelstaff. Geometry from speculari-

ties. Computer Vision., Second International Conference on,pages 394–403, 1988.

[2] Thomas Bonfort and Peter Sturm. Voxel carving for specu-lar surfaces. In Int. Conf. Computer Vision, pages 591–596,Nice, France, 2003.

[3] V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active con-tours. Int’l J. Computer Vision, 1997.

[4] L. D. Cohen and R. Kimmel. Global minimum for activecontour models: A minimal path approach. In IEEE Conf.Computer Vision and Pattern Recognition, pages 666–673,1996.

[5] L. C. Evans. Partial Differential Equations. American Math-ematical Society, 1998.

[6] I.M. Gelfand and S.V. Fomin. Calculus of Variations. DoverPublications, 2000.

[7] R. I. Hartley and A. Zisserman. Multiple View Geometry inComputer Vision. Cambridge University Press, 2000.

[8] R. Kimmel and A. M. Bruckstein. Global shape from shad-ing. In Computer Vision and Image Understanding, pages120–125, 1995.

[9] W.E. Lorensen and H.E. Cline. Marching cubes: a highresolution 3d surface reconstruction algorithm. ComputerGraphics (Siggraph’87 ), 21(4):163–169, 1987.

[10] K. Museth, D.E. Breen, R.T. Whitaker, and A.H. Barr. Levelset surface editing operators. ACM Transactions on Graph-ics, 21(3):330–8, 2002.

[11] S. Osher and J. A. Sethian. Fronts propagating withcurvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics,79:12–49, 1988.

[12] S. J. Osher and R. P. Fedkiw. Level Set Methods and Dy-namic Implicit Surfaces. Springer Verlag, 2002.

[13] S. Savarese and P. Perona. Local analysis for 3d reconstruc-tion of specular surfaces - part ii. In Proc. European Conf.on Computer Vision, pages 759–774, 2002.

[14] H. Schultz. Retrieving shape information from multiple im-ages of a specular surface. Pattern Analysis and MachineIntelligence, IEEE Transactions on, 16(2):195–201, 1994.

[15] J.A. Sethian. Level Set Methods and Fast Marching Meth-ods Evolving Interfaces in Computational Geometry, FluidMechanics, Computer Vision, and Materials Science. Cam-bridge University Press, 1999.

[16] J.E. Solem and A. Heyden. Estimating surface shape andextending known structure using specular reflections. In In-ternational Conference on Pattern Recognition, Cambridge,UK, 2004.

[17] J.E. Solem, H. Aanæs, and A. Heyden. Pde based shape fromspecularities. In Lewis D. Griffin and Martin Lillholm, edi-tors, Scale-Space Theories in Computer Vision, 4th Interna-tional Conference, Scale Space 2003, Isle of Skye, UK, vol-ume 2695 of Lecture Notes in Computer Science. Springer,2003.

[18] R. Swaminathan, S.K. Nayar, and M.D. Grossberg. Frame-work for designing catadioptric projection and imaging sys-tems. In Proc. ICCV-PROCAMS, Nice, France, 2003.

[19] D. Terzopoulos. Multiresolution Computation of Visible-Surface Representations. PhD thesis, Dept. of Electrical En-gineering and Computer Science, Massachusetts Institute ofTechnology, Cambridge, MA, January 1984.

[20] Ruigang Yang, Marc Pollefeys, and Greg Welch. Dealingwith textureless regions and specular highlights — a progres-sive space carving scheme using a novel photo-consistencymeasure. In Int. Conf. Computer Vision, pages 576–584,Nice, France, 2003.

[21] H.K. Zhao, S. Osher, B. Merriman, and M. Kang. Implicitand non-parametric shape reconstruction from unorganizedpoints using a variational level set method. In Computer Vi-sion and Image Understanding, pages 295–319, 2000.

[22] J. Y. Zheng and A. Murata. Acquiring a complete 3d modelfrom specular motion under the illumination of circular-shaped light sources. Pattern Analysis and Machine Intel-ligence, IEEE Transactions on, 22(8):913–920, 2000.

[23] J.Y. Zheng, Y. Fukagawa, and N. Abe. 3d surface estimationand model construction from specular motion in image se-quences. Pattern Analysis and Machine Intelligence, IEEETransactions on, 19(5):513–520, 1997.

[24] A. Zisserman, P. Giblin, and A. Blake. The informationavailable to a moving observer from specularities. Imageand Vision Computing, 7(1):38–42, 1989.


a variational analysis of shape from specularities using sparse … · 2009-01-16 · a variational...

Documents