Light Mixture Estimation for Spatially Varying White Balance

SIGGRAPH 2008

Eugene Hsu, (MIT, CSAIL)

Tom Mertens (Hasselt Univ. EDM)

Sylvain Paris (Adobe System)

Shai Avidan (Adobe System)

Frédo Durand (MIT, CSAIL)

Light Mixture Estimation for Spatially Varying White Balance

2

Outline

• Overview• White Balance• Material Color Estimation• Mixture Interpolation• Result• Relighting

Light Mixture Estimation for Spatially Varying White Balance

3

Overview

• Goal:– Perform a white balance technique

for scenes with two light types.

• Input:– Original image– Two light colors (user specified)

Light Mixture Estimation for Spatially Varying White Balance

4

White Balance

• Assumption– The interaction of light can be described using RGB c

hannels only, instead of requiring full spectra– Surfaces are Lambertian and non-fluorescent.– Color bleeding due to indirect illumination can be igno

red.– There are two illuminant types present in the scene a

nd their colors are known beforehand.– Scenes are dominated by only a small number of mat

erial colors.

Light Mixture Estimation for Spatially Varying White Balance

5

Light Mixture Estimation for Spatially Varying White Balance

6

Light Mixture Estimation for Spatially Varying White Balance

7

Material Color Estimation

• Natural scenes are dominated by a small set of material colors

[Omer and Werman 2004]

1. Sampling

2. Voting

3. Set Estimation

Light Mixture Estimation for Spatially Varying White Balance

8

Light Mixture Estimation for Spatially Varying White Balance

9

Material Color Estimation

Light Mixture Estimation for Spatially Varying White Balance

10

Material Color Estimation

Light Mixture Estimation for Spatially Varying White Balance

11

Mixture Interpolation

• Extends light mixture values over the entire image using interpolation.

• Image chromaticities are a linear blend of the material chromaticity multiplied by the light chromaticities.

• matting Laplacian [Levin et al. 2006]

Light Mixture Estimation for Spatially Varying White Balance

12

Mixture Interpolation

Light Mixture Estimation for Spatially Varying White Balance

13

Mixture Interpolation

• Color line model– Colors lie on a line in RGB space.

• Idea– Obtain the pixel opacities β by minimizing the quadrati

c βT Mβ, where M is the matting Laplacian.

Light Mixture Estimation for Spatially Varying White Balance

14

Mixture Interpolation

• Define Mij

– wk : window.

– Ii, Ij : the colors at pixels i and j.

– δij is 1 if i= j, 0 otherwise.

– μk, Σk : the mean and variance of pixel colors in wk.

– E3 is the 3×3 identity matrix.

– ε: regularizing constant.

Light Mixture Estimation for Spatially Varying White Balance

15

Mixture Interpolation

• Objective function

– α∗ : vector contains mixture constraints.– D : a diagonal matrix that selects the marked pixels

from the voting step.– λ : smoothing constant.

Light Mixture Estimation for Spatially Varying White Balance

16

Mixture Interpolation

Light Mixture Estimation for Spatially Varying White Balance

17

Result

• Synthetic inputs from multiple exposures• Single exposures with real mixed lighting

Light Mixture Estimation for Spatially Varying White Balance

18

Result

Light Mixture Estimation for Spatially Varying White Balance

19

Result

Light Mixture Estimation for Spatially Varying White Balance

20

Relighting

• Input– White balanced image.– Light mixture α.

• Output– Image that light colors are changed.

Light Mixture Estimation for Spatially Varying White Balance

21

Light Mixture Estimation for Spatially Varying White Balance

22

Relighting