poisson image editing - ustcstaff.ustc.edu.cn/.../courseware/dgp16_poissonimageediting.pdf ·...

Poisson Image Editing

Ligang LiuGraphics&Geometric Computing Lab

USTChttp://staff.ustc.edu.cn/~lgliu

Images in the Internet

Images everywhere!

Images Everywhere

• Digital camera

• Personal digital album

• Photoshop

Digital Image

Math Model of Image

• Continuous– f(x,y)

• Discrete– I (i,j)

– Pixel 

(R, G, B)

Image Processing

• Image filtering

• Image coding

• Image transformation

• Image enhancement

• Image segmentation

• Image understanding

• Image recognition

• …

Gradient Domain Image Editing

• Motivation:– Human visual system is very sensitive to gradient

– Gradient encode edges and local contrast quite well

• Approach:– Edit in the gradient domain

– Reconstruct image from gradient

Gradient Domain: View of Image

Membrane Interpolation

Membrane Interpolation

Poisson Equations

)(4 xρπG−=ΔΦ

Siméon Denis Poisson 

• His teachers: Laplace, Lagrange, …• Poisson’s terms:

– Poisson's equation 

– Poisson's integral

– Poisson distribution

– Poisson brackets

– Poisson's ratio 

– Poisson's constant

1781-1840, France

“Life is good for only two things: to study mathematics and to teach it.”

Background:

02 =+⋅+⋅+⋅+⋅+⋅+⋅ GfFfEfDfCfBfA yxyyxyxx

• Partial Differential Equations (PDE)

• The PDE's which occur in physics are mostly second order and linear:

0),,,,,( =yyxyxxyx ffffffE

– Hyberbolic• wave equation:

– Parobolic• heat equation:

– Elliptic• Laplace equation:

• Poisson equation:

:2BCA <⋅

:2BCA =⋅

2

2

21

tf

vf

∂∂

=Δ

fktf

Δ⋅=∂∂

:2BCA >⋅

0=Δf

ρ−=Δf

02 =+⋅+⋅+⋅+⋅+⋅+⋅ GfFfEfDfCfBfA yxyyxyxx

Poisson Equation

ρ−=Δf

yx 2

2

2

2

∂∂

+∂∂

≡Δ

),( yxρρ =

Boundary conditions

• Dirichlet boundary conditions:

• Neumann boundary conditions:

dsΩ∂Ω

Ω∂|f

Ω∂∂∂ |sf

Existence of solution

dsΩ∂Ω

The solution of an Poisson Equation is uniquelydetermined in , if Dirichlet boundary conditions or Neumann boundary conditions are specified on

ΩΩ∂

Discrete Poisson Equation  

jijijijijijiji

hfff

hfff

,2,1,1,

2,,1,1 22

ρ=−+

+−+ −+−+

ρ−=Δf

Matrix Nature of Poisson Equation

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−−

−−

−−

−−

−

33

23

13

32

22

12

31

21

11

33

23

13

32

22

12

31

21

11

41141

141141

141141

141141

14

bbbbbbbbb

fffffffff

Large sparse linear system

Poisson Equation Solver

• Direct method

• Iterative methods– Jacobi, Gauss‐Seidel, SOR

• Multigrid method

Variational interpretation

Ω∂Ω∂Ω=−∇= ∫∫ ||s.t minarg *2* ffff f v

Ω∂Ω∂ ==Δ ||s.t )( * ffdivf v

Euler Equation: 0=∂∂

−∂∂

−yx fff F

yF

xF

F

v is a guidance field, needs not to be a gradient field.

Physical Origins of Poisson Equation

• Electrostatic potential

• Gravitational potential

)(4 xρπG−=ΔΦ

0

)(ερ x

−=ΔΦ

Electrostatic potential

Charge Density

Electric Potential

Electric Field Φ

E

E

Φ

)(xρ

)(xρ

Φ−∇=E

30

21

4 rqq

πεrF =

Derivations

∫∫ =⋅VS

dvd0

)(ερ xsE

0

)(ερ x

−=ΔΦ

Gauss’s Law: V

S

ds

∫∫ ⋅∇=⋅VS

dvd EsEGauss’stheorem:

0

)(ερ xE =⋅∇

Φ−∇=E Poisson Equation

Φ

E

ρ

0

)(ερ x

−=ΔΦ

Φ−∇=E V

ddiv S

V

∫ ⋅≡=

→

sEEx

0

lim)()(

0ερ

Relationships

DensityPotential

Field

Φ E),()( 00 yxδρ =x

Gravitational potential

Mass Density

Gravitational Potential

Force Field

(acceleration)

g

Φ

)(xρ

Φ−∇=g

gF m=

m

)(xρ

Φ

3rmMGrF =

V

ddivG S

V

∫ ⋅≡=

→

sggx

0

lim)()(4 ρπ

Φ

g

ρ

Φ−∇=g

Relationships

DensityPotential

Field

)(4 xρπG−=ΔΦ

Analogy for Image

Image Density

Image (Potential)

Image Gradientg

I

)(xρ

I−∇=g

V

ddiv S

V

∫ ⋅≡=

→

sggx

0

lim)()(ρ

I

g

ρ

I−∇=g

Relationships

DensityPotential

Field

)(xρ−=ΔI

V

ddiv S

V

∫ ⋅≡=

→

sggx

0

lim)()(ρI−∇=g

)(xρ−=ΔI

)(xρ−=ΔI

Motivation for Image Editing

I

ρDensity

Potential

)(xρ−=ΔI

Poisson Image Editing

P. Pérez, M. Gangnet, and A. BlakeSIGGRAPH 2003

Seamless Cloning

Precise selection: tedious and unsatisfactory

Alpha-Matting: powerful but involved

Seamless cloning: loose selection but no seams?

Seamless Poisson Cloning

Ω∂Ω∂ =∇=Δ || s.t. )( BA IIIdivI

AI BI

Cloning by solving Poisson Equation

Why we do analogy for image?

• Easier in the image gradient domain

– Local editing         global effects

– Seamless ‐ cloning, editing, tilling

Variational Interpretation

Ω∂Ω∂Ω=∇−∇= ∫∫ ||s.t minarg *2*

BAf IIIII

Euler Equation: 0=∂∂

−∂∂

−yx fff F

yF

xF

F

AI∇ is a gradient field to be cloned.

Ω∂Ω∂ =∇=Δ || s.t. )( BA IIIdivI

ComposeDemo

Change Features

Change Texture

Conceal

Mix Lights

Change Colors

Change Colors

Seamless TilingSingle image

Seamless TilingMultipleimagestiled at random

Image Stitching

Alpha Blending / Feathering

Affect of Window Size

Affect of Window Size

Good Window Size

Gradient based Stitching

• Input images

Recap: Gradient Domain Image Editing

• Motivation:– Human visual system is very sensitive to gradient

– Gradient encode edges and local contrast quite well

• Approach:– Edit in the gradient domain

– Reconstruct image from gradient

Q&A