optimal transport in imaging sciences

1

Optimal Transportin Imaging Sciences

Gabriel Peyré

www.numerical-tours.com

http://www.numerical-tours.com

http://www.numerical-tours.com

Statistical Image Models

Optimal transport framework Sliced Wasserstein projection Applications

Application to Color Transfer

Source image (X )

Style image (Y )

Sliced Wasserstein projection of X to styleimage color statistics Y

Source image after color transfer

J. Rabin Wasserstein Regularization

Colors distribution: each pixel � point in R3


Input image Modified image



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )






Source image (X )

Style image (Y )





Other applications: Texture synthesisTexture segmentation


Input image Modified image



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )






Source image (X )

Style image (Y )





Overview

• Wasserstein Distance

• Sliced Wasserstein Distance

• Color Transfer

• Wasserstein Barycenter

• Gaussian Texture Models

Discrete measure:

Discrete Distributions

Xi � Rd�

i

pi = 1µ =N�1�

i=0

pi�Xi

Xi

Quotient space:

Discrete measure:

Constant weights: pi = 1/N .Point cloud

RN�d/�N


Xi � Rd�

i

pi = 1µ =N�1�

i=0

pi�Xi

Xi

Quotient space: A�ne space:

Discrete measure:

Constant weights: pi = 1/N . Fixed positions Xi (e.g. grid)HistogramPoint cloud

{(pi)i \�

i pi = 1}RN�d/�N


Xi � Rd�

i

pi = 1µ =N�1�

i=0

pi�Xi

Discretized image f � RN�d

N = #pixels, d = #colors.fi � Rd = R3

From Images to Statistics



Source image (X )

Style image (Y )





N = #pixels, d = #colors.fi � Rd = R3

f µf� Needs an estimator:� Modify f by controlling µf .


Disclamers: images are not distributions.



Source image (X )

Style image (Y )





N = #pixels, d = #colors.

Point cloud discretization:

fi � Rd = R3

f µf� Needs an estimator:

µf =�

i

�fi

� Modify f by controlling µf .


Xi




Source image (X )

Style image (Y )







fi � Rd = R3

Histogram discretization:


µf =�

i

�fi


µf =�

i

pi�Xi

pi =1

Zf

�

j

�(xi � fj)Parzen windows:


Xi




Source image (X )

Style image (Y )







fi � Rd = R3

Histogram discretization:


µf =�

i

�fi


µf =�

i

pi�Xi

pi =1

Zf

�

j

�(xi � fj)Parzen windows:

Today’s focus


Xi




Source image (X )

Style image (Y )




µX =�

i

�Xi

Colors: 3-DVector X � RN�d

(image, coe�cients, . . . )

Optimal Transport Distances

R

GB

Grayscale: 1-D

µX =�

i

�Xi

Colors: 3-D

�⇥ ⇥ argmin��N

�

i

||Xi � Y�(i)||p

Vector X � RN�d



R

GB

Grayscale: 1-D

Optimal assignment:

µX =�

i

�Xi

Colors: 3-D


�

i

||Xi � Y�(i)||p

Wasserstein distance: Wp(µX , µY )p =�

i

||Xi � Y��(i)||p

�� Metric on the space of distributions.

Vector X � RN�d



R

GB

Grayscale: 1-D

Optimal assignment:

µX =�

i

�Xi

Colors: 3-D


�

i

||Xi � Y�(i)||p

Wasserstein distance:

Projection on statistical constraints:ProjC(f) = Y � ��

Wp(µX , µY )p =�

i

||Xi � Y��(i)||p

�� Metric on the space of distributions.

C = {f \ µf = µY }

Vector X � RN�d



R

GB

Grayscale: 1-D

Optimal assignment:

Computing Transport Distances

Xi

Yi

Explicit solution for 1D distribution (e.g. grayscale images):

�� sorting the values, O(N log(N)) operations.

Higher dimensions: combinatorial optimization methods

Hungarian algorithm, auctions algorithm, etc.


Xi

Yi



�� O(N5/2 log(N)) operations.

�� intractable for imaging problems.

Higher dimensions: combinatorial optimization methods

Hungarian algorithm, auctions algorithm, etc.

µ =�

i

pi�Xi ⇥ =�

i

qi�Yi

�� Wp(µ, �)p solution of a linear program.

Arbitrary distributions:


Xi

Yi



�� O(N5/2 log(N)) operations.

�� intractable for imaging problems.

Overview



• Color Transfer



Approximate Sliced Distance

Key idea: replace transport in Rd by series of 1D transport. Xi

�Xi, ��Projected point cloud: X� = {�Xi, �⇥}i.

[Rabin, Peyre, Delon & Bernot 2010]

(p = 2)

SW (µX , µY )2 =�

||�||=1W (µX� , µY� )

2d�



�Xi, ��Sliced Wasserstein distance:

Projected point cloud: X� = {�Xi, �⇥}i.


(p = 2)

Theorem: E(X) = SW (µX , µY )2 is of class C1 and

SW (µX , µY )2 =�

||�||=1W (µX� , µY� )

2d�

�E(X) =�

��Xi � Y⇥�(i), �� d�.






where �� N are 1-D optimal assignents of X� and Y�.

(p = 2)

Theorem: E(X) = SW (µX , µY )2 is of class C1 and

SW (µX , µY )2 =�

||�||=1W (µX� , µY� )

2d�

�E(X) =�

��Xi � Y⇥�(i), �� d�.






where �� N are 1-D optimal assignents of X� and Y�.

�� Possible to use SW in variational imaging problems.�� Fast numerical scheme : use a few random �.

Sliced Assignment

X is a local minima of E(X) = SW (µX , µY )2

�� N , X = Y � �

Theorem:

Stochastic gradient descent of E(X):E�(X) =

�

��

W (X�, Y�)2Step 1: choose � at random.

Step 2: X(�+1) = X(�) � ��E�(X(�))

Sliced Assignment


�� N , X = Y � �

Theorem:


�

��


Step 2:

X(�) converges to C = {X \ µX = µY }.

X(�+1) = X(�) � ��E�(X(�))

Sliced Assignment


�� N , X = Y � �

Theorem:


�

��


Step 2:

X(�) converges to C = {X \ µX = µY }.

X(�+1) = X(�) � ��E�(X(�))

Sliced Assignment


�� N , X = Y � �

Theorem:

X(�) � ProjC(X(0))Numerical observation:

Final assignment

Overview



• Color Transfer



Input color images: fi � RN�3.

Color Histogram Equalization

⇥i =1N

�

x

�fi(x)



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )




f0

µ1

f1

µ0

f0

Optimal assignement:


min��N

||f0 � f1 ⇥ �||


⇥i =1N

�

x

�fi(x)



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )




f0

µ1

f1

µ0

f0

Transport:



min��N

||f0 � f1 ⇥ �||

T : f0(x) � R3 �� f1(�(i)) � R3


⇥i =1N

�

x

�fi(x)



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )




f0

µ1

f1

µ0

f0

T

Transport:

Equalization:



min��N

||f0 � f1 ⇥ �||

T : f0(x) � R3 �� f1(�(i)) � R3

f0 = T (f0) �� f0 = f1 � �


⇥i =1N

�

x

�fi(x)

T



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )






Source image (X )

Style image (Y )




f0

T (f0)

µ1

f1

µ0 µ1

f0

T

Solving is computationally untractable.

and is close to f0

f0 solves minf

E(f) = SW (µf , µf1)

Sliced Wasserstein Transfert

Approximate Wasserstein projection:

min��N

||f0 � f1 ⇥ �||

Solving is computationally untractable.

and is close to f0

f (0) = f0

(Stochastic) gradient descent:

At convergence: µf0= µf1

f (�+1) = f (�) � ��E(f (�))

f0 solves minf

E(f) = SW (µf , µf1)

f (�) � f0

Sliced Wasserstein Transfert

Approximate Wasserstein projection:

min��N

||f0 � f1 ⇥ �||

Input image f0

Target image f1

Transfered image f0

Color Transfert



Source image (X )

Style image (Y )






Source image (X )

Style image (Y )






Source image (X )

Style image (Y )




Transfered image f1

Color Exchange



Source image (X )

Style image (Y )

X ⇥� Y

Y ⇥� X




Source image (X )

Style image (Y )

X ⇥� Y

Y ⇥� X




Source image (X )

Style image (Y )

X ⇥� Y

Y ⇥� X




Source image (X )

Style image (Y )

X ⇥� Y

Y ⇥� X


Input image f0 Target image f1

Transfered image f0

Overview



• Color Transfer



µ�

µ1

µ3

W2(µ1, µ� )

W2(µ

2, µ

� )

W2 (µ

3 , µ �)�

i

�i = 1Barycenter of {(µi, �i)}Li=1:

µ� � argminµ

L�

i=1

�iW2(µi, µ)2

Wasserstein Barycenterµ2

µ�

µ1

µ3

W2(µ1, µ� )

W2(µ

2, µ

� )

W2 (µ

3 , µ �)

If µi = �Xi , then µ� = �X�

X� =�

i

�iXi

�� Generalizes Euclidean barycenter.

�

i


µ� � argminµ

L�

i=1

�iW2(µi, µ)2

Wasserstein Barycenterµ2

µ� exists and is unique.

µ�

µ1

µ3

Theorem:if µ0 does not vanish on small sets,

W2(µ1, µ� )

W2(µ

2, µ

� )

W2 (µ

3 , µ �)

If µi = �Xi , then µ� = �X�

X� =�

i

�iXi

�� Generalizes Euclidean barycenter.

�

i


µ� � argminµ

L�

i=1

�iW2(µi, µ)2

Wasserstein Barycenter

[Agueh, Carlier, 2010]

µ2

t �� µt is the geodesic path.

µt � argminµ

(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2

Special Case: 2 Distributions

Case L = 2:

µ1

µ2µ�

µ =�N

k=1 �Xi(k),Discrete point clouds:

Assignment: �⇥ ⇥ argmin��N

N�

k=1

||X1(k)�X2(�(k))||2

µ� =�N

k=1 �X�(k),

t �� µt is the geodesic path.

X� = (1� t)X1(k) + tX2(��(k))

µt � argminµ

(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2

Special Case: 2 Distributions

Case L = 2:

µ1

µ2µ�

where

� i = 1, . . . , L, µi =�

k �Xi(k)

Linear Programming Resolution

µ�

µ1

µ2

µ3

Discrete setting:

� i = 1, . . . , L, µi =�

k �Xi(k)

C(k1, . . . , kL) =�

i,j �i�j ||Xi(ki)�Xj(kj)||2


L-way interaction cost:

µ�

µ1

µ2

µ3

Discrete setting:

� i = 1, . . . , L, µi =�

k �Xi(k)

L-way coupling matrices:

�= 1

�= 1

�= 1

C(k1, . . . , kL) =�


P � PL



µ�

µ1

µ2

µ3

Discrete setting:

� i = 1, . . . , L, µi =�

k �Xi(k)


�= 1

�= 1

�= 1

Linear program:

C(k1, . . . , kL) =�


minP�PL

�

k1,...,kL

Pk1,...,kLC(k1, . . . , kL)

P � PL



µ�

µ1

µ2

µ3

Discrete setting:

� i = 1, . . . , L, µi =�

k �Xi(k)


�= 1

�= 1

�= 1

Linear program:

Barycenter:

X�(k1, . . . , kL) =�L

i=1 �iXi(ki)

C(k1, . . . , kL) =�


minP�PL

�

k1,...,kL

Pk1,...,kLC(k1, . . . , kL)

P � PL

µ� =�

k1,...,kL

Pk1,...,kL�X�(k1,...,kL)



µ�

µ1

µ2

µ3

Discrete setting:

µX that solves

minX

�

i

�i SW (µX , µXi)2

Sliced Wasserstein Barycenter

Sliced-barycenter:

Gradient descent:

µX that solves

minX

�

i


Ei(X) = SW (µX , µXi)2

X(�+1) = X(�) � ⇥�

L�

i=1

�i�Ei(X(�))


Sliced-barycenter:

Gradient descent:

Disadvantage:Non-convex problem

� local minima.

Advantages:

µX that solves

minX

�

i



X(�+1) = X(�) � ⇥�

L�

i=1

�i�Ei(X(�))


µX is a sum of N Diracs.

Sliced-barycenter:

Smooth optimization problem.

Gradient descent:

Disadvantage:Non-convex problem

� local minima.

Advantages:

µX that solves

µ1

minX

�

i



X(�+1) = X(�) � ⇥�

L�

i=1

�i�Ei(X(�))


µX is a sum of N Diracs.

Sliced-barycenter:

Smooth optimization problem.

µ2µ3

Raw imagesequence

Application: Color Harmonization

Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion

Color transfer

Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;

Raw image sequence

J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing


Color transfer


Raw image sequence



Color transfer


Raw image sequence


Raw imagesequence

ComputeWassersteinbarycenter



Color transfer


Raw image sequence



Color transfer


Raw image sequence



Color transfer


Raw image sequence


Raw imagesequence

ComputeWassersteinbarycenter

Project onthe barycenter



Color transfer


Raw image sequence



Color transfer


Raw image sequence



Color transfer


Raw image sequence



Color transfer


Sliced Wasserstein Projection on the barycenter



Color transfer





Color transfer




Overview



• Color Transfer



Exemplar f0

Texture Synthesis

Problem: given f0, generate f“random”

perceptually “similar”

analysis Probabilitydistributionµ = µ(p)

Exemplar f0

Texture Synthesis



analysis synthesisProbabilitydistributionµ = µ(p)

Exemplar f0

Outputs f � µ(p)

Texture Synthesis



analysis synthesisProbabilitydistributionµ = µ(p)

Exemplar f0

Outputs f � µ(p)

Gaussian models: µ = N (m,�), parameters p = (m,�).

Texture Synthesis



Input exemplar:d = 1 (grayscale), d = 3 (color)

N1

N2

N3

Images Videos

f0 � RN�d

Gaussian Texture Model

N1

N2


N1

N2

N3

Images Videos

Gaussian model:

m � RN�d,� � RNd�Nd

X � µ = N (m,�)

f0 � RN�d


N1

N2


N1

N2

N3

Images Videos

Gaussian model:


X � µ = N (m,�)

� highly under-determined problem.Texture analysis: from f0 � RN�d, learn (m,�).

f0 � RN�d


N1

N2


N1

N2

N3

Images Videos

Gaussian model:


X � µ = N (m,�)

Texture synthesis:given (m,�), draw a realization f = X(�).

� highly under-determined problem.

� Factorize � = AA� (e.g. Cholesky).� Compute f = m + Aw where w drawn from N (0, Id).

Texture analysis: from f0 � RN�d, learn (m,�).

f0 � RN�d


N1

N2

Stationarity hypothesis: X(· + �) � X(periodic BC)

Spot Noise Model [Galerne et al.]


Block-diagonal Fourier covariance:y(�) = �(�)f(�)y = �f computed as

where f(�) =�

x

f(x)e2ix1⇥1�

N1+

2ix2⇥2�N2



Block-diagonal Fourier covariance:y(�) = �(�)f(�)y = �f computed as

where f(�) =�

x

f(x)e2ix1⇥1�

N1+

2ix2⇥2�N2

Maximum likelihood estimate (MLE) of m from f0:

� i, mi =1N

�

x

f0(x) � Rd



Block-diagonal Fourier covariance:

�i,j =1N

�

x

f0(i + x)�f0(j + x) � Rd�d

y(�) = �(�)f(�)y = �f computed as

where f(�) =�

x

f(x)e2ix1⇥1�

N1+

2ix2⇥2�N2

MLE of �:


� i, mi =1N

�

x

f0(x) � Rd



Block-diagonal Fourier covariance:

�i,j =1N

�

x

f0(i + x)�f0(j + x) � Rd�d

y(�) = �(�)f(�)y = �f computed as

where f(�) =�

x

f(x)e2ix1⇥1�

N1+

2ix2⇥2�N2

�� = 0, �(�) = f0(�)f0(�)� � Cd�d

� is a spot noise �� = 0, �(�) is rank-1.

MLE of �:


� i, mi =1N

�

x

f0(x) � Rd


� Cd � C

Input f0 � RN�3 Realizations f

Example of Synthesis

Synthesizing f = X(�), X � N (m,�):

�� = 0, f(�) = f0(�)w(�)

� Convolve each channel with the same white noise.

w � N (N�1, N�1/2IdN )

Input distributions (µ0, µ1) with µi = N (mi,�i).

E0 E1

Ellipses: Ei =�x � Rd \ (mi � x)��1

i (mi � x) � c�

Gaussian Optimal Transport

Input distributions (µ0, µ1) with µi = N (mi,�i).

�0,1 = (�1/21 �0�

1/21 )1/2

E0 E1

W2(µ0, µ1)2 = tr (�0 + �1 � 2�0,1) + ||m0 �m1||2,

T

T (x) = Sx + m1 �m0

Theorem: If ker(�0) � Im(�1) = {0},

S = �1/21 �+

0,1�1/21

where

Ellipses: Ei =�x � Rd \ (mi � x)��1

i (mi � x) � c�

Gaussian Optimal Transport

Geodesics between (µ0, µ1): t � [0, 1] �� µt

Wasserstein Geodesics

(W2 is a geodesic distance)µt = argmin

µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2

Variational caracterization:


Optimal transport caracterization:

µ1µ0

µt = ((1� t)Id + tT )�µ0



µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2


µ�


Optimal transport caracterization:

µ1µ0

µt = ((1� t)Id + tT )�µ0

Gaussian case:µ0

µ1



µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2


µ�

� the set of Gaussians is geodesically convex.

µt = Tt�µ0 = N (mt,�t)

mt = (1� t)m0 + tm1

�t = [(1� t)Id + tT ]�0[(1� t)Id + tT ]

Geodesic of Spot NoisesTheorem:i.e. �i(�) = f [i](�)f [i](�)�.

f [t] = (1� t)f [0] + tg[1]

g[1](�) = f [1](�)f [1](�)�f [0](�)|f [1](�)�f [0](�)|

Then � t � [0, 1], µt = µ(f [t])Let for i = 0, 1, µi = µ(f [i]) be spot noises,

tf [0]f [1]

Geodesic of Spot Noises

0 1

Theorem:i.e. �i(�) = f [i](�)f [i](�)�.

f [t] = (1� t)f [0] + tg[1]

g[1](�) = f [1](�)f [1](�)�f [0](�)|f [1](�)�f [0](�)|

Then � t � [0, 1], µt = µ(f [t])Let for i = 0, 1, µi = µ(f [i]) be spot noises,

OT Barycenters

µ1 µ3

µ2

Optimal transportEuclidean Rao

2-D Gaussian Barycenters

Inpu

t

Spot Noise Barycenters

Exe

mpl

ars

Dynamic Textures Mixing

19/05/12 Static and dynamic texture mixing using optimal transport

1/1localhost/Users/gabrielpeyre/Documents/Dropbox/slides/2012-‐‑05-‐‑20-‐‑siims-‐‑textures/…/index.html

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start

images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

Abstract.- This paper tackles static and dynamic texture mixing by combining the statistical properties of an input set of images or videos. We focus on spot noisetextures that follow a stationary and Gaussian model which can be learned from the given exemplars. From here, we define, using optimal transport, the distancebetween texture models, derive the geodesic path, and define the barycenter between several texture models. These derivations are useful because they allow theuser to navigate inside the set of texture models, interpolating a new texture model at each element of the set. From these new interpolated models, new texturescan be synthesized of arbitrary size in space and time. Numerical results obtained from a library of exemplars show the ability of our method to generate newcomplex realistic static and dynamic textures.

Results following the geodesic path:

M A T L A B C O D E

1

24

5

6

0

3

7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7



<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start

images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>



M A T L A B C O D E

1

24

5

6

0

3

7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7



<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start

images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>



M A T L A B C O D E

1

24

5

6

0

3

7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7



<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start

images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>

<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>



M A T L A B C O D E

1

24

5

6

0

3

7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7

[0] [1] [2]f f f

0 1 2 3 4 5 6 7

[See web page]

Conclusion

Image modeling with statistical constraints�� Colorization, synthesis, mixing, . . .



Source image (X )

Style image (Y )




�� Fast sliced approximation.Wasserstein distance approach

Conclusion




Source image (X )

Style image (Y )




�� Fast sliced approximation.Wasserstein distance approach

Extension to a wide rangeof imaging problems.

in out�� Color transfert, segmentation, . . .

Conclusion

14 Anonymous

P (⇥⇥) P (⇥� ) P (⇥� c) P (⇥⇥) P (⇥� ) P (⇥� c)

P (⇥⇥) P (⇥� ) P (⇥� c) P (⇥⇥) P (⇥� ) P (⇥� c)

Fig. 4. Left: example of natural image segmentation. Below each image is displayed the 2-D histogram of the image (in the space of the two dominant color eigenvectors as providedby a PCA) for the whole domain � (which is P (⇥⇥)) and over the inside and outside thesegmented region.




Source image (X )

Style image (Y )




optimal transport in imaging sciences

Education

source image style image

image y stylej

r3source image

pixel source image x

color transferstyle

dvector x rn d x

projc f

d distribution