Image Recolorization for the ColorblindJia-Bin Huang1, Chu-Song Chen1, Tzu-Cheng Jen2, Sheng-Jyh Wang2

1 Institute of Information Science, Academia Sinica, Taipei, Taiwan2 Institute of Electronics Engineering, National Chiao Tung University, Hsinchu, Taiwan

Problem•People with color vision deficiency (CVD) have difficulty indistinguishing between some colors.

•How the colorblind perceive colors?

Background

•CVD results from partial or complete loss of function of one ormore types of cone cells.

•Simulation of color perception of people with CVD [1]•Related works addressing CVD accessibility:

I Guidelines for designers to avoid ambiguous color combinationsI (Semi-)automatic methods for recoloring images

Contributions•Generalize the concept of key colors in a image•Propose to measure the contrast between two key colors by thesymmetric KL divergence

• Interpolate colors to ensure local smoothness with a few keycolors

Reference1 H. Brettel, F. Vienot, and J.D. Mollon, “Computerized simulation

of color appearance for dichromats”, J. Optic. Soc. Amer. A,1997.

2 L. Jefferson and R. Harvey, “Accommodating color blindcomputer users,” in ACM SIGACCESS, 2006.

The Proposed Algorithm

• Image Representation via Gaussian Mixture ModelingI Use the CIEL*a*b* color spaceI Approximate the distribution by K Gaussians: p(x |Θ) =

∑Ki=1ωiGi(x |θi)

I Learn the parameters by the Expectation-Maximization (EM) algorithmI Select the optimal number of K by the Minimum Description Length (MDL) principle

•Target DistanceI Generalize the concept of key colors from “point” to “cluster”.I Use symmetric Kullback-Leibler (KL) divergence as our dissimilarity measure The symmetric KL

divergence: DsKL(Gi,Gj) = DKL(Gi||Gj) + DKL(Gj||Gi)I For Gaussians, analytical solutions exists

DsKL(Gi,Gj) = (µi − µj)T (Σ−1

i + Σ−1j )(µi − µj)+

tr(ΣiΣ−1j + Σ−1

i Σj − 2I)

•OptimizationI Define the color mapping functions Mi(·), i = 1, . . . ,KI The error introduced by the ith and jth key colors:

Ei,j = [DsKL(Gi,Gj)− DsKL(Sim(Mi(Gi)),Sim(Mj(Gj)))]2

I Introduce weights for each color values: αj = ||xj − Sim(xj)||I Obtain weight for each cluster:

λi =

∑Nj=1αjp(i |xj,Θ)∑K

i=1∑N

j=1αjp(i |xj,Θ)

I Rewrite the objective function

E =i=K∑i=1

j=K∑j=i+1

(λi + λj)Ei,j.

I Minimize the objective function via direct search optimization method.•Gaussian Mapping for Interpolation

I Compute the transformed colors using

T (xj)H = xH

j +K∑

i=1

p(i |xj,Θ)(Mi(µi)H − µH

i )

Future Directions•Relax the assumption of Gaussian distribution (e.g., use non-parametric modeling)•More principled optimization procedure•Subjective evaluation

Experimental Results

•Sample results

(a) (b) (c)Figure: (a) Original images. (b) Simulated views of the original images forprotanopia (first row), deuteranopia (second row), and tritanopia (third row).(c) Simulation results of the re-colored images.

•Comparison with [2]

(a) (c) (e)

(b) (d) (f)Figure: Comparison with [2]. (a) The original image. (b) The simulated viewof (a) for tritanopia. (c)(d) The re-colored result by the proposed method andits simulated view. (e)(f) The re-colored result by [2] and its simulated view.

Jia-Bin Huang et al. (Academia Sinica and National Chiao-Tung University, Taiwan) International Conference on Acoustics, Speech, and Signal Processing 2009