Automatic Coloring Using Factor Graph

Yuan Liu 1 Fan Jiang 1 Yangyi Lu 1

Abstract

The goal of our project is to build up a proba-bilistic factor graphical model for automatic col-oring task for the given gray scale templates. Wealso extend the model to the cases with colorconstraints. From the coloring patterns gener-ated from our model, we showed that the fea-tures we used can comprehensively represent col-oring properties and the factor graph model canindeed assign charming colorings for the inputtemplates.

1. IntroductionColored patterns with pleasing structures are very attrac-tive to human beings. People prefer delicate paintings,buildings and even well-structured and colored websites.However, while people can always easily distinguishpatterns that are pleasing from those unpleasing, creatingattractive pattern colorings is very time consuming andrequires much professional training, even skilled artistscannot promise a suitable coloring in a short time. So webelieve an efficient automatic coloring system will greatlyhelp web developers, architectures and even our daily life.

Colored pattern can be decomposed into two parts.One is pattern template, which is the decomposed spaceof regions, so that we know each separated region shouldbe assigned one color. The other is a set of colors, whichare assigned to the regions. In our project, we focus on thecases that the pattern templates are given and the regionsthat should be given the same color in a particular templateare also known. (For example, there are 10 separatedregions in one template and 5 colors are to be assigned toit. We may know 6 of the regions should be assigned thesame color, and the other 4 regions have their own colors.)

We use 256 popular colored patterns from 82 professional

*Equal contribution 1Carnegie Mellon University, U.S.A.. AU-THORERR: Missing \icmlcorrespondingauthor.

Proceedings of the 34 th International Conference on MachineLearning, Sydney, Australia, 2017. JMLR: W&CP. Copyright2017 by the author(s).

artists as the training dataset. By using a probabilisticfactor graph model that can be trained on example patternsand sampled to generate new colorings for a target patterntemplate, we can get a various attractive coloring set foreach pattern template through Markov Chain Monte Carlomethod after learning the colors’ distribution. Then userscan choose their favorite ones from the colorings providedto them.

We demonstrate the effectiveness of our model for avariety of pattern coloring scenarios. We compare thepattern colorings done by random coloring, artists coloringand our model. Pattern colorings done by probabilisticgraphical model and artists are shown to be significantlypreferred to random colorings.

In following parts of the paper, we will first showsome related work in section 2, introduce the dataset weare going to use in section 3. Then we will talk about themodel we built in section 4, its flow, components, learningand test. We will demonstrate our conclusion through theresults from our model in section 5 and 6. At last, we willtalk about the future directions of our project in section 7.

2. Related WorkProbabilistic Color-By-Numbers. This paper (Lin et al.,2013) is the baseline of our project. Starting with 5-coloring patterns dataset from COLOURlovers, the authorsbuild a model using factor graphs that can suggest patterncolorizations. Based on the idea of Color-By-Number,same color is used at different segments to form a coloredshape that can be recognized easily, for example, a flowerwith all red petals. Thus the picture can be divided into 5color groups, each group formed by same color segments.With the separation, the authors extract features for themodel. For a given color group, lightness, saturations andother color related properties are collected. Between anytwo color groups, the comparison and distance are alsogathered. For the whole picture, the color compatibilityis calculated using ODonovans model(O’Donovan et al.,2011). The three models are used as factors to inferencethe pattern colorization that has the highest probabilityof being as good as artists work. At the end, the authorsrecruited 16 people to rate a mixture of works from the

Submission and Formatting Instructions for ICML 2017

model and works from COLOURlovers artists. It turnedout that there is still gap in the aesthetic appealingnessfrom best works by this model to best works by artists,but the not-so-good work is good enough to serve asinspirational starting points for further creative work.

This will be the baseline of our project. We will tryto impose semantic constraints- for example, the sky isblue- to further improve the visual appealingness of thesuggested pattern colorization.

Creative Color Support Tools Sauvaget and hercolleagues describe a system that takes an image dividedinto segments and a set of colors and computes a harmo-nious assignment of colors to segments based on conrastof proportions guidelines. The system select colors froma fixed palette, and users can specify the set of colorsto be used and their relative amounts. However, in ourassumption, users do not have knowledge about colors tobe used, so this method is an improvement to Sauvaget’stool.

Natural Image Colorization. Related to pattern col-oring, researchers have developed many approaches tocolorizing grayscale photographic images. Some of thesemethods assume that neighboring pixels in space that havesimilar texture should have similar colors (Levin et al.,2004). According to this assumption, an energy functioncan be defined. By minimizing this energy function, theimage can be colored. In this project, we will also usesimilar tricks, by encoding the spatial relationship betweeneach pattern region as features in the graph model.

Color Compatibility. This paper is the first large-scale online studies of color preference (O’Donovanet al., 2011).It is also the first to use 5-color themes anduse-generated color combinations. The authors built amodel to analyze the 5-color themes color compatibility.The model is a combination of analysis of each color, eachcolor pairs, hue template, hue entropy and theme names,with three large 5- color theme datasets as training data.This model has been used in many fields that requires colorcompatibility. It will also be used in our project to calculatethe overall color compatibility of pattern colorizations.

3. DataThe data is downloaded from COLOURlovers website(COLOURlovers.com), which is a website for users toshare colorations for pictures and color creations. Thereare two kinds of data involved in this project. First is thetraining data, which is all from COLOURlovers patterns.The second part is the model input data, which is fromCOLOURlovers templates.

Template in COLOURlovers website is a greyscale picturethat shows shapes to be filled with colors. For example, inFigure 1, the leftmost picture shows a template. For eachtemplate, it is required to be filled with at most 5 colors.The pixels with same grey shade should be filled with thesame color. It is like the color-by-number game with 5color groups. To further decompose the template, we de-fine the pixels with same shades as a color group. Withineach color group, the connected pixels forms a segment.Thus a template can have at most 5 color groups, and eachcolor groups can have many segments. In the next section,we define various features on segments and groups. Wewill use these features in our factor graph model.

Pattern in COLOURlovers website is a designed templatefilled with colorations. For example, Figure 1 shows thesame templates, but with different colorations. We treatthese two as two different patterns. A pattern has samecolor groups and corresponding segments in a color groupas the underlying template, but a pattern also have Color,which is the coloration for each color group. In our model,we extract features from the coloration as well.

Some patterns are more visually appealing to people thanothers, our task in this project is to propose a model thatcan automatically generate visually appealing patterns. Weunderstand that visually appealingness is a subject concept,but there are some aesthetic rules that can be followed. Ourmodel models these rules and generate various colorationsuggestions, in the hope that some of these suggestions willbe visually appealing to viewers and some suggestions caninspire viewers.

In our model, all the data is fed using pictures in .png for-mat. Each picture is 200x200 pixels. We process thesepictures to get each component and extract features fromdifferent components. To train the model weights, we arefeeding the model with 256 pictures, found in (Lin et al.,2013) supplemental materials. In the future, we will getmore data directly from COLOURlovers website using pro-vided API.

Figure 1. Two colorations on same template. They are consideredtwo different patterns.

Submission and Formatting Instructions for ICML 2017

4. Method4.1. Model Overview

Figure 2. The model overview: training and test parts

Our model mainly contains two parts: the training part andtesting part. Training part consists of a factor graph model.Training images are fed into the factor graph. Weights arelearned using MCMC techniques. For the test part, a gray-scale template is the input. MCMC will be utilized to gen-erate color themes, and the model will pick the top col-orations. Then output of the model will be colored picturewith top scoring color themes. We will talk about the de-tails of the model in following sections.

4.2. Process Pictures

To make our model use the input images, we first have toprocess the picture from (200, 200, 3) array to usable com-ponents. Given a pattern picture, we first convert the pic-ture to greyscale, (200, 200, 1). We remove the bound-ary to remove the possible noise that will be introduced byboundary. Then we get the contour of color groups, andmake the contour clearer. This will help reduce the noiseas well. Then we get the 5 grey shades that have most pixelareas in the greyscale picture, convert the 5 shades area to 5color groups. Each group corresponds to a color in original(before grescale) picture. Then for each group, we furthergather connected pixels and make connected pixels a seg-ment. And stack several segments into a group. A segmentis represented as a list of coordinates, indicating the posi-tions in the image. We restrict the minimum size a segmentto be 20. This way, we can further decrease the noise byremoving elements such as contours, etc. Thus, an imageis now converted to at most 5 color groups, each group hasa corresponding color, and each group is consist of a list ofsegments, each segment is consist of a list of pixel coordi-nates.

Figure 3. Example of gray-scale picture

Figure 4. Visualization of five color groups separation

4.3. Unary Features

After converting the image to usable components, westarted to construct the factor graph. We have followingunary features given different components: color, segmentand group.

4.3.1. COLOR

With the pre-greyscale colors, we use the L*a*b colorspace instead of RGB. From L*a*b, we can easily get thecolor’s lightness and saturation.

Lightness. Lightness is the L component in L*a*bcolor space. It quantitatively reflects the brightness of acolor. Generally, we would expect background to havelow lightness and foreground groups to have high lightness.

Saturation. Saturation influences the grade of purityor vividness of a color/image. A desaturated image is saidto be dull, less colorful or washed out but can also makethe impression of being softer. The formula for saturationusing L*a*b is

√a2+b2√

a2+b2+L2.

Hue. Hue is defined technically as ”the degree towhich a stimulus can be described as similar to or differentfrom stimuli that are described as red, green, blue, andyellow”. In painting color theory, a hue refers to a purecolor - one without tint or shade. A hue is an element ofthe color wheel and it can be calculated directly from RGBform.

4.3.2. SEGMENT

Segment is the connected pixels of the same color. Thereare five segment features.

Relative Size. The number of pixels in current seg-ment divided by total pattern area (200 x 200 = 40,000).This feature can influence the pattern spatially.

Submission and Formatting Instructions for ICML 2017

Normalized Discrete Compactness The ratio of cur-rent segment’s perimeter and its area. We treat each pixelas a square. We expect a line to have low compactness anda square to have high compactness.

Elongation. Elongation is calculated by 1 - boxWidthboxHeight ,

where box is current segment’s minimum area boundingbox. Generally, we expect a square to have low elongation.

Centrality. Centrality indicates the Euclidean dis-tance between the segment’s centroid and the center of thepattern.

Role Labels. This feature indicates where the segment isin background (1) or foreground (0).

4.3.3. GROUP

Group is the combination of segments of same colors.There are four features for a group.

Relative Size. The area occupied by current colorgroup divided by total pattern area (200 x 200 = 40,000).

Segment Spread. The covariance matrix of currentcolor group’s segments’ centroids. This feature indicateswhether the segments in the group is centered around orspread out.

Segment Size Statistics. This feature contains thestatistics of current color group’s segments. It includesinformation like min, max, mean and standard deviation ofthe size of segments.

Number of Segments. The number of segments thatis contained in current color group. We require theunderlying segment to be at least 20 pixels.

Role Labels. Same as segment role labels, determin-ing if the color group is background or foreground.

For each feature, we normalize all continuous feature to ascale of [0, 1] so that in training phase, the weights will notsaturate or vanish. Role labels can only be 1 or 0, giventhe current components is background or foreground. Fornumber of segments, we use a mapping from number ofsegments, by [0-2, 2-4, 4-8, 8-16, 16-∞] to an integer ofrange [0, 4], then normalize it.

4.4. Pairwise Features

While color, segment and group features can model thecolor assignments according to the spatial features of theregion with unique color, they can not recover the strong

effect of one color group on another. So here, pairwisefeatures of adjacent color groups are introduced. Basedon that, we predict the distribution of color propertiesfor adjacent color groups. Further, background colorproperties are different from foreground parts in general,so in the pairwise features part, we focus on the pairs of thebackground with every foreground color groups. We havefollowing pairwise features: perceptual difference, relativedifference, relative saturation, and chromatic difference.

Perceptual difference. Perceptual difference is theEuclidean distance between two colors in L*a*b space andis the primary descriptor of ’contrast’ between two colorsthat we use in our model.

Relative Lightness. Relative lightness is the abso-lute difference between the L values of two colors inL*a*b space. This ’difference of intensities’ capturesanother significant type of contrast.

Relative Saturation. Relative saturation is the abso-lute difference between the saturation values from unaryfeatures of two colors. It helps capture whether two colorsshould be mutually saturated or not.

Chromatic Difference. Chromatic difference is thesquared fraction of perceptual distance. It measures thedifference between two colors after factoring out lightness.

4.5. Global Features - Compatibility

Beside unary and pairwise features, the overall colorharmony is also an important factor in the visual appeal-ingness of a picture. Global feature captures the colorcompatibility for the five-color theme.

The idea is from the paper (O’Donovan et al., 2011).In the paper, the authors examines three large dataset ofcolor themes, from Adobe Kuler, COLOURLovers, andAmazon Mechanical Turk. There are 46,137 themes inKuler, 178,086 in COLOURLovers, and Amazon Mechan-ical Turk dataset selected 10,743 Kuler themes with 40users rantings per theme, covering a range of poorly-ratedto highly-rated themes. Each color theme is convertedto a feature vector of dimension 334, and all features arenormalized to range [0, 1]. Each color theme also has arating from reviewers. The authors uses LASSO regressionto map from feature vector to rating.

minw,b

∑i

(wT yi + b− ri

)2+ λ||w||1

where w is the feature weights, y is feature vector, b is biasterm, and r is rating.

Submission and Formatting Instructions for ICML 2017

The LASSO regression results suggest which fea-tures are important in measuring color compatibility. Wepick 7 from the 334 features, whose weights are relativelylarge, showing below:

Mean Lightness. Positive, indicate a preference foroverall bright themes.

Max Lightness. Positive, indicate that users like darkercolors, as long as there is at least one bright color.

Max Min Difference in Lightness. Positive, indicatepreference of a spread of lightness.

Standard Deviation in Lightness. Negative, indicatepairing several bright and dark colors is a poor choice.

Standard Deviation in CIELab’s b dimension. Positive,indicate preference of similar colors instead of a transitionof different colors.

Max Min Difference in CIELab’s b dimension.Positive, high weight keeps the colors from becoming toosaturated.

Hue Entropy. Negative, penalty for having too manydistinct colors.

We adopted these features and together with the learnedweights to our model, showed in table below, to produce afinal compatibility score in range [1, 5].

Table 1. Weights for important color compatibility featuresFeature Name Feature Weights

Mean Lightness 0.0Max Lightness 0.04Diff Lightness 0.66

Stddev Lightness -0.5Stddev Lab b 0.43

Diff Lab b 0.20Hue Entropy -0.59

The global feature is calculated by dividing the compati-bility score by the maximum possible rating, and treat theresult as a probability.

φCompat(c1,...,c5) = ln (compat(c1, ..., c5)/5) (1)

4.6. Factor Graphs

We have already define unary, pairwise and global featurefunctions. Now we need to combine them into one scor-ing function that evaluates the overall quality of a patterncoloring.

Figure 5. A factor graph for an example template. Variable nodesare colored according to their corresponding color group in thepattern.c3 is the background color. c1, c2, c4and c5 are colors ofother groups. The connect background color c3 to another colorci, i 6= 3 to form a pairwise factor.

In this paper, we represent our model as a factor graph.A factor graph is a probabilistic graphical model that de-composes a complex probability distribution over multiplevariables into a set of smaller factors over subsets of thevariables. In our case, unary, pairwise and global featuresmake up the set of smaller factors.

In this paper we use C to represent color groups.Each group contains several segments, and these segmentsshould have the same color. Each color group has thefollowing form: Ci = (ci,P). Where ci is the color of thiscolor group and P is the spatial information given by thetemplate.

The factors connected to a single variable are derivedfrom our unary scoring functions; they includes the scorefor a color group and the scores for all segments in thatgroup:

FUnary(ci,P) = exp(φunary(ci,P) · wunary)

In this formula, wunary is model’s parameters for unaryfactors. φunary represents unary features.

The factors connecting two variables come from ourpairwise scoring functions and combine the scores for alladjacencies which involve segments from two differentcolor groups.

FPairwise(c3, ci,P) = exp(φpair(c3, ci,P) · wpair)

In this formula, c3 is the background color, c1, c2, c4 and c5are other groups’ colors. wpair is the model’s parametersfor pairwise factors.

Finally, the factor connected to all five color variables

Submission and Formatting Instructions for ICML 2017

enforces color compatibility:

FCompat(c1, ..., c5,P)= exp(φcompat(c1, · · · , c5,P) · wcompat)

The probability distribution encoded by this factor graphcan be shown in the following way:

p(c|P : w)

=1

Z(P, w)exp{FUnary + FPairwise + FCompat}

:=1

Z(P : w)

∏w∈W

exp(w · φw(c,P)))

Here, φw represents the features using the same weights w.

4.7. Weights Learning

The factor graph model we have defined is parameterizedby a vector of weights w. We set the weights such thatthe training examples have the highest probability underthe resulting model. We write the probability distributionencoded by the factor graph model as below:

p(c|P : w) =1

Z(P : w)

∏w∈W

exp(w · φw(c,P))

Then the log-likelihood of the weights given a dataset D ofpattern colorings is written as:

`(w : D) =∑

(P,c)∈D

(∑w∈W

w · φw(c,P)− lnZ(P : w))

The log-likelihood is maximized via gradient ascent:

∂

∂w`(w : D) =

∑(P,c)∈D

φw(c,P)− Ew[φw(c,P)]

However, it is intractable to calculate the expectationabove, since the probabilistic inference is an NP-hard prob-lem. According to (Lin et al., 2013), we use ContrastiveDivergence method to approximate the likelihood gradient:

CDkw(w : D) =

∑(P,c)∈D

(φw(c,P)− φw(c,P))

where c is the coloring obtained by running an MCMCchain for k steps from the initial coloring c. Then we canuse the above approximated gradient to perform gradientascent algorithm in order to update w. Here, we use theMetropolis-Hastings algorithm, using normal distributionas proposal function. We generate 1,000 samples with skip100 to approximate the derivation at each time.

4.8. Coloration

With the learned factor graph, now we are able to selectbest rating colorations for a given gray-scale template. Wealso use a MCMC algorithm — Metropolis-Hastings algo-rithm to generate colors for the given template. The dif-ference between here and weights learning part is that weonly apply MCMC to the three color features: lightness,saturation, and hue, with normal distribution for each fea-ture as the proposed distribution. All other features can beextracted from the given template and the color sampled.We feed these features to the factor graph and pick the bestrating color and template combination. Then we plot thedesignated color to corresponding color group in the tem-plate, and output the final colored picture.

4.9. Model Improvements - Constraints

When coloring some specific templates containing trees,flowers, sky, it is natural to prefer them to be colored green,red, and blue respectively. Although other coloration mightbe possible and creative, in general, it requires higher de-sign skills to make it harmonic with abnormal colors inusual patterns. So in our model, we apply soft color con-straints to prevent model from assigning abnormal colors tocertain patterns. In order to set up such constraints, we didsome modification during the process of generating colorsin testing part.

In usual cases without constraints, we apply MCMC meth-ods to three features of the color — lightness, saturationand hue by using normal distribution as the MCMC pro-posal. In the constraint cases, we change the proposal cor-responding to the hue, to a uniform distribution with spe-cific range centering on a designated value in order to makesure it will keep sampling reasonable hue values within thehue range of the fixed color. For example, when samplingfor leaves, the designated center hue value will be green.And the color generated will be green with slightly differ-ent hues.

In a formal expression,let xhue denote the hue variable inthe MCMC process.The way of sample new xhue is as fol-lows:xnewhue ∼ Unif(xhue − rangehalf , xhue + rangehalf )

5. Experiment Results5.1. Weights convergence

By using gradient ascent method to maximize the log-likelihood, we examine the norm of the gradient of weightsto show the convergence of weights learning. It is shownthat the general trend of the norm is decreasing. The rea-son why the norm does not converge to zero is due to theMCMC algorithm, we are not getting the exact gradient of

Submission and Formatting Instructions for ICML 2017

weights, the MCMC approximation may cause fluctuationto the final convergence results, which is shown in Figure(6).

Figure 6. the norm of the gradient of weights according to itera-tions

5.2. Factor Graph without Constraints

Figure 7. Left: input gray scale template Middle: Randomly col-ored Right: Colored patterns from our factor graph model

We trained our model using 256 pictures and input atemplate. The model generated a coloration for the giventemplate. As shown in Figure(7), the left picture isthe template, the middle picture is generated randomly,the right picture is the sample picture created by our model.

We can see that our model can successfully separatesall components in the graph. Colors in two close areasshare same saturation and the foreground color andbackground color are in harmony.

5.3. Factor Graph with Constraints

We will show how our model works with constraints setup. For example, for the left input gray scale template,the middle picture is the sample generated from the modelwithout constraints. Further, we can set up the leaf colorgroup to be green, our model predicts a colored pattern asthe right image. We can see that the leaf color group isindeed green now while remaining color are compatible.

Figure 8. Left: input template Middle: colored patterns withoutconstraints Right: colored patterns with constraints

6. ConclusionFactor graph is an efficient way to solve automatic coloringproblem, because factor graphs allow the problem to bedecomposed into groups of features and combines thefeatures in designed way. This decomposition to featurescorresponds well with the different color properties in apicture, which combining features together allow user todefine any rules.

It can be shown from our colored patterns resultsthat the features we used can extract the properties of thetraining patterns very well and the factor graph modeldid a great job in generating charming colorings. Thecombination of unary, pairwise and global compatibilityfeatures is comprehensive for the coloring model. Further,under constraints setting, the factor graph model can alsoassign personalized colorings.

Although to we authors, we think generated picturesfrom our model is better than random colorations, we don’thave a quantitative way to prove it. One way to prove itis to have reviewers review these pictures. If the averageratings of our model’s generated images are higher thanthat of randomly generated pictures and that of artistsgenerated pictures, then we can conclude that our modelcan replace the role of people in designing field. If theratings are lower than artists’ generated pictures, thenfurther research is needed to close the gap.

With that being said, we still believe that this auto-matic coloration model has extensive applications in reallife, either in pattern design field, or simply as inspirationsor suggestions in pattern colorations.

7. Future WorkIn this paper, we created a model that can generate 5-colorthemes for a given pattern template, considering the pat-tern’s segment features and color compatibility feature. Wefurther improve the model with soft constraints, such asleaf are set to be green. However, there are other fieldswhere this project can be further explored.

Submission and Formatting Instructions for ICML 2017

7.1. Number of colors constraints

For now, for simplicity, we only considered patterns that re-quire 5-color themes. However, if we want this automaticcoloration to be more applicable in real life, we have to re-lax this limit. To generate pictures in this kind of settings,we not only need a model that can calculate probability foreach color themes and patterns, but also a model that canautomatically separate an input gray-scale picture to seg-ments. To make the picture even more visually appealing,the model can classify segments with gradual changes, sothat the colors can have gradual changing effects. Thus re-laxing the 5-color limit will provide more intelligence tothe application.

7.2. 3D Automatic Coloration

One application of this automatic color decoration ishome design or building design, in which color and dis-tance factors have been considered a lot. To successfullyapply the project in related field, we have to be ableto convert the 2D model to 3D model. This involvesresearches on which are the important factors in deter-mining the spatial harmony in a 3D model. For example,usually people prefer lighter color in upper space anddarker color in bottom space. We can expand our modelto accommodate these information and produce 3D colortheme suggestions.

7.3. Object Recognition and Coloration

For now, our soft constraints is hard coded. For example,we found the color group number representing leaves,and hard set that color group to be in green color range.This prevents our generated picture from looking weird.However, we want to our model to be more intelli-gent by automatically detecting the leaves and assigningthe leaves with the color that most people paint leaves with.

As in paper (Zhen et al., 2012), they built a modelthat can automatically paint pictures using similar pictures,which are called reference pictures. In these pictures,each pixel is converted to feature vectors. The model usesK-nearest-neighbors to find pixel features in the templatethat are close to the pixel features in the reference picture.If the similarity is measured larger than 75% by thesimilarity metric, then the pixel will be color to the samecolor as the corresponding pixel in the reference picture.

Figure 9. Example of reference picture, input template, and out-put picture

We can adopt the main idea of the paper, while maintain adensity function for a certain pattern, since we have morethan one reference images in the training set. Besides learn-ing weights in the training model, each time we have tocompare the similarity between segments in template andcurrent training image. And final output picture will firstcolor segments that has been identified as similar to seg-ments in training pictures. Then with that as soft con-straints, use MCMC generate colors for rest segments andthen choose the top rated color themes.

ReferencesLevin, Anat, Lischinski, Dani, and Weiss, Yair. Coloriza-

tion using optimization. In ACM Transactions on Graph-ics (ToG), volume 23, pp. 689–694. ACM, 2004.

Lin, Sharon, Ritchie, Daniel, Fisher, Matthew, and Han-rahan, Pat. Probabilistic color-by-numbers: Suggestingpattern colorizations using factor graphs. ACM Transac-tions on Graphics (TOG), 32(4):37, 2013.

O’Donovan, Peter, Agarwala, Aseem, and Hertzmann,Aaron. Color compatibility from large datasets. ACMTransactions on Graphics (TOG), 30(4):63, 2011.

Zhen, Zhong, Yan, Gui, and Lizhuang, Ma. An automaticimage and video colorization algorithm based on patterncontinuity. In Audio, Language and Image Processing(ICALIP), 2012 International Conference on, pp. 531–536. IEEE, 2012.