Photogrowth: Non-Photorealistic Renderings Through AntPaintings

Penousal Machadomachado@dei.uc.pt

Luís Pereiralcper@student.dei.uc.pt

CISUC, Department of Informatics EngineeringPolo II of the University of Coimbra

3030 Coimbra, Portugal

ABSTRACTWe introduce photogrowth, an evolutionary approach to theproduction of non-photorealistic renderings of images. Thepainting algorithm – inspired by ant colony approaches –is described and explained, giving emphasis to its novel as-pects: the evolution of the sensory parameters of the ants;the production of resolution independent images; the render-ing lines of variable width. The experimental results high-light the range of imagery that can be evolved by the systemand show the potential of the approach for the productionof large-format artworks.

Categories and Subject DescriptorsI.2.2 [Artificial Intelligence]: Automatic Programming;I.3.3 [Computer Graphics]: Picture/Image Generation—Non-Photorealistic Rendering

General TermsAlgorithms

KeywordsEvolutionary Art, Non-Photorealistic Rendering, Ant Colony

1. INTRODUCTIONThe main goal of the research presented in this paper is the

creation of large-scale non-photorealistic renderings (NPRs)of input images. The predominant artistic sources of inspi-ration for this work are ornamentation techniques. From ascientific point of view, areas such as evolutionary NPR andartistic filter evolution are of particular relevance.

Our approach can be seen as the evolution of a filter thattransforms an input source image. The approach is inspiredon ant colony approaches: the trails of artificial ants are usedto produce an artistic rendering of the original. A interactiveGenetic Algorithm (GA) is used to evolve the parametersthat govern the behavior of ants’ species, allowing the user

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.GECCO’12, July 7-11, 2012, Philadelphia, Pennsylvania, USA.Copyright 2012 ACM 978-1-4503-1177-9/12/07 ...$10.00.

to guide the algorithm to areas of the search space thatshe/he finds promising.

The novel characteristics of our approach derive, directlyand indirectly, from the adoption of a scalable vector graph-ics, which contrasts with the pixel based approaches used inmost ant colony painting algorithms. This enables the cre-ation of resolution independent images and, as such, large-format artworks. The rendering algorithm represents thetrail of each ant through a continuous line of varying width,which contributes to the expressiveness of the artworks.

We begin with a short survey of related work. Next, in thethird section, we make a description of the photogrowth sys-tem, focusing on the behavior of the ants, on the evolution-ary algorithm, and on the previewing and rendering modes.In the fourth section we present experimental results, mak-ing a brief analysis. Finally, we draw some conclusions anddiscuss aspects to be addressed in future work.

2. STATE OF THE ARTIn this section we make a survey of related works, focus-

ing on systems that use artificial ants for image generationpurposes and on systems where evolutionary computation isemployed for NPR purposes.

Tzafestas [20] presents a system where artificial ants pick-up and deposit food, which is represented by paint, andstudies the self-regulation properties and complexity of thesystem and resulting images. Ramos and Almeida [18] ex-plore the use of ant systems for pattern recognition purposes.The artificial ants successfully detect the edges of the imagesproducing stylized renderings of the originals and smoothtransitions between different images. The artistic potentialof these approaches is explored in later works (e.g. [17]) andthorough his collaboration with the portuguese artist LeonelMoura, which resulted in several robotic swarm drawings(see e.g. [14]). Urbano [21, 22, 23, 24] presented severalmulti-agent systems based on artificial ants.

Aupetit et al. [1] present an interactive evolutionary com-putation system for the creation of ant paintings. They em-ploy a GA to evolve parameters of the rules that govern thebehavior of the ants. The artificial ants deposit paint onthe canvas as they move thus producing a painting. In alater study, Monmarche [13] refines this approach exploringdifferent rendering modes. Greenfield [5] presents an evo-lutionary approach to the production of ant paintings andexplores the use of behavioral statistics of the artificial antsto automatically assign fitness. In [6] Greenfield adopted amultiple pheromone model where ants movements and be-haviors are influenced (attracted or repelled) by both an en-

vironmentally generated pheromone and an ant generatedpheromone.

The use of evolutionary algorithms to create image filtersand NPRs of source images has been explored by several re-searchers. Focusing on the works where there was an artisticgoal, we can mention the research of: Neufeld and Ross ([19,15]), where Genetic Programming (GP, [8]), multi-objectiveoptimization techniques, and an empirical model of aesthet-ics are used to automatically evolve image filters; [9], whichevolved live-video processing filters through interactive evo-lution; [10], where GP is used to evolve image coloring fil-ters from a set of examples; [25], which employs GeneticAlgorithms (GAs) to evolve filters that produce images thatmatch certain features of a target image; Collomosse ([4, 3,2]), which uses image salience metrics to determine the levelof detail for portions of the image, and GAs to search forpainterly renderings that match the desired salience maps;[7] uses GP to evolve procedural textures for 3D objects; [11]employ GP to evolve assemblages of 3D objects that are anartistic representation of an input image.

3. PHOTOGROWTHPhotogrowth is composed of three main modules:

1. Evolutionary engine;

2. Previewing engine;

3. Rendering engine;

The evolutionary module is an interactive GA that allowsthe evolution of a series of parameters that govern the be-havior of the ants. The previewing module is responsiblefor the production of the ant paintings during the evolution-ary runs. The rendering engine is typically used offline toproduce high quality renderings of specific ant paintings.

A graphic user interface gives access to these modules. Itis composed of two windows (see Fig. 1: one depicts thepaintings produced by the ants of the current populationand allows the user to assign fitness to them, indicating avalue between 0 and 10); the other, allows the user to nav-igate through populations, change the input image, inspectand edit the genotype of a given individual, save and loadexperiments and individuals, invoke the rendering engine,etc.

The following sections present the photogrowth system.We begin by describing the behavior of our painting ants.Next we present the evolutionary algorithm, focusing on therepresentation and genetic operators. Finally we highlightthe differences between the previewing and rendering modes.

3.1 Painting AntsOur painting ants live on the 2D world provided by the

input image and they paint on a painting canvas that is ini-tially empty (i.e., black in the experiments reported in thispaper). Both living and painting canvas have the same di-mensions and the ants move simultaneously on both canvas.The painting canvas is used exclusively for depositing ink. Ithas no interference with the behavior of the ants. They sharethese worlds with other artificial ants of the same species.Each ant has a position, color, deposit transparency and en-ergy, all the remaining parameters are shared by the entirespecies. If the energy of an ant is bellow a given energythreshold it dies, if is is above a given threshold it generatesoffspring.

Figure 2: An ant with five sensory vectors.

The luminance of an area of the living canvas representsthe available energy at that point. Therefore, ants may gainenergy by traveling through light areas. The energy of anant is updated as follows:

energy = (energy + b(x, y) ∗ gain) ∗ decay (1)

where b(x, y) is a function that returns the luminance (in the[0,1] interval) of the area where the ant is placed, gain is ascalar that represents the energy gain rate, and decay is ascalar in the [0, 1[ interval that represents the energy decayrate. The energy consumed by the ant is removed from theenvironment, as will be explained later in detail.

The ants’ movement are determined by how they reactto light. Each ant senses the environment by “looking” inseveral directions (see Fig. 2). In the experiments describedin this paper we use 10 sensory vectors, each vector hasa given direction relative to the current direction of the antand a length. The sensory organs return the luminance valueof the area where each vector ends. To update the positionof an ant one calculates: the sum of the sensory vectorsdivided by their norms, multiplied by the luminance of theirend point and by the weight the ant gives to each sensor,the result is multiplied by a scaling scalar that representsthe ant’s base speed:

∆~p = vel ∗10X

i=1

~vi

|~vi|∗ b((x, y), ~vi) ∗ wi (2)

where, ∆~p is the displacement vector, (x, y) is the currentposition of the ant, vel is the ant’s base velocity; ~vi is thesensory vector i; b((x, y), ~vi) a function that returns the lu-minance of the area at coordinates (x, y) + ~vi; wi is theweight associated with ~vi.

Subsequently, to represent inaccuracy of movement andsensory organs, the direction ∆~p is perturbed by the additionPerlin noise [16] to its angle. Therefore, the position of theant at instant t + 1 is given by the following formula:

(x, y)t+1 = (x, y)t + perlinnoise(t, ∆~p) (3)

The ant simulation algorithm follows the following steps:

1. Initialization: n ants are placed on the canvas on pre-established positions; Each ants assumes the color ofthe area where it was placed; Their energy and deposittransparencies are initialized using the species param-eters;

Figure 1: Screenshot of the graphic user interface. Control panel on the left and current population of antpaintings on the right.

2. For each ant:

(a) Update the ant’s energy following formula 1;

(b) Update the energy of the environment;

(c) Place ink on the painting canvas;

(d) If the ant’s energy is bellow the death thresholdremove the ant from the colony;

(e) If the ant’s energy is above the reproduction thresh-old generate an offspring; The offspring assumesthe color of the position where it was createdand a percentage of the energy of the progeni-tor (which loses this energy); The offspring in-herits the velocity of the parent, but a pertur-bation is added to the angular velocity by ran-domly choosing an angle between descvelmin anddescvelmax (both values are species’ parameters);Likewise, the deposit transparency is inheritedfrom the progenitor but a perturbation is includedby adding a randomly choosen a value betweendtranspmin and dtranspmax;

(f) Update ant’s position following formulas 2 and 3;

3. Repeat since 2 until no living ants exist;

Steps (b) and (c) require further explanation. The con-sumption of energy of the environment is attained by draw-ing on the living canvas a black circle of size equal to energy∗consrate of given transparency. consrate and constrans areparameters of the species. In previewing mode ink is de-posited on the paining canvas by drawing a circle of thecolor of the ant – which is attributed when the ant is born –with a size given by energy ∗ depositrate and of given trans-parency. depositrate is a species parameters, the deposittransparency is a parameter of the ant.

3.2 Evolutionary EngineAn interactive GA is used to evolve the ant species’ pa-

rameters. The genotypes are tuples of floating point num-bers which encode the parameters of the ant species. Thesize of the genotype depends on the experimental settings.Table 1 presents an overview of the encoded parameters. Weuse a two point crossover operator for recombination pur-poses and a Gaussian mutation operator. We employ tour-nament selection and an elitist strategy, the highest rankedindividuals proceed, unchanged, to the next population.

3.3 Preview and Rendering ModesWe use scalable vector graphics in both preview and ren-

dering modes. This allows the generation of resolution inde-pendent ant paintings, which, to the best of our knowledge,is a novel feature of our approach. As previously stated,in preview mode, the ants draw circles while they move.In rendering mode the ants behave similarly in every way.However, when the painting is finished, the trail of each antis converted into a single line of variable width. This is at-tained by calculating external tangents to each pair of con-secutive circles drawn by the ant, drawing polygons amongthese points and performing a shape union between all cir-cles and polygons belonging to the same trail. In Fig. 3 wepresent a graphical portrayal of this process.

The differences between preview and rendering modes areusually not visible on screen, however, they have a significantimpact when the artworks are printed or at high zoom rates.Fig. 4 highlights the differences.

It is also important to notice that the color of an ant isdetermined at birth, thus the ants may carry this color toareas of the screen that possess different colors in the orig-inal image, making the ant paintings further deviate fromthe original image (see Fig. 7). As a consequence of this

Table 1: Parameters encoded by the genotypeName # Commentsgain 1 scaling for energy gainsdecay 1 scaling for energy decay

consrate 1 scaling for size of circles drawnon the living canvas

constrans 1 transparency of circles drawnon the living canvas

depositrate 1 scaling for size of circles drawnon the painting canvas

deposittransp 1 base transparency of circlesdrawn on the painting canvas

dtranspmin 1 limits for perturbation ofdeposit transparencywhen offsprings aregenerated

dtranspmax 1

initialenergy 1 initial energy of the startingants

deaththreshold 1 death energy tresholdbirththreshold 1 generate offspring energy

thresholddescvelmin 1

limits for perturbation ofangular velocity whenoffsprings are generated

descvelmax 1

vel 1 base speed of the antsnoisemin 1 limits for the perlin noise

generator functionnoisemax 1initialpositions 2 ∗ n initial coordinates of the n

ants placed on the canvassensoryvectors 2 ∗m direction and length of the m

sensory vectorssensoryweights 2 ∗m direction and length of the m

sensory vectors

Figure 3: On the top, three circles belonging to thesame trail and the polygons produced by tracinglines between the points of the external tangentsto each consecutive pair. On the bottom, the shaperesulting from the union of the three circles and twopolygons

Figure 4: Details of images produced in pre-view(top) and rendering (bottom) modes depictingthe differences in the rendering of the ants’ trails.

coloring method, trails of progenitor ants become interlacedwith those of descendants.

4. EXPERIMENTAL RESULTSThe results presented in this section were obtained by con-

ducting a series of informal experiments with the followingexperimental setup: Population Size = 15; Tournament size= 3; Crossover probability = 0.9; Mutation Probability =0.1 (per gene); Initial Position of the ants = center of thecanvas; Initial number of ants = 1. Thus, when the drawingstage starts each ant specie is represented by a single ant.However, due to the abundance of energy this ant tends togenerate offspring quickly. The length of each evolutionaryrun varied, typical runs had 30 to 40 generations.

The analysis of the experimental results of evolutionaryart systems, specially user driven ones, is subjective by na-ture. As such, more than presenting measures of perfor-mance that would be meaningless when considering the goalsof our system, we aim to:

1. Convey the overall feeling of the user experience whenworking with photogrowth through the presentation ofdifferent steps of the evolutionary process;

2. Highlight the different types of imagery that can beevolved with the system;

3. Show that the application of the same individual (antcolony) to different input images produces stylisticallysimilar renderings;

In Fig 5 we present snapshots of the 1st, 2nd, 20th and40th population of a typical evolutionary run. As it can beobserved, most of the individuals of the initial populationfail to portray the input image. In the second population,the percentage of images that depict the input image sig-nificantly increases. Nevertheless, the overall quality of theimages is relatively low, in the sense that they do not conveythe aesthetic preferences of the user. In the 20th populationwe observe the emergence of elliptical and organic ant trails

Figure 5: On the top row, snapshots of the 1st and 2nd populations of an evolutionary run. On the middlerow, the 20th population. On the bottom row, snapshot of the 40th population.

Figure 6: NPRs of two different images using the same evolved ant species.

(e.g., 3rd and 9th images of this population), a trait thatwill be favored by the user in subsequent runs. The 40thpopulation presents an wide variety of images of contrast-ing line widths and trail curvature scales. The organic anttrails that emerged on the 20th population are still presenton most of these images (although in some cases this is onlyobservable at higher resolutions). At this generation we alsoobserve the emergence of “abstract” portrayals of the origi-nal image (see figure 14th of this population). Progress fromthis point onwards tends to be slow and unsteady since, likein other interactive evolutionary art approaches, the usertends to often rewards novelty, resulting in frequent changesof selection criteria.

Fig. 6 presents the results of applying the same evolvedant species to two different images. As it can be observedthe distinguishing traits of the ant colony are preserved, re-sulting in renderings that share stylistic characteristics.

In Fig. 7 we present two color images produced by differ-ent ant species to highlight the range of imagery that canbe produced by photogrowth. The top image explores theuse of thick organic lines of varying width to produce anabstract rendering of the original image. The bottom imageis composed of thiner lines which become more intertwined.Both images take advantage of the coloring approach – thecolor of the ants is determined at birth – to transport thecolor of a region of the image to surrounding areas. Thiseffect is particularly noticeable in the bottom image.

5. CONCLUSIONSWe presented photogrowth, a novel evolutionary approach

to the production of NPRs of input images. The distinguish-ing characteristics of the system rest on the evolution of thesensorial organs of the ants, on the production of a scalablevector graphics output and on the rendering engine whichrepresents the ant trails through continuous lines of variablewidth. The experimental results illustrate the range of im-agery that can be produced by the system and demonstrateits adequacy for the production of large-format artworks.

Future research will focus on: (i) the improvement of theefficiency of the rendering engine – the production of an art-work in rendering mode can take from hours to days – whichcan be easily attained through parallelization since each anttrail can be independently processed; (ii) the developmentof automatic fitness assignment schemes. For this purpose,and considering the nature of our painting algorithm, we areparticularly interested in the use of statistics of the behav-ior of the ants to assign fitness [5] and in the combinationof this approach with ones that take into account the com-plexity and fractal dimension of the produced artwork (seee.g. [12]).

6. ACKNOWLEDGMENTSThis research is partially funded by the Portuguese Foun-

dation for Science and Technology. Project PTDC/EIA–EIA/115667/2009.

7. REFERENCES[1] S. Aupetit, V. Bordeau, N. Monmarche, C. Slimane,

and G. Venturini. Interactive Evolution of AntPaintings. In IEEE Congress on EvolutionaryComputation, volume 2, pages 1376–1383, Canberra,8-12 december 2003.

Figure 7: Examples of evolved color images.

[2] J. Collomosse. Evolutionary search for the artisticrendering of photographs. In J. Romero andP. Machado, editors, The Art of Artificial Evolution:A Handbook on Evolutionary Art and Music, pages39–62. Springer Berlin Heidelberg, 2007.

[3] J. P. Collomosse. Supervised genetic search forparameter selection in painterly rendering. InApplications of Evolutionary Computing,

EvoWorkshops 2006, pages 599–610, Budapest,Hungary, 2006.

[4] J. P. Collomosse and P. M. Hall. Genetic paint: Asearch for salient paintings. In Applications ofEvolutionary Computing, EvoWorkshops 2005, pages437–447, Lausanne, Switzerland, 2005.

[5] G. Greenfield. Evolutionary methods for ant colonypaintings. In F. Rothlauf, J. Branke, S. Cagnoni,D. W. Corne, R. Drechsler, Y. Jin, P. Machado,E. Marchiori, J. Romero, G. D. Smith, andG. Squillero, editors, Applications of EvolutionaryComputing, EvoWorkshops2005: EvoBIO,EvoCOMNET, EvoHOT, EvoIASP, EvoMUSART,EvoSTOC, volume 3449 of LNCS, pages 478–487,Lausanne, Switzerland, 30 Mar.-1 Apr. 2005. SpringerVerlag.

[6] G. Greenfield. Ant Paintings using a MultiplePheromone Model. In Bridges, London, England, 2006.

[7] A. Hewgill and B. J. Ross. Procedural 3d texturesynthesis using genetic programming. COMPUTERSAND GRAPHICS, 28:569–584, 2003.

[8] J. R. Koza. Genetic Programming: On theProgramming of Computers by Natural Selection. MITPress, Cambridge, MA, 1992.

[9] M. Lewis. Aesthetic video filter evolution in aninteractive real-time framework. In Applications ofEvolutionary Computing, EvoWorkshops2004:EvoBIO, EvoCOMNET, EvoHOT, EvoIASP,EvoMUSART, EvoSTOC, volume 3005 of LNCS,pages 409–418, Coimbra, Portugal, 5-7 Apr. 2004.Springer Verlag.

[10] P. Machado, A. Dias, and A. Cardoso. Learning tocolour greyscale images. The Interdisciplinary Journalof Artificial Intelligence and the Simulation ofBehaviour – AISB Journal, 1(2):209–219, 2002.

[11] P. Machado and F. Graca. Evolutionary pointillistmodules: Evolving assemblages of 3d objects. InM. Giacobini, A. Brabazon, S. Cagnoni, G. D. Caro,R. Drechsler, A. Ekart, A. Esparcia-Alcazar,M. Farooq, A. Fink, J. McCormack, M. O’Neill,J. Romero, F. Rothlauf, G. Squillero, S. Uyar, andS. Yang, editors, Applications of EvolutionaryComputing, EvoWorkshops 2008: EvoCOMNET,EvoFIN, EvoHOT, EvoIASP, EvoMUSART,EvoNUM, EvoSTOC, and EvoTransLog, Naples, Italy,March 26-28, 2008. Proceedings, volume 4974 ofLecture Notes in Computer Science, pages 453–462.Springer, 2008.

[12] P. Machado, J. Romero, and B. Manaris. Experimentsin computational aesthetics: An iterative approach tostylistic change in evolutionary art. In J. Romero andP. Machado, editors, The Art of Artificial Evolution:A Handbook on Evolutionary Art and Music, pages381–415. Springer Berlin Heidelberg, 2007.

[13] N. Monmarche, I. Mahnich, and M. Slimane. Artificialart made by artificial ants. In J. Romero andP. Machado, editors, The Art of Artificial Evolution:A Handbook on Evolutionary Art and Music, pages227–247. Springer Berlin Heidelberg, 2007.

[14] L. Moura. Swarm paintings – non-human. InARCHITOPIA Book, Art, Architecture and Science,pages 1–24. 2002.

[15] C. Neufeld, B. Ross, and W. Ralph. The evolution ofartistic filters. In J. Romero and P. Machado, editors,The Art of Artificial Evolution: A Handbook onEvolutionary Art and Music, pages 335–356. SpringerBerlin Heidelberg, 2007.

[16] K. Perlin. An image synthesizer. In P. Cole,R. Heilman, and B. A. Barsky, editors, Proceedings ofthe 12st Annual Conference on Computer Graphicsand Interactive Techniques, SIGGRAPH 1985, pages287–296. ACM, 1985.

[17] V. Ramos. On the implicit and on the artificial -morphogenesis and emergent aesthetics in autonomouscollective systems. In ARCHITOPIA Book, Art,Architecture and Science, pages 25–57. 2002.

[18] V. Ramos and F. Almeida. Artificial ant colonies indigital image habitats - a mass behaviour effect studyon pattern recognition. In In Dorigo, M., Middendorf,M., Stuzle, T. (Eds.): From Ant Colonies to ArtificialAnts - 2 nd Int. Wkshp on Ant Algorithms, pages113–116, 2000.

[19] B. J. Ross, W. Ralph, and Z. Hai. Evolutionary imagesynthesis using a model of aesthetics. In G. G. Yen,S. M. Lucas, G. Fogel, G. Kendall, R. Salomon, B.-T.Zhang, C. A. C. Coello, and T. P. Runarsson, editors,Proceedings of the 2006 IEEE Congress onEvolutionary Computation, pages 1087–1094,Vancouver, BC, Canada, 16–21 July 2006. IEEE Press.

[20] E. Tzafestas. Integrating drawing tools withbehavioral modeling in digital painting. InS. Ghandeharizadeh, S.-F. Chang, S. Fischer, J. A.Konstan, and K. Nahrstedt, editors, ACM MultimediaWorkshops, pages 39–42. ACM Press, 2000.

[21] P. Urbano. Playing in the pheromone playground:Experiences in swarm painting. In F. Rothlauf,J. Branke, S. Cagnoni, D. W. Corne, R. Drechsler,Y. Jin, P. Machado, E. Marchiori, J. Romero, G. D.Smith, and G. Squillero, editors, EvoWorkshops,volume 3449 of Lecture Notes in Computer Science,pages 527–532. Springer, 2005.

[22] P. Urbano. Consensual paintings. In F. Rothlauf,J. Branke, S. Cagnoni, E. Costa, C. Cotta,R. Drechsler, E. Lutton, P. Machado, J. H. Moore,J. Romero, G. D. Smith, G. Squillero, and H. Takagi,editors, EvoWorkshops, volume 3907 of Lecture Notesin Computer Science, pages 622–632. Springer, 2006.

[23] P. Urbano. Mimetic variations on stigmergic swarmpaintings. In N. Monmarche, E.-G. Talbi, P. Collet,M. Schoenauer, and E. Lutton, editors, ArtificialEvolution, volume 4926 of Lecture Notes in ComputerScience, pages 62–72. Springer, 2007.

[24] P. Urbano. The t. albipennis sand painting artists. InC. D. Chio, A. Brabazon, G. A. D. Caro, R. Drechsler,M. Farooq, J. Grahl, G. Greenfield, C. Prins,J. Romero, G. Squillero, E. Tarantino, A. Tettamanzi,N. Urquhart, and A. S. Etaner-Uyar, editors,EvoApplications (2), volume 6625 of Lecture Notes inComputer Science, pages 414–423. Springer, 2011.

[25] C. Yip. Evolving Image Filters. Master’s thesis,Imperial College of Science, Technology, and Medicine,September 2004.