agents with faces - what can we learn from lego minfigures?miniﬁgure faces. over the years, lego...

Agents With Faces - What Can We Learn FromLEGO Minifigures?

Christoph BartneckHIT Lab NZ, University of Canterbury,

PO Box 4800, 8410 Christchurch, New ZealandEmail: [email protected]

Mohammad ObaidHIT Lab NZ, University of Canterbury,

PO Box 4800, 8410 Christchurch, New ZealandEmail: [email protected]

Karolina ZawieskaIndustrial Research Institute for Automation and Measurements – PIAP,

Al. Jerozolimskie 202, 02-486 Warsaw, PolandEmail: [email protected]

Abstract—Emotional facial expressions are essential foragents. The LEGO company developed hundreds of facial ex-pressions for their Minifigures, which are often the centerpieceof LEGO construction. We investigate and present a summary ofthe development of the facial expression for all LEGO Minifiguresthat were released between 1975 and 2010. Our findings arebased on several statistical tests that are performed on datagathered from an online questionnaire. The results show thatthe LEGO company started in 1989 to dramatically increase thevariety of facial expressions. The two most frequent expressionsare happiness and anger and the proportion of happy faces isdecreasing over time. Through a k-cluster analysis we identifiedsix types of facial expression: disdain, confidence, concern, fear,happiness, and anger. Our cluster analysis shows that toy designhas become a more complex design space in which the imaginaryworld of play does not only consist of a simple division of goodversus evil, but a world in which heroes are scared and villainscan have superior smile. In addition we tested if the perceptionof the face changes when the face is presented in the context of acomplete Minifigure. The impression of anger, disgust, sadnessand surprise were significantly influenced by the presence ofcontext information. The distinctiveness of the faces was, however,not significantly improved. The variation in skin color did also notchange the perception of the Minifigure’s emotional expression.

I. INTRODUCTION

One of the application of agents, both screen based androbotics, is entertainment. Agents are of major importancein computer games and their facial expressions contributesignificantly to their success. But also robots, such as Sony’sAibo, have a clear goal to play with users. Playing is a verypopular activity for children and adults. It is to some degreesurprising that there still is a lot of debate about its scientificdefinition [1]. The role that play has in the development ofchildren has been studied from different perspectives. Mostscholars agree on the crucial importance of play not onlyfor developing children wellbeing but also their cognitive andemotional skills, regardless the variety of forms that play andtoys can take. Play, including playing with objects, is seen as anactivity that helps children to learn [2]. It is through pretendplay that children develop the capacity of abstract thought,i.e. thinking about symbols and meanings independently ofthe objects they represent [3]. Moreover, play allows childrenlearning to practice adult roles and decision-making skills as

well as work in groups and resolve conflicts [4].

From the historical perspective play might be treated as acultural practice that is being influenced by societal processesand technological innovations. The way toys are produced andconsumed as well as the way of thinking about childhood havechanged significantly over the centuries leading to the current“culture of the child” [5], [6].

A discussion about the relationship between playing withspecific toys and intellectual and emotional development isan open research question and has not reached a conclusion.It has been proved that toys might help learning, especiallythose designed for educational purposes, like LEGO bricks[7]. However, few studies have shown that some toys may havea negative impact, in particular on very young children (5-8years old). For example, research findings on the Barbie dollhave shown that playing with very thin dolls can cause girls’unhappiness with their bodies [8]. It is also an element of thebroader question of the gender bias in toys [9]. LEGO productscombine learning with playing but also raise questions aboutthe role of the design of toys and its impact on children.

The Danish company LEGO is one of the biggest toy man-ufacturers. Company founder Ole Kirk Kristiansen producedwooden toys as early as the 1930s and plastic toys startingin 1947 [10]. The LEGO brick was first patented in 1958 inDenmark [11] and in the following years across Europe and theUS. A well written summary of the LEGO company’s historyis available [5]. Today, LEGO bricks are sold in more than130 countries and in 2010 alone LEGO produced more than36 billion bricks [12]. On average, every person on earth ownsaround 75 bricks. LEGO is popular with children and adults.Many people never loose their fascination for LEGO and ahuge Adult Fan Of LEGO (AFOL) community has emergedover the years. Several books about the AFOL culture havereflect on this culture and the ideas of LEGO [13], [14].

The centerpiece of any LEGO set has to be the LEGOMinifigure (see Figure 1). The Minifigure is meticulouslyplaced within any building or vehicle at the end of con-struction. The Minifigure enables children to populate theirworlds with agents. They are no longer constraints to playwith objects, such as cars and houses, but they can putthemselves into these worlds through the Minifigure. They can

Bartneck, C., Obaid, M., & Zawieska, K. (2013). Agents with faces - What can we learn from LEGO Minfigures. Proceedings of the 1st International Conference on Human-Agent Interaction, Sapporo pp. III-2-1.

play roleplaying games and explore human relationships.

Fig. 1. A LEGO Minifigure

The Minifigure was first introduced in 1975 and refinedin 1978. The patent on this iconic design was granted in1979 [15]. The Minifigures soon became a grant success witharound 4 billion sold so far. The Minifigure has since thenbeen extended and modified [16]. One of the first changeswas the replacement of the torso stickers with prints that weremade directly onto the plastic. The stickers could come off dueto normal wear and the aging of the glue. In 1989 differentdesigns for the facial expression became available [16]. Untilthen, every Minifigure had the same enigmatic smile. Now,Minifigures could also be angry or scared. Including ethnicelements further extended the variety of faces. The Indiansin the Wild West theme made a start with distinct faces.They were the first faces that included a nose. In 2003 moreskin colors were introduced within the NBA theme. Thepopular basketball player Shaquille O’Neal was portrait ina natural dark brown skin color. This trend was expandedin the licensed themes, such as Harry Potter in 2004. Harrywas given a more natural skin color to better represent theactor Daniel Radcliffe. Further innovations in the Harry Pottertheme were the introduction of the double-sided heads. TheQuirell Minifigure was the first to have two face printed on thehead [16]. Rotating the head can quickly change the face of aMinifigure. The licensed themes have become a major part ofthe LEGO world with the Star Wars theme taking the leadingrole. The Star Wars Minifigures have caught the attention ofmany collectors and guides have been published [17].

The Minifigure also grew out of the LEGO sets. Already in1982 Minifigure key rings were introduced [18]. Minifiguresare also part of chess games, LED flashlights and books.Naturally they are also the main characters for most LEGOcomputer games. In 2010 LEGO introduced the independentMinifigure theme. Minifigures are now available that are no

longer part of any other set. They are marketed as collectableitems. Each series consists of 16 different Minifigures that areindividual sold in sealed and unmarked bags. The themes inwhich the LEGO company released its sets and Minifigures canbe classified by the Systema MinfiguræTaxonomy (see figure2).

The vast use and popularity of LEGO has motivated us toinvestigate how the LEGO Minifigures have evolved over thepast 35 years (1975-2010). In particular, this paper addressesthe users’ perception of the facial expressions on the LEGOMinifigure faces. Over the years, LEGO produced face bricksthat map the different facial expression states and facial exag-gerations in the style of cartoon. In this context, a facial cartoonexaggerates face features for a comical effect, and can create anentertaining, humorous, and cartoon-like description of a face.The head parts are mainly exaggerated to produce the cartoon-like facial effects that include the nose, eyes, eyebrows, lips,hair and ears. As LEGO bricks are considered toys, the useof a cartoon like exaggeration plays an important role in theLEGO construction, as it brings together a good entertainmentformat.

The work presented in this paper can lead other researchersin the field of understanding the science of play to investigatefurther the influence the LEGO Minifigures’ facial appearancehave on LEGO users over time. We believe that the extensiveand elaborate designs of faces on LEGO Minifigures can alsoinform the designers of other agents, such as computer gamecharacters and robots. The LEGO company has developedhundreds of designs and can therefore be considered one ofthe most extensive set of agent faces.

A. Facial Expressions of Emotions

[19] defines the bases of human emotions to involve “phys-iological arousal, expressive behaviors, and conscious experi-ence”. [20] proposed the following classifications: emotionsas expressions, emotions as embodiments, cognitive theoriesof emotions, emotions as social constructs and neural basisof emotions. Moreover, due to the complexity of definingemotions, [20] gave a comprehensive definition of emotionsas follows: “emotions are constructs (i.e. conceptual quantitiesthat cannot be directly measured) with fuzzy boundaries andwith substantial variations in expression and experience”. Inthe context of our study, we focus on the facial expression ofemotion, which is an expressive behavior that is triggered on anindividual’s face, due to the internal feeling (or emotional sate),and conveyed to the observer. Several researchers revealed thatfacial expressions are universal across cultures such as thework by [21], [22], [23]. The most widely used definition ofuniversal facial expression is defined by [24], and they are:disgust, sadness, happiness, fear, anger, surprise. In addition,other work, in psychology, addressed the importance of theintensity level of the facial expression of emotions, such asthe work by [25]. She studied facial expressions of emotionsbased on different intensity levels of Activation (arousal level,and it is expressed on face) and Evaluation (agreement level,and it is expressed through internal feelings). A number ofresearchers [26], [27] have used her findings to map differentintensities of basic facial expressions of emotion to the faceof virtual agents.


Fig. 2. Systema MinfiguræTaxonomy for the years 1975-2010


Moreover, facial expressions relate not only to the waypeople express emotions but also to how they interpret themwhile expressed by others. An attempt to understand the latter,for example, is an area of research in the field of AffectiveComputing (AC), which aims to detect the basic emotions fromthe face; the results can be applied in different areas, amongwhich animation, virtual humans and robotics [20], [28].

In this paper we present a study that is focused on in-vestigating how users observe the iconic representations ofthe facial expressions of emotions conveyed by the LEGOminifigures over the years. We allow participants to not onlydefine the observed emotional facial expression of the LEGOminifigures based on the basic universal emotions, but alsowith different intensities of the facial expressions.

Research in the field of Design & Emotions focuses on“understanding the emotions of product users, and on thedevelopment of tools and techniques that facilitate an emotion-focused design process” [29] while self-reports are used to“assess respondents behaviors, attitudes and subjective experi-ences, like moods, emotions or pain [30]. However, we invitedparticipants to evaluate LEGO facial expressions and nottheir own emotional reactions or preferences towards LEGOminifigures. Our research methods therefore takes a slightlydifferent approach than the established Design & Emotionresearch process, although a certain overlap certainly exists.

The limitation of the methodology we used lies in speci-ficity of questionnaires and the Likert-type scale: a predefinedset of answers does not allow participants expressing a fullrange of opinions. Nevertheless, in our opinion the use ofquestionnaires based on labels is a suitable and widely usedresearch technique to study six basic facial expressions [31],[32]. LEGO minifigures by definition provide a simplifiedrepresentation of human-like emotions and an in-depth analysisof all possible perceptions of LEGO facial expressions goesbeyond the scope of this study.

B. Design

The Minifigures consist of a head, torso, arms, hands, hipand legs (see Figure 3). The Minifigure has seven degrees offreedom and is exactly four standard bricks tall, which is equalto 4.1mm. A Minifigure can have accessories on its head, suchas hair, helmets and hats. Accessories are also often foundaround the neck, such as capes, or under the feed, such asflippers. Many Minifigures also hold items in their hands, suchas swords, tools and books. At times, hands, arms and legs arereplaced by special items, such as hooks and wooden legs.

The different parts of a Minifigure can be made of differentcolored plastics and prints can be made on the head, torso,arms, hip and legs. There are a great number of possibilitiesto combine the parts, which allows LEGO to provide anenormous variety of Minifigures. Two Minifigures may, forexample, only differ by the face that is printed on their head.

The face of the Minifigure is of particular importance,since it gives the strongest indicator of the emotional stateof the character. People both consciously and subconsciouslyuse facial expressions to communicate their emotions andintentions through variations in gaze direction, voice toneand gesture speed. Ekman showed that expressing emotions

Fig. 3. Anatomy of a LEGO Minifigure

through the face is a natural activity for humans and that ittakes considerable effort to mask them [24]. There has alsobeen a considerable debate on how much the context in whichan emotion appears influences its perception. Carroll andRussell pointed out that situational information does indeedinfluence how a face is perceived [33]. This result is of interestto the design of Minifigures, since the same head can becombined with different bodies.

For the first eleven years, only one smiley face wasproduced, but since then the number of different faces seemto have increased and also the themes that LEGO is producingsubjectively appear to become increasingly aggressive. TheBionicle theme could be the scariest theme at this point intime The Minifigures might not yet be as aggressive as thecharacters in the Bionicle theme, but skeleton warriors are alsoin their repertoire. In this study, we are trying to address thefollowing research questions:

1) What emotions do the face in the LEGO Minifigureexpress?

2) How did the emotional expression of the faces changeover time?

3) What influence does the context of the whole Minifig-ure have on the perception of its face?

II. METHOD

A. Setup

We photographed all the 3655 Minifigures that were re-leased between 1975 and 2010. We identified 628 differentheads and cut them out from the photographs. These 628photos of the faces were the basis of our experiment. Forheads that had two faces printed on it, we randomly selectedeither the front or the back face. This allowed us to have onlyone representative face per figure and it was not necessary toincrease the already large set of stimuli. We looked up the


year in which the head was first introduced from a database ofMinifigures [34]. We then randomly selected 100 heads. Forthese heads we randomly selected an associated Minifigure.We manually checked these Minifigures and six of them werenot suitable for our experiment, since the face was not clearlyvisible on the Minifigure. A helmet, for example, covered alarge portion of the face.

We created an online questionnaire that showed all the 628heads and the 94 Minifigures. The Participants were asked torate the emotional expression based on the scale shown inFigure 4. We utilized Amazon Mechanical Turk (MT) 1 torecruit participants and to administer the questionnaire. It hasbeen shown that results obtained through MT are comparableto those obtained through the conventional method of question-naires [35]. There is no substantial difference between resultsobtained through an online questionnaire and results receivedthrough MT.

B. Measurements

Each face was rated on five point Likert scales rangingfrom very weak to very intense. The selection categories of thefacial appearance are based on the work of Paul Ekman [24],who grouped the universal facial expressions into the followingsix categories: anger, disgust, fear, happiness, sadness, andsurprise. Each of these categories has a number of intermediatefacial expressions that are based on the intensity level andthe expression details. Therefore, we asked participants togive one rating on one of the six scales that were labeled:anger, disgust, fear, happiness, sadness, and surprise. With oneclick the participants thereby identified the emotional facialexpression and rated its intensity (see Figure 4).

C. Process

After reading the instructions, participants started ratingthe randomly presented images. The participants could rate asmany faces as they wanted, but they could not rate the sameimage twice. Participants received one cent per rating.

D. Participants

264 adult participants, located in the US, filled in the ques-tionnaire. MT automatically made sure that exactly 30 differentparticipants rated each image. To protect the privacy of itsworkers, MT does not directly allow to survey demographicdata and hence this data is not available for this study. Previoussurveys on the population of Mechanical Turk Users (MTU)reveals that MTUs from the US tend to be well educated,young, and with moderately high incomes, and roughly equallyas many males as females [36], [37]. Mechanical Turk has beenshown to be a viable, cost effective method for data collectionthat reduces threats to internal validity [38].

MT is only available for registered users, which doesinclude a Captcha test. MT has in addiction a reputationsystem in place which enables requesters and workers toprovide feedback. We can therefore assume that no automaticspam responses have been recorded. We performed a visualinspection to check for any obvious patterns in the data, suchas respondents always giving the same answer. We could notfind any obvious patterns.

1http://www.mturk.com/

Fig. 4. The rating scales

III. RESULTS

On average, participants rated 82.05 images with a standarddeviation of 155.3. The average response time per image was17.33 seconds. On average, each face was rated on 3.9 differentemotion scales with a standard deviation of 1.39. This indicatesthat many faces are to some degree ambiguous. The data forone face was corrupted due to a software failure and wastherefore excluded from the further analysis. The remaining627 faces form the basis for the statistical tests describedbelow.

A. Distribution of facial expressions

We calculated the most dominant emotional expression perface by first identifying on which emotional scale the faces wasrated most often. In case a face was rated 28 times as happyand two times as surprised then happiness was selected as thedominant emotion. In case of a tie, the emotional categorywith the higher average intensity was selected. For example,a face could have been rated 15 times as fear and 15 timesas surprise. If the average intensity rating of fear was higherthan the average intensity rating for surprise, then fear wasselected as the dominant emotion. Table I shows the count offaces per emotion based on the calculation of the dominantemotion per face. Most Minifigure faces have been rated as


happiness followed by anger. The other four emotions wereobserved considerably less.

TABLE I. COUNT OF FACE PER EMOTION

Emotion CountHappiness 324Anger 192Sadness 49Disgust 28Surprise 23Fear 11

1) Cluster Analysis: We performed a k-cluster analysis tocheck if the faces would fall into certain design patterns. Forthis analysis we used the all six emotion ratings for every face.If, for example, a face F was rated 20 times on the surprisescale with an average of 4.2 and 10 times on the fear scalewith an average of 3.1, then the data in Table II would berepresent face F.

TABLE II. DATA REPRESENTATION OF FACE F BASED ON AVERAGEINTENSITY RATINGS

Anger Disgust Fear Sadness Happiness SurpriseF 3.1 4.2

This data represents a non-weighted average. We triedseveral values for the number of clusters k, but at no setting ameaningful result could be obtained. Table III shows the finalclusters for k=6 after ten iterations.

TABLE III. FINAL CLUSTERS FOR K=6

Cluster1 2 3 4 5 6

Anger 1.3080 3.5048 3.7042 2.8381 3.1558 2.4135Disgust 1.6542 3.0613 2.0031 2.8871 1.7540 2.9300Fear 1.6842 1.0714 1.8583 2.6750 2.8611 2.6950Sadness 1.6423 1.5969 2.3145 3.7657 1.4250 2.1151Happiness 1.5012 2.3262 1.7079 2.4189 2.0848 1.6418Surprise 1.7256 2.9024 1.5967 2.8000 3.0926 2.3765

No clear clusters become visible. The results of this testshow that too many faces were rated on too many scales. Theaverage was, as already mentioned above, 3.9. It is not possibleto plot the six dimensional space that represents our data, butwe believe that our data would form a widely spread cloudof points. Using a weighted average would have not helped,since it would have not changed the fact that the faces wererated on too many different scales.

We therefore decided to repeat the cluster analysis only onthe basis of the frequency of the classifications. We ignoredthe intensity ratings. Using the example above, Face F wouldthen be represented as shown in Table IV.

TABLE IV. DATA REPRESENTATION OF FACE F BASED ON FREQUENCY

Anger Disgust Fear Sadness Happiness SurpriseF 20 10

A k-cluster analyses provides results for any k and wedecided to set k = 6 in order to check if it would result inthe same clusters as the emotional categories we presentedto the participants. The resulting six clusters did not matchthe emotional categories directly. Two variations of happinessand anger emerged and the clusters were significantly distant

to each other (p<0.001). Table V shows the six clusters, anexample face, its distance to the center of the cluster and howmany faces fall into each cluster. Two clusters that includea considerable amount of happiness have been identified. Weviewed some faces that are in the center of the cluster andinterpreted their expression. We labeled the more negative formof happiness as confidence. Also two types of anger haveemerged from the cluster analysis. One is a rather straightform of anger, while the other includes more mixed emotions.After reviewing some central faces, we interpreted this clusteras disdain. Cluster three loads strongly on the sadness emotion,but it does not seem to be as clean as for example the happinesscluster. We reviewed several central faces in this cluster andinterpreted them as ”Concern”.

B. Distribution of facial expressions across time

The faces might not only be unevenly distributed acrossemotional categories, but also across the years in which theywere released. We therefore plotted how many faces werenewly introduced per year. Figure 5 shows that the numberof new releases has grown substantially over the years.

0

10

20

30

40

50

60

70

80

90

100

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

Fig. 5. Number of new heads across years

It is of interest to see how the proportion of a certainemotion might have changed over time. Since the total numberof faces per years varies substantially, we used the proportionsof faces in a certain emotional category instead. If in ayear 20 new faces were released and 10 of them were ratedpredominantly as happy then the graph would indicate a valueof 50% (see Figure 6).

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

Happiness Anger Sadness Disgust Fear Surprise

Fig. 6. Proportion of emotional categories over time


TABLE V. THE SIX CLUSTERS OF FACES

ClusterCluster Nr. 1 2 3 4 5 6

Disdain Confidence Concern Fear Happiness Anger

FaceDistance 2.961 2.496 2.591 3.067 0.776 1.790Anger 10 4 4 2 1 21Disgust 9 3 3 1 0 4Fear 2 1 3 17 0 1Sadness 4 2 14 2 0 1Happiness 2 15 3 1 26 2Nr. Cases 80 140 57 13 198 139

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Anger

(a) Anger

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Disgust

(b) Disgust

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Fear

(c) Fear

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Happiness

(d) Happiness

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Sadness

(e) Sadness

Year20102005200019951990198519801975

Inte

nsity

5 .00

4.00

3.00

2.00

1.00

Surprise

(f) Surprise

Fig. 7. Scatterplots of emotional intensities across time

We next plotted all the faces across time (see Figure 7)based on their average intensity of their dominant emotion.Besides the obvious differences in frequency that have alreadybeen described in Table I we notice that the faces are veryscattered across the intensity scale for angry and happy faces.Faces in the other categories are more clustered. There are, forexample, only very few faces expressing a low intensity levelof fear (see Figure 7(c)).

We estimate a curve of best fit for each of the emotioncategories. A linear model turned out to be the best fit forall emotion categories, but the enormous spread of the dataresulted in models that are not able to significantly representthe data. For the angry faces, the linear model was only able toexplain 0.1 % of the variance. Table VI shows the R2 valuesand the significance level for each of the linear estimations.

TABLE VI. RESULTS OF THE LINEAR CURVE ESTIMATIONS ACROSSTHE SIX EMOTION CATEGORIES

Emotion R2 Sig.Happiness 0.010 0.069Anger 0.001 0.654Sadness 0.034 0.205Disgust 0.037 0.324Surprise 0.010 0.651Fear 0.017 0.699

C. Context

Next, we analyzed if the faces were perceived differ-ently depending on whether they were attached to a wholeMinifigure or not. We analyzed how the frequencies acrossthe six emotional categories may have changed. We conducteda related sample t-test in which the context (face or body)was the independent variable and the average frequencies peremotional category (anger, disgust, fear, sadness and happi-ness) were the dependent variables. Table VII shows the meanfrequencies of the emotional categories across the two con-texts. The mean for anger, disgust, sadness and surprise weresignificantly different. For anger and happiness the context ofthe body increased the mean frequency, while for disgust andsadness the context decreased the mean frequencies.

TABLE VII. MEAN, STANDARD DEVIATION AND P ACCROSSEMOTIONAL CATEGORIES.

Emotion Mean Head Mean Body Std.Dev. PAnger 7.968 8.839 3.597 0.022Disgust 3.462 2.688 2.468 0.003Fear 1.301 1.602 3.103 0.352Sadness 1.624 1.172 2.159 0.047Happiness 12.946 13.677 4.062 0.086Surprise 2.699 2.022 2.183 0.004

We then analyzed if the context may influence the distinc-tiveness of the face. Would the expression of a face becomeclearer if it was presented within the context of a wholeMinifigure? We performed a paired sample t-test in which the


(a) Natural (b) Yellow

Fig. 8. Harry Potter with two different skin colors

context was the independent variable. The number emotionalcategories a face was categorized on and the associated χ2

value were the dependent variable. A face is very distinct if itwas categorized into only on very few emotional categories.The mean χ2 value for the face condition (m = 0.0015) wasnot significantly (t(92) = −0.623, p = 0.535) higher than themean χ2 value for the body condition (m = 0.0039). Themean number of categories for the face condition (m = 3.98)was not significantly (t(92) = 0.592, p = 0.556) higher thanthe mean mean number of categories for the body condition(m = 3.90).

Finally, we analyzed if the change in the skin color that wasfirst introduces in the Harry Potter theme had any influence onhow the face was perceived. In our set of photographs of thefull Minifigure, we had included the Harry Potter Minifigurewith two different skin colors: Figure 8(a) shows the naturalskin color introduced in 2004 and Figure 8(b) shows thetraditional yellow LEGO skin color.

Table 8 shows how often the two Harry Potter Minifigureswere classified into the six emotion categories. We conducteda χ2 test on this data and although the assumption of the mini-mum expected cell frequency was slightly violated. 83% of thecells had a value of less than five. The χ2 test showed that therewas no significant difference (Pearson χ2 = 1.953, p = 0.856)between the ratings of the two Harry Potter Minifigures. Next,we conducted a t-test only on the intensity scores of thoseparticipants that had classified the Minifigures as happy, whichwere 19 for the natural and 22 for the yellow Harry Potter. Thet-test showed that there was no significant difference in theintensity scores (t(39) = −0.426, p = 0.672). The differentskin color did not result in a significantly different evaluationby the users.

TABLE VIII. COUNT OF EMOTIONAL CATEGORIES PER SKIN COLOR.

Color Happiness Fear Anger Sadness Disgust Surprise TotalNatural 19 3 1 3 3 1 30Yellow 22 1 1 2 2 2 30Total 41 4 2 5 5 3 60

IV. CONCLUSIONS

The number of new faces that the LEGO company intro-duces every year is increasing steadily. Creating variations ofMinifigures could possibly be more cost effective by creating

different face prints than torso prints. If the current trendcontinues, then soon every Minifigure in every set will beunique.

Only in the early 90s did the LEGO company start toproduce a greater variety of faces. Happiness and anger seemto be the most frequent emotional expression of the Minifigurefaces and their intensity is widely scattered. This scatter makesit very difficult to create a model that would adequatelyrepresent the development of faces over time. Still, we canobserve a trend over time that the proportion of happy facesdecrease and the proportion of angry faces increase. We havebeen able to identify six different clusters of faces. There aretwo different types of happiness and two different types ofanger.

Four out of six emotional categories were significantlyinfluenced by the context in which the face was presented.For anger the presence of the body increased the frequencyof how often the face was categorized as such. For disgust,sadness and surprise, the presence of the body decreasedthe frequency. The context nearly significantly increased thefrequency in the happiness category. We have to be carefulwith the interpretation of this result. A change in frequencydoes not necessarily mean that a face becomes more or lessdistinct. Our analysis of the faces distinctiveness shows thatthe availability of the context did not significantly increase thefaces distinctiveness.

The introduction of more natural skin colors did not changehow participants perceived the Harry Potter Minifigure. Sincewe only had this one sample of two different skin colors forthe same Minifigure, we cannot necessarily generalize to allof the Minifigures that have been released with two differentskin colors.

We have to consider this distribution of faces across emo-tional categories in the context of the LEGO themes. After all,most Minifigures are released in sets that belong to a certaintheme, such as Pirates or Harry Potter. It is our impression thatthe themes have been increasingly based on conflicts. Often agood force is struggling with a bad one. May it be the goodknights against the skeleton warriors or the space police againstalien criminals. But the facial expressions are not directlymatched to good and evil. Even the good characters sufferin their struggle and the villains can have a smug expression.In any case, the variety of faces has increased considerable.

We cannot help but wonder how the move from onlypositive faces to an increasing number of negative facesimpacts how children play. So far LEGO did at least notproduce classical military themes. There is no LEGO DesertStorm or LEGO D-Day. The Megablocks company, on theother hand, is producing a LEGO compatible construction toysthat do fill this market space. Their HALO line of products,which is directly related to the popular computer game of thesame name, is clearly embedded in a military culture. Othercompanies, such as Brickarms, are also already offering LEGOcompatible weapons for Minifigures.

But also LEGO has a considerable array of weapon systemsin their program, although the weapons mainly appear in thefictional themes. But their presence indicated that also LEGOis moving towards a more conflict based play themes. Thisdevelopment might be unavoidable to sustain a strong market


position. Still, LEGO might not be able to hold onto its highlypositive reputation. The children that grow up with LEGOtoday will remember not only smileys, but also anger and fearin the Minifigures’ faces.

Designers of agent faces should take great care to designthe expressions and to test their effect since toys play animportant role in the development of children. The exampleof the Minifigures show that to appeal to users it is necessaryto offer a wide range of emotional expressions that connectto the complex interaction scenarios of today’s users. Insteadof focusing on highly realistic expressions, it may be worth-while to increase the variability of expressions. A comic styleexpression is sufficient to convey a full spectrum of emotionsand intensities.

A. Limitations

Participants in this study could rate as many or as littlefaces as they wanted. The number of faces they rated variesgreatly and hence we do recommend a Baysian approach toconduct more advanced statistics for future studies as describedin [35].

REFERENCES

[1] B. Sutton-Smith, The ambiguity of play. Cambridge, Mass.: HarvardUniversity Press, 1997.

[2] J. Goldstein, D. Buckingham, and G. Brougere, Toys, games and media.Mahwah, NJ: Lawrence Erlbaum Associates, 2004.

[3] H. Rakoczy, M. Tomasello, and T. Striano, “How children turn objectsinto symbols: A cultural learning account.” in Symbol use and symbolrepresentation., L. Namy, Ed. New York: Erlbaum, 2005, pp. 67–97.

[4] K. R. Ginsburg, “The importance of play in promoting healthy childdevelopment and maintaining strong parent-child bonds,” Pediatrics,vol. 119, no. 1, pp. 182–191, 2007.

[5] M. Lauwaert, The place of play : toys and digital culture, ser. Media-Matters. Amsterdam: Amsterdam University Press, 2009.

[6] G. Cross, Kids’ stuff : toys and the changing world of Americanchildhood. Cambridge, Mass: Harvard University Press, 1997.

[7] S. Hussain, J. Lindh, and G. Shukur, “The effect of lego training onpupils’ school performance in mathematics, problem solving ability andattitude: Swedish data,” Educational Technology & Society, vol. 9, no. 3,pp. 182–194, 2006.

[8] H. Dittmar, E. Halliwell, and S. Ive, “Does barbie make girls wantto be thin? the effect of experimental exposure to images of dolls onthe body image of 5- to 8-year-old girls,” Developmental Psychology,vol. 42, no. 5, pp. 283–92, 2006.

[9] I. D. Cherney, L. Kelly-Vance, K. G. Glover, A. Ruane, and B. O.Ryalls, “The effects of stereotyped toys and gender on play assessmentin children aged 18-47 months,” Educational Psychology, vol. 23, no. 1,p. 95, 2003.

[10] D. Lipkowitz, The LEGO Book. New York: DK Publishing, 2009.[11] G. K. Christiansen, “Patent ”legetojsbyggeelement”, dk92683c,” 1958.[12] LEGO, “Company profile - an introduction to the

lego group,” Tech. Rep., 2011. [Online]. Avail-able: http://cache.lego.com/en-gb/sitecore/shell/controls/rich%20text%20editor/-/media/30e5da5c580a4141914659e26753c938.pdf

[13] J. Baichtal and J. Meno, The Cult of LEGO. San Francisco: No StarchPress, 2011.

[14] J. Bender, LEGO: A Love Story. Hoboken: Willey, 2010.[15] G. K. Christiansen and J. N. Knudsen, “Toy figure,” 1979.[16] N. Martell, Standing small: a celebration of 30 years of the Lego

minifigure. New York: DK Publisher, 2009.[17] DK Publishing, LEGO Star Wars Character Encyclopedia. New York:

DK Publishing, 2011.

[18] M. Steiner, LEGO collector: collector’s guide. Dreieich: Fantasia,2008.

[19] D. G. Myers, Theories of Emotions. Worth Publisher, 2004, ch.Theories of Emotions, pp. 500–529.

[20] R. A. Calvo and S. D’Mello, “Affect detection: An interdisciplinaryreview of models, methods, and their applications,” IEEE Trans. Affect.Comput., vol. 1, no. 1, pp. 18–37, Jan. 2010. [Online]. Available:http://dx.doi.org/10.1109/T-AFFC.2010.1

[21] P. Ekman, The face of man: expressions of universal emotions. NewGuinea village, New York: Garland STPM Press, 1980.

[22] N. H. Frijda, The emotions. Cambridge, England: Cambridge Univer-sity Press., 1980.

[23] R. Buck, Human motivation and emotion (2nd ed.). New York: Wiley,1980.

[24] P. Ekman and W. Friesen, Unmasking the Face. Englewood Cliffs:Prentice Hall, 1975.

[25] C. M. Whissel, “The dictionary of affect in language,” Emotion: Theory,Research and Experience, vol. 4, pp. 113–131, 1989.

[26] M. Obaid, R. Mukundan, M. Billinghurst, and C. Pelachaud, “Expres-sive MPEG-4 Facial Animation Using Quadratic Deformation Models,”in International Conference on Computer Graphics, Imaging and Visu-alisation (CGIV 2010). IEEE Computer Society, 2010.

[27] A. Raouzaiou, N. Tsapatsoulis, K. Karpouzis, and S. Kollias, “Parame-terized facial expression synthesis based on mpeg-4,” EURASIP J. Appl.Signal Process., vol. 2002, no. 1, pp. 1021–1038, Jan. 2002.

[28] C. Bartneck and M. J. Lyons, “Hci and the face: towards an artof the soluble,” in Proceedings of the 12th international conferenceon Human-computer interaction: interaction design and usability,ser. HCI’07. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 20–29.[Online]. Available: http://dl.acm.org/citation.cfm?id=1772490.1772494

[29] P. Desmet and P. Hekkert, “Special issue editorial: Design & emotion,”International Journal of Design, vol. 3, no. 2, pp. 1–6, 2009.

[30] N. Schwarz, Retrospective and Concurrent Self-Reports: The Rationalefor Real-Time Data Capture. Oxford University Press, 2007, pp. 11–26.

[31] T. Koda, T. Ishida, M. Rehm, and A. Elisabeth, “Avatar culture: cross-cultural evaluations of avatar facial expressions,” AI and Society, vol. 24,no. 3, pp. 237–250, 2009.

[32] Z. Ruttkay, “Cultural dialects of real and synthetic emotional facialexpressions,” AI and Society, vol. 24, no. 3, pp. 307–315, 2009.

[33] J. M. Carroll and J. A. Russell, “Do facial expressions signal specificemotions? judging emotion from the face in context,” Journal ofPersonality and Social Psychology, vol. 70, no. 2, pp. 205–218, 1996.

[34] C. Bartneck, The Unofficial LEGO Minifigure Catalog. Charleston:CreateSpace, 2011.

[35] C. Bartneck, K. Zawieska, A. Dunser, and E. Moltchanova, “Comparingthe quality of responses received from participants recruited withamazon’s mechanical turk to participants recruited through traditionalmethods,” International Journal of Human Computer Studies (submit-ted), 2012.

[36] P. G. Ipeirotis, “Demographics of mechanical turk,” New York Univer-sity, Tech. Rep., 2010.

[37] J. Ross, L. Irani, S. Silberman, A. Zaldivar, and B. Tomlinson, “Whoare the crowdworkers? shifting demographics in mechanical turk,”in Proceedings of the 28th of the international conference extendedabstracts on Human factors in computing systems. ACM, 2010, pp.2863–2872.

[38] G. Paolacci, J. Chandler, and P. G. Ipeirotis, “Running experiments onamazon mechanical turk,” Judgment and Decision Making, vol. 5, no. 5,pp. 411–419, 2010.


http://cache.lego.com/en-gb/sitecore/shell/controls/rich%20text%20editor/-/media/30e5da5c580a4141914659e26753c938.pdf

http://cache.lego.com/en-gb/sitecore/shell/controls/rich%20text%20editor/-/media/30e5da5c580a4141914659e26753c938.pdf

http://dx.doi.org/10.1109/T-AFFC.2010.1

http://dl.acm.org/citation.cfm?id=1772490.1772494

agents with faces - what can we learn from lego minfigures?miniﬁgure faces. over the years, lego...

Documents