interactive social agents from deep data

Interactive Social Agents from Deep Data

Joana Campos and Ana Paiva

INESC-ID and Instituto Superior Tecnico - Universidade de Lisboa,Av. Prof. Cavaco Silva, Taguspark 2744-016, Porto Salvo, Portugal

[email protected]

[email protected]

Abstract. The multidisciplinary challenge of modelling agents have beendriven by theory explaining social phenomena. Yet, these generic modelslack of expressiveness. For that reason, data-driven approaches to thedesign of agents have been pursued, mainly for modelling non-verbal be-haviour. In this paper we argue that real data is not only useful for thatmodality, but it can also assist agent’s design in different phases of theprocess at different levels of granularity. Furthermore, deep data, whichinform us about user’s perception, emotions and motivations is valu-able to build fluid interactions with virtual humans. We illustrate ourstance with two case studies where we study interpersonal conflict. Onestudy describes the design of agents to populate a serious game aimedat teaching conflict resolution skills to children and the other describesan experiment designed to extract deep data from a dyadic interactionprone to conflict emergence.

Key words: Virtual Agents, Design, Interpersonal Conflict, Data-driven

1 Introduction

Intelligent Virtual Agents (IVAs) are becoming commonplace given their wideapplication in several areas such as healthcare, education, simulation and games.Hence, more often, agents have to collaborate with humans, compete or even toact under a specific role in increasingly dynamic environments. To perform allthose tasks effectively, in a way that suits human standards, such agent systemsshould be able to decode other’s social signals, produce non-verbal behaviours,generate and manage dialogue, plan and decide, which involves real-time rea-soning and action. Inevitably, designing agents that are socially aware of theirinteractional partner can be a bewilderingly complex task.

Over the years, researchers have tackled the challenge of modelling and ex-pressing each of the aforementioned natural human modalities, by integratingthe knowledge and methodologies from computer science with concepts fromsociology, psychology or linguistics, to enumerate a few examples. Frequently,the development of these computational models is driven by theoretical con-cepts, which provides a systematic way to tackle a problem, towards a genericrepresentation of social phenomena. Although it is an utterly valid approach,

2 Joana Campos, Ana Paiva

researchers are confronted with several difficulties. First and foremost, the in-tegration of general theoretical concepts present in the social sciences literatureinto a program, requires interpretation and formulation of assumptions from thedevelopment side. Additionally, it is often hard to use the computational modelto represent very concrete cases, since the models are not expressive enoughwithout some authoring tweaks. This is mostly due to the lack of empirical datasupporting the formulated theories, which would allow us not only to justify au-thoring decisions, but also help us to better evaluate the behaviours generatedby our computational models.

In a nutshell, theoretical research tries to explain the macro-concepts andoffers a simplification of a complex phenomena, but it often disregards the spec-ification of those concepts into small details, which would help to derive concreteinstantiations for the agents’ behaviour. Hence, unsurprisingly, data-driven ap-proaches have become increasingly popular, because it allows the validation ofdesign choices empirically [16]. This type of approach may help researchers toachieve a better balance between simplification and realistic behaviour, by fo-cusing on the expressivity of the model. Yet, data-driven approaches start bycollecting enormous amounts of data for finding representativeness of types ofbehaviour. First, it can be a very daunting task to gather large amounts of data.Then, the more data we get, the more data has to be annotated and there is nolimit for the number of annotation layers, which depends on the granularity ofbehaviour one is seeking [16]. But regardless of these open research issues, thismethodology for designing intelligent agents enables the generation of modelsthat augment the agent’s interactive capabilities, mainly because it allows re-searchers to focus on small units of the interaction under the assumption thatpeople’s interactions are driven by very subtle cues. This cues are the scaffoldfor fluid interactions, which is the uttermost goal in virtual agent’s research.

According to this view, Morency et al. [13] focused on multi-modal behavioursand its predictive power. Their aim was to understand how humans use thebackchannel feedback (e.g. head-nods, paraverbals) to interact with other hu-mans and then generate dyadic conversational behaviour. Another example isthe work by Endrass et al. [6], who analysed a multi-modal corpura to investigateculture-specific interpersonal communication management. Her (theory-based)model of interpersonal communication was then continuously refined with sta-tistical data extracted from a video corpus of German and Japanese first-timemeetings [7].

Real data is not only useful for modelling non-verbal behaviours. Real anddeep data can feed the agent’s design process in several ways, depending on thecontext of application and the level of granularity required for a certain type oftask. Nevertheless, theory from diverse areas of research should not be discardedand an hybrid approach to the design of social agents is probably more adequate.In this paper, we present two case studies in which interpersonal conflict wasexplored with children. Our aim is to expose how different levels of granularityof data and theory combined, can help us to express the phenomenon in twodifferent contexts. Furthermore, we aim to highlight the importance of gathering

Interactive Social Agents rooted in Deep Data 3

deep data from potential users of a system (independently of its application).By deep data we mean, rich data representative of one’s perspectives, emotionsand motivations. We believe that these elements establish the context for manydyadic interactions, which are no more than a form of relating.

2 The Case of Interpersonal Conflict

In our work, we aim to explore how subtle forms of conflict evolve in socialinteractions, for a better representation of the phenomenon in dyadic interac-tions. A simplistic view of conflict is predominant in the Multi-Agent Systems’sliterature, where the phenomena is commonly addressed as a failure or synchro-nisation problem [15]. But conflict is more than that, it is a form of relating[4], in which emotions play a relevant role. Therefore, we started by articulatinginsights from the social sciences literature towards a more natural representationof conflict in agents’ behavioural systems [3]. Our position is that conflict is adynamic process and transitions between states are driven by an agent’s emo-tions, which are responsible for activating or deactivating it in a conflict loop.Yet, despite the massive research on conflict in the social sciences, there is stillsome uncertainty on how to translate the theory to more specific parametersthat together provide an adequate description of the phenomenon. Conflict hap-pens at different levels of the social interaction and it is not clear what actuallyhappens during this multi-level process.

To learn more about the conflict phenomena we turned to humans, in par-ticular children, to understand how the process unrolls. We explored conflictin two different contexts and hence we followed two different methodologies togather data. In one study (Case Study 1 in section 3) in , we were inter-ested in the high-level dynamics of the conflict phenomenon to represent it ina game intended for teaching conflict resolution skills to children [2]. The gamewas populated by virtual agents and we claim that the agents have to dependon emotional reactions to accurately respond to one’s expectations of naturalconflict scenarios. Along these lines, the agents have to be created based onreal data gathered from real people (children) without shortcomings for definingtheir behaviours. Therefore, to collect children’s perspective on the subject weemployed an adaptation of cultural probes described in the next section.

Following this, to get more detail, we were interested in dyadic factors ofinterpersonal conflict and how conflict can be modelled from an agent’s perspec-tive (Case Study 2 in section 4). Conflict requires interaction and emotionsthat regulate it. Thus, we created an experimental setting (described in section4) that reduces real-life to a mixed motive game, in which children’s previousexperiences and relation with the interactional partner play a relevant role inthe interaction. Our aim is to understand how subtle forms of conflict unroll, byanalysing micro-level behaviours and establish a link to high-level psychologicalconstructs.


3 Case Study 1: Capturing High-Level Deep Data

When the systems are task specific or, as it is the case, have specific learningobjectives is essential to adopt a methodology that iteratively involves the finaluser. Thus, in this work, to glimpse children’s emotional worlds and to assesswhether or not and under which conditions conflicts exist, we centered the userin the design process. As part of this process, we used an adaptation of culturalprobes — Conflict Probes —, which aim at gaining insight of realistic gamemechanics and narratives, by closely observing how children behave at school.Taking advantage of cultural probes’ characteristics , we expected candid re-sponses from children that would guide us from initiating actions of conflictepisodes to climax and finally the resolution strategies that they employed.

Conflict probe is a variation of the original package introduced by Gaver [8].Despite being an adaptation it was designed to embed the main characteristicsof this method. The probe study was designed to be user-centred, to allow self-documentation and also has an exploratory character [9]).

3.1 Aim

The purpose of this study was to understand the users’ social worlds, in termsof the ways in which children behave at school. By using cultural probes, weexpected to gain insight into these practices without being too close and inter-fering in the ways that children interact with each other. The aim of our probepack is to collect sensitive information about children’s individual feelings andbehaviours in situations that they identify as important to them. In that sense,we expected that children would show us how they themselves appraise theirsocial environment, encouraging us, as designers, to step back from our precon-ceived notions of their reality, and to identify novel and surprising aspects ofchildren’s lives.

3.2 Adaptation

The probe package was created in order to be suitable to this new context. First,we had to adapt the probes to the theme, to the environment in which they wouldbe employed and we had to create the specific materials for this study. Second,we would be working with children. For that reason, the tasks had to be adaptedin terms of ambiguity and provocative attitude.

3.3 Research Materials

The conflict probes (Figure 1) comprised a pack of 7 envelopes, each one with awritten task inside, and a set of materials that children could use to perform theactivities. In addition, a calendar (see Figure 2 below) was used as an anchorpoint to help children identify which envelope to open and when.

The tasks were designed in such a way as to create an evolving interactionwith the user by increasing the level of ambiguity as the participant explores the


Fig. 1. Probe pack

Fig. 2. Calendar to keep track of tasks.

delivered package. As they progress through the tasks, the children are able tobuild on the concepts apprehended in previous tasks. A description of each ofthe tasks is provided in the following section.

3.4 Participants

The cultural probe study focused on 51 children with ages between 9 and 12 yearsold. Conflict probes had to be completed in class during a five 5- week period.Each week teachers made available a 45-minute slot to the activity. Teacherssuggested such setting, because at home children would be influenced by theirparents. Teachers were asked not to intervene and every child had to completethe activities on their own.

3.5 Probe’s Tasks: Breaking down ”conflict” into pieces

When asked about conflict, children’s personal definition does not go much fur-ther than the word ”fight” [12]. For that reason, dividing the concept into piecesseemed to be essential in order to extract meaningful information from the chil-dren and at the same time make easier to them to talk about the subject.

The probe pack was composed by 7 self-contained tasks designed around fivedimensions of conflict: participants, causes, strategies, resolutions and outcomes.Each one of the tasks (if not all) would fit in each one these dimensions wouldbe dependent of its returns.

At the same time, as an attempt to engage the children, we presented themwhat we believe to be creative and constructive activities [22] to complete. The


toolkit offered activities for thinking (Bubbles), mapping (Social Network), feel-ing (Feelings and Anger Measurement), storytelling (You’re the writer), beingcreative and constructive (No Rules and Journal) (inspired on Sanders idea ofStrategic Visioning Workshops [17]).

Social Network (Participants)A key aspect of the dynamics of interpersonal conflict is how people relate

with each other and how their relationships change over time as a consequenceof conflict episodes. The first task was designed to get information about chil-dren’s social relationships. The social network task asks about the participantsemotional links to other children in school. At the same time, it tries to capturethe degree of closeness associated with different emotions. Figure 3 depicts theprovided material for this task. The yellow envelope contains a card that has theinstructions for this activity, a piece of paper with concentric circles, and a setof stickers. Each sticker has an emotion/feeling and a line. The children have towrite on that line the name of a person in their school that makes or made themfeel that way. Then, they have to choose one of the concentric circles on whichto affix it. The further from the centre, the less close to them that person is.

Fig. 3. Materials for the Social Network Task

Feelings (Outcomes)This task is complementary to the first one and tries to capture triggers for

emotions other than anger, which is the emotion most frequently associated withconflict. This and the social networks task were assigned in the first week, asthey are both quite simple. Children were asked to match a picture (sad, afraid,happy) with a reason for feeling that way, as shown in Figure 4.

Bubbles (Causes and Strategies)Describing scenarios from pictures is a suitable technique for broadening

children’s vocabulary, as suggested by Kreidler [12]. In Kreidler methodology a


Fig. 4. Materials for the Feelings Task

picture depicting a conflict was presented to the children, who had to describeconflict related issues such as what was happening or the reason for what hadhappened. Inspired by this, we asked them to do the opposite, as we do not intendto teach children about conflict (at least at this stage), but to extract realisticconflict scenarios. Our adaptation of this method asked children to use a set ofgiven words for describing a situation (see Figure 5). They were free to draw orwrite depending on their preference. The words fell into three main categories:feelings – sad, angry, scared, upset, worried; external actions (which have adirect impact on the situation) – I said, I did, I had; and internal triggers(which may influence the chosen action) – I felt, I thought, I wanted. The wordsfriend and school were also added. The words were general enough not to guidea child to any particular scenario, as might have happened if we had presentedthem with a picture. Also note that the word conflict was never mentioned. Theanswer sheet was divided into three areas identified as Before, What happened,and After. This subdivision may give us not just information about the set ofevents that lead to a conflict situation, but also the aftermath of the conflict.

You’re the writer (Strategies)Stories and storytelling have had an active role either in ethnography research

or in the HCI field, as they are practices claimed to preserve personal expression,relationships, conflicts and multiple perspectives [14].

This task is intended to explore various perspectives on conflict resolutionthat may be influenced by personality differences, given an initial scenario of aresource dispute. This kind of situation is one of the most usual sources of inter-personal conflict among children [18]. In childrens eyes, it is easily interpretedas a contest in which one party loses and the other wins [12]. Figure 6 showsthe material available for this task. At the top of the answer sheet there is astory that describes a conflict between a pair of boys and a pair of girls whoare disputing over a computer to finish a school project. Dialogue cues indicate


Fig. 5. Materials for the Bubbles Task

some tension between the two pairs, and the story ends up with one of the girlsyelling to the boys: ”Come on ” The children were asked to write a conclusionto the story from this point.

Fig. 6. Materials for You’re the writer Task

Thermomether (Causes and Outcomes)Anger is probably the emotion most intuitively associated with conflict [1].

What makes someone get angry is likely to vary a lot between children. Thistask was designed to get at the differences in intensity of various triggers foranger. We used an anger thermometer chart, inspired by an activity presented


by Kreidler [12]. In his book, Kreidler’s goal was to expand children’s vocabularyand to help them express what they are feeling. The thermometer is a metaphorfor a scale of anger. That is, the hotter it gets the more intense the angry feelingsare. The anger scale starts with annoyed, goes through cross, anger, furious andends with enraged (see Figure 7 above). The children were asked to write downtypical situations that make them reach each level of the scale.

Fig. 7. Materials for the Thermometer Task

No Rules (All)Inspired by the provocative nature of cultural probes and the structure of

storyboards, we provided children with a set of materials representative of con-cepts they already learned in the previous tasks. In the instructions we asked theparticipant to think about conflict and what that means to him or her. Then,using the provided materials, they are to depict a situation in which a conflicthappened or could happen. The envelope is full of stickers (see Figure 8 below),including dialogue bubbles, thermometers at different points of the anger scale,thumbs up/down, and the words what, who, where, and when. (This task wasnot designed to feed any of conflict dimensions.)

3.6 Implications for design

Information conveyed by the probes redesigns the researchers’ mental modelsabout a certain subject. Yet, how that data encourages subsequent decisionsfades away between the two stages. By using this methodology, not only dowe want to understand the problem at hand, but also to access real scenariosfrom the subjective data and make our agents more believable, in the sensethat children will be able to identify with them. We do not expect to find afinal model for the agents, but rather lay out relevant aspects for a design shiftof such characters. Note that the agent community hardly ever designed theiragents focusing on actual users during their development. Agents’ behavioursare most of the times generated by the designer’s opinion beforehand.


Fig. 8. Materials for the No Rules Task

The design shift that we look for is fed by the children’s emotions, attitudesand actions, which will shape our characters in the game - My Dream Theatre(MDT)1. How the agent will behave depends to the answer to questions asWhat are the causes of conflict? What makes them escalate? How do childrendeescalate? What do they feel afterwards whereas the confrontation was with afriend or other?. The probes contributed to the design, by creating an empathiclink with children’s common practices, emotions and perspectives on conflict.It also allowed us to extract generic information about conflict episodes andvocabulary. The probes helped us to understand the interaction flow and thusstratify how to explain conflict to children, by reinforcing aspects that are notclear to them. For example, from a high level perspective, it helped us to makeexplicit that conflict is not an isolated act, but rather a phenomenon that evolvesthroughout several stages. That fact is not usually clear for children.

In the serious game, that idea of a dynamic process was made explicit [2].That process is driven by the agent’s emotions, which are essential to understandhow to manage the conflicts between the characters (see Figure 9). Further,events that occur within the game are closely related to episodes reported inthe probes, as an attempt to create a meaningful experience. Additionally, aschildren more often than not do not employ effective strategies to cope with theirconflicts, the game presented to them an array of choices, extracted from theliterature (Thomas-Kilmann Model [19]), to help them manage conflict episodesin MDT. In that way, theory and deep data combined provide an environmentin which current practices meet other perspectives of conflict, to help childrento reason about conflict episodes more effectively.

4 Case Study 2: Capturing Low-Level Deep Data

As previously asserted, despite the massive research on conflict in the socialsciences, there is still some uncertainty on how to translate the theory to more

1 Graphics by Serious Games Interactive (SGI)


Fig. 9. An agent’s emotional reaction due to incompatible goals in the MDT game. Inthis case, the character ‘Maria’ is overtly manifesting her anger (her emotional stateis given by the thermometer). Unfortunately, the player was not able to prevent theagent’s emotional escalation.

specific parameters that together provide an adequate description of the phe-nomenon. The lack of a blue-print to represent conflict in interactive settingshas challenged researchers to find possible ways of representing conflict for theirpurposes. To create believable virtual agent’s in conflict we explore how an agentknows that it is in conflict, by exploring a human-human interaction in a mixed-motive game. Not only are we interested in understanding how one perceives sheis in conflict, but also what are the interpersonal strategies applied as part of hu-man adaptation to the interactional partner. We should not forget that conflictis more than a state of affairs or an overt manifestation of disagreement, it is adynamic process in which is not guaranteed that the parties will go through allstages towards climax. In fact, overt manifestations of conflict are not as usualas one might think.

In mixed-motive negotiations, conflicts are bound to emerge as participantshave opposed preferences and each one try to maximise their own gains. Suchexperimental setting, in which potential for conflict exists, acts as a model of asocial interaction that is the object of study. For this study we used a variationof the “Game of Nines”.

4.1 The “Game of Nines”

The “Game of Nines” is a mixed-motive bargaining game and it was firstly usedby Kelley et. al [11]. This bargaining game was selected because it creates an in-teresting setting, where the negotiators face dilemmas concerning their goals andforms of communication. Further, it requires that the players negotiate to dividea joint reward between themselves with competitive and cooperative incentives.


Each player holds cards from one to eight in their hands. At each trial, thenegotiators must agree on the cards they play such that their sum is nine or less.In each turn of the game a minimum necessary share (MNS) is assigned to eachnegotiator. This MNS value is only known by the person to whom it was assignedto. Therefore, the information about the other is incomplete. For a profitableagreement the negotiator has to bargain for a value above the MNS (e.g. if aplayer has a MNS equals to 4 and plays a 6 he will get 2 as a reward), withoutknowing the extent of concessions that the other can make. If the participantsdo not reach an agreement in a limited amount of time both get zero. Therefore,is of mutual interest to get to an agreement and it is in each person’s individualinterest to chose a division that is the most profitable to her (and thus, minimallyprofitable to the other).

As the negotiators are children we eliminated the time constraint2. Becauseof that, if children were not able to reach an agreement they both would get zeroand “the bank” would win as many points as the difference between 9 and thesum of their MNS values. In the end, both players individually have to makemore profit than “the bank”. This establishes that the lack of consensus costsand also adds a cooperative incentive, ensuring a mixed-motive relationship.

The experiment consisted in 5 rounds. In each round the participant took aMNS (a card with a number) from an envelop. The child was instructed not toshow the card during the trial and never agree on a value below that number. Atthe end of the round, each participant had to show to the other her MNS value.During each bargaining round the players had to jointly agree on a possiblecontract. Each contract corresponds to a card that is going to be played byplayer A and a card played by player B, so that their sum does not exceed 9.For example, if player A plays the card 6 a possible contract is player B to playthe card 3. The interests of the parties are always directly opposed. What ismost profitable for one player is least profitable for the other. Besides holdingthe cards ranging from one to eight, each player also holds a card that allowsthem to give up if they feel they are not able to achieve a viable agreement. Therules for not reaching a consensus apply here. The player that sums more pointsin the end wins the game.

4.2 Procedure

Before the game sessions children filled in a sociometric questionnaire and apersonality test. The former was applied, mainly to ensured that children inopposed poles (neglected and populars) or children that did not like each otherwere not paired together, given the sensitive nature of this experiment3.

For the experiment, each dyad was collected from the classroom and a braceletto measure their electrodermal activity was immediately attached to their wrists.

2 We verified in pilot sessions that this factor was making them not to pay attentionto what was happening in the game.

3 The results from both questionnaires are beyond of the scope of this paper and arenot going to be discussed here.


Then, children were conducted to a room made available for the purpose. Theparticipants were sit face-to-face at the opposite ends of a table, on which a cardboard was there to assist them through the game. After the explanation of therules, the participants were “walked through” two rounds of the game to learnits mechanics. After that they were left alone to play the game (Figure 10).

To motivate the participants to do well, we told the players that the personwho accumulates more points during the game would win a prize. In the end,both children won a prize, but the winner was able to choose between two options(one item was better than the other).

Fig. 10. Two girls playing the Game of Nines

4.3 Data Collection and Recording

The “Game of Nines” data was collected in a public school in country X. Intotal, 22 children (13 girls and 9 boys) aged 10 to 12 years-old, participatedin dyadic sessions of the game. All dyads, with one exception, are same-sexparticipants. Opt-out consent forms were provided to all parents or guardians ofthose children. All games were video and audio recorded.

Two cameras recorded the interaction. Each one of the cameras was di-rected to one of the children, capturing their face and hand movements. A audiorecorder was used due to the poor performance of the cameras microphones. Thethree elements were later synchronized.

4.4 Annotations

The basic actions of the game were manually annotated along with gaze, smilesand speech transcriptions. Furthermore, action tendencies, emotions and othersocial signals were also annotated by two psychologists, using the ELAN4 soft-ware.

4 http://tla.mpi.nl/tools/tla-tools/elan/


The latter set of annotations are based on the EASI Model (Emotions As So-cial Information Model) presented by Van Kleef and DeDreu [20] who advocatea more social approach to emotions, that is how emotions in a social interactionshape behaviour. That seems appropriate to our analysis, since a true under-standing of the deep data requires a focus in interpersonal effects of emotion.Hence, the annotation layers to describe the behaviour in this social interactionare the following:

Emotion arise from an individual’s appraisal of the situation and individualsmay use those to their social decisions. Four groups of emotion were con-sidered. Happiness, joy, contentment; Anger, frustration, irritation; Sadness,distress, disappointment and worry; and finally guilt, regret and embarrass-ment.

Social Signals are related to the emotions. Were considered and annotatedsignals of affiliation, opportunity, dominance, aggression, supplication andappeasement.

Action Tendencies in this model are conceptualised in terms of Horney’s the-ory [10]. In decision making, one can be moving towards, enrolling in morecooperative activities, moving away by taking a passive stance in the interac-tion or moving against the other, in which non-cooperative actions are taken.These action tendencies framed here in the context of decision making, havebeen also used to define a taxonomy of conflict behaviour [21] [5].

The model also tries to explain the interaction between these three elements.The EASI model provides a framework for understanding the effects of one’semotions in one’s action and the competitive or cooperative nature of the inter-action. The focus on interpersonal emotions and their effect on the individual’sactions tendencies may help to explain why some conflicts subside whereas otheremerge.

4.5 Implications for design

The employed experimental paradigm attempts to create a natural setting thatresembles a real-life situation. It has increased the complexity for interpretation,but on the other hand the findings can be generalised to create more naturalsituations (e.g. design agent’s decision process such that the agent is able tobypass or ignore a conflict). The analysis of the behaviours according to theEASI model may shed a light on the cooperative and competitive moves presentin the interactions between children and how their interpersonal strategies made,most of the times, conflict not to be manifested (e.g. which moves contribute tomove forward in the conflict process and what moves makes it go backwards).Guided by this model, we attempt to shed a light in conflict dynamics and itsunderlying processes for a more natural agent’s behaviours. Furthermore, duringevaluation data can be used to verify the acceptability of the interaction, insteadof asking users directly about their impression of the agent’s behaviour [16].


5 General Discussion and Conclusions

Deep data from human-human interactions serves as an useful tool for develop-ing and modelling IVAs. In this paper we explore two experimental paradigmsto access users’ perceptions, emotions and motivations in two different levels ofgranularity, for two different purposes. On one hand, we explored how high-leveldeep data can inform us about children’s perspective of conflict. Conflict Probesplayed a reinvigorating role in our pre-inventive design phase. The theory thatgrounded the tasks’ structure gave us the opportunity to learn about conflict. Italso helped us understand how conflict can be explained to children, which hada direct effect on the design of the agents in the game. The probes’ returns veri-fied our earlier findings in showing the unsophisticated knowledge children haveabout conflict and the difficulty they have in recognizing its various dimensionsas part of a coherent whole.

To explore a different angle, we presented an experiment, where we try toreplicate a situation in which participants do not have full and accurate insightinto the structure of the social situation nor the information necessary to resolveit effectively. The Game of Nine reduces real-life to a mixed-motive game, inwhich children apply interpersonal strategies to adapt to the other. Our aim is toexplore the interaction in term of emotions, social signals and action tendenciesaccording to the EASI Model to better understand how conflict unrolls. Addingto this, other social signals as gaze cues, smiles and electrodermal activity mayalso help to make of the “fuzzy” dyadic interaction.

It is our belief that different granularities of data are required to representmore natural IVAs that suit the human standards. Data can be useful at differentstages of design for different purposes as described in this paper. Yet, theory fromdifferent areas of expertise is also essential to model agent’s behaviours. In fact,theory and data combined are essential tools for building more natural agent’sbehaviours.

Acknowledgements

This work was supported by national funds through FCT - Fundacao para aCiencia e Tecnologia, under project PEst-OE/EEI/LA0021/2013 and a scholar-ship (SFRH/BD/75342/2010) granted by FCT.

References

1. R. F. Baumeister, A. Stillwell, and S. R. Wotman. Victim and perpetrator accountsof interpersonal conflict- autobiographical narratives about anger.pdf. Journal ofPersonality and Social Psychology, 59(5), 1990.

2. J. Campos, C. Martinho, G. Ingram, A. Vasalou, and A. Paiva. My dream theatre:Putting conflict on center stage. In Proceedings of the 8th International Conferenceon the Foundations of Digital Games, 2013.


3. J. Campos, C. Martinho, and A. Paiva. Conflict inside out: a theoretical approachto conflict from an agent point of view. In Proceedings of the 12th InternationalConference on AAMAS, 2013.

4. C. Castelfranchi. Conflict ontology. Computational Conflicts, Conflict Modellingfor Distributed Intelligent Systems, pages 21–40, 2000.

5. M. Deutsch, P. T. Coleman, and E. C. Marcus. The handbook of conflict resolution:Theory and practice. Jossey-Bass, 2006.

6. B. Endrass, E. Andre, L. Huang, and J. Gratch. A data-driven approach to modelculture-specific communication management styles for virtual agents. In Proceed-ings of the 9th International Conference on AAMAS, 2010.

7. B. Endrass, E. Andre, M. Rehm, and Y. Nakano. Investigating culture-relatedaspects of behavior for virtual characters. Autonomous Agents and Multi-AgentSystems, 2013.

8. W. W. Gaver, T. Dunne, and E. Pacenti. Cultural probes. Interactions, 11(5),1999.

9. T. Hemmings, A. Crabtree, T. Rodden, K. Clarke, and M. Rouncefield. Probingthe probes. In Proc. of Participatory Design Conference, 2002.

10. K. Horney. Our inner conflicts: A constructive theory of neurosis. WW Norton &Company, 1945.

11. H. H. Kelley, L. L. Beckman, and C. S. Fischer. Negotiating the division of areward under incomplete information. Journal of Experimental Social Psychology,3(4), Oct. 1967.

12. W. J. Kreidler. Conflict Resolution Through Children’s Literature. Scholastic Pro-fessional Books, 1994.

13. L.-P. Morency, I. de Kok, and J. Gratch. A probabilistic multimodal approach forpredicting listener backchannels. Autonomous Agents and Multi-Agent Systems,20(1), 2010.

14. M. J. Muller. Participatory design: the third space in HCI. 2003.15. H. J. Mller and R. Dieng. On conflicts in general and their use in AI in particular.

In Computational Conflicts: Conflict Modeling for Distributed Intelligent Systems,with Contributions by Numerous Experts. Springer, 2000.

16. M. Rehm and E. Andr. From annotated multimodal corpora to simulated human-like behaviors. In Modeling Communication with Robots and Virtual Humans.Springer, 2008.

17. E. Sanders. Generative tools for codesigning. In Collaborative Design. London:Springer-Verlag, 2000.

18. C. U. Shantz. Conflicts between children. Child Development, 58(2), 1987.19. K. W. Thomas. Conflict and conflict management. In Handbook of Industrial and

Organizational Psychology, Dunnette, M.D. (Ed.). Randy McNally, Chicago, 1976.20. G. A. Van Kleef, C. K. W. De Dreu, and A. S. R. Manstead. An interpersonal

approach to emotion in social decision making: The emotions as social informationmodel. Advances in experimental social psychology, 42, 2010.

21. E. V. d. Vliert. Complex Interpersonal Conflict Behaviour: Theoretical Frontiers.Psychology Press, 1997.

22. P. Wyeth and C. Diercke. Designing cultural probes for children. In Proceedingsof the 20th conference of the computer-human interaction special interest group(CHISIG) of Australia - OZCHI, page 385, New York, USA, 2006. ACM Press.

interactive social agents from deep data

Documents