mutual information as a measure of coordination in ...€¦ · coordination in collaborative...
TRANSCRIPT
Mutual Information as a Measure ofCoordination in Collaborative Interaction
Dari TrendafilovUniversity of Hertfordshire
School of Computer ScienceHatfield, UK
Alexander MayeUniversity Medical Center Hamburg-Eppendorf
Department of Neurophysiology and PathophysiologyHamburg, Germany
Daniel PolaniUniversity of Hertfordshire
School of Computer ScienceHatfield, UK
Roderick Murray-SmithUniversity of Glasgow
School of Computing ScienceGlasgow, Scotland, UK
Andreas K EngelUniversity Medical Center Hamburg-Eppendorf
Department of Neurophysiology and PathophysiologyHamburg, [email protected]
We present an information-theoretic approach for quantifying the level of coordination between cooperatingparties engaged in a computer-mediated collaborative interaction. The approach builds on Shannon’s mutualinformation, as a task-independent objective measure, which captures the level of corelation between theactions of interacting agents. We introduce the approach through two characteristic examples and discussthe challenges in designing a reliable measure and the amount of modelling effort required. Our initial resultssuggest the potential of this measure in supporting designers of collaborative systems and in providing moresolid theoretical foundations for the science of Human-Computer Interaction.
1. INTRODUCTION
A fluid, engaging collaboration between peopleconnected remotely via a computer has long beena goal of technology-mediated interaction. Modernhardware increasingly facilitates the emergence ofexciting high-bandwidth, tightly-coupled, continuousinteraction styles by getting new sensing, processingand feedback capabilities. Multi-player games, takingplace in virtual environments, are just one exampleof such systems. More recent work expands furtherthe research towards human–robot collaboration.
Cooperation in the real world emerges as a distinctcombination of innate and learned behaviour accord-ing Tomasello (2009), and collaborative systems tapinto both individual and social processes such asmutual perception, joint attention, turn-taking, andmutual entrainment. Research on the interactive di-mensions of mutual perception (Auvray et al. (2009))show that rich perceptions are possible even with aminimal channel of communication, which has directpractical implications. Causes and consequences ofdisrupted coupling between agents is studied in thecontext of deficits in social cognition abilities like inpeople with autism spectrum disorders. However, inorder to analyse such social contingencies we need
coupling measures in place. Standard performancemetrics, such as success rate or completion time,might not be applicable, nor descriptive for a particu-lar system, hence, in order to evaluate such systemswe need new objective measures.
One current challenge is the development of a for-mal measure, quantifying the level of coordinationbetween participants of computer-mediated environ-ments. A rigorous measure of coordination couldhelp provide a firm foundation for designers to treatand evaluate collaborative systems in a generalfashion. An analytical tool, characterising coordina-tion in real time, could give direct insight into thedetailed interactions that evolve as people engageand disengage from contact with each other. In thispaper we present initial results of the applicationof the information-theoretic approach introduced inTrendafilov et al. (2015) to a labyrinth game sce-nario and discuss its sensitivity and potential forcharacterizing the dynamics in social interaction.Variants of Shannon’s mutual information are typi-cally used in the identification of the ’flow of infor-mation’ in a given system (Ay and Polani (2008),Wheeler (1990)), where the joint information stemsfrom a common past (Matsumoto and Tsuda (1988);Schreiber (2000)).
c© Copyright is held by the author/owner(s). 1Proceedings of International Workshop on Ubiquitous andCollaborative Computing (iUBICOM’15) – Lincoln, UK
iUBICOM’15
Figure 1: In this example each of the two players controlsonly one axis of the labyrinth game. First, player 1 bringsthe ball to position A without any interaction from player 2.By reversing the roles, player 2 then brings it to position B.Finally, both players need to cooperate in order to pot theball in the green target and not in one of the wholes.
2. COLLABORATIVE INTERACTION
Real-time collaborative systems enable the simul-taneous interaction between multiple participants;however, for simplicity we will consider the bidirec-tional coupling of two agents only and will focuson the perceptual crossing in what Tomasello et al.(2005) call a ’dyadic engagement’. Collaborationcould take place in proximal interaction, involving ajointly manipulated physical object, or in distal inter-action, where the agents are not in direct physicalcontact with each other.
An example of proximal interaction is presentedin Figure 1, which shows a collaborative labyrinthgame, consisting of a maze with holes and a ball.The goal is to guide the ball through the mazeby slating the board while preventing it from fallinginto holes. By constraining the dimensionality of thesensorimotor coupling, this scenario facilitates theemergence of cooperative strategies.
An example of distal interaction is presented inFigure 2, depicting a collaborative target acquisitiongame, which requires partners’ coordinated actionsand is performed in a shared mediated environment.In recent studies (Trendafilov et al. (2014, 2011))we investigated the dynamics of social interactions,building on the minimalist experimental paradigm ofLenay et al. (2007), and exploring the emergenceof cooperative strategies using limited modes ofcommunication. In these studies the interaction waslimited to scrolling a finger on a touch-sensitivetactile device. Given the all-or-none nature of the
Figure 2: An example of perceptual interaction, involvingthe exploration of a shared one-dimensional space,utilising touch-sensitive tactile devices. When the avatar ofan agent (green/black) overlaps with a fixed object (blacksquare, on the left), the object is perceived by that agent.When the avatars of A and B (opposite in black and green)meet, the agents perceive each other.
sensory feedback, the perception of an object waspossible only by means of dynamic exploration of theshared one-dimensional space. More precisely, thespatial characteristics of an object could be definedby specific ’laws of sensorimotor contingencies’,which make it possible to anticipate the sensoryconsequences of one’s actions in the course of anactive exploration (O’Regan and Noe (2001), Noe(2005)).
In both of the examples, standard performancemetrics may not be descriptive for the level ofcoordination, since high levels of coupling mayachieve low levels of performance, and vice versa, assuggested in Trendafilov et al. (2015). For example, asmooth coordinated performance (Figure 2), may beslow in locating objects, resulting in a lower score.Alternatively, jumping from one object to the next,could appear as a less coordinated behaviour, butat the same time lead to a higher score.
3. MEASURE OF COORDINATION
In recent work (Trendafilov et al. (2015)) weproposed an information-theoretic approach toquantify coordination, based on mutual information,which is defined for random variables XA and XB
given XS as follows
Ip(XA : XB |XS) =∑xS
p(xS)Ip(XA : XB |xS), (1)
where
Ip(XA : XB |xS) =
=∑xA
p(xA|xS)∑xB
p(xB |xA, xS) logp(xB |xA, xS)
p(xB |xS).
2
iUBICOM’15
Controller
BA
Controller
ActuatorsMotor signal
SensorsFeedback
SensorsFeedback
ActuatorsMotor signal
SharedWorld
Action Stimuli
ActionStimuli
Environment
Figure 3: Schematic diagram of a dyad cognitive sensorimotor loop in a shared computer-mediated environment.
Furthermore
Ip(XA : XB |XS) ≤ min(Hp(XA), Hp(XB)), (2)
which gives a characteristic upper bound, providedby the minimum of the two entropies. In thisformalism XA and XB denote the actions of theagents A and B respectively, and XS denotes thestate of the shared environment. Intuitively, themutual information in Equation 1 captures how wellwe can predict the behaviour of A, if we know thebehaviour of B, for a given state of the sharedenvironment S. If XA and XB are conditionallyindependent, then the coordination is zero, meaningthat there is no corelation between the actions of Aand B. Equation 2 provides the range of this measurefor a particular stochastic model and can serve as abenchmark in system evaluation.
4. STOCHASTIC MODEL
In order to apply Equation 1 to experimental data, weneed an empirical approximation of the probabilitydistribution. This, in turn, requires the definition ofa discrete stochastic model (i.e. random variablesXA, XB , XS) for a particular system, and thepopulation of conditional probabilities from empiricaldata. The approach relies to a great extent on thequality of the model – the more accurate the model,the more reliable the measure it implies. In thediscretisation of the continuous sets of actions andenvironmental states, we need to find a trade-offbetween space granularity and measure reliability,which is influenced by the density of our empiricaldata. Higher resolution spaces usually require largeramounts of data to provide a reliable empiricaldistribution, as data sparsity could bias the model.
Feedback in the perception–action loop is subjectto disturbances, such as noise and delays, whichaffect the quality of experimental data and haveimplications on the modelling process. Lag isinevitable and can be attributed to properties ofthe human cognitive and sensorimotor system,input/output devices and software (Figure 3).Sampling rates of input and update rates of outputdevices are major contributors. Lag is increased
further due to ’software overhead’ – a looseexpression for a variety of system-related factors.Communication modes, network configurations,number crunching, and application software allcontribute. In human–human mediated interaction,however, we also have to account for the variableresponse time of a human decision maker, which– unlike machines – varies across individuals anddepends on various factors, making it unpredictable.Other sources of disturbances are different typesof noise due to digital sensor imprecision, humansensorimotor inaccuracy, transmission interference,etc. In order to tackle the effect of these factorson experimental data, we need to apply advancedfiltering and delay compensation techniques, prior tothe computation of this measure.
5. RESULTS
We applied the above method to data collected in alabyrinth game study. A sample of force data, derivedfrom the electromyograms (EMG) of the players’fingers, is shown in Figure 4 (top).
0 0.5 1 1.5 2 0
20
40
60
Time (sec)
Forc
e
Player A
Player B
0 0.5 1 1.5 2 0
20
40
60
Time (sec)
Forc
e
Player A
Random
Figure 4: Time series from an experimental trial yielding0.42 bits of coordination (top). Player A vs. a quasi-randomsine wave yielding 0.27 bits of coordination (bottom).
3
iUBICOM’15
0 10 20 30 40 50 600.3
0.35
0.4
0.45
0.5
0.55
Trial
Coord
ination (
bit)
Figure 5: Levels of coordination of 60 consecutive trialssuggesting the learning effect.
Using a variant of Equation 1, which ignores the ballposition as environmental state (XS), we computedthe mutual information of XA and XB , representingboth players’ switches in finger force trajectories.Encoding switches with binary random variables inour stochastic model implies a theoretical upperbound of 1 bit. Applying a delay compensation in therange of 20ms, we computed the corelation betweenswitches across trials. To validate the results, wegenerated a quasi-random sine wave with similarfrequency characteristics (Figure 4 bottom), andcomputed the corelation between this artificial curveand player A’s data, which resulted in 0.27 bitsof coordination, whereas the coordination betweenplayer A and player B measured 0.42 bits. Theevolution of the coordination level across all 60trials of one pair shows signs of a learning effect(Figure 5), represented by a gradual increase from0.4 to 0.45 bits. Further analysis is required todetermine the precise relationship between ourmeasure and standard performance metrics.
The corresponding results of the distal interactionexperiment can be found in Trendafilov et al. (2015).
6. CONCLUSION
In this paper we present an application of mutualinformation to measure coordination between collab-orating parties in the context of two representativecomputer-mediated systems. Our initial results showinteresting trends, however, detailed sensitivity anal-ysis is required to further explore the properties ofthis approach. A rigorous measure could give directinsight into the convergence properties of mutualentrainment and could help provide a firm foun-dation for designers in making informed decisionswhen evaluating collaborative systems. Applying ourmethod, however, requires prior modelling – themore accurate the models, the more costly theyare to create, but the more reliable the measurethey imply. The aim of this paper is to raise theawareness of the research community about thepotential of systematic quantification of coordinationin computer-mediated environments.
ACKNOWLEDGMENT
The authors would like to acknowledge support bythe H2020-641321 socSMCs FET Proactive project.
REFERENCES
Auvray, M., C. Lenay, and J. Stewart (2009).Perceptual interactions in a minimalist virtualenvironment. New Ideas in Psychology 27 (1), 32–47.
Ay, N. and D. Polani (2008). Information flows incausal networks. Advances in Complex Systems.
Lenay, C., I. Thouvenin, A. Gunand, O. Gapenne,J. Stewart, and B. Maillet (2007, August).Designing the ground for pleasurable experience.In Conference on designing pleasurable productsand interfaces, Helsinki, Finland.
Matsumoto, K. and I. Tsuda (1988). Calculation ofinformation flow rate from mutual information. J.Phys. A: Math. Gen. 21(6).
Noe, A. (2005). Action in perception. Cambridge MA:MIT Press.
O’Regan, J. K. and A. Noe (2001). A sensorimotoraccount of vision and visual consciousness.Behavioral and Brain Sciences 24, 939–973.
Schreiber, T. (2000). Measuring information transfer.Phys. Rev. Lett. (85), 461–464.
Tomasello, M. (2009). Why We Cooperate. BostonReview Books.
Tomasello, M., M. Carpenter, J. Call, T. Behne,and H. Moll (2005). Understanding andsharing intentions: the origins of cultural cognition.Behavioral and Brain Sciences 28, 675–735.
Trendafilov, D., S. Lemmela, and R. Murray-Smith(2014). Negotiation models for mobile tactileinteraction. Mobile Social Signal Processing, 64–73.
Trendafilov, D., D. Polani, and R. Murray-Smith(2015, March). Model of coordination flowin remote collaborative interaction. In IEEEUKSim-AMSS 17th International Conference onComputer Modelling and Simulation, UKSim2015(UKSim2015), Cambridge, United Kingdom.
Trendafilov, D., Y. Vazquez-Alvarez, S. Lemmela, andR. Murray-Smith (2011). Can we work this out?:an evaluation of remote collaborative interaction ina mobile shared environment. Proc. MobileHCI,499–502.
Wheeler, J. A. (1990). Complexity, entropy and thephysics of information. Santa Fe Studies in theSciences of Complexity , 328.
4