contextual dissonance: design bias in sensor-based experience sampling methods
DESCRIPTION
Paper presented at #ubicomp13. Full paper pdf here: http://www.cl.cam.ac.uk/~nkl25/publications/papers/lathia_ubicomp13.pdfTRANSCRIPT
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
@neal_lathia, k. rachuri, c. mascolo (@cecim), j. rentfrowcomputer laboratory, university of cambridge#ubicomp13
Contextual Dissonance?
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
You are tasked with researching X (e.g., X = emotions) in daily life using ubiquitous tech; so you decide to build a system that will:
●Ask participants for assessments of the X they experience
●Collect sensor data to 'objectively' measure participants' contexts and quantify their behaviour
Research Scenario
why would you do this?
●… to explore whether machine learning approaches could infer people's subjective responses/complex behaviours
●… to understand the extent that the broad set of sensor data reflects self-reported behaviour
“...automated tracing is widely used to provide insight into what and when; however, it does not provide the why...”
Froehlich et al.
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
“...researchers are faced with concrete decisions regarding design [...] studies have often been classifed into the three categories of interval-, signal-, and event-contingent protocols...”
Bolger et. al
ESM design: how should I ask questions?
“...sampling to capture data from the sensors of the phone cannot be performed continuously, as this will drain the battery rapidly. However, conservative sampling leads to the loss of valuable behavioural data...”
K. Rachuri
sensor design: how should I sample from sensors?
Both of these design protocols will affect the quantity and quality of data that you receive from participants.
● Shouldn't sense everything all the time: triggers a survey based on a particular sensor
●Ask for subjective responses and, while doing so, sample data from other sensors to gather behavioural signals
Research Scenario
We built a system like this. It includes: sensor data collection, ESM interfaces, etc., and remote reconfguration.
Open Source Smartphone Libraries for Computational Social ScienceN. Lathia, K. Rachuri, C. Mascolo, G. Roussos. 2nd ACM Workshop on Mobile Systems for Computational Social Science.
as an aside...
22 users; 1-month; questions about mood & current context (location, sociability); background sensing from many sensors; triggers remotely reconfgured weekly.
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
… by asking the “what if” question:
Your ESM protocol is driven by the accelerometer's state: questionnaires will be triggered based on when the participant is moving.
Example Research Scenario
...skews sampling towards the later hours of the day
But all sensors have their own distribution...
… so how have I skewed my other results?
P( state(sensor1) = b | state(accelerometer) = a) ~ P( state(sensor1)) = b
Data that I would have received by continuous sampling
Data received by triggering on one sensor's state.
Bias?
Accelerometer ~ Non-Stationary10.61% of the data is non-stationary.
When it is, participants are:95.23% non-silent; 39.24% at home; 14.43% communicating with others.
Full Sample vs. Accelerometer Trigger
Non-silent?37.78% | 95.23%
Communicating with others?4.60% | 14.43%
More Examples?
Microphone ~ Silent/Non-SilentAccelerometer ~ Moving/Not-Moving
Location ~ Home/AwayScreen ~ Using the device
SMS/Calls ~ Communicating with othersProximity ~ Near the phone
Microphone ~ Non-Silent37.78% of the data is non-silent.
When it is, participants are:26.75% non-stationary; 47.12% at home; 9.48% communicating with others.
Full Sample vs. Microphone Trigger
Moving?10.61% | 26.75%
Communicating with others?4.60% | 9.48%
Dissonance; a tension or clash resulting from the combination of two disharmonious elements
Dissonance; between using sensor states to trigger ESM surveys while using sensor data to quantify context and behaviour.
Ok; so replace the accelerometer trigger with sampling uniformly across time.
Example Research Scenario
temporal sampling is more likely to fnd your participant in “dominant” contexts, e.g., at home.
But the response data I get back from participants will not be affected by the choices that I make... right?
Research Scenario
1-month; 4 groups with random weekly trigger orders: (a) screen, (b) communication events, (c) immediately during non-silence, (d) some time after non-silence
“4 of the 6 tests found that the negative affect ratings (and 2 out of 6 for the positive ratings) were signifcantly different from one another with at least 90% confdence.”
who are you with?
alone 33.33% of the time (screen trigger) to 60.77% of the time (microphone trigger)
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
Where do we go from here?Opportunities for more research...
Generalise sensor-enhanced experience sampling tool. Currently in alpha testing.
Working with Android sensors?Try out library!
One of the goals is to enable easy and quick access to sensor data in 2 lines of code.https://github.com/nlathia/SensorManager
Contextual Dissonance:Design Bias in Sensor-Enhanced Experience Sampling Methods
@neal_lathia, k. rachuri, @cecim, j. rentfrowACM Ubicomp 2013
References
● Smyth and Stone. “Ecological Momentary Assessment Research in Behavioral Medicine.” Journal of Happiness Studies 2003.
● Froehlich et al. “MyExperience: A System for In Situ Tracing and Capturing User Feedback on Mobile Phones.” ACM MobiSys 2007.
● Froehlich et al. “UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green Transportation Habits” ACM CHI 2009.
● Rachuri. “Smartphones Based Social Sensing: Adaptive Sampling, Sensing and Computation Offloading.” PhD Thesis 2013.
● Bolger et. al. “Diary Methods: Capturing Life as it is Lived” Ann. Rev. Psychology 2003.