My Experience

Jon FroehlichCSE490f, October 17, 2006

A Context-Aware Tool for In Situ Data Collection

2

Challenge

• Naturalistic data collection is – time-consuming, – costly, – resource intensive

• Desktop-based studies often in controlled usability labs– Context/environment not typically an issue

3

Goal Today

• Introduce In Situ Self-Report Methods– Convince you that they are useful!

• Studying human behavior• Validating, assessing, building UbiComp apps

• Introduce our new tool– The My Experience (Me) Tool– Context-aware self-report app for mobile devices

• Go over some XML• Questions/Answers

4

In Situ (“in place”)

• Studying people in naturalistic settings:– Direct observation– Indirect observation– Diary method– Experience Sampling Method (ESM)

ESM

The Experience Sampling Method

6

History

• Larson / Csikszentmihalyi [1983]– Procedure for studying what

people• Do• Think• Feel

– Asking individuals to provide systematic self-reports

• Random occasions• During waking hours

7

Primary Sampling Technique

+

Called “signal-contingent” sampling...

(Random Beeps)

8

Other Sampling

• Interval-contingent sampling– Sample on experiences at fixed times– Good for time series data– Typically less burdensome to subjects

• They begin to expect prompts

• Event-Contingent sampling– Report on experiences based on event of interest– Subject must be “cognitively-engaged” into own

actions

Benefits of ESM

Psychological Perspective

10

Immediacy

• Reduce recall memory bias– Important for qualitative data [Barrett 1998]

• Difficult to remember mood, feeling, thoughts of particular events retrospectively

11

Multiple Assessments

• Multiple assessments over time allows for studying within-person processes [Conner 2004]– Time-series data– Observe patterns– Look for correlations between elements

• Medication taken• Perceived pain

– Calibrate responses per subject

12

Natural Setting

• Naturalistic data collection method– Outside the lab

• “Ecologically valid”

– Studying behaviors in real-life situations…

13

Studies

• Psychology/Medical Sciences*– Smoking, Asthma, Pain– Alcoholism/binge drinking; migraine

headaches, eating disorders– Self-esteem, depression coping, flow– Many more…

* List lifted from Conner 2004

ESM Modernized

15

Computerized ESM

• Advantages– Ensures compliance– Sophisticated presentation

• Conditionals• Probabilities• “Question pools”

– Record reaction times– Data already in computer

• reduces data entry error

16

Computerized ESM

• Disadvantages– Input constraints (limited free response)– Human factors

• Small screen, buttons, etc.• Requires some prior experience with technology

– Costs• Particularly for large-n subject studies

17

Context-Triggered Sampling

• New sampling technique– First introduced by Intille et al [2003] with

Context-Aware Experience Sampling (CAES) Tool

• Use sensor data to achieve more targeted triggers

18

Immediacy

• Allows us to validate/assess context-aware algorithms– “Did you just finish jogging?” Yes/No– “Are you at work right now?” Yes/No– “Did you just finish your conversation?” Yes/No

19

Multiple Assessments

• Provide training data for machine learning– Models tailored per subject

• Look for contextual features where algorithm performed well and not-so-well

20

Natural Setting

• Validate user interfaces, sensors, algorithms, etc.– Within the environment of actual deployment

21

HCI/UbiComp Studies

• Computerized ESM– Personal Server [Consolvo et al 2003]– Location disclosure [Consolvo et al 2004]

• Context-triggered ESM– Interruptability [Intille et al 2005]

• Using the CAES Tool

– Place preferences [Froehlich et al 2006]• Using the Me Tool

My Experience Tool

23

My Experience Tool

• Advantages– Multi-media capture (audio, video, etc.)– Reduces some human factor issues

• Audio playback of questions/answers• Settable fonts, colors, sizes• Simplified interaction

– Real-time wireless connectivity– Context-triggers

• Sensor combinations…

– Modern platform support• Mobile phones and PDAs

24

My Experience Tool

• Disadvantages– XML input file with sensor scripts

• Limited usability for non-programmers• Brightside: Looking into creating a front end!

– Equipment costs• Currently requires modern device

– Windows Mobile 5.0

• Brightside: Prices continue to decline– Reuse equipment

25

Context-Awareness

• Advanced sensor support– Scenario: Fitness Study

• Detect: Running• Wait to prompt…

– Scenario: Elderly Study• Detect: Medication bottle picked up• Trigger survey if it’s past lunch and not detected

– Scenario: Sensor failure• Watchdog• Trigger survey if no sensor state change

26

Evolving Context-Awareness

• Use machine learning– Real-time customization of inferencing

algorithms– Hopefully prompts become more targeted– Provide evidence that algorithms being tested

can be tailored per person

Example Usage

Voting With Your Feet

28

Voting With Your Feet

• Investigated relationship between place visit behaviors and place preference– How often you go to a place…– How far you travel to get there…

• 4-week study, 16 participants– Participants recruited from Seattle area– My Experience Tool– Online web diaries

29

ESM Triggers

• Two triggers– Mobility

• Using GSM signals, can detect movement• When stationary for 10 mins, trigger survey

– Time• Essentially a fail-safe• No movement sensed for 1 hr, trigger survey

30

High Level Results

• 4,295 ESM questionnaires administered– 3,458 Completed (80.5%)

• 368 web diary sessions completed• On average,

– 28 days of ESM data per participant– 216 completed ESM surveys/participant– 1.5 minute survey completion time

• 1,981 individual place visits– 862 public place visits (~1.9/day)

31

Sensor vs. Time Triggered Surveys

0%

20%

40%

60%

80%

100%

Store OtherPublic

Restaurant Café In Transit Bar Work Personal Home

Place Category

% C

om

ple

ted

Sensor Triggered

Time Triggered

Demo

33

34

35

Planned Studies

• UbiFit 2.0 (Summer 2006)– w/Sunny Consolvo et al

• Elderly Care (Fall 2006)– w/Beverly Harrison et al

• Rehabilitative Medicine (Planning Phase)– w/Mark Harniss & Kurt Johnson

Feedback!

Jon Froehlichjfroehli@cs.washington.edu

University of WashingtonComputer Science and Engineering

37

Compliance

• Can be an issue…– Stone et al [2002]– Paper diaries fitted with photosensors that

detected light and recorded when the binder was open and closed

• Self-report compliance: 90%• Actual compliance: 11%

38

My Experience Tool

• C# (.NET CF 2.0)

39

Goals

• Extend computerized self-report to mobile phones

• Provide evidence to support context-triggered sampling

• Use machine learning techniques to customize sampling per subject in real-time