the hidden stories behind missing data
TRANSCRIPT
partially joint work with Henry Potts and Katarzyna Stawarz; partially with the Help4Mood Consortium
The Hidden Stories Behind Missing Data
Maria WoltersUniversity of Edinburgh
[email protected]@mariawolters
http://www.slideshare.net/mariawolters
Overview
❖ What is Missing Data?
❖ The Story of Appropriation: Help4Mood
❖ The Story of Monitoring Work: Activity Trackers
❖ The Story of Hidden Missing Data: Electronic Health Records
Missing Data
❖ informally: observations that we would like to be there, or that should be there, but that are not
❖ Statistical treatment differs according to whether missing data are
❖ completely random
❖ predictable from existing data
❖ not predictable from existing data
Goal❖ Tell the hidden stories behind missing data by
understanding and describing data generation processes
❖ qualitatively for deeper understanding
❖ quantitatively to feed into data analysis and visualisation
❖ Current stage: setting out problem space in order to pick appropriate models
The Appropriation of Help4Mood
depressioncomix.tumblr.com
Depression is a change
relative to an individual baseline
Help4Mood: Supporting People with Depression
• daily monitoring • of activity using
actigraph • of mood, thought
patterns & psycho-motor symptoms using talking head GUI
• weekly one-page reports to clinicians
Maria K. Wolters, Juan Martínez-Miranda, Soraya Estevez, Helen F. Hastie, Colin Matheson (2013). Managing Data in Help4Mood AMSYS ICST DOI: 10.4108/trans.amsys.01-06.2013.e2
One Help4Mood per Day❖ Users interact with Help4Mood once a day
❖ Sessions can be short, medium, or long
❖ Planning algorithm ensures
❖ different kinds of data are collected regularly, e.g.
❖ mood: every day
❖ speech: 1-3 times per week
❖ sessions are varied
Patient Focused Development1. Actigraphy
2. Mood Tracker +Actigraphy
3. Mood Tracker,Thought Patterns,Speech + Actigraphy
4. Mood Tracker,Thought Patterns,Behavioural Activation,Relaxation,Speech + Actigraphy
Case Studies
Pilot Randomised Controlled
Trial
People with history of depression
People with current depression
Pilot RCT Participants❖ Participants with Major Depressive Disorder (SCID
diagnosed)
❖ Use Help4Mood for 4 weeks
❖ Background measures include demographics and attitudes to computers
❖ (Pre/Post measures to establish change)
❖ Qualitative interviews at intake and debriefing for those randomized to Help4Mood
Participants and Usage Patterns
❖ 18 in Romania, 7 in Scotland, 2 in Spain (EU Project)
❖ 14 treatment as usual (age 42 years +/- 10), 13 Help4Mood (age 35 +/- 12)
❖ None formally tracked or measured their mood before, but some used introspection
❖ Half used it regularly, but that was not daily; instead, it was 2-3 times per week. Why? Appropriation
What is Going On With Me?
The monitoring part helped me understand some things [. . .] sometimes I did not realize how I felt that day, how happy I was or how active I was. The system helped me observe these things and also control them. (RO14, female, 20–29)
Help4Mood as Coping[. . .] when I felt very upset, very low, I preferred longer sessions [. . .] because the session is pretty much focused on analyzing the thinking patterns of depressed people, and those recovering from depression. It gave me space to reflect on my own thoughts and feelings and gave me alternative ways that I could resonate with. (RO17, female, 30–39)
The most important thing for me was those relaxation exercises and I think that it would have been very useful for the patient to have direct access to them. (RO03, female, 50–59)
It Doesn’t (Quite) Work This Way
http://imgarcade.com/1/depressed-stick-figure/
Well-DesignedMonitoring
Self-Help Internet-Based
Therapy
+
http://www.acog.org/About-ACOG/ACOG-Departments/Long-Acting-Reversible-Contraception
https://www.osneybuyside.com/forget-big-data-just-collect-smart-data/
Protected Peer Support
http://www.clipartsfree.net/small/3977-game-piece-group-clipart.html
It’s a complex adaptive system
http://www.thebolditalic.com/articles/3609-the-stick-figure-guide-to-kicking-depression
Individualised monitoring based on what person has & does
Coping and getting better:• Twitter, exercise, kindness• Friends • Medications• GP
Productive (!) self insightand reflection
The Story of Monitoring Work
The Chore of Monitoring
If at all possible, it would be good not to have the same questions every day; or even if the questions are the same, the phrasing should be different. At some point it gets boring—I think this could be changed. (RO15, female, 30–39)
Self-Reflection is Hard Work
“This wasn’t very pleasant. Because you don’t go to therapy every day. You wouldn’t go every day; you would go maybe once a week or two or three times maybe, but not every day. It’s a bit too much to use it every day.” (P01, Case Studies)
TRACKER
whatswimming
weightlifting
team sports
self-reporteffort
e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015
TRACKER
whenworried well
motivated for change
not tracking during lazy days
whatswimming
weightlifting
team sports
stigmaself-report
effort if there is a need
e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015
TRACKER
when
who job (e.g. nurses)
allergies
wrist anatomy
forgetting
to wear to bringto charge
worried well
techy
motivated for change
not tracking during lazy days
whatswimming
weightlifting
team sports
style / fashion
stigmaself-report
effort if there is a need
e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015
http://whitemenwearinggoogleglass.tumblr.com/image/51710586195
http://www.cio.com/article/2452759/health/wearable-techs-dilemma-too-much-data-not-enough-insight.html
Apple Watch Hermès
TRACKER
when
who job (e.g. nurses)
allergies
wrist anatomy
forgetting
to wear to bringto charge
worried well
techy
motivated for change
not tracking during lazy days
device
breaks
no longer holds charge
lost / stolen
whatswimming
weightlifting
team sports
no Internetstyle / fashion
stigmaself-report
effort
not synchingproperly
if there is a need
e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015
http://thoughtstipsandtales.com/2014/11/06/fitbit-fun-ten-months-later/
http://thoughtstipsandtales.com/2015/03/05/fabulous-fitbit-accessory-to-keep-the-clasp-from-opening/
TRACKER
when
who
forgetting is a function of- user characteristics- illness- external stress
fit with identity
device
what
detailed user modeldevice model
connectivity model
fit with activities
effort required to track activity
fit with ulterior need(why tracking?)
TRACKER
when
who
forgetting is a function of- user characteristics- illness- external stress
fit with identity
device
what
detailed user modeldevice model
connectivity model
fit with activities
effort required to track activity
fit with ulterior need(why tracking?)
All except for ulterior need and identity amenable to formal/ quantitative modelling from appropriate discipline.
When looking at studies:
People will do things for research, to benefit others, to be part of something bigger,
and maybe also to earn some money,
that they wouldn’t do for themselves.
The Story of Hidden Missing Data
Hiding in Electronic Health Records
❖ Data go dark because of quality issues in data entry and management (e.g., Chan et al., 2014, Medical Care Research and Review)
❖ People go to the doctor when worried about something, which increases likelihood of detection of other problems.
❖ So does diabetes really increase your cancer risk, or is your cancer more likely to be spotted in regular check ups? (e.g, Badrick and Renehan, 2014, Eur J Cancer)
We can only detect what is being observed. Does that mean regular screening and monitoring?
What is the human cost of a false positive?
What is the human cost of a false negative?
What are the numbers needed to treat?
Questions? [email protected]
http://www.thebolditalic.com/articles/3609-the-stick-figure-guide-to-kicking-depression
Individualised monitoring based on what person has & does
Coping and getting better:• Twitter, exercise, kindness• Friends • Medications• GP
Productive (!) self insightand reflection