experience)sampling)and) ecological)momentary)assessmentexperience)sampling)and)...
TRANSCRIPT
Experience Sampling and Ecological Momentary Assessment
Arthur A. Stone Department of Psychiatry and Behavioral Science
Stony Brook University Stony Brook, NY USA
For the Conference on “The Future of Survey Research,” November 2012
Disclosures: AAS is a Senior ScienNst with the Gallup OrganizaNon and Senior Consultant with PRO ConsulNng, inc.
Overview
• Why go into a day in detail?
• Conceptual posiNon • Methods for ESM and EMA • Pros and Cons from a survey perspecNve
• AlternaNves to ESM/EMA – End of Day Diaries – Yesterday Diaries – ReconstrucNng Yesterday
Why go into a day in detail?
• To capture data on experiences and behavior as accurately as possible – To most capture daily experiences and behavior without fear
of recall bias
– To generate high levels of Ecological Validity • To study Nme usage
– Experiences or acNviNes changing? • To study diurnal paWerns of experience and behavior • To examine within-‐day contemporaneous and lagged
associaNons among experiences and behaviors • To link experienNal and behavioral data with real-‐
Nme, physiological data
Conceptual posiNon
Conceptual PosiNon Regarding Measurement
• “Experienced UNlity” versus “Remember UNlity” • Immediate informaNon is stored in Experien(al Memory
– Stored-‐lived, fleeNng • Memories of experiences are stored in Episodic Memory • Beliefs about experiences are stored in Seman(c Memory
• As Recall Period increases, a shi[ from ExperienNal/Episodic to SemanNc – Robinson & Clore’s work on reacNon Nme
Conceptual PosiNon Regarding Measurement
• Cogni(ve Heuris(cs impact recollecNon – Rules of Thumb, but out of awareness
– Peak-‐end – Current levels
• As a proxy for recall period • As altering accessibility to past memories
– Variability – Dura(on neglect
• These issues are recognized by survey research world and by regulatory agencies – Bradburn’s influenNal 1987 Science arNcle on
autobiographical memory in surveys
Conceptual PosiNon Regarding Measurement
• Concept: Ecological Validity • SituaNons need to be
representaNvely sampled from the relevant universe of situaNons
• Bias can be introduced into measurement
Egon Brunswik
Conceptual PosiNon Regarding Measurement
• Procedures to avoid bias due to recall – Limit recall period – ReconstrucNon/reinstanNaNon of recall period – More precise quesNons about sought a[er informaNon – Limit queries to informaNon that can be recalled
• e.g., salient events may be possible to recall • e.g.., difficulty with highly fluctuaNng states
• ImplicaNons for Survey Research – If only short periods are reasonable, how can surveys
incorporate such assessments – What do single, short-‐term assessments mean?
• Are they of any value? • Are they reasonable for characterizing groups? • What Ns are needed in single assessment surveys?
Methods for ESM and EMA
Overview of CollecNon Methods
MORNING NIGHT
MORNING NIGHT
MORNING NIGHT
MORNING NIGHT MORNING NIGHT
1
2
3
4, 5
?
Real-‐Nme Data Capture
• Capturing Real-‐(me Experience – Experience Sampling Method (ESM)
• Csiksezentmahalyi
• Paper-‐and-‐pencil – Ecological Momentary Assessment (EMA)
• Context of medical research
• GeneralizaNon of ESM to sampling other modaliNes – EMA assesses phenomena at the moment they occur – EMA is dependent upon careful Nming of assessments – EMA methods usually, but not always, involve a substanNal number of repeated observaNons
– EMA measurements are made in the environments that subjects typically inhabit
Electronic Diary Screens
Real-‐Nme Data Capture
• PROs – No recall bias – tapping experiences – Mood can be associated with environmental
qualiNes – Can capture diurnal rhythms
• CONs – Point esNmates – Sampling framework – Can miss important events – Burdensome/expensive
• ModificaNons – “Coverage” model: Small retrospecNve periods
(minutes/hours) to increase coverage of day
AlternaNves to ESM/EMA End-‐of-‐Day Diaries
End-‐of-‐Day Diaries
• EOD diaries are the oldest method for capturing near real-‐Nme data
• Require recall for up to 24 hours – LiWle evidence that major bias due to most heurisNcs
• Mode of AdministraNon (applies to following methods) – Paper-‐and-‐pencil
• “Parking Lot” compliance • Forward-‐filling
– Electronic diary • Time-‐date stamp
– InteracNve Voice Recording (IVR) • Few visual cues
– Internet • Household availability of internet access
End-‐of Day Diaries
• PROs – Rapid assessment of day’s experience
– Allows for extended length studies – Low parNcipant burden – RelaNvely inexpensive
• CONs – Possible recall bias over day, though liWle – Poor resoluNon of day
• Diurnal rhythms
• False PA—NA associaNon
– Compliance issues with some Modes-‐of-‐AdministraNon
End-‐of Day Diaries
• NaNonal Study of Daily Experience – Part of MIDUS survey
(Almeida) – 1,031 respondents; 8
consecuNve brief telephone interviews
– Daily NA: Worthless; Hopeless; Nervous; Restless; Fidgety
– Daily PA: Cheerful; In Good Spirits; Extremely Happy; Calm and Peaceful; SaNsfied; Full of Life
AlternaNves to ESM/EMA Yesterday Diaries
Recall of Yesterday
• Ask person to report about yesterday – Advantage: Can be administered anyNme during day
– Advantage: Covers the enNre day – Disadvantage: Longer recall period
• Gallup OrganizaNon Daily Survey – Since 2 January 2008, 1,000 RDD interviews per day – Several quesNons about Yesterday – Did you experience the following feelings during A LOT OF
THE DAY yesterday? How about __________? NO/YES
– Enjoyment, Sadness, Stressed, Worried
– Data presented is of 337,000 interviews conducted in 2008
Recall of Yesterday
Recall of Yesterday
Recall of Yesterday
• PROs – Very brief assessment
– Low parNcipant burden – Low cost
• CONs – Longer recall period – Current affect may impact recall
– Low resoluNon regarding the day
Day ReconstrucNon Method (DRM)
• Beyond EOD diaries by incorporaNng reconstrucNon of recall period
• IntenNon was to develop a method that could reproduce ESM/EMA from the perspecNve of yesterday
• RaNonale: ReconstrucNon of day via Nme-‐use – Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone,
A. A. (2004). A survey method for characterizing daily life experience: the day reconstrucNon method. Science, 306(5702), 1776-‐1780
Day ReconstrucNon Method (DRM)
• Texas; Columbus, OH and Rennes, France
N ≈ 800-‐900; samples of women age 18+ ini(ally. • Respondents were given 3 envelopes, which were
opened sequenNally. Each envelope contained a quesNonnaire. The quesNonnaires were labeled “Packets” 1-‐3 – Packet 1 contained standard life saNsfacNon quesNons and
some demographics.
– Packet 2 asked respondents to segment the preceding day (a work day) into episodes, like scenes in a movie (mean = 15 episodes); didn’t collect diaries from them at the end.
– Packet 3 asked detailed quesNons about the seung of each episode and about their feelings (shown next)
Day ReconstrucNon Method (DRM)
Day ReconstrucNon Method (DRM)
Day ReconstrucNon Method (DRM)
Day ReconstrucNon Method (DRM)
American Time Use Survey (ATUS) [Princeton Affect & Time Use Survey (PATS)]
• 24-‐hour Nme diary, beginning at 4:00 AM the day before the interview (yesterday) and ending at 4:00 AM on the day of the interview (today)
• Collect all acNviNes engaged in during the day with ATUS instrument
• NIA-‐supported “Experimental Module”: Randomly selects 3 awake episodes in proporNon to duraNon (without replacement) and ask quesNons about feelings a[er complete Nme diary
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
• Does DRM method faithfully reproduce EMA? – Small scale study (n=168) comparing PATS with EMA
– Three days of 6 EMA random reports – Following day telephone interview about the day
American Time Use Survey (ATUS) Princeton Affect & Time Use Survey (PATS)
• PROs – RelaNvely rapid assessment of emoNons
– Sampling of episodes is possible – Associated with Nme-‐usage
– MulNple modes of administraNon
• CONs – Technologically sophisNcated – Skilled interviewers necessary – Costly – 30-‐45 minutes
Transforming Methods for Survey Research
Transforming
• Several on-‐going efforts to create “survey-‐friendly” versions of single-‐day
• Context: Wellbeing assessment • Jacqui Smith at ISR/U Michigan
• Stony Brook’s versions
• Are being tested in: – Health ReNrement Survey
– English Longitudinal Study of Aging – ROBUST
Transforming
From the ISR group
Transforming
From the ISR group
Transforming
From the Stony Brook group.
Transforming Small-‐scale study comparing affect for the day between duraNon-‐weighted DRM and “overall yesterday” assessments. (N=49).
SWB-‐SF: SubjecNve Wellbeing, Short-‐Form
Transforming
Transforming
Summary
• There is considerable interest in characterizing daily experiences and behaviors in real-‐Nme or near real-‐Nme
• There should be a clear raNonale for aWempNng to collect detailed daily informaNon in large-‐scale surveys, because there are logisNc challenges
• Several studies have been successful with alternaNves to ESM/EMA
• However, it is not yet enNrely clear how well EOD and Yesterday approaches replicate EMA, though there is promising recent evidence