assessing moderated effects of mobile...

Assessing Moderated Effects

of Mobile Health

Interventions

on Behavior

Susan Murphy

09.28.16

HeartSteps

JITAI

JITAIs

JITAIs

JITAIs

2

The Dream!

“Continually Learning Mobile Health Intervention”

• Help you achieve, and maintain, your desired

long term healthy behaviors

– Provide sufficient short term reinforcement to enhance

your ability to achieve your long term goal

• The ideal mobile health intervention

– will engage you when you need it and will not intrude

when you don’t need it.

– will adjust to unanticipated life events

Example: Heart Steps

oGoal: Develop an mobile activity coach

for individuals who have coronary artery

disease

oThree iterative studies:

o 42 day pilot micro-randomized study with

sedentary individuals,

o 90 day micro-randomized study,

o 365 day personalized study3

Heart Steps

Data from: Wearable band → activity and

sleep quality; Smartphone sensors →

busyness of calendar, location, weather;

Self-report→ stress, utility

a) In which contexts should the smartphone

provide the user with an activity suggestion?

b) On which evenings should the smartphone

ping the user to plan his/her physical activity

over the next day?4

5

Data from wearable devices that

sense and provide treatments

• On each individual: ��, ��, ��, … , �� , �� , ���, …

• t: Decision point

• ��: Observations at tth decision point (high

dimensional)

• ��: Action at tth decision point (treatment)

• ���: Proximal outcome (e.g., reward, utility, cost)

6

Examples

1) Decision Points (Times, t, at which a

treatment can be provided.)1) Regular intervals in time (e.g. every 10

minutes)

2) At user demand

Heart Steps: a) approximately every 2-2.5 hours

(activity suggestions) and

b) every evening (planning prompts)

7

Examples

2) Observations Ot

1) Passively collected (via sensors)

2) Actively collected (via self-report)

Heart Steps: classifications of activity,

location, weather, step count, busyness of

calendar, usefulness ratings, adherence…….

8

Examples

3) Actions At

1) Types of treatments that can be provided at

a decision point, t

2) Whether to provide a treatment

Heart Steps: a) tailored activity suggestion (yes/no)

and b) evening planning prompt (yes/no)

9

Tailored Walking

Activity Suggestion

10

Examples

4) Proximal Outcome Yt+1

Heart Steps: a) Step count over next 30 minutes

(activity suggestions),

b) Step count on following day (planning prompts)

11

Continually Learning Mobile Health

Intervention

1) Trial Designs: Are there effects of the actions on the

proximal response? experimental design

2) Data Analytics for use with trial data: Do effects vary by

the user’s internal/external context,? Are there delayed

effects of the actions? causal inference

3) Learning Algorithms for use with trial data: Construct a

“warm-start” treatment policy. batch Reinforcement

Learning

4) Online Algorithms that personalize and continually update

the mHealth Intervention. online Reinforcement Learning

12

Heart Steps Micro-Randomized Trial

For each of n individuals and at each of T decision

points, treatment is repeatedly randomized:

a) Activity suggestion (T=210 randomizations)

• Provide a suggestion with probability .6; do nothing

with probability .4

b) Evening planning prompt (T=42)

• Prompt activity planning for following day with

probability .5; do nothing with probability .5

13

Conceptual Models

Generally data analysts fit a series of increasingly more

complex models:

Yt+1 “~” α0 + α1T Zt + β0 At

and then next,

Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St

and so on…

• Yt+1 is subsequent activity over next 30 min.

• At = 1 if activity suggestion and 0 otherwise

• Zt summaries formed from t and past/present observations

• St potential moderator (e.g., current weather is good or not)

14

Conceptual Models

Generally data analysts fit a series of increasingly more

complex models:

Yt+1 “~” α0 + α1T Zt + β0 At

and then next,

Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St

and so on…

α0 + α1T Zt is used to reduce the noise variance in Yt+1

(Zt is sometimes called a vector of control variables)

15

Causal Effects

Yt+1 “~” α0 + α1T Zt + β0 At

β0 is the effect, marginal over all observed and all

unobserved variables, of the activity suggestion on

subsequent activity

Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St

β0 + β1 is the effect when the weather is good (St=1),

marginal over other observed and all unobserved variables,

of the activity suggestion on subsequent activity.

16

Data Scientist’s Goal

• Challenges:

• Time-varying treatment (At , t=1,…T)

• “independent” variables: Zt, St that may be

affected by prior treatment

• Develop data analytic methods that are

consistent with the scientific understanding of

the meaning of the β regression coefficients

• Robustly facilitate noise reduction via use of

controls, Zt

17

“Centered and Weighted

Least Squares Estimation”

• Simple method for complex data!

• Enables unbiased inference for a causal,

marginal, treatment effect (the β‘s)

• Inference for treatment effect is not biased by

how we use the controls to reduce the noise

variance in Yt+1

https://arxiv.org/abs/1601.00237

18

Application of the “Centered and

Weighted Least Squares Estimation”

method in an initial analysis of

Heart Steps

19

Heart Steps Pilot Study

On each of n=37 participants:

a) Activity suggestion, At

• Provide a suggestion with probability .6

• a tailored sedentary-reducing activity suggestion

(probability=.3)

• a tailored walking activity suggestion (probability=.3)

• Do nothing (probability=.4)

• 5 times per day * 42 days= 210 decision points

20

Conceptual Models

Yt+1 “~” α0 + α1Zt + β0 At

Yt+1 “~” α0 + α1Zt + β0 At + β1 At dt

• t=1,…T=210

• Yt+1 = log-transformed step count in the 30 minutes after

to the tth decision point,

• At = 1 if an activity suggestion is delivered at the tth

decision point; At = 0, otherwise,

• Zt = log-transformed step count in the 30 minutes prior to

the tth decision point,

• dt =days in study; takes values in (0,1,….,41)

21

Pilot Study Analysis

Yt+1 “~” α0 + α1Zt + β0 At , and

Yt+1 “~” α0 + α1Zt + β0 At + β1 At dt

Causal Effect Term Estimate 95% CI p-value

β0 At

(effect of an activity suggestion)

�=.13 (-0.01, 0.27) .06

β0 At + β1 At dt

(time trend in effect of an

activity suggestion)

�

= .51

�

= -.02

(.20, .81)

(-.03, -.01)

<.01

<.01

22

Heart Steps Pilot Study

On each of n=37 participants:

a) Activity suggestion

• Provide a suggestion with probability .6

• a tailored walking activity suggestion

(probability=.3)

• a tailored sedentary-reducing activity suggestion

(probability=.3)

• Do nothing (probability=.4)

• 5 times per day * 42 days= 210 decision points

23

Pilot Study Analysis

Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t

• A1t = 1 if walking activity suggestion is delivered at the tth

decision point; A1t = 0, otherwise,

• A2t = 1 if sedentary-reducing activity suggestion is

delivered at the tth decision point; A2t = 0, otherwise,

24

Initial Conclusions

• The data indicates that there is a causal

effect of the activity suggestion on step

count in the succeeding 30 minutes.

• This effect is primarily due to the walking

activity suggestions.

• This effect deteriorates with time

• The walking activity suggestion initially increases

step count over succeeding 30 minutes by 271

steps but by day 20 this increase is only 65 steps.

25

Heart Steps Pilot Study

On each of n=37 participants:

b) Evening planning prompt, At

• Provide a prompt with probability .5• Prompt using unstructured activity planning for

following day with probability=.25

• Prompt using structured activity planning for

following day with probability=.25

• Do nothing with probability=.5

• 1 time per day * 42 days= 42 decision points

26

Conceptual Models

Yt+1 “~” α0 + α1Zt + β0 At

Yt+1 “~” α0 + α1Zt + β0 At Wt + β1 At (1-Wt)

• t=1,…T=42

• Yt+1 = square root-transformed step count on the day after

the tth day,

• At = 1 if activity planning prompt on the evening of the tth

day; At = 0, otherwise,

• Zt = square-root step count on the tth day,

• Wt =1 if Sunday through Thursday; Wt =0, otherwise

27

Pilot Study Analysis

Yt+1 “~” α0 + α1Zt + β0 At , and

Yt+1 “~” α0 + α1Zt + β0 At Wt + β1 At (1-Wt)

Causal Effect Term Estimate 95% CI p-value

β0 At

(effect of planning)

�= 1.7 ns ns

β0 At Wt + β1 At (1-Wt)

(effect of planning for

weekday (Wt =1) and

for weekend(Wt =0)

�

= 3.6

�

<0

(.74, 6.4)

ns

<.02

ns

28

Heart Steps Pilot Data

On each of n=37 participants:

b) Evening planning prompt

• Provide a prompt with probability .5• Prompt using unstructured activity planning for

following day with probability=.25

• Prompt using structured activity planning for

following day with probability=.25

• Do nothing with probability=.5

• 1 time per day * 42 days= 42 decision points

29

Conceptual Model

Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t

• Yt+1 = square root-transformed step count on the day after

the tth day,

• A1t = 1 if unstructured activity planning prompt on the

evening of the tth day; A1t = 0, otherwise,

• A2t = 1 if structured activity planning prompt on the

evening of the tth day; A2t = 0, otherwise,

• Zt = square-root step count on the tth day,

30

Pilot Study Analysis

Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t t=0,…T=41

• A1t = 1 if unstructured activity planning prompt on the

evening of the tth day; A1t = 0, otherwise,

• A2t = 1 if structured activity planning prompt on the

evening of the tth day; A2t = 0, otherwise,

Causal Effect Term Estimate 95% CI p-value

β0 A1t + β1 A2t β��

= 3.1

�

>0

(-.22, 6.4)

ns

.07

ns

31

Heart Steps Pilot Study Analysis

Yt+1 “~” α0 + α1Zt + β0 A1t Wt + β1 A1t (1-Wt)

+ β2 A2t Wt + β3 A2t (1-Wt)

A1t = 1 if unstructured activity planning prompt on the

evening of the tth day; A1t = 0, otherwise,

Wt =1 if Sunday through Thursday; Wt =0, otherwise

Causal Effect Estimate 95% CI p-value

β0 A1t Wt + β1 A1t (1-Wt)

+ β2 A2t Wt + β3 A2t (1-Wt)

�

= 5.3

�

<0, �

>0,

β�

<0

(2.2, 8.5)

all ns

<.01

all ns

32

Initial Conclusions

• The data indicates that there is a causal

effect of planning the next day’s activity on

on the following day’s step count

• This effect is due to the unstructured planning

prompts.

• This effect occurs primarily on weekdays

• On weekdays the effect of an unstructured

planning prompt is to increase step count on the

following day by approximately 780 steps.

33

Discussion

Problematic Analyses

• GLM & GEE analyses

• Random effects models & analyses

• Machine Learning Generalizations:

– Partially linear, single index models & analysis

– Varying coefficient models & analysis

--These analyses do not take advantage of the micro-

randomization. Can accidentally eliminate the advantages of

randomization for estimating causal effects--

34

Discussion

• Randomization enhances:

– Causal inference based on minimal structural

assumptions

• Challenge:

– How to include random effects which reflect

scientific understanding (“person-specific” effects)

yet not destroy causal inference?

35

It takes a Team!