some markov models for direct observation of behavior · direct observation of behavior...

Some Markov models for direct observation of behavior

James E. Pustejovsky

Northwestern University

May 29, 2013

Direct observation of behavior

• Quantities of interest • Prevalence: proportion of time that a behavior occurs

• Incidence: rate at which new behavioral events begin

• Intensity, contingency, others

• Applications in psychology and education research • Measurement of teaching practice

• Measurement of student behavior

• Evaluating interventions for individuals with disabilities

• Other examples in animal behavior, organizational psychology, social work, exercise physiology

2

Observation recording methods

• How to turn direct observation of a “behavior stream” into data?

• Continuous recording methods • Produce rich data, amenable to sophisticated modeling

• Effort-intensive

• Discontinuous recording methods • Less demanding methods needed in field settings

• Momentary time sampling

• Interval recording

• Other possibilities?

3

Outline

• Model for behavior stream as observed

• Momentary time sampling

• Interval recording

• Some novel proposals

• Efficiency considerations

4

A model for the behavior stream

5

Session time

Interim times

Event durations

D1 D2 D3

E0 E1 E2 E3

1

1 0

is not occuring at time

is occu

0 behavior( )

ring at tim1 beh e avior

0j

i i j

j i

t

tY t

I Dt E D

Behavior stream MTS Interval recording Novel proposals Efficiency

Assumptions

1. Event durations:

2. Interim times:

3. Event durations and interim times are all mutually independent.

4. Process is in equilibrium.

Under this model:

• Prevalence ϕ = μ / (μ + λ)

• Incidence ζ = 1 / (μ + λ)

Alternating Poisson Process

6

Exp(1/ ), 1, 2, , ..~ 3 .jD j

Exp(1/ ), 1, 2, , ..~ 3 .jE j



Momentary time sampling (MTS)

7

• (K + 1) moments, equally spaced at intervals of length L.

• Observer records the presence or absence of a behavior at each moment

• Recorded data are

( ), 0,...,k Y kL k KX

X0 = 0 X1 = 1 X2 = 0 X3 = 0 X4 = 1 X5 = 1 X6 = 1 X7 = 1 X8 = 0

Session time

Model for MTS data • Under the alternating Poisson process, X1,…,XK follow a

discrete-time Markov chain (DTMC) with two states (see e.g., Kulkarni, 2010).

8

X=0 X=1

p0(L) 1 - p0(L)

p1(L) 1 - p1(L)

0

1

( ) Pr( ( ) 1| (0) 0) 1 exp(1 )

( ) Pr( ( ) 1| (0) 1) (1 )exp(1 )

tt Y t Y

tp t

p

Y t Y


MTS model, continued

• Maximum likelihood estimators of ϕ and ζ have closed form expressions (Brown, Solomon, & Stephens, 1975).

• But under more general models.

• Extensive literature, lots of generalizations • stopping rules for observation time

(Brown, Solomon, & Stephens, 1977, 1979; Griffin & Adams, 1983)

• Irregular observation times (e.g., Cook, 1999)

• Random effects to describe variation across subjects (e.g., Cook et al., 1999)

9

E X


Partial interval recording (PIR)

10

• Divide period into K intervals, each of length L.

• For each interval, observer records whether behavior occurred at any point during the interval.


U1 = 1 U2 = 1 U3 = 0 U4 = 1 U5 = 1 U6 = 1 U7 = 1 U8 = 1

Session time

[0, )

0 1 , 1,..., .L

k IU Y k L t dt k K


PIR, continued • Unlike MTS, the mean of PIR data is not readily

interpretable:

11

(1 ) 1 exp1

LE U


Model for PIR data • Define Vk as the number of consecutive intervals where

behavior is present:

• Under the alternating Poisson process, V1,…,VK follow a DTMC on the space {0,1,2,3,…}.

12

max 0 : U 0 .k jk j kV

V=0 V=1

π01 1 - π01

V=2 V=3 …

1 – π12 1 – π23 1 – π34

π12 π23 π34


Whole interval recording (WIR)

13

• Divide period into K intervals, each of length L.

• For each interval, observer records whether behavior occurred for the duration of the interval.


• Equivalent to PIR for absence of event.

W1 = 0 W2 = 0 W3 = 0 W4 = 0 W5 = 1 W6 = 1 W7 = 0 W8 = 0

Session time

[0, )

1 , 1,..., .L

k IW L Y k L t dt k K


Augmented interval recording (AIR)

14

• Divide period into K/2 intervals, each of length 2L.

• Use MTS at the beginning of each interval, to record Xk-1.

• If Xk-1 = 0, use PIR for the remainder of the interval.

• If Xk-1 = 1, use WIR for the remainder of the interval.

X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0

U1 = 1 U2 = 1 W3 = 1 W4 = 0

Session time


Model for AIR data

15

• Define Zk = Uk + Wk + Xk .

• Under the alternating Poisson process, Z1,…,ZK/2 follow a DTMC on {0,1,2,3}, with transition probabilities πab= Pr(Zk = b | Xk-1 = a)

Z=0 U=0, W=0, X=0

Z=1 U=1, W=0, X=0

Z=2 U=1, W=0, X=1

Z=3 U=1, W=1, X=1

π01

π00

π00

π01

π02

π11

π11

π02

π12

π13

π12

π13


Intermittent transition recording (ITR)

16

• Divide period into K/2 intervals, each of length 2L.

• Use MTS at the beginning of each interval, to record Xk-1.

• Record time until next transition as Tk.

X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0

T1 T2 T3 >2L T4

Session time


• Under the alternating Poisson process, T1,X1, …,TK/2,XK/2 have the property that

F(Tk,Xk | T1,X1,…,Tk-1,Xk-1) = F(Tk,Xk|Xk-1)

Model for ITR data

17 Behavior stream MTS Interval recording Novel proposals Efficiency

X=0 X=1

p0(L-t) 1 - p0(L-t)

p1(L-t) 1 - p1(L-t)

T | X=1

T | X=0

fμ(t)

fλ(t)

Asymptotic relative efficiency


• Procedure p, q ∈ {MTS, PIR, AIR, ITR}

• are maximum likelihood estimators based on procedure p

• are asymptotic variances based on inverse of expected information matrix.

• Asymptotic relative efficiency of p versus q

ˆVˆ ˆ,R

ˆE

VA

q

p q

p

ˆVˆ ˆ,R

ˆE

VA

q

p q

p

ˆ ˆ,p p

,ˆ ˆp pV V

PIR AIR ITR

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

MT

SP

IRA

IR

0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence

Tru

e in

cid

en

ce

(fr

eq

ue

ncy p

er

inte

rva

l)

0.5

1.0

2.0

asymptotic relative efficiency

Asymptotic relative efficiency: Prevalence


PIR AIR ITR

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

MT

SP

IRA

IR

0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence

Tru

e in

cid

en

ce

(fr

eq

ue

ncy p

er

inte

rva

l)

0.95

1.00

1.05


blue = column is more efficient red = row is more efficient

PIR AIR ITR

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

MT

SP

IRA

IR

0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence

Tru

e in

cid

en

ce

(fr

eq

ue

ncy p

er

inte

rva

l)

0.5

1.0

2.0


Asymptotic relative efficiency: Incidence


blue = column is more efficient red = row is more efficient

• Evaluating these models & methods • Field testing

• When is it okay to treat ML estimates from individual sessions as “pre-processing”?

• Lots still to do • Build data-collection software

• Extensions to between-period regression models

• Random period/subject effects

• PIR, AIR, ITR under other distributional assumptions?

Future work

21

Questions? Comments?

[email protected]

mailto:[email protected]

PIR model • Transition probabilities are uglier than MTS:

where and f(j) is the j-fold recursion of f.

23

/ )

,

(

1 1 1 (Pr 1| 0)1 L

j j k k

jV j V j fe

/ (1 )

/(1 )( )

1

( )

(1 )

L

L

qf q

q e

e

AIR, continued

24

Event occurring at time (k-1)L?

Event ends before time kL?

Event begins before time kL?

Xk-1=1, Uk=1

Xk-1=0, Wk=0

Yes

No

Wk=1

Wk=0

Yes

No

Vk=1

Vk=0

Yes

No


AIR model

25 Extras

• Transition probabilities are given by

2 /

00

10 0

Le

2 /

01 0

11 0

(2 )

( )

1

2

L L

p L

e p

02 0

2 /

12 01

(2 )

(2 )L

p

e p

L

L

03

2 /

13

0

Le

• Under alternating Poisson process,

ITR model

26

1 1 /2 /2 0

/2

1

1

1

, ,..., , |

; , ,;| , |

K K

K

k k k

k

k k

f t x

f t x

x t x

x f x t


some markov models for direct observation of behavior · direct observation of behavior...

Documents