Using observational methods to measure treatment integrity in

psychosocial intervention research

Kimberly S Van Haitsma PhD Director Polisher Research Institute

Madlyn amp Leonard Abramson Center for Jewish Life (formerly Philadelphia Geriatric Center)

1425 Horsham Road North Wales PA 19454-1320

215 371 1895 kvanhaitsmaabramsoncenterorg

Presented at the Gerontological Society of America 64th Annual Scientific Meeting

Pre-Conference Workshop Current and Future Challenges in Designing Behavioral Interventions From Randomized Trials to Community Implementation

November 18th 2011 1230 to 430 1

K2A Knowledge to Action Framework for Public Health

EVALUATION

Translation Supporting Structures Research

Supporting Structures

Institution-

alization

Supporting

Structures

DISCOVERY

STUDIES

EFFICACY

STUDIES

EFFECTIVENESS

AND IMPLEMENTATION

STUDIES

TRANSLATION PHASE

RESEARCH PHASE

INSTITUTION-

ALIZATION

PHASE

KNOWLEDGE

INTO

PRODUCTS

DISSEMINATION

PRACTICE

INSTITUTION-

ALIZATION

DECISION

TO

TRANSLATE

ENGAGEMENT

DECISION

TO

ADOPT

DIFFUSION

Practice-based

Discovery

Practice-based

Evidence

Wilson K Brady T ampLesesne C (2011) An organizing framework for Translation in public health The knowledge to action framework Preventing chronic disease Public health research practice and policy 8(2)1-7 wwwcdcgovpcdissues2011mar10_0012htm

K2A Knowledge to Action Framework for Public Health

EVALUATION

Translation Supporting Structures Research

Supporting Structures

Institution-

alization

Supporting

Structures

DISCOVERY

STUDIES

EFFICACY

STUDIES

EFFECTIVENESS

AND IMPLEMENTATION

STUDIES

TRANSLATION PHASE

RESEARCH PHASE

INSTITUTION-

ALIZATION

PHASE

KNOWLEDGE

INTO

PRODUCTS

DISSEMINATION

PRACTICE

INSTITUTION-

ALIZATION

DECISION

TO

TRANSLATE

ENGAGEMENT

DECISION

TO

ADOPT

DIFFUSION

Practice-based

Discovery

Practice-based

Evidence

Wilson K Brady T ampLesesne C (2011) An organizing framework for Translation in public health The knowledge to action framework Preventing chronic disease Public health research practice and policy 8(2)1-7 wwwcdcgovpcdissues2011mar10_0012htm

Overview of presentation

bull Illustration of how real-time observations can be used to enhance fidelity in psychosocial intervention studies

ndash What behavior observation measures are available

ndash What aspects of fidelity measurement can be optimized by real time observational techniques

3

An illustrative example The Individualized Positive Psychosocial Intervention Study

Funded by

bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of

RT)

ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention

ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior

ndash Methodology focused on real-time observation of dyadic interactions

Intervention Study Design

bull Randomized Controlled Trial Residents randomly assigned to one of two

intervention conditions (IPPI or Attention Control) or Usual Care

CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition

bull Residents (n=180)

bull CNAs (n=84)

bull Number of real-time observations (n=2638 occasions)

Resident Group Assignments

Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)

bull Content of activity selected based on resident preferences

Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine

aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational

Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did

not

Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors

Intervention delivery focused on Communication Skills of CNAs

7

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

An illustrative example The Individualized Positive Psychosocial Intervention Study

Funded by

bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of

RT)

ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention

ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior

ndash Methodology focused on real-time observation of dyadic interactions

Intervention Study Design

bull Randomized Controlled Trial Residents randomly assigned to one of two

intervention conditions (IPPI or Attention Control) or Usual Care

CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition

bull Residents (n=180)

bull CNAs (n=84)

bull Number of real-time observations (n=2638 occasions)

Resident Group Assignments

Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)

bull Content of activity selected based on resident preferences

Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine

aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational

Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did

not

Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors

Intervention delivery focused on Communication Skills of CNAs

7

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Intervention Study Design

bull Randomized Controlled Trial Residents randomly assigned to one of two

intervention conditions (IPPI or Attention Control) or Usual Care

CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition

bull Residents (n=180)

bull CNAs (n=84)

bull Number of real-time observations (n=2638 occasions)

Resident Group Assignments

Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)

bull Content of activity selected based on resident preferences

Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine

aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational

Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did

not

Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors

Intervention delivery focused on Communication Skills of CNAs

7

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Resident Group Assignments

Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)

bull Content of activity selected based on resident preferences

Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine

aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational

Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did

not

Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors

Intervention delivery focused on Communication Skills of CNAs

7

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Intervention delivery focused on Communication Skills of CNAs

7

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in

dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Prescribed Positive Communication Behaviors

Greeting Saying hello goodbye handshake

Explanation Explains what activity is or will be occurring

Courtesy Saying excuse me please thank you yoursquore welcome

Offers choice Offers choice by explicitly identifying 2 alternatives

Asks preference Solicits resident preferences for past present or future activities

Praisecompliment Expresses praise compliment or approval

Reassurance Provides reassurance or validates feelings

Positive touch Gently touching resident stroking arm or back holding hands hugging kissing

Positiveneutral prompt Verbal prompt to initiate an activity without negative tone

Nonverbal prompt Points shows something to orient toward object

Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects

Proscribed Negative Communication Behaviors

Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior

Talks to other Talks to third party while interacting with resident

Rebukedisapprove Reprimand admonish express disapproval

Negative touch Handles resident roughly grabs resident moves resident without warning

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes

related to Quality of Life

9

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Definitions of Observed Resident Behavior Variables

Positive Behavior

Task Engagement engages in conversation manually manipulates or gestures toward an object

Positive Verbal coherent conversation responding to question

Very Positive Verbal complimenting joking singing

Positive Touch physically receptive to another person gently touching stroking hugging kissing

Negative behavior

Aggression hitting kicking throwing things grabbing spitting

Uncooperative pulling away saying ldquonordquo turning head or body away

Very Negative Verbal swearing screaming mocking making strange noises

Negative Verbal incoherent repetitious statements muttering

General Restlessness pacing fidgeting disrobing repetitive movement

Eyes Closed Sits with eyes closed

Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Resident Affective Responses

11

Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel

K (1999) Observed affect and quality of life in dementia

Further affirmations and problems Journal of Mental Health

and Aging 5 69-81

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Detailed information about measurement properties of observational measures of outcomes

12

bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A

Review of Current Methods and Measures for Use with Older Adults with Dementia Research in

Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008

Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649

Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102

Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009

Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x

Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606

Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x

Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

wwwnolduscom

Observational Method used in the IPPI study ldquoThe Observerrdquo

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Measuring Behavior with The Observer

Customized coding scheme

Precisely detail

behaviors amp affect

Time-stamped event log

Annotate behavior via simple key press Coding can be done live or via videorecording

Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity

Maintain time-stamped event log Synchronize with multimodal data

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Pocket Observer be free to move

Code on a handheld device

Mobile observations outdoor use

Add behaviorsmodifiers on the handheld

Supports Smart phones rugged handhelds

PDAs (Windows Mobile 6x)

Gather amp score behavior data live using

Pocket Observer

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Data collection Continuous Coding via Video

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

The ObserverXT Analyzing the data

Descriptive statistics

Frequency How many times did the resident strike out

Duration How long did the resident express pleasure

Latency rate per minute percent observation inter-rater

reliability etc

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451

What is fidelity in psychosocial intervention research

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Receipt Recipient

Assures that the treatment has been received

and understood by the individual

Enactment Recipient

Monitors that the individual performs

treatment-related behavioral skills and

cognitive strategies in relevant real life

settings as intended

What aspects of fidelity are optimally measured by real- time observational

methods

19

Focus Purpose

Design Study

Ensure that a study can adequately test its

hypotheses in relation to the underlying

theory and clinical processes

Training Interventionist

Ensure that interventionists have been

satisfactorily trained to deliver the

intervention to participants

Delivery Interventionist

Ensure that the intervention is delivered as

intended

Fidelity of Treatment Delivery

20

Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia

Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial

interventions to nursing home residents with dementia

Percent who used communication behaviors

CNA Communication Behaviors Total N=84 ()

Po

sit

ive

Greeting 86

Explanation 89

Courtesy 79

Offers choice 80

Asks preference 98

Praisecompliment 79

Reassure 60

Positive announcement 89

Positive touch 96

Positive verbal prompt 92

Non-verbal prompt 89

Task engagement 71

Neg

ati

ve

Negative announcement 1

Ignores resident 6

Talks to another 87

Rebukedisapprove 13

Negative touch 0

Benefits of real- time observation in measuring treatment delivery integrity

22

bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2

bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively

1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24

Fidelity and Study Design How Observational Methods Can Help

23

bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2

bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1

Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice

1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89

Fidelity of Treatment Delivery

20

Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia

Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial

interventions to nursing home residents with dementia

Percent who used communication behaviors

CNA Communication Behaviors Total N=84 ()

Po

sit

ive

Greeting 86

Explanation 89

Courtesy 79

Offers choice 80

Asks preference 98

Praisecompliment 79

Reassure 60

Positive announcement 89

Positive touch 96

Positive verbal prompt 92

Non-verbal prompt 89

Task engagement 71

Neg

ati

ve

Negative announcement 1

Ignores resident 6

Talks to another 87

Rebukedisapprove 13

Negative touch 0

Benefits of real- time observation in measuring treatment delivery integrity

22

bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2

bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively

1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24

Fidelity and Study Design How Observational Methods Can Help

23

bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2

bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1

Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice

1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89

Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial

interventions to nursing home residents with dementia

Percent who used communication behaviors

CNA Communication Behaviors Total N=84 ()

Po

sit

ive

Greeting 86

Explanation 89

Courtesy 79

Offers choice 80

Asks preference 98

Praisecompliment 79

Reassure 60

Positive announcement 89

Positive touch 96

Positive verbal prompt 92

Non-verbal prompt 89

Task engagement 71

Neg

ati

ve

Negative announcement 1

Ignores resident 6

Talks to another 87

Rebukedisapprove 13

Negative touch 0

Benefits of real- time observation in measuring treatment delivery integrity

22

bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2

bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively

1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24

Fidelity and Study Design How Observational Methods Can Help

23

bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2

bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1

Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice

1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89

Benefits of real- time observation in measuring treatment delivery integrity

22

bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2

bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively

1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24

Fidelity and Study Design How Observational Methods Can Help

23

bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2

bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1

Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice

1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89

Fidelity and Study Design How Observational Methods Can Help

23

bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2

bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1

Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice

1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89

Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674

Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia

Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Ne

gati

ve C

NA

co

mm

un

icat

ion

b

eh

avio

rs

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Po

siti

ve C

NA

co

mm

un

icat

ion

be

hav

iors

Greeting

Explanation

Courtesy

Offers choices

Asks preference

Praisecompliment

Reassurance

Positive verbal prompt

Positive announcement

Positive touch

Non-verbal prompt

General conversation

Verbally engaged in task

Physically engaged in task

Correlation is significant at the 0001 level (2-tailed)

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement

Ignores resident

Talks to another 08 12 09

Rebukedisapprove 06 12

Negative touch

Greeting 35 16 38 20

Explanation 46 20 51 24

Courtesy 47 27 46 32

Offers choices 34 21 42 13

Asks preference 41 19 52 16

Praisecompliment 38 35 54 20

Reassurance 16 22 10

Positive verbal prompt 29 08 48 11

Positive announcement 34 12 37 14

Positive touch 22 14 40 23

Non-verbal prompt 43 26 42 16

General conversation 66 09 19

Verbally engaged in task 39 43 90 21

Physically engaged in task 07 12 22 20

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement 11

Ignores resident

Talks to another 26 10 28 08

Rebukedisapprove 10

Negative touch

Greeting 15 11 -09 08

Explanation 15 15 -12 20 12

Courtesy -14

Offers choices 11 12 -12

Asks preference 17 14 -14 10 11

Praisecompliment 07 -13

Reassurance 20 12

Positive verbal prompt 13 17 -12 13 10

Positive announcement 08 11 -10 12

Positive touch 09 12 -08

Non-verbal prompt 09 -07 -15 07 09

General conversation 11 09 -08 -18 09 09

Verbally engaged in task 10 -12

Physically engaged in task 15

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement 11

Ignores resident

Talks to another 08 12 09 26 10 28 08

Rebukedisapprove 06 12 10

Negative touch

Greeting 15 11 -09 08 35 16 38 20

Explanation 15 15 -12 20 12 46 20 51 24

Courtesy -14 47 27 46 32

Offers choices 11 12 -12 34 21 42 13

Asks preference 17 14 -14 10 11 41 19 52 16

Praisecompliment 07 -13 38 35 54 20

Reassurance 20 12 16 22 10

Positive verbal prompt 13 17 -12 13 10 29 08 48 11

Positive announcement 08 11 -10 12 34 12 37 14

Positive touch 09 12 -08 22 14 40 23

Non-verbal prompt 09 -07 -15 07 09 43 26 42 16

General conversation 11 09 -08 -18 09 09 66 09 19

Verbally engaged in task 10 -12 39 43 90 21

Physically engaged in task 15 07 12 22 20

Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Ne

gati

ve C

NA

co

mm

un

icat

ion

b

eh

avio

rs

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Po

siti

ve C

NA

co

mm

un

icat

ion

be

hav

iors

Greeting

Explanation

Courtesy

Offers choices

Asks preference

Praisecompliment

Reassurance

Positive verbal prompt

Positive announcement

Positive touch

Non-verbal prompt

General conversation

Verbally engaged in task

Physically engaged in task

Correlation is significant at the 0001 level (2-tailed)

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement

Ignores resident

Talks to another 08 12 09

Rebukedisapprove 06 12

Negative touch

Greeting 35 16 38 20

Explanation 46 20 51 24

Courtesy 47 27 46 32

Offers choices 34 21 42 13

Asks preference 41 19 52 16

Praisecompliment 38 35 54 20

Reassurance 16 22 10

Positive verbal prompt 29 08 48 11

Positive announcement 34 12 37 14

Positive touch 22 14 40 23

Non-verbal prompt 43 26 42 16

General conversation 66 09 19

Verbally engaged in task 39 43 90 21

Physically engaged in task 07 12 22 20

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement 11

Ignores resident

Talks to another 26 10 28 08

Rebukedisapprove 10

Negative touch

Greeting 15 11 -09 08

Explanation 15 15 -12 20 12

Courtesy -14

Offers choices 11 12 -12

Asks preference 17 14 -14 10 11

Praisecompliment 07 -13

Reassurance 20 12

Positive verbal prompt 13 17 -12 13 10

Positive announcement 08 11 -10 12

Positive touch 09 12 -08

Non-verbal prompt 09 -07 -15 07 09

General conversation 11 09 -08 -18 09 09

Verbally engaged in task 10 -12

Physically engaged in task 15

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Greeting 35 38

Explanation 46 51

Courtesy 47 46

Offers choices 42

Asks preference 41 52

Praisecompliment 38 35 54

Reassurance

Positive verbal prompt 48

Positive announcement 37

Positive touch 40

Non-verbal prompt 43 42

General conversation 66

Engaged in task 39 43 90

Physically engaged in task

Why focus on positive emotion as an outcome of choice

1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash

1378 doi 101098rstb20041512

2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686

doi1010370003-066X607678

3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home

Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences

Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1

bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23

Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Ne

gati

ve C

NA

co

mm

un

icat

ion

b

eh

avio

rs

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Po

siti

ve C

NA

co

mm

un

icat

ion

be

hav

iors

Greeting

Explanation

Courtesy

Offers choices

Asks preference

Praisecompliment

Reassurance

Positive verbal prompt

Positive announcement

Positive touch

Non-verbal prompt

General conversation

Verbally engaged in task

Physically engaged in task

Correlation is significant at the 0001 level (2-tailed)

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement

Ignores resident

Talks to another 08 12 09

Rebukedisapprove 06 12

Negative touch

Greeting 35 16 38 20

Explanation 46 20 51 24

Courtesy 47 27 46 32

Offers choices 34 21 42 13

Asks preference 41 19 52 16

Praisecompliment 38 35 54 20

Reassurance 16 22 10

Positive verbal prompt 29 08 48 11

Positive announcement 34 12 37 14

Positive touch 22 14 40 23

Non-verbal prompt 43 26 42 16

General conversation 66 09 19

Verbally engaged in task 39 43 90 21

Physically engaged in task 07 12 22 20

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement 11

Ignores resident

Talks to another 26 10 28 08

Rebukedisapprove 10

Negative touch

Greeting 15 11 -09 08

Explanation 15 15 -12 20 12

Courtesy -14

Offers choices 11 12 -12

Asks preference 17 14 -14 10 11

Praisecompliment 07 -13

Reassurance 20 12

Positive verbal prompt 13 17 -12 13 10

Positive announcement 08 11 -10 12

Positive touch 09 12 -08

Non-verbal prompt 09 -07 -15 07 09

General conversation 11 09 -08 -18 09 09

Verbally engaged in task 10 -12

Physically engaged in task 15

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement 11

Ignores resident

Talks to another 08 12 09 26 10 28 08

Rebukedisapprove 06 12 10

Negative touch

Greeting 15 11 -09 08 35 16 38 20

Explanation 15 15 -12 20 12 46 20 51 24

Courtesy -14 47 27 46 32

Offers choices 11 12 -12 34 21 42 13

Asks preference 17 14 -14 10 11 41 19 52 16

Praisecompliment 07 -13 38 35 54 20

Reassurance 20 12 16 22 10

Positive verbal prompt 13 17 -12 13 10 29 08 48 11

Positive announcement 08 11 -10 12 34 12 37 14

Positive touch 09 12 -08 22 14 40 23

Non-verbal prompt 09 -07 -15 07 09 43 26 42 16

General conversation 11 09 -08 -18 09 09 66 09 19

Verbally engaged in task 10 -12 39 43 90 21

Physically engaged in task 15 07 12 22 20

Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Ne

gati

ve C

NA

co

mm

un

icat

ion

b

eh

avio

rs

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Po

siti

ve C

NA

co

mm

un

icat

ion

be

hav

iors

Greeting

Explanation

Courtesy

Offers choices

Asks preference

Praisecompliment

Reassurance

Positive verbal prompt

Positive announcement

Positive touch

Non-verbal prompt

General conversation

Verbally engaged in task

Physically engaged in task

Correlation is significant at the 0001 level (2-tailed)

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement

Ignores resident

Talks to another 08 12 09

Rebukedisapprove 06 12

Negative touch

Greeting 35 16 38 20

Explanation 46 20 51 24

Courtesy 47 27 46 32

Offers choices 34 21 42 13

Asks preference 41 19 52 16

Praisecompliment 38 35 54 20

Reassurance 16 22 10

Positive verbal prompt 29 08 48 11

Positive announcement 34 12 37 14

Positive touch 22 14 40 23

Non-verbal prompt 43 26 42 16

General conversation 66 09 19

Verbally engaged in task 39 43 90 21

Physically engaged in task 07 12 22 20

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement 11

Ignores resident

Talks to another 26 10 28 08

Rebukedisapprove 10

Negative touch

Greeting 15 11 -09 08

Explanation 15 15 -12 20 12

Courtesy -14

Offers choices 11 12 -12

Asks preference 17 14 -14 10 11

Praisecompliment 07 -13

Reassurance 20 12

Positive verbal prompt 13 17 -12 13 10

Positive announcement 08 11 -10 12

Positive touch 09 12 -08

Non-verbal prompt 09 -07 -15 07 09

General conversation 11 09 -08 -18 09 09

Verbally engaged in task 10 -12

Physically engaged in task 15

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Greeting 35 38

Explanation 46 51

Courtesy 47 46

Offers choices 42

Asks preference 41 52

Praisecompliment 38 35 54

Reassurance

Positive verbal prompt 48

Positive announcement 37

Positive touch 40

Non-verbal prompt 43 42

General conversation 66

Engaged in task 39 43 90

Physically engaged in task

Why focus on positive emotion as an outcome of choice

1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash

1378 doi 101098rstb20041512

2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686

doi1010370003-066X607678

3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home

Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences

Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1

bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23

Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Ne

gati

ve C

NA

co

mm

un

icat

ion

b

eh

avio

rs

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Po

siti

ve C

NA

co

mm

un

icat

ion

be

hav

iors

Greeting

Explanation

Courtesy

Offers choices

Asks preference

Praisecompliment

Reassurance

Positive verbal prompt

Positive announcement

Positive touch

Non-verbal prompt

General conversation

Verbally engaged in task

Physically engaged in task

Correlation is significant at the 0001 level (2-tailed)

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt 12

Negative announcement

Ignores resident

Talks to another 08 12 09

Rebukedisapprove 06 12

Negative touch

Greeting 35 16 38 20

Explanation 46 20 51 24

Courtesy 47 27 46 32

Offers choices 34 21 42 13

Asks preference 41 19 52 16

Praisecompliment 38 35 54 20

Reassurance 16 22 10

Positive verbal prompt 29 08 48 11

Positive announcement 34 12 37 14

Positive touch 22 14 40 23

Non-verbal prompt 43 26 42 16

General conversation 66 09 19

Verbally engaged in task 39 43 90 21

Physically engaged in task 07 12 22 20

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement 11

Ignores resident

Talks to another 26 10 28 08

Rebukedisapprove 10

Negative touch

Greeting 15 11 -09 08

Explanation 15 15 -12 20 12

Courtesy -14

Offers choices 11 12 -12

Asks preference 17 14 -14 10 11

Praisecompliment 07 -13

Reassurance 20 12

Positive verbal prompt 13 17 -12 13 10

Positive announcement 08 11 -10 12

Positive touch 09 12 -08

Non-verbal prompt 09 -07 -15 07 09

General conversation 11 09 -08 -18 09 09

Verbally engaged in task 10 -12

Physically engaged in task 15

Very neg

verbal Neg

verbal Gen

restless Eyes closed

Uncoop- erative

Aggres- sion

Pos verbal

Very pos verbal

Task en-gagement

Pos touch

Negative prompt

Negative announcement

Ignores resident

Talks to another

Rebukedisapprove

Negative touch

Greeting 35 38

Explanation 46 51

Courtesy 47 46

Offers choices 42

Asks preference 41 52

Praisecompliment 38 35 54

Reassurance

Positive verbal prompt 48

Positive announcement 37

Positive touch 40

Non-verbal prompt 43 42

General conversation 66

Engaged in task 39 43 90

Physically engaged in task

Why focus on positive emotion as an outcome of choice

1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash

1378 doi 101098rstb20041512

2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686

doi1010370003-066X607678

3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home

Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences

Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1

bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23

Why focus on positive emotion as an outcome of choice

1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash

1378 doi 101098rstb20041512

2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686

doi1010370003-066X607678

3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home

Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences

Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1

bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23

Pearson Correlations (N=2638 real-time observation occasions)

Resident Positive Affect Resident Negative Affect

Pleasure Interest Anger Anxiety Sadness

Po

sit

ive C

NA

co

mm

un

icati

on

be

ha

vio

rs

Greeting

Explanation

Courtesy

Offers choice

Asks preference

Praise

compliment

Reassure

Positive

announcement

Positive touch

Positive verbal

prompt

Non-verbal

prompt

Verbal task

engagement

Physical task

engagement

Neg

ati

ve C

NA

co

mm

un

icati

on

beh

avio

rs

Negative

announcement

Ignores resident

Talks to another

Rebuke

disapprove

Negative touch

Correlation is significant at the 0001 level (2-tailed)

Pleasure Interest Anger Anxiety Sadness

Greeting 24 11

Explanation 30 12

Courtesy 38 14

Offers choice 25 11

Asks preference 34 14

Praise

compliment 42 12

Reassure 12

Positive

announcement 26 09

Positive touch 25 12

Positive verbal

prompt 27 12

Non-verbal

prompt 37 20

Verbal task

engagement 51 10

Physical task

engagement 14 07

Negative

announcement

Ignores resident

Talks to another 09 07

Rebuke

disapprove 07

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting 15

Explanation 17 20 08

Courtesy

Offers choice 12

Asks preference 10 16

Praise

compliment

Reassure 11 08 13

Positive

announcement 09 09

Positive touch 13

Positive verbal

prompt 14

Non-verbal

prompt

Verbal task

engagement 07

Physical task

engagement 17

Negative

announcement

Ignores resident

Talks to another 17 09

Rebuke

disapprove

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting 24 11 15

Explanation 30 12 17 20 08

Courtesy 38 14

Offers choice 25 11 12

Asks preference 34 14 10 16

Praise

compliment 42 12

Reassure 12 11 08 13

Positive

announcement 26 09 09 09

Positive touch 25 12 13

Positive verbal

prompt 27 12 14

Non-verbal

prompt 37 20

Verbal task

engagement 51 10 07

Physical task

engagement 14 07 17

Negative

announcement

Ignores resident

Talks to another 17 09 09 07

Rebuke

disapprove 07

Negative touch

Pearson Correlations (N=2638 real-time observation occasions)

Resident Positive Affect Resident Negative Affect

Pleasure Interest Anger Anxiety Sadness

Po

sit

ive C

NA

co

mm

un

icati

on

be

ha

vio

rs

Greeting

Explanation

Courtesy

Offers choice

Asks preference

Praise

compliment

Reassure

Positive

announcement

Positive touch

Positive verbal

prompt

Non-verbal

prompt

Verbal task

engagement

Physical task

engagement

Neg

ati

ve C

NA

co

mm

un

icati

on

beh

avio

rs

Negative

announcement

Ignores resident

Talks to another

Rebuke

disapprove

Negative touch

Correlation is significant at the 0001 level (2-tailed)

Pleasure Interest Anger Anxiety Sadness

Greeting 24 11

Explanation 30 12

Courtesy 38 14

Offers choice 25 11

Asks preference 34 14

Praise

compliment 42 12

Reassure 12

Positive

announcement 26 09

Positive touch 25 12

Positive verbal

prompt 27 12

Non-verbal

prompt 37 20

Verbal task

engagement 51 10

Physical task

engagement 14 07

Negative

announcement

Ignores resident

Talks to another 09 07

Rebuke

disapprove 07

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting 15

Explanation 17 20 08

Courtesy

Offers choice 12

Asks preference 10 16

Praise

compliment

Reassure 11 08 13

Positive

announcement 09 09

Positive touch 13

Positive verbal

prompt 14

Non-verbal

prompt

Verbal task

engagement 07

Physical task

engagement 17

Negative

announcement

Ignores resident

Talks to another 17 09

Rebuke

disapprove

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting

Explanation

Courtesy 38

Offers choice

Asks preference

Praise

compliment 42

Reassure

Positive

announcement

Positive touch

Positive verbal

prompt

Non-verbal

prompt 37

Verbal task

engagement 51

Physical task

engagement

Negative

announcement

Ignores resident

Talks to another

Rebuke

disapprove

Negative touch

Summary Pros and Cons afforded by real time observation of treatment fidelity

compared to other methods

30

Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice

Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings

Thank you

31

Pearson Correlations (N=2638 real-time observation occasions)

Resident Positive Affect Resident Negative Affect

Pleasure Interest Anger Anxiety Sadness

Po

sit

ive C

NA

co

mm

un

icati

on

be

ha

vio

rs

Greeting

Explanation

Courtesy

Offers choice

Asks preference

Praise

compliment

Reassure

Positive

announcement

Positive touch

Positive verbal

prompt

Non-verbal

prompt

Verbal task

engagement

Physical task

engagement

Neg

ati

ve C

NA

co

mm

un

icati

on

beh

avio

rs

Negative

announcement

Ignores resident

Talks to another

Rebuke

disapprove

Negative touch

Correlation is significant at the 0001 level (2-tailed)

Pleasure Interest Anger Anxiety Sadness

Greeting 24 11

Explanation 30 12

Courtesy 38 14

Offers choice 25 11

Asks preference 34 14

Praise

compliment 42 12

Reassure 12

Positive

announcement 26 09

Positive touch 25 12

Positive verbal

prompt 27 12

Non-verbal

prompt 37 20

Verbal task

engagement 51 10

Physical task

engagement 14 07

Negative

announcement

Ignores resident

Talks to another 09 07

Rebuke

disapprove 07

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting 15

Explanation 17 20 08

Courtesy

Offers choice 12

Asks preference 10 16

Praise

compliment

Reassure 11 08 13

Positive

announcement 09 09

Positive touch 13

Positive verbal

prompt 14

Non-verbal

prompt

Verbal task

engagement 07

Physical task

engagement 17

Negative

announcement

Ignores resident

Talks to another 17 09

Rebuke

disapprove

Negative touch

Pleasure Interest Anger Anxiety Sadness

Greeting

Explanation

Courtesy 38

Offers choice

Asks preference

Praise

compliment 42

Reassure

Positive

announcement

Positive touch

Positive verbal

prompt

Non-verbal

prompt 37

Verbal task

engagement 51

Physical task

engagement

Negative

announcement

Ignores resident

Talks to another

Rebuke

disapprove

Negative touch

Summary Pros and Cons afforded by real time observation of treatment fidelity

compared to other methods

30

Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice

Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings

Thank you

31

Summary Pros and Cons afforded by real time observation of treatment fidelity

compared to other methods

30

Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice

Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings

Thank you

31

Thank you

31