adjusting our focus! measuring meaningful clinical outcomes

Paediatric Symposium '11 Building a Brighter Future 1

Adjusting our FOCUS!

Measuring Meaningful Clinical Outcomes

F Focus on the O Outcomes of C Communication U Under S Six


Team FOCUS

• Prof. Nancy Thomas-Stonell, PI• Dr. Bruce Oddson, Co-PI• Dr. Peter Rosenbaum, Co-PI• Dr. Karla Washington, Post-Doctoral Fellow• Ms. Bernadette Robertson, Research Coordinator• Ms. Joan Walker, Research Assistant

Funding

SickKids Foundation

Canadian Institutes of Health Research (CIHR)

Bloorview Childrens Hospital Foundation


Eleven Research Partners in Five Provinces Across Canada

• Eastern Healthcare, St. Johns, NL• Nova Scotia Hearing and Speech Centres, NS • Beyond Words, Preschool Speech and Language Program, York Region,

ON• Waterloo Preschool Speech and Language Program, ON• Holland Bloorview Kids Rehabilitation Hospital, ON• Hamilton Preschool Speech and Language Service, ON• Technology Access Clinic, ON• Wellington-Dufferin Guelph Region Preschool Sp & Lang Service, ON• ErinoakKids Centre for Treatment and Development & Halton-Peel

Preschool Speech and Language Program, ON• Calgary Health Region, AB• BC Centre for Ability, BC


A Typical Day in the Life of a SLP!

Example prepared with thanks to Laurie Graham


Preschool Speech-Language Pathologist

• 4 year old boy, A.B., presents for service

• Formal assessment results (currently available standardized tests) indicate a moderate speech and language disorder


Assessment reveals…

•Has a lot of trouble with sounds

– i.e. ‘ish’ instead of ‘fish’; ‘

–og’ instead of ‘frog’

•Has trouble with pronouns

– i.e. often says ‘he’ instead of ‘she’


Parents reports the boy is:

• extremely frustrated, has tantrums

• teased at school

• kids and teacher have trouble understanding him

• shy, evidence of low self-esteem


Clinical Goals for 8 week treatment block

• A.B. will produce /f/ in word initial position in structured settings 80% of the time.

• A.B. will produce ‘she’ appropriately in phrases in structured settings 80% of the time.

• Speech-Language Pathologist documents parent comments in client file.


Eight weeks later…

• During the last session, the Speech-Language Pathologist decides her goals have not been met as the child is only performing tasks at a 50% success rate.

• The parents state that tantrums have decreased, A.B. is no longer being teased at school, and seems more confident in communicating.


The importance of parent comments

• The Speech-Language Pathologist has chosen to include many parent comments in her client notes because they give her an indication of the child’s interactions with others.


The importance of parent comments

– Interaction is fundamental to the development of communication - the more you interact, the more you practice communication skills.

–The ability to communicate with peers and teachers is fundamental to academic and social success (i.e. group work).


Parent comments continued..

–Including parent comments in client notes, although recommended by regulatory bodies of the profession, is not required.

–There are no valid and reliable measures (i.e. tests/questionnaires) that capture the real-world changes observed by parents and clinicians!


Outcome measurement for A.B.

• The Speech-Language Pathologist wishes she could more thoroughly document the behavioural, interactive and social changes seen by A.B.’s parent.

• She suspects that A.B.’s gains are associated with therapy (not just normal development) but there is no way to prove her hypothesis.


Decision time:

• There are other children on the waitlist.

• Given the limited funding available, A.B. is not offered a second block of therapy.


Increasing concern

• Clinicians, researchers and disability advocates are concerned that changes which may be important results of therapy are overlooked.


What are clinically meaningful outcomes?


World Health Organization (WHO) Health Frameworks

• In 1980 WHO (1980) came out with their first health framework the International Classification of Impairments, Disabilities and Handicaps (ICIDH)

• Impairment (what’s broken)

•Disability (what can’t you do)

•Handicap (limitations in the real world)

Impairment Disability Handicap


Body Functions & Structures

Activities Participation

Health Condition

Environmental Factors

PersonalFactors

International Classification of Functioning, Disability and Health (ICF-2003)Children & Youth Version (ICF-CY - 2007)


ICF & ICF-CY Domains

• Body Functions: Physiological

– (e.g., voice, oral motor, speech production)

• Body Structures: anatomical

– (e.g., hearing loss; CL/P)


ICF & ICF-CY Domains

•Activities:

–Tasks and actions by an individual.

– ‘Capacity’ - performance of a task in a standard environment.

•Participation:

– Involvement in a life situation.

– ‘Performance’ performance of tasks in a in the current environment.


Contextual Factors

•Environmental Factors:

•external influences on functioning and disability related to physical, social and attitudinal world. (stairs, culture, support system)

•Personal Factors:

•internal influences on functioning and disability (personality influences on coping style)


ICF Health Framework

•Has positive and negative components.

•Uses a bi-directional model.

•Doesn’t take developmental stages into account.

–e.g. temper tantrums/frustrations for

2 year olds


ICF-CY Health Framework

•New codes to capture the functional characteristics of a developing child. Expanded codes include:– Learning new skills

– Play

– Adaptability

– Persistence

– Exploration


Why use ICF-CY model?

•Outcomes need to be evaluated across ICF-CY domains.

•Several studies (Sarno, 1969, Aten, 1986) have noted poor correlation between body structure/function outcomes and activity and participation outcomes

–depends not only on skill levels, but also personality, coping skills, social support systems...


How do we measure these outcomes?


We need treatment outcome measures!

We need outcomes measures to evaluate the impact of treatment on children’s lives.


Outcomes vs. Outcome Measures

•Any consequence of healthcare is an outcome!

Outcome = environment + treatment + client + severity


There are many types of outcomes.

•Avoiding adverse affects of care (nobody dies)

•Improved physiologic status ( phonation time)

•Reduction in symptoms ( dysfluencies)

•Improved functional status (use telephone)

•Minimizing costs

•Minimizing length of care


Outcome Measure

•A treatment outcome measure is a validated test designed to measure change in function.

•It measures, in quantitative terms, the impact of routinely delivered care on client’s lives.


Treatment Outcome Measures need to be proven to work!

•Garbage in – garbage out!

•Information generated by outcome studies is only useful if the measure is clinically useful and scientifically sound (van der Putten et al., 1999).


Why can’t we use our standardized tests?


Standardized tests...

• Determine the presence or absence of a communication disorder. They do not change.

• They provide too little information (insufficient number and variety of items) to monitor progress.

Huang, Hopkins & Nippold (1997). Satisfaction with Standardized Language Testing: A survey of Speech-Language Pathologists. Language Speech & Hearing Services in Schools 28, 12-29.


Treatment Outcome measures

•Outcomes measures at a minimum need to be proven to reliably distinguish between children who improve from therapy and those who do not improve.


Why use treatment outcome measures?

•To improve treatment services in an evidence-based manner.

•To measure clinically important change.

•To determine optimal length for treatment.

•To select the best treatment approach for each child.


CASLPA Position Statement on Outcome Measures

•CASLPA encourages and supports the use and development of outcome measures by speech-language pathologists and audiologists

•Outcome measures should be used to improve practice in an evidence-based manner in the best interests of clients.

• CASLPA Position Statement on Outcome Measures – May, 2010 (www.caslpa.ca)

ASHA Convention November 18-20 2010


The FOCUS journey began in 1998

•Holland Bloorview Kids Rehabilitation Hospital wanted an outcome measure for speech-language therapy that could be used across programs.

•Diverse population

–CP/CLP/ABI


Began our search for a treatment outcome measure.

•TOMS and AusTOMS are very broad measures of change. Scale has many descriptors. Hard to know what changed.

•GAS (individualized and time consuming).

•ASHA NOMS had no proven reliability or validity. We completed a two-year study to evaluate the NOMS. Results indicated poor sensitivity to change.


What do we do now?

•…with no existing valid, reliable and responsive communication outcome measure for preschool children available?


Development of the ‘FOCUS’

•F ocus on the •O utcomes of •C ommunication•U nder •S ix

• Thomas-Stonell, N., Oddson, B., Robertson, B. & Rosenbaum, P. Development of the FOCUS (Focus on the Outcomes of Communication Under Six), a communication outcome measure for preschool children. Developmental Medicine and Child Neurology: 2010, 52:47-53. ]


Our Goal

To develop a valid, reliable, responsive treatment outcome measure that captures ‘real world’ changes following speech and language treatment.


Developing the FOCUS

•In the previous outcome study, we collected data from parents of 210 preschool children receiving speech-language treatment and their clinicians.

(Thomas-Stonell, Oddson, Robertson & Rosenbaum, Predicted and Observed Outcomes in Preschool Children Following Speech and Language Treatment: Parent and Clinician Perspectives. JCD 42 (2009) 29-42.)



•They were asked to describe the changes they observed in their child during/following therapy.

–My child is now able to…

–What other changes did you see?

–Why is that important?


Method: 6 Linked-Steps

1. Content analysis of our descriptive data to create FOCUS items.

2. Test the measure with clinicians and families.

3. Revise the measure using the parent and clinician feedback.

4. Test the revised measure again.

5. Revise measure a second time.

6. Test measure a third time to obtain preliminary reliability and validity data.


Content Analysis

•Content analysis is the “systematic, objective analysis of message characteristics” to make valid inferences from text. (Neuendorf, 2002)


Content Analysis

•Identify recurring categories of change and calculate percentages of occurrence for each category.

•The recurring categories reflected the ICF-CY framework.


Coding Comments

Working Slides



• FOCUS is driven by DATA,

• no preconceived ideas

• FOCUS items were developed from categories cited by >10% of parents & clinicians.

• Resulted in 200 items, reduced to 103.

• Items used parents’ own wording.


Sample Body Functions Item Development


Item Development: Body Functions

Parent Comment

“Pronounces words much more clearly (specifically F sounds, L sounds when prompted - he still has work to do with L's)”

Category/ICF-CY coding: Body Functions

–Articulation Functions; b320

FOCUS Item

“My child’s speech is clear.”


Sample Activities/CapacityItem Development


Sample Item Development: Activities

Parent Comment

“Says more words. Put more words together.”

Category/ICF-CY coding: Activities/Capacity

Communicating – producing

–Speaking; d330

FOCUS Item

My child can string words together.


Sample ParticipationItem Development


Sample Item Development: Participation

Parent Comment

“His play with peers has improved in terms of sharing, turn-taking, following conversations, acting less aggressively.”

Category/ICF-CY coding: Activities & Participation

– Complex Interpersonal Interactions d720

FOCUS Item

“My child plays well with other children.”


Sample Personal FactorsItem Development


Sample Item Development: Personal Factors

Parent Comment

“More confident in playing with peers or entering a new group.”

Category/ICF-CY coding: Personal Factors

– Coping Style/Behavior Pattern

FOCUS Item

My child makes friends easily.


Initial FOCUS – 103 items

•Body Functions 9 %

•Activity/Capacity 28 %

•Participation/Performance 54 %

•Personal Factors 20 %

•Environmental Factors 3 %

**Percentages exceed 100 as some items had 2 codes.


Criterion-referenced

•Performance is judged according to pre-stated criterion.

•Take a verbal ‘snapshot’ of child’s skills at Time 1 and Time 2 and use the changes in the scores to measure change.

•Developed a parent and a clinician version.


Response Categories #1

Not at all like my child

A little like my child

Somewhat like my child

A fair bit like my child

Quite a bit like my child

Very much like my child

Exactly like my child

My child talks a lot.


Response Categories #2

Can not do at all

Can do with a great deal of help

Can do with a lot of help

Can do with a bit of help

Some-times does without help

Often does without help

Can always do without help

My child plays well with other children.


Sample FOCUS Form

FOCUS: Focus on the Outcomes of Communication Under Six

How do I know?

# Part I Not at all like my client

A little bit like my client

Some-what like my client

A fair bit like my client

Quite a bit like my client

Very much like my client

Exactly like my client

Ob

serv

ed

Par

ent

rep

ort

1 My client’s speech is clear. 2 My client speaks slowly when not understood.

3 My client can string words together.

4 My client speaks in complete sentences.

5 My client uses correct grammar when speaking.

6 My client talks a lot.

7 My client is confident communicating with adults who know my client well.

8 My client uses language to communicate new ideas.

Start Time: __________ ID:_________


Sample FOCUS Items

1. My client’s speech is clear.

2. My client speaks slowly when not understood.

3. My client can string words together.

4. My client speaks in complete sentences.

5. My client uses correct grammar when speaking.


Sample FOCUS Items

6. My client talks a lot.

7. My client is confident communicating with adults who know my client well.

8. My client uses language to communicate new ideas.


FOCUS Instructions

•FOCUS designed for children from birth to 6 years.

•If children are too young to complete some of the items, parents and clinicians need to score the items as

“Not at all like my child”.

•This allows these emerging skills to be measured.


Instruction Example

•A typical child of 15 months is probably only speaking in one-word phrases, so the response to the item:

•“My child uses correct grammar when speaking”

•would be “Not at all like my child”.


FOCUS Definitions

•“Talking”, “tell”, “speaks”, “speech”

and “words” refers to verbal speech.

•For example,

•“My child talks a lot.” refers to verbal communication.


FOCUS Definitions

•“Communicating”, “conversations”, “participates” and “asking” can be any form of communication.

– (e.g. pecs, AAC, sign)

•For example

•“My child will ask for help.”


FOCUS Definitions

•Some children using AAC began to verbalize during their speech therapy.

•This is a very important functional change.

•We needed to ensure that the FOCUS could capture this change.


FOCUS Phase 1 Testing (N = 74)

•FOCUS revised using measurement science. • Data driven!

•Items were deleted if: – Poor distribution of scores

– Poor completion rate

– Not sensitive to change

– Redundant

– Items not clear


Clinician and Parent Feedback

•Difficulty completing the ‘school’ items in Nova Scotia, (more rural setting).

•Both parents and clinicians requested more questions for younger children.

•They suggested items such as babbling, imitation…


Revisions

•Broadened definition of ‘school’.

•Added 5 new items for younger children.– My child is reluctant to talk.

– My child takes turns.

– My child does not interact with others.

– My child is independent.

– My child uses immature language.

– My child uses words to request items.


Second FOCUS Testing (N = 65)

•FOCUS reduced to 77 items, including 5 new items for young children.

•Tested again with different parents.•High internal consistency indicated that the FOCUS items had some redundancy.

–Parents: = .98

–Clinicians: = .83•FOCUS revised and reduced to 50 items. [Thomas-Stonell et al., 2010]


Results: Item Distribution

Initial FOCUS

• Body Functions 9 %

• Activities/Capacity 28 %

• Participation/Perf. 54 %

• Personal Factors 20 %

• Environ. Factors 3 %

•Increased Activities and Participation items.

•One Body Function item remains (Speech Rate).

Final FOCUS







Results: Item Distribution

Initial FOCUS






•Remaining FOCUS items demonstrated the most sensitivity to change.

Final FOCUS







Phase 3 Testing

•Factor analysis indicates

one construct!

•FOCUS has 50 items.

•FOCUS takes 10 minutes to complete.


Communicative Participation

•“Communication in life situations where knowledge, information, ideas or feelings are exchanged.” (Eadie et al, 2006)

•Life situation = communication within a social context.

•Exchange = reciprocal nature of communication.


Communicative Participation

•The fundamental feature of ‘communicative participation’ is the complex interaction between the speaker and the social context.


Where are we now?

Reliability and Validity Study

CIHR 2009 - 2011


FOCUS Journey

•Evaluating other outcomes measures 1998

•Coding collected comments 2002

•Seek development funds 2003

•FOCUS development study 2005

•FOCUS validation study 2009


Reliability

•Parents completed the FOCUS twice, 7 days apart.

•Parent test-retest reliability was high!

•The same clinician scored the FOCUS twice during a 30 day no treatment interval (N = 19).

•Clinician test-retest reliability was high.


Clinician Inter-Rater Reliability

•Two different clinicians administered the FOCUS on the same child twice within a 30 day no-treatment interval.

•Clinicians’ inter-rater reliability was also very high.


Validity Testing


Construct Validity

•Construct validity is the extent to which a measure correlates with the construct is was designed to measure. (Streiner & Norman, 1995)

•Generally, a number of independent studies are required to establish the credibility of a measure.


Preliminary Validity Testing – PEDS-QL

•Parents of 22 children completed the FOCUS and the Pediatric Quality of Life Inventory (PedsQL) at the start and end of a treatment block.

•Higher FOCUS scores at the end of treatment correlated with higher PedsQL total scores (r = .466, p = .029).


Preliminary Validity Testing – PEDS-QL

•Higher FOCUS scores were specifically correlated with higher scores in the psychosocial domain - emotional, social and school functioning (r = .518, p = .013).

•Positive correlations between FOCUS scores and the PedsQL indicate that the FOCUS has construct validity.


Preliminary Validity TestingConstruct Hypothesis

•The FOCUS will measure more change during a Treatment Interval than during the Wait List Interval.

(assuming treatment works!)


Demographics

•43 preschool children with communication impairments from:

– Holland Bloorview Kids Rehabilitation Hospital,

– KidsAbility

– Alberta Health Services.•Mean age = 2.7 yrs. (age range = 1.25 – 4.8 yrs)

•63% of participants were boys.


Communication Function Classification System (CFCS) (Hidecker, 2008)

•Level I: Effective Sender and Receiver with unfamiliar and familiar partners

•Level II: Effective but slower paced Sender and/or Receiver with unfamiliar and familiar partners Level III: Effective Sender and Receiver with familiar partners

•Level IV: Sometimes Effective Sender and/or Receiver with familiar partners

•Level V: Seldom Effective Sender and Receiver even with familiar partners


Severity

•Children ranged in CFCS from 1 (mild) to 5 (severe).

•The majority of the children (70%) were classified in Level IV – Level V.

•51% of the children also had a diagnosis of developmental delay.


Methods

•Parents and clinicians completed the FOCUS at assessment, start and completion of a treatment block.

•On average, there were 36 days between assessment and start of treatment.

•On average, there were 107 days between the start and end of treatment.

•Ave amount of treatment provided was 9.7 hours.


Preliminary FOCUS Results

•Significant change was noted by both parents and clinicians after treatment. No change was noted during the waiting list period.

•Parents and clinicians score identical amounts of change from T2 – T3.

•FOCUS demonstrates both stability and the ability to measure change.


Validity Testing – VABS II(Washington, 2011)

•Progress measured by the FOCUS was compared to progress measured by the Vineland Adaptive Behavior Scales (VABS-II)

•VABS II selected as it assesses communication skills as well as broader participation (i.e., Socialization) skills.


Method

•Sixty-seven parents of preschool children ages 3 to 6 years old with communication disorders participated.

•Parents recruited from one of three agencies:

1.Holland Bloorview Kids Rehabilitation Hospital (Integrated Education and Therapy Program)

2.Toronto Preschool Speech and Language Services – West Quadrant

3.University of Western Ontario, tykeTALK


Preschoolers’ Group Description

•Group 1 – Communication Disorder only and receiving intervention

•Group 2 – Communication Disorder and a developmental mobility impairment and receiving intervention

•Group 3 - Control participants, on waitlist for intervention


Method

•Fifty-two children received direct group or individual intervention with an SLP

•Fifteen children acted as waiting list controls.

•A different SLP completed VABS-II and FOCUS by telephone with the parent following treatment.


VABS-II

Measure Purpose

1. Vineland Adaptive Behavior Scales –II (VABS-II; Sparrow, Cicchetti, & Balla, 2005)

Assessment of everyday adaptations for four major domains, including socialization. Raw scores used to establish participation skills

•Interpersonal Relationships•Play & Leisure Time •Coping Skills


VABS-II

• Interpersonal Relationships

– Demonstrates friendship seeking behaviors with others the same age (e.g., “Do you want to play?”)

•Play & Leisure Time

– Plays simple make-believe activities with others (e.g., plays dress-up, pretends to be superheroes)

•Coping Skills

– Ends conversation appropriately (e.g., says “Good-bye”).


VABS-II Response Options

Response Option Description

2 Usually

1 Sometimes or partially

0 Never

DK Don’t know


Results

•Changes on the FOCUS and VABS-II Socialization domain are significantly correlated.

•Participants receiving intervention experienced significantly greater gains compared to controls


Parent Comments - Intervention

Parent Comments - Intervention

Assessment

Will play at cousin's house, can be sociable and “hands on”, initiates and makes requests

Re-check 2

More confidence, more likely to initiate, sometimes asks to play with others

Re-check 1

Sometimes will wait his turn, will share with others and can follow nonverbals


Discussion

• Correlations between the FOCUS and the VABS-II Socialization domain demonstrates construct validity

• The FOCUS is another measure of Participation, although somewhat different from the VABS-II• SLP administered/supervised

• Shorter administration time

• Sensitive to changes in communication-level participation


Participation Predictors

•Wanted to know which factors predicted the Participation changes measured by the FOCUS

•Multiple regression analyses were preformed on the results of the 52 children who received therapy.


Predictors

Measure/Procedure Predictor Variable1. Demographic Information Age, sex

2. The matrices component of the Kaufman Brief Intelligence Test –II (K-BIT2; Kaufman & Kaufman, 2004)

Nonverbal IQ

3. Communication Domain of the Vineland Adaptive Behavior Scales (VABS-II; Sparrow et al., 2005)

Pre-Tx Communication (parent)

4. Communication Function Classification System (CFCS; Hidecker et al., 2008)

Pre-Tx Communication (SLP)

5. Socialization Domain of the VABS-II Pre-Tx Participation Skills

6. Physician/SLP Report Presence of a Physical Disability

7. SLP/Parent Report English as a Second Language

8. SLP Report Amount of Direct Intervention


General Results

•SLP treatment has a positive effect on children’s ability to participate in their world!

•Specific factors unique to children predicted improved Participation skills

•Factors unique to children’s environment may be predictive as well…


Case Study #1Pretreatment Participation Skills

Parent Description of Participation Skills

“Sometimes goes to parties on weekends. Will go to grandmother's house. Does play well with other kids. Can take turns.”


Case Studies

Three Preschool Children Attending Speech-Language Therapy

(Washington, 2010)


Case Study # 1: Child with Communication Disorder and Mobility Impairment

•5 yrs 3 month old boy with Pierre Robin Syndrome.•Mild physical impairment (GMFCS Level 1) due to club foot.•Some fine motor difficulties (OT)•Communication disorder secondary to cleft lip and palate.


Case Study # 1Pretreatment Communication Skills

•Describe your child’s communication abilities. (e.g., listening and talking skills)

Exceptional Ability

Above Average

Good AbilityAverage Ability

Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

ParentSLP



•CFCS level = 3 (Hidecker, 2008)

‘Effective sender and receiver with familiar partners’

•Difficulties with speech sounds and resonance. Mild expressive language difficulties.

Parent Description of Communication Skills

“Okay communication. Pronunciation is hard for strangers to understand. Better with repetition. Makes it hard for others to understand him, but great personality.”


Case Study # 1Pretreatment Participation Skills

•Describe your child’s participation skills.

(e.g., Does your child play at other children’s homes or go to birthday parties or other social events?)

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & Parent




“Sometimes goes to parties on weekends. Will go to grandmother's house. Does play well with other kids. Can take turns.”


Case Study # 1 Treatment

•15.5 hours of group treatment

•Total duration = 29 weeks.

Treatment Goals– Mark final consonants in words with hard contact,

– /t,d,f,s/-all word positions,

– Improve consonant blends,

– Reduce nasal turbulence on fricatives.


Case Study # 1 Pre - Post Treatment Scores

Parent FOCUS Change

– Pre 280

– Post 336 + 56 points

VABS Communication

– Pre 120


VABS Socialization

– Pre 98



Case Study # 1Post Treatment Communication Skills

•Describe your child’s communication abilities since the last interview.

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

ParentSLP SLPParent

Orange = pretreatment Green = Post Treatment




•“/l / & /s/ have improved. He is better. Clearer to others, especially non-family members. Now he is using more and longer sentences.”

•“This is very important because other people can understand him better now.”


Case Study # 1 Post Treatment Participation Skills

•Describe your child’s participation abilities since the last interview.

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & Parent

SLP & Parent



Case Study # 1Post treatment Participation Skills


•“He takes turns and listens better. He responds to questions better.”

•“This is important because he can be with other people better and not be sad.”

Other Observations

•“He has become better overall. He is talking and playing more.”


Case Study # 1High Change FOCUS Items [> 3]

•My child’s speech is clearer. + 5

•My child can string words together.

•My child speaks in complete sentences.

•My child uses correct grammar when speaking.

•My child can communicate independently with adults who do not know my child well.


Case Study # 1High Change FOCUS Items

•My child’s communication skills get in the way of learning.

•My child will try to carry on a conversation with adults who do not know my child well.

•My child can communicate effectively with adults who do not know my child well.



•Many of the play and peer items were scored at level 6 (Often does without help) at the start of treatment.

•Most of these items also improved, but they could only improve by 1 point.

•Therefore they were not included in the ‘high change’ items described above.


Case Study # 2


Case Study # 2: Child with Communication Disorder and Mobility Impairment

•4 yrs 1 month old boy with Cerebral Palsy (Spastic Quad).

•GMFM = 4

•Uses a wheelchair most of the time; Also has a walker.

•CFCS = 1

•“An effective communicator in most situations”



•Describe your child’s communication abilities. (e.g., listening and talking skills)

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

ParentSLP &




“Still developing vocabulary. Using 4-5 word sentences. Learning new words and word approximations, but I don’t always know what he wants which leads to frustration on his part.”





Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLPParent




“Not always sociable. No mobility issues affect this. He does not imitate and changing activities is difficult.”



•41 hours of group treatment•Total duration = 29 weeks.

Treatment Goals– Increase vocabulary.

– Improve understanding and use of concepts.

– Improve understanding and use of action words.

– Appropriate responses to questions.

– Expand sentence length.



Parent FOCUS Change

– Pre 270

– Post 246 - 24 points

– Follow-UP 309 + 63 points (+ 39 points)

VABS Total Score

– Pre 89


– Follow-UP 136 + 1 point (+ 47 points)




Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & ParentSLP &

Parent





•“He is talking a lot more now and answers questions appropriately.”

•“This is very important to us because now we are sure about what he wants/needs. We feel better about addressing his needs. We feel like better parents.”


Case Study # 2Post Treatment Participation Skills



Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLPParent


SLP Parent


Case Study # 2Post treatment Participation Skills


•“Increased initiation noted. Increased attention during circle time.”

•“This is important because he can interact with others.”

Other Observations

•“He is more aware that his actions lead to results. Have an impact on others and his environment.”


Case Study # 2Follow-Up Communication Skills


Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & Parent

SLP


Blue = Follow-UP

SLP & Parent

Parent


Case Study # 2Follow-Up Communication Skills


•“May not say much, but vocabulary has definitely improved. Increased grammar (possessive form).

•“This is important because it helps him communicate with his peers and allows him to find new ways of expressing himself.”


Case Study # 2Follow-Up Participation Skills


Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLPParent


Blue = Follow-Up

SLP Parent

SLP & Parent


Case Study # 2Follow-Up Participation Skills


•“Great. He is highly engaged in circle time. He still needs help physically but is willing to participate.” “This is great because he has initiative!”

Other Observations

“ He has better memory and is more curious about world. He is showing likes/dislikes more and starting to assert himself beyond food preference (e.g. Dressing).”


Case Study # 3


Case Study # 3: Child with Communication Disorder

•3 yrs 6 month old boy.

•Severe speech and language disorder.

•Difficulties with both receptive and expressive language

•CFCS Level = 4

•“Inconsistent Sender and/or Receiver with familiar partners”



•Describe your child’s communication abilities. (e.g., listening and talking skills).

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

ParentSLP




“Poor clarity of speech. Delayed grammar.”

“Late talker.”

“People don’t understand him. He is limited in expressing himself to others because they don’t understand.”





Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & Parent




“He is very sociable and entertaining.”

“He makes friends easily.”



•8 hours of group treatment

•Total duration = 5 weeks

Treatment Goals

– Improve expressive language.

– Increase MLU / expand phrases.

–Teach vocabulary using themes.



Parent FOCUS Change

– Pre 246


VABS Communication

– Pre 94


VABS Socialization

– Pre 117




•Describe your child’s communication abilities. (e.g., listening and talking skills).

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

ParentSLP



Case Study # 3 Post Treatment Communication


“His speech still not clear.

“He is trying to make sentences.”

“His vocabulary has improved but he still has difficulty with concepts such as first/middle/last and with following instructions.”


Case Study # 3 Post Treatment Participation



Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

Exceptional Ability

Above Average


Low Ability

Below Average

Limited Ability

7654321

SLP & Parent

Parent



Case Study # 3 Post Treatment Participation Skills


“He is talking more. He finds games to play. He initiates more.”

“He is more likeable and has more friends now.”

Other Observations

“He is more confident now.”


Case Study # 3High Change FOCUS Items [>3]

•My child’s communication skills get in the way of learning. [+5]

•My child’s communication skills limit his independence. [+5]

•My child waits for her/his turn to talk. [+4]


Case Study # 3High Change FOCUS Items [=3]

•My child’s is confident communicating with adults who do not know my child well.

•My child is understood the first time when s/he is talking with other children.

•My child takes turns.

•My child can tell stories that make sense.



•Even though the parent did not rate the communication skills as improved, she felt that participation skills had improved a lot!

•She was no longer concerned that communication skills were interfering with independence and learning.


FOCUS

Limitations:

•The 7 point rating scale may not have been sensitive enough to capture communication changes (change from low ability to average ability).


Discussion

•Improvement was measured by the FOCUS, VABS communication and VABS socialization domain scores.

•In Case # 1 and # 2, the parent noted improved communication skills. In Case # 3, the parent did not rate communication skills as improved.

•For all children, there were improvements in participation scores!


FOCUS

•An outcome measure that only measured changes in specific communication skills (e.g., MLU, expressive grammar, articulation) would have missed many of the positive changes associated with treatment.

•Even when there were no identified concerns with participation pretreatment, improvements were noted after treatment.


Summary


Summary

•The use of a newly developed measure of paediatric ‘participation’ outcomes, the FOCUS, has provided evidentiary support that speech and language intervention can have a broad and positive effect on progress in participation skills!

ASHA Convention November 18-20 2010


Examples of ‘Real Life’ Impact of Speech-Language Therapy

•More sociable.

•Understood better by others.

•Improved attention and listening skills.

•Improved play with peers.

•Increased communication with others.

•Less frustration/improved confidence.

•Fewer negative behaviors/temper tantrums.


Summary

•An outcome measure that focuses solely on speech and language skills (i.e. impairments) would miss the large changes associated with communicative participation.


Summary

Preliminary results suggest that…

The FOCUS is successfully measuring the ‘real world’ communication outcomes corresponding at the ICF level of participation.


FOCUS Journey continues…

•Evaluating other outcomes measures 1998

•Coding collected comments 2002

•Seek development funds 2003

•FOCUS development study 2005

•FOCUS validation study 2009

•Dissemination of the FOCUS 2011


Final Thoughts

•The evaluation of outcomes in the field of speech-language pathology would benefit from the development and implementation of additional measures of communicative participation.


Acknowledgements

A special thank you to all of the families and clinicians who participated in these studies.


[email protected]

adjusting our focus! measuring meaningful clinical outcomes

Documents

language program

bcpaediatric symposium

shepaediatric symposium

speech centres

moderate speech

language service

brighter futureassessment

brighter futureparents