hands’ smh evaluation · 2019. 6. 7. · summary overall, hands’ smh is beneficial on average,...

Hands’ SMH Evaluation Summary Feedback and Recommendations

2014

Evaluators: Tracey Curwen, PhD & Glen Sharpe, EdD

Acknowledgements: Erin Brock, Nathalie Ouelette, Michelle Dermenjian

Goals

Provide research evidence re SMH programs

Identify most effective Hands’ SMH service

Provide recommendations based on evidence

Literature research and Hands’ SMH outcomes

Curwen & Sharpe, 2014

Summary

Overall, Hands’ SMH is beneficial

On average, clients improve, similar to literature

No differences among service models: mobile,

group, or classroom effectiveness

SMH services overall appear to be directed at few

children with moderate to severe concerns (*cf SNAP)

Most effective with younger population and those

who have greater difficulties

Focus on this population

with clear protocols of identifying mental health need

using most cost effective method


Process

1) Literature review

2) Program information

3) Program outcome data


1) Literature Review

2 models

School-based SMH services (SBMH)

Provided by school personnel

Expanded SMH services (ESMH)

combines traditional SBMH community-based mental health

programs/multidisciplinary care

Focus on ESMH - model used by Hands


1) Literature review continued

Most samples did not meet “clinical cut-off” criteria

Significant decrease/improvement

moved from “no concern” to a lower score in the “no

concern” range

Others in clinical range

Significant decrease

Remain in clinical range

Not meeting criteria for program efficacy

1st Review of Literature


1) Literature review continued

Refocused review

Why programs are effective or ineffective

Program Components & Characteristics

Dosage and duration

Service administrator

Fidelity (implementation consistency)

Theory/framework/focus

Delivery

School characteristics

Intervention characteristics


3) Hands’ SMH Outcomes

Data provided by Hands

Last 3 years

Data included:

Age

Gender

BCFPI at intake to agency

CAFAS pre- and post- SMH


3) Hands’ SMH Outcomes

N = 543 children and youth

70% male

Age: M = 10.53 (SD = 3.27)

Range 4 – 18 years

4-5 = 4% 6-12 = 67% 13-18 = 29%

All children/youth had at least 1 SMH program

16 different SMH services

32% multiple services concurrent

37% services following SMH

35% services prior to SMH

Only 1st SMH considered

Participants


1st SMH Participants

0

10

20

30

40

50

60

70

80

90

4-5 6-12 13-18

Perc

en

tag

e

Age Group

Male Female


First SMH Service

48.9%

25.6%

15.8%

5.9%

1.3

0.9

0.7

0.4

0.2

0.2

Mobile Group Consult

Classroom Student Specific Consultation Summer School Program

Monitoring Transitions Follow-up

SIS Intense Intervention


Outcomes

Of 543 SMH cases

232 CAFAS at pre- & post-1st SMH

All 232 combined:

Significant Improvement No improvement

Total Score Community

School/Work Substance Use

Home Thinking

Behaviour Towards Others Caregiver – material needs

Moods/emotions Caregiver - social support

Self-Harm

p < .01


Recall: Criteria for Evaluation

1) Investigate clinically significant change initially met or exceeded clinically elevated cutoffs

improved to a non-clinical range post-treatment

Investigated actual scores

2) The change must be considered reliable level of change must be sufficient enough to rule out random

fluctuations or measurement error

Used a stringent p value

CAFAS concern levels:

0 = none,

10 = minimal,

20 = moderate,

30 = severe


Pre-post CAFAS Scores

0

2

4

6

8

10

12

14

16

18

20

CA

FA

S S

co

re

Pre-SMH Post- SMH

moderate

minimal


BCFPI Mean Scores

0

10

20

30

40

50

60

70

80

Mobile Group Classroom Consult

T-s

co

res

Internalizing** Externalizing** Total Problems**

Clinically concerning

Sub-clinical


Moderate – Severe Sample

Selected only those in the moderate to severe

range in any of the following:

Home

Behaviour towards others

Moods/emotion

n = 185

mobile (61%), group (24.3%), classroom (10.8%), and

consultation (3.8%)

Consultation dropped: only 8 participants

n = 177


Change scores by SMH service

0

1

2

3

4

5

6

7

8

9

10

Mean

Ch

an

ge S

co

re

Mobile (n = 113) Group (n = 45) Classroom (n = 20)

MANCOVA with age and pre-scores covaried


Single or Concurrent Hands’ Services

CAFAS scale

Only SMH

(n = 110)

Concurrent SMH+

(n = 68)

School/Work

6.3 (8.2)

6.2 (10.8)

Home

4.8 (8.3)

3.5 (11.3)

Behaviour Towards Others**

9.6 (10.9)

7.1 (10.1)

Moods/Emotions**

6.2 (9.1)

3.4 (6.9)

Note: pre-SMH CAFAS scores and age at SMH service were covaried

**p < .01


Summary

Many empirical investigation samples are NOT in

the clinical range pre-SMH

Similar profiles for Hands’ SMH clients

Many clients did not have pre-SMH assessment

measures completed

Unclear WHY referred to SMH

Does not fit clinical level of MH need

Presenting issue is unclear and thus its EBP

How is the service to be received determined?


Summary continued

Age is important to outcomes

Pre-SMH scores are important to outcomes

No difference in outcomes between mobile,

group, or classroom at exit

For those with at least one elevated Mental Health

score

Level of functioning 3-6 months later not assessed


General Recommendations

Identify those most in need of mental health

services

Concerns in clinical or sub-clinical range

Services and intervention based on empirical evidence

Conduct assessments including:

Protocol linking specific need (e.g., depression) and

service most likely to benefit (e.g. mobile)

Parent, teacher, and child

Evaluate changes seen by each


General Recommendations continued

Focus services on Younger age groups

With more concerning functioning scores

Implement the most cost effective service among mobile, group, classroom

Ensure clients meet eligibility criteria Or consider refocusing the purpose of SMH services

Do not provide too many services at once Clear assessment should allow for more focused

service/fewer services at one time

Keep the number of goals manageable *


General Recommendations continued

Consider new working relationship with school

personnel

Many children were not clinically concerning

May be a classroom or teacher-student issue

How SMH providers may work with/assist teachers differently


Limitations and Recommendations

Many children did not have an outcome measure

standard protocol to make SMH decisions

Re-consider data collection system and measures

Unclear how specific service is deemed necessary

Why group vs mobile vs consult?

Why multiple services?


Service System Partnership

School Boards and Hands

You and we are doing what the literature reveals

Our story is not unique-but not as effective as it could be

Students/Children and Youth

All can be referred to Hands for a variety of services

Priority is to link clinical levels of need with appropriate system

response (level of need, intervention type, location, capacity)

SMH Services System Partnership

Provide most cost effective, ethical, evidence based service at

appropriate levels in our collective

Which Student Should Be Priorized?

Severe Problem Identification

CAS

Pediatrician

School

…or probation, or family who’s environment is disrupted

Opportunities

Increased inclusion of teacher in assessment and

treatment planning process for children and youth

Generalization of skills (from therapy) into classroom

With intensive intervention modelling and coaching

Increasing capacity of classroom milieu to support

child and not solely seeking child change

ongoing mental health issues (1:5) can be reflected in

chronic behavioural challenges whose intensity

intervention can moderate, but not necessarily eliminated

Maximizing Service System Capacity

75 schools and limited SMH staffing

We, collectively, need to do something different in

the system with collective mental health resources

Current Capacity

Hands serves over 2000 files across CYMH programs for

students who attend schools which should be priorized for

school based mental health services

Maximizing Service System Capacity

WHAT SYSTEM DO WE WANT FOR OUR

CHILDREN AND YOUTH?

Partnering to provide equitable services

for children and youth

hands’ smh evaluation · 2019. 6. 7. · summary overall, hands’ smh is beneficial on average,...

Documents