doing effectiveness research at the county level
DESCRIPTION
Doing Effectiveness Research at the County Level. Robert Landry, Ph.D. Yolo County Department of Alcohol, Drug and Mental Health. Contact: [email protected]. Problem. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/1.jpg)
Doing Effectiveness Research at the County Level
Robert Landry, Ph.D.
Yolo County Department of Alcohol, Drug and Mental Health
Contact: [email protected]
![Page 2: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/2.jpg)
Problem• We are charged with helping the most difficult welfare
recipients move from a culture of welfare dependency to independence.
• We must not only address traditional substance abuse and mental health issues. We need to find new ways to facilitate what is essentially a cultural transition.
• There has been little research on how to do this, let alone whether it is even possible.
• This presentation covers Yolo county’s attempt to come to grips with this problem.
![Page 3: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/3.jpg)
Questions• What percentage of CalWORKs participants are referred for
Alcohol Drug and Mental Health services?
• How many of these participants enter treatment?
• What barriers do they face?
• How are the barriers related to engagement in ADMH treatment and to success in employment services?
• Do clients get better in treatment?
• Does ADMH treatment help clients achieve greater financial independence?
![Page 4: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/4.jpg)
What percentage of CalWORKs participants are referred for Alcohol Drug and Mental Health
services?
26%
![Page 5: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/5.jpg)
How many of these participants enter treatment?
![Page 6: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/6.jpg)
What barriers do clients face?
• In a program with a broader mandate than simply treating mental health problems, we found it necessary to summarize the clients’ employment barriers.
• We developed an Employment Barriers Checklist.
• We also measured the clients’ symptom distress with the BASIS 32 and their social functioning with the
CA-QOL.
![Page 7: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/7.jpg)
Percentage of Clients with Barriers
Mental Health 85.3 Alcohol/Drug 60.6 Domestic Conflict 55.8 Attitude 35.5 Education 32.9 Children 30.4 Work 25.6 Medical Problems 22.4 Housing 19.6 Legal 16.8 Transportation 15.9
![Page 8: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/8.jpg)
Client Reported Symptomatology and Quality of Life
• The Basis32 and the CA-QOL were administered during assessment (n = 288).
• The BASIS32 is a symptom inventory completed by the client.
• We compared the means of our sample to that of the normative female in-patient group at discharge .
• The CalWORKs group is slightly worse than the in-patient group at discharge, indicating CalWORKs ADMH clients are reporting serious pathology.
![Page 9: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/9.jpg)
How are the barriers related to engagement in ADMH treatment and success in employment
services?
Method Tasks:
• Collapse the ADMH termination status into a treatment and no treatment group.
• Collapse the CWES disposition (what the client was does after termination) into a participation and no participation group.
• Decide which barriers, or independent variables, we will measure.
![Page 10: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/10.jpg)
57%
43%
![Page 11: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/11.jpg)
69%
31%
![Page 12: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/12.jpg)
Frequency of Cancellation/No Shows For Clients Who Start Treatment
MISSEDAPPOINTMENTS
PERCENT
0 51.31 or 2 37.3
3+ 27.4
There was a significant relationship between the number of no shows and starting treatment.
(Pearson’s Chi Squared 17.599 df = 2 p < .001)
![Page 13: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/13.jpg)
Frequency of Cancellation/No Shows For Clients Who Participate in CWES
MISSEDAPPOINTMENTS
PERCENT
0 46.11 or 2 47.3
3+ 24.6
There was no difference between the 0 missed session group and the 1or 2 missed session group. However, there
was a large difference starting with 3 missed sessions (Pearson’s Chi Squared 11.147 p = .004)
![Page 14: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/14.jpg)
Client Reported Symptomatology and Quality of Life
• The Basis32 and the CA-QUAL were administered during assessment (n = 288).
• There was no relation found between client reported symptomatology or quality of life and participation in either treatment or employment services.
![Page 15: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/15.jpg)
Percentage of Clients Who Participate in CWES By Number of Barriers
NUMBER OF BARRIERS
PERCENT
2-4 50%5-6 40%7+ 14%
![Page 16: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/16.jpg)
Barriers associated with starting
treatment:
• Anxiety
• Depression
• Interpersonal/Axis II Problems
• No High School Diploma
• Diagnosed Chronic Physical Disability
![Page 17: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/17.jpg)
Barriers associated with a lack of participating in CWES:
•AD abuse
•Poor Attendance
•Past Domestic Violence (inverse relationship)
•Unstable Housing
• Never Worked.
![Page 18: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/18.jpg)
Conclusions about Barriers and Starting Treatment or Participating in CWES
• The presence of barriers increases the chances of starting treatment and decreases the chances of participating in CWES.
• Assessment no shows were associated with poor participation in both treatment and CWES.
![Page 19: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/19.jpg)
Do clients get better in treatment?
• Tracking therapeutic and quality of life changes is essential for determining the impact of treatment on employment.
• The BASIS 32 and CA-QOL are readministered every 15 sessions and at termination.
• Significant symptom reduction was indicated by the BASIS 32 and significant improvements in quality of life were indicated on 3 of the 9 CA-QOL scales.
![Page 20: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/20.jpg)
How Does Starting Treatment Affect the Chances That a Client Will Participate in CWES?
Started Treatment
Probability of CWES
Participation
NO 36%
YES 58%
(Pearson’s Chi Squared p < .001).
![Page 21: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/21.jpg)
Is there a causal relationship between treatment and success in employment?
•Experimental designs provide the most powerful proof of causality, but can be ethically and legally problematic.
•We recommend A B designs and other time study designs.
•We tracked quarterly change in treatment completion and found the percentage of clients who completed treatment increased by 38% over 2 years.
•The percentage of treatment completers who participated in CalWORKs and found employment stayed constant at 56-58%.
•This indicates there is a causal relationship between treatment and employment.
![Page 22: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/22.jpg)
Conclusions: Outcome Research
• The clients self report high levels of symptomatology.
• The degree of symptomatology within the group referred for treatment
is not a good predictor of participation in treatment or CWES.
• Therapy drop out rates are very high.
• Most clients report they improve in therapy.
• There is a relationship between starting treatment and participation in
employment services.
![Page 23: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/23.jpg)
Conclusions: Program Implications
• More than 2 missed assessment appointments warrants flagging for alternate
interventions.
• Never employed participants need alternate interventions.
• There needs to be a high degree of integration of therapy and other components in
order to address interrelated issues.
• Therapy should have a significant focus on motivating clients to overcome their life
barriers.
• Intensive case management may be needed to engage the most intractable clients.
• Once the most intractable clients are engaged there needs to be graduated
pathways towards increasing responsibility, or an effective SSI advocacy program if
indicated.
![Page 24: Doing Effectiveness Research at the County Level](https://reader036.vdocument.in/reader036/viewer/2022062721/568135c8550346895d9d2877/html5/thumbnails/24.jpg)
The Prerequisites For Outcome Assessment
• A manager committed to outcome assessment.
• A user friendly database that makes life easier for staff by:
– Simplifying standard case management paperwork tasks.
– Collects most data as a bi-product of what they are doing anyway.
– Provides the staff with quick access to useful information.
• A commitment of resources to the start up process.
• Someone familiar with research to help with the design and number
crunching.