2012 cbhc conference panel september 28, 2012 breckenridge, co innovative evaluation: collaborating...

2012 CBHC Conference PanelSeptember 28, 2012

Breckenridge, CO

Innovative Evaluation: Collaborating to Develop

Population-based Measurement for the Future

CBHC Panel Abstract, Session # 605

This multi-disciplinary panel will discuss how the future direction in behavioral health care informs the need to develop and implement program outcome measures which reflect an integrated and collaborative approach to service delivery and affect positive change throughout Colorado’s community behavioral health system.

CBHC Panelists & Facilitator Michael Allen, LCSW, CAC III, MBA – Vice President, Health

Network/TeleCare, AspenPointe

Sharon Raggio, LPC, LMFT, MBA - Chief Executive Officer, Colorado West, Inc.

Vicki Rodgers, MS, LPC – Vice President, Clinical Systems Administration, Jefferson Center for Mental Health

Richard Swanson, Ph.D., J.D. - Executive Director, Aurora Research Institute

John Mahalik, Ph.D., MPA – Director, Data & Evaluation, Colorado Division of Behavioral Health, Office of Behavioral Health, CDHS

CBHC Panelists’ PresentationsClinical focus:

How do clinical staff know performance measures make a difference for providers and practitioners?

What are behavioral health service providers/clinicians using to monitor client progress?

AspenPointe – Wellness and chronic disease focus

Colorado West – BASIS-24, RCCO focus

Jefferson Center – CCAR, client-directed outcome informed treatment

Aurora Research Institute – Recovery and client treatment rating scales

Past, Present and Future Paradigm shift from outputs and structure to

outcomes and results.Mental health and substance abuse performance indicators

task forcesAccess, customer satisfaction, continuity of care, quality and

appropriateness of care, outcomes (quality of life)

‘Future is now’ with foci on integration, prevention and wellness.Consumers lack information to select providers by quality

and performance.Without objective data to demonstrate value, behavioral

healthcare becomes vulnerable to becoming a commodity purchased solely based on price.

IHI’s ‘triple aim’ is to optimize the health system by accounting for:Experience of the individual; Health of a defined population; Cost for the population.

C-Stat Initiative – What is it? CDHS Director Reggie Bicha strategic initiative for 2012

is a performance-based analysis strategy that will allow CDHS programs and services:To better focus on and improve performance outcomes; To make more informed, collaborative decisions; To align efforts and resources to affect positive change.

Collaborative approach to affect change at every level – in concert with provider leadership, Divisions determine strategies for improvement and implement strategies, while Executive Leadership help to reduce barriers to the Divisions’ success.

Goals are to collect timely data, increase transparency, conduct regular executive meetings to assess the effectiveness of the strategies, and to identify new performance measures, all in support of continuous quality improvement.

DBH, Providers and C-Stat Process C-Stat initiative in developmental phase, as a

collaborative, iterative process.Leadership and Technical Advisory GroupsPerformance Improvement Plans (PIPs)

C-Stat changes through monthly updated data and presentations to CDHS leadership.Access to SUD services, reduction SU, lesser MH severity,

maintenance of housing and employment, reduce SUD and MH drop-out rates,

Benchmarks/goals and CY 2011 statewide averages

Broad sphere of influence with feedback from CDHS, Governor’s Office, OBH, DBH, and community behavior health provider network.

AspenPointe

Michael Allen, LCSW, CAC III, MBA

Data and information sharing will be critical to successful integrated care

It will be expected that providers demonstrate quality outcomes and return-on-investment to funders

Data will be crucial to providing a common language/common lexicon between disparate healthcare systems

Integrated care will be an essential component in Medicaid re-procurement

Looking Through the Data ‘Crystal Ball’…

“Services that engage and support clients in making behavioral changes related to diet, nutrition, smoking cessation, and physical activity should be included in the package of integrated services.”

CBHC, Input for the Design of the Request for Proposals for Behavioral Health Services Contracts, September 7, 2012

How can the Department (HCPF) achieve greater integration of services at the point of care?

Whole Person Wellness

An integrated/whole-person form of disease management

DCM is a confidential, clinically proven program for screening, managing, and supporting people with depression

CDM is a confidential, clinically proven program for screening, managing, and supporting people with chronic conditions and co-occurring mental health diagnoses

Telephonic

Depression Care Management (DCM)/Chronic Disease Management (CDM)

DepressionAsthmaType II DiabetesHeart DiseaseChronic Pain (new)

Conditions Managed

TeleCare Program Goals:

Help clients understand their disease and the impact both physically and emotionally

Educate clients on ways to self-manage their diseaseAssess for mental health needsProvide resources and information to help reduce

barriers to successWith the client leading the way, develop a plan with

achievable, measureable goalsEncourage and support each step of the wayCelebrate each success no matter how small

Focus is on strengths!

Client Identification, Outreach and Engagement

Assessment

Self Management Support

Coordination of Care

Care Manager’s Role

Functional health and well being using the Short Form 12 (SF-12 v2 ®), a 12-question health inventory.

Depression severity using a tool developed for the PCP setting, the Patient Health Questionnaire-9 (PHQ-9).

Condition specific tools using condition-specific tools designed to screen for asthma, diabetes, chronic pain and heart disease and monitor symptom severity.

Global Assessment of Functioning (GAF) recorded “pre-” and “post-” discharge.

Colorado Patient Assessment Record (CCAR) Survey recorded “pre-” and “post-” discharge.

Colorado Health Partnerships (CHP) Adult Outcome Questionnaire recorded “pre-” and “post-” discharge.

Client satisfaction with the program using the AspenPointe TeleCare satisfaction survey.

Evaluation Tools

DIABETES

Program ParticipantsInformation was analyzed for 40 clients participating in the AspenPointe TeleCare Diabetes Care Management Program. The information in this report represents approximately one year of program participation.

Changes in General Physical and Mental HealthChanges in health over the 12 month period were evaluated by the clients’ responses to the SF-12 interview made up of 12 questions related to both general physical and mental health.

SF-12 ResultsNote: Higher Scores Indicate Improvement

** after domain name indicates statistically significant change

0

10

20

30

40

50

60

70

80

90

100

Sco

re

Pre Follow-up

Changes in Self-Care Activities Changes in Diabetes Self-Care Activities after Approximately 12 Months of Program Participation

Responses to Satisfaction ItemsScore mean calculated with 1=Strongly Disagree to 4=Strongly AgreeA higher score indicates stronger agreement with an item or group of items

StatementItem Mean

ScoreStrongly

Agree

Agree

Disagree

StronglyDisagree

Mean for All Items

3.68 69% 31% <1% --

1. I am treated with respect and compassion. 3.77 77% 23% -- --

2. My Care Manager listens to me and understands my situation.

3.66 66% 34% -- --

3. My Care Manager is knowledgeable about my disease and its treatment.

3.66 66% 34% -- --

4. My Care Manager considers any cultural consideration I may have when work with me and making recommendations.

3.77 77% 23% -- --

5. My Care Manager and I discuss when I need to talk with my doctor about my disease.

3.59 59% 41% -- --

6. My Care Manager helps me identify and manage my symptoms.

3.68 68% 32% -- --

7. I am offered telephone appointments at convenient days and times.

3.63 66% 31% 3% --

8. I believe my information was handled in a confidential manner.

3.71 71% 29% -- --

9. I am satisfied with the help/service I received through this Chronic Disease Management program.

3.69 69% 31% -- --

10. I would recommend this Chronic Disease Management program to another person.

3.74 74% 26% -- --

IMPROVED POPULATION HEALTH

Decrease in severity of depression/chronic condition symptoms

Increase in functional health and well-being

Increase in self-care and self-management of condition

IMPROVED PATIENT EXPERIENCE

Excellent scores on Satisfaction Survey

DECREASED PER CAPITA COSTS

TBD = Opportunity!

Outcomes Summary – “Triple Aim”

Contact Information

Michael Allen, LCSW, CACIII, MBA

Vice PresidentHealth Network and TeleCare6208 Lehman DriveColorado Springs, CO 80918Phone (719) [email protected]

mailto:[email protected]

mailto:[email protected]

Colorado West, Inc.

Sharon Raggio, LPC, LMFT, MBA

Colorado West, Inc. Measurement is important to our futureCW currently measures:

No shows-indication of engagementTime to first appt (intake)-indication of walk in accessTime to first therapy appt (second contact)-used with

productivity measures to determine staffing needsproductivityHospital recidivismQuarterly client satisfactionSuicide ratesBASIS 24

In Development with RCCO

Outreach Specialist for Behavior ChangeCW employee paid by RCCOGoal is to outreach for behavior change, regardless of having a MH diagnosis

Have a 3 level triage tool used to ID need for this level of intensity

Go to people’s homes who are:Discharged from the acute care hospital

Identified by PCP

Continued…..Other elements that the RCCO is tracking.

Cost of clients care pre ICC interventionsCost of client care post ICC interventionsNumber of Hospital admits pre ICC interventions

Number of Hospital admits post ICC interventions

Number of ER services pre ICC interventionsNumber of ER services post ICC interventions

Jefferson Center for Mental Health

Vicki Rodgers, MS, LPC

Data Informed. Results Driven.

We have been busy…Overall, we have been working to improve the

quality of our use of data – including clinical outcomes.

We will know we have achieved ongoing improvement in our quality in use of data when we…Ask early in the decision making process, “What does

data tell us about that?”Use principles of a learning organization in our daily

work and planning.Use data to be accountable for our personal, team, and

organizational outcomes.Use data well for future planning, forecasting, and continuous quality improvement.

Meaningful Change

There are two types of outcome measures that can describe meaningful change in the lives of behavioral health consumers at Jefferson Center:First, goals that are personalized to each

individual’s needs and desires for their life regarding their struggle with a behavioral health condition.

The other type of meaningful change is the kind that is measured across individuals, programs, and systems to provide statistically valid and reliable results about the outcomes that occur as a result of treatment.

Using the CCARSometimes the CCAR gets a bad reputation because it is

required and many staff have not received feedback on the results that they have entered year after year. However, after careful review of results in CCAR change over time, we have found 3 valid and reliable measures that we use to monitor meaningful, statistically significant change the individual experiences and the health of various populations and have made these key performance indicators.CCAR Symptom Severity - Change between time 1 and

time 2 – Quality of service indicator.CCAR Level of Functioning – Change between time 1

and time 2 – Quality of service indicator.CCAR Hope Domain – Change between time 1 and time 2

- Quality of service indicator related to Resilience and Recovery.

CCAR ScoresSome of the good things about using CCAR scores

are that:They can be compared across individuals and the

organization by many different factors, such as, age or payer.

There are many years of data available for comparison, research, and review.

It is information that is collected already.It is in an easy to use a rating scale format and in

some programs we are comparing the client and clinician rating of these items.

Results may be compared between organizations.Clinicians understand these 3 measures and can see

the relevance to their day-to-day work with clients.

Moving on to Client Directed Outcome Informed Treatment (CDOI)Behavioral health outcomes that measure

positive, meaningful change for consumers are those that focus on improving behavioral health and functioning and decreasing health risks and behavioral health symptoms that interfere with the individual’s ability to function in life domains.

Behavioral health outcomes for individuals need to be realistic, obtainable and measured on an on-going basis. In order for these outcomes to measure meaningful change for a consumer, they must include consumer input and involve on-going assessment.

Barry Duncan’s presentations on a Client Directed Outcome Informed approach to treatment and change suggest:“The quality of the patient's participation . . . [emerges] as the

most important determinant of outcome."“The quality of the alliance is a more potent predictor of outcome

than orientation, experience, or professional discipline.”“Feedback improves Outcomes.”“When clients are not benefiting it provides the opportunity to do

your best work. It gives you the possibility of being helpful to everyone.”

“Change happens early.”

…and our Director of Effectiveness, Alan Girard, has posted on the Heart and Soul of Change website that:“CDOI provides a practical and realistic way to privilege the client’s voice, participate in and bear witness to the change process.”

Relevance of CDOI

Every session feedback!CDOI Outcome Measures are done at every

session:Outcome Rating Scale (ORS): Individually,

Interpersonally, Socially, Overall – Client’s rating of their progress in treatment.

Session Rating Scale (SRS): Relationship, Goals & Topics, Approach or Method, Overall – Client’s rating of the alliance with therapist.

The measures are important and the conversations are essential. Change happens as a result of the conversation.

Asking these questions increases the engagement between client and therapist which increases the achievement of meaningful outcomes for the client.

Outcome Rating ScaleBlue line on graph: This line represents client ORS

scores for 10 sessions rating how they feel progress in treatment is going. The circled area indicates that most clients experience

significant change for the better in sessions 1 – 3. This data informs about the importance of fully engaging clients early in the therapeutic process.

Also, a change of 5 or more points between sessions 1 and 10 indicates meaningful change has taken place for the client. We are doing well in this area!

The client sees their graphs each week and can view their progress along the way. Research indicates that a client’s progress happens much more slowly after 10 sessions so our goal is that more than half of clients reach the clinical cutoff by the 10th session. We are achieving this goal.

Session Rating ScaleRed line on graph: This line represents client SRS

scores for 10 sessions rating how they feel the strength of their alliance is with the clinician. The clinician also see the client’s Session Rating Scale

scores and can discuss how the clinician can better engage with the client. We find that this discussion reduces blaming clients for non-compliance, no-shows, and lack of response.

When the client is about done with treatment, their SRS scales may begin to fall.

Clinicians can also compare their client’s ORS and SRS scores to an international database of other clinicians. Trust us, although average is good in comparing scores – none of our clinicians want to be average!

Next Steps:Using these measures and service costs to understand potential

cost reduction for this population or subpopulations.Using these measures for similar groups of people with various

behavioral health diagnoses and physical health diagnoses.Preparing baselines for new information that can be compared

to current CCAR and CDOI measures.Reviewing how to use client input in more meaningful ways to

further refine supports in resilience and recovery.Continue developing meaningful measures for certain

populations of clients, such as, persons 60 and better or persons with criminal justice involvement.

Continue development of comparison of client/clinician ratings on the CCAR items and as key performance indicators.

Develop better methods to measure homelessness, employment, and housing status.

Feel free to give me a call!

Vicki K. Rodgers, MS, LPC

Vice President, Clinical Systems AdministrationJefferson Center for Mental Health4851 IndependenceWheat Ridge, CO [email protected]

Aurora Research Institute

Richard Swanson, Ph.D., J.D.

An Aurora Research Institute Presentation

Recovery and Client Treatment

Ratings

Recovery and Client Treatment

RatingsPRESENTED AT THE

Colorado Behavioral HealthCare Council

September 28, 2012

Richard M. Swanson, Ph.D., J.D.Aurora Research Institute

48

RecoveryRecovery

At the heart of the recovery movement is the idea that instead of focusing on the disease or pathology of (serious mental illness) … emphasis is placed on the potential for growth in the individual.

Patrick McGuire, February, 2000. APA Monitor, 31, No. 2.

People facing life challenges (such as serious illness, trauma, disability, or disadvantage) are resilient and can significantly improve the healing process when they have access to knowledge, self-help resources, skilled professionals, sustaining environment, and social justice.

Courtenay Harding, vision statement of theInstitute for the Study of Human Resilience, Boston University

49

Recovery ModelRecovery Model

New ModelCombination of traditional clinical and

recovery model

Process

Content

Empowerment of client

Client guided

Clinician facilitates

50

Adult, Middle Childhood, and Early Childhood Client Treatment RatingsAdult, Middle Childhood, and Early Childhood Client Treatment Ratings

Client Information

Social Support

Hope

Empowerment

Community Involvement

Overall Recovery Involvement

Overall Symptoms

Family Functioning

The Adult and Child Client Treatment Rating forms are questionnaires that asks consumers and parents to rate their mental health functioning.

There are eight separate sections:

51

Recovery Approach Overview Recovery Approach Overview

Purpose of Client Treatment Ratings Actively involve consumers in their own treatment. Consumers have opportunity to rate domains

(sections) from their perspective. Consumers also rate symptom recovery from their

own perspective.

Improve Therapeutic Experience Therapists want and welcome client input. Client feedback can be taken into account for

treatment planning. Client feedback may enhance rapport between

therapist and client.

52

Social Support Rating ScaleSocial Support Rating Scale

53

Hope Rating ScaleHope Rating Scale

54

Empowerment Rating ScaleEmpowerment Rating Scale

55

Community Involvement Rating ScaleCommunity Involvement Rating Scale

56

Overall Recovery Involvement Rating ScaleOverall Recovery Involvement Rating Scale

57

Overall Symptoms Rating ScaleOverall Symptoms Rating Scale

58

Family Functioning Rating ScaleFamily Functioning Rating Scale

59

Defining Mental Health RecoveryDefining Mental Health Recovery

A journey of healing and transformation

enabling a person with a mental health

problem to find a meaningful life in a

community of his or her choice while

striving to achieve his or her full potential.

60

Client Treatment and CCARRatings of Recovery: PsychometricsClient Treatment and CCARRatings of Recovery: Psychometrics

Reliability Analysis

CTRAdult

CTRMiddle

CTREarly

CTRAll ages

CCARAdult

CCARMiddle

CCAREarly

CCARAll ages

Alpha .83 .81 .77 .82 .91 .90 .86 .90

Items 5 5 5 5 5 5 5 5

N 2710 1090 292 4092 3734 1960 346 6042

61

Client Treatment Ratings of RecoveryClient Treatment Ratings of Recovery

MeanTime 1

MeanTime 2

SignificanceAverage

Change Score

CTR – Social Support 6.44 6.83 .000 .39

CTR – Hope 6.11 6.68 .000 .57

CTR – Empowerment 6.05 6.37 .000 .32

CTR – Community Involvement 4.76 5.29 .000 .53

CTR – Overall Recovery 6.06 6.52 .000 .46

CTR – Overall Symptoms 5.66 6.02 .000 .36

CTR – Family Functioning 6.18 6.49 .000 .31

CTR – Scale Score 29.33 31.66 .000 2.23

Means and Significance Level

N = 440

Modal 6 months

62

CCAR Recovery Outcome DomainsCCAR Recovery Outcome Domains

Means and Significance Level

MeanTime 1

MeanTime 2

SignificanceAverage

Change Score

CCAR – Social Support 6.22 6.48 .000 .26

CCAR – Hope 6.34 6.65 .000 .31

CCAR – Empowerment 6.15 6.39 .000 .23

CCAR – Activity 5.61 5.91 .000 .29

CCAR – Overall Recovery 5.67 6.00 .000 .33

CCAR – Overall Symptoms 5.35 5.59 .000 .24

CCAR – Scale Score 29.99 31.42 .000 1.43

N = 1636

Model 12 months

63

Client Treatment Ratings of RecoveryReliability AnalysesClient Treatment Ratings of RecoveryReliability Analyses

CTRSocial

Support

CTRHope

CTREmpowerment

CTRCommunityInvolvement

CTROverall

Recovery

CTROverall

Symptoms

CTRFamily

Functioning

CTRSocial Support

1

4110

CorrelationSig. (2-tailed)N

CTRHope

.484 **.000

4110


CTREmpowerment

.380 **.000

4111

.516 **.000

4112


CTR Community Involvement

.440 **.000

4100

.485 **.000

4094

.451 **.000

4093


CTROverall Recovery

.425 **.000

4096

.494 **.000

4094

.592 **.000

4094

.522 **.000

4095


CTR Overall Symptoms

.295 **.000

4095

.474 **.000

4093

.371 **.000

4092

.381 **.000

4095

.365 **.000

4092


CTRFamily Functioning

.529 **.000

4099

.487 **.000

4094

.413 **.000

4092

.474 **.000

4100

.452 **.000

4095

.413 **.000

4095


CTRScale Score

.704 **.000

4113

.768 **.000

4112

.765 **.000

4096

.765 **.000

4096

.788 **.000

4097

.490 **.000

4092

.611 **.000

4115


** Correlation is significant at the 0.001 level (2-tailed)

Inter-Scale Correlations

64

CCAR Reliability AnalysesCCAR Reliability Analyses

CCAR SocialSupport

CCARHope

CCAR Empowermen

tCCAR Activity

CCAR OverallRecovery

CCARSocial Support

1

6042


CCARHope

.722 **.000

6042


CCAREmpowerment

.640 **.000

6042

.685 **.000

6042


CCARActivity

.761 **.000

6042

.734 **.000

6042

.777 **.000

6042


CCAROverall Recovery

.504 **.000

6042

.507 **.000

6042

.553 **.000

6042

.648 **.000

6042


CCARScale Score

.856 **.000

6042

.866 **.000

6042

.866 **.000

6042

.921 **.000

6042

.743 **.000

6042



Inter-Scale Correlations

65

Client Treatment Ratings of Recoveryby CCAR Validity AnalysisClient Treatment Ratings of Recoveryby CCAR Validity Analysis


* Correlation is significant at the 0.05 level (2-tailed)

CTR SocialSupport

CTRHope

CTR Empowerm

ent

CTR Community Involvemen

t

CTR OverallRecovery

CTR Overall Symptoms

CTR Family Functioning

CTR Scale Score

CCARSocial Support

.177 **.000

3359

.246 **.000

3347

.152 **.000

3350

.197 **.000

3337

.161 **.000

3307

.172 **.000

3319

.196 **.000

3337

.246 **.000

3358


CCARHope

.161 **.000

3359

.188 **.000

3347

.147 **.000

3350

.192 **.000

3337

.145 **.000

3307

.151 **.000

3319

.183 **.000

3337

.219 **.000

3358


CCAREmpowerment

.177 **.000

3359

.196 **.000

3347

.133 **.000

3350

.244 **.000

3337

.151 **.000

3307

.141 **.000

3319

.190 **.000

3337

.239 **.000

3358


CCARActivity

.197 **.000

3359

.234 **.000

3347

.173 **.000

3350

.242 **.000

3337

.181 **.000

3307

.181 **.000

3319

.226 **.000

3337

.271 **.000

3358


CCAROverall Recovery

.150 **.000

3359

.216 **.000

3347

.124 **.000

3350

.265 **.000

3337

.138 **.000

3307

.230 **.000

3319

.201 **.000

3337

.222 **.000

3358


CCARScale Score

.206 **.000

3359

.257 **.000

3347

.174 **.000

3359

.257 **.000

3337

.185 **.000

3307

.208 **.000

3319

.236 **.000

3337

.285 **.000

3358


CCAR Overall Symptom Severity

.248 **.000

3359

.162 **.000

3347

.108 **.000

3359

.177 **.000

3337

.133 **.000

3307

.148 **.000

3319

.215 **.000

3337

.218 **.000

3358


66

CCAR and CTR Scores by Division,Team, and TherapistCCAR and CTR Scores by Division,Team, and Therapist

CCAR CTR

Baseline Time 2 Baseline Time 2

Community Involvement 3 7 9 8

Empowerment 4 7 1 6

Hope 6 8 8 9

Overall Recovery Involvement 4 9 9 8

Social Support 4 9 9 9

Family Functioning 7 6

67

CCAR and CTR Scores by Division,Team, and TherapistCCAR and CTR Scores by Division,Team, and Therapist

CCAR CTR

Baseline Time 2 Baseline Time 2


Empowerment 5 7 8 9

Hope 7 7 9 9




68

CCAR and CTR Scores by Division and TeamCCAR and CTR Scores by Division and Team

CCAR CTR

Baseline (151)

Time 2 (151)

Baseline (67)

Time 2 (67)


Empowerment 5 5 6 5

Hope 6 6 6 6




69

CCAR and CTR Scores by Division and TeamCCAR and CTR Scores by Division and Team

CCAR CTR

Baseline (43)

Time 2 (43)

Baseline (11)

Time 2 (11)


Empowerment 5 6 6 6

Hope 6 6 6 6




70

CCAR and CTR Scores by DivisionCCAR and CTR Scores by Division

CCAR CTR

Baseline (217)

Time 2 (217)

Baseline (121)

Time 2 (121)


Empowerment 5 5 6 6

Hope 6 6 6 6




71


CCAR CTR

Baseline (301)

Time 2 (301)

Baseline (162)

Time 2 (162)


Empowerment 6 6 5 6

Hope 6 6 6 7




72


CCAR CTR

Baseline (743)

Time 2 (743)

Baseline (240)

Time 2 (240)


Empowerment 6 6 5 6

Hope 6 6 5 6




Contact MeContact Me

11059 East Bethany Drive, Suite 105 • Aurora, Colorado 80014 • TEL 303-617-2675 • FAX 303-617-2397

Aurora Research Inst i tute

Richard M. Swanson, Ph.D., J.D.

Executive DirectorAurora Research Institute

11059 E. Bethany Dr., Suite 105Aurora, CO 80014(303) 617-2574

[email protected]

2012 cbhc conference panel september 28, 2012 breckenridge, co innovative evaluation: collaborating...

Documents

behavioral health care

office of behavioral

health networktelecare

individual health

future slide

performance outcomes

cdhs slide

new performance measures