fy17 18 medi-cal specialty mental health external … and... · beneficiaries under the provisions...

FY17–18 MEDI-CAL SPECIALTY MENTAL HEALTH

EXTERNAL QUALITY REVIEW

MARIN MHP FINAL REPORT

Behavioral Health Concepts, Inc.

5901 Christie Avenue, Suite 502

Emeryville, CA 94608

[email protected]

www.caleqro.com

855-385-3776

Prepared for:

California Department of

Health Care Services (DHCS)

Review Dates:

January 10 – 11, 2018

- 2 -

Marin County MHP CalEQRO Report Fiscal Year 2017–18

TABLE OF CONTENTS MARIN MHP SUMMARY OF FINDINGS ............................................................................................ 5

Introduction ................................................................................................................................................................................ 5

Access ............................................................................................................................................................................................ 6

Timeliness.................................................................................................................................................................................... 6

Quality ........................................................................................................................................................................................... 6

Outcomes ..................................................................................................................................................................................... 7

INTRODUCTION...................................................................................................................................... 8

Validation of Performance Measures ............................................................................................................................... 8

Performance Improvement Projects ................................................................................................................................ 9

MHP Health Information System Capabilities .............................................................................................................. 9

Validation of State and County Consumer Satisfaction Surveys ........................................................................... 9

Review of Recommendations and Assessment of MHP Strengths and Opportunities ................................ 9

PRIOR YEAR REVIEW FINDINGS, FY16-17 .................................................................................. 11

Status of FY16–17 Review of Recommendations ..................................................................................................... 11

Changes in the MHP Environment and Within the MHP—Impact and Implications ................................ 15

PERFORMANCE MEASUREMENT .................................................................................................... 17

Total Beneficiaries Served ................................................................................................................................................. 18

Penetration Rates and Approved Claim Dollars per Beneficiary ....................................................................... 18

High-Cost Beneficiaries ....................................................................................................................................................... 22

Timely Follow-up After Psychiatric Inpatient Discharge ...................................................................................... 23

Diagnostic Categories .......................................................................................................................................................... 24

Performance Measures Findings—Impact and Implications .............................................................................. 25

PERFORMANCE IMPROVEMENT PROJECT VALIDATION ....................................................... 27

Marin MHP PIPs Identified for Validation ................................................................................................................... 27

Clinical PIP—Improving Follow-Up After Hospital Discharge ........................................................................... 29

Non-clinical PIP—Casa Rene Occupancy Rate .......................................................................................................... 30

PIP Findings—Impact and Implications ...................................................................................................................... 30

PERFORMANCE AND QUALITY MANAGEMENT KEY COMPONENTS ................................... 32

Access to Care ......................................................................................................................................................................... 32

Timeliness of Services ......................................................................................................................................................... 33

Quality of Care ........................................................................................................................................................................ 34

Key Components Findings—Impact and Implications .......................................................................................... 37

CONSUMER AND FAMILY MEMBER FOCUS GROUPS ................................................................ 39

Consumer/Family Member Focus Group 1 ................................................................................................................ 39

Consumer/Family Member Focus Group 2 ................................................................................................................ 40

Consumer/Family Member Focus Group Findings—Implications ................................................................... 42

INFORMATION SYSTEMS REVIEW ................................................................................................. 43

- 3 -


Key Information Systems Capabilities Assessment (ISCA) Information Provided by the MHP ........... 43

Summary of Technology and Data Analytical Staffing ........................................................................................... 44

Current Operations ............................................................................................................................................................... 45

Priorities for the Coming Year ......................................................................................................................................... 46

Major Changes Since Prior Year ...................................................................................................................................... 46

Other Significant Issues ...................................................................................................................................................... 47

Plans for Information Systems Change ........................................................................................................................ 47

Current Electronic Health Record Status ..................................................................................................................... 48

Personal Health Record ...................................................................................................................................................... 48

Medi-Cal Claims Processing .............................................................................................................................................. 49

Information Systems Review Findings—Implications .......................................................................................... 50

SITE REVIEW PROCESS BARRIERS ................................................................................................. 52

CONCLUSIONS ....................................................................................................................................... 53

Strengths and Opportunities ............................................................................................................................................ 53

Recommendations................................................................................................................................................................. 55

ATTACHMENTS .................................................................................................................................... 56

Attachment A—On-site Review Agenda ...................................................................................................................... 57

Attachment B—Review Participants ............................................................................................................................. 58

Attachment C—Approved Claims Source Data ......................................................................................................... 63

Attachment D—PIP Validation Tools ............................................................................................................................ 64

- 4 -


LIST OF TABLES AND FIGURES Table 1: MHP Medi-Cal Enrollees and Beneficiaries Served in CY16, by Race/Ethnicity

Table 2: High-Cost Beneficiaries

Table 3: PIPs Submitted by MHP

Table 4: PIP Validation Review

Table 5: PIP Validation Review Summary

Table 6: Access to Care Components

Table 7: Timeliness of Services Components

Table 8: Quality of Care Components

Table 9: Distribution of Services, by Type of Provider

Table 10: Summary of Technology Staff Changes

Table 11: Summary of Data Analytical Staff Changes

Table 12: Primary EHR Systems/Applications

Table 13: EHR Functionality

Table 14: MHP Summary of Short Doyle/Medi-Cal Claims

Table 15: Summary of Top Three Reasons for Claim Denial

Figure 1A: Overall Average Approved Claims per Beneficiary, CY14-16

Figure 1B: Overall Penetration Rates, CY14-16

Figure 2A: Foster Care Average Approved Claims per Beneficiary

Figure 2B: Foster Care Penetration Rates, CY14-16

Figure 3A: Latino/Hispanic Average Approved Claims per Beneficiary, CY14-16

Figure 3B: Latino/Hispanic Penetration Rates, CY14-16

Figure 4A: 7-day Outpatient Follow-up and Rehospitalization Rates

Figure 4B: 30-day Outpatient Follow-up and Rehospitalization Rates

Figure 5A: Beneficiaries Served, by Diagnostic Categories, CY16

Figure 5B: Total Approved Claims by Diagnostic Categories, CY16

- 5 -


MARIN MHP SUMMARY OF FINDINGS

Beneficiaries Served in Calendar Year 2016 2,213

MHP Threshold Language(s) Spanish

MHP Size Medium

MHP Region Bay Area

MHP Location San Rafael

MHP County Seat San Rafael

Introduction

Marin County, located just north of San Francisco, is a medium sized county spanning 520 square

miles with a total population of 252,409 residents. It is geographically located in the San Francisco

Bay Area of California (the San Francisco-Oakland-Hayward California Metropolitan Statistical

Area). The county seat is San Rafael.

Ranching and dairying are the primary industries in the rural areas of West Marin. Other industries

in the county include movie and video production, computer software, communications equipment,

printing and the manufacturing of ceramics, candles, and cheese.

The population is 50.8 percent female and 80 percent White; however, similar to other areas of

California, the Latino population (15.5 percent of the total population) is the fastest growing

demographic in Marin (43 percent increase since 2000). Neighborhoods like Marin City and San

Rafael/Canal have a higher concentration of families of color. Spanish is the only threshold

language in Marin County, although most county documents are also available in Vietnamese.

The Population Health Institute continues to rank Marin as the healthiest county in California. This

includes indicators for mortality, health behaviors, and social and economic factors. However,

health status is not distributed equally throughout the county as, according to the same ranking

system, Marin County ranks in the bottom 50 percent regarding adult excessive drinking and

accidental substance overdoses.

The Behavioral Health and Recovery (BHRS) has three services locations: San Rafael (the Kerner

Campus), Greenbrae (the Bon Air campus), and Point Reyes Station. The administrative office is in

San Rafael, in a separate location. Medi-Cal insures 17 percent of the overall population and 34

percent of children aged 0-5.

- 6 -


During the fiscal year 2017-2018 (FY17-18) review, California External Quality Review

Organization (CalEQRO) reviewers found the following overall significant changes, efforts, and

opportunities related to access, timeliness, quality, and outcomes of the Mental Health Plan (MHP)

and its contract provider services. Further details and findings from EQRO-mandated activities are

provided in this report.

Access

The new contract with Traditional Behavioral Health (TBH) in February 2017 has added seven full

time equivalents (FTEs) of psychiatry staffing, helping to alleviate capacity and access issues for

psychiatric and medication management services.

The MHP’s collaboration with multiple community based services to improve access continues to

grow.

The opening of the ten-bed Casa Rene Crisis Residential facility in 2014 has increased access to

either divert or provide step-down level of care from hospitalization.

The BHRS website can be accessed in six languages besides English. It is comprehensive in listing

services available and how to access them.

Timeliness

Some contractors are not in the Electronic Medical Record (EMR) and therefore their timeliness

data are not captured.

Issues with timeliness to psychiatric treatment has been resolved with the addition of seven FTEs

of psychiatry staffing.

A 2 percent no-show rate for appointments with non-psychiatrist clinicians was reported this

review cycle. This included 1 percent for adults and 5 percent for children. The MHP believes this

data to be incomplete.

The MHP continues to set a high priority on expanding their continuum of crisis services,

particularly an additional new mobile response team to be added during the current MHSA Three

Year Plan period. This is in addition to the mobile response teams launched in 2015. The CSU is

being renovated.

Quality

Multiple changes in the management staff have occurred in the past year, including the Behavioral

Health Director, Quality Improvement Coordinator, Medical Director, one Adult System of Care

(ASOC) Division Director, and one Children System of Care (CSOC) Director.

- 7 -


The TBH contract alleviated the need for the use of locum tenens psychiatrists. This has added to

the quality of the psychiatric services that BHRS can offer in a variety of ways, including continuity

of provider for the consumer.

BHRS’ last state triennial audit was June 5-8, 2017. The results showed a positive trend in

compliance and quality of service delivery with a 95 percent compliance in system review. This

included 100 percent in network adequacy and array of services.

The goal of Marin’s BHRS Workforce Education and Training (WET) program is to develop a

workforce that is culturally competent, linguistically and culturally reflective of the communities

that are served, and able to offer integrated treatment for co-occurring disorders. This includes

scholarships for consumer and family members to complete certificate courses in mental health,

alcohol and other drugs (AOD), and/or domestic violence peer counseling.

Along with other county departments, the MHP is co-sponsoring trainings for post-traumatic stress

disorder (PTSD) with veterans; juvenile probation training on collection and use of demographic

data for providing culturally appropriate intervention strategies and techniques; and, sponsoring

training on African American topics focused on men.

The goal to hire culturally appropriate staff for the population served resulted in a change in the

profile of interviewers to be more culturally diverse. The MHP noted this as a data driven change in

response to outcomes of interviews of applicants.

The MHP provided the FY17-18 QI Work Plan and the FY16-17 QI Work Plan evaluation. The work

plans evidenced high-level goals with measurable targets.

BHRS Peer Counselor I and II positions were instituted. Co-Occurring Peer Education (COPE)

training from WET through the Mental Health Services Act (MHSA) funds involves a nine-month

program consisting of a three-month peer counseling course and a six-month peer specialist course.

The Marin County Drug Medi-Cal/Organized Delivery System (DMC/ODS) waiver went into effect in

April 2017.

Outcomes

Performance Outcomes and Quality Improvement (POQI) results are analyzed and compared to

previous years for program planning.

- 8 -


INTRODUCTION The United States Department of Health and Human Services (HHS), Centers for Medicare and

Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid

Managed Care programs by an External Quality Review Organization (EQRO). External Quality

Review (EQR) is the analysis and evaluation by an approved EQRO of aggregate information on

quality, timeliness, and access to health care services furnished by Prepaid Inpatient Health Plans

(PIHPs) and their contractors to recipients of State Medicaid managed care services. The CMS (42

CFR §438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations)

rules specify the requirements for evaluation of Medicaid managed care programs. These rules

require an on-site review or a desk review of each Medi-Cal Mental Health Plan.

The State of California Department of Health Care Services (DHCS) contracts with 56 county Medi-

Cal MHPs to provide Medi-Cal covered specialty mental health services (SMHS) to Medi-Cal

beneficiaries under the provisions of Title XIX of the federal Social Security Act.

This report presents the FY17-18 findings of an EQR of the Marin MHP by the California External

Quality Review Organization, Behavioral Health Concepts, Inc. (BHC).

The EQR technical report analyzes and aggregates data from the EQR activities as described below:

Validation of Performance Measures1

Both a statewide annual report and this MHP-specific report present the results of CalEQRO’s

validation of eight mandatory performance measures (PMs) as defined by DHCS. The eight PMs

include:

• Total beneficiaries served by each county MHP;

• Total costs per beneficiary served by each county MHP;

• Penetration rates in each county MHP;

• Count of Therapeutic Behavioral Services (TBS) beneficiaries served compared to the 4%

Emily Q. Benchmark2;

• Total psychiatric inpatient hospital episodes, costs, and average length of stay (LOS);

1 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validation of Performance

Measures Reported by the MCO: A Mandatory Protocol for External Quality Review (EQR), Protocol 2, Version 2.0,

September, 2012. Washington, DC: Author. 2 The Emily Q. lawsuit settlement in 2008 mandated that the MHPs provide TBS to foster care children meeting certain at-risk

criteria. These counts are included in the annual statewide report submitted to DHCS, but not in the individual county-level

MHP reports.

- 9 -


• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;

• Post-psychiatric inpatient hospital 7-day and 30-day Specialty Mental Health Services

(SMHS) follow-up service rates; and

• High-Cost Beneficiaries (HCBs), incurring approved claims of $30,000 or higher during a

calendar year.

Performance Improvement Projects3

Each MHP is required to conduct two Performance Improvement Projects (PIPs)—one clinical and

one non-clinical—during the 12 months preceding the review. The PIPs are discussed in detail later

in this report.

MHP Health Information System Capabilities4

Using the Information Systems Capabilities Assessment (ISCA) protocol, CalEQRO reviewed and

analyzed the extent to which the MHP meets federal data integrity requirement for Health

Information Systems (HIS), as identified in 42 CFR §438.242. This evaluation included a review of

the MHP’s reporting systems and methodologies for calculating PMs.

Validation of State and County Consumer Satisfaction Surveys

CalEQRO examined available consumer satisfaction surveys conducted by DHCS, the MHP, or its

subcontractors.

CalEQRO also conducted 90-minute focus groups with beneficiaries and family members to obtain

direct qualitative evidence from beneficiaries.

Review of Recommendations and Assessment of MHP Strengths

and Opportunities

The CalEQRO review draws upon prior years’ findings, including sustained strengths, opportunities

for improvement, and actions in response to recommendations. Other findings in this report

include:

3 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validating Performance

Improvement Projects: Mandatory Protocol for External Quality Review (EQR), Protocol 3, Version 2.0, September 2012.

Washington, DC: Author. 4 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). EQR Protocol 1: Assessment

of Compliance with Medicaid Managed Care Regulations: A Mandatory Protocol for External Quality Review (EQR), Protocol

1, Version 2.0, September 1, 2012. Washington, DC: Author.

- 10 -


• Changes, progress, or milestones in the MHP’s approach to performance management —

emphasizing utilization of data, specific reports, and activities designed to manage and

improve quality.

• Ratings for key components associated with the following three domains: access, timeliness,

and quality. Submitted documentation as well as interviews with a variety of key staff,

contracted providers, advisory groups, beneficiaries, and other stakeholders inform the

evaluation of the MHP’s performance within these domains. Detailed definitions for each of

the review criteria can be found on the CalEQRO website, www.caleqro.com.

http://www.caleqro.com/

- 11 -


PRIOR YEAR REVIEW FINDINGS, FY16-17 In this section, the status of last year’s (FY16-17) recommendations are presented, as well as

changes within the MHP’s environment since its last review.

Status of FY16–17 Review of Recommendations

In the FY16-17 site review report, the CalEQRO made a number of recommendations for

improvements in the MHP’s programmatic and/or operational areas. During the FY17-18 site visit,

CalEQRO and MHP staff discussed the status of those FY16-17 recommendations, which are

summarized below.

Assignment of Ratings

Met is assigned when the identified issue has been resolved.

Partially Met is assigned when the MHP has either:

• Made clear plans and is in the early stages of initiating activities to address the

recommendation; or

• Addressed some but not all aspects of the recommendation or related issues.

Not Met is assigned when the MHP performed no meaningful activities to address the

recommendation or associated issues.

Key Recommendations from FY16-17

Recommendation #1: Conduct an analysis of the current options to provide stable

psychiatric coverage (e.g., increased telepsychiatry, utilization of nurse practitioners, and

smaller caseloads of locum tenens) and implement the strategies that are most feasible and

promising.

Status: Met

• Multiple providers of psychiatric coverage were lost within a short interval of time,

primarily due to retirements over the past few years. The MHP focused on recruitment

efforts while utilizing locum tenens to augment this workforce.

• The high cost of living in Marin County and scarcity of housing relative to salary and

benefits were barriers to the recruitment process.

• After analyzing available options, approval was received from the Marin County Board

of Supervisors (BOS) to enter into a contract with TBH for 7.0 psychiatry FTEs. As of

August 2017, this has allowed the MHP to return to full psychiatric staffing and

eliminate the use of locum tenens providers.

- 12 -


• The MHP also continues to utilize Psychiatric Nurse Practitioners across an array of

adult programs, and to provide a training site for University of California San Francisco

forensic psychiatry fellows and psychiatric nurse practitioners in the county jail.

Recommendation #2: Establish a designated forum with representation by staff and peers at

each level of the MHP, and/or ensure attendance by staff and peers at system planning

meetings conducted by MHP administration.

Status: Met

• To facilitate bi-directional flow of information, the MHP Director initiated quarterly

town hall meetings for all staff. The purpose of these meetings was to disseminate

important information regarding MHP activities and changes, and to address agenda

items that were solicited in advance from the staff.

• An anonymous electronic staff survey of the town hall meetings was conducted after

these meetings to get feedback on these meetings. There were 68 responses (one-half

from non-attendees). The results of this survey were reflected to staff to increase

transparency and encourage future meeting attendance. The results were used to

inform changes in the format of the town hall meetings to allow open discussion

devoted to improving communication strategies to the beginning of the meeting.

• Contract provider staff were not included in the MHP Town Hall meetings and stated

that regular contract provider meetings with executive leadership declined to two in

2017. Contract providers identified that meetings, occurring at least quarterly, would be

useful in improving bidirectional communication and collaboration.

• The monthly Policy Meeting, traditionally attended by BHRS senior managers and

supervisors, has been opened to BHRS line staff. In Spring 2017, supervisors were

encouraged to invite two to three of their staff to each meeting and to rotate the staff

who attend, with the goal to provide the opportunity for access to all.

• The BHRS, in a further effort to increase staff representation in system planning

activities, initiated an Electronic Medical Record (EMR)/practice management (PM)

user’s group this past year. This group is multi-purpose and elicits input from multiple

levels of end users of both ShareCare and Clinicians Gateway applications regarding

system issues. The User’s Group provides input on system enhancements, tests changes

in development, and will be used to spearhead the search efforts for a new EMR/PM

suite in the upcoming year(s).

• Key members of the MHP Administration, Fiscal and Information Technology (IT), and

Quality Management (QM) identified the need for a BHRS Intraweb to provide

information in accessible and organized format to all staff. This is currently in

development and will contain a central calendar, staff and program directory, as well as

program descriptions and eligibility/referral information. It will also support workflows

and projects across the MHP. Soft launch is planned for early 2018.

- 13 -


Recommendation #3: Hire a project manager to oversee project tracking and assignment of

Information Technology staff to those projects, ensuring that limited staff can focus on

completing of initiatives that will support the department.

Status: Partially Met

• The MHP was unable to hire a project manager, however the MHP had enhanced IT

staffing due to an overlap of approximately six months between the outgoing IT

supervisor and the incoming IT supervisor.

• This overlap at the supervisor level allowed for thorough training and knowledge

transfer to take place. It also gave the incoming supervisor, who has substantial IT

project management experience, time to consider the routine and ad hoc reporting and

other tasks for which the unit is responsible and devise a plan to execute these tasks

with the highest degree of automation and efficiency possible.

• The IT supervisors reviewed the status of the electronic health record (EHR) and

practice management applications, open tickets and upgrade requests in light of the

intent to procure a new EMR/PM system.

Recommendation #4: Conduct a program-by-program analysis of consumers’ needs for

ongoing services through the mental health plan; identify those consumers who no longer

meet criteria; and develop and execute a plan to move those consumers to the most

appropriate provider or placement.

Status: Met

• Several activities took place during the past year that focused on ensuring that

consumers were provided services by the appropriate level of care. Utilization

management activities conducted by quality management staff, program staff and

access/authorization staff occurred throughout the year. Consumers that were assessed

as not meeting medical necessity for current level of care were transitioned to the

appropriate level of care.

• Additional activities occurred among both the programs of the MHP and between the

MHP and external partners.

o Adult Full-Service Partnership (FSP) and Bridge Teams: Supervisors of the two

multi-disciplinary case management (Bridge) teams and the three adults FSP teams

met weekly. The clinical teams identified individuals who would benefit from an

increase or decrease in level of care in clinical case conferences; supervisors then

brought forward these recommendations to the supervisors meeting. The

Supervisors created target criteria and a process to facilitate program transfers.

They began piloting their new tool, the Case Presentation Protocol for Program

Transfers in the Fall of 2017.

- 14 -


o Children FSP and Youth and Family Services (YFS) review the Child and Adolescent

Needs and Strengths (CANS)at each administration meeting (every six months) and

staff review, with consumers, the note that is in the chart as part of their plan

development. Per the CANS protocol, the CANS may be used not only in assessment

and plan development, but also in discharge planning. Embedded in the CANS are

criteria that would indicate that the client may not need FSP or Youth and Family

Services level of service, and may be ready to transition to Beacon level of services.

o Whole Person Care: The MHP is piloting enhanced use of clinicians in care

transitions as part of its Whole Person Care efforts. This focuses particularly on

efforts to utilize clinicians to facilitate movement from Institutions of Mental

Disease (IMD) to lower levels of care.

o Successful step-down from MHP services requires strong communication and

relationships with key external partners. Below is information on partners and the

system for communication in place:

▪ Kaiser Permanente: Senior managers from the MHP continued to

meet quarterly with senior managers from Kaiser Permanente

Department of Psychiatry and Chemical Dependency in San

Rafael to enhance communication and problem solving at a

policy level. Point persons at the program level within both

organizations were identified and care coordination/transfers of

care are primarily routed through these individuals.

▪ Marin Community Clinic (MCC)/Federally Qualified Health

Centers (FQHC): Representatives from BHRS and MCC (including

the Medical Directors of each entity) meet at regular intervals to

problem-solve transfer of care issues. The entities created a joint

“Level of Care Transition Form” to aid in transfers of client care

between the specialty mental health and the mild to moderate

level of care.

▪ Beacon Health Options: The MHP continues to refine transfer of

care procedures between specialty care and mild to moderate

care clinicians who provide care through Beacon. Conference

calls were held with Beacon supervisors in which Beacon

introduced their new case management capability, allowing

them to more successfully connect referred consumers to

ongoing care.

- 15 -


Changes in the MHP Environment and Within the MHP—Impact

and Implications

Discussed below are any changes since the last CalEQRO review that were identified as having a

significant effect on service provision or management of those services. This section emphasizes

systemic changes that affect access, timeliness, and quality, including any changes that provide

context to areas discussed later in this report.

Access to Care

• The BOS approved and BHRS contracted with TBH in February 2017 for 7.0 FTEs of

psychiatric providers. This addressed access issues and resolved further need for locum

tenens staffing.

• The BOS approved implementation of Assisted Outpatient Treatment (AOT), under

Assembly Bill (AB) 1421. Staff recruitment are underway. The treatment component of

AOT will be provided by a new FSP team, Integrated Multi-Service Partnership Assertive

Community Treatment (IMPACT). The MHP will initiate the FSP team in 2018.

• The MHP is opening a clinic location in Novato in early 2018 to better meet the needs of

consumers in the northern part of the county.

• The opening of the ten-bed Casa Rene Crisis Residential facility has increased access to

either divert or provide step-down level of care from hospitalization.

Timeliness of Services

• Issues with timeliness to psychiatric treatment has been resolved with the addition of

seven FTEs of psychiatry staffing.

• Average no-shows for clinicians other than psychiatrists is reported as 2 percent. This

unusually low percentage leads the MHP to conclude that this rate is likely inaccurate

and possibly not all no-shows are being documented by clinical staff.

• Rehospitalization rates for adults remains high. The MHP identified a 30-day acute

inpatient readmission rate of 14 percent for adults. The clinical PIP attempts to address

this through increased targeting of post discharge follow-up engagement.

Quality of Care

• Behavioral Health Director, Dr. Suzanne Tavano, PhD separated from the MHP in

January 2018. Recruitment in underway and interviews are planned for mid-January

2018.

- 16 -


• The Quality Improvement Coordinator (QIC) retired from the MHP in December 15,

2017. The new QIC was hired October 2017, providing a two-month overlap for training

and knowledge transfer.

• A new medical director came on board June 2017, replacing the previous medical

director who had held the position on an interim basis since May 2016.

• Two division directors left, the Adult Chief separated from the MHP in July 2017 and

Child Chief retired December 2017.

• Marin County went live with the Drug/Medi-Cal Organized Delivery System Waiver

implementation in April 2017. As part of this implementation, the MHP opened its first

directly-operated substance use disorder treatment program (Road to Recovery),

addressing the needs of adults with co-occurring complex use disorders and Serious

Mental Illness.

• The MHP has created an evidence-based practice (EBP) lead position to expand the

appropriate implementation and use of EBPs across all programs.

• A team of 14 staff and supervisors attended a two-day Recovery Oriented Practice

(ROP) training held at The Village in Los Angeles. Upon return the team formed

subcommittees with the intent to design and initiate implementation of ROP in Marin.

• A new Residential Unit Supervisor position was created. The position oversees

placements into Board and Care, IMD, skilled nursing facility (SNF) and State hospital

facilities, as well as transition from these facilities back into community settings.

Consumer Outcomes

• The County of Marin has signed on to the national Stepping Up Initiative, which will

leverage partnerships and resources across local behavioral health and justice

departments to ultimately reduce the population of incarcerated individuals with

serious mentally illnesses by implementing evidence-based alternative treatment and

diversion approaches. Increased mental health staffing in the jail was approved and

staffing is in recruitment. Recruitment for court liaisons/case managers funded under

Proposition 47 is also underway.

• The recently opened Empowerment Clubhouse in Marin City provided resources to an

area previously underserved. The Clubhouse is staffed and run by consumers. The

Clubhouse promotes strengths-based programs that include socialization activities,

skill-development, peer counseling, mentoring, and support groups. It is open to all.

- 17 -


PERFORMANCE MEASUREMENT As noted above, CalEQRO is required to validate the following PMs as defined by DHCS:

• Total beneficiaries served by each county MHP;

• Total costs per beneficiary served by each county MHP;

• Penetration rates in each county MHP;

• Count of TBS Beneficiaries Served Compared to the 4% Emily Q. Benchmark (not

included in MHP reports; this information is included in the Annual Statewide Report

submitted to DHCS);

• Total psychiatric inpatient hospital episodes, costs, and average LOS;

• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates;

• Post-psychiatric inpatient hospital 7-day and 30-day SMHS follow-up service rates; and

• HCBs incurring $30,000 or higher in approved claims during a calendar year.

HIPAA Suppression Disclosure:

Values are suppressed to protect confidentiality of the individuals summarized in the data sets

where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be

required to prevent calculation of initially suppressed data, corresponding penetration rate

percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

- 18 -


Total Beneficiaries Served

Table 1 provides detail on beneficiaries served by race/ethnicity.

Starting with CY16 performance measures, CalEQRO has incorporated the ACA Expansion data in

the total Medi-Cal enrollees and beneficiaries served. See Attachment C, Table C1 for the

penetration rate and approved claims per beneficiary for just the CY16 ACA Penetration Rate and

Approved Claims per Beneficiary.

Penetration Rates and Approved Claim Dollars per Beneficiary

The penetration rate is calculated by dividing the number of unduplicated beneficiaries served by

the monthly average enrollee count. The average approved claims per beneficiary served per year

is calculated by dividing the total annual dollar amount of Medi-Cal approved claims by the

unduplicated number of Medi-Cal beneficiaries served per year.

Regarding calculation of penetration rates, the Marin MHP uses the same method used by CalEQRO.

- 19 -


Figures 1A and 1B show 3-year (CY14-16) trends of the MHP’s overall approved claims per

beneficiary and penetration rates, compared to both the statewide average and the average for

medium MHPs.

- 20 -


Figures 2A and 2B show 3-year (CY14-16) trends of the MHP’s foster care (FC) approved claims per

beneficiary and penetration rates, compared to both the statewide average and the average for

medium MHPs.

- 21 -


Figures 3A and 3B show 3-year (CY14-16) trends of the MHP’s Latino/Hispanic approved claims

per beneficiary and penetration rates, compared to both the statewide average and the average for

medium MHPs.

- 22 -


High-Cost Beneficiaries

Table 2 compares the statewide data for High-Cost Beneficiaries for CY16 with the MHP’s data for

CY16, as well as the prior two years. HCBs in this table are identified as those with approved claims

of more than $30,000 in a year.

See Attachment C, Table C2 for the distribution of the MHP beneficiaries served by approved claims

per beneficiary (ACB) range for three cost categories: under $20,000; $20,000 to $30,000; and

those above $30,000.

- 23 -


Timely Follow-up After Psychiatric Inpatient Discharge

Figures 4A and 4B show the statewide and MHP 7-day and 30-day outpatient follow-up and

rehospitalization rates for CY15 and CY16.

- 24 -


Diagnostic Categories

Figures 5A and 5B compare the breakdown by diagnostic category of the statewide and MHP

number of beneficiaries served and total approved claims amount, respectively, for CY16.

MHP self-reported percent of consumers served with co-occurring (substance abuse and mental

health) diagnoses: 31%.

- 25 -


Performance Measures Findings—Impact and Implications

Access to Care

• The number of eligibles increased slightly from CY15 to CY16, while beneficiaries

remained stable, correlating to a year over year drop in overall penetration rate of

approximately 0.30 percent. The MHPs CY16 overall penetration rate is approximately

10 percent greater the medium county average and approximately 15 percent greater

than the statewide average.

• The MHP’s FC penetration rate increased slightly from CY15 to CY16 and is comparable

to both medium county and statewide averages.

• The MHP’s Latino/Hispanic penetration remained stable from CY15 to CY16, but is

approximately 20 percent less than the medium county average and approximately 35

percent less than the statewide average.


• In CY16, the MHP’s 7- and 30-day outpatient follow-up rates after discharge from a

psychiatric inpatient hospitalization increased from the rates in CY15 and are greater

than CY16 statewide averages.

Quality of Care

• The MHP’s average overall approved claims per beneficiary remained stable from CY15

to CY16 and is approximately 70 percent greater than both medium county and

statewide averages in CY16.

• The MHP’s average FC approved claims per beneficiary increased each year from CY14

to CY16 and is approximately 50 percent greater than the medium county average and

approximately 40 percent greater than the statewide average in CY16.

• The MHP’s average Latino/Hispanic approved claims per beneficiary remained stable

from CY15 to CY16 and is approximately 75 percent greater than the medium county

average and approximately 70 percent greater than the statewide average in CY16.

• Varying from the statewide diagnostic pattern, a primary diagnosis of Psychotic

disorders accounted for the largest percentage of beneficiaries served by the MHP in

CY16. The MHP had a lower rate of Disruptive and Adjustment disorders and a zero-

percentage rate of deferred diagnoses (compared to the statewide average of 5

percent).

• Consistent with its diagnostic pattern, the MHP’s percentage of total approved claims

for individuals with Psychotic disorders was greater than that of any other diagnostic

category and approximately 75 percent greater than the statewide average in CY16.

- 26 -


Consumer Outcomes

• The MHP’s 7- and 30-day rehospitalization rates increased from their CY15 rates and

exceed statewide averages.

- 27 -


PERFORMANCE IMPROVEMENT PROJECT

VALIDATION A Performance Improvement Project (PIP) is defined by CMS as “a project designed to assess and

improve processes and outcomes of care that is designed, conducted, and reported in a

methodologically sound manner.” The Validating Performance Improvement Projects Protocol

specifies that the EQRO validate two PIPs at each MHP that have been initiated, are underway, were

completed during the reporting year, or some combination of these three stages. DHCS elected to

examine projects that were underway during the preceding calendar year.

Marin MHP PIPs Identified for Validation

Each MHP is required to conduct two PIPs during the 12 months preceding the review. CalEQRO

reviewed and validated two MHP-submitted PIPs, as shown below.

Table 3 lists the findings for each section of the evaluation of the PIPs, as required by the PIP

Protocols: Validation of Performance Improvement Projects.5

Table 3: PIPs Submitted by Marin MHP

PIPs for Validation

# of PIPs PIP Titles

Clinical PIP 1 Improving Follow-up After Hospital Discharge

Non-clinical PIP 1 Casa Rene Occupancy Rate

Table 4, on the following page, provides the overall rating for each PIP, based on the ratings given to

the validation items: Met (M), Partially Met (PM), Not Met (NM), Not Applicable (NA), Unable to

Determine (UTD), or Not Rated (NR).

5 2012 Department of Health and Human Services, Centers for Medicare and Medicaid Service Protocol 3 Version 2.0,

September 2012. EQR Protocol 3: Validating Performance Improvement Projects.

- 28 -


Table 4: PIP Validation Review

Item Rating

Step PIP Section Validation Item Clinical Non-

clinical

1 Selected Study

Topics 1.1 Stakeholder input/multi-functional team M M

1.2 Analysis of comprehensive aspects of enrollee needs, care, and services

M M

1.3 Broad spectrum of key aspects of enrollee care and services M M

1.4 All enrolled populations M M

2 Study Question 2.1 Clearly stated M M

3 Study 3.1 Clear definition of study population M M

Population 3.2 Inclusion of the entire study population M M

4 Study

Indicators 4.1 Objective, clearly defined, measurable indicators PM PM

4.2 Changes in health status, functional status, enrollee satisfaction, or processes of care

PM PM

5 Sampling Methods

5.1 Sampling technique specified true frequency, confidence interval and margin of error

NA NA

5.2 Valid sampling techniques that protected against bias were employed

NA NA

5.3 Sample contained sufficient number of enrollees NA NA

6 Data Collection 6.1 Clear specification of data M M

Procedures 6.2 Clear specification of sources of data M M

6.3 Systematic collection of reliable and valid data for the study population

M PM

6.4 Plan for consistent and accurate data collection M M

6.5 Prospective data analysis plan including contingencies UTD PM

6.6 Qualified data collection personnel PM M

7 Assess

Improvement Strategies

7.1 Reasonable interventions were undertaken to address causes/barriers

M M

8 Review Data Analysis and

8.1 Analysis of findings performed according to data analysis plan PM M

Interpretation

of Study Results 8.2 PIP results and findings presented clearly and accurately M M

8.3 Threats to comparability, internal and external validity PM UTD

8.4 Interpretation of results indicating the success of the PIP and follow-up

UTD PM

9 Validity of

Improvement 9.1 Consistent methodology throughout the study UTD M

9.2 Documented, quantitative improvement in processes or outcomes of care

PM PM

9.3 Improvement in performance linked to the PIP PM UTD

9.4 Statistical evidence of true improvement UTD UTD

9.5 Sustained improvement demonstrated through repeated measures

NM UTD

- 29 -


Table 5 provides a summary of the PIP validation review.

Table 5: PIP Validation Review Summary

Summary Totals for PIP Validation Clinical PIP Non-clinical

PIP

Number Met 13 15

Number Partially Met 7 6

Number Not Met 1 0

Number Applicable (AP)

(Maximum = 28 with Sampling; 25 without Sampling) 25 25

Overall PIP Rating ((#Met*2) +(#Partially Met))/(AP*2) 66% 72%

Clinical PIP—Improving Follow-Up After Hospital Discharge

The MHP presented its study question for the clinical PIP as follows:

“Will implementing interventions including clear written procedures and identification and

clarification of BHRS staff roles and responsibilities during hospitalization and discharge follow-up,

along with enhancing permissions to BHRS CSU and Access contact log improve MHP follow-up

after hospitalization rate within 7 days by 5% and 30 days by 5% by Feb. 2019 as measured by an

improved tracking mechanism that drives workflow?”

Date PIP began: February 2017

Status of PIP: Active and ongoing

The PIP hypothesis postulates that if the MHP increases the percentage of clients who receive

timely follow-up appointments after a psychiatric inpatient discharge (i.e., within 7 days and 30

days), then this will consequently decrease the number of clients who are readmitted to psychiatric

hospital within 30 days of discharge. The PIP focuses on streamlining communication between

hospitals and the MHP during a psychiatric inpatient stay and puts mechanisms in place at multiple

points post-discharge to ensure better outcomes for clients. The goal is to reduce readmission to

the hospital by increasing clients who receive follow-up post discharge.

Relevant details of these issues and recommendations are included within the comments found in

the PIP validation tool.

The technical assistance (TA) provided to the MHP by CalEQRO consisted of discussion during the

review of wording of original study question to include clinical outcome for consumers. The need

for and wording of a new indicator and intervention to address the rewording of the study question

- 30 -


was also discussed. Future TA was scheduled as a conference call for the week following the on-site

review. EQRO offered and the MHP agreed to ongoing TA for facilitating progress in the PIP this

coming year.

Non-clinical PIP—Casa Rene Occupancy Rate

The MHP presented its study question for the non-clinical PIP as follows:

“Will admission to Casa Rene from the County’s Crisis Stabilization Unit and other referring sources

ensure that adult who are experiencing a psychiatric crisis have at least 5% reduction in CSU

utilization and 5% in-patient psychiatric hospital utilization within 60 days post discharge from

Casa Rene?”

Date PIP began: February 2017

Status of PIP: Active and ongoing

The PIP aims to implement interventions that allow Casa Rene, the county’s Crisis Residential Unit,

to fully utilize its bed capacity to the extent that the demand for diversion and/or step-down beds

are needed. The MHP will measure effects on increasing utilization of Case Rene to reduction in CSU

utilization or inpatient psychiatric hospitalization within 60 days post discharge from Casa Rene.

Interventions are directed towards resolving barriers and creating a more effective access system

to expedite admissions to Casa Rene.

Relevant details of these issues and recommendations are included within the comments found in

the PIP validation tool.

The TA provided to the MHP by CalEQRO consisted of assisting the MHP in creating a consumer

outcome for the study question. The original study question lacked any benefit for the consumer

and spoke only to increasing bed utilization. Upon discussion, it was clarified that a benefit for the

consumers who needed crisis residential level of care had been assumed. The MHP corrected the

study question, and will track and analyze the results. Future TA was scheduled as a conference call

for the week following the on-site review. EQRO offered and the MHP agreed to ongoing TA for

facilitating progress in the PIP this coming year.

PIP Findings—Impact and Implications

Access to Care

• The clinical PIP addresses access to follow-up services post discharge from psychiatric

inpatient hospitalization.

- 31 -


• The non-clinical PIP goal is to implement interventions to address barriers to access to

crisis residential treatment.


• The clinical PIP looks at timeliness to follow-up services within seven and 30 days post

discharge from inpatient psychiatric hospitalization.

• Lowering readmissions to inpatient psychiatric hospitalization is a goal of the clinical

PIP.

Quality of Care

• The clinical PIP addresses quality of care through interventions that decrease

readmissions to inpatient hospitalizations. The PIP focuses on streamlining

communication between hospitals and the MHP during a consumer’s psychiatric

inpatient stay to enable follow-up opportunities for the consumer post discharge.

• The non-clinical PIP attempts to implement interventions that allow the crisis

residential unit to fully utilize its bed capacity to the extent that the demand for

diversion from hospitalization and/or step-down level of care is appropriate. This

presumes to have the consequence of lower inpatient and CSU admissions following

discharge from crisis residential.

Consumer Outcomes

• The effect of follow-up interventions post discharge from psychiatric inpatient

hospitalization attempts to lower the readmissions to inpatient within 30 days of

discharge.

• The consumer outcomes the non-clinical PIP hopes to achieve through admission to the

crisis residential unit includes diversion from hospitalization and a decrease in CSU

utilization. The goal is to deliver the lowest level of care appropriate for the consumer.

- 32 -


PERFORMANCE AND QUALITY MANAGEMENT

KEY COMPONENTS CalEQRO emphasizes the MHP’s use of data to promote quality and improve performance.

Components widely recognized as critical to successful performance management include an

organizational culture with focused leadership and strong stakeholder involvement, effective use of

data to drive quality management, a comprehensive service delivery system, and workforce

development strategies that support system needs. These are discussed below, along with their

quality rating of Met (M), Partially Met (PM), or Not Met (NM).

Access to Care

Table 6 lists the components that CalEQRO considers representative of a broad service delivery

system that provides access to consumers and family members. An examination of capacity,

penetration rates, cultural competency, integration, and collaboration of services with other

providers forms the foundation of access to and delivery of quality services.

Table 6: Access to Care Components

Component Quality Rating

1A Service accessibility and availability are reflective of cultural competence principles and practices

M

The cultural competency work plan includes increasing bilingual/bicultural Spanish speaking

support service workers in West Marin and Spanish speaking therapists to work with

beneficiaries with mild to moderate diagnoses.

The MHP endeavors to equitably distribute available scholarships to consumers and family

members to complete a certificate course in mental health, AOD and/or domestic violence. Peer

counselor positions are tied to education. The MHP has developed a robust scholarship program

that offers funds through a WET grant for consumer and family members to consider a vocation

in Drug and Alcohol, Domestic Violence, and other mental health disorders. The criteria for

selection include having lived experience, being 18 years of age or older, residing in Marin

County, and being a member of a cultural minority group.

The MHP is co-sponsoring with other county departments trainings for PTSD with veterans;

juvenile probation training on collection and use of demographic data for providing culturally

appropriate intervention strategies and techniques; and sponsoring training on African

American topics focused on males.

1B Manages and adapts its capacity to meet consumer service needs M

- 33 -


The MHP has an impressive geographic information system (GIS) mapping system for network

adequacy that is used to make program decisions.

An active Cultural Competency Advisory Board (CCAB) is comprised of BHRS management,

BHRS line staff, contract agency providers, consumer advocates, consumers, community leaders

from ethnic communities and an administrative aide to one of the county’s Supervisors. There

are three existing working committees within the Board: Training, Policy, and Access. The 21-

member board is tasked to analyze data, review existing improvement plans, examine practice

approaches and make recommendations related to policy, service delivery, staffing and training

needs, and system improvements. QM staff provides data for the CCAB, and there is shared

participation in both the QIC and CCAB on the management, staff and consumer level.

The new psychiatrist contract has added seven FTEs. This has addressed capacity issues and

timeliness of access to psychiatric services that were issues in the last EQR.

The MHP is analyzing data to address the reasons for the lower than expected penetration rate

for Latino/Hispanic adults.

The MHP is opening a clinical location in Novato to better meet the needs of consumers in the

northern area of the county.

A team of 14 staff and supervisors attended a two-day ROP training held at The Village in Los

Angeles. Upon return, the team formed subcommittees with the intent to design and initiate

implementation of a ROP in Marin.

A First Episode Psychosis program is being developed to increase identification and effective

use of treatment and ancillary resources for individuals experiencing a first psychotic episode.

1C Integration and/or collaboration with community-based services to improve access

M

BHRS strives to offer integrated services through collaboration with partner-stakeholders,

community based organization, and public and private organizations. Examples of this

collaboration include Buckelew Counseling Services, Huckleberry Youth Programs, Jewish

Family and Child Services, Center of Restorative Practices (in collaboration with other Marin

county programs), and individual providers that are part of the service delivery network for

network adequacy.


As shown in Table 7, CalEQRO identifies the following components as necessary to support a full-

service delivery system that provides timely access to mental health services. This ensures

successful engagement with consumers and family members and can improve overall outcomes,

while moving beneficiaries throughout the system of care to full recovery.

- 34 -


Table 7: Timeliness of Services Components


2A Tracks and trends access data from initial contact to first appointment M

The MHP utilizes a 15-day standard. Adult services are reported as averaging 7.5 days (88.96%

meeting standard) and children/youth services averaged 8.8 days (83.54% meeting standard).

2B Tracks and trends access data from initial contact to first psychiatric appointment

M

The MHP contracted with TBH to resolve insufficient psychiatric staffing and now has seven

FTEs on-site psychiatric providers. The MHP also continues to utilize psychiatric nurse

practitioners throughout the system of care. The MHP utilizes a 15-day standard. Adult services

are reported as averaging 15.7 days (64% meeting standard) and children/youth services

average 8.5 days (100% meeting standard).

2C Tracks and trends access data for timely appointments for urgent conditions

M

The MHP set a high priority on expanding their continuum of crisis services, particularly new

mobile response teams and a crisis stabilization unit. The MHP utilizes a 180-minutes standard.

Adult services are reported as averaging 109 minutes (79% meeting standard) and

children/youth services averaged 91 minutes (90% meeting standard).

2D Tracks and trends timely access to follow-up appointments after hospitalization

M

Post-hospital follow-up is the subject of a PIP, and average time to first appointment post-

hospital discharge is 6 days, within the HEDIS standard. The MHP utilizes a 7-day standard.

Adult services are reported as averaging 6 days (79.2% meeting standard) and children/youth

services are reported as averaging 5.6 days (76.2% meeting standard).

2E Tracks and trends data on rehospitalizations M

The MHP identified a 30-day acute inpatient readmission rate of 14% for adults and 2% for

children/youth. The clinical PIP addresses readmissions and attempts to lower this rate for

adults.

2F Tracks and trends no-shows NM

The MHP identified a no-show standard of 10% for clinicians and psychiatrists. The psychiatry

no-show rate is 11% for adults and 16% for children/youth. While the clinician no-show rate is

1% for adults and 5% for child/youth, the MHP suspects that not all clinician no-show data is

being identified and recorded, therefore the data are likely incomplete.

Quality of Care

- 35 -


In Table 8, CalEQRO identifies the components of an organization that is dedicated to the overall

quality of care. Effective quality improvement activities and data-driven decision making require

strong collaboration among staff (including consumer/family member staff), working in

information systems, data analysis, clinical care, executive management, and program leadership.

Technology infrastructure, effective business processes, and staff skills in extracting and utilizing

data for analysis must be present in order to demonstrate that analytic findings are used to ensure

overall quality of the service delivery system and organizational operations.



3A Quality management and performance improvement are organizational priorities

M

The MHP provided the FY17-18 work plan and the FY16-17 work plan evaluation. The work

plans evidenced high level goals, with measurable target goals along the way. The QI work plan

is well-written with clear and measurable objectives, making it easier to conduct the year end

evaluation.

CSU Supervisors developed and presented a training curriculum for a full day “CSU Core

Competency Training” for all staff, with a focus on enhancing the safety and quality of care.

Training was provided to clinical staff on the impact of complex trauma in early childhood and

how this informs assessment and treatment.

3B Data are used to inform management and guide decisions M

The MHP data analysis of beneficiary demographics and staff cultural and linguistic

characteristics led to a change in their recruitment and interviewing process and requirements

of culturally diverse interviewers.

Barcodes with reporting unit identification were added to survey forms allowing for improved

tracking of distributed surveys and aiding in the accuracy of data collection and analysis for

programmatic decisions.

3C Evidence of effective communication from MHP administration, and stakeholder input and involvement on system planning and implementation

PM

Consumer and family members are informed by case managers, therapists, doctors and the local

National Alliance of the Mentally Ill (NAMI) Chapter. MHP administration sends a weekly email

with updates to a 500-plus mailing list, including consumers and family members, upon their

- 36 -




request. The CCAB, QIC, and the Mental Health Board all include consumer and family member

representation.

The MHP Director initiated quarterly All Staff Town Hall Meetings with the goal of improving

bidirectional communication and flow of information.

Contract providers noted that there had been reduced and irregular contract provider group

meetings with leadership over the past year.

3D Evidence of a systematic clinical continuum of care M

Consumers and family members in focus groups confirmed involvement in the establishment of

and/or modification to treatment plans. The MHP utilizes CANS and Milestones of Recovery

Scale (MORS) as part of their effort to assess when consumers are ready to step-down to a lower

level of care.

A new Residential Unit Supervisor position was created. The position oversees placements into

Board and Care, IMD, SNF and State Hospital facilities, as well as transition from these facilities

back into community settings.

Marin County has established an initial protocol for presumptive transfer for youth from other

counties into Marin. They are collaborating with a regional group of counties to find a protocol

and system to utilize between the counties for presumptive transfer issues versus waivered

youth who still require service authorization requests (SARs).

3E Evidence of consumer and family member employment in key roles throughout the system

M

The MHP has positions both within the county as well as through its contract providers that are

designated for consumer and family member employment. Both Wellness Centers (Marin City

and San Rafael) are staffed and run by consumers. The MHP was an active member in the Bay

Area Workforce Co-Learning Collaborative to develop a curriculum and strategies for employers

to support consumers and family members in the workplace.

The job classifications of BHRS Peer Counselor I/II have been established at the MHP. Career

ladders are in place allowing for advancement with experience, although supervisory positions

do not currently exist at the MHP. The Wellness Centers, however, do have supervisory

positions. Community Action Marin (CAM), a contract provider, also has supervisory positions

filled by family members.

The goal of Marin’s BHRS WET Program is to develop a workforce that is culturally competent,

linguistically and culturally reflective of the communities that are served, and able to offer

- 37 -




integrated treatment for co-occurring disorders. Some of the key strategies have included

training and mentoring to assist consumers and family members to enter the public behavioral

health and recovery services workforce and offering scholarships for consumers/family

members (61 have been offered since 2015).

3F Consumer run and/or consumer driven programs exist to enhance wellness and recovery

M

The Enterprise Resource Center (San Rafael) and the recently-opened Empowerment Clubhouse

(Marin City) are both staffed and run by consumers. Open to all, both centers promote strengths-

based programs that include socialization activities, skill-development, peer counseling,

mentoring, and support groups. Such groups include: Life Management with Crisis Planning,

[How to Facilitate] Groups Made Easy, Wellness and Recovery Action Planning (WRAP), and a

Dual Diagnosis Support Group, among others. The San Rafael location is open seven days a week.

Consumers are informed about the availability of the centers upon hospital discharge and/or

through case managers or therapists.

3G Measures clinical and/or functional outcomes of consumers served M

Timeliness data is analyzed for capacity issues. The MHP looks to see where there is a need to

increase staff, change staff according to needs for culturally specific providers, or change

workflow systems to enhance timeliness. The implementation of KIDnet, a cloud based CANS

application, was completed. The following outcome tools are utilized by the MHP:

• Children/Youth (MHP) - CANS. • Children/Youth FSP and TAY FSP - CANS, Partnership Assessment Form (PAF),

MORS, Key Event Tracking (KET), and 3M (bi-polar depression screen). • Adults (MHP) – Patient Health Questionnaire (PHQ-9), Alcohol Use Disorder

Identification Test (AUDIT), Drug Abuse Screening Test (DAST), and American Society of Addiction Medicine assessment (ASAM)

• Adult FSP – PAF, MORS, KET, and 3M (Quarterly Reports) • • CSU – Broset Violence Checklist (BVC)

3H Utilizes information from Consumer Satisfaction Surveys M

The MHP presented analysis of POQI results and charts/graphs, which included comparison to

previous years. This data is used to assess needs for program changes.

Key Components Findings—Impact and Implications

Access to Care

- 38 -


• The new MHP clinic location in Novato gives access to better meet the needs of

consumers in the northern part of the county.

• The new Clubhouse in Marin City addresses a previously underserved population that

had no mental health services option in that part of the county.


• The MHP has increased its timeliness of response to urgent care requests through the

mobile crisis teams and crisis stabilization unit.

• Some contractors are not included in the EMR and therefore their timeliness data are

not captured.

Quality of Care

• New activities have been implemented to increase response rate to POQI. These

included volunteer recruitment, snacks for consumers participating, staff training in

presenting the POQI to the consumer, and advanced notice given to staff about POQI

week.

• The MHP has created an EBP lead position to expand the appropriate implementation

and use of EBPs across all programs.

• MHP staff have received 19 trainings on cultural competency in the past six-month

period. These include, but are not limited to, information on treatment for veterans, the

LBGTQ community, and issues concerning African Americans with a focus on males.

Consumer Outcomes

• It is unclear if the results of the POQI and the trends it demonstrates are disseminated

to all stakeholders.

• Some peer staff interviewed were under the impression that without a clinical license,

there were no opportunities for upward mobility.

- 39 -


CONSUMER AND FAMILY MEMBER FOCUS

GROUPS CalEQRO conducted two 90-minute focus groups with consumers and family members during the

site review of the MHP. As part of the pre-site planning process, CalEQRO requested two focus

groups with 8 to 10 participants each, the details of which can be found in each section below.

The consumer/family member focus group is an important component of the CalEQRO site review

process. Obtaining feedback from those who are receiving services provides significant information

regarding quality, access, timeliness, and outcomes. The focus group questions are specific to the

MHP being reviewed and emphasize the availability of timely access to care, recovery, peer support,

cultural competence, improved outcomes, and consumer and family member involvement. CalEQRO

provides gift certificates to thank the consumers and family members for their participation.

Consumer/Family Member Focus Group 1

CalEQRO requested a culturally diverse group of 8-10 parents/caregivers of child/youth

beneficiaries who are mostly new clients who have initiated/utilized services within the past 12

months. The focus group was comprised of a combination of foster parents and parents. The

participants were all female and included three disparate ethnicities. The focus group was held on

January 10, 2018 at Room 109, 3230 Kerner Blvd (Youth Family Services), San Rafael, CA 94901.

Number of participants: 3

Only one participant entered services within the past year and so experiences are merged with

other participants for purposes of confidentiality.

General comments regarding service delivery that were mentioned included the following:

• Participants reported that all children have therapists whom they see 2-3 times a week.

• None of participants’ children who are receiving services have psychiatrists or are

prescribed psychotropic medications.

• One participant reported having a parent partner and finds this helpful.

• The participants reported that there are parenting classes available. Also, one reported

that the therapist did a parent group and it was both educational and supportive to the

parent.

• All participants know who to call in an urgent situation. All stated they would call or text

the therapist during regular business hours and outside of that call a parent partner or

the police if necessary.

- 40 -


• Participants report being involved in their children’s treatment planning and ongoing

care.

• Two of the participants endorsed that their children have been improving in the last 12

months of services.

• None of the participants noted transportation as an issue, however housing was noted

as an issue.

• All participants reported that their language and cultural needs are met by the staff

providing services.

• Participants stated the therapists keep them informed on what is going on within the

MHP.

• Two surveys each year are given to the participants to record their assessment of the

access, timeliness and quality of services. There was no certainty as to whether this was

the POQI or another survey.

Recommendations for improving care included the following:

• Counselors in elementary schools to begin services earlier in the child’s life.

• More attention needs to be paid to illicit drug availability in middle and high schools.

• The children who are in services need more activities, not just treatment.

Interpreter used for focus group 1: Yes Language(s): Spanish

Consumer/Family Member Focus Group 2

CalEQRO requested a culturally diverse group of adult beneficiaries who are mostly new clients

who have initiated/utilized services within the past 12 months. The focus group was comprised of

five consumers and one consumer and family member. The participants were three males and three

females and included both Caucasian/White and African American/Black ethnicities. The focus

group was held on January 11, 2018 at Room 105, 3240 Kerner Blvd (Connection Center, Marin

Health & Wellness Campus), San Rafael, CA 94901.

Number of participants: 6

Only one participant entered services within the past year and so experiences are merged with

other participants for purposes of confidentiality.

General comments regarding service delivery that were mentioned included the following:

- 41 -


• The participants generally agreed they had sufficient access to therapist appointments.

The frequency ranged from weekly to once every two weeks depending on the need.

• None of the participants had received either group therapy or family therapy. Most

services provided were one-to-one.

• Five of the six participants see a psychiatrist. The frequency ranges from every two

weeks to every four months depending on the need.

• All participants reported that they have a case manager. The involvement with the

consumer varies according to need. Some see the case manager five times a week, while

others connect every two weeks, again, depending on the need.

• Participants were aware of support groups offered through BHRS. These included

mindfulness, Seeking Safety, Dialectical Behavior Therapy (DBT), emotional support

group among others. Alcoholic Anonymous was mentioned as a support, although not a

formal county resource. There were also support groups for those who resided in a

halfway house.

• The participants reported that if they had an urgent situation they would text or call the

therapist or call the case manager. When these people are unavailable the recording

refers the consumer to 9-1-1 for psychiatric emergencies.

• All participants agreed that they have a say in their treatment plan and ongoing

treatment. The majority report discussing their treatment with the therapist on a

regular basis.

• The participants expressed appreciation for the availability of job training and

education through Buckelew programs and Integrated Community Services (ICS),

including Support and Treatment After Release (STAR) and Helping Older People Excel

(HOPE).

• The participants all agreed that the counselor and/or therapist ask for their input in

how to improve services. They are also asked to do a survey at least one time a year.

Recommendations for improving care included the following:

• More Whistlestop tickets for transportation services.

• A better variety of groups offered, to include a survivor group.

• The participants agreed that services were accessible and helpful to their wellness and

recovery. One complaint was that ICS is sometimes difficult to contact.

Interpreter used for focus group 2: No

- 42 -


Consumer/Family Member Focus Group Findings—

Implications

Access to Care

• Both consumer groups reported that there were no issues accessing services. They

reported that changes in level of services was handled efficiently.

• The parent/caregiver of youth beneficiaries reported that school based services were

available to their children, which increased their access to services.


• Participants noted that the increase in psychiatrist availability had improved their

services and ability to be medication compliant.

• None of the participants noted any lag time from request for services to offered

appointment.

Quality of Care

• The adult consumers reported a variety of resources for support groups and treatment

modalities offered to them by BHRS. They find these useful to their wellness and

recovery and to increasing their coping skills and resiliency.

• Language and cultural competency in service delivery appeared adequate within all the

services discussed in the focus groups.

Consumer Outcomes

• Consumers reported that they felt that they received services that facilitated their

wellness and recovery.

• Consumers are asked for their input by therapists and counselors and are given surveys

to provide feedback on the services they receive.

- 43 -


INFORMATION SYSTEMS REVIEW Understanding an MHP’s information system’s capabilities is essential to evaluating its capacity to

manage the health care of its beneficiaries. CalEQRO used the written response to standard

questions posed in the California-specific ISCA, additional documents submitted by the MHP, and

information gathered in interviews to complete the information systems evaluation.

Key Information Systems Capabilities Assessment (ISCA)

Information Provided by the MHP

The following information is self-reported by the MHP through the ISCA and/or the site review.

Table 9 shows the percentage of services provided by type of service provider.

Table 9: Distribution of Services, by Type of Provider

Type of Provider Distribution

County-operated/staffed clinics 25%

Contract providers 73%

Network providers 2%

Total 100%

Percentage of total annual MHP budget dedicated to supporting information technology operations

(includes hardware, network, software license, IT staff): 3%

The budget determination process for information system operations is:

☐ Under MHP control

☐ Allocated to or managed by another County department

☒ Combination of MHP control and another County department or Agency

- 44 -


MHP currently provides services to consumers using a telepsychiatry application:

☐ Yes ☒ No ☐ In pilot phase

Summary of Technology and Data Analytical Staffing

MHP self-reported technology staff changes (Full-time Equivalent [FTE]) since the previous

CalEQRO review are shown in Table 10.

Table 10: Technology Staff

IS FTEs (Include Employees

and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled Positions

3 1 1 0

- 45 -


MHP self-reported data analytical staff changes (in FTEs) that occurred since the previous CalEQRO

review are shown in Table 11.

Table 11: Data Analytical Staff

IS FTEs (Include Employees

and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled Positions

3 1 0 0

The following should be noted with regard to the above information:

• A new IT Supervisor began in March 2017.

• The analytic staff addition is a Department Analyst II; this position was added in

February 2017.

Current Operations

• The MHP continues to utilize the ShareCare/Clinicians Gateway system. Marin HHS

Technical Services staff contribute to the BHRS IT support by providing management of

hardware resources and desktop support.

Table 12 lists the primary systems and applications the MHP uses to conduct business and manage

operations. These systems support data collection and storage, provide electronic health record

(EHR) functionality, produce Short-Doyle/Medi-Cal (SD/MC) and other third-party claims, track

revenue, perform managed care activities, and provide information for analyses and reporting.

Table 12: Primary EHR Systems/Applications

System/Application Function Vendor/Supplier Years Used

Operated By

ShareCare Practice Management

The Echo Group 6 Marin BHRS-IT

Clinician’s Gateway EHR Krassons 11 Marin BHRS-IT RxNT eRx RxNT 8 Marin BHRS-IT

MySQL PES Tracking PES Medication

Tracking, PES Access Log

Marin HHS Technical Services

3 Marin HHS Technical Services

ImaVisor Document Imaging

Marin HHS Technical Services

2 Marin HHS Technical Services

- 46 -


Priorities for the Coming Year

• Create a business plan and begin RFP development for replacement of the

ShareCare/Gateway system.

• Complete development and rollout of an intranet site for BHRS.

• Continue ongoing vendor issue management with weekly communication of high

priority issues.

• Create TBH physicians review reports, including note completion timeliness tracking.

• Continue planning and design of the Marin Health Gateway Health Information

Exchange (HIE) with partners. The HIE will initially include four local FQHCs and Marin

General Hospital. User permissions and role definitions are currently being defined.

Meetings are held monthly. Due to the varied participants and varied resources of

participants, an HIE implementation target date has not yet been identified.

• Develop a process with QA to accurately categorize, analyze and appropriately act on

internal QA disallowed services.

Major Changes Since Prior Year

• In October 2017, the implementation of a cloud based CANS system, KIDnet, operated

by Advanced Metrics (AMS) was completed. One and a half years of paper CANS

documents were transferred to the cloud based system.

• A contract provider database was implemented in July 2017 to assist contract monitors

track and manage vendor contracts and associated expenditures. The database

currently contains only mental health contract data, but substance use service contracts

will be added in FY18-19.

• The BHRS IT Supervisor initiated an EHR/Practice Management User’s group. This

group is multi-purpose, and elicits input from multiple levels of end users of both

ShareCare and Clinician’s Gateway applications. The User’s Group provides input on

system enhancements, tests changes in development, and will assist in the search efforts

for a new EHR/Practice Management system.

• A Utilization Review (UR) Access database was created to allow utilization review staff

to more easily search services and provide information to assist in generating the

necessary figures for reporting. The MHP reports that the database is much more

efficient for UR staff than the previous Crystal report process.

• Report enhancements were made: staff role for a consumer is visible, enhanced

confidential client identification, and caseload reports were scheduled to run monthly.

- 47 -


• A new report was developed to assist clinicians in tracking timeliness of progress note

completion and pending note status.

• Delayed CSI file submissions were addressed; back logged files were submitted; and, file

submissions are now current.

• The POQI survey technical process was improved by utilizing Teleforms software. IT

staff were able to add barcodes and reporting unit identification to the survey forms.

The pre-labeled program information creates efficiency in the tracking of distributed

surveys and aids in the accuracy of data collection and analysis.

• The MHP can now perform reconciliation and more detailed analytics on 835 and 837

files by using PowerShell (a command line scripting language) with a Dimension

Reports flat file.

Other Significant Issues

• The MHP experienced enhanced IT staffing during half of the last year due to a County

approved six-month overlap between the outgoing/retiring IT Supervisor and the

incoming IT Supervisor. This double staffing at the supervisor level allowed for

thorough training and knowledge transfer. It also gave the incumbent and new

incoming supervisor the ability to review the current status of the EHR/PM

applications, including current functionality, open tickets and upgrade requests, in

preparation for developing a business plan for a new EHR/PM procurement.

Plans for Information Systems Change

• The MHP is considering a new system, but there is no formal plan in place and no

project team assigned to accomplish it. The MHP is exploring the possibility of a new

system due to the difficulty in maintaining and updating a dual vendor system,

ShareCare (The Echo Group)/ Clinicians Gateway (Platton Technologies).

- 48 -


Current Electronic Health Record Status

Table 13 summarizes the ratings given to the MHP for EHR functionality.

Table 13: EHR Functionality

Rating

Function System/Application Present Partially Present

Not Present

Not Rated

Alerts X

Assessments Clinician’s Gateway X

Care Coordination X

Document imaging/storage ImaVisor X

Electronic signature—consumer

Clinician’s Gateway X

Laboratory results (eLab) Quest Diagnostics X

Level of Care/Level of Service

Clinician’s Gateway X

Outcomes KIDnet X

Prescriptions (eRx) Rx/NT X

Progress notes Clinician’s Gateway X

Referral Management X

Treatment plans Clinician’s Gateway X

Summary Totals for EHR Functionality: 8 2 2 0

Progress and issues associated with implementing an electronic health record over the past year

are discussed below:

• The implementation of KIDnet, a cloud based CANS application, was completed.

• Quest Diagnostics lab results can be viewed via an electronic portal. However, lab

orders remain in paper format due to a lack of Clinician’s Gateway (CG) Vendor,

Krasson’s Inc. resources required to move forward with the implementation of full eLab

functionality.

Consumer’s Chart of Record for county-operated programs (self-reported by MHP):

☐ Paper ☐ Electronic ☒ Combination

Personal Health Record

- 49 -


Do consumers have online access to their health records either through a Personal Health Record

(PHR) feature provided within the EHR, consumer portal, or third-party PHR?

☐ Yes ☒ No

If no, provide the expected implementation timeline.

☐ Within 6 months ☒ Within the next year

☐ Within the next two years ☐ Longer than 2 years

Medi-Cal Claims Processing

MHP performs end-to-end (837/835) claim transaction reconciliations:

If yes, product or application:

Local SQL database, Dimension reports and PowerShell.

Method used to submit Medicare Part B claims:

☐ Paper ☒ Electronic ☐ Clearinghouse

Table 14 summarizes the MHP’s SDMC claims.

Table 15 summarizes the most frequently cited reasons for claim denial.

☒ Yes ☐ No

- 50 -


• Denied claim transactions with the reason code of missing, incomplete, invalid ICD-10

diagnosis or condition are generally re-billable within the State guidelines

Information Systems Review Findings—Implications

Access to Care

• The MHP’s Hispanic/Latino CY16 penetration rate continues to be lower than both

medium county and statewide averages.


• The MHP believes clinician no-show data is incomplete.

Quality of Care

• Barcodes with reporting unit identification were added to survey forms allowing for

improved tracking of distributed surveys and aiding in the accuracy of data collection

and analysis.

• An EHR / performance management user group was initiated.

• A contract provider database was implemented to assist contract monitors to track and

manage vendor contracts.

• A Utilization Review database was created for staff to more easily search services and

provide information to assist in providing the necessary figures in their reporting.

• A report was developed to assist clinicians in tracking timeliness of progress note

completion.

• The implementation of full eLab functionality is hindered by limited resources at CG

Vendor, Krasson’s Inc.

• The BHRS IT Supervisor initiated an EHR/practice management user’s group.

Consumer Outcomes

- 51 -


• The implementation of KIDnet, a cloud based CANS application, was completed.

- 52 -


SITE REVIEW PROCESS BARRIERS The following conditions significantly affected CalEQRO’s ability to prepare for and/or conduct a

comprehensive review:

• There were no barriers or conditions that significantly affected CalEQRO’s ability to

prepare for and/or conduct this review.

- 53 -


CONCLUSIONS During the FY17-18 annual review, CalEQRO found strengths in the MHP’s programs, practices, or

information systems that have a significant impact on the overall delivery system and its

supporting structure. In those same areas, CalEQRO also noted opportunities for quality

improvement. The findings presented below relate to the operation of an effective managed care

organization, reflecting the MHP’s processes for ensuring access to and timeliness of services and

improving the quality of care.

Strengths and Opportunities

Access to Care

Strengths:

• The MHP received approval from the board of supervisors to enter into a contract with

TBH. This allowed for a psychiatry staffing increase of 7.0 FTEs.

• The MHP opened a clinic location in Novato to better meet the needs of consumers in

the northern part of the county.

• The new Clubhouse in Marin City addresses a previously underserved geographic area

and population.

Opportunities:

• The MHP’s Hispanic/Latino CY16 penetration rate continues to be lower than both

medium county and statewide averages.

• The MHP’s percentage of high cost beneficiaries is more than twice the statewide

average.


Strengths:

• The timeliness issue for first psychiatrist visit was resolved with the new psychiatrist

staff through TBH.

Opportunities:

• The MHP is under the impression that non-psychiatrist clinician no-show data are

incomplete.

• Some contractors are not included in EMR and therefore their timeliness data are not

captured.

- 54 -


Quality of Care

Strengths:

• A UR database was created for staff to more easily search services and provide information

to generate the necessary figures in their reporting.

• A report was developed to assist clinicians in tracking timeliness of progress note

completion.

• Due to adequate psychiatry staffing, the use of locum tenens psychiatrists is no longer

necessary.

• A change in the composition of teams that interview job candidates to ensure

recruitment interviews are more culturally diverse was data driven from information

captured in the recruitment process.

• The MHP has added an EBP lead position to expand the appropriate implementation

and use of EBPs across all programs.

Opportunities:

• The implementation of full eLab functionality is hindered by limited resources at CG

Vendor, Krasson’s Inc.

• Contract providers noted reduced and irregular contract provider group meetings with

leadership over this past year.

• The MHP does not have analytical tools in place to assess outcomes of the EBPs from a

programmatic standpoint.

Consumer Outcomes

Strengths:

• The implementation of KIDnet, a cloud based CANS application, was completed. Paper

format charts were scanned into this cloud based system.

• The BOS approved the first classified peer counselor positions.

• COPE training involves a nine month program consisting of a three month peer

counseling course and a six month peer specialist course. This is tied to the peer

counselor classifications.

• The BHRS participated in a multi-county collaboration to design curriculum for

organizations and the community for effective use of peer counselors.

Opportunities:

- 55 -


• It is unclear if the results of the POQI and the trends it represents are disseminated to all

stakeholders.

Recommendations

• Examine the high cost beneficiary population, including, but not limited to, diagnosis,

polypharmacy, hospitalization length of stay and post hospitalization follow-up

timeliness and readmissions, to establish the cause of the high percentage of high cost

consumers. Enact processes and/or procedures as indicated by the data to reduce this

population where appropriate.

• Collaborate with community partners to design outreach programs to address low

penetration rates for Latino/Hispanic adult beneficiaries. Track and trend results.

• Implement regular contract provider group meetings with leadership. Beginning the

meetings with an open discussion format, which was implemented in the MHP Town

Hall meetings, would assist in increasing bidirectional communication and collaboration

with contract providers.

• Assure adequate project management capacity is included in the project plan for the

replacement of the electronic health record/practice management system, in light of

current staffing of information technology.

• Establish a process to ensure clinician no-show data are entered into Clinician’s

Gateway. Assure compliance to the newly developed process by monitoring no-show

rates.

- 56 -


ATTACHMENTS

Attachment A: CalEQRO On-site Review Agenda

Attachment B: On-site Review Participants

Attachment C: Approved Claims Source Data

Attachment D: CalEQRO Performance Improvement Plan (PIP) Validation Tools

- 57 -


Attachment A—On-site Review Agenda

The following sessions were held during the MHP on-site review, either individually or in

combination with other sessions.

Table A1—EQRO Review Sessions - Marin MHP

Opening Session – Changes in the past year; current initiatives; and status of previous year’s recommendations

Use of Data to Support Program Operations

Disparities and Performance Measures/ Timeliness Performance Measures

Quality Improvement and Outcomes

Performance Improvement Projects

Acute Care Collaboration and Integration

Health Plan and Mental Health Plan Collaboration Initiatives

Clinical Line Staff Group Interview

Clinical Supervisors Group Interview

Consumer Employee Group Interview

Consumer Family Member Focus Group(s)

Contract Provider Group Interview –Quality Management

Validation of Findings for Pathways to Mental Health Services (Katie A./CCR)

ISCA/Billing/Fiscal

EHR Deployment

Site Visit to Innovative Clinical Programs: Innovative program/clinic that serve special populations or offer special/new outpatient services.

- 58 -


Attachment B—Review Participants

CalEQRO Reviewers

Lynda Hutchens, NCC, LMFT – Lead Quality Reviewer Melissa Martin-Mollard, PhD – 2nd Quality MH/DMC Reviewer Tom Trabin, PhD, MSM – DMC/ODS EQRO Deputy Director Lisa Farrell – Information Systems Reviewer Deb Strong – Consumer/Family Member Consultant Additional CalEQRO staff members were involved in the review process, assessments, and

recommendations. They provided significant contributions to the overall review by participating in

both the pre-site and the post-site meetings and in preparing the recommendations within this

report.

Sites of MHP Review

MHP Sites

Marin Health and Human Services

Behavioral Health and Recovery Services

Connection Center

Marin Health & Wellness Campus

3240 Kerner Blvd.

San Rafael, CA 94901

Contract Provider Sites

Buckelew Programs

Casa Rene (Crisis Residential Unit)

1109 Sir Francis Drake Blvd.

Kentfield, CA 94904

- 59 -


Table B1 - Participants Representing the MHP

Last Name First

Name Position Agency

Arnao

Pilar

Support Service

Worker

HHS

Ballard

Lisa

Mental Health

Unit Supervisor

HHS

Bhambra

John

Deputy

Compliance

Officer

HHS

Bordeaux

Robert

Mental Health

Practitioner

HHS

Buck

Alison

Homeward

Bound

Cain

Sarah

Utilization

Review

Specialist

HHS

Carpenter

Jayne

Sr. Accounting

Assistant

HHS

Chao

Chua

Program

Manager

Children and

Family

Services

Condon

Catherine

MHSUS Program

Manager

HHS

Crutsinger

Lauren

Executive

Director

Seneca

Family of

Agencies

Devido

Jeffrey

Chief Addiction

Specialist

HHS

Donnell-Abaci Maria Mental Health

Unit Supervisor

HHS

Duvall

Cammie

Utilization

Review

Specialist

HHS

Giampaoli

Domenico

Technology

Systems

Coordinator

HHS

- 60 -



Last Name First


Goncalves

Gustavo

Administrative

Services

Technician

HHS

Gray

Erin

Residential

Services

Supervisor

HHS

Hall

Jordan

Senior Program

Coordinator

HHS

Haro Egda Case Manager HHS

Herrera

Veronica

Administrative

Services

Associate

HHS

Holcombe

Sean

Crisis Specialist HHS

Kaiser

Dawn

Division

Director:

Quality

Management

HHS

Lagleva

Cesar

Ethnic Services

Manager

HHS

Loi

Ngoc

Mental Health

Unit Supervisor

HHS

Main

Galen

Department

Analyst

HHS

Marquard-Byrd

LaRee

Unit Supervisor HHS

Martin

Keely

Administrative

Services

Associate

HHS

Moghbel

Pamela

Administrative

Services Officer

HHS

Moreno

Nina

Mental Health

Practitioner

HHS

Moreno-Peraza Connie Adult/Older

Adult Division

Director

HHS

- 61 -



Last Name First


Nightingale

Vicki

Administrative

Services

Associate

HHS

Norris-Alvarez

Shelley

Social Service

Worker

HHS

Ojeda

Claribel

Department

Analyst

HHS

Olivera

Luis

Support Service

Worker

HHS

Ongwongsakul

Walter

Department

Analyst

HHS

Paler

Todd

Program

Manager

HHS

Pierce

D.J.

MHSUS Division

Director

HHS

Player

Tamara

CEO Buckelew

Programs

Prenter

Geoffrey

Mental Health

Practitioner

HHS

Rajparia

Amit

Medical

Director

HHS

Robinson

Brian

Unit Supervisor HHS

Rothery David

Compliance

Officer

HHS

Santiago

Dario

Mental Health

Practitioner

HHS

Saucedo

Maritza

HHS

Sciacca

Laura

Intern Training

Supervisor

HHS

Smith

Katie

Utilization

Review

Specialist

HHS

Steffy

Leigh

SUS Data Mgmt.

Comp.

HHS

Swanson

Hilary

Mental Health

Practitioner

HHS

- 62 -



Last Name First


Sweeney

Mary Kay

unknown Homeward

Bound

Tavano

Suzanne

Director, BHRS HHS

Tognotti

Angela

Unit Supervisor HHS

Webster

William

Utilization

Review

Specialist

HHS

Wilbur

Steve

Quality

Improvement

Coordinator

HHS

Williams

Mike

Quality

Assurance

Sunnyhills

Children’s

Services

Zane Catharine

Unit Supervisor HHS

Zvanovec

Denise

Assistant CFO HHS

- 63 -


Attachment C—Approved Claims Source Data

Approved Claims Summaries are provided separately to the MHP in a HIPAA-compliant manner.

Values are suppressed to protect confidentiality of the individuals summarized in the data sets

where beneficiary count is less than or equal to eleven (*). Additionally, suppression may be

required to prevent calculation of initially suppressed data, corresponding penetration rate

percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

Table C1 shows the penetration rate and approved claims per beneficiary for just the CY16 ACA

Penetration Rate and Approved Claims per Beneficiary. Starting with CY16 performance measures,

CalEQRO has incorporated the ACA Expansion data in the total Medi-Cal enrollees and beneficiaries

served.

Table C2 shows the distribution of the MHP beneficiaries served by approved claims per beneficiary

range for three cost categories: under $20,000; $20,000 to $30,000, and those above $30,000.

- 64 -


33Attachment D—PIP Validation Tools

PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 CLINICAL PIP

GENERAL INFORMATION

MHP: Marin

PIP Title: Improving Follow-Up After Hospital Discharge

Start Date: 02/1017

Completion Date: Ongoing

Projected Study Period: 24 months

Completed: Yes ☐ No ☒

Date(s) of On-Site Review: 01/10-11/2018

Name of Reviewer: Lynda Hutchens

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☒ Active and ongoing (baseline established and interventions started)

☐ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☐ Inactive, developed in a prior year

☐ Submission determined not to be a PIP

☐ No Clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish): The PIP hypothesis postulates that if the MHP increases the percentage of clients who receive timely follow-up appointments (7 days and 30 days measured) after a psychiatric inpatient discharge, this will consequently decrease the number of clients who are readmitted to psychiatric hospital within 30 days of discharge. The PIP focuses on streamlining communication between hospitals

- 65 -


and the MHP during a psychiatric inpatient stay and puts mechanisms in place at multiple possible points post-discharge to ensure better outcomes for clients. The goal is to reduce readmission to the hospital by increasing clients who receive follow-up post discharge.

ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)

Component/Standard Score Comments

1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to Determine

A multi-disciplinary team with participation from the different business units involved in ensuring that clients receive timely follow-up care after hospitalization discharge. Staff involved directly with patient care, subject matter experts, administrative and data analytic support and management sponsors responsible for supporting change were include. The MHP reports that they plan to involve consumers in the project once it is at the point of maturity where their expertise can be best utilized. This point in time was not specific.

1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☒ Met

☐ Partially Met

☐ Not Met


Post-hospital follow-up performance measured by MHP at regular intervals. Data from October 2016 and January 2017 shown in Step 1 of PIP submission tool. New intervention will track readmits.

Select the category for each PIP:

Clinical:

☐ Prevention of an acute or chronic condition ☐ High volume services

☒ Care for an acute or chronic condition ☐ High risk conditions

Non-clinical:

☐ Process of accessing or delivering care

1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☒ Met

☐ Partially Met

☐ Not Met


All county responsible clients hospitalized for treatment of a mental health disorder and discharged from an acute inpatient setting inpatient were eligible.

- 66 -


1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:

☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other

☒ Met

☐ Partially Met

☐ Not Met


Totals 4 Met 0 Partially Met 0 Not Met 0 UTD

STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing?

Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative:

Will implementing interventions including clear written procedures and identification and clarification of BHRS staff roles and responsibilities during hospitalization and discharge follow up, along with enhancing permissions to BHRS CSU and Access contact log improve MHP follow up after hospitalization rate within 7 days by 5% and 30 days by 5% by Feb. 2019 as measured by an improved tracking mechanism that drives workflow? Will these interventions also decrease the number of clients re-hospitalized within 30 days after psychiatric hospital discharge by at least 5%.

☒ Met

☐ Partially Met

☐ Not Met



STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:

☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☒ Other

☒ Met

☐ Partially Met

☐ Not Met


All consumers who are admitted to inpatient psychiatric treatment. See #1.3

- 67 -


3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☒ Utilization data ☐ Referral ☐ Self-identification

☐ Other: <Text if checked>

☒ Met

☐ Partially Met

☐ Not Met


All consumers who are admitted to inpatient psychiatric treatment.


STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators:

1. % of clients that receive follow-up care within 60 days after psychiatric inpatient discharge.

2. % of clients that receive follow-up care within 7 days after psychiatric hospital discharge.

3. % of clients that receive follow-up care within 30 days after psychiatric hospital discharge.

4. % of clients that receive follow-up care by type of follow-up.

5. % of clients that are re-hospitalized within 30 days after psychiatric hospital discharge

☐ Met

☒ Partially Met

☐ Not Met


Indicators are objective, defined, and measurable. The consumer outcome is measured by % of re-hospitalized patients within 30 days. This need to be broken into those who received aftercare in 7 or 30 days and those who did not.

- 68 -


4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☐ Health Status ☒ Functional Status

☐ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☐ Yes ☒ No

Are long-term outcomes implied? ☒ Yes ☐ No

☐ Met

☒ Partially Met

☐ Not Met


Outcomes are assumed to be/equivalent to reduction of rehospitalizations.


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

<Text>

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


- 69 -


Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD

STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

☒ Met

☐ Partially Met

☐ Not Met


6.2 Did the study design clearly specify the sources of data?

Sources of data:

☐ Member ☐ Claims ☐ Provider

☒ Other: Utilization data

☒ Met

☐ Partially Met

☐ Not Met


- 70 -


6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☒ Met

☐ Partially Met

☐ Not Met


Data used to evaluate impact of PIP was collected using various tools and by various staff within BHRS. This included:

Current Inpatient Log (MS Excel Spreadsheet) – used by various staff within organization to include CSU team and the placement/hospital liaison team to notify of a psychiatric admission.

TAR Log (MS Access Database) – Log contains information of admit and discharge from an inpatient psychiatric unit and for whom treatment authorization has been requested. Utilized by UR team and updated by UR support staff when they receive discharge records from hospital.

Clinician’s Gateway (EMR) – This is the MHP’s electronic medical record and is used by the Hospital Liaison staff among others to document clients who are already linked to a BHRS program. Follow-up services for registered clients are recorded in this system.

ShareCare (EPM) – The MHP’s practice management system used to capture and obtain discharge data and follow-up service data for clients. System interfaces with MHP’s EMR.

MCRT Log – Updated by Mobile Crisis Team and used by QM department analyst to obtain data regarding clients who had follow-up services after psychiatric discharge.

BHRS CSU & Access Contact Log – Used by various teams within BHRS including the Access Team, hospital liaisons, Transition Team, and CSU to document various types of information on pre-consumers and active clients.

- 71 -


6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☐ Survey ☐ Medical record abstraction tool

☐ Outcomes tool ☐ Level of Care tools

☐ Other: See 6.3

☒ Met

☐ Partially Met

☐ Not Met


See 6.3

6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

☐ Met

☐ Partially Met

☐ Not Met

☒ Unable to Determine

It was not clear what the specific plan for analysis was and how the MHP would deal with untoward results. Not enough information to grade.

6.6 Were qualified staff and personnel used to collect the data?

Project leader:

Name: Dawn Kaiser

Title: LCSW, CPHQ

Role: QM Division Director

Other team members:

Names: Various staff within BHRS

☐ Met

☒ Partially Met

☐ Not Met


There was no clarification of specific staff members who completed this function other than “various staff”.


- 72 -


STEP 7: Assess Improvement Strategies

7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

1. UR Specialist: Conduct regular concurrent review with inpatient hospital social workers. Notify hospital liaisons and outpatient treatment team regarding high priority targets (imminent discharges, placement challenges, and high-risk situations).

2. Hospital liaisons required to document encounters in Clinician’s Gateway (CG) for registered clients and in the BHRS CSU and Access Log to document for non-registered individuals (pre-consumers).

3. Create written materials to inform hospitals and internal staff regarding procedures to: Casa Rene, Transition, and Access Team. Distribute referral procedures packet to all hospitals and store on Intraweb.

4. Clarify BHRS staff roles during discharge follow up phase.

5. Enhance permissions to BHRS CSU and Access Contact Log to allow increased communication among responsible parties.

6. Repurpose the use of the “Urgent” feature in the Access Contact Log to track high risk consumers who need follow up including post-hospital consumers. Report generated will be used in daily Access Team “morning huddle”.

☒ Met

☐ Partially Met

☐ Not Met



- 73 -


STEP 8: Review Data Analysis and Interpretation of Study Results

8.1 Was an analysis of the findings performed according to the data analysis plan?

This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


The new indicators will produce more analysis. MHP needs to analyze rehospitalization broken down into those who received post discharge follow-up and when and those who did not.

8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled? ☒ Yes ☐ No

Are they labeled clearly and accurately? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: 10/16 – 1/17 baseline and 2/17 – 6/17 measurements

Indicate the statistical analysis used: ______n/a________

Indicate the statistical significance level or confidence level if available/known: _______% _X_Unable to determine

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


New indicator will have to be tracked and analyzed. This includes breaking down into those consumers who received post discharge follow-up and whether within 7 or 30 days and those who did not. Did post discharge follow up decrease rehospitalization within 30 days as the study question asks? Did the interventions increase post hospitalization follow-up?

- 74 -


8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

<Text>

Conclusions regarding the success of the interpretation:

<Text>

Recommendations for follow-up:

<Text>

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


This will need to be expanded upon into the next year’s data.


STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask: At what interval(s) was the data measurement repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The PIP required a redesign of the Study question and addition of interventions, which make it necessary to wait until the next review to evaluate.

9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☐ Yes ☐ No

Clinical significance: ☐ Yes ☐ No

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


Some improvement was noted, but not over a long enough time period or analyzed sufficiently to state statistical or clinical significance.

9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☒ Fair ☐ High

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


The difference between 7-day follow-up and 30-day follow-up is minimal.

- 75 -


9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☐ Moderate ☐ Strong

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☐ Partially Met

☒ Not Met

☐ Not Applicable


There was insufficient narrative or data tables over a long enough period to prove sustained improvement.


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No

PIP is to be continued and is redesigned with clinical outcome for consumers for this coming year.

ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS

Conclusions:

The PIP as originally presented on site did not measure what effect increased f/u post hospitalization had on readmissions within 30 days?

Data shows that their f/u rates for 7 and 30 days are higher than state average, however the readmission rate is also high.

PIP study question was redesigned to include measuring readmissions within 30 days as an outcome.

- 76 -


Recommendations:

Redesign of PIP to include consumer clinical outcomes was accomplished through TA. This involved changes/revisions to the study question, indicators, and interventions that were appropriate.

EQRO recommends the MHP create system to find out why readmission rates are as high as they are given that follow-up rates are also high.

Also recommended is breaking out data to analyze the consumers who readmit within 30 days, which received follow-up in 7 days, 30 days, or not at all.

Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☒ Confidence in PIP results cannot be determined at this time

- 77 -


PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY17-18 NON-CLINICAL PIP

GENERAL INFORMATION

MHP: Marin

PIP Title: Casa Rene Occupancy Rate

Start Date: 02/2017

Completion Date: Ongoing

Projected Study Period: 24 months

Completed: Yes ☐ No ☒

Date(s) of On-Site Review: 01/10-11/2018

Name of Reviewer: Lynda Hutchens

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☒ Active and ongoing (baseline established and interventions started)

☐ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☐ Inactive, developed in a prior year

☐ Submission determined not to be a PIP

☐ No Non-clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish):

The PIP aims to implement interventions that allow Casa Rene, the county’s Crisis Residential Unit, to fully utilize its bed capacity to the extent that the demand for diversion and/or step-down beds are needed. The MHP will measure the ability of Case Rene to reduce CSU utilization and psychiatric hospitalization within 60 days post discharge from Casa Rene. Interventions are directed towards resolving barriers and creating a more effective access system to admission to Casa Rene in an expeditious timeframe.

- 78 -


ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)


1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☒ Met

☐ Partially Met

☐ Not Met


Marin County Mental Health Plan formed a team consisting of Behavioral Health and Recovery Services, Buckelew Programs, and Casa Rene stakeholders. Team members included clinicians, upper and mid-level managers, administrative and quality management staff. Individuals were selected based on being: subject matter experts, having direct involvement in relevant processes, or having the responsibility/authority to support program change. The team met at regular intervals during the duration of the project. Throughout this review period, subgroups of the teams met because the MHP staff and contract provider (Buckelew) staff primarily met separately. This strategy will be reconsidered for activities that occur during the upcoming year. This begs the question where are the consumers on this team?

1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☒ Met

☐ Partially Met

☐ Not Met


The MHP QM Dept. tracks occupancy rates at Casa Rene (CRU) – the only short-term crisis residential facility in the county. Occupancy rates have not over time conformed to expectations.

Nationally, CRUs were established to lower cost, to offer community based treatment, to reduce ED visits, and to divert hospitalization and/or incarceration. CRUs were meant to achieve these outcomes while also producing patient outcomes comparable to or superior than those of institutionalized care. Cost issues were discussed in narrative of Step 1, but are not relevant to the PIP.

- 79 -


Select the category for each PIP:

Clinical:

☐ Prevention of an acute or chronic condition ☐ High volume services

☐ Care for an acute or chronic condition ☐ High risk conditions

Non-clinical:

☒ Process of accessing or delivering care

1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☒ Met

☐ Partially Met

☐ Not Met


The MHP seeks to increase Casa Rene utilization, with the goal of diverting consumers from hospitalization and CSU by 5% within 60 days of discharge from Case Rene.

1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:

☒ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other

☒ Met

☐ Partially Met

☐ Not Met


All adults, over 18 years of age, experiencing an acute psychiatric crisis who meet criteria to be referred to a crisis residential facility in Marin county.


STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing?

Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative:

“Will admission to Casa Rene from the County’s crisis Stabilization Unit and other referring sources ensure that adults who are experiencing a psychiatric crisis have at least 5% reduction in CSU utilization and 5% in-patient psychiatric hospital utilization within 60 days post discharge from Casa Rene?”

☒ Met

☐ Partially Met

☐ Not Met


Initial study question: “Will streamlining the referral process and removing barriers to admission to Casa Rene from the County’s Crisis Stabilization Unit and other referring sources help to increase utilization of bed capacity and ensure that adult who are experiencing a psychiatric crisis have optimal access to crisis residential services?” This study question uses the term “optimal”, but this is not operationalized. The MHP needs to show right care, right time, for right length of time. There is no consumer benefit in this question.

Revised study question includes a consumer outcome.


- 80 -


STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:

☒ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language ☐ Other

☒ Met

☐ Partially Met

☐ Not Met


The study population for this PIP includes any adult, 18 years of age or older, experiencing an acute psychiatric crisis who meets criteria to be referred to a crisis residential facility such as Casa Rene in the Marin county community.

- 81 -


3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☐ Utilization data ☒ Referral ☐ Self-identification

☐ Other: <Text if checked>

☒ Met

☐ Partially Met

☐ Not Met


● Adults 18 and older.

● Marin County Resident.

● Experiencing a psychiatric crisis.

● At risk for Hospitalization or re-hospitalization.

● Able to engage in services voluntarily.

● Exclusions: Active suicidal thoughts and accessible plan or the inability to communicate to staff when feeling they must act on impulses.

○ Psychotic and unable to respond to redirection, in need of a locked setting for safety.

○ Actively trying to hurt those around them. A plan to hurt someone that is able to be carried through without the ability to communicate to staff when feeling they must act on impulses.

○ Actively trying to hurt self, or has a plan to hurt self (that can result in catastrophic injury) that is able to be carried through and the person is unable to communicate to staff when they are feeling they must act on impulses.

○ Has major medical issues that are beyond management of an outpatient clinic level including symptoms of alcohol or drug withdrawal. Type 1 Diabetics are excluded unless they are able to demonstrate that they can self-administer necessary medication. Medical conditions that require oxygen or in-home nursing/medical care. Incontinent and unable to manage supplies independently.

○ History of recent violence; Casa Rene staff and BHRS staff will mutually decide the admission is contraindicated.

○ Registered sex offender.


- 82 -


STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators:

1. Monthly Occupancy Rate

2. Average time from referral to admission

3. Percent of admissions occurring over the weekend

☐ Met

☒ Partially Met

☐ Not Met


The indicators need to be better defined. Baseline should not be a range. In #1 and #2, the baselines need more specifics.

4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☐ Health Status ☒ Functional Status

☐ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☐ Yes ☒ No

Are long-term outcomes implied? ☒ Yes ☐ No

☐ Met

☒ Partially Met

☐ Not Met


Initial presentation of PIP included no consumer outcomes.


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


- 83 -


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

<Text>

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not Applicable


Totals

0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD

STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

☒ Met

☐ Partially Met

☐ Not Met


Primary quantitative data source is admission and discharge data in ShareCare, the practice management system. This data can be accessed via a query or can be viewed on the Casa Rene dashboard.

6.2 Did the study design clearly specify the sources of data?

Sources of data:

☐ Member ☐ Claims ☐ Provider

☒ Other: Primary quantitative data source is admission and

discharge data in ShareCare, the practice management system. This data can be accessed via a query, or can be viewed on the Casa Rene Dashboard.

☒ Met

☐ Partially Met

☐ Not Met


- 84 -


6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☐ Met

☒ Partially Met

☐ Not Met


The narrative of the PIP submission did not explain who will collect the data, what data will be collected, how often will it be collected? Who will analyze the data, how often will it be analyzed and reported?

6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☐ Survey ☐ Medical record abstraction tool

☐ Outcomes tool ☐ Level of Care tools

☒ Other: ShareCare Admit/Discharge information

☒ Met

☐ Partially Met

☐ Not Met


This data source is ultimately reliable, but also is subject to lag since it relies on data entry staff.

6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

☐ Met

☒ Partially Met

☐ Not Met


The MHP had a perspective analysis plan that was not implemented in full. There was no plan for what is the plan if the data doesn’t show what the MHP predicts? More information is needed on what will be collected, how often and specifically by whom. Who will analyze the data and how often?

6.6 Were qualified staff and personnel used to collect the data?

Project leader:

Name: Dawn Kaiser

Title: LCSW, CPHQ

Role: QM Division Manager

Other team members:

Names: Stephanie Schuman, Coordinator Buckelew

Cody Milner, Admin Services Associate Quality and Compliance

☒ Met

☐ Partially Met

☐ Not Met


Data to be collected on ongoing basis.


- 85 -


STEP 7: Assess Improvement Strategies

7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

1. Clarify admission criteria and train staff on all shifts on admission process.

2. Develop new staffing structure to support admission policy and procedure.

3. Educate referral Sources on client admission criteria.

4. Unify admission workflows among the three referral sources.

5. Implement on site CSU psychiatry staffing.

6. Access to Golden Gate Pharmacy weekend system.

7. Deploy MHP staff to expedite referral to admission process.

☒ Met

☐ Partially Met

☐ Not Met



STEP 8: Review Data Analysis and Interpretation of Study Results

8.1 Was an analysis of the findings performed according to the data analysis plan?

This element is “Not Met” if there is no indication of a data analysis plan (see Step 6.5)

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Section 8 of the PIP submission includes charts and graphs of data outcomes.

8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled? ☒ Yes ☐ No

Are they labeled clearly and accurately? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


- 86 -


8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: ___________________

Indicate the statistical analysis used: _________________________

Indicate the statistical significance level or confidence level if available/known: _______% ______Unable to determine

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

See comments

Conclusions regarding the success of the interpretation:

Unclear

Recommendations for follow-up:

Redesign PIP to include consumer outcomes that are quantifiable. This was accomplished through TA process.

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


Data analysis for this project has been challenging for the MHP. Casa

Rene referrals are paper documents, and although admissions and

shift notes are captured in the EMR, the most definitive admission

and discharge data is hand entered into the Practice Management

system by the contractor and is often not complete until after a lag.

Data has been tracked and interpreted differently between the

contractor and the MHP, and the two entities are collaborating to

reconcile these differences, which seem to be definitional to some

extent. Tracking timeliness from referral to admission has been

difficult given the manual nature of the process – changes to the EMR

would make timeliness more apparent, however enhancements to

the EMR are not being pursued given that the MHP intends to

procure a new system. Tracking of percent occupancy was relatively

easy, but did not provide the transparency that would exist if the

actual number of free beds were known in a “live” manner to referral

sources. Overall occupancy rate can also be skewed by other factors,

such as short lengths of stay, as became apparent during this analysis.

The MHP needs to clearly define how they will measure and analyze

performance indicators 4 and 5.

- 87 -



STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask: At what interval(s) was the data measurement repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☒ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Data analysis was conducted by the same staff using the same methodology. Quantitative improvements can be attributed to substantive changes in staffing and workflows. Additional improvements are warranted, and the MHP plans to pilot the use of a staff position that is not part of CSU staffing ratios to complete referrals and facilitate admissions. Longer term implementation of interventions is warranted, with monthly monitoring to measure the impact on the occupancy rate, with an eye to further interventions that may be needed.

9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☐ Yes ☐ No

Clinical significance: ☐ Yes ☐ No

☐ Met

☒ Partially Met

☐ Not Met

☐ Not Applicable


The MHP did present some data that showed improvement in admissions to the crisis residential unit. However, no consumer outcomes were presented.

9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☐ Fair ☐ High

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The new indicators and measurements over coming year will better answer this question.

9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☐ Moderate ☐ Strong

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


Not enough information was given to grade the statistical evidence of true improvement.

- 88 -


9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☐ Partially Met

☐ Not Met

☐ Not Applicable


The PIP changed and then was abandoned and therefore the MHP was unable to produce measurements over comparable times.


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No

ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS

Conclusions:

The initial presentation of the PIP aimed to implement interventions that allow Casa Rene, the county’s Crisis Residential Unit, to fully utilize its bed capacity to the extent that the demand for diversion and/or step-down beds are needed. This goal lacked consumer outcomes (i.e. why this would benefit the consumer). The MHP revised this to a study question that included consumer outcomes which included 5% reduction of CSU utilization and 5% reduction in hospitalization within 60 days post discharge from Casa Rene.

- 89 -


Recommendations:

While the study question was revised, no revision was made of the initial “Brief Description of the PIP) that included the goal and what the PIP is attempting to accomplish.

Rewriting this portion of the PIP would give it increased credibility.

Continue TA process with EQRO for facilitation of best outcome for this PIP.

Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☒ Confidence in PIP results cannot be determined at this time

fy17 18 medi-cal specialty mental health external … and... · beneficiaries under the provisions...

Documents