measuring value and outcomes for continuous quality ... value and outcomes for continuous quality...
TRANSCRIPT
Measuring Value and Outcomes for Continuous Quality Improvement
Noelle Flaherty MS, MBA, RN, CCM, CPHQ1
Jodi Cichetti, MS, RN, BS, CCM, CPHQ
Leslie Beck, MS 1
Amanda Abraham MS 1
Maria Uriyo, PhD, MHSA, PMP 1
1. Johns Hopkins Healthcare LLC, Baltimore Maryland
Corresponding Author: Noelle Flaherty, MS, MBA, RN, CCM, CPHQ
Biographical Sketches:
Noelle Flaherty, MS, MBA, RN, CCM, CPHQ is Director of Quality Improvement at Johns
Hopkins HealthCare LLC. (JHHC). Ms. Flaherty earned her BA from Bryn Mawr College, BS
from Johns Hopkins School of Nursing, MS in Health Services Leadership and Management
from the University of Maryland School of Nursing, and MBA from the University of Baltimore.
Jodi Cichetti, MS, RN, BS, CCM, CPHQ is the Senior Director of Quality and Clinical
Improvement, WellSpan Health System, and past Senior Director Medical Management, Johns
Hopkins Health System.
Leslie Beck, MS, is the Health Services Project Manager at JHHC. Ms. Beck served as Co-Chair
on the JHHC Diversity Management Leadership Committee and is a member on the Johns
Hopkins Berman Institute of Bioethics Patient Advisory Group. Ms. Beck holds a Bachelor of
Measuring Value and Outcomes for Continuous Quality Improvement
2
Science degree in Business Management and a Master of Science degree in Healthcare
Management from the Johns Hopkins University Carey Business School in Baltimore, Maryland.
Amanda Abraham, MS, is the Data Analyst for Quality Improvement at JHHC. She received her
Bachelor’s degree from Penn State University in Health Policy & Administration and a Master’s
degree from George Mason University in Health Systems Administration with a concentration in
Executive Management Her primary role as an analyst is to provide actionable data on the
Value-Based Purchasing project and Medicaid line of business (Priority Partners MCO) to the
administrative team.
Maria Uriyo, PhD, PMP, is the Project Manager for NCQA Accreditation at JHHC. She received
her PhD in Food Science, Master’s in Food Chemistry from Virginia Polytechnic Institute &
State University and a Masters in Health Systems Administration from Georgetown University.
In her current role Dr. Uriyo manages the NCQA accreditation process for JHHC.
Acknowledgement: The Quality Improvement Team developed the framework under direction
of Dr. Chester Schmidt, Chief Medical Officer, JHHC.
Disclosures: No grants or external support funded this work.
Measuring Value and Outcomes for Continuous Quality Improvement
3
Abstract
Quality of care is critically important to health plans, systems and care providers striving
for a healthier population at a decreased cost. Prioritizing action related to population health
and quality of care is a challenge for leaders in the healthcare industry. Poor performance on
quality outcome measures can cost health plans and providers millions of dollars in missed
revenue, fines, penalties and other related health care costs. Challenges for the health care
industry include hundreds of National Quality Forum (NQF) endorsed measures, variations
between government, payer and provider quality indicators and related definitions, as well as
current or future financial penalties linked to different measures by regulatory and accreditation
bodies. In response to the need for meaningful and actionable health care initiatives, the Johns
Hopkins HealthCare LLC Quality Improvement (QI) Department developed a framework for
quality project planning based on a mathematical model in order to easily identify, forecast and
target population health measures. This effort is in alignment with Johns Hopkins Medicine’s
mission to continuously reduce preventable harm, improve patient outcomes and enhance the
value and equity of care around the world by advancing the science of patient safety and quality
through discovery, implementation, education, evaluation, and collaborative learning.
Keywords: Value Based Purchasing; Performance Improvement; Pay for Performance (P4P)
Introduction
Johns Hopkins HealthCare LLC (JHHC) administers health care benefits and services for
over 408,000 lives enrolled in managed care organizations, government and employer sponsored
programs. JHHC supports plan members through added benefits, including outreach, disease
Measuring Value and Outcomes for Continuous Quality Improvement
4
management, complex case management and health education. Quality Improvement (QI)
Program activities support and promote the JHHC mission to improve the lives of our plan
members by providing access to high quality, cost effective, member-centered healthcare.
Additionally, the JHHC QI program supports the Johns Hopkins Medicine (JHM) mission to
improve the health of the community and the world by setting the standard of excellence in
medical education, research, and integration. The QI Department works collaboratively with
Johns Hopkins entities to ensure that members are receiving exceptional health care.
Annually, health outcomes for our covered members are measured through the
Healthcare Effectiveness Data and Information Set (HEDIS®). Based on the HEDIS
® results, the
JHHC Quality Improvement (QI) Department will propose projects to improve patient outcomes.
A standard work method was needed to efficiently and effectively focus and allocate resources to
measure directed projects that will improve care quality, and also provide optimal impact to
quality ranking results for the health plan. In response, the QI Department developed a
framework for quality project planning based on a mathematical model to easily identify,
forecast and target population health quality measures.
Healthcare outcomes and patient/member satisfaction data are important indicators in
measuring care quality, and when monitored with due diligence, support good financial
stewardship. Quality measurement and improved outcomes are a high priority due in part to the
importance of care quality for the patient population, and also to the potential financial impact of
pay for performance programs including value based purchasing (VBP), the Centers for
Medicare and Medicaid Services (CMS) Five-Star Quality Rating System (Stars), and other pay
for performance (P4P) programs. From a financial perspective, the return on investment for
Measuring Value and Outcomes for Continuous Quality Improvement
5
improving health outcomes and patient/member satisfaction can be estimated in the millions of
dollars for some organizations.
Health care systems and leadership are challenged with developing strategies for the
management of quality improvement projects due to the sheer number of quality measures and
related data metrics, and the competing priorities for resources in the healthcare environment.
The National Quality Forum (NQF) has endorsed over 700 measures1 that have undergone peer
review, and are considered clinically relevant to population health and quality of care. Measure
types include process, outcome, intermediate clinical outcome, efficiency and cost/resource
utilization.1 Quality measures that are used by health plans and federal and state agencies are
based on NQF endorsed measures. For example, the Maryland Department of Health (MDH) has
a VBP program for HealthChoice, which is the Medicaid managed care program2, ten (10) of the
thirteen (13) VBP measures are NQF endorsed measures, while the other three (3) were
developed by the MDH. In addition to the NQF measures, the Agency for Healthcare Research
and Quality (AHRQ) National Quality Measure Clearinghouse includes over 2,000 measures,
including some measures that overlap with NQF. 3
In order to manage the volume and variety of measures, a standard framework is needed
to support a focus on clinically relevant opportunities with the greatest positive impact on
population health.4 Meltzer & Chung evaluated thirteen (13) AHRQ quality measures and
proposed a framework for prioritization called “net health benefit”. This clinical framework
focuses on the evaluation of cost and population health outcomes. Porter’s model is another
method for prioritization that focuses on the value for the patient5. Value is quantified through an
analysis of health outcomes relative to costs. Similar to the model proposed by Meltzer &
Chung, Porter’s model is population health based and clinically driven5. The framework
Measuring Value and Outcomes for Continuous Quality Improvement
6
developed by our health plan includes the clinical and cost elements, but also includes
administrative and effort scoring with the goal of the best impact on patient health, patient
experience and cost of care.
Methods
In 2012, the QI Department analyzed patient outcomes data as defined by HEDIS® and
identified many opportunities for improvement. The QI Department had limited resources to
impact all of the measures, and identified a need to develop a targeted approach to quality
improvement initiative planning. The measures that were targeted by the QI team were specific
to the health plan accreditation requirements from the National Committee for Quality Assurance
(NCQA®
), which is a key indicator of health plan excellence. The QI team created an initiative
planning framework to focus on decision making and appropriately allocate resources.
Institutional Review Board Approval (IRB) was not required for the framework development or
associated data analysis.
The framework developed by the QI Department has three (3) phases, and includes
consideration of multiple factors, including national and regional benchmarks, administrative
effort, individual circumstances of the business, and variations in population demographics. This
framework includes tools for quality improvement reporting, planning and intervention. The
framework is cost effective and does not require technology beyond standard data collection of
quality measures.
In the first phase of the framework, previous year results are used to prioritize measures
with the greatest opportunity for improvement when compared to national or regional
benchmarks. The evaluation of previous year results includes the following:
Measuring Value and Outcomes for Continuous Quality Improvement
7
1. Basic population evaluation components: age, gender, geography, and ethnicity.
2. Determine multifaceted population evaluation components identified in previous quality
program evaluations, targeting best practices and opportunities for improvement.
3. A barrier analysis.
4. Identify population targets by geography and population size.
5. Literature review of best practices.
Figure 1: Example of an evaluation of a behavioral health measure
After completing the evaluation and barrier analysis, measures are color coded to
highlight opportunity for improvement. Green indicates maximum improvement opportunity.
Yellow is the next best point, and red indicates minimal movement or need to maintain. Minimal
or need to maintain usually indicates measures that are in the goal range (threshold or target
when compared to benchmarks). The use of color coding and benchmarks in this first phase are
shown in the example in Figure 2. Although NCQA®
benchmarks are used in the example; other
national or regional benchmarks can be applied using the same process. The color coding can be
modified to meet the business or population health requirements of the organization.
Measuring Value and Outcomes for Continuous Quality Improvement
8
Figure 2
After the measures are color coded, the methodology focuses on measures that are
identified as having the best or next best opportunities for improvement (color coded green or
yellow). The JHHC QI department may also decide to focus on high performing measures (color
coded red) if there is a business reason (P4P) or population health rationale to focus on those
measures.
During phase two of the methodology, a percentage to goal value is calculated for each
measure using current year (prospective) data to identify likelihood of success based on current
data. The percent to goal (PG) of a measure moving to the next threshold percentile level is
calculated using the following formula: PG= CCR/NBPTR. In this formula, PG is the percent to
goal; CCR is current compliance rate and NBPTR is the national benchmark percentile target
rate. This calculation helps focus attention to the amount of effort that will be needed to meet
the next percentile benchmark. Some measures might be identified as potentially reaching the
target based on little to no effort. Figure 3 is an example of the second phase of the framework
for CY 2018 planning. The measures in this example have demonstrated improvement, so a
control plan can be put in place for these measures and other measures can be targeted in C 2018.
HEDIS® Measures2017 Final HEDIS
Rate
2017 NCQA®
National
Percentile
Benchmark
Threshold
Additional 2018
HEDIS® Points (if
next benchmark
attained)
Breast Cancer Screening 76.15 75th 0.180
Chlamydia Screening in Women 43.07 50th 0.285
Follow-up After Hospitalization for Mental
Illness - within 7 days66.67 90th 0.000
Measuring Value and Outcomes for Continuous Quality Improvement
9
Figure 3
During phase three, the QI Department meets internally to review the priority measures
identified in the first two phases. The QI Director then meets with business leaders, physicians
and other stakeholders to learn about any business needs or strategies already underway that
could positively (or negatively) impact the quality outcomes for the current year. Factors to
consider prior to scoring measures include specifics related to the financial history of project
planning, patient/member population, company workforce, and historical quality approach.
After gathering input from the QI team, business leaders and other stakeholders, the QI
Director scores each measure using four pre-defined categories: 1) administrative effort; 2)
population impact / relevance; 3) reporting requirements; and 4) anticipated expense.
Definitions of the categories are as follows:
1) Administrative effort is defined as the level or intensity of the total business work effort
related to quality or care management programs;
2) Population impact or relevance is defined as the number of opportunities within the
measure or the relevance assigned. The lower the point score, the higher the anticipated
impact on the population or higher relevance;
3) Reporting effort is defined as the level of complexity for ongoing monitoring and
evaluation of effectiveness of quality projects; AND
Measuring Value and Outcomes for Continuous Quality Improvement
10
4) Anticipated expense includes projected costs for the quality program and potential
financial risks such as VBP penalties, potential loss of contracted members, or other
regulatory fines. The amount of expense is compared with the potential financial gain or
loss.
Each category is scored on a scale of 0-5. Effort scoring is a total cumulative score
(maximum of 20 points) where the higher point value represents the higher/heavier work
effort comparatively for corporate consideration and project planning. Measures identified as
having a lower point value may be selected for rapid cycle quality improvement projects.
Measures with a higher point value that are selected for quality initiatives may need more
time for planning and budgeting to successfully manage the work effort. The cumulative
score can be considered in addition to the other factors in determining the most valuable
quality measure for corporate focus and quality planning for improved clinical and reported
patient outcomes and enhanced value. Figure 4 is an example of the third phase of the
framework that was used for initiative planning in CY 2016.
Figure 4
Results
In 2012, the QI team used the framework and identified breast cancer screening as a
measure for improvement for the commercial line of business. A women’s health screening
Measuring Value and Outcomes for Continuous Quality Improvement
11
brochure was sent to members with opportunities in 2011 and 2012. A significance validation
was calculated for women who received health reminders. The evaluation of the members who
received a mailing had a significantly better rate of compliance (p value <0.0001) than those who
did not receive the mailing in 2011. Based on evaluation, the brochure mailings likely served as
a benefit to remind the member to schedule the appointment for breast cancer screening. HEDIS
® scores increased for this measure from 70.82% (50th
percentile) in Calendar Year (CY) 2012
to 76% (75th
percentile) in CY 2013. The QI Initiative Project expanded in 2014 and 2015 and
the plan has maintained 75th
percentile since CY 21013, indicating that a control plan is in place
to maintain performance for this measure.
Another measure that was identified for improvement using this framework was the
quality measure for follow up care after a mental health hospitalization. A project was initiated
in 2013, and was refined and updated in 2016. As a result of the ongoing focus on this measure,
the results in 2016 demonstrated a statistically significant improvement (9.93%). This
improvement also resulted in an increase in overall accreditation score for the plan, contributing
to the US Family Health Plan ongoing “Excellent” accreditation. Figure 5 demonstrates the
improvement in this measure over time and with a sustained effort to improve the coordination
of care project for mental health hospitalization within seven (7) days.
Measuring Value and Outcomes for Continuous Quality Improvement
12
Figure 5
In addition to the measure specific successes referenced above, overall quality results for
JHHC health plans have been above national standards. Results for 2017 (CY 2016) that
demonstrates the overall success of the framework for one of our programs includes:
– NCQA® Excellent accreditation for US Family Health Plan, the highest level of
health plan accreditation
– NCQA® Rating 5 out of 5, indicating that the US Family Health Plan is a highly
rated national plan.
– The highest possible percentile ranking (NCQA® 90th percentile) for the
following HEDIS® measures identified through the framework: Cervical Cancer
Screening, Comprehensive Diabetes Care, Follow-Up After Hospitalization for
Mental Health – 7 days
Measuring Value and Outcomes for Continuous Quality Improvement
13
– Ongoing compliance for the Breast Cancer Screening measure at the 75th
percentile level.
– Significant improvement (6.90 %) of compliance with chlamydia screening for
women.
Limitations
The framework and associated tools are not automated and do not incorporate statistical
analysis using large data sets. Access to quality data, including administrative, pharmacy,
laboratory, claims, encounters and electronic data is needed to maximize the use of the
framework. Changes within health care, including changes to clinical practice guidelines, quality
measure specifications, payer plan benefits or decreased access to care can result in lower than
expected results. Although leadership input is engaged for the effort scoring, strategic decisions
may be made by leadership that could result in selection of quality measures that are not
identified as priority through the framework.
Discussion
For most quality measures, the framework is applied to the current year using past results
and a minimum of three months’ of quality data. When there is a new quality measure, or when
there are significant revisions to the specification for an established measure, projected trended
rate and percentage to goal are not applicable because there is no historical data available to
calculate the projected trend rate. In these circumstances, the goal percentile ranking is listed as
requiring the greatest effort.
There are measures that require chart audit/abstraction of a sample population, so
prospective administrative data is not available. An example is the HEDIS® measure for
controlling blood pressure6, which cannot be easily measured for a health plan throughout the
Measuring Value and Outcomes for Continuous Quality Improvement
14
year because the measure is based on a representative sample of the entire population, and
measures the last blood pressure of the calendar year. For this type of measure, the projected
rate is derived from the two basic assumptions that past patterns will persist into the future, and
measurable fluctuations in past trends will recur regularly and can be projected into the future.
Previous year’s eligible reporting population and information on total plan membership is
referenced.
Conclusions
The framework provides a standard approach to prioritizing quality improvement projects
with the goal of improving patient outcomes. Executive decisions are made easier by clear
recommendations from subject matter experts that are supported by data. The final product for
executive decision making is an objective report inclusive of a pre-determined and consistent
methodology, that once applied, supports quality improvement prioritization and focus for
intervention. The model developed by the JHHC QI Department has been effective over the
past five years, and quality measures identified through the framework have maintained or
improved each year when the proposed projects were approved and funded. In addition to
proven maintenance or improvement of quality measure performance (by measured year)
identified by the framework, there is a correlated improvement in overall quality ranking.
Implications
The NQF has recognized the importance of developing meaningful measures as well as
improving and prioritizing existing measures, as evidenced by the NQF 2016-2019 Strategic
Plan to answer “an unmet need for NQF to lead, prioritize, and collaborate to drive measurement
that can result in better, safer and more affordable healthcare for patients, providers, and
Measuring Value and Outcomes for Continuous Quality Improvement
15
payers.”1
CMS has also recognized the need for a core measure set to focus on patient outcomes.
CMS, in collaboration with America’s Health Insurance Plans (AHIP), released seven set of
quality measures in 2016.7 The work between CMS and AHIP is ongoing, but does not address
the need for health plans, health systems and providers to prioritize quality improvement
activities and related interventions to improve patient health care. The framework developed
and implemented by the JHHC QI department supports both the NQF and CMS goals related to
identifying key measures for prioritization to improve health outcomes. This framework can be
easily modified for adoption for use in a variety of settings, including health plan/ payor,
Accountable Care Organization (ACO), hospital and provider.
References
1 National Quality Forum (NQF). Healthcare Measurement. Retrieved from
http://www.qualityforum.org/NQF_Strategic_Direction_2016-2019.aspx on June 7th, 2016.
2 Maryland Department of Health and Mental Hygiene. HealthChoice. Retrieved from
https://mmcp.dhmh.maryland.gov/healthchoice/pages/Home.aspx on July 22, 2016.
3 Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse.
Retrieved from http://www.qualitymeasures.ahrq.gov/index.aspx on June 7th, 2016.
4 Meltzer, D. O., & Chung, J. W. (2014). The population value of quality indicator reporting: a
framework for prioritizing health care performance measures. Health Affairs, 33(1), 132-139.
5 Porter, M. What is Value in Health Care? The New England Journal of Medicine 363:26
(2015): 2477-2481.
6 National Committee for Quality Assurance (NCQA). HEDIS ® & Performance Measurement.
Retrieved from http://www.ncqa.org/tabid/59/Default.aspx on June 7th, 2016.
Measuring Value and Outcomes for Continuous Quality Improvement
16
7 Centers for Medicare & Medicaid Services. (2016). CMS and major commercial health plans,
in concert with physician groups and other stakeholders, announce alignment and simplification
of quality measures, Retrieved from CMS.gov on July 22, 2016.