hcia final narrative report - highvaluecollaborative.com · 3 pr_1029pe_final_dart.xlsx measures to...

1 PR_1029PE_FINAL_Dart.xlsx

HCIA Final Narrative Report

Federal Agency and Organization Element to Which Report is Submitted:

Centers for Medicare & Medicaid Services

Federal Grant or Other Identifying Number Assigned by Federal Agency:

● i.e. CMS Awardee Number (1C1CMS33####) 1C1CMS331029PE

Recipient Organization Name: Trustees of Dartmouth College

Recipient Organization Address: ● Including zip code

Dartmouth Medical School 11 Rope Ferry Rd

Hanover, NH 03755-1404 Reporting Period End Date:

● Month, Day, Year (e.g., 12/31/2012; 3/31/2013/; 6/30/2013)

6/30/2016

Certification: I certify to the best of my knowledge and belief that this report is correct and complete for performance of activities for the purposes set forth in the award documents.

Name and Title of Awardee Project Director (Certifying Official):

Allison Hawke, Assoc. Director of Operations

Phone Number: ● Area code, Number, and Extension 603-676-7149

Email Address: [email protected]

Date Report Submitted: ● Month, Day, Year 09/28/2016

mailto:[email protected]

mailto:[email protected]


Final Narrative Report High Value Healthcare Collaborative: Engaging Patients to Meet the Triple Aim Health Care Innovation Award (HCIA) July 1, 2012 – June 30, 2016

Section 1: Final Report The contents of this report are solely the responsibility of the authors and do not necessarily represent the official views of the Department of Health and Human Services or any of its agencies.

Project Goals: The High Value Healthcare Collaborative (HVHC) program set out to engage patients and implement shared decision making (SDM) for participants facing hip, knee or spine surgery decisions, and those with diabetes or congestive heart failure. Methods included a variety of patient engagement (PE) interventions and the use of shared decision making tools and techniques including Decision Aids (DAs) and health coaching to ensure that participants are partners in their care decisions. The goals of this initiative are to improve quality, outcomes, and cost of care by:

1. advancing best practice care models for patients considering hip, knee, or spine surgery and for patients with diabetes or congestive heart failure (CHF);

2. implementing shared decision making (SDM) interventions (e.g., decision tools, coaching) for preference-sensitive decisions (hips, knees, and spine) and patient engagement interventions (e.g., decision tools, motivational interviewing, coaching, and navigation skills) for patients with diabetes and CHF.

Improvement Targets: • Reduce cost: Reduce inappropriate rates (relative rate) of hip, knee and spine surgeries and

episode costs resulting in 5% total cost reduction; for complex patients with diabetes or CHF: reduce hospitalizations by 10% (aggregate relative rate) and reduce aggregate cost of annual episodes by 2%. Overall, the project goal was to reduce the total cost of care for this population by $48.8 Million over three years of the demonstration. Note: we did not include partial year 2015 in our calculations (see explanation below) so used prorated target of $39M for 2013-2014 analytics.

• Improve health: Improve health status measures (function, pain) for >50% of patients considering hip, knee, and spine surgery at one year; for complex patients with diabetes or CHF: reduce emergency department visit rates and hospitalizations by 10% (aggregate relative rate)

• Improve care: Refer >50% of eligible patients to SDM and of those referred, >50% of referred patients/families participate in SDM intervention

Executive Summary

Context: Through the Dartmouth Atlas (http://www.dartmouthatlas.org/) we have reported on the tremendous variation in health care delivery across the U.S. This variation is responsible for billions of dollars in health care spend, year after year. We suggest that as much as 30% of that amount could be saved. The HVHC was formed to address several issues related to this variation in a value-based manner. The HVHC goal is to advance the best care in the right place, at the right time, and in the most efficient and effective manner. The HVHC has been working for years to establish the highest value of care across all HVHC members and others who are interested. We are establishing measurable processes and


measures to be reported transparently to the public for the benefit of our patients and their families. It is our belief that much of the variation in health care delivery is due to patients receiving care for which, if well informed, they might not choose. For this reason, we have chosen to work with CMMI to advance decision tools that aid patients and their families in making treatment decisions. Getting to the "right rate" of elective procedures using evidence-based decision making tools will likely decrease both the variation and amount of the overall healthcare spend for the U.S. ultimately by billions of dollars.

Progress against our stated goals follows (NOTE: “cost” refers to cost to Medicare and payers: claims reimbursement):

Reduce cost: We observed a decrease in 90-day episode actual costs of 3.1% and 4.2% for patients with hip and knee surgeries, respectively (or 5.7% and 5.9% using standardized costs), falling short of our 5% target for actual costs savings. We observed a decrease in annual episode actual costs by .5% and 1.8% for CHF and diabetes patients, respectively (or .6% and 1.7% using standardized costs), falling short of our 2% target for actual cost savings. Total actual cost savings for Medicare was $20.9M (or $27.5M in standardized costs), falling short of our prorated goal of $39M for 2013-2014. If we assume similar trends for two more quarters, the total savings estimate would be $25M and $33M for actual and standardized costs, respectively, and still falling short of the actual $48.8M target.

• Improve health: We observed improved health status measures (function, pain) for 51% of patients with diabetes or CHF and 60% of patients considering hip, knee, and spine surgery at 6+ months, meeting our goal of >50%. ED visits increased by 4% and 3.7% and inpatient hospital admissions decreased by 3.8% and 2.0% for patients with diabetes or CHF, respectively; these outcomes fell short of our 10% reduction target for ED visits and hospitalizations.

• Improve care: The summary ratios of member-submitted enrolled-to-eligible patient counts by condition were as follows: CHF: 0.78, diabetes: 0.55, hip: 0.78 and knee: 0.78. While there are likely data consistency and quality issues inherent in manually tallied data, these ratios are still well above the goal of 50% of eligible patients enrolled.

While we did not hit these cost targets for CMS, the CMMI award has provided an extremely rewarding opportunity to collaborate with our member institutions from across the US with a focus on better evidence-based choices, better care, smarter spending, and healthier people. The value of uncovering and addressing identified challenges and leveraging lessons learned through collaborative implementation were the first important steps to achieving our goals in the longer term. We learned that two years is not enough time for a collaborative as large and diverse as ours to implement common patient engagement interventions requiring such disruptive changes to workflow and clinical practice. Implementing any intervention across so many disparate systems is challenging and the SDM focus was a particular challenge given the predominant fee-for-service payment model that our members primarily work in. Collecting consistent, standardized data from 17 different systems (many of which have multiple IT systems within their healthcare systems, others who still use paper) also presented challenges. These are representative problems as our nation considers healthcare reform at the level of the delivery systems.

Developing the infrastructure necessary to manage the data and create feedback reports has had many benefits and challenges alike, anticipated and unanticipated. CMMI provided the opportunity for our members to accelerate their use SDM and patient engagement tools, representing one of the largest national efforts to our knowledge to implement SDM of tools into standard practice of care. The result


from three years of funding with limited time to implement the necessary integrated system changes, considerably underestimates the potential long term effectiveness of such efforts.

Overall, we were successful in moving the needle toward value-based patient care and setting the stage for future success. We now have the ability to integrate complex, disparate information from CMS and member Electronic Medical Record (EMR) health system data (including self-reported outcomes directly from the patient) across multiple organizations. Access to CMS data allows us to better inform the public, health systems, and their providers on how best to deliver population and value-based care in quantifiable dashboard reports. We are seeing that various patient engagement interventions are being maintained at HVHC sites and we plan to use our new data infrastructure and web-based dashboard reports to continue monitoring the longer term impact of these efforts on choice, quality, outcomes, and cost.

Section 2. Accomplishments & Lessons Learned We set out to engage patients and implement shared decision making (SDM) for participants facing hip, knee or spine surgery decisions, and those with diabetes or congestive heart failure. Methods included a variety of patient engagement (PE) interventions (such as length of stay expectation management, depression screening, pre-op training, etc.) and the use of shared decision making tools and techniques including Decision Aids (DAs) and health coaching to ensure that participants are partners in their care decisions. To accomplish these goals, we worked with the four founding members (Dartmouth-Hitchcock, Denver Health, Intermountain, Mayo) in year 1 to design the SDM interventions, conduct pilot tests, evaluate results, refine the protocol, and codify and disseminate a common intervention for the 11 remaining members to implement in year 2.

The SDM intervention included common referral algorithms for defining eligible patients; processes and roles for processing the referrals and the SDM intervention; data collection tools and processes to collect patient-reported decision quality and experience measures from patients and families pre- and post-SDM intervention, post-clinical encounter, and one-year post-SDM intervention; a limited number of decision aids selected based on accepted criteria; consultation with the health coach before and after watching the decision aid. While members were required to implement a common core protocol (same cohorts, referral triggers, measures), they were allowed flexibility to tailor the protocol to their local environments to ensure local success and also allow for natural experiments across sites.

Key factors for successful implementation included achieving leadership accountability and support; recruiting of 2-3 health coaches per member based on common qualifications; multiple stages of SDM training tailored to different audiences; multi-disciplinary teams designing new process of care; strong communication to patients, families, clinicians and staff; and transparent results with rapid adjustment based on what was working and what was not.

We made considerable progress in this complex and disruptive change to clinical practice. We were successful in building the data infrastructure needed to process and link data from the EHR with CMS claims data and in building web-based portals to make comparative reports of process and outcomes analytics available to members to monitor and adjust their improvement efforts. We were also very successful in training workforce across the country in health coaching techniques and in standing up a learning network among these coaches to share best practices. Finally, we made strides in breaking through cultural and bandwidth barriers to build new processes and tools into the flow of clinical practice.

Implementation milestones included:


Hired 36.7 FTE health coaches Trained and certified 304 health coaches Developed condition-specific Implementation Guides (Hip & Knee, Spine, Diabetes, CHF) Initiated a Health Coach Learning Collaborative (HCLC), engaging health coaches from all members

in an interactive monthly call to share learnings on barriers and facilitators of change Implemented a web-based Survey Administration Tool (SAT) to collect patient-reported experience

and outcomes at more than 136 clinical sites Distributed nearly 20,000 Health Dialog video decision aids and summary booklets Built data infrastructure to process, clean, link, and analyze member-submitted data with CMS

claims data Launched a web-based data reporting portal for members to monitor comparative outcomes near

real-time and adjust interventions accordingly

We also faced and made progress in overcoming many barriers. Both the integration of the interventions into the member systems and the measuring of exposures and outcomes proved challenging due to the complex nature of 21st century health-care delivery. Our efforts with the spine intervention were particularly challenging given how difficult it is for clinicians, health coaches, and staff to determine which spine decision aids should be administered prior to surgical consults given that the choice of decision aid was highly dependent on an accurate spine diagnosis, which spine physicians did not believe could not be done in primary care. Therefore, our analytics for spine interventions were thwarted by a low number of events so we did not include episode costs for spine surgeries in the final analysis. (On the other hand, some sites within HVHC have successfully used SDM for years and have some of the lowest rates of spine surgeries, with the best outcomes quality/cost/time, in the U.S.)

Overview of methods

Methods to attribute patients to the correct care facility were important for (1) assigning patients and cost allocations over time and (2) the ability to appropriately benchmark results comparators. We used the Physician Hospital Network (PHN) attribution of Bynum, et al. to assign patients to hospitals for rate denominators (J. P. W. Bynum, et al. "Assigning Ambulatory Patients and Their Physicians to Hospitals: A Method for Obtaining Population-Based Provider Performance Measurements." Health Services Research. 2007:42(1): 45-62). Results from the evaluation of PHN assignments revealed that a significant number of surgeries were performed outside of PHN assigned hospitals. Given this evidence, we do not think the PHN methodology provided accurate and precise enough denominators to measure the rate changes we had proposed for surgical events and the corresponding effectiveness of the interventions. We assume that misclassification was random and biased our observed results towards no effect. Attributions for CHF and diabetes also likely have misclassification, but to a lesser extent. We do not think the attribution problem is unique to this project. Attribution models continue to be challenging in the current ACO modelling and alternative payment models being implemented under the ACA.

Cost and event outcome measures for these analyses were evaluated in two ways:

1. Method 1: Year-over-year change among all HVHC members involved in the CMMI award (all of their hospitals) was compared between calendar years 2013 and 2014 using CMS claims data, member-submitted data, and patient-reported data, summarized in the HVHC Monitoring Measures (which are measures proposed and approved by CMMI as part of the post-award operational plan). We only used two complete years of data to evaluate change within all of HVHC. Dollar amounts for these comparisons were inflation adjusted to 2013 dollars. The


analyses were not adjusted for patient characteristics. Patients for diabetes and CHF were attributed to hospitals based on the PHN model described above. Patients for surgical conditions were attributed to the actual hospital where surgeries were conducted for all measures except for surgical rates, which were calculated based on PHN attribution (in order to establish the denominator). Longitudinal analysis was reported on calendar quarter or year, irrespective of launch date. See Appendix E for limitations and technical notes on Method 1.

2. Method 2: For target outcome measures, we limited our analysis to Participating and Intervention Hospitals (definitions below) using claims-based criteria for identifying patient cohorts. For most of these analyses, we also compared changes in outcomes for Participating and Intervention Hospitals to change in matched Comparator Hospitals over time. Change relative to Comparators was estimated at the marginal level and we assumed a homogeneous effect among all hospitals. These two cohort analyses were conducted as follows:

a. Intervention Hospitals included all HVHC-member hospitals participating in the CMMI award

at the onset of the project and provided cost-improvement targets by condition as part of the post-award operational plan. The figures present dollars adjusted to 2013 dollars, although we did not adjust for inflation in the difference-in-difference modeling. The analyses were adjusted for patient characteristics. Longitudinal analyses were all reported on calendar quarter or year, irrespective of launch date. We considered this analysis to be more appropriate for our cost savings targets since we built the projected estimates based on these hospitals and our targets were to be achieved in the award period, irrespective of launch dates.

b. Participating Hospitals included all hospitals that met inclusion criteria for provision of interventions based on member-submitted enrollment data. Dollar amounts for these comparisons were inflation adjusted to 2013 dollars. The analyses were adjusted for patient characteristics. Longitudinal analyses were reported relative to individual launch dates. We considered this analysis to be more appropriate for studying the impact of the interventions because we could compare outcomes pre- and post-launch dates for each hospital.

Details on the difference-in-difference design used, inclusion criteria and statistical models are provided in Appendix B, Analysis Design & Statistical Methodology.

Triple Aim Target 1: Cost Savings NOTE: “Cost” refers to cost to Medicare and payers: claims reimbursement.

GOAL: Reduce inappropriate rates (relative rate) of hip, knee and spine surgeries, and episode costs resulting in 5% total cost reduction (from baseline total bundled costs); for complex patients with diabetes or CHF: reduce hospitalizations by 10% (aggregate relative rate) and reduce aggregate cost of annual episodes by 2%.

Method 2a: Comparison of change for HVHC Intervention Hospitals 2013-2014

Year-over-year change in actual and standardized cost among all HVHC Intervention Hospitals for the target cost outcomes is shown in Table 1. Observed changes were estimated using Method 2a:


• We observed a decrease in 90-day episode actual costs of 3.1% and 4.2% for patients with hip and knee surgeries, respectively (or 5.7% and 5.9% using standardized costs), falling short of our 5% target for actual costs savings.

• We observed a decrease in annual episode actual costs by .5% and 1.8% for CHF and diabetes patients, respectively (or .6% and 1.7% using standardized costs), falling short of our 2% target for actual cost savings.

• Total actual cost savings for Medicare was $20.9M (or $27.5M in standardized costs), falling short of our prorated goal of $39M for 2013-2014.

• If we assume similar trends for two more quarters, the total savings estimate would be $25M and $33M for actual and standardized costs, respectively, and still falling short of the actual $48.8M target.

Note: though we did not achieve the percentage change we were hoping for, we were moving in the right direction and believe with more time we might have achieved our goals.

Condition Outcome Mean

Outcome 2013

Mean Outcome

2014 Percent Change n 2014

Total Change

Actual Costs 2013-2014

Total Change

Standardized Cost 2013-2014

CHF Actual Annual Cost $34,199 $34,028 -0.5% 21,578 -$3,691,449

Standardized Annual Cost $38,329 $38,100 -0.6% 21,578 -$4,946,701

Diabetes Actual Annual Cost $15,667 $15,388 -1.8% 43,403 -$12,094,017

Standardized Annual Cost $18,285 $17,967 -1.7% 43,403 -$13,800,262

Hip Surgery

90-day Episode Cost $24,280 $23,533 -3.1% 1,725 -$1,289,321

Standardized 90-day Episode Cost $28,247 $26,637 -5.7% 1,725 -$2,776,917

Knee Surgery

90-day Episode Cost $27,345 $26,196 -4.2% 3,285 -$3,774,652

Standardized 90-day Episode Cost $30,972 $29,158 -5.9% 3,285 -$5,959,301

TOTAL (2013-2014): -$20,849,439 -$27,483,181

Add two quarters -$25,019,327 -$32,979,817

Table 1: Summary of change in inflation-adjusted cost outcomes between 2013 and 2014 within HVHC Intervention Hospitals (Method 2A)


Method 2a: Comparison of change for HVHC Intervention Hospitals relative to Comparators

The analyses comparing the change relative to Comparator Hospitals includes all Intervention Hospitals as described in Method 2a. Results assume that the intervention started in the first quarter of the award period (1QR: July, 2012) for all HVHC Intervention Hospitals. HVHC Intervention Hospitals were defined as having an HVHC member-reported, condition-specific cost-savings target for 2012 or 2013 at the onset of the project. Using this definition and timing allows only two quarters of “pre” intervention to be evaluated for parallel slopes assumption of the difference-in-difference model and likely misclassifies several quarters of pre-intervention as post-intervention.

Surgical Episode Costs (2a)

Average costs for total-hip and -knee replacement 90-day episodes in intervention and comparator groups are shown in Figure 1. While costs in both the intervention and comparator groups had a downward trend over the term of the project, change in cost in Intervention Hospitals was not statistically significantly different than change in Comparator Hospitals. For Figure 1 and 2 below, there is a “pre” period and a series of quarters post-intervention (or years, for diabetes and CHF). In these plots, we are NOT

Figure 1: Average 90-day episode cost and standardized cost for total-hip and -knee replacement among HVHC Intervention Hospitals and matched Comparators, by intervention quarter.


connecting the points because the number of Intervention Hospitals can change over the length of the observation period. We do not want to give the impression that change seen in the plot is due to the intervention when it could actually be due to a high- (or low-) performing hospital reaching the end of its observation time and not being included. Additionally, the “n” on the X-axis is the number of hospitals providing data points for that quarter. All cost measures are inflation adjusted to 2013 dollars.

Chronic Condition Annualized Costs (Method 2a)

Annualized cost estimates for the chronic conditions included in the project are presented in Figure 2 Reference source not found.. While HVHC Intervention Hospitals had aggregate lower costs, change in annualized cost was not statistically significantly different from Comparators over the course of the project.

Figure 2: Average annualized cost and annualized standardized cost for diabetes and CHF patients assigned to PHNs of HVHC Intervention Hospitals and matched Comparators, by year since the Intervention Hospitals started their interventions.


Method 2b: Comparison of change for active HVHC Participating Hospitals relative to comparators

We repeated the comparisons of change in cost using only hospitals that met the definition of HVHC Participating Hospitals, defined in Appendix B, Analysis Design & Statistical Methodology. In addition to the difference of inclusion criteria for included Participating Hospitals, these longitudinal analyses are relative to Participating Hospital specific intervention start-dates.

Surgical Episode Costs (Method 2b)

Estimated changes in Participating Hospital total-hip replacement 90-day episode cost and standardized-cost episode-cost were not significantly different from those in matched comparators. Similarly, for total-knee replacement, the predicted 90-day episode cost and standardized-cost episode-cost changes were not significantly different between Participating Hospitals and Comparators. The average episode-cost and standardized cost by intervention quarter by Participating and Comparator Hospitals are shown in Figure 3 for hip and knee replacement, respectively.

Figure 3: Average cost and standardized cost for total-hip and -knee replacement among HVHC Participating Hospitals with active interventions for the reporting quarter and matched Comparators, by intervention quarter. Shaded quarters have low numbers of Participating Hospitals and are not considered stable.


Chronic Condition Annualized Costs (Method 2b)

For diabetes and CHF among Participating Hospitals, average annualized costs and average annualized costs in matched Comparators are shown in Figure 4. In intervention years 1 and 2, both actual and standardized modeled costs increased faster in the Participating Hospitals than Comparator Hospitals according to the model. The results of the models are summarized in Table 5 in Appendix D – Difference-in-Difference results for CHF and Diabetes.

Figure 4: Average annualized actual cost and standardized cost for diabetes and CHF patients assigned to PHNs of HVHC Participating Hospitals with active Intervention for the reporting quarter and matched Comparators, by year since start of intervention. Year 3 is shaded due to the low number of Participating Hospitals. The estimates are not considered stable.


Hip and Knee Surgical Rates in PHNs (Method 2b)

Total-joint replacement rates per 1,000 PHN assigned patients among Participating and Comparator Hospitals are shown in Figure 5. For total-hip replacement, change in rates was not significantly different between Intervention Hospitals and Comparator Hospitals. The model of total-knee replacement rates indicated that Participating Hospitals in intervention quarters 7 and 8 reduced knee replacement rates more than their Comparators. We interpret this finding with caution since it may be an artifact of the population-average estimate in the model and the ecologic nature of using surgeries in PHN assigned patients to the rate numerator for this analysis.

Figure 5: Rates of total-hip and –knee replacement for Participating Hospitals and Comparator Hospital groups by intervention quarter. Quarters 8 and 9 are shaded due to the low number of Intervention Hospitals in those quarters; they are not considered stable estimates.


Triple Aim Target 2: Health Outcomes GOAL: Improve health status measures (function, pain) for >50% of patients considering hip, knee, and spine surgery at one year; for complex patients with diabetes or CHF: reduce emergency department visit rates and hospitalizations by 10%.

Health Status Measures Figure 6 shows the proportion of patients with a PROMIS global physical health score improvement from baseline score over the length of the project. While the reported scores increased in the post-baseline questionnaires, the number of participants completing the last survey at 6+ months (n=384) was 27% of the first survey completed three months after baseline (n=1414). We observed improved health status measures (function, pain) in 51% of patients with diabetes or CHF and 60% of patients considering hip, knee, and spine surgery at 6+ months. As noted in the Results discussion, the patients participating in the intervention were under treatment and therefore likely to improve. Note: These kinds of patient-reported measures are essential to understanding the true effectiveness of the treatments offered. To date these have been in limited use in actual clinical practice.

ED admissions for chronic-condition PHN-assigned patients (Method 1) Table 2 details the year-over-year change in ED visit rate by PHN-assigned patients with diabetes or CHF. Within HVHC hospitals (Method 1), the overall admission rate per 1000 PHN assignees per year increased approximately 4%. A general upward trend in ED visit rate by quarter, that also shows seasonal oscillations, can be seen in Figure 7. ED visits increased by 4% and 3.7% for patients with diabetes or CHF, respectively; these outcomes fell short of our 10% reduction target. Note that this cohort represents all hospitals for our participating member systems, not just the Participating Hospitals or Intervention Hospitals.

Figure 6: PROMIS Global Physical Health Score Improvement (Monitoring Measure SS118d) by chronic and surgical conditions

Table 2: Year-over-year comparison of ED visit rate by PHN assigned patients with diabetes or CHF


In-patient admissions for chronic-condition PHN assigned patients (Method 1)

Table 3 shows year-over-year change in in-patient admissions among PHN assigned patients with CHF or diabetes. Inpatient hospital admissions decreased by 3.8% and 2.0% for patients with diabetes or CHF, respectively; these outcomes fell short of our 10% reduction target. Note: this cohort represents all hospitals for our member systems, not just the Participating Hospitals or Intervention Hospitals.

Table 3: Year-over-year change in in-patient admissions among PHN assigned patients with CHF or

diabetes.

Figure 7: Non-Admitting ED Visit Rates (Total attributed population, diabetes and CHF) from HVHC Monitoring Measures of beneficiaries assigned to HVHC member PHNs


Triple Aim Target 3: Quality of Care GOALS: >50% eligible patients referred to SDM, >50% of referred patients participate in SDM

Member-reported Enrollment and Eligibility Counts

For a period in late 2013 through 2014, HVHC member systems reported the number of eligible patients and enrolled patients for the SDM interventions their Participating Hospitals were providing. Note: this does not include other PE interventions as the focus was on SDM for this target. The average number of patients enrolled per month compared to the number of eligible patients screened is shown by condition and member in Figure 8. The ratio of enrolled patients to eligible patients was highly variable within each condition (as expected). Each institution has multiple facilities and different rates of uptake for implementation, e.g., all care and processes are local and therefore each site had unique challenges and requirements towards implementation. The summary ratios of enrolled to eligible patients by condition are as follows: CHF: 0.78, diabetes: 0.55, hip: 0.78 and knee: 0.78. Summaries are reported as ratios since the reported number of enrolled patients sometimes exceeded the reported number of eligible patients for some member-condition combinations (see Discussion Section). While there are likely data consistency and quality issues inherent in manually tallied data, these ratios are still well above the goal of 50% of eligible patients enrolled.


Additional Analyses Additional analyses (unrelated to the enrollment targets) about the population follow:

Interventions provided

In aggregate, the project delivered over 204,000 patient engagement interventions and over 36,000 shared decision making interventions to 144,880 patients over the nearly 2.5-year intervention period. As can be seen in the cumulative enrollment charts, by condition (Figure 9) and category of intervention (Figure 10), about 90% of interventions were delivered in the 2nd and 3rd years of the intervention as

Figure 8: Eligible and enrolled patients for SDM intervention by condition and HVHC member. Data are normalized to monthly average counts and reflect member-reported data provided in late 2013 - 2014. Hip = Hip Osteoarthritis; Knee = Knee Osteoarthritis; CHF = Congestive Heart Failure;


expected due to varied implementation launch dates.

Figure 9: Cumulative enrollment in HVHC interventions by condition. CHF = Congestive Heart Failure;

Figure 10: Cumulative enrollment in HVHC interventions by condition. SDM = Shared Decision Making Interventions PE = Patient Engagement Interventions


Age of intervention recipients

We estimated the proportion of patients receiving interventions who were Medicare beneficiaries using age-at-enrollment as a proxy for Medicare eligibility. Among patients receiving an intervention, 55% were age 65 or older. With the exception of diabetes, a majority of patients in each condition were 65 or older. However, there was substantial heterogeneity by Participating Hospital (Table 4).

Table 4: Enrolled patients and percentage age 65 and older by intervention condition CHF = Congestive Heart Failure; Hip OA = Hip Osteoarthritis; Knee OA = Knee Osteoarthritis Member = HVHC member health care system.

Enrollment Relative to PHN Cohorts In order to evaluate the penetration of interventions into target populations and compare the PHN attributions to actual events, we evaluated the ratio of intervention-enrolled patients over the age of 65 –our proxy for Medicare eligibility – to the number of PHN-assigned patients for chronic conditions and to the number surgeries in patients assigned to a PHN for surgical conditions. We used average monthly counts of events and divided the PHN population by 12 to get comparable counts. Generally, there were, as expected, many more intervention-enrolled patients than surgical events within a member’s PHNs and the number of patients with a chronic condition within a PHN exceeded the number of patients with a condition receiving interventions. However, as can be seen in Figure 11, there was substantial heterogeneity between members. We interpret these findings with caution given the limits of PHN attribution in assigning beneficiaries to hospitals and the short time frame from funding to attempted implementation.


Figure 11: Comparisons of intervention-enrolled patients age 65+ to number of surgeries among PHN assigned beneficiaries or number of PHN assigned beneficiaries with condition. All counts normalized to monthly values. PHN = Physician Hospital Network; HVHC Member = Health System participating in the CMMI work Hip = Hip Osteoarthritis; Knee = Knee Osteoarthritis; CHF = Congestive Heart Failure


Discussion Evidence targets were reached Progress against our stated goals follows (NOTE: “cost” refers to cost to Medicare and payers: claims reimbursement):

• Reduce cost: We observed a decrease in 90-day episode actual costs of 3.1% and 4.2% for patients with hip and knee surgeries, respectively (or 5.7% and 5.9% using standardized costs), falling short of our 5% target for actual costs savings. We observed a decrease in annual episode actual costs by .5% and 1.8% for CHF and diabetes patients, respectively (or .6% and 1.7% using standardized costs), falling short of our 2% target for actual cost savings. Total actual cost savings for Medicare was $20.9M (or $27.5M in standardized costs), falling short of our prorated goal of $39M for 2013-2014. If we assume similar trends for two more quarters, the total savings estimate would be $25.0M and $33M for actual and standardized costs, respectively, and still falling short of the actual $48.8M target.

• Improve health: We observed improved health status measures (function, pain) for 51% of patients with diabetes or CHF and 60% of patients considering hip, knee, and spine surgery at 6+ months, meeting our goal of >50%. ED visits increased by 4% and 3.7% and inpatient hospital admissions decreased by 3.8% and 2.0% for patients with diabetes or CHF, respectively; these outcomes fell short of our 10% reduction target for ED visits and hospitalizations.

• Improve care: The summary ratios of member-submitted enrolled-to-eligible patient counts by condition were as follows: CHF: 0.78, diabetes: 0.55, hip: 0.78 and knee: 0.78. While there are likely data consistency and quality issues inherent in manually tallied data, these ratios are still well above the goal of 50% of eligible patients enrolled.

Potential reasons for these results are detailed in Section 3.

Limitations of Evaluation Limitations of PHNs

An important design decision in the internal evaluation of this project was to use an ecologic hospital-level analysis rather than an individual patient-based analysis. The reason for this decision was to prevent selection bias due to non-random recruitment of participants into the intervention program. Evidence supporting the decision came from the qualitative investigation of the implementation of the CHF Implantable Cardiac Defibrillator decision aid, where "cherry picking" of patients for the intervention was a theme in stakeholder interviews (Matlock D, Jones J. HVHC ICD Decision Aid Webinars. https://www.youtube.com/watch?v=NpVxKcFtuII&list=PLBJktjqodu74xuDBUBwD5ToBjXtzQ4gRW&index=1. Accessed 2016-08-30). PHNs also provided denominator populations for evaluating non-episode based events and costs in populations where patients were likely to receive services from many different providers.

Additionally, it was hypothesized that clinics participating in the interventions would (1) be more likely to observe other best practices, and (2) interventions that were part of the project would “spill over” and be taken up by clinicians who were not part of the intervention. These potential effects would be likely to

https://www.youtube.com/watch?v=NpVxKcFtuII&list=PLBJktjqodu74xuDBUBwD5ToBjXtzQ4gRW&index=1

https://www.youtube.com/watch?v=NpVxKcFtuII&list=PLBJktjqodu74xuDBUBwD5ToBjXtzQ4gRW&index=1


improve outcomes within the entire patient population, not just patients assigned to the intervention. With more than just the exposed population "exposed," the measured effect of exposure would be expected to be biased downward in a patient-level analysis.

While using ecologic-level analysis prevented selection bias, it provided opportunities for both ecologic fallacy and misclassification. Patients assigned to a PHN with an intervention condition may not have been exposed to an intervention or even had services provided by their PHN-assigned hospital. Similarly, patients who received an intervention at an Intervention Hospital may have been assigned to a non-Intervention Hospital.

The PHN attribution method for assigning beneficiaries to hospitals provides a method to create a population of beneficiaries who would probably go to their PHN assigned hospital if they were admitted. The original work defining the PHN algorithm found that about two-thirds of medical admissions were at the beneficiary’s PHN-assigned hospital (J. P. W. Bynum, et al. "Assigning Ambulatory Patients and Their Physicians to Hospitals: A Method for Obtaining Population-Based Provider Performance Measurements." Health Services Research. 2007:42(1): 45-62).

We found that the proportion of knee arthroplasties in patients assigned to HVHC PHNs that occurred at the patient’s assigned PHN was 47% and among patients getting a knee arthroplasty at HVHC PHNs, the proportion assigned to the hospital where they received the surgery was 51%.

As a further investigation into how PHN attribution could affect our measures, we investigated the proportion of services within the geographic catchment area of one member's delivery system that were provided outside of that system. The member-provided proportion of each type of service examined was highly variable and dependent on the condition (surgical or chronic) as well as types of services provided by the member. At an even more detailed level, we observed issues in attribution for a newly structured referral center at a specialty orthopedic hospital that wasn't represented in the higher level analysis.

Enrollment data

As noted in Results Table 2, depending on the condition, about a quarter to a half of all interventions in HVHC were provided to patients under age 65, our proxy for not being a Medicare beneficiary. Age as a proxy measure for Medicare beneficiary status is not perfect. Approximately 17% of Medicare patients are under age 65 and have permanent disabilities (An Overview of Medicare. The Henry J. Kaiser Family Foundation. http://kff.org/medicare/issue-brief/an-overview-of-medicare Accessed 2016-08-29.). Additionally, 31% of Medicare beneficiaries are enrolled in Medicare Advantage plans (Medicare Advantage Enrollees as a Percent of Total Medicare Population. http://kff.org/medicare/state-indicator/enrollees-as-a-of-total-medicare-population/. Accessed 2016-09-06.). While non-Medicare patients under 65 and Medicare advantage beneficiaries were included in the CMMI Award for receiving interventions, their results were not included in the evaluation as stated by the terms of the CMMI award. Assuming a similar effect in younger and older intervention participants, increasing the number of observations available would increase the precision of the effect estimates.

We were also limited in our ability to quantify the number of SDM interventions provided at a given facility and the number of eligible patients for those interventions. Our quantification of intervention exposures was limited to member-reported counts for a subset of the project period and patient-level enrollment of the SDM intervention provided at the member level rather than individual Participating Hospitals. We noted certain anomalies in member-reported patient counts; for example, some months the number of enrolled patients was higher than the number of eligible patients, which members attributed to timing

http://kff.org/medicare/issue-brief/an-overview-of-medicare

http://kff.org/medicare/state-indicator/enrollees-as-a-of-total-medicare-population/

http://kff.org/medicare/state-indicator/enrollees-as-a-of-total-medicare-population/


issues (e.g., a given patient might have been reported as eligible one month, but wasn’t enrolled until the following month). These constraints were due to the manually-tallied patient counts submitted by members and the heterogeneous ways interventions were administered within member facilities.

Heterogeneity in amount of intervention provided

As noted in Section 2, Triple Aim 3, there was substantial heterogeneity in the ratio of patients eligible for SDM (proxy: PHN assigned beneficiaries or number of surgeries) to interventions provided. Assuming an effective intervention, the effect of the intervention should be a function of the proportion of eligible patients exposed to that intervention. While we could not measure the number of eligible patients directly, we decided the most robust available estimate of the penetration of the intervention into the eligible population was the ratio of SDM interventions to PHN assigned chronic condition patients or surgeries performed at the hospital level. For chronic conditions and event rates, this measure uses PHNs and so shares the limitations of PHNs.

Follow-up period

This project was funded for three years and the number of interventions provided increased substantially starting in year 2. About 42% of interventions were provided in year 3.

Patients enrolled in interventions towards the end of the project had limited time to contribute change results, particularly if there was a lag between intervention application and change in outcomes. For instance, a diabetes patient enrolled in project-quarter 11 would contribute less than two quarters of results, if the effect of the intervention took more than two quarters to become measurable, change in the patient's outcome would not contribute to the measured results.

Follow-up time is particularly important for the year-over-year comparisons. Different intervention start-times made for different follow-up times at different Intervention Hospitals. Heterogeneity in intervention start-times and lengths likely had an effect on the amount of change that was measurable in the year-over-year comparisons.

Limiting calendar year analyses to 2013-2014

Additionally, we excluded the 2015 partial year (Jan-Jun) from our calendar year analyses for two reasons: (1) variability observed in the analysis by quarter; and (2) this partial year included a lower number of enrollees both due to the half year time period and due to enrollment dropping off in anticipation of grant funding coming to an end (the problem with “short term funding” to solve a major, life threatening, health issue).

Limits of comparison model

There are several limitations to inaccuracies in analysis inherent to real-world observational studies. Our choice of a difference-in-difference model to quantify the results of the project had limitations. The model compares changes in outcomes in Intervention Hospitals with changes in matched Comparator Hospitals over time. We had no way of knowing if the Comparator Hospitals were also doing PE (Patient Engagement) or SDM (Shared Decision Making) interventions, or other interventions, so our conclusions from the models cannot estimate the effect of the interventions, but rather compare Intervention


Hospitals to Non-Intervention Hospitals. Anecdotally, we know of at least one HVHC member that provided different SDM interventions for arthroplasty in a hospital that was not part of the HVHC intervention.

Additionally, hospitals that had implemented the use of knowledge-based tools to aid in difficult medical decisions prior to the CMMI work and were already performing at a high level would not be capable of large amounts of change. When the evaluation model estimates change at the aggregate level of all Intervention Hospitals, as our evaluation was designed, these hospitals dilute the amount of population-level change observed. Examples of heterogeneity in outcomes among members have been documented as part of investigations related to this project. For surgical conditions, Weeks et al. (Weeks WB, Schoellkopf WJ, Sorensen L, Masica AL, Nesse RE, Weinstein JN. High Value Healthcare Collaborative: Episodes of care for hip and knee replacement surgery. Journal of Arthroplasty, 2016. (accepted).) found several fold variation in hip and knee replacement rates between HVHC members. For chronic conditions, Herndon et al. (in preparation) also found an over two-fold variation in the rate of ambulatory visits in medically-homed diabetes patients among HVHC members.

While our difference-in-difference analysis models adjusted for several observable potential confounders, the possibility that we failed to adjust for one or more patient-level confounders that biased results are likely.

Additionally, while we did adjust for time, confounding of results by the differential environmental changes occurring across HVHC members and Comparators is impossible to include in any of our models. Measures of the effects of the SDM interventions must be evaluated in the face of health reform in various stages occurring across all organizations and national cost-curve changes that make it difficult to discern the impact of this HVHC effort from those of others and national reforms.

Lastly, given the complex implementation involving clinics and hospitals within systems, a statistical power analysis at the design phase was not feasible. While our models may not have estimated statistically significant differences, for some conditions, HVHC costs were consistently below Comparators.

Health Status Measures

Patient reported measures are essential for us to better understand effectiveness, outcomes of care, in contrast to simple total cost of care measures. These measures are important since they help us determine if we are actually improving health as determined by the patient and society. We used improvement over baseline score in the PROMIS Global Physical Health Score to measure change in self-reported health status. While the reported scores increased in the post-baseline questionnaires, the number of participants completing the last survey at 6+ months (n=384) was 27% of the first survey completed three months after baseline (n=1414). While follow-up bias is a concern, we did not distinguish whether non-response was due to late enrollment (so the follow-up survey would be after the project period) or loss-to-follow-up. An additional caveat to interpretation is that since the respondents were receiving care for their conditions, we would expect some improvement in scores.

Section 3: Challenges and Lessons Learned

Implementing any intervention across so many disparate systems for multiple conditions is challenging, let alone while simultaneously being engaged in the new rules and regulations following the ACA. While overall, we did not meet our planned targets, many lessons can be learned from our experience and we have built the foundation for continued improvement over time for the PE and SDM interventions as well


as our ability tom monitor the outcomes. While the funding period was adequate to begin implementation improvements, the time necessary to identify and address barriers and challenges, as well as sustain and evaluate the program was clearly insufficient. We have, for the first time, created a national collaboration across the entire country in multiple disciplines with tens of thousands of physicians and millions of patients. The CMMI project was HVHC’s first major challenge has and it has set the stage for future success. Additional monitoring time, resources and the institutional restructuring is necessary to standardize processes across health systems and overcome barriers. The challenges and lessons learned summarized below highlight the complexity of this program.

Health Coaching: sustaining and enhancing health coaching in practice Sites identified ways to sustain and enhance health coaching so its practice can successfully support a new culture of truly patient-centered care and collaboration, wherein patients receive only what they want and need when well informed.

Lessons learned include:

● Leadership from the top of the organization is critical ● Recruitment of the right person to fill the change leadership position can be a lengthy endeavor. ● Cross covering work allows more availability to support patients and providers and less risk from

attrition. ● Integrating health coaches into the care team allows them to forge strong relationships with the

care team as well as with patients ● Imbed the health coach role within the clinical team so that the team can define the role together

to become part of routine care ● Coaches should “live” within the clinic space and have designated space to see patients. ● Health coaching is resource and time dependent and hard to scale ● Make protected time for the stakeholders involved to develop a process and workflow ● Health coaches without clinical backgrounds found it difficult to deliver the intervention to patients

with complex medical conditions. ● Mitigation strategies included exploring the hiring of an RN health coach, delivering a training video

with health coaches on CHF, and identifying a clinician lead to support health coaches and answer clinical questions.

● Health coaches supporting patients with chronic conditions benefit from additional training in motivational interviewing, wellness coaching and disease-specific education

Identifying Patients for Interventions Identifying all patients within the organization who are eligible for education and treatment decision support was challenging. Additionally, identification of patients at the right point in time added additional complexity. This barrier was significant for all members and required multiple levels of mitigation. Lessons learned as reported by HVHC Members include:

● Develop an organizational process flow map early in the implementation process to help identify areas to target for patient eligibility screening

● Utilize multiple screening methods to ensure all eligible patients are identified ● Increase provider education and awareness of the program ● Incorporate SDM decision aid and survey administration as part of the physician order process ● Routinely reinforce the importance of SDM as an integral component of patient engagement


● Identify patients early in the process (further upstream, more often effective if done with the primary care team before reaching the specialist) in order to initiate the process at a time that aligns with the patient's’ needs.

● Review clinical calendars with the care team increases efficiency ● Ensure that responsibility for patient identification does not lie solely with the providers ● Invest in automation of the identification of appropriate patients ● Develop a “Hot Button” or reminder question in the ordering system within the orders to queue

providers to refer patients to the program Electronic Health Records

Though integrating an SDM process into the electronic health record is challenging, members, even at the time of this report, are beginning to see success. Several options for sustainability of capturing patient-generated health data (PGHD) and providing access to DAs have seen early success, including integration of SDM questionnaires, referrals and health coach notes into the electronic health record. Examples from HVHC Members include:

● Many healthcare organizations still have multiple electronic medical record systems which create significant barriers for communication, data sharing, and access to tools and patient information.

● Integrate SDM questionnaires into the EMR (Epic) by utilizing questionnaire series functionality. This functionality allows questionnaires to be accessible via MyChart and Welcome (two patient-facing functions of Epic) prior to clinical visits and repeated at desired follow-up intervals.

● Integrating these questionnaires into the Epic system provides immediate access of PGHD to clinical staff within the patients’ EMR and availability at the point of care.

● Creating an EHR health coach documentation note and a task team improved integration of workflow, patient recruitment and increased program visibility.

● Create a “Health Coach” appointment in the EHR ● Set up the EMR system to collect the quality and monitoring data needed early in the process

Clinician Buy-In / Clinical Champions

Clinician buy-in and identifying clinical champions are critical for success. Examples include:

● Understanding the impact of rival activities (e.g., implementing a new EHR) and physician incentive payments, and clinic staffing issues is important

● Work with physicians who fully support the SDM program for their patient population ● Identify multiple clinical champions early in the process, engage them in planning and

implementation and monitoring progress ● Involve many groups and stakeholders across the system (data analysts, physicians, health

coaches, nurses, pharmacists, etc.) and bring these groups together regularly to plan, implement and monitor progress

● Take advantage of mid-course corrections, if a clinician is not able to champion the effort, work to identify a replacement.

Section 4: Planned Activities Funded by the HVHC members, we intend to examine the ongoing effects of health care innovations implemented through this CMMI award. This new study will allow for an extended period of evaluation and monitoring of each program utilizing CMS as well as member-submitted data. The evaluation model will


continue utilizing approved monitoring measures for each project. This evaluation will provide the analytic evidence to help determine the long-term impact and success of sustainability.

Section 5: Stories from the Field The following real world stories are provided by our participating members as a way to highlight the value of the intervention; positive impacts on groups of participants, individual participants, or health care workers; difficulties overcome in the field; and reactions of patients, family members, or staff members.

Decision support tools (DAs, Ottawa Personal Decision Guide etc.) BAYLOR: Decision Support Tools - Remote Care Managers are trained to use the Ottawa Guide and Mayo Clinic SDM tools. Free access (no cost) and ease of use of the OPDG and Mayo Clinic tools have facilitated adoption.

BETH ISRAEL DEACONESS MEDICAL CENTER (BIDMC): Decision Support Tools - Although we will not be sustaining health coaching as part of the participating practices, administrative staff working with clinical champions will continue to identify patients who might benefit from DAs and make those available in advance of the first visit with the surgeon. We will continue to use the Health Dialog DAs as long as they are available, and will work to identify a set of resources that can be available through the patient portal or in hard copy. Additionally, the electrophysiologist / physician champion at BIDMC will continue to offer the ICD decision aid DVD, paper-based tools, and decision coaching will continue to those heart failure patients who qualify for primary prevention ICD.

DARTMOUTH-HITCHCOCK (DART): Decision Support Tools - We will continue to use the decision support tools as they are embedded into practice. We also have embedded the surveys and data collections. There is buy-in from both teams of providers.

EASTERN MAINE HEALTH SYSTEM (EMHS): Decision Support Tools - Health Care Coordinators will have access to Health Dialog decision aid tools to use with the appropriate patients in their panel. They also intend to continue using the Ottawa Personal Decision Guide to support SDM.

INTERMOUNTAIN HEALTHCARE (IHC): Decision Support Tools - Our delivery system is in the process of developing a strategic plan for continued use of SDM based on what we’ve learned. This may lead to a reduced number of decision aids being used and others being added (e.g., hospice). Ongoing use of SDM is being “baked into” our routine work and included in our benefit design packages for commercial insurers such that sustainability of this work is insured.

MAINEHEALTH (MAINE): Decision Support Tools - CHF: Heart Failure nurses participate in active planning with inpatients. Methods for success included the addition of health coaches and educators in incorporating the use of the Living with Heart Failure decision aid and integrating health coach follow up into patient workflows. Another source of success was from our continued effort and discussion with the MaineHealth Heart Failure workgroup team as well as with HVHC staff to identify areas where we might align our work at a system level to extend the reach of the intervention. Finally, multiple discussions with EP cardiologists lead to a very successful implementation of the ICD decision aid with select patients.

MAYO CLINIC (MAYO): Decision Support Tools - Interest in exploring development of institutional ICD decision aid was garnered due to this work.


NORTH SHORE LIJ (NSLIJ): Decision Support Tools - Orthopedic practice sites where CMMI efforts occurred will continue to distribute DAs to patients considering hip, knee, or spine surgery. Practice administration have championed the continued use of the DAs and clinical staff (PAs, RNs) and were supportive of transitioning to using the DAs with patients. DAs will most likely be distributed to patients by clinical support staff and reviewed with a PA or RN. Internal medicine will continue to distribute DAs to patients with complex chronic conditions. Practice administration championed the continued use of the DAs and clinical staff is supportive of transitioning to using the DAs with patients. DAs will most likely be distributed to patients by clinical support staff and reviewed with a PA or RN.

PROVIDENCE (PROVID): Decision Support Tools - Hip & Knee: Providence has funded a $25K pilot project to mail Hip & Knee booklets to members of its health plan who are referred to a group of orthopedic surgeons. With little success at engaging front office staff in initiating SDM prior to an appointment, we decided to intervene at the point of referral. Patients will be called to verify their diagnosis of osteoarthritis, asked about their current needs for decision support using the SURE tool, and then contacted again after 2-weeks to ascertain the utility of the booklets and their current decision status.

PROVIDENCE (PROVID): Decision Support Tools - Spine: Shared decision-making continues to be a vital part of our Spine Continuum of Care project, nurses hand-select patients to receive materials specific to their diagnosis. Over the course of the CMMI grant eight new primary care clinics and several additional neurosurgeons were added to the project, and six more clinics are joining soon.

VIRGINIA MASON MEDICAL CENTER (VMMC): Decision Support Tools - DAs are distributed via appointment scheduling. Organization support to integrate DAs with our scheduling system and patient portal were vital to our success. Organizational goals and local state policies to continuing giving DAs also provided extra incentive to design a sustainable process.

UNIVERSITY OF CALIFORNIA AT LOS ANGELES (UCLA): Decision Support Tools – We have two ongoing projects that involve the use of DAs (patients with herniated discs, and patients with prediabetes). Each of these projects is an early phase, but we are using lessons learned from the CMMI project to improve our patient recruitment, patient retention, and clinician engagement strategies.

Training and education BAYLOR: Training & Education - The Health Coach Training offered by HVHC during the CMMI project is now being offered on site by the Education Manager in the Office of Patient Experience. Current courses include a four (4) hour Health Coach Training course and a one (1) hour Introduction to SDM course offered through the Patient Care Academy. Future development includes courses specific to providers and integration of SDM and patient engagement into the provider communication education series.

DARTMOUTH-HITCHCOCK (DART): Training and Education - Through this project we were able to train many Dartmouth-Hitchcock care coordinators on the use of the OPDG and they are now utilizing this within their practice.

NORTH SHORE LIJ (NSLIJ): Training & Education - Interested PAs and RNs from the orthopedic practice site will be trained by our health coaches on the importance of SDM, health coaching skills and techniques, and how to use decision support tools. Training will occur for physicians and staff who have found the program to be useful to their practice and improving patients' satisfaction and decision making quality. Clinician buy in at implementation sites contributed to this success. Our orthopedic staffing model also


contributed to this success because PAs and RNs have greater access to the patients and are better suited to implement SDM processes and deliver clinical information.

UNIVERSITY OF IOWA (UIOWA): Training and Education - To some extent, we will attempt to use the train-the-trainer model (promoted during the training of the CMMI Health Coaching sessions) to educate those patients who are involved with this intervention. We will continue to draw upon and spread many of the concepts that were introduced through this HCIA intervention (e.g., motivational interviewing, helping patients with decisional conflict) as we recognize the importance of these issues when dealing with this particular patient population.

VIRGINIA MASON MEDICAL CENTER (VMMC): Training & Education -We have developed standard work for these processes so that staff can be trained and easily pick up the work. We report our progress to the organization quarterly to help communicate the efforts. These two strategies have made it easier to sustain this work on a broad level.

UNIVERSITY OF CALIFORNIA AT LOS ANGELES (UCLA): Training & Education - To some extent, we will attempt to use the train-the-trainer model (promoted during the training of the CMMI Health Coaching sessions) to educate those patients who are involved with this intervention. Identifying patients for interventions

UNIVERSITY OF IOWA(UIOWA): Identifying Patients - In the spine setting, we are in the planning stages of an intervention that shares similarities to this CMMI intervention. Using an online decision-aid, we will identify herniated disc patients and attempt to engage these patients with education and SDM techniques to help them consider their preferences and align their care decision with those preferences, while also incorporating physician input.

VIRGINIA MASON MEDICAL CENTER (VMMC): Identifying Patients - We have a scheduling process in which patients are identified for the interventions. Patients are screened for hip, knee and spine issues in each call center which initiates a PRM questionnaire to be distributed to patients. We also developed rules in our EHR to automatically identify patients for the interventions that will be improved as we learn of the long term impact on patient outcomes. By removing a human operator from this process, we no longer have to worry about ongoing training and sustainment with expected staff turnover over time. Organization support to integrate these processes with our EHR played a big role to our success in this work.

Health Coaches

BETH ISRAEL DEACONESS MEDICAL CENTER (BIDMC): Diabetes - The diabetes educators will continue to incorporate health coaching into their care of BIDMC patients seen at Joslin Diabetes Center. We expect to use coach training on demand to make this training available to educators who did not participate during the CMMI award period. Because we incorporated health coaching into an existing service - diabetes education - our ability to sustain this activity is much greater both in terms of personnel and in terms of clinical champions. Similarly, because the diabetes educators have acknowledged value from coach training they are able to make a case for continued training and education.

DARTMOUTH-HITCHCOCK(DART): Health Coaches - We showed that patients and clinicians found value in health coaching performed by non-clinical health coaches. Two of our coaches are now supported by the Regional Primary Care Center to continue their work with patients with chronic conditions. The Stanford Better Choices, Better Health Self-Management groups have been particularly well received with several


groups being offered this summer. Our Health coaches, in their new role, plan to become trainers for the Stanford Self-Management groups so they can train facilitators throughout D-H.

EASTERN MAINE HEALTH SYSTEM (EMHS): Health Coaches - We utilized RN Care Coordinators who were already working for EMHS' Accountable Care Organization, Beacon Health, as Health Coaches for the primary care practices. These RN care coordinators are now equipped to provide SDM techniques with their patient panels.

NORTH SHORE LIJ (NSLIJ): Health Coaches - One of our health coaches will be transitioning into a newly created position where she will be applying her skills in SDM and patient engagement to work with NSLIJ's Bundled Payments Care Improvement Initiative (BPCI) for orthopedic conditions. As a Program Engagement Specialist, she will be working with our care management organization to engage team members, patients, and physicians to provide coordinated care, improve care management programs, and promote the integration care management teams in our hospitals. The role will focus on hip and knee surgical candidates designated to be part of the BPCI. In this role she will work toward improving patient activation by identifying and incorporating person-specific barriers to high quality health outcomes in development with multi-modal patient engagement training programs. We successfully achieved transitioning the health coach role into a patient engagement position through outreach with department chairs and program leads who had project missions that aligned with patient engagement activities. We met with leadership to advocate for these initiatives to explore how aspects of our SDM program could bolster their efforts. We were able to leverage the unique skills of our health coach to successfully match staffing needs for the BPCI initiative.

PROVIDENCE (PROVID): Health Coaches - We will continue reinforcing and expanding the health coach pool through training in the diabetes education area, and others as desired. It is uncertain at this point whether the diabetes educators will be the primary coaches, or whether we will add coaches to assist the educators. We are experimenting with both options. We will build on the lessons learned from this project as we continue to develop a more patient-centered model of diabetes coaching and support for self-management. This project provided an excellent foundation on which to continue to test other innovative ways to provide coaching support. We are now testing a mobile platform using similar concepts, and also allows secure texting and video chat. VIRGINIA MASON MEDICAL CENTER(VMMC): Health Coaches - We designed a phone-based model for health coaching that is initiated by the patient completing the decision aid. Nurses have an hour of protected time each afternoon to perform calls and it has worked well and is actively being spread to other areas. Our lean management system, the Virginia Mason Production System, played a big role in creating this process. Local state initiatives like the Dr. Robert Bree Collaborative added extra incentive for us to implement a strong model for health coaching. We have also integrated health coaching into our care management model for heart failure and diabetes. It is now the standard work for our nurse care managers to health coach patients as part of care management visits. Aligning health coaching with our existing processes was the biggest factor to success and sustainability of these patient engagement strategies. Clinician buy-in / clinical champions

DARTMOUTH-HITCHCOCK (DART): Clinician buy-in / clinical champions - Clinicians value this service and several championed for continuation of health coaching which led to the addition of two health coaches in the Regional Primary Care Center’s budget.


DENVER HEALTH (DENVER): There seemed to be considerable clinician buy-in at the microsystem level within the orthopedic department. The staff, including the physicians and nurses, saw the benefit in the program, not only to free up their time from the reduction of inappropriate patients, but also with having better informed patients before the consult. During a short time when we did not have a health coach, the orthopedic staff was extremely accommodating and advocated for the program to continue.

NORTH SHORE LIJ (NSLIJ): Clinician buy in - Although our SDM program will not be sustained in the orthopedic practice in full, we received strong support from one of our clinicians. He has communicated the importance of SDM to his colleagues and staff and he will continue to implement aspects of the program, including having his support staff trained by our health coaches. The clinician was extremely supportive of the program, and although his efforts could not sustain the program in the orthopedic practice in line with our funding end date, he will continue to champion SDM and deliver program services to his patients.

MAINEHEALTH (MAINE): Under the guidance of John Pier, physician champion of SDM, the Spine Center's SDM process has flourished. The additional resources provided by the CMMI HVHC funding allowed them to grow their SDM program and realize the importance of the collecting patient reported outcomes data. In effort to sustain the work funded by CMMI/HVHC, the spine center created a position for a quality data coordinator (QDC) and hired Melinda Beyer, health coach for HVHC related work. Mindy will now be responsible for gathering and evaluating clinical data from the organization; and monitoring and managing core measures and other quality improvement care processes, interpreting performance and sharing outcomes with Medical Staff, Patient Care Services staff and leaders to promote compliance with outcome expectations. She will also utilize her experience to continue the SDM training for staff at MaineHealth.

VIRGINIA MASON MEDICAL CENTER (VMMC): Clinician buy-in/clinical champions - We have strong leadership in our spine and orthopaedic sections who embraced this work. Our strategic plan places the patient at the top of our strategic pyramid which also helped to gain buy-in from our clinical teams since this work was the right thing to do for the patient. We assigned clinical champions to oversee the design of this work in the context of clinical care for these patients. That combined with strong leadership were the recipe for success.

UNIVERSITY OF CALIFORNIA AT LOS ANGELES (UCLA): Clinician buy-in - We have been involving clinicians involved in our MyMEDS program for the past year. Throughout this process, physicians involved have become increasingly familiar with SDM and its potential to have a positive effect on patient experiences/satisfaction and health outcomes.


Integration of information and tools into the electronic medical record

EASTERN MAINE HEALTH SYSTEM (EMHS): Integration of Information and Tools into the EMR - The HOOS and KOOS have both been built in the EMR and are now available for our clinicians to utilize.

VIRGINIA MASON MEDICAL CENTER (VMMC): Integration of Information and Tools into the EMR - We integrated PGHD and DAs with our scheduling system, patient portal and EHR which was the biggest factor to sustaining these interventions within our organization. We have an organizational goal focused on integrated systems; having this support during the award was crucial to executing a sustainable solution for patient engagement across Virginia Mason.

DARTMOUTH-HITCHCOCK (DART): Integration of information and tools into the electronic medical record - We created a health coach encounter that does not require a clinician co-sign within the EMR which is being used not only by our health coaches but also by the Live Well Work Well health coaches and those trained in facilitation of advance care planning conversations. We also created a template health coach note and action plan for the EMR.

INTERMOUNTAIN HEALTHCARE (IHC): Integration of information and tools into the electronic medical record - The PHQ-9 is embedded into our medical records; we also have clinical decision support around a care process model for pre-diabetes and diabetes care. Organizational priorities in diabetes and CHF have led to Board goals and funded activities that target these patients. For CHF, we have developed appropriate use guidelines and care process models that are embedded into the EHR.

VIRGINIA MASON MEDICAL CENTER (VMMC): We integrated PGHD and DAs with our scheduling system, patient portal and electronic health record (EHR) which was the biggest factor to sustaining these interventions within our organization. We have an organizational goal focused on integrated systems; having this support during the award was crucial to executing a sustainable solution for patient engagement. Patient-Generated Health Data (PGHD)

BAYLOR: Patient Generated Health Data - Select providers have begun piloting OBERD to collect PROMs. The OBERD project was launched during the CMMI project, so providers who participated in the OBERD pilot were not part of the CMMI project. Providers are able to participate in OBERD for any orthopedic condition. It is not restricted to hip and knee osteoarthritis. OBERD is partially integrated in to the EHR scheduling software.

BETH ISRAEL DEACONESS MEDICAL CENTER (BIDMC): is currently piloting a tool for collecting patient-generated health data electronically and displaying them in the EHR. This tool is being piloted in spine with the goal of implemented more broadly in other clinical areas. This tool will allow patients to complete the instruments in advance of a visit. Scores will be displayed for clinician to track changes over time.


INTERMOUNTAIN HEALTHCARE(IHC): Patient Generated Health Data - We are working to harmonize measures across the system and create a platform for collecting these data securely through the patient portal. No final decision has yet been made on where this information will be stored in the medical record; Intermountain is in the midst of an EHR conversion from a homegrown system to a vendor-based system.

VIRGINIA MASON MEDICAL CENTER(VMMC): Patient-Generated Health Data - Patient-generated health data (PGHD) was also integrated in our EHR and patient portal so that all patients receive requests prior to their appointments. This automated and mistake-proofed the process for distributing PGHD. Organizational and IT support to perform this integration was a key component to sustainability. Data collection (e.g. gathering and reporting clinical and administrative data)

VIRGINIA MASON MEDICAL CENTER (VMMC): Data Collection - We organized data marts to structure clinical and administrative data for the purposes of reporting. Analytic and IT support was the biggest factor to sustaining data collection. Our HVHC participation is led through our analytics department which makes the data collection more feasible.

Section 6: Pulse Check

No additional information Section 7: Final Report Close-out

Note: categories below are provided by CMMI for required reporting

Major activities that occurred during the three-year award period. Hired 36.7 FTE health coaches. Over the course of the award members hired and on-boarded 36.7 health coach FTE’s. As described in the Health Coach Position Description developed for the project, the overall goal of the Health Coaching shared decision making intervention is to improve the care received and the health of patients receiving services. To reach this goal the Health Coach were responsible for: employing high quality, consistent health coaching strategies to support patients and families; ensuring maximal participation of the appropriate patients and families in the SDM program; effectively engaging patients and families in shared decision making; consistently implementing shared decision making processes and tools; ensuring accurate and complete collection and recording of quality measures, patient-reported data and process measures. Trained and certified 304 health coaches through a distance learning training program. Three forty-eight (348) health coaches enrolled in the course (auditors were accepted) and 304 received certification since project inception. This total far exceeds our target of 60 health coaches. This accomplishment is due to the success of the training program and the push to spread and sustain the effort. Students from a variety of backgrounds enrolled in the course: diabetic educators, primary care staff, physical and occupational therapists, social workers, and case managers. Several sites continue to provide in-house training to a variety of audiences, including physicians, nurses, care coordinators and medical assistants. A manuscript was prepared and submitted to Implementation Science reporting the results.


K Clay, J Riley, “Scaling Up – A Report from the High Value Healthcare Collaborative Distance Learning Program for Decision Coaching” 2016 Developed condition-specific Implementation Guides (Hip & Knee, Spine, Diabetes, CHF). Each implementation guide summarizes the rationale and steps for implementation of shared decision making (SDM), patient-reported outcome measures (PROMs) in the routine clinical care of the specific identified patient population (Project Goal 2), and other PE interventions. In addition, steps for implementing specific best practice improvement projects are summarized (Project Goal 1). The guides clarify that implementation of SDM and PROMs is supported by Centers for Medicare and Medicaid Innovation (CMMI) award, and comprises the core requirements for participation. The four additional improvement projects are encouraged and implementation of all innovations were optional. Initiated a Health Coach Learning Collaborative (HCLC). HCLC members met once a month with separate calls for health coaches focusing on hip, knee and spine projects and those working on diabetes and CHF projects. The main goal of the HCLC was to provide a multi-disciplinary, multi-institutional forum to share site updates, provide implementation support, troubleshoot barriers and challenges, and share successes and struggles. Implemented the web-based Survey Administration Tool (SAT) at more than 136 clinical sites. The HVHC in collaboration with Dynamic Clinical Systems developed and implemented a web-based Survey Administration Tool (SAT) which provided patients with access to patient surveys and linkage to decision aids across multiple conditions. This integrated health information technology (HIT) tool was implemented at member systems for patients to access: patient decision support tools (web-based DAs); surveys to collect patient-generated health data (PGHD); and summary reports for patients and clinicians. Distributed nearly 20,000 Health Dialog video decision aids and summary booklets. In addition to patients accessing video decision aids (DAs) through the SAT tool, HVHC members ordered nearly 20,000 hard copy DAs and booklets directly from Health Dialog to distribute to patients. A total of 6,547 hard copy DAs were ordered by HVHC members in the no-cost extension period alone (7/1/2015 – 12/31/2015) as they prepared to sustain their efforts. Launched the web-based Insight Tool for member reporting. The “Insight” tool (now called “Vantage”) provides members with the ability to generate their own web-based longitudinal and comparative reports on improvement tracking measures. The HVHC Insight Tool is a web-based secure portal for member access to CMS comparative reports. For authorized users logging on with two-factor authentication, the tool provides members with access to comparative, longitudinal reporting of quality and outcomes measures, with drill-down capability to hospital-specific levels.

• Omissions and changes in major project activities. The Year-2 notice of award included a Programmatic Corrective Action Plan (CAP) which changed the focus of our project substantially.


“The project shall be focused on implementing Shared Decision Making across all participating HVHC sites. All members must implement SDM interventions in hip, knee, and spine, unless specific exceptions are approved by CMS. Members may optionally implement SDM interventions in chronic conditions projects (diabetes, CHF). Other patient engagement projects may be implemented as long as the specific site is also implementing SDM for that condition and funding remains.”

Our original goals were to (1) advance best practice care models across all five conditions and (2) implement SDM interventions for preference sensitive conditions (hip, knee and spine surgeries) and patient engagement interventions for patients with diabetes and CHF. The major cost savings projected for the project was tied to the chronic condition population and specifically to the goal of advancing best practice care and patient engagement models for these patients. This new requirement meant that in order to implement the patient engagement and best practice care models to reach our cost savings targets for the chronic conditions (as originally proposed), we were required to also implement Shared Decision Making for diabetes and CHF. This new requirement resulted in a narrowed focus for CHF patients as there are very few treatment options that are considered preference sensitive and therefore eligible for SDM. The forced implementation of SDM for ICD insertion for CHF patients resulted in major delays in implementing the interventions across the board.

Describe any changes in key personnel for the project. There were no changes in key personnel for the project.

For projects involving computer applications, describe any changes that were made in the method of data entry, the specific data to be encoded, software, hardware, file systems or search strategies. Not applicable - this project focused on implementation of the Shared Decision Making and Patient Engagement interventions.

Briefly describe any efforts that were made to publicize the results of the program. The HVHC standard model for public reporting includes communication strategies that cross all levels of the health system, ensuring intra- and inter-organizational sharing. The following efforts were made to publicize progress of the program:

1) HVHC web portal – access for members to view cost, quality, and process outcomes for the HVHC member organizations; these include longitudinal reports of measures and populations associated with best practice interventions implemented at those organizations (including cost of care, utilization of services, survival, functional outcomes, etc.);

2) Local, Regional and National meetings – training workshops, presentations and posters will be submitted for inclusion at local (e.g. health system symposia), regional (e.g. Center for Rural Emergency Services and Trauma conference, New England region), and national meetings of healthcare leaders (e.g. annual meeting of the American College of Health Executives and World Congresses) and clinician-researchers (such as Academy Health’s Annual Research Meeting and the semi-annual meetings of the American Association of Health Economists and the International Health Economics Association);


3) Peer-reviewed Journals –findings from our implementations are published in a wide array of peer-reviewed journals including specialty, implementation science, online journals, etc.

Plans there are to complete the project after the award period, how project activities will be funded, and when they are likely to be completed.

See Section Planned Activities above. Evaluation of this program will be continued after the award period funded by the Trustees of Dartmouth College for a period of at least 3-5 years. The importance of continuing the programs is significant allowing us to:

• extend the length of time necessary to see sustained changed in health care improvements; • leverage CMMI federal investments; • continue cross-walking member-submitted data with CMS data critical for feedback

reporting; • leverage critical infrastructure developed to process the CMS and member-submitted data

files to report process, outcomes and benchmarking data to our members via web-based portal; and

• maintain the focus and engagement of our member systems in sustainability efforts through feedback reporting so they can understand their opportunities and continued progress toward program goals.

Other planned activities include working with strategic partners to advance this work. Such potential partners include: commercial payers, state alternative payment models (e.g., Vermont all payer), and federal alternative payment models (e.g., Next Gen); pharmaceutical companies, the Department of Defense, PCORI, NIH, private industry and foundations (e.g., the Arnold Foundation). We may also expand our work to include regenerative medicine and clinical trials.

Describe the audiences for the project. Indicate the nature, size, geographic reach, sex and age of the audience and assess the impact that the project had on this audience. The project focused on all health systems, clinics and ambulatory surgery facilities caring for patients 18 years of age and older, both male and female considering hip, knee, or spine surgeries and patients with diabetes and/or congestive heart failure (CHF), health care systems, and clinicians (physicians, nurses, physical therapists etc.). The project had national reach and represents the largest of the CMMI awards in terms of scope and geographic reach. HVHC has locations in 31 states (see section entitled “Additional Analyses” above for more details on the population). The impact of this work is underestimated using quantifiable measures. That this large subgroup of health systems has come together to work in a ground-up approach has never been done before. It is the opinion of the PI that more work and greater support is the only way to true health care and delivery system reform will occur, in an organized and meaning way. These organizations are committed to real change but without support and coordination will continue with fragmented one-off reform that is driven by finances and not quality and not truly integrated delivery that is necessary for real change.


What, if any, is the long term impact of your project for improving health care? Long term impact is being assessed through a new study funded by the HVHC members that will allow for an extended period of evaluation and monitoring of each program utilizing CMS data as well as High Value Healthcare Collaborative (HVHC) member-submitted data. The evaluation model will continue utilizing approved monitoring measures for each project. This evaluation will provide the analytic evidence to help determine the long-term impact and success of sustainability for both efforts. We have seen with our assessment of bundles and use of resources, there remains significant variation in spending and outcomes (see publications listed below). Various systems are moving at different paces to make the reforms needed for value-based payments but there are no benchmarks on what “best” looks like. HVHC is all about determining and sharing what “best” could be, not just in spending but in the all-important patient-reported outcomes wherein we can truly measure value.

Publications associated with this project over the award period. 1. High Value Healthcare Collaborative: Episodes of care for hip and knee replacement surgery, W.

Weeks et. al (Accepted - Journal of Arthroplasty) 2. Scaling Up: A report from the High Value Healthcare Collaborative Distance Learning Program

for Decision Coaching, K. Clay et. al. (under review Journal of Patient Education and Counseling)

3. Diabetes Visit Frequency, B. Herndon et. al. (in preparation) 4. Sharing Best Practices for Hip & Knee, J. Nassens et. al. (in preparation) 5. Determinants of simultaneous versus staged bilateral total knee replacement: a multi-

institutional study, I. Tomek et. al. (pending submission to Journal of Bone and Joint Surgeries) 6. Tomek IM, Sabel AL, Froimson MI, Muschler G, Jevsevar D, Koenig KM, Lewallen DG, Naessens

JM, Westrich J, Weeks WB, Weinstein JN. A collaborative of leading health systems finds wide variations in total knee replacement delivery and takes steps to improve value. Health Affairs 2012; 30(6): 1329-1338. PMID: 22571844.

7. Weinstein JN, Nesse RE, James BC, Harrison AM, Cosgrove DM, MacKenzie TD, Gabow P, Colacchio TA, Weiss LT, Weeks WB. The High Value Healthcare Collaborative. A national collaborative to improve value in healthcare delivery: structure, process, challenges, and early lessons learned. Health Affairs 2012: online appendix: http://content.healthaffairs.org/content/31/6/1329/suppl/DC1

Supporting materials associated with this project over the award period. [Final supporting materials should be uploaded into the Publications Portal].

Health Coach Position Description Template Health Coach Overview Hip & Knee Implementation Guide Spine Implementation Guide Diabetes Implementation Guide CHF Implementation Guide Implementation Toolkit Content Description Enrollment Roster Intervention Descriptions

Indicate whether or not any subject inventions were made. No inventions were made through the award.


Appendix A – Patient Engagement Driver Diagram


Appendix B– Analysis design and statistical methodology

Year-over-year outcome comparisons (Method 1) used in the cost-change evaluation were estimated from HVHC monitoring-measures summary data (Appendix C). Annual estimates were calculated using the average (mean) quarterly outcome value weighted by the number of PHN-assigned patients in the quarter. Hip and knee episode costs were calculated as the average (mean) episode cost in surgeries occurring in HVHC member hospitals. Dollar amounts were inflation adjusted to 2013 dollars.

We used a difference-in-difference design to estimate the difference in change in outcomes associated with hospitals participating in the intervention project relative to matched Comparators. Patients were attributed to hospitals using the Physician-Hospital-Network method (J. P. W. Bynum, et al. "Assigning Ambulatory Patients and Their Physicians to Hospitals: A Method for Obtaining Population-Based Provider Performance Measurements." Health Services Research. 2007:42(1): 45-62) regardless of the hospital(s) the patient was admitted to, or, in the case of surgical events, the hospital where surgery was performed (for rates).

Statistical significance for all hypothesis tests was set at the p<0.05 level.

Cost outcomes were modeled as Medicare paid-amounts and as standardized costs using the HVHC modification of the Health Partners Total Care Relative Resource Value methodology (HealthPartners. Total Care Relative Resource Value (TCRRV™) Overview Methodology. https://www.healthpartners.com/ucm/groups/public/@hp/@public/documents/documents/dev_057426.pdf. Accessed 2016-08-30.). Actual cost was inflation adjusted to 2013 dollars.

Comparator Hospital Selection: HVHC hospitals were matched with up to five Comparator Hospitals based on geographic proximity; number of beds; number of surgeries or patients with the condition under study; teaching status (medical school affiliation, residency, and membership in the Council of Teaching Hospitals); urban or rural location (based on Census Department RUCA classification) and Joint Commission accreditation and critical-access status. The population of candidate Comparator Hospitals consisted of all hospitals in Health Referral Regions containing at least one HVHC member hospital. The “optimal” algorithm in the %match SAS macro published by Mayo Clinic (Bergstralh EJ and Kosanke JL. Computerized Matching of Cases to Controls Technical Report Number 56. 1995. http://www.mayo.edu/research/documents/biostat-56pdf/doc-10026923. Accessed 2016-08-30.) was used to match Intervention Hospitals to Comparators and results were reviewed by HVHC analysts to ensure that the program provided valid matches.

The change in outcomes associated with HVHC hospitals was estimated using a mixed model with random intercepts for each hospital or hospital and Intervention-Comparator cluster depending on the data set. Fixed effects were included in the model to adjust for patient attributes, including age, sex, race, comorbid conditions (Charlson score) and poverty (proportion population in patient’s Zip code below poverty). All continuous variables were broken into sets of discrete categories. Cost data were log transformed to better approximate a normal distribution. In-patient admissions were estimated using Poisson mixed-effects models and surgery rates were estimated using logistic regression mixed models. Outcomes were grouped into three-month calendar quarters to minimize variance-bias trade-off and quarters were assumed to be statistically independent. The parameter estimated for each intervention by quarter interaction was assumed to be the difference associated with the intervention. One model including all Intervention-Comparator clusters was fit to event-based outcomes (episode costs) and separate models for each Intervention-Comparator cluster were fit for population-based outcomes (surgery and in-patient

https://www.healthpartners.com/ucm/groups/public/@hp/@public/documents/documents/dev_057426.pdf

http://www.mayo.edu/research/documents/biostat-56pdf/doc-10026923


admission rates, annual costs for chronic conditions). The parallel slopes assumption for the difference-in-difference design was assessed by testing for a non-zero effect of the intervention-by-time interaction in the two quarters (evaluation method 2a) or four quarters (evaluation method 2b) prior to the start of the intervention. For models fit at the cluster level, a summary measure of effect was estimated using inverse-variance weighting (Greenland S, O'Rourke K. 33. Meta-Analysis. In Rothman KJ, Greenland S, Lash TL (eds). Modern Epidemology 3rd edn. Philadelphia, PA: Lippincott, Williams & Wilkins, 2008). Outcomes were annualized and weighted by the number of quarters the beneficiary was observed within the intervention-year when chronic condition outcomes were analyzed by intervention-year and data was collected by intervention-quarter (Ellis RP, Ash A. Refinements to the Diagnostic Cost Group (DCG) Model. Inquiry 1995; 32: 418-429).

Intervention change model "2a" used the presence of member-reported cost-targets “HVHC Intervention Hospitals” as an inclusion criterion. For evaluation model "2b", “HVHC Participating Hospitals” were included in the analysis based on self-reported intervention start-dates and matching of CMS claims (for surgical events) or PHN assignment (for chronic conditions) to HVHC hospitals as well as requiring a minimum number of self-reported interventions. Additionally, for individual-hospital analyses, hospital-comparator clusters with "pre" period data not meeting the parallel slopes assumption for an outcome were excluded from that outcome-specific analysis.

This PHN attribution strategy is an ecological-level analysis, which is a standardized approach across all markets. An ecological-level analysis is the only feasible way to do attribution for large-scale analyses due to the unavailability of drill-down data that would be needed for more specific attribution. We explored the extent to which more specific attribution may better inform our detailed analyses by conducting a sub-analysis using context-specific data for one delivery system to understand the extent to which services were received outside the health system. For example, in this sub-analysis, we observed issues in attribution for a newly structured referral center at a specialty orthopedic hospital that wasn't represented in the higher level analysis. It is possible that some of this inconsistency was due to outdated data in the higher level attribution. We will continue to explore these types of sub-analyses for other systems in HVHC to inform our continuing evaluation of improvement projects.


Appendix C – Monitoring measures As part of the project, HVHC collected monitoring measures describing outcomes related to project goals. The lists below show the relevant measures that were submitted to CMMI and that were used in the evaluation of the project.

Reduce cost monitoring measures

o SM-150a: Total Medicare Part A & B Payment Calculation (standardized pricing). o SM-150b: Total Medicare Part A & B Payment Calculation (actual payment)

Health outcomes monitoring measures

o CM-5: Rates of Total Knee Replacement Surgeries o CM-6: Rates of Total Hip Replacement Surgeries o CM-7: Rates of Spine Surgeries o CM-8: All-Cause Hospital Admission Rates o CM-8a: All-Cause Hospital Admission Rates – Diabetes o CM-8b: All-Cause Hospital Admission Rates – CHF o CM-10: Ambulatory Visits o SM-143: Non-Admitting All-Cause ED Visit Rate o SM-143a: Non-Admitting All-Cause ED Visit Rate – Diabetes o SM-143b: Non-Admitting All-Cause ED Visit Rate – CHF o SS-118a: PROMIS Global Mental Health Score Baseline o SS-118c: PROMIS Global Physical Health Score Baseline o SS-119a: Depression Screener Baseline

Quality of care monitoring measures

o CM-9: Percent of All-Cause Hospital Readmissions o CM-9a: Percent of All-Cause Hospital Readmissions – Diabetes o CM-9b: Percent of All-Cause Hospital Readmissions – CHF o CM-1: Diabetes Intervention Participants with Current Hemoglobin A1c Measurement o CM-2: Baseline Blood Pressure & LDL Cholesterol Proportions [RETIRED] o CM-2a: Diabetes Intervention Participants with Current Blood Pressure Measurement o CM-2b: Diabetes Intervention Participants with Current LDL Cholesterol Measurement o SM-94: Diabetes Intervention Participants with Hemoglobin A1c In Control o SM-95: Diabetes Intervention Participants with Blood Pressure In Control o SM-96: Diabetes Intervention Participants with LDL Cholesterol In Control o CM-11: Patient Referral to Survey [RETIRED] o CM-12: Patient Survey Response Rates o CS-2: Patient Satisfaction with Decision Aids o CS-3: Patient Satisfaction with Survey Tool o CS-5a: Patient Health Confidence Baseline o CS-5b: Patient Health Confidence Change o CS-6: Patient’s Perception of Survey Importance o SS-118b: PROMIS Global Mental Health Score Improvement o SS-118d: PROMIS Global Physical Health Score Improvement o SS-119b: Depression Screener Change


Appendix D: Difference-in-Difference Results for CHF and Diabetes

This table is included to show the details of the difference-in-difference results for CHF and Diabetes. We include this detail because the analysis found significant differences between the Intervention Hospitals and the Comparators. We view this analysis with caution due to the inconsistent pattern and wide confidence intervals (particularly in Year 3). Many hospitals did not have interventions in place long enough to include three years’ worth of post-launch data.

Table 5. Estimated change in chronic costs relative to matched Comparator Hospitals for Intervention Hospitals with interventions for chronic conditions

Condition Outcome Intervention Year

Percent Difference 95% CI

CHF standardized cost 1 6.1 (2.1, 10.3)

2 7.7 (2.4, 13.3) 3 -0.9 (-32, 44.5) actual cost 1 5.4 (1.3, 9.6) 2 6.2 (0.9, 11.8) 3 2 (-30, 49.2) Diabetes standardized cost 1 1.2 (-1.1, 3.4) 2 2.9 (0.3, 5.7) 3 -8.4 (-23, 9.2) actual cost 1 1.2 (-1.1, 3.6) 2 3.4 (0.7, 6.3) 3 -1 (-17, 18.1)

Appendix E: Data Limitations and Technical Notes for Monitoring Measures (Method 1) General Limitations of All files

• Under-diagnosis or system-specific culture that affects coding choice / internal coding rules, including

o imprecise coding or recording of patient information,

o provider belief systems that affect their selection of codes,

o internal data systems that do not collect data in the required format

o data systems designed for billing purposes, rather than the collection of clinical research data

• Non-reporting of necessary data elements

• Non-adherence to data structure requirements (naming, coding, time frame, record duplication)

• Health system's location and its effect on the patient population; a particular institution's case mix, size, or rural location may render it unique from other hospitals in the collaborative.


• Matching between the various files is not perfect: in some cases, the required data are not available for matching, in others, the data are not formatted the same for all files, or the data are present, but do not result in successful matches due to changes or errors in data recording, collection, or transmission

• Small number issues when looking at specific conditions, procedures, or by specific health system or site

o Figures that do not meet the suppression rules or numbers that are too small to meet standards of statistical reliability are not displayed.

CMS Data-based Measures (for which denominator is PHN)

Since enrollment data collection began at different times for members, the number of reporting sites will not be consistent across the entire reporting time frame, and for those who do have the same start date, they may actually have started their interventions after the listed date. Therefore, sites with the same start date are not necessarily comparable to one another.

CMS data are based on the Medicare population and therefore apply only to the 65 and older and population.

CMS data are current through Q32014. We are missing Q42014 and had to create an annualized file for the third time period in our presented data, resulting in Q42013's data being included in both the 2013 and 2014 measures.

Standardized pricing methodology is based off the Health Partners® logic, which is typically used for commercial claims and then normalized to the claim allowed amounts. Therefore, total standardized costs are not directly comparable to payment amounts.

Total payment amounts and standardized prices have been adjusted for outliers by truncating at the 99th percentile within HRR prior to calculating the means.

PHN attribution is done for an attribution year, resulting in a denominator population defined for a whole year at a time. As some patients die over the course of the year and no new patients enter the population, the denominators become smaller for each quarter within a given year. If we assume that all patients are equally likely to die at any given point in a year, then this shouldn’t affect the measures.

Using PHN attribution for creating the denominator population used in surgical rates can result in figures that do not match what the health system actually experiences, as surgical cases are more likely to be referred to hospitals outside their defined primary care system.

Member Data-based Measures (for which patients are identified as enrollees)

Member data was submitted directly from each member's data system. Not all institutions were able to meet all the requirements, resulting in missing files or information necessary for the creation of analytic files. Due to their complexity and volume of measures, lab and vitals files were particularly challenging for members.

Overall, data entry errors, missing data, and site-specific data processing rules negatively affected data quality. Data that were not within valid ranges, were missing or coded to unknown, or did not make sense given the data parameters, were excluded from the analytic files.

Though valid data ranges were vetted internally and based on clinical recommendations, there is the potential that some valid measures were excluded based on the valid range criteria.

Diabetes-specific Considerations

Specific diabetes measures of interest were systolic and diastolic blood pressure, hemoglobin A1c, and LDL cholesterol, which were identified by specific LOINC codes submitted in the vitals and lab files, the most


difficult files for members. Observations were dropped if they did not have a valid test value or date. Observations with 0000 or blank times were converted to missing, and those with missing times had to be treated differently for a matched blood pressure set of measures to be identified.

Lab / test results were not always recorded in the same manner and relied on a combination of LOINC codes and values for the identification of the measure and the test result. Relying on a combination of different codes for one measure introduced additional chance for data errors.

Though the lab values of our test measures were numeric, the data field was alphanumeric, which led to the reporting of test results using text descriptors that did not always translate correctly into a number. The proportion of cases affected by this was small.

The member-based diabetes intervention measures aggregates all intervention participants, independent of the amount of time they spent in the intervention. The proposed next release of the measure will provide separate tables for newly enrolled participants and participants who have been in the intervention for 3 months, 6 months and >6 months so any effect of the intervention over time is diluted.

Patient Data-based Measures (for which patients have completed surveys)

In some cases, particularly with SAT-like data from Dartmouth and UIOWA, it is unclear which (if any) of the five condition categories the patient falls under, and / or whether they are a chronic or (potentially) surgical patient. These small number of cases were excluded from the analysis.

Sepsis data were reported through different mechanisms, and as a result, different time periods were included based on the data collection mechanism rather than the condition.

• DART SAT-like data only runs through quarter 10, while SAT data and UIOWA SAT-like data run through quarter 11.

All patient-generated health data is subject to issues with: response rates, missing data, attrition, and bias.

• Analysis of patient responses is dependent on the patient starting or completing the survey at all (see CM-12), as well as the patient’s willingness to answer specific pertinent question(s).

• All follow-up data: Patient response rates typically drop off in follow-up data. In addition, there is the time lapse factor (particularly for members who started later and have less data), e.g. you can’t have a 6-month follow-up for a patient who was first seen 2 months ago.

• An example of bias is the completion of patient satisfaction questions by patients who self-selected into a group by their willingness to complete a survey in the first place.

CM-12: Analysis of completion / compliance rate of surveys requires header data on all surveys attempted, complete or otherwise. SAT-like data from DART and UIOWA only included complete surveys and is consequently not included in the analysis.

CS-3: The question asking for the patient’s rating of the SAT is skipped a little more frequently than other satisfaction questions. It may also not be answered if data is entered by proxy on behalf of the patient for whatever reason.

hcia final narrative report - highvaluecollaborative.com · 3 pr_1029pe_final_dart.xlsx measures to...

Documents