health plan enrollee experience survey (qhp€¦ · part 1 profiles the content, organization, and...

2016 Qualified Health Plan Enrollee Experience Survey

(QHP Enrollee Survey)

Colorado

August 2016

By the Centers for Medicare & Medicaid Services

i

Table of Contents

Part 1: Executive Summary .................................................................................................................. 1 Overview ................................................................................................................................................. 2 Key Terms ............................................................................................................................................... 3 How This Report Is Organized ................................................................................................................ 3 How Scores Are Compared ..................................................................................................................... 3 How To Use This Report ......................................................................................................................... 4 Public Reporting and Marketing Use ...................................................................................................... 4

Part 2: Detailed Results ........................................................................................................................ 5 Getting Care Quickly (Composite) .......................................................................................................... 6

Getting Needed Care Right Away .................................................................................................... 7 Routine Care .................................................................................................................................... 8

Getting Needed Care (Composite) ....................................................................................................... 10 Easy to Get Care ............................................................................................................................. 11 Getting Appointments with Specialists .......................................................................................... 12

Getting Information in a Needed Language or Format (Composite) ................................................... 13 Needing an Interpreter .................................................................................................................. 14 Forms in Language of Choice ......................................................................................................... 15 Forms in Format of Choice ............................................................................................................. 16

How Well Doctors Communicate (Composite) .................................................................................... 17 Easy To Understand ....................................................................................................................... 18 Listen Carefully ............................................................................................................................... 19 Show Respect ................................................................................................................................. 20 Spend Enough Time ....................................................................................................................... 21

How Well Doctors Coordinate Care and Keep Patients Informed (Composite) ................................... 22 Medical Records ............................................................................................................................. 23 Test Results .................................................................................................................................... 24 Up to Date ...................................................................................................................................... 25 Test Results Quickly ....................................................................................................................... 26 Prescription Medications ............................................................................................................... 27 Getting Help ................................................................................................................................... 28

Health Plan Customer Service (Composite) ......................................................................................... 29 Information Needed ...................................................................................................................... 30 Courtesy and Respect .................................................................................................................... 31

Getting Information About the Health Plan and Costs of Care (Composite) ....................................... 32 Provide Needed Information ......................................................................................................... 33 Cost for Services ............................................................................................................................. 34 Cost for Prescription Medication ................................................................................................... 35

Enrollee Experience With Cost (Composite) ........................................................................................ 36 Payment for Necessary Services .................................................................................................... 37 Out of Pocket ................................................................................................................................. 38 Decided Not to Get Care ................................................................................................................ 40 Did Not Fill Prescription ................................................................................................................. 41

Single‐Item Measures ........................................................................................................................... 42

ii

Overall Ratings ...................................................................................................................................... 46 Enrollees’ Rating of the Health Plan .............................................................................................. 46 Enrollees’ Rating of All Health Care ............................................................................................... 47 Enrollees’ Rating of Personal Doctors ............................................................................................ 48 Enrollees’ Rating of Specialists ...................................................................................................... 49 Global Rating .................................................................................................................................. 50

Part 3: Background and Methodology ................................................................................................ 51 Methodology ........................................................................................................................................ 52

The Survey Instruments ................................................................................................................. 53 Data Collection ............................................................................................................................... 54 Sample Selection and Eligibility Criteria ........................................................................................ 54

QHP Eligibility (Which QHPs Have To Participate) .................................................................. 54 Enrollee Eligibility for QHP Survey/QHP Enrollee Survey Population ..................................... 55 Sample Frame Generation ....................................................................................................... 55

Survey Implementation ................................................................................................................. 55 Sample Disposition ......................................................................................................................... 56

Data Analysis ........................................................................................................................................ 58 Use of Composite Measures .......................................................................................................... 58 Case‐Mix Adjustment ..................................................................................................................... 58

Statistical Details of Case Mix Adjustment .............................................................................. 60 Significance Testing ........................................................................................................................ 80 Assessing Reliability of Scores ........................................................................................................ 81 Relationship Between QHP Enrollee Survey and Quality Rating System (QRS)............................. 83

Appendix 1: Tabular Display of Results............................................................................................... 84

1

Part 1: Executive Summary

2

Overview

Section 1311(c)(4) of the Affordable Care Act (ACA) (42 U.S.C. 13031) directs the Secretary of the U.S.

Department of Health and Human Services (HHS) to establish an enrollee satisfaction survey system

with the purpose of evaluating enrollee experiences with Qualified Health Plans (QHPs) offered through

the Health Insurance Marketplaces (HIMs) and the Small Business Health Options Program (SHOP). The

Centers for Medicare & Medicaid Services (CMS) has developed the Qualified Health Plan Enrollee

Experience Survey (QHP Enrollee Survey) to collect data for three main purposes:

To inform consumer decision‐making in choosing a QHP

To inform quality improvement efforts for QHPs

To allow for effective oversight (assist Marketplaces in overseeing QHPs, and state and federal

regulators in overseeing Marketplaces)

The QHP Enrollee Survey is administered to enrollees of all QHPs by HHS‐approved QHP Enrollee Survey

vendors, using a standardized protocol to facilitate QHP comparison within and across

Marketplaces. The results of the QHP Enrollee Survey, along with clinical measures collected through the

Quality Rating System (QRS), will be the subject of a pilot with consumers in six states using the Health

Insurance Marketplace website (HealthCare.gov) in 2016 and will be available to all Marketplace users in

2017. State‐based Marketplaces may choose to report data in 2016.

The QHP Enrollee Survey was designed to capture accurate and reliable information from consumers

about their experiences with their health plan and the health care they received through their QHP. The

survey includes a core set of questions on key areas of care and service, with some questions grouped to

form composites. Survey data is used to examine QHP performance and to create national benchmarks

for care. The QHP Enrollee Survey will be conducted annually to assess the experiences of enrollees in

QHPs in each Marketplace.

The 2016 survey was conducted between January and May 2016 to measure QHP enrollees’ experiences

with your plan. Although beneficiaries provide ratings of their specific “qualified health plans,” the

reporting unit is not a single health insurance plan but rather all plans of the same type (provider

organization [EPO], health maintenance organization [HMO], point of service [POS], or preferred

provider organization [PPO]) offered by a given QHP issuer within a particular Marketplace. In future

years, the reporting unit might change based on analysis of enrollee experiences.

3

Key Terms

The following terminology is used in this report:

QHP Issuer: The commercial insurance provider offering one or more health coverage options in

a given Marketplace.

Product type: Insurance plans available through Marketplaces include EPO, HMO, POS, or PPO

plans; offering catastrophic, bronze, silver, gold, or platinum levels of coverage; and covering all

or part of a Marketplace’s geographic area.

QHP reporting unit: This report summarizes the results of enrollees’ experiences in each type of

plan: EPO, HMO, POS, or PPO. Results for all of a QHP issuer’s plans of a single type within one

Marketplace are combined. For example, AlphaCare offers a bronze‐ level HMO, two silver‐level

HMOs, a gold‐level HMO, and one PPO each at the gold and platinum levels. AlphaCare would

receive two reports: One on enrollees’ experiences across all of its HMOs, and one on all of its

PPOs. A QHP issuer could receive up to four reports for a single Marketplace if it had at least one

plan of every type.

How This Report Is Organized

This report provides results from the 2016 QHP Enrollee Survey for all QHP Issuers in your Marketplace.

Part 1 profiles the content, organization, and purpose of this report.

Part 2 presents detailed results for several composite measures and the individual survey

questions that make up those composites, results for single questions from the survey, and

rating measures that summarize enrollees’ overall satisfaction with various aspects of their

coverage and care. For each composite or single question, QHP Issuers and their product types

are listed in order of performance, from highest mean score to lowest. Part 3 provides some

background information about the survey and describes sampling protocols and other

methodological topics.

How Scores Are Compared

Scores in this report are compared against national average scores for all enrollee ratings of QHPs

nationwide. In 2016, for the 2017 Open Enrollment Period, CMS will conduct a pilot test to display

scores to help consumers compare QHPs in six states using the Federally facilitated Marketplace

website.1 Frequency tables that display unadjusted responses (not case‐mix adjusted) to all survey items

1 The composite structure for all measures displayed within this report aligns with the CAHPS Health Plan Survey

5.0; however the composites used within the Quality Rating System (QRS) differ slightly. As a result, the scores displayed within this report will differ from those generated through the Quality Rating System. Additional information about this difference is discussed in “Part 3: Background and Methodology”.

4

are shown at the end of this section. Description of case‐mix adjustment can be found in Part 3. When a

question received 11 or fewer responses, the answer will be left blank.

How To Use This Report

Marketplaces can use this report to expand their understanding of enrollees’ experiences with health

plans within the Marketplace and relative to national averages. Marketplaces may wish to use findings

in this report to help QHPs identify opportunities to improve or to identify examples of high‐performing

QHPs that can serve as models. Over time, multiple years of data will be available and Marketplaces will

be able to examine trends in performance among QHPs.

Public Reporting and Marketing Use

Beginning in 2017 and beyond, results of the QHP Enrollee Survey will be publicly reported on

Marketplace websites. A pilot test will take place in six states in 2016.

CMS will publish guidelines related to public display of quality rating information by Marketplaces to

inform consumer QHP selection in 2017 (for the 2018 coverage year).

CMS has issued guidance for QHPs that wish to include information from the QHP Enrollee Survey or

QRS in marketing information. This includes the use of appropriate disclaimers and the use of data from

the most recent survey and QRS ratings. For specifics, please see the Quality Rating System and

Qualified Health Plan Enrollee Experience Survey: Technical Guidance for 2016.

5

Part 2: Detailed Results

6

The following pages provide detailed results of the 2016 QHP Enrollee Survey, including each QHP

Issuer’s reporting unit performance on the composite measures and their component questions, as well

as the results of individual survey questions. Description of case‐mix adjustment can be found in Part 3.

Results are displayed based on each reporting unit’s mean score, from highest to lowest. This means

that QHP Issuers and reporting units will not necessarily appear in the same order for each composite or

measure, as performance may vary. In the stacked bar graphs, blue indicates the best response (such as

“always” or 9‐10 on a 0‐10 scale) and orange indicates the worst.

Getting Care Quickly (Composite)

Figure 1 shows how QHP reporting units in your Marketplace performed on “Getting Care Quickly,” a

composite of survey questions 4 and 6. The figure shows the national distribution and whether each

reporting unit’s mean score was significantly better or worse than the national average. A blank row

indicates that too few enrollees answered the question to permit reporting. All statistics are adjusted for

case‐mix. Results for the individual questions included in this composite measure are shown on

subsequent pages.

Figure 1. Composite: Getting Care Quickly

blank blank Mean How Do Reporting

Units in Your Marketplace Compare to the National Average? (p < .05)

blank

blank blank

National Average

76.96 —

Rocky Mountain HMO ‐ HMO

82.32 above average

Rocky Mountain HMO ‐ PPO

78.45 average

Kaiser Foundation Health Plan of Colo. ‐ HMO

75.75 average

7



HMO Colorado, Inc., dba HMO Nevada ‐ HMO

75.52 average

Colorado Choice Health Plans ‐ HMO

73.39 average

Humana Health Plan ‐ HMO

71.74 average

Items in This Composite

Getting Needed Care Right Away

Figure 2. Question 4: In the last 6 months, when you needed care right away, how often did you get

care as soon as you needed?*



blank

blank blank

National Average

79.27 —


85.07 above average


82.69 average

8




76.75 average


75.89 average


73.64 average


72.33 average

* This question was included in the 2016 QRS.

Routine Care

Figure 3. Question 6: In the last 6 months, how often did you get an appointment for a check‐up or

routine care at a doctor’s office or clinic as soon as you needed?*



blank

blank blank

National Average

74.70 —


79.55 above average

9




75.59 average


74.27 average


74.20 average


73.14 average


71.15 average


10

Getting Needed Care (Composite)

Figure 4 shows how QHP reporting units in your Marketplace performed on “Getting Needed Care,” a

composite of survey questions 11 and 33. The figure shows the national distribution and whether the



case‐mix. Results for the individual questions included in this composite are shown on subsequent

pages.

Figure 4. Composite: Getting Needed Care



blank

blank blank

National Average

78.06 —


80.85 average


80.47 average


80.11 average


77.59 average


75.05 average


67.35 below average

11


Easy to Get Care

Figure 5. Question 11: In the last 6 months, how often was it easy to get the care, tests, or treatment

you needed?*



blank

blank blank

National Average

79.36 —


83.33 above average


80.83 average


78.92 average


76.35 average


74.72 below average


71.35 below average


12

Getting Appointments with Specialists

Figure 6. Question 33: In the last 6 months, how often did you get an appointment to see a specialist

as soon as you needed?*



blank

blank blank

National Average

76.73 —


82.77 above average


80.44 average


79.38 average


77.60 average


73.74 average


63.34 below average


13

Getting Information in a Needed Language or Format (Composite)

Figure 7 shows how QHP reporting units in your Marketplace performed on “Getting information in a

needed language or format,” a composite of survey questions 13, 49, and 51. The figure shows the

national distribution and whether the reporting unit’s mean score was significantly better or worse than

the national average. A blank row indicates that too few enrollees answered the question to permit

reporting. All statistics are adjusted for case‐mix. Results for the individual questions included in this

composite are shown on subsequent pages.

Figure 7. Composite: Getting information in a needed language or format



blank

blank blank

National Average

69.05 —


90.05 above average


86.62 above average


83.01 above average


73.05 average


70.14 average


51.32 below average

14


Needing an Interpreter

Figure 8. Question 13: In the last 6 months, when you needed an interpreter at your doctor’s office or

clinic, how often did you get one?*



blank

blank blank

National Average

71.07 —


blank — —


blank — —


blank — —


blank — —


blank — —


blank — —


15

Forms in Language of Choice

Figure 9. Question 49: In the last 6 months, how often were the forms that you had to fill out available

in the language you prefer?*



blank

blank blank

National Average

87.40 —


95.48 above average


94.67 above average


93.29 above average


92.75 above average


92.09 average


88.59 average


16

Forms in Format of Choice

Figure 10. Question 51: In the last 6 months, how often were the forms that you had to fill out

available in the format you needed, such as large print or Braille?*



blank

blank blank

National Average

57.40 —


blank — —


blank — —


blank — —


blank — —


blank — —


blank — —


17

How Well Doctors Communicate (Composite)

Figure 11 shows how QHP reporting units in your Marketplace performed on “How Well Doctors

Communicate,” a composite of survey questions 16, 17, 18, and 19. The figure shows the national

distribution and whether the reporting unit’s mean score was significantly better or worse than the

national average. A blank row indicates that too few enrollees answered the question. All statistics are

adjusted for case‐mix. Results for the individual questions included in this composite are shown on

subsequent pages.

Figure 11. Composite: How Well Doctors Communicate



blank

blank blank

National Average

89.48 —


90.87 average


89.77 average


89.68 average


89.34 average


88.41 average


86.71 average

18


Easy To Understand

Figure 12. Question 16: In the last 6 months, how often did your personal doctor explain things in a

way that was easy to understand?



blank

blank blank

National Average

89.91 —


91.15 average


90.99 average


90.36 average


89.63 average


89.60 average


86.62 below average

19

Listen Carefully

Figure 13. Question 17: In the last 6 months, how often did your personal doctor listen carefully to

you?



blank

blank blank

National Average

89.49 —


91.03 average


90.46 average


90.20 average


89.87 average


88.65 average


85.61 below average

20

Show Respect

Figure 14. Question 18: In the last 6 months, how often did your personal doctor show respect for

what you had to say?



blank

blank blank

National Average

91.70 —


93.94 average


92.04 average


91.45 average


90.59 average


89.90 average


89.26 average

21

Spend Enough Time

Figure 15. Question 19: In the last 6 months, how often did your personal doctor spend enough time

with you?



blank

blank blank

National Average

86.81 —


88.71 average


86.36 average


86.11 average


86.09 average


85.74 average


84.73 average

22

How Well Doctors Coordinate Care and Keep Patients Informed

(Composite)

Figure 16 shows how reporting units in your Marketplace performed on “How Well Doctors Coordinate

Care and Keep Patients Informed,” a composite of survey questions 20, 22, 23, 25, 28, and 31. The figure

shows the national distribution and whether the reporting unit’s mean score was significantly better or

worse than the national average. A blank row indicates that too few enrollees answered the question to

permit reporting. All statistics are adjusted for case‐mix. Results for the individual questions included in

this composite are shown on subsequent pages.

Figure 16. Composite: How Well Doctors Coordinate Care and Keep Patients Informed



blank

blank blank

National Average

83.45 —


82.63 average


80.70 average


80.36 average


79.69 average


79.65 below average


79.56 average

23


Medical Records

Figure 17. Question 20: When you visited your personal doctor for a scheduled appointment in the

last 6 months, how often did he or she have your medical records or other information about your

care?*



blank

blank blank

National Average

91.95 —


92.24 average


92.21 average


91.98 average


90.91 average


90.54 average


89.93 average


24

Test Results

Figure 18. Question 22: In the last 6 months, when your personal doctor ordered a blood test, x‐ray, or

other test for you, how often did someone from your personal doctor’s office follow up to give you

those results?*



blank

blank blank

National Average

81.17 —


82.94 average


82.26 average


81.41 average


80.35 average


80.12 average


78.59 average


25

Up to Date

Figure 19. Question 25: In the last 6 months, how often did your personal doctor seem informed and

up‐to‐date about the care you got from specialists?*



blank

blank blank

National Average

75.28 —


73.18 average


71.41 average


71.04 average


70.70 average


65.14 below average


65.13 below average


26

Test Results Quickly

Figure 20. Question 23: In the last 6 months, when your personal doctor ordered a blood test, x‐ray, or

other test for you, how often did you get those results as soon as you needed them?*



blank

blank blank

National Average

85.21 —


88.03 average


86.54 average


84.97 average


84.40 average


83.43 average


80.59 below average


27

Prescription Medications

Figure 21. Question 28: In the last 6 months, how often did you and your personal doctor talk about

all the prescription medicines you were taking?*



blank

blank blank

National Average

82.29 —


83.18 average


82.64 average


81.50 average


80.28 average


77.31 average


74.02 below average


28

Getting Help

Figure 22. Question 31: In the last 6 months, did you get the help you needed from your personal

doctor’s office to manage your care among these different providers and services? *



blank

blank blank

National Average

81.01 —


81.44 average


80.00 average


75.47 average


74.80 average


71.61 average


71.47 average


29

Health Plan Customer Service (Composite)

Figure 23 shows how reporting units in your Marketplace performed on “Health Plan Customer Service,”

a composite of survey questions 43 and 44. The figure shows the national distribution and whether the



case‐mix. Results for the individual questions included in this composite are shown on subsequent

pages.

Figure 23. Composite: Health Plan Customer Service



blank

blank blank

National Average

77.15 —


88.20 above average


87.04 above average


80.16 average


74.46 average


65.95 below average


63.50 below average

30


Information Needed

Figure 24. Question 43: In the last 6 months, how often did your health plan’s customer service staff

give you the information or help you needed?*



blank

blank blank

National Average

68.90 —


82.99 above average


79.92 above average


73.41 average


61.30 below average


55.22 below average


52.72 below average


31

Courtesy and Respect

Figure 25. Question 44: In the last 6 months, how often did your health plan’s customer service staff

treat you with courtesy and respect?*



blank

blank blank

National Average

85.40 —


94.15 above average


93.41 above average


87.62 average


86.91 average


76.67 below average


74.29 below average


32

Getting Information About the Health Plan and Costs of Care

(Composite)

Figure 26 shows how reporting units in your Marketplace performed on “Getting Information About the

Health Plan and Costs of Care,” a composite of survey questions 37, 39, and 41. The figure shows

whether each reporting unit’s mean score was significantly better or worse than the national average. A

blank row indicates that too few enrollees answered the question to permit reporting. All statistics are

adjusted for case‐mix. Results for the individual questions included in this composite are shown on

subsequent pages.

Figure 26. Composite: Getting Information About the Health Plan and Costs of Care



blank

blank blank

National Average

55.32 —


57.81 average


54.18 average


52.04 average


51.60 average


50.89 average


49.11 average

33


Provide Needed Information

Figure 27. Question 37: In the last 6 months, how often did the written materials or the Internet

provide the information you needed about how your health plan works?*



blank

blank blank

National Average

57.62 —


60.99 average


54.61 average


53.46 average


51.96 below average


51.13 below average


49.90 below average


34

Cost for Services

Figure 28. Question 39: In the last 6 months, how often were you able to find out from your health

plan how much you would have to pay for a health care service or equipment before you got it?*



blank

blank blank

National Average

51.40 —


54.25 average


51.14 average


48.94 average


47.23 average


44.86 average


41.35 average


35

Cost for Prescription Medication

Figure 29. Question 41: In the last 6 months, how often were you able to find out from your health

plan how much you would have to pay for specific prescription medications?*



blank

blank blank

National Average

57.59 —


60.33 average


58.24 average


56.12 average


54.40 average


54.33 average


53.96 average


36

Enrollee Experience With Cost (Composite)

Figure 30 shows how reporting units in your Marketplace performed on “Enrollee Experience With

Cost,” a composite of survey questions 54, 55, 56 and 57. The figure shows whether each reporting

unit’s mean score was significantly better or worse than the national average. A blank row indicates that

too few enrollees answered the question to permit reporting. All statistics are adjusted for case‐mix.

Results for the individual questions included in this composite are shown on subsequent pages. Note

that for this composite and all of its measures, “Never” is the best possible answer.

This composite and individual items within it are not part of the QRS and are not publicly reported. They

are intended solely to be used for quality improvement. The items in this composite are presented with

the standard case‐mix adjustment found in Exhibit 6 in Part 3: Background and Methodology. For

purposes of comparison, Exhibits 45‐49 in Part 3 also provides the means for this composite and its

component measures if the metal level of QHP offerings is included as a case mix adjuster. Because

metal level relates directly to enrollees’ deductibles and out‐of‐pocket expenses, health plan metal level

may affect enrollees’ perception of cost‐related issues for their health plan. A reporting unit that is

composed of mostly gold or platinum plans could receive different responses than a reporting unit in

which plans are primarily bronze or silver, in which out‐of‐pocket costs might be higher.

Figure 30. Composite: Enrollee Experience With Cost



blank

blank blank

National Average

78.59 —


79.34 average


78.37 average


76.30 average

37




76.13 average


74.78 below average


73.22 below average


Payment for Necessary Services

Figure 31. Question 54: In the last 6 months, how often did your health plan not pay for a service that

your doctor said you needed?



blank

blank blank

National Average

78.02 —


79.73 average


79.39 average

38




77.73 average


76.53 average


74.12 below average


70.77 below average

Out of Pocket

Figure 32. Question 55: In the last 6 months, how often did you have to pay out of your own pocket

for care that you thought your health plan would pay for?



blank

blank blank

National Average

76.49 —


76.57 average

39




75.63 average


74.92 average


73.99 average


73.08 below average


70.49 below average

40

Decided Not to Get Care

Figure 33. Question 56: In the last 6 months, how often did you delay or not visit a doctor because you

were worried about the cost? Do not include dental care.



blank

blank blank

National Average

75.06 —


73.63 average


70.78 below average


69.72 below average


69.64 below average


69.55 below average


67.77 below average

41

Did Not Fill Prescription

Figure 34. Question 57: In the last 6 months, how often did you delay or not fill a prescription because

you were worried about the cost?



blank

blank blank

National Average

84.79 —


88.37 above average


87.96 above average


86.31 average


82.17 average


81.99 average


81.77 average

42

Single‐Item Measures

Figure 35. Question 8: In the last 6 months, how often were you able to get care you needed from a

doctor’s office or clinic after regular office hours?*



blank

blank blank

National Average

60.09 —


85.08 above average


83.95 above average


76.93 average


74.28 average


51.54 average


45.06 average


43

Figure 36. Question 45: In the last 6 months, how often did the time that you waited to talk to your

health plan’s customer service staff take longer than you expected?*



blank

blank blank

National Average

55.61 —


77.78 above average


70.34 above average


68.02 above average


54.33 average


43.08 below average


39.25 below average


44

Figure 37. Question 47: In the last 6 months, how often were the forms for your health plan easy to fill

out?*



blank

blank blank

National Average

65.45 —


72.64 above average


70.13 average


65.19 average


61.75 average


60.23 average


55.63 average


45

Figure 38. Question 48: In the last 6 months, how often did the health plan explain the purpose of a

form before you filled it out?*



blank

blank blank

National Average

68.57 —


80.50 above average


71.69 average


67.38 average


62.13 average


61.42 average


56.57 average


46

Overall Ratings

Enrollees’ Rating of the Health Plan

Figure 39. Question 52: Rating of health plan: Using any number from 0 to 10, where 0 is the worst

health plan possible and 10 is the best health plan possible, what number would you use to rate your

health plan in the last 6 months?



blank

blank blank

National Average

72.21 —


72.64 average


69.49 below average


68.11 below average


66.67 below average


65.09 below average


62.12 below average


47

Enrollees’ Rating of All Health Care

Figure 40. Question 10: Rating of all health care: Using any number from 0 to 10, where 0 is the worst

health care possible and 10 is the best health care possible, what number would you use to rate all

your health care in the last 6 months?



blank

blank blank

National Average

81.56 —


82.74 average


80.45 average


78.52 below average


77.99 below average


77.79 below average


74.95 below average


48

Enrollees’ Rating of Personal Doctors

Figure 41. Question 26: Rating of personal doctor: Using any number from 0 to 10, where 0 is the

worst personal doctor possible and 10 is the best personal doctor possible, what number would you

use to rate your personal doctor?



blank

blank blank

National Average

88.36 —


88.27 average


87.80 average


87.40 average


86.15 average


85.74 below average


85.16 average


49

Enrollees’ Rating of Specialists

Figure 42. Question 35: Rating of specialist: We want to know your rating of the specialist you saw

most often in the last 6 months. Using any number from 0 to 10, where 0 is the worst specialist

possible and 10 is the best specialist possible, what number would you use to rate the specialist?



blank

blank blank

National Average

86.65 —


87.62 average


86.58 average


85.94 average


85.68 average


83.53 average


82.71 below average


50

Global Rating

Figure 43. Question 53: Global Rating of Health Plan: Using any number from 0 to 10, where 0 is not at

all likely and 10 is extremely likely, how likely is it that you would recommend this health plan to a

friend or family member?



blank

blank blank

National Average

70.33 —


69.96 average


68.83 average


64.73 below average


64.39 below average


58.57 below average


56.48 below average

51

Part 3: Background and Methodology

52

QHPs offered through Marketplaces are assessed using a reliable and valid survey instrument

administered and scored according to a standardized methodology that meets QHP certification

requirements. CMS used the CAHPS®2 principles (http://www.ahrq.gov/cahps/about‐

cahps/principles/index.html) to inform the QHP Enrollee Survey development.

QHPs are very similar to the health plans assessed by the CAHPS Health Plan Survey; thus, CMS adopted

the current version of this survey, the CAHPS 5.0 Health Plan Survey (Adult Medicaid) as the core of the

QHP Enrollee Survey. The QHP Enrollee Survey includes all questions in the core CAHPS Health Plan 5.0

(Adult Medicaid). The survey also adds several new domains and items based on a comprehensive

review of the literature and related surveys, consumer focus groups, public comments, stakeholder

discussions, and input from a Technical Expert Panel. These additional topics include care coordination,

and information provided by the health plan.

The QHP Enrollee Survey uses a 6‐month reference period, consistent with other CAHPS instruments. To

facilitate administration comparability across QHP issuers, neither a QHP issuer nor a survey vendor may

change the wording of the survey questions, the response categories, or the order of the questions. The

survey vendor may make minor modifications to the format and layout of the questionnaire, adhering to

the formatting parameters specified in the “Data Collection” section of the Quality Assurance Guidelines

and Technical Specifications (QAG) manual

(https://qhpcahps.cms.gov/sites/default/files/upload/2015_QHP_Survey_QAG_111414.pdf).

Many of the items contained within the QHP Enrollee Survey are preceded by screener questions. This

process confirms that only those sampled enrollees for whom a particular item is relevant answer the

pertinent items following each specific screener question.

In addition to English, the QHP Enrollee Survey questionnaire is available in Spanish and Traditional

Chinese (Mandarin) translations.

The QHP Enrollee Survey produces comparable data on enrollees’ experience of care, allowing objective

and meaningful comparisons among QHP domains that are important to consumers. The survey was

fielded nationally by HHS‐approved QHP Enrollee Survey vendors from January to May 2016. The 2016

data will be used to examine QHP performance and to create national benchmarks for care.

Beginning with the 2016 implementation of the QHP Survey, survey results will be used by consumers to

assist in the selection of a QHP. In addition, QHP issuers can use survey results to identify areas for

quality improvement. Administrators and policy makers can also rely on the measures to devise,

implement, and monitor quality improvement efforts and to make policy decisions.

Methodology

For the 2016 QHP Enrollee Survey, the sampling/reporting unit has been defined at the level of product

type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred

Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through the Marketplace in

2 CAHPS® is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ).

53

a particular state. For example, XYZ issuer’s HMOs offered through the Marketplace in Florida would be

considered a single sampling unit. Note that, depending on the way a QHP issuer packages its plan

offerings, the sampling unit might include anything from a single QHP to many QHPs spanning all

categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). Only QHPs with more than 500

enrollees are required to conduct the QHP Enrollee Survey.

The Survey Instruments

The 2016 QHP Enrollee Survey questionnaire includes a total of 85 questions; however, many questions

are preceded by screener questions to ensure that enrollees answer only the items that are applicable

to them. For analysis and reporting purposes, some questions are combined into composites including:

1. Getting needed care

2. Getting care quickly

3. Getting information in a needed language or format

4. How well doctors communicate

5. How well doctors coordinate care and keep patients informed

6. Getting information about the health plan and costs of care

7. Health plan customer service

8. Enrollee experience with cost

Exhibit 1 shows the survey questions that comprise each composite; however, note that the composites

in this report differ slightly from the composites generated for the 2016 Quality Rating System (QRS).

Additional changes may be made to the questionnaire and/or the composite structure as the program

evolves. For example, the 2016 survey includes the “Enrollee Experience With Cost” composite and its

component measures, as well as the global rating question.

Exhibit 1. Composites and Items in the 2016 QHP Enrollee Survey

QRS Composite QHP Enrollee Survey Composite QHP Enrollee Survey Questions

Access to Care Getting Care Quickly 4, 6

Access to Care (cont.) Getting Needed Care 11, 33 (8 in QRS only)

Access to Information Getting information about the health plan and costs of care

37, 39, 41

Care Coordination How well doctors coordinate care and keep patients informed

20, 22, 23, 25, 28, 31

Cultural Competence Getting information in a needed language or format

13, 49, 51

Plan Administration Health plan customer service 43, 44 (45, 47, 48 in QRS only)

(Not part of QRS) How well doctors communicate 16, 17, 18, 19

54

QRS Composite QHP Enrollee Survey Composite QHP Enrollee Survey Questions

(Not part of QRS) Enrollee experience with cost 54, 55, 56, 57

Rating of all Health Care Single Item Measure (Global Ratings)

10

Rating of Health Plan Single Item Measure (Global Ratings)

52

Rating of Personal Doctor Single Item Measure (Global Ratings)

26

Rating of Specialist Single Item Measure (Global Ratings)

35

Single‐items Single‐item questions in QHP survey that are included in QRS (including questions on aspiring use, flu vaccination, and smoking cessation)

8, 32, 33, 45, 47, 48, 60, 62, 63, 64, 65, 66, 67, 68, 69

Data Collection

The QHP Enrollee Survey is conducted by HHS‐approved survey vendors. Survey vendors must meet or

exceed the minimum business requirements established by CMS. Seven survey vendors administered

the 2016 QHP Enrollee Survey in accordance with guidelines and specifications outlined in the 2016 QHP

Enrollee Survey Quality Assurance Guidelines and Technical Specifications.

Sample Selection and Eligibility Criteria

QHP Eligibility (Which QHPs Have To Participate)

As specified by section 1311(c)(4) of the Affordable Care Act, only QHPs with more than 500 enrollees

are required to conduct the QHP Enrollee Survey. For the 2016 survey, CMS issued guidance stating that

this requirement would be applied at the reporting unit level and that issuers should use July 1, 2015, as

the anchor date for determining whether they met this threshold. Additionally, CMS issued guidance

that the QHPs that meet all three of the following criteria are eligible to participate in the 2016 QHP

Enrollee Survey:

1. It must have a 14‐digit Standard Component ID number (SCID) assigned in the Health Insurance

Oversight System (HIOS) database and

2. That SCID must be offered on an Individual Marketplace and/or a Small Business Health Options

Program (SHOP) Marketplace and

3. It must provide family and/or adult‐only medical coverage, regardless of whether or not it also

covers dental care or vision care.

Child‐only QHPs, Stand‐alone Dental Plans (SADPs), and Stand‐alone Vision Plans were not required to

field the 2016 QHP Enrollee Survey. In addition, the QHPs issuers that offered family and/or adult health

55

insurance through the a Marketplace for plan year 2015 but opted to not offer coverage through the

Marketplace for plan year 2016 were exempted from participating in the 2016 QHP Enrollee Survey.

Enrollee Eligibility for QHP Survey/QHP Enrollee Survey Population

QHP enrollees were eligible to be included in the 2016 QHP Enrollee Survey if they met all of the

following requirements:

Enrollee was 18 years or older as of December 31, 2015

Enrollee was enrolled in an eligible QHP for at least the last 6 months as of December 31, 2015,

with no more than one 31‐day break in enrollment during the 6 months

Enrollee received his/her primary health coverage through the eligible QHP

Enrollees in QHPs offered outside the Marketplace (off‐Marketplace) were not eligible.

Sample Frame Generation

QHP issuers were responsible for generating a list of all enrollees who met the eligibility criteria

(sampling frame) for each reporting unit using specifications outlined in the 2016 QHP Enrollee Survey

Quality Assurance Guidelines and Technical Specifications. Subsequently, QHP issuers were required to

submit the sample frame for each reporting unit to an NCQA‐certified HEDIS3 Compliance Auditor to

verify the integrity of the sample frame. Once the sample frame was approved by the HEDIS Compliance

Auditor, the frame was then provided to the QHP issuer’s designated survey vendor.

The CMS‐approved survey vendors were responsible for drawing the sample of enrollees who received

the QHP Enrollee Survey. In order to select this sample, vendors ensured that one enrollee per covered

family unit was included in the sample frame using a simple random sample of individuals covered under

the same policy. Subsequently, vendors drew a simple random sample of at least 1,300 individuals for

each reporting unit. If a reporting unit had fewer than 1,300 enrollees eligible for the survey, then

survey vendors included all individuals in the sample. QHP issuers were allowed to request larger

sampling frames to facilitate subgroup analysis, up to 1,690.

Survey Implementation

The 2016 QHP Enrollee Survey was fielded between January and May 2016 to measure QHP enrollees’

experiences with the care they received through their QHP in the previous 6 months. Data collection

was conducted using a mixed‐mode approach that used mail, telephone, and Web surveys. Sampled

individuals received a pre‐notification letter, up to two mailings of a paper survey, a reminder letter

between the first and second survey mailing, and up to 6 telephone follow‐ups to enrollees who did not

respond on the Web. The pre‐notification and reminder letters also offered sampled enrollees who had

a language preference of English the option to complete the survey online; issuers had the option to

offer an online version of the Spanish survey as well.

3 HEDIS® is a registered trademark of the National Committee for Quality Assurance (NCQA).

56

The mail and telephone surveys were available in English, Spanish, and Traditional Chinese (Mandarin);

however, administration of the survey in Chinese was optional. Exhibit 2 presents the timeline of the full

survey implementation.

Exhibit 2. Survey Implementation Timeline

Activity Date

Conditionally approved survey vendors contract with QHP issuers to administer the QHP Enrollee Survey

Aug. 2015–Dec. 2015

Survey vendors submit all mail, telephone, and online materials to CMS for review Nov. 2‐17, 2015

Survey vendors submit draft QHP client list Jan. 5, 2016

Deadline for QHP Issuers to authorize a survey vendor through the QHP Enrollee Survey website

Jan. 5, 2016

Survey vendor receives validated sample frame from QHP issuer along with confirmation that an NCQA‐certified HEDIS Compliance Auditor has validated the sample frame. Survey vendor draws a survey sample from the validated sample frame.

Jan.–Feb. 2016

Survey vendors administer QHP Enrollee Survey Jan.–May 2016

Submission Test Period: Survey vendors successfully submit at least one interim data file to the submission link on the Project website to test data submission protocols

April 6‐8, 2016

Data Submission Period May 11‐25, 2016

Deadline for Data Submission May 25, 2016

Sample Disposition

The sample disposition and response rates for the 2016 QHP Enrollee Survey are presented in Exhibit 3.

Of the 376,962 QHP enrollees sampled for the survey, 18,229 (4.8 percent) enrollees were classified as

ineligible because they were no longer enrolled in the health plan they were sample for, institutionalized

for the duration of the field period, deceased, or mentally or physically incapable and did not consent

for a proxy to complete the survey on their behalf. QHP Enrollees under the age of 18 are ineligible and

are excluded from the sample frame.

The QHP Enrollee Survey Project Team utilizes Response Rate 3 (RR3) established by the American

Association for Public Opinion Research (AAPOR)4 to calculate the response rate for the QHP Enrollee

Survey. This response rate accounts for the fact that a proportion of the remaining cases with an

unknown eligibility will be ineligible. The formula estimates the proportion of cases that would be

ineligible based on the rate of ineligibility within each reporting unit. A summary of this response rate is

shown below:

4 The American Association for Public Opinion Research. 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th edition. AAPOR.

57

Where

C = Completed Surveys

E = Partial Completed Surveys

U = Cases with Unknown Eligibility

O = Other Disposition

R = Refusal

I = Ineligible

X = Proportion of cases eligible for this survey, which is calculated as:

The response rate, after accounting for both ineligible enrollees, is 30 percent (75,719 completed

surveys).

Exhibit 3. Sample Disposition and Response Rates

Disposition QHP Enrollee Survey Nationally

Marketplace [For all plans in Marketplace]

Response Rate 30.0% 34.5%

Complete 75719 1953

Web 8718 230

Mail 51113 1336

Telephone 15888 387

Ineligible 18229 197

Institutionalized 35 —

Deceased 328 —

Not Eligible or on “Do not survey” list 17866 190

Nonrespondents 283014 5458

Partial Complete 2810 45

Language Barrier 3302 30

Mentally or Physically Incapacitated 176 —

Refusal 44710 1196

Non Response 158507 3099

Blank Survey 993 17

Bad Address/Phone 72516 1067

58

Data Analysis

Data from the QHP Enrollee Survey were weighted to reflect the number of survey‐eligible enrollees in

each reporting unit and produce summary statistics. The sections below describe the major analysis

steps.

Use of Composite Measures

When a survey covers many topics, it is useful to combine items about similar topics into composites. In

most cases the items in a composite are given equal weight in calculating the composite score. The

exception is the measure “How Well Doctors Coordinate Care and Keep Patients Informed,” a composite

of survey questions 20, 22, 23, 25, 28, and 31. The two survey items about test results are averaged

together prior to being combined with the other four items that comprise the composite. In cases where

both items are non‐missing, this method has the effect of down‐weighting these two items each by half;

the combined score for the two “tests” items thus has a weight of one.

Case‐Mix Adjustment

Certain respondent characteristics, such as age, education, and health status, are not under the control

of the health plan but are related to the enrollee’s survey responses. To ensure that comparisons among

reporting units reflect differences in performance rather than differences in case‐mix, it is necessary to

adjust for such respondent characteristics when generating QHP Enrollee Survey scores.

Previous analyses of CAHPS data have shown that self‐reported global health status, self‐reported

mental health status, age, and education account for some variation among health plans on the CAHPS

global ratings. In general, individuals reporting better physical and mental health, those who are older,

and those with less education gave higher ratings. CMS evaluated the impact of these variables and

determined that they qualified as case‐mix adjusters for the QHP Enrollee Survey scores.

CMS conducted case‐mix analysis to identify additional case‐mix adjusters to use when scoring data

from the QHP Enrollee Survey by examining how well candidate adjusters predicted variation in

reporting units’ performance and how much reporting units differed on these candidate adjusters. This

information determined which of the candidate adjusters would have the greatest impact on scores

when included in the case‐mix model. The additional case‐mix adjusters that were identified by these

analyses and used to adjust scores displayed in this report are: survey language, survey mode, a variable

indicating whether another person helped the respondent complete the questionnaire, and a

constructed variable used to indicate the presence of chronic conditions. Exhibit 4 shows the variables

selected for use in the case‐mix adjustment , and the options selected as the reference standard.

59

Exhibit 4. Case‐Mix Adjusters

Case‐mix Variable Survey Question Variable Coding

Education What is the highest grade or level of school that you have completed?

1) 8th grade or less

2) Some high school, but did not graduate

3) High school graduate or GED [REFERENCE]

4) Some college or 2‐year degree

5) 4‐year college graduate

6) More than 4‐year college degree

Global Health Rating In general, how would you rate your overall health?

1) Excellent [REFERENCE]

2) Very good

3) Good

4) Fair

5) Poor

Mental Health Rating In general, how would you rate your overall mental or emotional health?

1) Excellent [REFERENCE]

2) Very good

3) Good

4) Fair

5) Poor

Age What is your age? 1) 18 to 24

2) 25 to 34 [REFERENCE]

3) 35 to 44

4) 45 to 54

5) 55 to 64

6) 65 to 74

7) 75 or older

Survey Language From data collection vendor English [REFERENCE]

Spanish

Chinese

Survey Mode From data collection vendor Mail

Telephone [REFERENCE]

Web

Chronic Conditions and Medications

Q65. In the past 6 months, did you get health care 3 or more times for the same condition or problem?

Q66. Is this a condition or problem that has lasted for at least 3 months?

Q67. Do you now need or take medicine prescribed by a doctor?

Q68. Is this medicine to treat a condition that has lasted for at least 3 months?

0) No care (Q65=no), no medications (Q67=no)

1) Acute care (Q65=yes) and/or acute medications (Q67=yes)

2) Chronic care (Q65=yes and Q66=yes) or chronic medications (Q67=yes and Q68=yes)

3) Chronic care and chronic medications (all 4 questions = yes)

60

Case‐mix Variable Survey Question Variable Coding

Received Help Responding Did someone help you complete this survey?

1) Yes

2) No [REFERENCE]

Statistical Details of Case Mix Adjustment

Case‐mix adjustment is performed as part of the scoring of QHP Enrollee Survey composites, individual

items, and global ratings. Coefficients obtained in linear regression models used for scoring estimate the

tendency of patients to respond more positively or negatively to a given measure. The adjustments

needed to counter that tendency are obtained by multiplying the case‐mix coefficients by (‐1.0). Prior to

adjusting the scores, missing values for each adjuster are imputed by substituting the reporting unit

mean for those missing values. Case‐mix adjustments for the eight composites are shown in Exhibits 5

and 6 (for composites, the adjustments shown are the average of the adjustments across the items that

belong to the composite). Case‐mix adjustments for the five global ratings appear in Exhibit 7. There are

no coefficients for reference categories.

Each reporting unit score is adjusted using a regression model predicting the score that would be

obtained if the case‐mix associated with the reporting unit was equal to the average case mix across all

reporting units nationally. Case‐mix adjusted data are used to compare the scores for each reporting

unit to the national mean scores (see Significance Testing below). The case‐mix adjusted data show how

each reporting unit would compare to other reporting units if the enrollees in each reporting unit had

the same distribution of characteristics with respect to age, education, self‐reported physical and

mental health status, proxy status, survey language and mode, and chronic conditions.

Thus, whether the scores of a given reporting unit are adjusted upward or downward for a given

measure depends not only on these case‐mix adjustments, but also on the case‐mix of that reporting

unit relative to the national average of these case‐mix characteristics. These national means are shown

in Exhibits 8, 9 and 10. Specifically, the total case‐mix adjustment for a given reporting unit is the sum of

a series of products, where each product multiplies the adjustment (Exhibits 5, 6 and 7) by the deviation

of the reporting unit’s mean on the corresponding case‐mix variable from the national mean on that

case‐mix variable (Exhibits 8, 9 and 10).

Exhibit 5. Case‐Mix Regression Adjustments for QHP Survey Composites

Predictor

Getting Care

Quickly

Getting

Needed Care

Access to

Health Plan

Information

Doctors

Coordinate

Care

Education blank blank blank blank

8th grade or less 0.082 0.1 ‐0.141 0.059

Some high school, but did not graduate 0.056 0.008 0.015 ‐0.036

Some college or 2‐year degree 0.004 0.036 0.096 0.05

4‐year college graduate 0.024 0.088 0.112 0.066

More than 4‐year college degree 0.077 0.149 0.259 0.109

Global Health Rating blank blank blank blank

61

Predictor

Getting Care

Quickly

Getting

Needed Care

Access to

Health Plan

Information

Doctors

Coordinate

Care

Very good 0.022 0.07 ‐0.011 0.075

Good 0.117 0.16 0.001 0.121

Fair 0.136 0.245 0.059 0.143

Poor 0.289 0.318 ‐0.006 0.245

Mental Health Rating blank blank blank blank

Very good 0.065 0.075 0.082 0.105

Good 0.114 0.078 0.085 0.125

Fair 0.038 0.129 0.077 0.103

Poor 0.136 0.213 0.232 0.199

Age blank blank blank blank

18 to 24 ‐0.126 ‐0.065 0.058 ‐0.091

35 to 44 ‐0.071 ‐0.063 0.06 ‐0.033

45 to 54 ‐0.073 ‐0.091 0.079 ‐0.137

55 to 64 ‐0.203 ‐0.233 0.027 ‐0.189

65 to 74 ‐0.167 ‐0.179 ‐0.061 ‐0.215

75 or older ‐0.357 ‐0.058 ‐0.258 ‐0.159

Survey Language blank blank blank blank

Spanish ‐0.301 ‐0.267 ‐0.152 0.071

Chinese 1.105 0.906 0.635 0.737

Survey Mode blank blank blank blank

Web ‐0.002 0.071 0.022 0.079

Mail ‐0.023 ‐0.019 0.023 0.074

Chronic Condition and Medications blank blank blank blank

blank ‐0.097 ‐0.044 ‐0.051 ‐0.03

Received Help Completing Survey blank blank blank blank

Yes 0.189 0.063 0.072 0.053

Exhibit 6. Case‐Mix Regression Adjustments for QHP Survey Composites ‐ cont.

Predictor

Information

Language or

Format

Doctors

Communicate

Well

Health Plan

Customer

Service

Enrollee

Experience

with Cost


8th grade or less 0.189 0.003 ‐0.02 0.069

Some high school, but did not graduate

0.021 ‐0.024 ‐0.072 0.01


62

Predictor

Information

Language or

Format

Doctors

Communicate

Well

Health Plan

Customer

Service

Enrollee

Experience

with Cost




Very good 0.199 0.072 0.028 0.061

Good 0.273 0.123 0.078 0.139

Fair 0.311 0.172 0.121 0.25

Poor 0.283 0.267 0.201 0.383


Very good 0.17 0.069 0.056 ‐0.007

Good ‐0.219 0.121 0.074 ‐0.001

Fair ‐0.292 0.144 0.002 0.063

Poor ‐0.245 0.171 0.078 0.106


18 to 24 1.061 ‐0.005 ‐0.136 ‐0.019

35 to 44 0.428 0.011 ‐0.092 ‐0.029

45 to 54 0.45 ‐0.025 ‐0.125 ‐0.014

55 to 64 0.341 ‐0.07 ‐0.222 ‐0.046

65 to 74 0.473 ‐0.061 ‐0.253 ‐0.1

75 or older 1.06 ‐0.063 ‐0.176 ‐0.127


Spanish ‐0.146 ‐0.013 ‐0.07 0.088

Chinese 0.78 0.842 0.54 0.139


Web ‐0.146 0.034 0.103 ‐0.064

Mail ‐0.161 0.032 0.079 ‐0.048


blank ‐0.086 ‐0.024 ‐0.004 0.029


Yes 0.411 0.065 ‐0.004 ‐0.066

Exhibit 7. Case‐Mix Regression Adjustments for QHP Survey Global Ratings

Predictor

Rating of

Health Care

Rating of

Personal

Doctor

Rating of

Specialist

Rating of

Health Plan

Recommend

to Friends

and Family

Education blank blank blank blank blank

8th grade or less ‐0.166 ‐0.011 ‐0.148 ‐0.524 1.53

63

Predictor

Rating of

Health Care

Rating of

Personal

Doctor

Rating of

Specialist

Rating of

Health Plan

Recommend

to Friends

and Family


‐0.093 ‐0.109 ‐0.073 ‐0.37 0.119

Some college or 2‐year degree 0.204 0.11 0.124 0.331 0.122

4‐year college graduate 0.253 0.193 0.248 0.525 0.264

More than 4‐year college degree 0.418 0.305 0.398 0.832 0.822

Global Health Rating blank blank blank blank blank

Very good 0.253 0.246 0.175 0.175 ‐0.436

Good 0.604 0.365 0.325 0.458 ‐0.177

Fair 0.94 0.578 0.522 0.71 ‐0.232

Poor 1.226 0.754 0.768 1.314 ‐0.25

Mental Health Rating blank blank blank blank blank

Very good 0.108 0.176 0.159 0.095 ‐0.733

Good 0.2 0.295 0.288 0.106 ‐0.946

Fair 0.326 0.311 0.43 0.255 0.473

Poor 0.485 0.332 0.228 0.712 0.296

Age blank blank blank blank blank

18 to 24 ‐0.222 ‐0.081 0.342 ‐0.369 ‐0.444

35 to 44 ‐0.192 ‐0.121 ‐0.05 ‐0.167 ‐0.42

45 to 54 ‐0.259 ‐0.21 ‐0.266 ‐0.143 0.348

55 to 64 ‐0.519 ‐0.328 ‐0.414 ‐0.27 0.61

65 to 74 ‐0.645 ‐0.333 ‐0.47 ‐0.569 0.982

75 or older ‐0.975 ‐0.681 ‐0.957 ‐1.095 0.173

Survey Language blank blank blank blank blank

Spanish ‐0.804 ‐0.419 ‐0.265 ‐0.652 ‐0.22

Chinese 1.122 1.217 1.298 0.158 0.487

Survey Mode blank blank blank blank blank

Web 0.086 0.189 0.361 0.388 ‐0.026

Mail ‐0.048 0.147 0.297 0.2 ‐1.326

Chronic Condition and Medications

blank blank blank blank blank

blank ‐0.107 ‐0.065 ‐0.05 ‐0.227 0.753

Received Help Completing Survey blank blank blank blank blank

Yes 0.09 0.016 ‐0.052 0.052 ‐0.544

64

Exhibit 8. Case‐Mix Adjusters – National Mean Values

Variable

Getting Care

Quickly

Getting

Needed Care

Access to

Health Plan

Information

Doctors

Coordinate

Care


8th grade or less 0.0236 0.022 0.0143 0.0254

Some high school, but did not graduate 0.0482 0.0459 0.0314 0.05

High school graduate or GED 0.2666 0.2626 0.2069 0.2789





Excellent 0.1359 0.1356 0.1487 0.1253

Very good 0.3509 0.3535 0.3649 0.3425

Good 0.3579 0.3573 0.3424 0.3689

Fair 0.133 0.1316 0.1223 0.1402

Poor 0.0223 0.022 0.0217 0.0231


Excellent 0.3067 0.3051 0.3085 0.2996

Very good 0.3484 0.3506 0.3522 0.3469

Good 0.2454 0.2452 0.2399 0.2521

Fair 0.0844 0.0842 0.0833 0.0866

Poor 0.0151 0.0148 0.0161 0.0149


18 to 24 0.0224 0.0228 0.0228 0.0176

25 to 34 0.1075 0.1079 0.1357 0.0867

35 to 44 0.1088 0.1068 0.1151 0.1002

45 to 54 0.2037 0.2015 0.1956 0.2017

55 to 64 0.5335 0.537 0.5098 0.5676

65 to 74 0.0218 0.0219 0.019 0.0239

75 or older 0.0022 0.0022 0.0021 0.0023


English 0.977 0.9789 0.984 0.9761

Spanish 0.0227 0.0208 0.0157 0.0236

Chinese 0.0003 0.0003 0.0003 0.0003


Web 0.1273 0.1267 0.1609 0.1209

Mail 0.6749 0.6793 0.6313 0.688

65

Variable

Getting Care

Quickly

Getting

Needed Care

Access to

Health Plan

Information

Doctors

Coordinate

Care

Telephone 0.1977 0.194 0.2078 0.1911


Average value 1.6536 1.6677 1.5814 1.7636


Yes 0.0425 0.0412 0.0349 0.0422


Variable

Information

Language or

Format

Doctors

Communicate

Well

Health Plan

Customer

Service

Enrollee

Experience

with Cost


8th grade or less 0.0527 0.0254 0.0203 0.0247

Some high school, but did not graduate 0.0615 0.0499 0.0401 0.0497

High school graduate or GED 0.2413 0.2788 0.2308 0.2746





Excellent 0.1569 0.1253 0.1508 0.1614

Very good 0.3381 0.3425 0.359 0.3629

Good 0.3394 0.3691 0.3403 0.3411

Fair 0.1395 0.1401 0.1278 0.1159

Poor 0.026 0.0231 0.022 0.0187


Excellent 0.3009 0.2996 0.3126 0.3238

Very good 0.3332 0.347 0.346 0.349

Good 0.2564 0.252 0.2422 0.2379

Fair 0.0924 0.0867 0.0839 0.076

Poor 0.017 0.0148 0.0153 0.0133


18 to 24 0.0295 0.0177 0.0215 0.0278

25 to 34 0.1369 0.0869 0.1187 0.1192

35 to 44 0.129 0.1003 0.1115 0.1133

45 to 54 0.2166 0.2015 0.1971 0.2035

55 to 64 0.4552 0.5675 0.5273 0.5127

65 to 74 0.0291 0.0239 0.0218 0.0215

66

Variable

Information

Language or

Format

Doctors

Communicate

Well

Health Plan

Customer

Service

Enrollee

Experience

with Cost

75 or older 0.0037 0.0023 0.0022 0.002


English 0.9316 0.976 0.9759 0.975

Spanish 0.0676 0.0236 0.0237 0.0246

Chinese 0.0009 0.0004 0.0004 0.0005


Web 0.1188 0.121 0.1422 0.1168

Mail 0.5751 0.6877 0.6615 0.6718

Telephone 0.3061 0.1913 0.1964 0.2114


Average value 1.5002 1.7629 1.608 1.3839


Yes 0.068 0.0421 0.0376 0.0424


Variable

Rating of

Health Care

Rating of

Personal

Doctor

Rating of

Specialist

Rating of

Health Plan

Recommend

to Friends

and Family

Education blank blank blank blank blank

8th grade or less 0.0218 0.0253 0.0175 0.0241 0.0243


0.0452 0.0497 0.0393 0.0492 0.0496

High school graduate or GED 0.2611 0.2786 0.2398 0.2746 0.2754

Some college or 2‐year degree 0.3351 0.3338 0.3353 0.3273 0.327

4‐year college graduate 0.1836 0.1699 0.1952 0.1819 0.1813

More than 4‐year college degree 0.1533 0.1427 0.1729 0.1429 0.1426

Global Health Rating blank blank blank blank blank

Excellent 0.133 0.1254 0.1168 0.1612 0.1617

Very good 0.3534 0.3427 0.3364 0.3637 0.3637

Good 0.3595 0.3688 0.3631 0.3412 0.3407

Fair 0.132 0.1401 0.1547 0.1155 0.1153

Poor 0.0222 0.023 0.029 0.0185 0.0187

Mental Health Rating blank blank blank blank blank

Excellent 0.3034 0.2999 0.2929 0.3238 0.3241

Very good 0.3511 0.3469 0.3487 0.3497 0.3497

Good 0.2461 0.252 0.2489 0.2374 0.2373

67

Variable

Rating of

Health Care

Rating of

Personal

Doctor

Rating of

Specialist

Rating of

Health Plan

Recommend

to Friends

and Family

Fair 0.0847 0.0865 0.093 0.0759 0.0757

Poor 0.0148 0.0148 0.0165 0.0131 0.0132

Age blank blank blank blank blank

18 to 24 0.0224 0.0176 0.0182 0.028 0.028

25 to 34 0.1081 0.0867 0.0974 0.1198 0.1197

35 to 44 0.1068 0.1002 0.0982 0.113 0.1132

45 to 54 0.2015 0.2017 0.1985 0.2035 0.2036

55 to 64 0.5375 0.5675 0.5628 0.5124 0.5121

65 to 74 0.0215 0.0239 0.0226 0.0213 0.0213

75 or older 0.0022 0.0023 0.0025 0.002 0.002

Survey Language blank blank blank blank blank

English 0.9795 0.9761 0.9826 0.9757 0.9756

Spanish 0.0203 0.0236 0.0173 0.0239 0.0239

Chinese 0.0003 0.0003 0.0001 0.0005 0.0004

Survey Mode blank blank blank blank blank

Web 0.1285 0.1214 0.1363 0.1176 0.1165

Mail 0.679 0.6867 0.6846 0.677 0.6759

Telephone 0.1925 0.1919 0.1791 0.2053 0.2076

Chronic Condition and Medications

blank blank blank blank blank

Average value 1.6803 1.764 1.886 1.3866 1.3801

Received Help Completing Survey blank blank blank blank blank

Yes 0.0408 0.0422 0.0403 0.0426 0.0423

Exhibits 11‐53 show the unadjusted and adjusted mean scores in the original scale of the measure, and a

0‐100 score. The “Unadjusted Mean” in the first column is the mean, or average, of responses from

eligible cases from each reporting unit, using the original response scale. The “Adjusted Mean” in the

second column differs from the first column in that the mean is both weighted (using sampling weights)

and case‐mix adjusted. The “0‐100 Score” in the final column takes the adjusted mean from the second

column and expands it to the same 0‐100 scale that is used for public reporting of CAHPS measures. The

linear expansion makes 0 the worst possible score and 100 the best possible score and uses the formula

y=100*(x‐a)/(b‐a), where y is the 0‐100 score, x is the adjusted score on the original scale, b is the

highest possible score on the original scale, and a is the lowest possible score on the original scale. For

example, if a given reporting unit has a mean score of 3.43 on a how‐often type measure with a

minimum possible value of 1 and a maximum possible value of 4, the conversion would look like this:

100*(3.45 – 1)/(4 – 1) = 81.67

68

Due to rounding, scores in other sections of this report may differ slightly from those in this table.

Exhibit 11. Adjusted and Unadjusted Means, and 0–100 Scores for Q10: Overall Rating of All Health

Care

Item Unadjusted Mean

Adjusted Mean

0‐100 Score

Rocky Mountain HMO ‐ PPO 8.44 8.27 82.74

Rocky Mountain HMO ‐ HMO 8.07 8.04 80.45

Colorado Choice Health Plans ‐ HMO 7.93 7.85 78.52

HMO Colorado, Inc., dba HMO Nevada ‐ HMO 7.90 7.80 77.99

Kaiser Foundation Health Plan of Colo. ‐ HMO 7.84 7.78 77.79

Humana Health Plan ‐ HMO 7.39 7.50 74.95

Exhibit 12. Adjusted and Unadjusted Means, and 0–100 Scores for Q26: Overall Rating of Personal

Doctor


Adjusted Mean

0‐100 Score







Exhibit 13. Adjusted and Unadjusted Means, and 0–100 Scores for Q35: Overall Rating of Specialist


Adjusted Mean

0‐100 Score







Exhibit 14. Adjusted and Unadjusted Means, and 0–100 Scores for Q52: Overall Rating of Health Plan


Adjusted Mean

0‐100 Score


69


Adjusted Mean

0‐100 Score






Exhibit 15. Adjusted and Unadjusted Means, and 0–100 Scores for Q53: Global Rating: Recommend to

Family and Friends


Adjusted Mean

0‐100 Score







Exhibit 16. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Getting Care Quickly


Adjusted Mean

0‐100 Score







Exhibit 17. Adjusted and Unadjusted Means, and 0–100 Scores for Q4: Member Got Urgent Care as

Soon as Needed


Adjusted Mean

0‐100 Score






70


Adjusted Mean

0‐100 Score


Exhibit 18. Adjusted and Unadjusted Means, and 0–100 Scores for Q6: Member Got Routine Care as

Soon as Needed


Adjusted Mean

0‐100 Score







Exhibit 19. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Getting Needed Care


Adjusted Mean

0‐100 Score







Exhibit 20. Adjusted and Unadjusted Means, and 0–100 Scores for Q11: Member Got Care, Tests, Tx


Adjusted Mean

0‐100 Score







71

Exhibit 21. Adjusted and Unadjusted Means, and 0–100 Scores for Q33: Member Got Specialist Appt

as Soon as Needed


Adjusted Mean

0‐100 Score







Exhibit 22. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: How Well Doctors

Communicate


Adjusted Mean

0‐100 Score







Exhibit 23. Adjusted and Unadjusted Means, and 0–100 Scores for Q16: Doctor Explained Things


Adjusted Mean

0‐100 Score







Exhibit 24. Adjusted and Unadjusted Means, and 0–100 Scores for Q17: Doctor Listened Carefully


Adjusted Mean

0‐100 Score




72


Adjusted Mean

0‐100 Score




Exhibit 25. Adjusted and Unadjusted Means, and 0–100 Scores for Q18: Doctor Showed Respect


Adjusted Mean

0‐100 Score







Exhibit 26. Adjusted and Unadjusted Means, and 0–100 Scores for Q19: Doctor Spend Enough Time


Adjusted Mean

0‐100 Score







Exhibit 27. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: How Well Doctors

Coordinate Care and Keep Patients Informed


Adjusted Mean

0‐100 Score







73

Exhibit 28. Adjusted and Unadjusted Means, and 0–100 Scores for Q20: Medical Records


Adjusted Mean

0‐100 Score







Exhibit 29. Adjusted and Unadjusted Means, and 0–100 Scores for Q22: Follow‐Up


Adjusted Mean

0‐100 Score







Exhibit 30. Adjusted and Unadjusted Means, and 0–100 Scores for Q23: Got Results


Adjusted Mean

0‐100 Score







Exhibit 31. Adjusted and Unadjusted Means, and 0–100 Scores for Q25: Informed and Up‐to‐Date


Adjusted Mean

0‐100 Score





74


Adjusted Mean

0‐100 Score



Exhibit 32. Adjusted and Unadjusted Means, and 0–100 Scores for Q28: Prescription Meds


Adjusted Mean

0‐100 Score







Exhibit 33. Adjusted and Unadjusted Means, and 0–100 Scores for Q31: Got Help Needed


Adjusted Mean

0‐100 Score







Exhibit 34. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Getting Information

About the Health Plan and Costs of Care


Adjusted Mean

0‐100 Score







75

Exhibit 35. Adjusted and Unadjusted Means, and 0–100 Scores for Q37: Written Materials or Internet


Adjusted Mean

0‐100 Score







Exhibit 36. Adjusted and Unadjusted Means, and 0–100 Scores for Q39: Have to Pay for Service or

Equipment


Adjusted Mean

0‐100 Score







Exhibit 37. Adjusted and Unadjusted Means, and 0–100 Scores for Q41: Have to Pay for Prescription

Meds


Adjusted Mean

0‐100 Score







Exhibit 38. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Health Plan Customer

Service


Adjusted Mean

0‐100 Score



76


Adjusted Mean

0‐100 Score





Exhibit 39. Adjusted and Unadjusted Means, and 0–100 Scores for Q43: Health Plan Customer Service

Gave Help Needed


Adjusted Mean

0‐100 Score







Exhibit 40. Adjusted and Unadjusted Means, and 0–100 Scores for Q44: Health Plan Customer Service

Treated with Respect


Adjusted Mean

0‐100 Score







Exhibit 41. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Getting Information in a

Needed Language or Format


Adjusted Mean

0‐100 Score






77


Adjusted Mean

0‐100 Score


Exhibit 42. Adjusted and Unadjusted Means, and 0–100 Scores for Q13: Got Interpreter


Adjusted Mean

0‐100 Score

Colorado Choice Health Plans ‐ HMO — — —

HMO Colorado, Inc., dba HMO Nevada ‐ HMO — — —

Humana Health Plan ‐ HMO — — —

Kaiser Foundation Health Plan of Colo. ‐ HMO — — —

Rocky Mountain HMO ‐ PPO — — —

Rocky Mountain HMO ‐ HMO — — —

Exhibit 43. Adjusted and Unadjusted Means, and 0–100 Scores for Q49: Forms in Preferred Language


Adjusted Mean

0‐100 Score







Exhibit 44. Adjusted and Unadjusted Means, and 0–100 Scores for Q51: Forms in Format Needed


Adjusted Mean

0‐100 Score

Colorado Choice Health Plans ‐ HMO — — —

HMO Colorado, Inc., dba HMO Nevada ‐ HMO — — —

Humana Health Plan ‐ HMO — — —

Kaiser Foundation Health Plan of Colo. ‐ HMO — — —

Rocky Mountain HMO ‐ PPO — — —

Rocky Mountain HMO ‐ HMO — — —

78

Exhibit 45. Adjusted and Unadjusted Means, and 0–100 Scores for Composite: Enrollee Experience

With Cost


Adjusted Mean

ML Adjusted Mean

0‐100 Score

Rocky Mountain HMO ‐ PPO 3.42 3.38 3.38 79.34

Rocky Mountain HMO ‐ HMO 3.39 3.35 3.36 78.37

Kaiser Foundation Health Plan of Colo. ‐ HMO 3.30 3.29 3.34 76.30

Humana Health Plan ‐ HMO 3.26 3.28 3.26 76.13

Colorado Choice Health Plans ‐ HMO 3.26 3.24 3.32 74.78

HMO Colorado, Inc., dba HMO Nevada ‐ HMO 3.24 3.20 3.28 73.22

Exhibit 46. Adjusted and Unadjusted Means, and 0–100 Scores for Q54: Payment for Necessary

Services


Adjusted Mean

ML Adjusted Mean

0‐100 Score







Exhibit 47. Adjusted and Unadjusted Means, and 0–100 Scores for Q55: Out of Pocket


Adjusted Mean

ML Adjusted Mean

0‐100 Score







79

Exhibit 48. Adjusted and Unadjusted Means, and 0–100 Scores for Q56: Decided Not to Get Care


Adjusted Mean

ML Adjusted Mean

0‐100 Score







Exhibit 49. Adjusted and Unadjusted Means, and 0–100 Scores for Q57: Did Not Fill Prescription


Adjusted Mean

ML Adjusted Mean

0‐100 Score







Exhibit 50. Adjusted and Unadjusted Means, and 0–100 Scores for Q8: After Hours Care


Adjusted Mean

0‐100 Score







Exhibit 51. Adjusted and Unadjusted Means, and 0–100 Scores for Q45: Customer Service Takes a Long

Time


Adjusted Mean

0‐100 Score



80


Adjusted Mean

0‐100 Score





Exhibit 52. Adjusted and Unadjusted Means, and 0–100 Scores for Q47: Forms Easy to Fill Out


Adjusted Mean

0‐100 Score







Exhibit 53. Adjusted and Unadjusted Means, and 0–100 Scores for Q48: Purpose of Forms Explained


Adjusted Mean

0‐100 Score







Significance Testing

Two‐tailed t‐tests were used to assess whether the case‐mix adjusted mean for each reporting unit

differed significantly from the overall mean for all reporting units in the nation. Reporting unit scores

that are significantly different from the national mean at the p<0.05 level are indicated. In accordance

with confidentiality requirements, scores are not reported for any item or composite with fewer than 11

observations. If one reporting unit has a higher score but performs worse in the statistical test than

plans with similar scores, it may be because the reporting unit has a small sample size or large variance,

or both.

81

Assessing Reliability of Scores

For each item or composite reported here, CMS calculated interunit reliability (IUR). Interunit reliability

is the average reliability of a measure across all reporting units, and calculated as: IUR = b/(b + v ̄ / n̄ )

where b is the between‐unit variance, n̄ is the average number of usable observations per reporting unit

(RU), and v ̄ is the average within‐unit variance among all RUs. The resulting number has a range from 0

to 1 and summarizes the fraction of variation in RU mean scores that is attributable to actual differences

among RUs (“signal”) rather than random variation due to sampling of individuals (“noise”). An IUR close

to 1 indicates that sampling variability is negligible, while IUR close to 0 means that all observed

differences among RUs are due to sampling variation, and thus the measure does not distinguish quality

among RUs.

In order to provide as much information as possible in this report, we have chosen not to flag or

suppress measures based on reliability.

Exhibit 54 shows the IUR for each composite, item and global rating. However, regardless of the level of

reliability, scores are not reported for measures with fewer than 11 responses.

Exhibit 54. Interunit Reliability (IUR)

Item

Total

Observations

Used in Scoring

Average Number

of Observations

per Reporting Unit IUR

q10: Overall Rating of All Health Care 51984 169 0.78

q26: Overall Rating of Personal Doctor 44231 143 0.65

q35: Overall Rating of Specialist 29765 97 0.44

q52: Overall Rating of Health Plan 73159 237 0.91

Q53: Global Rating 73421 238 0.92

Composite: Getting Care Quickly 52855 171 0.78

q04:Member Got Urgent Care as Soon as Needed 22040 72 0.65

q06:Member Got Routine Care as Soon as Needed 47785 155 0.74

Composite: Getting Needed Care 54579 177 0.82

q11:Member Got Care‐Tests‐Tx 51886 168 0.84

q33:Member Got Specialist Appt as Soon as Needed 30692 100 0.67

Composite: How Well Doctors Communicate 44553 144 0.66

q16:Dr Explained Things 44410 144 0.66

q17:Dr Listened Carefully 44347 144 0.59

q18:Dr Showed Respect 44378 144 0.53

q19:Dr Spent Enough Time 44329 143 0.6

Composite: How Well Doctors Coordinate Care and Keep Patients Informed 44554 144 0.68

q20:Medical Records 44201 143 0.69

q22:Follow‐Up 35434 115 0.74

82

Item

Total

Observations

Used in Scoring

Average Number

of Observations

per Reporting Unit IUR

q23:Got Results 35464 115 0.68

q25:Informed and Up‐to‐Date 22245 72 0.55

q28:Prescription Meds 38939 126 0.56

q31:Got Help Needed 6026 20 0.37

Composite: Health Plan Customer Service 35271 115 0.66

q37:Written Materials or Internet 27842 91 0.69

q39:Have to Pay for Service or Equipment 12240 40 0.49

q41:Have to Pay for Prescription Meds 14038 46 0.49

Composite: Getting Information About the Health Plan and Costs of Care 29185 95 0.84

q43:HP Customer Service Gave Help Needed 29006 95 0.83

q44:HP Customer Service Treated with Respect 29047 95 0.77

Composite: Getting Information in a Needed Format or Language 14069 46 0.75

q13:Got Interpreter 1370 7 0.18

q49:Forms in Preferred Language 13116 43 0.73

q51:Forms in Format Needed 395 3 0.18

Composite: Enrollee Experience With Cost 73774 239 0.87

q54: Payment for Necessary Services 70008 227 0.72

q55: Out of Pocket 71074 230 0.85

q56: Decided Not to Get Care 72976 236 0.87

q57: Did Not Fill Prescription 72464 235 0.75

Reliability of the scores is affected by a number of other factors, including the fraction of the reporting

unit’s respondents who are eligible to answer an item based on their experiences, the variability of

responses within the reporting unit, and the amount by which reporting units differ from each other

nationally on that measure. Reliability summarizes the influence of these factors on the precision of the

comparisons of a reporting unit’s score to national distributions. Measures with higher reliability are

better at detecting differences in performance among reporting units, and thus will tend to classify a

relatively large number of reporting units as either above or below average, even when differences

among units appear small. Measures with lower reliability are not as sensitive in detecting differences in

performance among reporting units, and will tend to produce results where the vast majority of

reporting units are classified as average, even when differences among reporting units appear large.

Within a given measure, reporting units that show low‐reliability for that measure (Ri) typically are those

with fewer respondents; and low reliability could also reflect greater variability in responses from

enrollees in that reporting unit. Across measures, more low‐reliability scores will be reported for

measures with fewer responses (more respondents for whom the measure does not apply), less

variation in scores across reporting units, and more variability in scores within each reporting unit.

83

Relationship Between QHP Enrollee Survey and Quality Rating System (QRS)

Section 1311(c)(3) of the ACA directs the Secretary of Health and Human Services (HHS) to develop a

quality rating system for QHPs based on quality and price. CMS requires that QHP issuers report health

care quality information for its QHPs offered through a Marketplace as a condition of certification and

participation in the Marketplace.

The survey results in this report are designed to match CAHPS measures and composites as closely as

possible, to allow QHP issuers to assess quality across different products. Presentations in this report

may not align with the QRS. While QRS ratings are based on a combination of survey, administrative and

clinical data to produce ratings that consumers can use to make informed choices about which health

plan to purchase through the Marketplace, the survey results in this report are designed to match

CAHPS measures and composites as closely as possible, to allow QHP issuers to assess quality across

different products.

The QRS measure set consists of measures that address areas of clinical quality management; enrollee

experience; and plan efficiency, affordability, and management. The survey measures in the QRS

measure set are taken from the QHP Enrollee Survey as described in Exhibit 3, above, and include

composites on access to care, care coordination and other topics. QHP issuers also will collect data for

QRS clinical measures using administrative sources or a hybrid of administrative and medical record

sources. These include preventive care, screenings and other topics. CMS will calculate QRS scores and

ratings and provide the summary‐level survey scores and overall 5‐star rating to QHP issuers. In 2017,

for the 2018 open enrollment period, QRS results will be publicly reported on Marketplace websites to

help consumers compare and choose QHPs. A beta test in six states is planned for 2016.

Detailed measure specifications for the QRS can be found in “QRS Measure Technical Specifications,”

which can be found on the CMS website at http://www.cms.gov/Medicare/Quality‐Initiatives‐Patient‐

Assessment‐Instruments/QualityInitiativesGenInfo/Health‐Insurance‐Marketplace‐Quality‐

Initiatives.html.

84

Appendix 1: Tabular Display of Results

85

Table 1. Composite: Getting Care Quickly

blank N Never or

Sometimes Usually Always

Average (0‐100)

Higher Is Better

How Do Reporting Units in Your Marketplace

Compare to the National Average? (p < .05)

National Average 52855 19.42% 24.09% 56.49% 76.96 —


290 13.72% 22.71% 63.57% 82.32 above average


218 20.37% 20.27% 59.36% 78.45 average


254 23.68% 20.76% 55.55% 75.75 average


142 21.14% 27.28% 51.58% 75.52 average


260 21.74% 30.92% 47.34% 73.39 average


136 26.86% 24.91% 48.23% 71.74 average

Table 2. Getting Needed Care Right Away

blank N Never or


Average (0‐100)

Higher Is Better





125 10.76% 20.46% 68.78% 85.07 above average


71 — 15.73% 67.70% 82.69 average


44 — 26.64% 53.67% 76.75 average


110 21.93% 22.75% 55.33% 75.89 average


91 20.62% 31.77% 47.60% 73.64 average


70 26.00% 23.30% 50.70% 72.33 average

86

Table 3. Getting Routine Care

blank N Never or


Average (0‐100)

Higher Is Better





260 16.71% 24.95% 58.35% 79.55 above average


223 25.46% 18.77% 55.77% 75.59 average


133 22.61% 27.92% 49.48% 74.27 average


204 24.18% 24.82% 51.00% 74.20 average


241 22.88% 30.06% 47.06% 73.14 average


122 27.74% 26.52% 45.75% 71.15 average

Table 4. Composite: Getting Needed Care

blank N Never or


Average (0‐100)

Higher Is Better





143 16.12% 22.63% 61.25% 80.85 average


237 14.28% 27.65% 58.06% 80.47 average


313 12.11% 32.49% 55.40% 80.11 average


265 15.63% 32.39% 51.98% 77.59 average


271 22.18% 24.98% 52.84% 75.05 average


148 27.91% 33.83% 38.27% 67.35 below average

87

Table 5. Easy to Get Care

blank N Never or


Average (0‐100)

Higher Is Better





231 10.59% 28.09% 61.33% 83.33 above average


297 11.38% 32.96% 55.66% 80.83 average


139 16.80% 25.93% 57.27% 78.92 average


261 21.13% 24.49% 54.38% 76.35 average


247 16.39% 40.31% 43.30% 74.72 below average


141 23.17% 34.09% 42.74% 71.35 below average

Table 6. Appointments With Specialists

blank N Never or


Average (0‐100)

Higher Is Better





82 15.46% 19.32% 65.22% 82.77 above average


143 14.87% 24.47% 60.66% 80.44 average


186 12.86% 32.00% 55.14% 79.38 average


141 17.99% 27.21% 54.79% 77.60 average


148 23.24% 25.47% 51.29% 73.74 average


82 32.66% 33.55% 33.79% 63.34 below average

88

Table 7. Composite: Getting Information in a Needed Language or Format

blank N Never or


Average (0‐100)

Higher Is Better





89 — — 99.04% 90.05 above average


25 — — 85.06% 86.62 above average


71 13.53% — 72.00% 83.01 above average


46 22.95% — 66.91% 73.05 average


32 — — 61.05% 70.14 average


117 50.11% — 48.83% 51.32 below average

Table 8. Needing an Interpreter

blank N Never or


Average (0‐100)

Higher Is Better





— — — — — —


— — — — — —


— — — — — —


— — — — — —


— — — — — —


— — — — — —

89

Table 9. Language of Choice

blank N Never or


Average (0‐100)

Higher Is Better





30 — — 97.06% 95.48 above average


112 — — 93.73% 94.67 above average


89 — — 91.21% 93.29 above average


25 — — 90.59% 92.75 above average


46 — — 88.21% 92.09 average


65 — — 83.07% 88.59 average

Table 10. Format of Choice

blank N Never or


Average (0‐100)

Higher Is Better





— — — — — —


— — — — — —


— — — — — —


— — — — — —


— — — — — —


— — — — — —

90

Table 11. Composite: How Well Doctors Communicate

blank N Never or


Average (0‐100)

Higher Is Better





214 — 16.53% 78.32% 90.87 average


198 6.66% 16.78% 76.56% 89.77 average


117 — 18.16% 76.61% 89.68 average


221 5.93% 19.82% 74.25% 89.34 average


119 8.84% 14.62% 76.54% 88.41 average


200 9.81% 18.78% 71.41% 86.71 average

Table 12. Doctors Easy To Understand

blank N Never or


Average (0‐100)

Higher Is Better





219 — 19.80% 76.86% 91.15 average


116 — 18.60% 77.99% 90.99 average


214 — 19.41% 76.41% 90.36 average


198 6.49% 17.60% 75.92% 89.63 average


117 — 13.95% 78.22% 89.60 average


199 9.81% 19.17% 71.02% 86.62 below average

91

Table 13. Doctors Listen Carefully

blank N Never or


Average (0‐100)

Higher Is Better





196 — 15.44% 78.82% 91.03 average


214 — 18.41% 76.63% 90.46 average


115 — 15.52% 78.56% 90.20 average


220 6.05% 18.33% 75.62% 89.87 average


117 9.87% 12.21% 77.91% 88.65 average


199 10.48% 21.05% 68.46% 85.61 below average

Table 14. Doctors Show Respect

blank N Never or


Average (0‐100)

Higher Is Better





213 — 9.00% 86.58% 93.94 average


198 — 11.88% 82.14% 92.04 average


114 — 13.27% 81.62% 91.45 average


221 5.83% 15.93% 78.25% 90.59 average


198 6.82% 15.49% 77.69% 89.90 average


118 8.58% 12.31% 79.12% 89.26 average

92

Table 15. Doctors Spend Enough Time

blank N Never or


Average (0‐100)

Higher Is Better





213 7.02% 19.31% 73.67% 88.71 average


197 8.43% 22.19% 69.38% 86.36 average


117 9.08% 20.01% 70.92% 86.11 average


115 — 25.25% 68.28% 86.09 average


219 8.52% 25.21% 66.27% 85.74 average


198 12.12% 19.42% 68.46% 84.73 average

Table 16. Composite: How Well Doctors Coordinate Care and Keep Patients Informed

blank N Never or


Average (0‐100)

Higher Is Better





199 15.08% 17.83% 67.09% 82.63 average


214 15.61% 20.28% 64.11% 80.70 average


221 16.29% 23.26% 60.45% 80.36 average


121 15.09% 24.29% 60.62% 79.69 average


199 16.22% 21.82% 61.95% 79.65 below average


118 16.70% 21.45% 61.85% 79.56 average

93

Table 17. Medical Records

blank N Never or


Average (0‐100)

Higher Is Better





196 — 13.22% 82.23% 92.24 average


115 — 16.94% 80.22% 92.21 average


218 — 13.68% 81.13% 91.98 average


198 5.41% 14.74% 79.85% 90.91 average


211 5.83% 15.21% 78.96% 90.54 average


118 — 16.48% 77.74% 89.93 average

Table 18. Test Results

blank N Never or


Average (0‐100)

Higher Is Better





92 14.28% 16.77% 68.94% 82.94 average


167 13.43% 18.37% 68.20% 82.26 average


91 12.72% 24.18% 63.10% 81.41 average


145 16.76% 18.65% 64.59% 80.35 average


163 16.70% 19.06% 64.24% 80.12 average


158 19.57% 16.28% 64.15% 78.59 average

94

Table 19. Up to Date

blank N Never or


Average (0‐100)

Higher Is Better





100 27.20% 16.74% 56.06% 73.18 average


99 22.42% 25.68% 51.90% 71.41 average


96 21.40% 36.28% 42.32% 71.04 average


56 21.15% 34.23% 44.62% 70.70 average


60 33.44% 22.98% 43.58% 65.14 below average


88 28.64% 31.59% 39.77% 65.13 below average

Table 20. Test Results Quickly

blank N Never or


Average (0‐100)

Higher Is Better





92 — 13.41% 76.15% 88.03 average


159 9.76% 18.93% 71.31% 86.54 average


145 12.29% 15.33% 72.38% 84.97 average


90 11.18% 18.73% 70.09% 84.40 average


167 10.61% 24.09% 65.30% 83.43 average


163 11.66% 31.59% 56.75% 80.59 below average

95

Table 21. Prescription Medication

blank N Never or


Average (0‐100)

Higher Is Better





187 15.71% 16.14% 68.15% 83.18 average


179 13.79% 19.58% 66.63% 82.64 average


173 16.90% 19.72% 63.38% 81.50 average


103 14.21% 25.67% 60.12% 80.28 average


104 21.53% 17.47% 61.00% 77.31 average


167 23.34% 20.33% 56.34% 74.02 below average

Table 22. Getting Help

blank N Never or


Average (0‐100)

Higher Is Better





28 — — 63.51% 81.44 average


22 — — 58.32% 80.00 average


33 — 33.02% 49.46% 75.47 average


28 — — 54.79% 74.80 average


31 — — 43.24% 71.61 average


18 — — — 71.47 average

96

Table 23. Composite: Health Plan Customer Service

blank N Never or


Average (0‐100)

Higher Is Better





145 — 23.02% 71.13% 88.20 above average


182 8.20% 20.59% 71.21% 87.04 above average


167 15.71% 23.07% 61.22% 80.16 average


194 22.86% 25.68% 51.46% 74.46 average


98 34.24% 26.10% 39.66% 65.95 below average


79 35.00% 30.12% 34.88% 63.50 below average

Table 24. Information Needed

blank N Never or


Average (0‐100)

Higher Is Better





145 9.57% 30.64% 59.79% 82.99 above average


179 14.61% 27.92% 57.47% 79.92 above average


165 24.03% 24.04% 51.93% 73.41 average


193 38.89% 29.02% 32.09% 61.30 below average


98 48.11% 25.63% 26.26% 55.22 below average


79 49.85% 24.25% 25.90% 52.72 below average

97

Table 25. Courtesy and Respect

blank N Never or


Average (0‐100)

Higher Is Better





181 — 13.26% 84.95% 94.15 above average


145 — 15.39% 82.48% 93.41 above average


194 6.84% 22.33% 70.84% 87.62 average


167 7.40% 22.10% 70.50% 86.91 average


98 20.36% 26.58% 53.06% 76.67 below average


79 20.15% 35.99% 43.86% 74.29 below average

Table 26. Composite: Access to Information

blank N Never or


Average (0‐100)

Higher Is Better





148 40.33% 30.49% 29.18% 57.81 average


186 42.79% 33.61% 23.59% 54.18 average


205 50.45% 21.85% 27.70% 52.04 average


186 49.30% 29.25% 21.46% 51.60 average


112 50.40% 30.16% 19.44% 50.89 average


117 54.08% 26.24% 19.68% 49.11 average

98

Table 27. Provide Needed Information

blank N Never or


Average (0‐100)

Higher Is Better





120 36.73% 38.27% 25.01% 60.99 average


166 44.22% 35.01% 20.77% 54.61 average


89 50.56% 30.81% 18.63% 53.46 average


148 49.37% 33.94% 16.69% 51.96 below average


138 46.50% 41.35% 12.15% 51.13 below average


103 50.51% 38.92% — 49.90 below average

Table 28. Cost for Services

blank N Never or


Average (0‐100)

Higher Is Better





61 43.14% 29.67% 27.19% 54.25 average


75 45.88% 28.65% 25.47% 51.14 average


66 51.32% 28.49% 20.19% 48.94 average


84 54.19% 22.38% 23.43% 47.23 average


30 56.94% — — 44.86 average


39 67.69% — — 41.35 average

99

Table 29. Cost for Prescriptions

blank N Never or


Average (0‐100)

Higher Is Better





72 35.96% 30.82% 33.22% 60.33 average


54 41.09% 23.52% 35.39% 58.24 average


50 44.00% 25.76% 30.24% 56.12 average


53 43.67% 30.55% — 54.40 average


81 52.91% — 38.96% 54.33 average


56 47.16% 25.29% 27.56% 53.96 average

Table 30. Composite: Enrollee Experience With Cost

blank N Always or Usually Sometimes Never

Average (0‐100)

Higher Is Better





310 14.61% 26.40% 58.99% 79.34 average


429 15.99% 25.57% 58.45% 78.37 average


379 19.06% 22.48% 58.46% 76.30 average


199 17.20% 27.43% 55.37% 76.13 average


390 20.60% 23.54% 55.86% 74.78 below average


196 21.31% 26.87% 51.82% 73.22 below average

100

Table 31. Payment for Necessary Services


Average (0‐100)

Higher Is Better





292 15.61% 23.15% 61.24% 79.73 average


400 15.22% 25.54% 59.23% 79.39 average


182 15.92% 27.25% 56.83% 77.73 average


369 19.86% 19.80% 60.34% 76.53 average


364 21.94% 21.54% 56.52% 74.12 below average


187 22.39% 31.35% 46.27% 70.77 below average

Table 32. Out of Pocket


Average (0‐100)

Higher Is Better





413 16.62% 30.05% 53.33% 76.57 average


294 14.87% 36.84% 48.29% 75.63 average


194 18.21% 28.50% 53.29% 74.92 average


368 20.24% 26.53% 53.23% 73.99 average


374 20.25% 29.82% 49.93% 73.08 below average


186 23.12% 32.27% 44.61% 70.49 below average

101

Table 33. Decided Not to Get Care


Average (0‐100)

Higher Is Better





306 22.18% 25.23% 52.59% 73.63 average


376 24.26% 25.75% 50.00% 70.78 below average


198 24.40% 27.80% 47.79% 69.72 below average


194 26.23% 25.32% 48.45% 69.64 below average


424 25.26% 28.32% 46.42% 69.55 below average


389 27.96% 25.81% 46.23% 67.77 below average

Table 34. Did Not Fill Prescription


Average (0‐100)

Higher Is Better





306 5.76% 20.38% 73.86% 88.37 above average


422 6.84% 18.36% 74.81% 87.96 above average


373 9.79% 16.12% 74.09% 86.31 average


195 10.27% 26.17% 63.56% 82.17 average


191 13.49% 18.55% 67.96% 81.99 average


379 14.32% 18.72% 66.95% 81.77 average

102

Table 35. Single Item After Hours

blank N Never or


Average (0‐100)

Higher Is Better





15 — — 65.15% 85.08 above average


23 — — 65.59% 83.95 above average


14 — — — 76.93 average


24 — — 59.22% 74.28 average


12 — — — 51.54 average


22 62.74% — — 45.06 average

Table 36. Single Item Longer Than Expected


Average (0‐100)

Higher Is Better





142 15.70% 29.73% 54.57% 77.78 above average


183 22.96% 33.35% 43.69% 70.34 above average


165 26.66% 28.33% 45.01% 68.02 above average


193 39.51% 39.94% 20.56% 54.33 average


79 51.46% 31.76% 16.78% 43.08 below average


98 58.78% 29.45% 11.77% 39.25 below average

103

Table 37. Single Item Easy to Fill Out

blank N Never or


Average (0‐100)

Higher Is Better





107 16.77% 45.65% 37.58% 72.64 above average


64 20.52% 44.41% 35.06% 70.13 average


87 33.21% 32.81% 33.97% 65.19 average


29 — 32.80% — 61.75 average


46 31.20% 43.36% 25.44% 60.23 average


30 57.50% — — 55.63 average

Table 38. Single Item Explain Purpose

blank N Never or


Average (0‐100)

Higher Is Better





113 12.73% 24.73% 62.54% 80.50 above average


87 26.46% 25.36% 48.18% 71.69 average


27 — — 42.45% 67.38 average


64 38.25% 27.66% 34.08% 62.13 average


44 34.92% 27.75% 37.33% 61.42 average


30 50.14% — — 56.57 average

104

Table 39. Rating of Health Plan

blank N 0‐6 7‐8 9‐10

Average (0‐100)

Higher Is Better





307 33.71% 32.41% 33.89% 72.64 average


423 40.59% 31.85% 27.56% 69.49 below average


374 40.45% 31.90% 27.64% 68.11 below average


392 40.80% 31.30% 27.90% 66.67 below average


194 44.45% 27.15% 28.40% 65.09 below average


201 45.56% 30.80% 23.63% 62.12 below average

Table 40. Rating of Health Care

blank N 0‐6 7‐8 9‐10

Average (0‐100)

Higher Is Better





232 13.93% 35.21% 50.86% 82.74 average


295 14.82% 41.03% 44.14% 80.45 average


248 19.11% 40.58% 40.32% 78.52 below average


140 19.90% 37.35% 42.75% 77.99 below average


261 20.25% 39.75% 40.00% 77.79 below average


142 19.83% 45.36% 34.81% 74.95 below average

105

Table 41. Rating of Personal Doctors

blank N 0‐6 7‐8 9‐10

Average (0‐100)

Higher Is Better





115 — 28.88% 65.97% 88.27 average


197 8.96% 21.80% 69.24% 87.80 average


218 7.95% 26.38% 65.67% 87.40 average


211 7.84% 27.21% 64.95% 86.15 average


198 11.35% 27.23% 61.42% 85.74 below average


120 9.94% 27.68% 62.37% 85.16 average

Table 42. Rating of Specialists

blank N 0‐6 7‐8 9‐10

Average (0‐100)

Higher Is Better





141 12.29% 17.67% 70.04% 87.62 average


180 11.64% 22.77% 65.59% 86.58 average


136 12.01% 25.79% 62.20% 85.94 average


82 14.04% 16.13% 69.83% 85.68 average


77 13.75% 25.09% 61.16% 83.53 average


145 14.75% 30.89% 54.36% 82.71 below average

106

Table 43. Global Rating

blank N 0‐6 7‐8 9‐10

Average (0‐100)

Higher Is Better





425 36.47% 30.06% 33.47% 69.96 average


309 38.63% 27.80% 33.57% 68.83 average


395 41.24% 24.88% 33.87% 64.73 below average


373 43.74% 24.11% 32.15% 64.39 below average


194 49.96% 24.51% 25.53% 58.57 below average


201 51.76% 24.88% 23.37% 56.48 below average

health plan enrollee experience survey (qhp€¦ · part 1 profiles the content, organization, and...

Documents