medicaid reporting errors in four national surveys: acs, cps, meps, and nhis
DESCRIPTION
Presentation by Brett Fried at the 2014 Joint Statistical MeetingsTRANSCRIPT
Medicaid Reporting Errors in Four National Surveys: ACS, CPS, MEPS, and NHIS
Brett Fried
State Health Access Data Assistance Center (SHADAC)University of Minnesota
JSM, BostonAugust 6, 2014
Acknowledgments
• Funding for this work is supported by the Census Bureau
• Coauthors:– Brett O’Hara: Census Bureau– Kathleen Call, Michel Boudreaux, Joanna
Turner: SHADAC
2
Background
• Administrative data on public assistance programs are not sufficient for policy making– Not always timely– Do not measure percent uninsured– Limited number of high quality covariates– Do not include policy relevant outcome
measures• Population surveys fill these gaps
– Yet they universally undercount public program enrollment described in administrative data 3
The problem• Every survey said to “undercount”
Medicaid enrollment• If counts of Medicaid are biased
downward, counts of other coverage and uninsurance are biased upward
• The size of the undercount varies by survey
• How does the ACS undercount and survey response error compare to other federal surveys?
4
Data and methods
• Four National Surveys– 2009 American Community Survey (ACS)– 2005 Current Population Survey (CPS)– 2002 National Health Institute Survey (NHIS)– 2003 Medical Expenditure Panel
Survey/Household Component (MEPS/HC)• Medicaid Statistical Information System
(MSIS)• Compare enrollment counts: all values• Compare linked data: explicit values only
5
Compare National Surveys: Medicaid Question
6
Reference Period Recall Period
Includes Institutionalized
Pop.
Can exclude state CHIP
ACSPoint in time
(PIT) NA Y N
CPSEver during the
year15-17
months N Y
NHISPoint in time
(PIT) NA N Y
MEPS/HCMonthly/Ever
during the year 3-5 months N N
ALL VALUES
COUNTS PRIOR TO LINKING:
7
Counts: Why do we care about counts?
–What public see when they see estimates of enrollment
– Illustrates the problem of concept alignment
–How the undercount is generally reported
8
Counts: Adjustments to align content between data sources
• Administrative Data:– Exclude most partial benefit Medicaid
enrollees– Remove duplicates and deceased
individuals– Exclude institutionalized (except ACS)– Exclude CHIP & SCHIP (except ACS &
MEPS)• Survey Data:
– CPS & NHIS - Exclude CHIP, SCHIP and other public program enrollees
– ACS and MEPS cannot exclude CHIP, SCHIP or other means tested public program enrollees
9
Counts: Compare to MSIS: All Values
10
Sources: SNACC reports and 2009 American Community Survey and 2009 Medicaid Statistical Information System
Percent different from MSIS
Raw Adjusted
ACS 2009 .2% 16%
CPS 2005 -38% -32%
NHIS 2002 -39% -26%
MEPS/HC 2003 -25% -6%
WHAT PEOPLE ACTUALLY REPORT (EXPLICIT VALUES ONLY)
LINKED FILE RESULTS:
11
Linked data: How do we link the data?
• Use linking methodology developed by the Census Bureau’s Center for Administrative Records Research and Applications – Personal Identification Key (PIK)
• Research conducted at the MN Census Research Data Center located at the University of Minnesota– http://mnrdc.umn.edu/
12
Linked and unlinked data and misreports
13
Accuracy of Medicaid Reporting: What they report in the ACS linked to MSIS
14
Sources: SNACC reports and 2009 ACS and MSIS.
“Correct Reports” “Misreports”(False Negatives)
MedicaidAny Other Coverage
Uninsured
ACS 2009 77% 13% 10%
CPS 2005 59% 23% 17%
NHIS 2002 67% 24% 9%
MEPS/HC 2003 84% 8% 8%
Percent Linked: Age (“correct reports”)
15
Sources: SNACC reports and 2009 ACS and MSIS.• The age groups used for the ACS analyses are slightly different.
0-17 18-64 65+
ACS 2009* 81%(0-18)
72%(19-64)
71%
CPS 2005 62% 56% 60%
NHIS 2002 68% 65% 66%
MEPS/HC 2003 86% 81% 85%
Percent Linked: Percent of poverty (“correct repots”)
16
Sources: SNACC reports and 2009 ACS and MSIS* The percent of poverty categories used for the ACS were slightly different.
0-124% 125-199% 200%+
ACS 2009* 83%(0-138%)
72%(139-200%)
64%(201+%)
CPS 2005 66% 52% 40%
NHIS 2002 73% 56% 41%
MEPS/HC 2003 88% 83% 73%
Linked and unlinked data and misreports
17
Percent of those not linked: Misreports of those who are not linked
18
Sources: SNACC reports and 2009 ACS and MSIS
Percent of those not linked
ACS 2009* unknown
CPS 2005 3%
NHIS 2002 1%
MEPS/HC 2003 unknown
Summary of results• The percent of “misreports” in the ACS
appears in line with other surveys • The percent of “misreports” is lower in all
four surveys for – Children and – Low income individuals
• The percent of “misreports” in all four surveys that are reported as uninsured is lower or equal to the percent reported as other sources of coverage
19
Implications• Care should be taken in making any
adjustments to Medicaid counts because it is important to consider– survey and administrative data universes
and definitions,– “misreports” of uninsurance as compared to
other sources of coverage,– offsetting “misreports” of Medicaid among
those not linked. • ACA health reforms present new
challenges. – Differentiating Medicaid from exchange
enrollment.– Large increase in the number of Medicaid
enrollees.– Linking could help to sort out these changes.
20
