matching the research design to the study question

Matching the Research Design to the Study Question

Lisa Simpson, MD, MPHJuly 25, 2011

Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question Implications for the CER enterprise

2

AcademyHealth: Improving Health & Health Care

As the professional society for health services researchers and health policy analysts, our mission is three-fold:

1. Support the development of health services research

2. Facilitate the use of the best available research and information

3. Assist health policy and practice leaders in addressing major health challenges

We work to both “push” the production of research and promote the “pull” by decision makers

3

In Other Words…

We help build the knowledge base AND

move knowledge into actionthrough programs and services for

research producers and research users.

4

Overview Overview of AcademyHealth Matching the Research to the Need

National Quality Strategy

5

National Quality Strategy

Secretary of HHS mandated by Affordable Care Act to develop a National Strategy for Quality Improvement in Health

Requires strategic plan and implementation metrics for national, state, and local efforts to improve health care quality on common aims, priorities and goals

6

National Quality Strategy: Three Aims

Better Care: Improve the overall quality, by making health care more patient-centered, reliable, accessible, and safe.

Healthy People/Healthy Communities: Improve the health of the U.S. population by supporting proven interventions to address behavioral, social and, environmental determinants of health in addition to delivering higher-quality care.

Affordable Care: Reduce the cost of quality health care for individuals, families, employers, and government.

7

National Quality Strategy: Six Priorities

Safer care Effective care coordination Person and family centered care Prevention and treatment of leading

causes of mortality Supporting better health in communities Making care more affordable

8

NQS and IOM: Priorities in Sync Safer care Effective care

coordination Person and family

centered care Prevention and

treatment of leading causes of mortality

Supporting better health in communities

Making care more affordable

Methods to eradicate MRSA and reduce HAI

Effectiveness of comprehensive care programs

None explicitly “patient-centered”

Primary prevention methods vs. clinical interventions

Effectiveness of dissemination and translation techniques

Reduce healthcare disparities including literacy

Localized prostate cancer management strategies

9

NQS and FCC CER Safer care Effective care

coordination Person and family

centered care

Prevention and treatment of leading causes of mortality

Supporting better health in communities

Making care more affordable

None explicitly on safety

Decision support tools

Focusing on immediate, specific patient needs; family as surrogate decision makers

Prevention

Focus on priority populationsDelivery system strategies,

including community-based models

10

National Quality Strategy Principles

1. Person-centeredness and family engagement

2. Specific health considerations

3. Eliminating disparities in care

4. Aligning the efforts of public and private sectors

5. Quality improvement

6. Consistent national standards

7. Primary care will become a bigger focus

8. Coordination will be enhanced

9. Integration of care delivery

10. Providing clear information

CER focused at the individual level, responsive

to patient needs

CER examining discrete &/or complex

comorbid conditions

Importance of sub group analyses, examination of context, & within group

heterogeneity

Enterprise dependent on public & private sources

Examination of system level effectiveness and co-occurring

interventions

Methodologic standards for priority setting, engaging patients

& research

Focus on real world setting and heterogeneous patient populations as

well as practice variations

Implementation and Dissemination CER

Importance of system level studies

11

So is this…

12

A brief (& Selected) History of Time…

1989AHCPR

1999AHRQ

2003MMA

2009ARRA

2009ACA

Outcomes PORTS

OutcomesCERTS

Quality & Safety

AHRQ EHC Program

Priority Conditions Payer focus Public input

CER enterprisePriority Questions

Methods & Standards

Stakeholder input

PCORIPublic/Private Funding

Priority QuestionsMethods & Standards

Stakeholder input

13

Over That Time…

A more than tripling of the NIH budget in 20 years

~$10 million in 1991 to $31 billion Growing understanding of the

many gaps in quality, outcomes Pervasive disparities The “inexorable cost curve”

Increasing demand for research ROI

For identified needs to drive funding

Improved methods for seeking public, user input

Increasing emphasis on priority setting

Focus on comparisons Expansion beyond discrete

clinical comparisons Focus on methodologic

standards

14

It is still wine, but…

15

CER Defining Characteristics

Objective of directly informing clinical or health policy decision

Compares at least 2 alternatives Results at population and subgroup level

16Tunis, 2009

“70 Kilo White Male”

17

CER Defining Characteristics

Objective of directly informing clinical or health policy decision

Compares at least 2 alternatives Results at population and subgroup level Measures outcomes important to patients Methods and data sources appropriate for the

decision of interest Conducted in real world settings

18Tunis, 2009

CER vs PCOR: is there a difference?

Patient-Centered Outcomes Research

Comparative Effectiveness Research

Comparisons

No patient reported data

Not comparative

Patient reported outcomes

Subgroups

19

So before we can match the study design to the question, we have to match the

question to the need.And that need varies depending on who

you ask and how you ask.

20

Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question

21

Who is asking the question? Why?

Policymakers Coverage, delivery system regulations, payment, quality

Payers Coverage, payment, quality

Industry Regulatory requirements, impact on innovation

Providers Health care choices

Consumers/Patients Health care choices

22

The design depends on who is asking the question

Constructing relationships Constructing questions Constructing answers Constructing messages

(Translation)

Source: Zerzan, JT et al. “Improving State Medicaid Policies with Comparative Effectiveness Research: A Key Role for Academic Health Centers.” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 695-700.

Categories of CER Methods

Systematic reviews of existing research Decision modeling, with or without cost information Retrospective analysis of existing clinical or

administrative data– Significant new investments in data infrastructure

Prospective non-experimental studies, including registries

Experimental studies, including randomized clinical trials (RCTs)

Tunis, 2009

The skeptical, “purist” view...

[…] “if American medicine is to avoid the mistakes of high-dose chemotherapy, hormone therapy, antiarrhythmic agents, and numerous other failed interventions, that our profession adopted on the basis of inferior evidence.”

Source: Lauer, MS. “Will Academia Embrace Comparative Effectiveness Research?” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 671-673.

Matching the Designs

Source: Thorpe KE et al, Journal of Clinical Epidemiology, 2009

Matching the Designs

Pediatric quality demonstrations 10 grants, 18 states, 47 projects Multiple stakeholders

– CMS– State policymakers– Health plans– Providers– Patients & families

27

Considerations

Degree of harm– Individual morbidity & mortality, versus population outcomes

Amount of existing evidence– System level decisions even more “evidence free”

Market entry vs prevalent practice Measuring the fidelity &/ adaptation of the intervention Opportunity to reconsider the choice

– Mid course corrections– Coverage with evidence development– Lifecycle approach to evidence development

28

Matching the Design to the Question

Observational Studies

RCTs

Balance

Rigor

Relevance

Timeliness

29

All methods have a role

Inevitable tradeoff between internal validity and feasibility, generalizability, cost, time

The nature of the research question, and the decision maker will influence best practices

Experimental studies will have a crucial role in CER, and there is need for improving design and implementation

Non-experimental methods hold great promise, particularly as methods are refined and data infrastructure is improved

Tunis, 2009

31

Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question Implications for the CER enterprise

32

An approach that advances a balanced portfolio of activities including:

Variety of study designs Evidence generation Evidence translation

Methodologic research Infrastructure enhancements

Data Workforce(s)

An approach that builds in “meta-learning”

A high performing evidence system

Source: Adapted from Van Lare, Conway & Sox, NEJM, 2010

33

Questions?

Lisa Simpson, MB, BCh, MPH, FAAPPresident and CEO