laurie anderson, phd, mph centers for disease control & prevention february 10, 2004

Laurie Anderson, PhD, MPH

Centers for Disease Control & Prevention

February 10, 2004

Outline

I. Difference between a literature review and a systematic review

II. Purpose of the Guide to Community Preventive Services: systematic reviews & evidence-based recommendations

III. Steps in carrying out a systematic reviews

IV. An example

The difference between

a literature review and

a systematic review

When making decisions about the choice of an intervention

The body of intervention literature can be quite large, inconsistent, and uneven in quality.

Literature Reviews

A “literature review” has typically been used to provide background information for intervention selection.

These reviews present a group of studies, with

strengths and weaknesses discussed selectively

and informally.

Traditional literature reviews have several shortcomings:

•The process is subjective. There are few formal rules, two reviewers might reach different conclusions.

•Lack explicit criteria for excluding inappropriate or poorly done studies.

Typically a literature review counts the number of studies supporting, or not supporting, an intervention

i.e. positive, negative, or no effect

but ignores sample size, effect size and research design.

This can lead to erroneous conclusions about intervention effectiveness.

In literature reviews conflicting findings may lead to a conclusion that an intervention is ineffective or the research is uninterpretable.

Systematic reviews exploit divergent findings by examining potential explanations --- treatment differences, setting differences, etc. --- because conflicting outcomes may tell us where an intervention is likely to succeed or fail.

Systematic reviews use numeric and narrative information fully:

•a small effect across several studies may be significant

•program characteristics can be used to explain the effect.

Literature reviews are an inefficient way to extract program & outcome information, particularly if the number of studies is large, e.g.>30.

•It is impossible to mentally juggle relationships among so many variables.

Systematic Reviews

Another approach to the literature is systematic reviews.

A systematic review takes in account:

•the precise purpose of the review

i.e. stating a research hypothesis

•how studies are selected & included

Systematic reviews can answer:

•is there publication bias?

•are intervention programs similar enough to combine?

•what is the distribution of study outcomes?

Systematic review can answer:

•are outcomes related to research design?

•are outcomes related to characteristics of programs, participants, and setting?

•what are the needs for future research?

Purpose of the Guide to Community Preventive Services: systematic reviews & evidence-based recommendations

Are we building on a foundation of existing knowledge?

Explosive growth of scientific information

• too much to keep up with

• contradictory results

Increasing public doubt about scientific findings

Systematic reviews for research synthesis

• combine many studies with different methods and results

• look for consistencies in set of findings

• more robust than single study

• may pinpoint why studies differ

• shows what is effective and why

What counts as evidence?

Type 1 Type 2

Determinants or

associations between

risk and an outcome

Relative

effectiveness of

different interventions

“Something should

be done”

“This should

be done”

The Guide to Community Preventive Services

What strategies, targeted to which groups will:

•promote healthy choices?

•prevent disease and injury?

•improve environmental conditions to promote health?

Community Guide Topics

Environmental Influences• Sociocultural Environment

• Physical Environment

Risk Behaviors Specific Conditions• Tobacco Use• Alcohol Abuse/Misuse• Other Substance Abuse• Poor Nutrition• Inadequate Physical Activity• Unhealthy Sexual Behaviors

• Vaccine Preventable Disease• Pregnancy Outcomes• Violence• Motor Vehicle Injuries• Depression• Cancer• Diabetes• Oral Health

Central Questions

• program effectiveness

• feasibility of implementation

• acceptability to the population

• unanticipated harms (or benefits)

• cost-effectiveness

Steps to Conducting a Review

• Assemble a review team• Develop conceptual framework• Prioritize review topics• Define specific intervention for review• Search for and retrieve evidence• Rate quality of evidence• Summarize evidence• Translate into a recommendation

Assemble a Review Team

• Multiple perspectives and backgrounds – Improve completeness and accuracy of information – Reduce impact of individual/institutional

perspectives– Enhance usefulness of products

Develop Conceptual Approachto the Topic

• The Logic Model

LOGIC FRAMEWORK ILLUSTRATING CONCEPTUAL APPROACH TO NUTRITION & COMMUNITY HEALTH

Interventions Modifiable Determinants

Food Supply Factors -Agriculture policy

-Nutrition policy-Science and technology

-Food production, processing, storage and distribution

-Food fortification-Food safety

Environmental Factors

Availability & Price-Neighborhoods

-Schools-Worksites

-Homes-State and National

-Food and nutrition

assistance programs

Consumer Demand-Household resources

-Nutrition knowledge

-Cultural practices

-Psychosocial characteristics

-Taste and preferences

-Advertising and marketing

POPULATION FOOD INTAKE

Food Consumption Patterns (e.g. fruits, vegetables)

Intake of Nutrients and Food Components

VitaminsMinerals

FiberFats

Other food constituentsDietary supplements

AlcoholEnergy balance

Life StageRequirements

Pregnancy Lactation Childhood

Adolescence Adulthood

Older Adulthood

Intermediate Outcomes

Community Health

OutcomesPhysiologic Indicators

Growth

Adipose tissueMusculoskeletalGastrointestinal

MetabolicCardiovascularReproductiveImmunologicalNeurological

Genetics,Co-morbidities

Morbidity

Mortality

Measures ofHealth & Fitness

PhysicalActivityPatterns

Prioritize Intervention Topics

• Preventable disease burden

• Common practices that are questionable

• New approaches that are promising

• Topic of keen public health interest

Priority Ranked List of Topics 

1. Food & beverage availability and price in schools.

2. Comprehensive community approaches to increase fruity & vegetable intake.

3. Food and beverage advertising to children.

4. Food & beverage availability, price, portion size, and labeling in restaurants.

5. Tax on sodas and snack foods.

6. Farm subsidies and production of fruits, vegetables, & whole grains.

7. Food choice and nutrition education in food assistance programs.

8. Nutrition and weight management counseling in healthcare settings.

9. Breast-feeding.

10 Product labeling in grocery stores, restaurants and vending machines.

11. Food & beverage availability and price in worksites.

12. Use of dietary supplements across the lifespan.

Specify the Review Question

• What population?

• What interventions or risk factors?

• What comparisons?

• What outcomes?

Example Question Do multi-component, school-based nutrition interventions

which may include:

– curricula (nutrition & physical activity)– food availability, accessibility, price– policy and environmental changes

improve nutritional behavior and nutritional status of children and adolescents in developed countries?

Environmentalsupport for healthy

choices

Nutrition & healthmessages

Promotion ofself-awareness,

self-efficacy

Multi-component

School-based

NutritionIntervention

s

Physiologic Indicatorsbody size &compositiongrowthfitnessHealth statusblood pressurecholesterolSchoolAchievementattendanceparticipationperformance

School Policy

Behaviors dietary intake physical activity

Knowledge nutritional needs food content

Attitudes self-care body image

Abilities self-assessment behavioral change skills media literacy

Analytic Framework

Inclusion Criteria

• Characteristics derived from the focused question

• Additional characteristics– Methodological– Publication dates– Languages– Relevant outcomes

Find Information

Select strategies for finding studies: – Database searches– Reviews of reference lists– Gray literature– Consultation with experts

Assess Quality

• Suitability of study design

•see handout

• Quality of study execution

•see handout

Suitability of Study Design

• Greatest– Prospective and – Concurrent comparison

• Moderate– Retrospective or– Multiple measurements over time; no

concurrent comparison

• Least– Single before and after measurement; no

concurrent comparison

Strength of Study Execution

• Description– Population– Intervention

• Sampling

• Measurement – Exposure– Outcome

• Analysis

• Interpretation of results

• Other

Exclude Studies Below Some Quality Threshold

• Exclude studies with limited execution (i.e., with many important threats to validity)

• Sometimes exclude studies with least or moderate suitability of design

Research Synthesis for Public Health Policy & Practice Decisions:

Systematic review of United States studies of .08 blood alcohol concentration laws

Lowering legal blood alcoholconcentration (BAC) limits

•Alcohol-related crashes cause 16,000 deaths and 300,000 injuries each year in the US

•Laws that lower BAC from 0.10 to 0.08 existed in less that half of US states

Analytic Framework:Lower Blood Alcohol Concentration Laws

LowerB.A.C.Laws Alcohol-

relatedcrashes

Drinking &driving

behaviors

PerceivedRisk ofArrest

Social normsregarding

drinking anddriving

Fatal &non-fatalinjuries

OUTCOME

Evidence base for .06 BAC Laws

Searched journal articles, technical reports and conference proceedings.

Nine studies were identified evaluated state BAC laws– all of sufficient design and quality.

All studies reported data from police incident reports of crashes on public roadways.

Evidence base for .06 BAC Laws

Eight of the nine studies reported percent change in alcohol-related fatalities (post-law period vs. pre-law period.

Other outcome data included public knowledge and perception of impaired driving laws, self-report of impaired driving, impaired driving arrests.

–50 -40 -30 -20 -10 0 10 20 30

Percent Change

Median percent change: -7% Interquartile range: -15%, -4%

States 15 States (1)* VA (2) VT (3,4,5) UT (5,3,4) OR (4,5,3) NC (6,2) NM(2) NH (2) ME(4,5,3) KS (2) FL (2) CA (5,7,8,3,4)

Percent Change in Measures of Alcohol -Related Motor Vehicle Fatalities, by State

Median percent change: -7%

Passed .08 BAC law in 2002

Passed .08 BAC law before the new national standard

States with .08 BAC Laws, 2002

Passed .08 BAC law in 2001

Who Is the Audience?

•Public health departments

•Health care systems

•Purchasers of health care

•Government and foundations

•Community organizations

Lack of Persuasive Evidence

• Lack of evidence does not mean that interventions don’t work

• Insufficient evidence may point to a

research agenda

Issues of Evidence

• We don’t have evidence about everything

• Enormous amount of evidence yet to review

• New evidence may change recommendations

• Capturing complexity

• Urgent needs and limited resources

• Participatory research

Translating Scientific Knowledge into Action

Task ForceRecommendations

andDissemination

Knowledge for InformedDecisions

Change in environments& behaviors

Program & policy

selection

Community Health

Outcomes

Implementation:• standards & protocols• program planning• funding decisions• policies & laws• research investments

www.TheCommunityGuide.org