applied health research lecture august 2015 (00000002)
DESCRIPTION
AHRTRANSCRIPT
10/08/2015
1
latrobe.edu.au/chcp CRICOS Provider 00115M
Communicate to Vaccinate: the COMMVAC systematic reviewJessica KaufmanResearch Officer and PhD CandidateGuest Lecture ‐ Applied Health Research10 August 2015
2
Outline
Communication for childhood vaccination
Identifying a topic for a systematic review
Defining the review’s question (PICO)
Searching and screening studies
Assessing study quality
Evidence synthesis
Findings and implications
10/08/2015
2
3
A snapshot of childhood vaccination
GLOBAL: In 2013, 21.8 million infants worldwide did not receive the full series of basic immunisations (WHO, UNICEF 2014)
LOCAL: In 2014, Victoria had 7 outbreaks of measles with 75 notified cases (RUSSO 2015)
Strategies to increase uptake:
Supply‐side interventions: vaccine access andavailability, new technologies, health personnel
Demand‐side / consumer level interventions:communication (e.g. reminders to parents, education, decision aids)
4
Examples of communication strategies (interventions) for childhood vaccination
Informing or educating people about risks and benefits of vaccines and where and when to get them
Reminding or recalling to appointments
Teaching people how to communicate more effectively
Supporting parents and helping facilitate decision making about vaccination
Involving community members in planning and evaluating vaccination programs
10/08/2015
3
5
Vaccine‐preventable disease outbreaks
Why do we need evidence‐based vaccination communication?
Effective communication enables:
Rapid rebuttal of misinformation
Community dialogue to achieve widespread support
Education of parents, communities and vaccinators
Consistent and effective advocacy of vaccine benefits
NEED TO KNOW WHAT WORKS: many trials but few efforts to synthesise available evidence globally
Ineffective communication may:
Consume time and resources
Undermine trust in health system
Have no or adverse effects on health outcomes
6
Communicate to Vaccinate (COMMVAC) projects
COMMVAC 1 (2010 – 2012) and COMMVAC 2 (2013‐2017)
Aim: To build the evidence base and improve communication about childhood vaccinations with parents and communities in Low and Middle Income Countries (LMICs)
International research collaboration with partners from Australia, Norway, France, Switzerland, South Africa, Nigeria, Mozambique, and Chile
10/08/2015
4
7
Evidence mapping
Run test search for trials (CENTRAL database) and other literature (MEDLINE)
Count articles evaluating each intervention
Map articles to the communication they evaluated
Orange = Trials
Blue = Not trials
Systematic review an intervention where there are trials
8
Identifying a topic for a systematic review
Deliberative forums with key stakeholders to prioritise review topics
Representatives of governmental and non‐governmental agencies, consumer groups and experts
Face to face (New York, Ottawa) and online
Identified high‐priority communication
10/08/2015
5
9
COMMVAC systematic reviews
Gold standard for evidence of effectiveness
“Face to face interventions for informing or educating parents about early childhood vaccination”
“Community‐directed interventions for informing and/or educating about early childhood vaccination”
Kaufman, J., Synnot, A., Hill, S., Willis, N., Horey, D., Lin, V., et al. (2013). Face to face interventions for informing or educating parents about early childhood vaccination. Cochrane Database Syst Rev, 5, CD010038.
Saeterdal, I., Glenton, C., Austvoll-Dahlgren, A., Munabi-Babigumira, S., & Lewin, S. (2014). Community-directed interventions for informing and/or educating about early childhood vaccination. Cochrane Database SystRev, 11, CD010232.
10
Defining the review’s question
P I C O
Population Intervention Comparison Outcomes
10/08/2015
6
11
Population
Health worker motivation program
Training for health professionals in vaccine delivery
Vaccine programme funding
More convenient organisation
of care
Incentives for consumers or health
professionals
Reminders for parents
Interpreters
Community vaccination events
Decision aids
Parenting skills training for new parents
Community coalitions to plan and implement vaccine program
Educational materials for parents
Parent‐held symptom
diary to track adverse events
Vaccination support hotline
Training for health
professional to improve their communication
skills
Chart prompt reminders for
health professionals that appear during consultations
Not IncludedIncluded
(involve communication and consumers or communities)
12
Defining the review’s question
P I C O
Population Intervention Comparison Outcomes
Parents /caregivers
10/08/2015
7
13
Intervention
Complex non‐clinical intervention
Clear definition at the protocol stage
Informing v. educating: literature review indicates there is no agreement about the difference between informing and educating
“Relevant interventions are those which make consumers aware of the practical or logistical factors associated with vaccination or seek to enable them to understand the meaning and relevance of vaccination for themselves, their family or community.” (KAUFMAN 2013)
14
Defining the review’s question
P I C O
Population Intervention Comparison Outcomes
Parents / caregivers
Any face‐to‐face communication to inform or educate
10/08/2015
8
15
Comparisons
What is the intervention compared to when it is evaluated?
Face to face communication directed to individual parents v. usual care
Face to face communication directed to groups of parents v. usual care
Face to face communication directed to individual parents v. face to face communication directed to groups of parents
Face to face communication type A v. face to face communication type B
16
Defining the review’s question
P I C O
Population Intervention Comparison Outcomes
Parents / caregivers
Any face‐to‐face communication to inform or educate
‐ F2F directed to individual parents v. usual care
‐ F2F directed to groups of parents v. usual care
‐ F2F directed to individual parents v. groups of parents
‐ F2F type A v. F2Ftype B
10/08/2015
9
17
Outcomes
What might be affected by this intervention? (e.g. behaviours, health status, thoughts or beliefs)
Which outcomes might be important to different people (e.g. parents v. doctors?)
Systematic reviews have to respond to trials but can also lead future research with carefully selected outcomes
Review literature first to understand the outcomes different trials are using
Process outcomes (e.g. knowledge and intention to vaccinate) are as important as endpoint outcomes (e.g. vaccination status)
Need to consider timepoints too – when was the outcome measured?
18
Defining the review’s question
P I C O
Population Intervention Comparison Outcomes
Parents /caregivers
Any face‐to‐face
communication to inform or educate
‐ F2F directed to individual parents v. usual care
‐ F2F directed to groups of parents v. usual care
‐ F2F directed to individual parents v. groups of parents
‐ F2F type A v. F2F type B
* Vaccination status of child
* Parents’ knowledge or understanding of vaccination
‐ Parents’ intention to vaccinate child
‐ Parents’ experience of the intervention
‐ Cost of implementing the intervention
‐ Adverse effects of intervention
“Face to face interventions for informing or educating parents about early childhood vaccination”
10/08/2015
10
19
Searching and screening studies
Professionally developed search strategy across many databases
Two authors screen results separately and compare
Inclusion criteria
Study design: RCTs or cluster RCTs
Children and vaccines: Children under 5 or 6 (school age), any WHO recommended routine early childhood vaccine (HPV excluded)
Interventions: not combined with any other intervention (e.g. reminders, improved access)
20
Included studies
Screened 9119 abstracts and titles
Included 7 studies
6 RCTs and 1 cluster RCT
Settings
HIC: Australia (2 studies), Canada, United States LMIC: Pakistan (2 studies), Nepal Mix of home and clinic
Participants
2978 participants across 7 studies Groups or individual parents, expectant parents or other guardians 3 studies targeted high‐risk mothers (drug users, adolescents, low socio‐
economic status)
Interventions
Mix of single and multi‐session interventions
10/08/2015
11
21
Outcomes measured
Primary outcomes
Vaccination status of child (measured by most studies) Parents’ knowledge or understanding of vaccination (measured by some
studies)
Secondary outcomes
Parents’ intention to vaccinate child (measured by no studies)
Parents’ experience of the intervention (measured by no studies)
Cost of implementing the intervention (measured by one study)
Adverse effects of intervention (measured by no studies)
22
Assessing the study quality
Quality: “the methodological strength of a relevant study, or how able it is, through its design and its conduct, to prevent systematic errors, or bias” (RYAN 2013)
“Biased studies are more likely to produce misleading results than those that are rigorously designed and conducted” (RYAN 2013)
RISK OF BIAS:
Type of bias DescriptionRelevant domains in the
Collaboration’s ‘Risk of bias’tool
Selection biasSystematic differences between baseline characteristics of the groups that are compared
• Sequence generation• Allocation concealment
Performance bias
Systematic differences between groups in the care that is provided, or in exposure to factors other than the interventions of interest.
• Blinding of participants and personnel
• Other potential threats to validity
Detection biasSystematic differences between groups in how outcomes are determined
• Blinding of outcome assessment
• Other potential threats to validity
Attrition biasSystematic differences between groups in withdrawals from a study
• Incomplete outcome data
Reporting biasSystematic differences between reported and unreported findings
• Selective outcome reporting(HIGGINS 2011)
10/08/2015
12
23
Synthesisingthe evidence
24
Synthesising the evidence
Meta‐analysis is combining and analysing data from more than one study at once (BASTIAN 2014)
Heterogeneity is variation or diversity…
In the participants, interventions and outcomes (clinical heterogeneity)
In study design and risk of bias (methodological heterogeneity)
In the intervention effects being evaluated (i.e. the observed effects are more different from each other than one would expect due to chance alone) (statistical heterogeneity) (HIGGINS 2011)
If calculation of statistical heterogeneity is high, your assumptions are unsafeand you should not pool your results.
10/08/2015
13
25
Narrative synthesis
Narrative synthesis: describing results of included studies in a written summary
Group studies by intervention, comparison, or other meaningful category
Provide the same elements of information about each study in the same order to avoid being selective or emphasising certain findings
Example:
Comparison 1: Face to face interventions directed to individual parents versus control
Primary outcome: Immunisation status
o Immunisation status measured 3 months after a single‐session intervention
» Population was diverse
» Overall effect was uncertain
» Evidence of low quality
» Description of each study’s effect
(KAUFMAN 2013)
26
Forest plots
Forest plot: a visual representation of the results of each study
Horizontal lines are confidence intervals, or margin of error for each result
Below is a NARRATIVE SYNTHESIS forest plot. A meta‐analysis would include a diamond at the bottom showing the overall summary
10/08/2015
14
27
Forest plots
Forest plot: a visual representation of the results of each study
Horizontal lines are confidence intervals, or margin of error for each result
Below is a NARRATIVE SYNTHESIS forest plot. A meta‐analysis would include a diamond at the bottom showing the overall summary
28
Summary of findings table
Organised by the outcomes
Includes assessment of quality, description of effect and how certain we are of that effect
Included at the start of all Cochrane reviews
10/08/2015
15
29
Summary of findings ‐ narrative
Face to face strategies do not consistently improve either immunisation rates or parent knowledge and understanding of vaccination, but the evidence was low to very low quality for these outcomes
Only one study measured cost and no studies measured other outcomes
The results of this review are limited by the small number of included studies, small number of outcomes measured and problems with study methods
30
Implications for research and practice
Effect size may depend on baseline vaccination rates
Impact may differ for populations with low literacy or limited access to printed or online materials
Future studies should measure process outcomes including knowledge or intention to vaccinate
10/08/2015
16
31
Next steps
COMMVAC 2 project – create evidence‐based global guidance on implementing vaccination communication interventions for program managers and policy makers in LICs
PhD – Develop a core outcome set so future research can help inform better communication design and delivery, and future trials can be meta‐analysed
32
References
Bastian, H. (2014) 5 key things to know about meta‐analysis. Scientific American Blog Network. http://blogs.scientificamerican.com/absolutely‐maybe/5‐key‐things‐to‐know‐about‐meta‐analysis/
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org
Kaufman J, Synnot A, Ryan R, Hill S, Horey D, Willis N, Lin V, Robinson P. Face to face interventions for informing or educating parents about early childhood vaccination. Cochrane Database of Systematic Reviews 2013, Issue 5. Art. No.: CD010038.
Russo, K. (2015) Immunisation Update April 2015. Networking Health Victoria. www.lmmml.org.au/LiteratureRetrieve.aspx?ID=191955 (accessed 6/8/15).
Ryan R, Hill S, Prictor M, McKenzie J; Cochrane Consumers and Communication Review Group. Study Quality Guide. May 2013 http://cccrg.cochrane.org/authorresources (accessed 30/7/15).
Saeterdal I, Lewin S, Austvoll‐Dahlgren A, Glenton C, Munabi‐Babigumira S. Interventions aimed at communities to inform and/or educate about early childhood vaccination. Cochrane Database of Systematic Reviews 2014, Issue 11. Art.No.: CD010232.
WHO, UNICEF. (2014) Progress towards global immunization goals ‐ 2013. http://www.who.int/immunization/monitoring_surveillance/slidesglobalimmunization.pdf
10/08/2015
17
Thank you
latrobe.edu.au/chcp CRICOS Provider 00115M
www.commvac.com