applying research in public health
TRANSCRIPT
www.hertsdirect.org
Applying Research in Public Health Greater Importance than Ever!
Jim McManusDirector of Public Health
www.hertsdirect.org
Some problems
• Massive literature on applying research in practice
• Lots of problems in the literature – Sometimes conceives receivers of research
as in deficit of knowledge mode– Focuses often on one side – the research
quality or the organisation – not synergistic enough
www.hertsdirect.org
Thesis
• Applying research needs a partnership between researchers and policymakers from the inception if possible
• Two sides of a coin : the quality of the research and the agility and leadership of the organisation
• Develop a roadmap jointly
www.hertsdirect.org
Context
• Cuts to public services• Increasing demand on NHS services• Financial unsustainability of current system• We need research which can help us keep
people independent and healthy, and reduce need for public services
www.hertsdirect.org
Nationally funded research
• http://www.nihr.ac.uk/funding/public-health-research-programme.htm
• http://www.nets.nihr.ac.uk/programmes/phr• http://
www.nihr.ac.uk/funding/school-for-public-health-research.htm
• Great these are funded but we need more emphasis on translatable research and interventions, and evaluating existing ones
www.hertsdirect.org
Family Nurse Partnership – applying the evaluation to public health decision making• Evaluation report
says doesn’t work not cost effective
• FNP says it does other valuable things
• Where do these contradictory views fit in policymaker decisions?
• Issues raised:– Appraisal of research– By products of
interventions and roundedness of evaluations
– Applicability
www.hertsdirect.org
Research on effects of e-cigarettes
• Most of it unappliable because of design flaws or even repeating existing badly done research
• In vitro studies which don’t reflect real life• Speculation and going beyond the research
doesn’t help
www.hertsdirect.org
Research on Diet
Is your research clear and conclusive enough to give definitive guidance to people?
If yes – be clear and don’t be vague
If no – then shut up and continue researching until it is clear enoughA moral responsibility on scientists NOT to overstate their case and generalizability
www.hertsdirect.org
The Problem
• Outcomes Frameworks• Connecting evidence with outcomes• Connecting evidence with interventions• Interests of policymakers and interests of
researchers
www.hertsdirect.org
So how do you go about applying research into public health practice? • Lots of literature on this• Some very complex and complicated models• A Proposal from me
1. Treat it like a change management process2. Think about two major sides to a coin 3. Project and process manage it4. Fidelity and sensitivity test
www.hertsdirect.org
Change Management
• Lessons from ESRC evaluations of large scale IT programmes
• Lessons from NHS and Social Care research on applying research
www.hertsdirect.org
Levels and Phasing
• Which level is the intervention?– Societal– Environmental?– Biological?– Intrapersonal?
• Which phase of time?– 1 year, 2, 3?
www.hertsdirect.org
Return on Investment as part of the argument• Variable response from commissioners to this• Too much use of poorly developed ROI models in
evidence• ROI is not always the answer and sometimes doesn’t
even help• Most ROI models aren’t real world enough• Depends on how robust it is• Who cashes the returned cheque? If agency A invests
but Agency Y gets the return, where is the incentive?
www.hertsdirect.org
Return on Investment Tools
• Helps if the evidence is also very clear • Helps if there is a timescale to the return and
clarity on who gets it• The policy/commissioning/research partnership
is crucial to making ROI useful
www.hertsdirect.org
What is evidence-based public health?
Many definitions◦ From Evidence-Based Medicine to Evidence-Based Public Health
Evidence-based public health is defined as the development, implementation, and evaluation of effective programs and policies in public health through application of principles of scientific reasoning, including systematic uses of data and information systems, and appropriate use of behavioral science theory and program planning models. (Brownson, Ross C., Elizabeth A. Baker, Terry L. Leet, and Kathleen N. Gillespie, Editors. Evidence-Based Public Health. New York: Oxford University Press, 2003.)
◦ E-Roadmap to Public Health Practice Concepts (New Hampshire Institute for Health Policy and Practice) Developing, implementing, and evaluating public health programs or public
health policies (in public health terms an "intervention") that have 1) data demonstrating their effectiveness and 2) a grounding in a health behavior theory or ecological model of health.
www.hertsdirect.org
What is evidence-based public health?
• Many definitions– “The process of integrating science-based
interventions with community preferences to improve the health of populations.” (Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: An evolving concept. Am J Prev Med. 2004 Dec;27(5):417-21.)
www.hertsdirect.org
Steps for EBPH
• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is
already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan
O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.
www.hertsdirect.org
Coin side 1: The Research itself
• Internal and external validity• Applicability• Clarity of methods and outcomes• Translatability• Intervention readiness• Engagement with policymakers
www.hertsdirect.org
Coin side 2: the organization and its agility
Macro Level
• Relevance to organization’s strategy• Champion and leadership
Meso Level
• Articulating the change needed• Resource, programme planning
Micro Level
• Making it attractive and easy for people to do• Champions and leadership
www.hertsdirect.org
Example: Behavioural Sciences
• Review of Research• Synthesis into policy paper• Applying resource• Articulating work and programme plan• Delivery and championing• Integration into strategy
– This is very high level and much more involved in this
www.hertsdirect.org
Communication - Writing style as an example
Academic• Heavily referenced• Third person • Passive voice• Tentative conclusions
Local Govt• Active voice• Summaries• Recommendations• Costings• Policy options• Impact measurements• Feasibility• risk
www.hertsdirect.org
Step 1: Develop an initial statement of the issue
Use PICO (from EBM) to begin define question◦ P: Population◦ I: Intervention◦ C: Comparison◦ O: Outcome
Use a logic model to begin your strategic planning◦ Inputs, activities, outputs, results (short/long term)◦ W.K. Kellogg Foundation Logic Model Development Guide
http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
◦ Logic model tutorials (CDC) http://apps.nccd.cdc.gov/dashoet/logic_model_1/menu.html http://apps.nccd.cdc.gov/dashoet/logic_model_2/index.html
www.hertsdirect.org
Steps for EBPH
• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is
already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan
O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.
www.hertsdirect.org
Mapping the Policy Process
• General Context issues – domestic and international.
• Specific Policy Issues (i.e. the policy cycle)• Who are the Stakeholders? (Stakeholder analysis)
– Arena: government, parliament, civil society, judiciary, private sector.
– Level: local, national, international • Process matrix + political influence ratings• What is their Interest and Influence?
[Sources: M. Grindle / J. Court ]
www.hertsdirect.org
Ontario Drug Programme (Khan et al,2014)http://dx.doi.org/10.1016/j.healthpol.2014.06.007 • Policymakers have cited barriers to using evidence,
including lack of research relevance and timeliness. • .Although reports often demonstrate an increase in
research relevance, rarely do they provide concrete methods of enhancing research timeliness
• . Additionally, the impact of researcher–policymaker collaborations is not well-discussed.
www.hertsdirect.org
Be clear about how feasible it is to do this
Option A Option B Option C
Effectiveness Very Positive Positive No impact
Flexibility Very Positive Positive Positive
Sustainability Positive Positive Negative
Political Feasibility High Medium Low
Administrative Feasibility
High Medium Low
Time Short Medium Long
Cost High Medium Low
Suggested criteria example