relevance to policy
Post on 20-Jan-2022
3 Views
Preview:
TRANSCRIPT
Assessing Relevance from the Policy Perspective
David Elliott MD, MScMedical Advisor, NS Department of Health
Adjunct Assistant Professor, Dalhousie University Department of Community Health and Epidemiology *
* The opinions expressed herein are the author’s alone and do not necessarily reflect the position of the Nova Scotia Department of Health or Dalhousie University
Policy RelevancePolicy Relevance
This is a slightly modified version of the slideshow given at the Symposium. It lacks the context given by the verbal presentation. I have attempted to compensate with some added text overlays. The initial slides parse out the words in the title to illustrate points regarding policy, technology and the challenges therein.
To view this presentation fullscreen in your browser hit F11 then CTRL+L. If you have saved this presentation to disk and are using the standalone Adobe viewer, hit CTRL+L to view it in slideshow mode. (Which illustrates another point – many people don’t understand how to properly use the technology they already have!)
DE
Policy RelevancePolicy Relevance
Relevance Relevance -- definitionsdefinitions
� Closely connected or appropriate to the matter at hand – WordPerfect Condensed Oxford Dictionary
� Microsoft Word Thesaurus (no dictionary)
• Significance
• Bearing
• Application
• Importance
• Consequence
Policy RelevancePolicy Relevance
Relevance Relevance -- definitionsdefinitions
� Closely connected or appropriate to the matter at hand – WordPerfect Condensed Oxford Dictionary
� Microsoft Word Thesaurus (no dictionary)
• Significance
• Bearing
• Application
• Importance
• Consequence
Note that I have just done a
technology comparison and
have concluded that the
WordPerfect technology is
superior. However, most
people now use Word as a
consequence of the tied
selling activities of the
monopoly Operating
System provider, Microsoft.
Which illustrates another
factor in the real world of
how technology gets
applied.
Note that I have just done a
technology comparison and
have concluded that the
WordPerfect technology is
superior. However, most
people now use Word as a
consequence of the tied
selling activities of the
monopoly Operating
System provider, Microsoft.
Which illustrates another
factor in the real world of
how technology gets
applied.
Policy RelevancePolicy Relevance
Policy definitionsPolicy definitions
� “What we usually do”
� “Who gets what”
� “Who gets, who pays, who gets paid” - Canadian health economist, Bob Evans
� “The process by which governments translate their political vision into programmes & actions to deliver ‘outcomes’ – desired changes in the real world” - UK Centre for Management and Policy Studies
� “People who enjoy eating sausage and respect the law should not watch either being made” - Otto von Bismarck
Policy RelevancePolicy Relevance
PerspectivePerspective
� perspective >noun
• the art of representing three-dimensional objects on a two-dimensional surface so as to convey the impression of height, width, depth, and relative distance.
• a view or prospect.
• a particular way of regarding something.
• understanding of the relative importance of things.
� originally in the sense 'optics': from Latin perspectiva ars 'science of optics'
Policy RelevancePolicy Relevance
PerspectivePerspective
� perspective >noun
• the art of representing three-dimensional objects on a two-dimensional surface so as to convey the impression of height, width, depth, and relative distance.
• a view or prospect.
• a particular way of regarding something.
• understanding of the relative importance of things.
� originally in the sense 'optics': from Latin perspectiva ars 'science of optics'
Note that the word
perspective is being used
“off label” or not in the
manner in which it was
originally intended,
illustrating another real
world technology challenge.
Note that the word
perspective is being used
“off label” or not in the
manner in which it was
originally intended,
illustrating another real
world technology challenge.
Policy RelevancePolicy Relevance
Health Technology can be so Complex
Policy RelevancePolicy Relevance
Or not so Complex . . .
Policy RelevancePolicy Relevance
or pretty darn Simple
. . . but may not always be
used as intended (off-label use)
. . . and this device, although
designated by manufacturer
to be “single-use only,” was
likely disinfected and is being
reused
Policy RelevancePolicy Relevance
Introduction Introduction
� First try to get some understanding of the significance of the title . . .
• What is “Policy”
• Who decides it, who implements it, and
• How is it informed
� HT decision scenarios
� What determines relevance
� Information needs
� Unique challenges
Policy RelevancePolicy Relevance
Policy Making LevelsPolicy Making Levels
� Judicial
� Legislative
� Regulatory
� Administrative
� Procedural
F
O
R
M
A
L
I
S
M
D
I
S
C
R
E
T
I
O
N
Policy RelevancePolicy Relevance
HT Policy LevelsHT Policy Levels
� Macro
• Dealing with high cost, one or few-of-a-kind technologies –often cross-jurisdictional
• Decisions for standardization on single product, technique for a jurisdiction
• High need for coordination and dialogue
� Meso
• Health Region or facility
� Micro
• Processes for ensuring proper principles are followed in making more minor HT decisions
Policy RelevancePolicy Relevance
But is policy the right concept here?But is policy the right concept here?
� Insofar as we can call any HT decision ‘policy,’then this is about policy
� But really we are most concerned with tools for decision-making, recognizing that a lot of these decisions are “routine”
� While policy may determine what facts we look for and how we interpret them, many HT decisions are subject to policy but not policy themselves
Policy RelevancePolicy Relevance
NeedsNeeds
� Obviously the type of information requirements between these levels may differ
� The timescale for making decisions can vary widely
� The cost impact is different
� Oversight of outcomes or consequences may range from close scrutiny to none
Policy RelevancePolicy Relevance
Overview Overview
� The decision-maker’s milieu
Budget
More meetings
80 eMails/dayPersonnelConflict
No time
Meetings
Firefighting
Polls
Consumers
Policy RelevancePolicy Relevance
Needs in the DecisionNeeds in the Decision--Making ProcessMaking Process
� Information gatherer
• Policy analyst or Content expert
� 1st order decision-maker
• director or executive director
� 2nd order
• Administrative decisions (ADM / Deputy)
• “Political” decisions (minister / cabinet)
� Needs
• Full background
• Highly technical
� Needs
• Detailed synthesis and options
� Needs
• Highly structured
• Main points
• Options / consequences
Policy RelevancePolicy Relevance
So then . . .So then . . .
� It follows that different decision-making levels require different products
• Regardless of efforts to produce products for different levels, there will have to be contextualization depending on jurisdiction and audience
• Local factors may trump otherwise relevant evidence
Policy RelevancePolicy Relevance
The Place of EvidenceThe Place of Evidence
� Everyone talks about “Evidence-Based” policies
� Evidence is good to have, but,• Sometimes you have evidence, but it isn’t the evidence you need
• Evidence is in conflict - contradictory/equivocal
• Other times evidence (scientific) is not enough
� Decision-makers must always make decisions under some condition of uncertainty (and be willing to do so!)
� Rely on experience (which is a form of evidence in itself)
Policy RelevancePolicy Relevance
Health Technology Health Technology –– Friend or Foe?Friend or Foe?
� While health technologies can and have helped in prevention and treatment
� They are unpredictable elements in managing healthcare
• Raise Expectations
• Cost-benefit
• Usage profile exceeds indications
Policy RelevancePolicy Relevance
I could save him if
we had one of
those new
LC-32D7Us
I could save him if
we had one of
those new
LC-32D7Us
Clinical Pressure & ImperativesClinical Pressure & Imperatives
Policy RelevancePolicy Relevance
HT Evaluations vs. Real WorldHT Evaluations vs. Real World
0 1 2 3 4
# of Interventions
Benefit
Cost
“Stacking”
Costs are additive
Benefits are not
Cost benefit often out of context of useCost benefit often out of context of use
Policy RelevancePolicy Relevance
Evaluation & Screening ProcessesEvaluation & Screening Processes
� Vary by technology and cost
• Drugs• Subject to more centralized decision making • ‘ Culture of evidence’
• Devices & Techniques• Evaluation process varies with individual or aggregate costs• Technical changes without immediate purchase implications may occur without any formal evaluation whatsoever.
Policy RelevancePolicy Relevance
Scenarios Scenarios –– How technology gets inHow technology gets in
� “Big ticket” items ~>$1M
• Multi-jurisdiction / Multi-level funding
• Highly formal process
• Can be politicized
� Midrange ~>$100K - $1M
• Single jurisdiction / facility
• Multidisciplinary Teams
� Small change ~<$100K
• Department or Materials Management
Policy RelevancePolicy Relevance
Oh yes, we
gotta have
one of those…
Oh yes, we
gotta have
one of those…
Policy RelevancePolicy Relevance
Scenarios Scenarios –– How technology gets in contHow technology gets in cont’’dd
� The back door
• Surgeon buys equipment
• Equipment part of clinical trial and is “donated” to facility at end of trial (e.g., Bone Densitometer used in osteoporosis Rx trial)
• Foundation raises money in community
• Internal innovation
� “The Big Deal”
• Large scale purchase agreements & tied selling
Policy RelevancePolicy Relevance
The midrange scenarioThe midrange scenario
Department
Heads
Biomedical
Engineering
Materials
Management
VPs Clinical
& Finance
Clinical
Advocate
HT
Decision
Policy RelevancePolicy Relevance
Department
Heads
Biomedical
Engineering
Materials
Management
VPs Clinical
& Finance
Clinical
Advocate
HT
Decision
The midrange scenarioThe midrange scenario
What does
it cost?
What does
it cost?
We think it works
and we want it
RCT & other trial
results
We think it works
and we want it
RCT & other trial
results
Comparison of
competing
technology &
suppliers
Comparison of
competing
technology &
suppliers
Standardization of
equipment across
departments
Standardization of
equipment across
departments
If they get it, we
want our request
filled as well
If they get it, we
want our request
filled as well
What do we do with
the stuff we’ve
already got?
What do we do with
the stuff we’ve
already got?
Policy RelevancePolicy Relevance
RelevanceRelevance
� In the topic submission and prioritization process, someone had to suggest a HT topic and it had to get enough votes to get studied• But did the subsequent assessment cover sufficient clinical scenarios and have proper sensitivity analysis
• Did it get done in time to affect decisions, or has an interim or permanent decision been taken that renders the HTA moot
� The HT(P)A, regardless of how well done, has to be contextualized to the local situation
Policy RelevancePolicy Relevance
Keys to RelevanceKeys to Relevance
� On target• Applies to the population and circumstances at hand
� In time• Information and options are available within an adequate timeframe for decisionmaking
� Trusted• Agency producing information has track record for quality, unbiased work
• Faith in generally accepted methods
Policy RelevancePolicy Relevance
Relevance contRelevance cont’’dd
� Effectiveness and cost-benefit scenarios must encompass a variety of real world possibilities
• E.g. DES and initial use in high risk populations at higher risk of restenosis
� Model the effect of market competition on price
• E.g. DES and price changes after introduction of competing products and alternate formats
Policy RelevancePolicy Relevance
The DecisionThe Decision--makermaker
� Has huge demands on their time
� May have no formal support structure for making HT decisions (smaller provinces)
� May not be a content expert
• Often there is one short opportunity to make a point
• Takes comfort, though, in being able to quote an impeccable authority
Policy RelevancePolicy Relevance
Products and Processes for the HT FutureProducts and Processes for the HT Future
� The two greatest needs:
• Information
• Understanding
Policy RelevancePolicy Relevance
InformationInformation
� Timely, relevant, credible and contextualized HT appraisals with policy overlay and discussion of options
� Horizon scanning
� Knowledgebase – online searchable collection of
• Q&As
• Rapid assessments
• Other resources for HT decision-making
� Liaison Officers
• Individualized responses to field HT questions
Policy RelevancePolicy Relevance
Processes toward Increasing UnderstandingProcesses toward Increasing Understanding
� Capacity building• Liaison officers
� Strategic ‘meta’-questions• Challenges arising from moral-ethical overlays that accompany some technologies. E.g. genetic
• Help understanding and articulating cost-benefit analysis: the go or no-go threshold
• Adequacy of HT policy making in terms of knowledge and resources
� Develop ‘Culture of Evidence’
Policy RelevancePolicy Relevance
SummarySummary
� Policy is made at many levels
� Inputs to policy-making are varied
� Packaging information for decision-makers is difficult
• information “too soon” or “too complex” is not useful
• information, however “evidence-based” it may be, may not be relevant to the heart of the decision needed
top related