identifying evidence for decision-analytic models
DESCRIPTION
Identifying evidence for decision-analytic models. Suzy Paisley DoH Research Scientist in Evidence Synthesis Consensus Working Group on the Use of Evidence in Economic Decision Models : Establishing the Current Situation MRC HSRC Workshop 25 July 2005. DoH Award. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/1.jpg)
Identifying evidence for decision-analytic models
Suzy PaisleyDoH Research Scientist in Evidence Synthesis
Consensus Working Group on the Use of Evidence in Economic Decision Models : Establishing the Current Situation
MRC HSRC Workshop 25 July 2005
![Page 2: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/2.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
DoH Award
To develop a systematic approach to searching for evidence to inform decision-analytic models
Characteristics and sources of evidence to inform parameter values
Use of evidence in informing other parts of the modelling process
![Page 3: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/3.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Today’s presentation
Why an issue Current practice
– Identification of evidence– Use of evidence
Implications of current practice
![Page 4: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/4.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Why an issue?
Environmental issues Own experience
– Undertaking HTA searches– Peer review of HTA reports
![Page 5: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/5.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
‘Typical’ HTA
Decision problem / research question Syntheses of existing evidence Systematic review of clinical-
effectiveness Decision-analytic model of cost-
effectiveness
![Page 6: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/6.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Decision problemsearch question
P = colorectal cancer
I = irinotecan
C = any relevant
O = survival
#1 colorectal
#2 irinotecan
#3 rct
#4 #1 and #2 and #3
![Page 7: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/7.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Modelling decision problems
Effectiveness Costs Resource use / activity Health states Utility values Indirect comparators Longer term outcomes ‘Other’ interventions Natural history Epidemiology
![Page 8: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/8.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice
Analysis of 27 technology assessment reports (TARs)
Review of HTA guidelines Content analysis of modelling guidelines
![Page 9: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/9.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Reports of models
Use evidence which cannot be identified systematically by PICO focused searches(21/27 (78%) assessments for NICE Paisley, 2001)
Report PICO focussed searches(4/21 (19%) reported additional ‘ad hoc’ searches Paisley, 2001)
Issues of transparency
![Page 10: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/10.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Reports of modelsTypes
EffectivenessCostsResource use / activityHealth statesUtility valuesIndirect comparatorsLonger term outcomes‘Other’ interventionsNatural historyEpidemiology
Sources
‘Published papers’Routine dataReference sourcesLocal / clinical /
expert opinionSponsor submissions
Uses
Parameter valuesModel structureSensitivity analysisValidation / consistency / calibration
![Page 11: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/11.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Methodological guidance
HTA Guidelines Modelling guidelines
![Page 12: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/12.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
HTA guidelines (n=32)
Identification of evidence:
No mention 13 (41%)Systematic review approach 13 (41%)Broad approach 2 (6%)Approach unclear 2 (6%)Not available 2 (6%)
Name data sources 21 (66%)
![Page 13: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/13.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Modelling guidelines
ISPOR Principles of Good Practice (Weinstein et al, 2003)
1 explicit reference to systematic reviews of the literature
41 implicit references to a process of seeking evidence
30/41 (73%) references form statements within recommendations
![Page 14: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/14.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Modelling guidelines
Model structure 10 (33%)
Data identification 7 (23%)
Data modelling 7 (23%)
Data incorporation 0 (0%)
Model validation 6 (20%)
![Page 15: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/15.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:
Conclusions
Reports of models– Report PICO focussed searches– Use a wide range of evidence from a wide
range of sources throughout the modelling process
Modelling and HTA guidelines– Give limited explicit guidance– Give an implicit indication that range of
sources should be used throughout the modelling process
![Page 16: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/16.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Implications for searching:
Types of evidence
Effectiveness Costs Resource use / activity Health states Utility values Indirect comparators Longer term outcomes ‘Other’ interventions Natural history Epidemiology
Defining search questions
Defining the evidence base
Transferable / ‘proxy’ data
Grouping questions
![Page 17: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/17.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Implications for searching:
Sources of evidence
‘Published papers’ Routine data Reference sources Local / clinical /
expert opinion Sponsor
submissions
Prioritising appropriate sources
![Page 18: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/18.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Implications for searching:
Uses of evidence
Parameter values Model structure Sensitivity analysis Validation /
consistency / calibration
Iterative nature of modelling process
Matching retrieval of evidence to use of evidence
![Page 19: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/19.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Other issues
Aim of the search– What evidence?– What for?– How much?
Methods of selecting evidence Alternative search methods Reporting standards
![Page 20: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/20.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Conclusions
The process of identifying evidence for models needs to take account of the following issues:
The range of types, sources and uses of evidence The process of identifying relevant evidence bases
and defining search questions The iterative nature of model development How much evidence is required Reporting standards
![Page 21: Identifying evidence for decision-analytic models](https://reader035.vdocument.in/reader035/viewer/2022062309/568142e3550346895daf3f78/html5/thumbnails/21.jpg)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Conclusions
Identifying evidence for models is different to identifying evidence for systematic reviews.