automated decision support for evidence-based benefit-risk...
Post on 18-Mar-2020
5 Views
Preview:
TRANSCRIPT
Introduction Benefit-risk Discussion References
Automated decision support for evidence-basedbenefit-risk decision making
Gert van Valkenhoef
Department of Epidemiology, University Medical Center Groningen (NL),Faculty of Economics and Business, University of Groningen (NL)
Escher meeting, 2 December 2011Amersfoort, The Netherlands
Introduction Benefit-risk Discussion References
Escher 3.2 Goals
Develop a drug information system:
Effective knowledge access and management
Answer drug efficacy and safety questions
in an efficient, transparent and accountable waywithin and across compoundsfor a broad audience (including regulators)
Improve consistency in regulatory decision making
Based on systematic review and meta-analysis
Introduction Benefit-risk Discussion References
Problem 1: effective knowledge access
Review of existing systems:
Evidence-based decision making time-consuming/error-prone
No comprehensive source of trial information existsTrial information is insufficiently structuredLack of systems to use the data
Missed opportunities to introduce more structure
Trial registration, regulatory submission and systematic review
It is unclear how the information should be structured
Prototypes should be developed now, to discover this
[Tervonen et al., 2010, van Valkenhoef et al., 2011b]
Introduction Benefit-risk Discussion References
Problem 2: transparent decision making
Regulatory benefit-risk decision making:
Based on pivotal evidence from trials
But not using formal decision modeling
Lack of (the use of) formal methods:
Hides reasoning supporting decision
Causes lack of transparency / traceability
Raises doubt about consistency
Causes of informal process:
Lack of suitable models/tools
Weighting criteria potentially controversial
Introduction Benefit-risk Discussion References
PhD overview
The ADDIS system & database
Automating network meta-analysis
Benefit-risk modeling
Introduction Benefit-risk Discussion References
ADDIS: Aggregate Data Drug Information System
Assisted evidence synthesis and benefit-risk assessment
Based on a database of clinical trials
Focussed on aggregated data
ADDIS is our (partial) solution [van Valkenhoef et al., 2011e]
Database is assumed available (or delivered by company)
Gathering the data is still a major hurdle!
But the ADDIS system / data model is a first step
Introduction Benefit-risk Discussion References
Evidence synthesis
If > 2 studies provide information on the same outcome
This information must be combined (synthesis)
Implemented through meta-analysis
Decisions involving > 2 alternatives
Require synthesis of networks of trials
Network meta-analysis
Existing technique, requiring manual modeling
Automated for use in ADDIS [van Valkenhoef et al., 2011a,c]
Introduction Benefit-risk Discussion References
Benefit-risk modeling in ADDIS
Available methods:
BRAT framework
‘Lynd & O’Brien’ model
SMAA (MCDA) based method
Based directly on the evidence from:
Single trials
Evidence synthesis
Introduction Benefit-risk Discussion References
BRAT Framework
Provides a number of steps to perform
Define decision context, identify outcomes, identify datasources, ... (similar to MCDA)
Defines useful visual summaries
But only applies to pair-wise comparisons
Does not define how to make trade-off decisions
Left open in frameworkADDIS models fill the gap
[Coplan et al., 2011]
Introduction Benefit-risk Discussion References
Quantitative benefit-risk modeling
Based on clinical trials or (network) meta-analysis
Making trade-offs explicit
But allowing for imprecise/vague/incomplete preferences
Making uncertainty explicit
[Tervonen et al., 2011, van Valkenhoef et al., 2011d]
Introduction Benefit-risk Discussion References
A simple stochastic model
The ‘Lynd & O’Brien’ model:
Based on cost-effectiveness analysis techniques
Compares 2 alternatives
On 2 criteria (benefit vs. risk)
Sample (∆B,∆R) values from a joint distribution
Plot them on a plane
Count how many points are below the threshold µ
[Lynd and O’Brien, 2004]
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µB better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µ
B better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µ
B better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
Trade-off
Trade-off
µ
B better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µThe acceptability threshold.
We are willing to ‘pay’ µ
units risk to get 1 unit of
benefit.
B better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µThe acceptability threshold.
We are willing to ‘pay’ µ
units risk to get 1 unit of
benefit.
B better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µB better
A better
p = aa+b
count b
count a
Introduction Benefit-risk Discussion References
SMAA BR analysis
The Lynd & O’Brien model is limited to 2x2 problems.
Stochastic Multi-criteria Acceptability Analysis (SMAA)allows m × n problems:
m alternativesevaluated on n criteriaperformance of alternative i on criterion j : Ci,j ∼ f (ci,j)
[Tervonen et al., 2011, van Valkenhoef et al., 2011d]
Introduction Benefit-risk Discussion References
SMAA BR analysis
SMAA models for benefit-risk:
Can be based on a single trial
Or (network) meta-analysis
And is implemented in ADDIS
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
SMAA modelbased on networkmeta-analysis.
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
Measurements (input distributions).
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
Model without preference information.
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
Model without preference information.
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
Preferences for severe depression.
Introduction Benefit-risk Discussion References
SMAA example (ADDIS)
Preferences for mild depression.
Introduction Benefit-risk Discussion References
Relevance: EMA BR methodology project
Approach/method Relevance to regulators UsefulnessProbabilistic simulation Can illuminate the risk/benefit trade-off when uncertainty is a major
feature of a regulatory decision.Medium
Bayesian statistics Can integrate evidence and its uncertainty, both pre- and post-approval, with multiple criteria in decision models.
High
MCDA Multi-criteria decision analysis extends decision theory to accommo-date multiple, conflicting objectives. Provides common units of valuefor both benefits and risks.
High
Table: MTC/SMAA integrates 2 of 3 quantitative approaches rated’High’ on usefulness, and 1 rated ’Medium’.
Introduction Benefit-risk Discussion References
Summary
ADDIS (Aggregate Data Drug Information System)
Decision support system
Evidence based regulatory decision making
ADDIS incorporates
statistical methods
decision modeling
in an automated framework
Introduction Benefit-risk Discussion References
Conclusions
ADDIS can enable
formal, explicit and transparent BR assessment
while taking uncertainty into account
The implemented decision modeling allows
imprecise or (wholly or partially) missing preferences
may enable a decision without the need for exact weights
Making preference elicitation
less time consuming
more feasible in groups
less controversial (unless controversial trade-off pivotal)
Introduction Benefit-risk Discussion References
Key lessons & future work
Key lessons
Decision support system based on trials data feasible
Requires semantically structured data sets
Statistical analysis should be tailored to decision
Requires semantically structured data sets
But structured data is hardly available
Future work
Beyond proof-of-concept
Further statistical & decision modelingEven better data modelingValorization / product development
Make structured repository of clinical trials data (?!)
Introduction Benefit-risk Discussion References
P. M. Coplan, R. A. Noel, B. S. Levitan, J. Ferguson, and F. Mussen.Development of a framework for enhancing the transparency, reproducibilityand communication of the benefit-risk balance of medicines. ClinicalPharmacology and Therapeutics, 89(2):312–315, 2011. doi:10.1038/clpt.2010.291.
L. D. Lynd and B. J. O’Brien. Advances in risk-benefit evaluation usingprobabilistic simulation methods: an application to the prophylaxis of deepvein thrombosis. Journal of Clinical Epidemiology, 57(8):795–803, 2004.doi: 10.1016/j.jclinepi.2003.12.012.
T. Tervonen, B. de Brock, P. de Graeff, and H. Hillege. Current status andfuture perspectives on drug information systems. In Proceedings of the 18thEuropean Conference on Information Systems, ECIS 2010, Pretoria, SouthAfrica, 2010.
T. Tervonen, G. van Valkenhoef, E. Buskens, H. L. Hillege, and D. Postmus. Astochastic multi-criteria model for evidence-based decision making in drugbenefit-risk analysis. Statistics in Medicine, 30(12):1419–1428, 2011. doi:10.1002/sim.4194.
Introduction Benefit-risk Discussion References
G. van Valkenhoef, , G. Lu, B. de Brock, H. Hillege, A. E. Ades, and N. J.Welton. Automating network meta-analysis. Submitted manuscript, 2011a.
G. van Valkenhoef, T. Tervonen, B. de Brock, and H. Hillege. Deficiencies inthe transfer and availability of clinical evidence in drug development andregulation. Submitted manuscript, 2011b.
G. van Valkenhoef, T. Tervonen, B. de Brock, and H. Hillege. Algorithmicparametrization of mixed treatment comparisons. Statistics and Computing,2011c. doi: 10.1007/s11222-011-9281-9. (in press).
G. van Valkenhoef, T. Tervonen, J. Zhao, B. de Brock, H. L. Hillege, andD. Postmus. Multi-criteria benefit-risk assessment using networkmeta-analysis. Journal of Clinical Epidemiology, 2011d. doi:10.1016/j.jclinepi.2011.09.005. (in press).
G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, and H. Hillege.ADDIS: a decision support system for evidence-based medicine. Submittedmanuscript, 2011e.
top related