biomedcom evidem a practical framework to support healthcare decisionmaking, assess quality of...
Post on 05-Jan-2016
212 Views
Preview:
TRANSCRIPT
BioMedComBioMedCom
EVIDEM
A practical framework toA practical framework tosupport healthcare decisionmaking, support healthcare decisionmaking, assess quality of evidence, and assess quality of evidence, and define explicit data needsdefine explicit data needs
Mireille M Goetghebeur PhDMireille M Goetghebeur PhD, Monika Wagner PhD, Hanane , Monika Wagner PhD, Hanane Khoury PhD, Randy Levitt PhD, Lonny J Erickson PhD, Donna Khoury PhD, Randy Levitt PhD, Lonny J Erickson PhD, Donna
Rindress PhDRindress PhD
BioMedCom Consultants incBioMedCom Consultants incMontreal, CanadaMontreal, Canada
Evidence and Value: Impact on DEcisionMaking -A practical framework applying Multi-Criteria
Decision Analysis to support healthcare decisionmaking
BioMedComBioMedCom
Healthcare decisionmaking is a complex process that requires Healthcare decisionmaking is a complex process that requires simultaneous integration of numerous disparate types of evidence simultaneous integration of numerous disparate types of evidence and value judgmentsand value judgments
There is a need, nationally and internationally for a transparent There is a need, nationally and internationally for a transparent access to evidence and values on which decisions are based for access to evidence and values on which decisions are based for coverage/use of healthcare interventions. coverage/use of healthcare interventions.
Ad-hoc assessment of a healthcare intervention without breakdown Ad-hoc assessment of a healthcare intervention without breakdown into components may result in loss of information and possibly into components may result in loss of information and possibly biased valuationbiased valuation
Healthcare decisionmaking could be facilitated by structuring Healthcare decisionmaking could be facilitated by structuring evidence and value judgments on which it is based into a practical evidence and value judgments on which it is based into a practical and transparent architectureand transparent architecture
Transparency will enhance understanding of healthcare decisionsTransparency will enhance understanding of healthcare decisions
Background & HypothesesBackground & Hypotheses
BioMedComBioMedCom
Breakdown components of healthcare decisionmaking into Breakdown components of healthcare decisionmaking into practical instruments structuring and quantifying practical instruments structuring and quantifying assessment of healthcare interventions to facilitate assessment of healthcare interventions to facilitate decisionmakingdecisionmaking
Build an iceberg architecture to provide multiple layers of Build an iceberg architecture to provide multiple layers of transparent access to components of decisionmakingtransparent access to components of decisionmaking
ObjectivesObjectives
Ad hoc assessment
Burden of disease Budget impact
Type of medical service Healthcare savings
Improvement of efficacy Equity
Cost-effectiveness Etc…
Ad hoc assessment
Burden of disease Budget impact
Type of medical service Healthcare savings
Improvement of efficacy Equity
Cost-effectiveness Etc…
Ultimately, optimize health by best use of healthcare Ultimately, optimize health by best use of healthcare interventionsinterventions
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix International standards
Decisionmaking body requirementsCriteria of quality
Synthesized evidence
For each component of Value MatrixFully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix International standards
Decisionmaking body requirementsCriteria of quality
Synthesized evidence
For each component of Value MatrixFully referenced
Extrinsic value
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix International standards
Decisionmaking body requirementsCriteria of quality
Synthesized evidence
For each component of Value MatrixFully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
BioMedComBioMedCom
Conceptual frameworkConceptual frameworkComponents of decisionmakingComponents of decisionmaking
1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443.
BioMedComBioMedCom
Quality of of
evidence
Scientific judgment Value judgment
Decisionmaking process
Intrinsicvalue
of product
Extrinsicvalue
of product
1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443.
Based on international standards Comprehensive analyses can be
systematized Not highly dependent on evaluator
perspective
Similar value system Requires value judgment Dependent on evaluator
perspective
No shared value systemDependent on
region/institution priorities
Improvement over existing treatment
None Lower
Major Higher
Efficacy
Validity of efficacy trial
Low Small
High Large
Number of patients Etc…
Type of population treated (equity)
Low priority? High priority?
Vulnerable vs productive patients
Etc…
EVIDEM – What?EVIDEM – What?Practical iceberg architecture to support Practical iceberg architecture to support decisionmakingdecisionmaking
BioMedComBioMedCom
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Intrinsic value of treatment
Value Matrix Multi Criteria Decision Analysis
(MCDA)
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
BioMedComBioMedCom
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
BioMedComBioMedCom
EVIDEM – Who?EVIDEM – Who?Communication tool to connect stakeholdersCommunication tool to connect stakeholders
Decisionmakers
Data producers
EVIDEM investigators & experts
Transparent
communication
tool
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
Intrinsic value of treatment
Value MatrixMulti Criteria Decision Analysis
(MCDA)
Decision
Quality of evidence
Quality Matrix Criteria of quality
International standards Decisionmaking body requirements
Synthesized evidence
Key info for each component of Value Matrix
Fully referenced
Extrinsic value
Evidence
Full text source documents (published, registries, proprietary)
BioMedComBioMedCom
EVIDEM - Why?EVIDEM - Why?Sharing of value judgments and structured multilevel Sharing of value judgments and structured multilevel access to dataaccess to data
Focus on valueSharing of weights & scores
Transparent access
to quality of data and methods
Transparent multilayeraccess to evidence
Limit obstruction of thinking process
by large amount of data
Can be explored more easily
More transparent and understandable
EVIDEM – How?EVIDEM – How?Integrated process – example “Improvement of Integrated process – example “Improvement of efficacy”efficacy”
Value Matrix component T3: Improvement of medical service - efficacy
Extrinsic value
Decision
Value Matrix (MCDA)
Evidence - Full text sources/links
RegistriesPublished
trialsProprietary
data
Meta-Analyses
(Cochrane)
Synthesized clinical evidence
References
Quality Matrix: quality of clinical evidence
Component Questions Comments Score
Clinical data How comprehensive is the clinical program? Are results across trials consistent?Are standard elements relevant and valid?(COMPLETE INSTRUMENT BELOW -1 per comparator)
BioMedComBioMedCom
2
BioMedComBioMedCom
EVIDEM tools – MCDA Value Matrix EVIDEM tools – MCDA Value Matrix 15 components defining intrinsic value of treatment15 components defining intrinsic value of treatment
Cluster Components of value assessment (alphabetical by cluster) Weight
1-5 Synthesized information*
Score 0-3
Value
Quality of dossier and evidence
Q1 Adherence to requirements of decisionmaking body (dossier)
Q2 Comprehensiveness and accuracy of evidence presented (dossier)
Q3 Relevance and validity of evidence (literature and dossier)
Disease impact
D1 Disease severity (risk of death & disability, acuteness)
D2 Size of population affected by disease
Treatment
T1 Current clinical guidelines on treatment or products of same class
T2 Current treatment limitations
T3 Improvement of medical service – efficacy
Independent of product
Structured summary of evidence for
treatment
Scale anchors and guidelines
Value score (linear model)
T4 Improvement of medical service – safety & tolerability
T5 Improvement of medical service – patient reported outcomes, convenience & adherence
T6 Public health interest (prevention & risk reduction)
T7 Type of medical service (symptom relief, prolonging life, cure etc.)
Economics
E1 Budget impact of reimbursing treatment on drug plans
E2 Cost-effectiveness of treatment
E3 Savings with treatment (impact on healthcare spending excluding treatment cost)
Total Aggregated value score (% of maximum score)
*Available information from public domain and manufacturer dossier
Quality Matrix
Components identified from literature review and current decisionmaking processes, and selected to fulfill MCDA guidelines on clustering, completeness, redundancy, operationality & mutual independence (Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.; National Economic Research Associated. Multi-criteria analysis manual 2005. http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf Accessed Jan2007)
BioMedComBioMedCom
EVIDEM tools - Quality MatrixEVIDEM tools - Quality Matrix12 types of evidence, 3 criteria of quality12 types of evidence, 3 criteria of quality
Structured based on criteria defining quality and on requirements from more than 20 decisionmaking bodies worldwide ( INTERFACE database. 2000-2008. www.biomedcom.org); Assessment for each component based on instruments derived from quality standards (e.g., CONSORT, CHEC, STROBE, Siegel et al, Mauskopf et al) as well as guidelines and processes from decisionmaking bodies
BioMedComBioMedCom
Pilot study in Canadian contextPilot study in Canadian contextProof of concept using historical casesProof of concept using historical cases
Historical cases: 10 medicines (cardiovascular diseases, endocrinology, infectious diseases neurology, oncology, ophthalmology,) using data from literature review and manufacturer dossiers submitted to the Canadian Common Drug Review (CDR) and Québec Conseil du Médicament (CM)
QM scores: EVIDEM investigators; VM weights and scores, and feedback on process: Canadian Value Panel
composed of representative stakeholders across Canada (decisionmakers, specialists, generalists, nurses, pharmacists, health economists/epidemiologists)
Feasibility: Algorithm developed to operationalize each cell of the matrices; applicable to any therapeutic areas and jurisdiction
Practicality: 30 min on average to apply VM by stakeholders; about 250 hrs to build the structured package of fully traceable information (quality, synthesized format and full text access) and value scores
Feedback from panelist: Value of EVIDEM in structuring evidence, assessing strengths and weaknesses systematically, and sharing values and value scores
Limitations of pilot: limited access for panelists to underlying source data (electronic interface to be developed); extrinsic components not covered (to be explored)
BioMedComBioMedCom
Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for clusters 1 & 2Value assessment example for clusters 1 & 2
ClusterComponents of value assessment
(alphabetical by cluster)Scale anchors
% Max Value score – Mean (SD)
Quality of dossier and evidence
Q1 Adherence to requirements of decisionmaking body
0=Low adherence3=High adherence
56 (17)
Q2 Comprehensiveness and accuracy of evidence presented
0=Many gaps / inaccurate3=Comprehensive and accurate
72 (14)
Q3 Relevance and validity of evidence 0=Low relevance/validity3=High relevance/validity
78 (17)
Disease impact
D1 Disease severity (risk of death & disability, acuteness)
0=Not severe (minor inconvenience)3=Very severe
100 (0)
D2 Size of population affected by disease 0=Extremely rare disease3=Common disease
22 (27)
Anchors defined from a societal perspective, i.e., optimizing health at the societal levelAnchors defined from a societal perspective, i.e., optimizing health at the societal level Score options translate to 0% (score of 0), 33% (score of 1), 66% (score of 2) and 100% (score of 3) Score options translate to 0% (score of 0), 33% (score of 1), 66% (score of 2) and 100% (score of 3)
BioMedComBioMedCom
ClusterComponents of value assessment
(alphabetical by cluster)Scale anchors
% Max Value score – Mean (SD)
Treatment
T1 Current clinical guidelines on treatment or products of same class*
0=Not recommended3= Strong first-line recommendationNA=not available
67 (21)
T2 Current treatment limitations 0=No or very minor limitations3=Major limitations
89 (17)
T3 Improvement of medical service – efficacy 0=Lower efficacy than comparator treatments presented3=Major improvement in efficacy vs comparator
treatments presented
72 (25)
T4 Improvement of medical service – safety & tolerability
0=Lower safety / tolerability than comparator treatments presented
3=Major improvement in safety / tolerability vs comparator treatments presented
61 (14)
T5 Improvement of medical service – patient reported outcomes (PRO), convenience & adherence
0=Worse PRO / lower convenience / lower adherence vs comparator treatments presented
3=Major improvement
50 (18)
T6 Public health interest (prevention & risk reduction)
0=No risk reduction compared to no intervention3=Major risk reductionNA=not applicable
NA†
T7 Type of medical service (symptom relief, prolonging life, cure etc.)
0=Minor service3=Major service
72 (14)
Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for cluster 3Value assessment example for cluster 3
BioMedComBioMedCom
ClusterComponents of value assessment
(alphabetical by cluster)Scale anchors
% Max Value score – Mean (SD)
Economics
E1 Budget impact of reimbursing treatment on drug plans
0=Strong impact on drug plan budget (significant additional expenditures)3=Substantial savings for drug plans
61 (25)
E2 Cost-effectiveness of treatment 0=Not cost-effective3=Highly cost-effective
89 (17)
E3 Savings with treatment (impact on healthcare spending excluding treatment cost)
0=Substantial additional healthcare spending, excluding product cost3=Substantial savings in healthcare spending, excluding product cost
67 (21)
Pilot study - Value MatrixPilot study - Value Matrix Value assessment example for cluster 4Value assessment example for cluster 4
BioMedComBioMedCom
Potential applicationsPotential applications Transparent access to evidence and valuesTransparent access to evidence and values
EVIDEM framework
BioMedComBioMedCom
Adapt framework to existing
decisionmaking processes
Basis to develop extrinsic value components
Application axisProspective
Application axisRetrospectiveValidate process
in various jurisdictions
Explore context of past decisions
Generate data on quality of evidence
Planning tool for researchers and intervention developers to meet
explicit needsDevelop methodology to generate data tailored to critical data needs
Collaboration axisResearch planning
Define explicit needs of decisionmakers
Establish reasonable requirements
Collaboration axisExplicit data needs
BioMedComBioMedCom
Future developmentsFuture developments
Collaborative studies and iterative processes to explore the value of EVIDEM in context
Interactive web based architecture integrating evidence and value for various healthcare interventions
Expected outcome of a systematized and shareable approach for data access and value assessment is to optimize resources, decisions and health
BioMedComBioMedCom
AcknowledgmentsAcknowledgments
EVIDEM Canadian Value panelEVIDEM Canadian Value panel: : Jean-Francois Bussières BPharm MSc MBA FCSHP, Director, Pharmaceutical Practice Research Unit, Dept of Pharmacy, CHU Sainte-Justine Research Center; Clinical Associate Professor, Faculty of Pharmacy, University of Montréal.
Benoit Cossette BPharm MSc, Pharmacist, Coordinator, Drug Disease Management Program, Fleurimont Hospital, Sherbrooke University Hospital Centre.
Doug Coyle PhD, Associate Professor, Faculty of Medicine, Epidemiology and Community Medicine, University of Ottawa.
Cheri Deal MD PhD FRCPC, Associate Professor and Program Director, Endocrine Service, CHU Sainte-Justine Research Center, University of Montréal.
Roland Grad MD CM MSc CCFP FCFP, Associate Professor, Department of Family Medicine, McGill University.
Christine Lee MD, FRCPC Assistant Professor, Director of the Microbiology Residency Program, Department of Pathology and Molecular Medicine, McMaster University.
Mitchell Levine MD FRCPC FISPE, Professor, Director of Centre for Evaluation of Medicines Department of Clinical Epidemiology & Biostatistics, Department of Medicine, McMaster University
Diane Lowden RN MSc, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University.
John Mancini MD FRCP, Professor/Program Director Continuing Medical Education, Department of Cardiology, University of British Columbia.
Paul Oh MD FRCPC, Medical Director of the Cardiac Rehab and Secondary Prevention Program, Department of Medicine, Clinical Pharmacology and Toxicology, University of Toronto.
Genevieve Tousignant N MScN, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University.
Wendy Ungar PhD, Senior Scientist, Child Health Evaluative Sciences, The Hospital for Sick Children; Associate Professor, Health Policy, Management, and Evaluation, University of Toronto.
Marie-Claude Vanier BPharm MSc, Clinical Associate Professor, Faculty of Pharmacy, University of Montréal; Pharmacist UMF-GMF Cité de la Santé de Laval.
Funded by an unrestricted research grant from Pfizer CanadaFunded by an unrestricted research grant from Pfizer Canada
BioMedComBioMedCom
Thank youThank you
EVIDEMEVIDEM tools are open accessEVIDEM tools are open access
We welcome input and collaborationsWe welcome input and collaborationsPlease contact the EVIDEM Group @Please contact the EVIDEM Group @
www.evidem.org
mireille_goetghebeur@biomedcom.orgmireille_goetghebeur@biomedcom.org
top related