ppt - innovative approaches database study design and analysis
TRANSCRIPT
Innovative Approaches to Database Study Design
and Analysis: Opportunities and Challenges
David Neasham, PhD MSc MFPH
Andrew Cox, PhD
31 January 2013
Background
2
Navigating medical products successfully to market and beyond
is highly complex – compounded by the fact that understanding the needs and criteria
of stakeholders (whether regulatory, payor, patient or prescriber) often requires
substantial effort and strategic thinking. Anticipating stakeholder needs for effective
planning allows for evidence to be produced that will be of value to stakeholders.
With the changing regulatory and payor environment, what are the implications
for database study design and analysis in the future?
Background (Cont’d)
3
This webinar explores these issues, highlighting opportunities, challenges, presenting
real-world examples and providing practical guidance.
Agenda
Introduction: From a regulatory and payor perspective
Increasing need for database studies (regulatory, payor, value demonstration,
positioning drug in market, treatment switching and patient-perspective studies)
Rationale
Approaches to database study design/analysis 1. Database study programmes in Europe
2. Study designs
3. Data mining (e.g., treatment patterns and patient profiling)
4. New and evolving data sources: Drug Utilisation (label) studies
Summary
Discussion/Q&A
Current Landscape....
FDA requirements
EMA requirements
Early drug
development
Phase III drug
development
Regulatory
submissions
Lic
ence
Lau
nch
Product
promotion
RMPs/REMS
US value demonstration
EU/national value
demonstration
pre-authorisation
eHCD studies have typically taken place within the framework of RMPs/REMS additional studies. However,
increasingly, eHCDs are being used in comparative-effectiveness research for payors and earlier in drug development
to investigate issues such as treatment switching patterns and patient profiling.
Safety surveillance
payors
Regulators Benefit: risk
requirements
EU/national value
demonstration
post-authorisation
Evidence for prescribers?
Additional studies
EU = Europe; EMA: European Medicines Agency; FDA: Food and Drug Administration; US = United States; eHCD = Electronic Health Care Database;
RMP: Risk Management Plan; REMS: Risk Evaluation and Mitigation Strategies
5
Paradigm Shift
Regulation 1235/2010 and Directive 2010/84/EC
on EU pharmacovigilance published on 31 December
2010
On July 2012, legislation was activated
International Conference on Harmonisation Technical
Requirements for Registration of Pharmaceuticals for
Human Use (ICH) E2C (R2) guideline Periodic
Safety Update Reports (PSURs) will become basis
of periodic benefit-risk evaluation reports
Periodic Benefit-Risk Evaluation Report (PBRER) is intended to be a common standard
for periodic benefit-risk evaluation reporting on marketed products
January 2013, the European Medicines Agency (EMA) will expect market authorisation
holders to submit the new PSUR
Requirements: PSUR will change from being largely a document based on long
line listings, narratives and simple sales-based incidence statistics, to a complex
benefit-risk tool
Data Sources for Benefit-risk Profile
Data contributing to the benefit-risk profile of a medicine usually generated
from at least three sources
1. Randomised controlled trials (RCTs)
2. Post-marketing surveillance (spontaneous reporting systems)
3. Pharmacoepidemiology studies
7
Comparative-effectiveness
Effectiveness research assesses whether an intervention does more good than
harm when provided under conditions of routine care
Efficacy data comparing a drug with placebo are rarely relevant in routine practice
where one or more alternative therapies are available for most conditions
Many randomised efficacy
trials exclude patient
populations that will use most
of the drugs, including older
adults and patients with
multiple morbidities
Most RCTs
for drug approval
Goal of comparative-
effectiveness ratio
(CER)
Effectiveness (does it
work in routine care?) Efficacy (can it work?)
Placebo
comparison
(for usual
care)
Active
comparison
(head-to-
head)
Figure adapted from: Comparative-effectiveness research (CER), Goal of CER
in contrast to preapproval randomised controlled trials (RCTs). Clinical Pharmcology and Therapeutics. 2011 December; 90(6): 778.
8
9
Phase I + II Phase
IIIa Phase
IIIb Phase
Iva Phase
IVb
More comparative-
effectiveness
research
Value of efficacy
versus comparative-
effectiveness
information
More efficacy
research (placebo
and active controlled)
Drug life cycle
Figure adapted from: Comparative-effectiveness research (CER), The increasing value of CER during the drug life style.
Clinical Pharmcology and Therapeutics. 2011 December; 90(6): 778.
Comparative-effectiveness (Cont’d)
Safety Challenges in Pre-approval Phase
Challenges of safety signal identification during pre-approval phase
Samples often narrowly defined – therefore may not be truly representative of final
treatment population
Pre-approval studies cannot usually be statistically powered to identify readily low
frequency safety events of concern
Short time frames: mid to long-term effects cannot be thoroughly evaluated
10
Approval
Post -marketing
Pharmacovigilance Preclinical Clinical
Product life cycle FIM Ph I Ph II Ph III Ph IV
• Toxicology (e.g.,
genotoxicity assays)
• In silico
analysis/structural alerts
• Margin of safety and no
observed adverse effect
level defined
• Phase I and II: dose
ranging, efficacy and
toxicity
• Phase III: demonstration
of efficacy and safety
signal monitoring
• Phase IV: spontaneous
reporting systems
Exposure
in humans (potential denominator)
Statistical power
Product Life Cycle and Safety Studies
Approval
11
Post-marketing Pharmacovigilance
Limitations of post-marketing spontaneous reporting systems
Patterns of spontaneous reporting vary by country, increase during early post-launch
period and are affected by publicity → spontaneous reporting rates biased
Limited population level denominator information
For common events (e.g., myocardial infarction) difficult to separate actual signals
from background noise
12
Approval
Preclinical Clinical Post-marketing
Pharmacovigilance
Product life cycle FIM Ph I Ph II Ph III Ph IV
• Toxicology (e.g.,
genotoxicity assays)
• In silico
analysis/structural alerts
• Margin of safety and no
observed adverse effect
level defined
• Phase I and II: dose
ranging, efficacy
and toxicity
• Phase III: demonstration
of efficacy and safety
signal monitoring
Exposure in humans
(potential denominator)
Statistical power
Product Life Cycle and Safety Studies
Reporting bias
• Phase IV: spontaneous
reporting systems
Approval
13
Post-marketing Database Studies
This includes large validated medical record databases where data collected
systematically over long time periods
Incidence of potential safety events can be evaluated in general population
Studies are sufficiently-powered
Study replication is relatively easy (important Bradford-Hill criterion for causality)
No self-reporting bias
Rare and common safety events can be evaluated
14
Clinical
Product life cycle FIM Ph I Ph II Ph III Ph IV
• Toxicology (e.g.,
genotoxicity assays)
• In silico
analysis/structural
alerts
• Margin of safety and no
observed adverse effect
level defined
• Phase I and II: dose
ranging, efficacy and
toxicity
• Phase III: demonstration
of efficacy and safety
signal monitoring
Exp
osu
re in h
um
ans (p
oten
tial den
om
inato
r) Product Life Cycle and Safety Studies
Preclinical Post-marketing
Pharmacovigilance
Sufficiently powered
Easy study replication
Large scale
database
studies
No self-reporting bias
Phase IV: spontaneous
reporting systems
Approval
15
Current Landscape....
FDA requirements
EMA requirements
Early drug
development
Phase III drug
development
Regulatory
submissions
Lic
ence
Lau
nch
Product promotion
RMPs/REMS
US value demonstration
EU/national value
demonstration
Pre-authorisation
eHCD studies have typically taken place within the framework of RMPs/REMS additional studies. However
increasingly, eHCDs are being used in comparative effectiveness research for payors and earlier in drug development
to investigate issues such as treatment switching patterns and patient profiling.
Safety surveillance
payors
Regulators Benefit: risk
requirements
EU/national value demonstration
post-authorisation
Evidence for prescribers?
Additional studies
EMA: European Medicines Agency ; FDA: Food and Drug Administration ; RMP: Risk Management Plan;
REMS: Risk Evaluation and Mitigation Strategies
Develop an integrated strategic approach in Europe
Payor research: demonstrating added value to payor; differentiating product from
competitors
Compliance with Committee on Human Medicinal Products (CHMP) requests
for safety information and/or concerns over potential drug utilisation patterns
Economies of scale
Payors are now demanding the use of naturalistic data often based on their
own information, to run observational studies to determine reimbursement,
pricing and formulary status
Data synergies between payor requirement and regulatory studies
Approaches to Database Study Design/Analysis
1. Database study programmes in Europe
17
Database Study Programmes in Europe
18
What is the payor asking for?
Reduced cost to the system
RCT’s ideal, but increasingly, requesting observational studies based
on real-world patient level data
Studies better suited to meet their needs
Database Study Programmes in Europe (Cont’d)
19
Challenge of the data from Client’s perspective?
PROs CONs
• Often the eHCD sources are
what the authorities are asking for
• Large representative real-world
populations
• Longitudinal individual level 1˚ or 2˚
care data, linkable
• Potential for innovative approaches
• Can be very cost-effective
• Data quality
Limitations
Need expertise in understanding
how the data are structured, linked
Level of clinical information
• Privacy and patient confidentiality
major issues
• Access and expertise needed
Database Study Programmes in Europe (Cont’d)
Define research question Pilot study(s)
Review data source options
Develop protocol(s)
• Define clear research
question(s) for
assessing comparative
-effectiveness/safety of
product vs. comparator
treatments
Include:
• A critical appraisal
of effectiveness
outcome measures
and methodologies
across the literature
• A qualitative and/or
quantitative
assessment of specific
outcome definition(s)
• Hypotheses on key
points of differentiation
of product vs.
comparators
• Conduct pilot study(s) in
suitable markets to test the
analytical plan (CER)
• Assess the availability
of electronic
healthcare databases
in the EMA region and
their suitability for
conducting CER
and/or post-
authorisation safety
(PAS) studies
• Develop a view on the
desirability of
conducting such
studies in these
databases
• Identify pilot study
opportunities
• Identify any important
gaps in database
availability and
develop action plans
• Combine the results
from the research
question, methodology
review, and database
evaluation to develop
core protocols: (i)
CER; (ii) PAS study
Develop full EMEA
database study
programme
• Conduct PAS study in
nominated countries
PAS study
Approaches to Database Study Design/Analysis
21
Levels of data according to study needs
eHCD alone
eHCD + linkage to register(s)
eHCD + physician text comments
eHCD + de novo data collection CRF or electronic CRF (eCRF)
eHCD + combinations of ii-iv
e-Registers (indication-specific)
Cohort studies + linkage to eHCD
2. Study designs
Hybrid database-case report forms (CRF) studies
Study Designs
22 22
Sample frame for identifying patients (cf. de novo recruitment)
Efficient method (saving time and money)
Rationale
Additional clinical data required
Validate cases
Outcome measures needed (e.g., hospitalisation)
Key requirement of regulator/payor (e.g., validation of mortality data through
linkage to deaths registration system)
GP supplemental data and/or patient supplemental data
Study Designs (Cont’d)
23
Background
Risk of seizures, venous thromboembolism (VTE) and stroke among patients
with Alzheimer’s Disease (AD)
Stroke reportedly associated with AD, though the direction of the association
is uncertain
Example: UBC Study using eHCD + Physician text comments + de
novo data collection (CRF)
Objective
To measure and compare risk of seizure, VTE and stroke in patients
with and without AD
To test an algorithm validating outcomes, by integrating three data sources
To examine the relation between AD and stroke type
Study Designs (Cont’d)
24
Incidence Rates for Stroke for Overall Study
Cohort and by Age and Gender
Patient
Characteristic
AD patients Non-AD patients
Crude RR 95% CI N
Number of
Events
Person-
Years
of Follow-
Up
Incidence
Rate* 95% CI N
Number of
Events
Person-
Years
of Follow-
Up
Incidence
Rate* 95% CI
Total 9,953 361 22,682 15.9 14.4 17.6 9,953 439 35,673 12.3 11.2 13.5 1.29 1.12 1.49
Males
50-69 443 16 1,379 11.6 7.1 18.9 443 8 2,161 3.7 1.9 7.4 3.13 1.27 8.46
70-74 461 16 1,217 13.2 8.1 21.5 461 23 1,971 11.7 7.8 17.6 1.13 0.56 2.23
75-79 740 29 1,678 17.3 12.0 24.9 742 40 2,825 14.2 10.4 19.3 1.22 0.73 2.02
80-84 803 26 1,576 16.5 11.2 24.2 804 50 2,519 19.9 15.0 26.2 0.83 0.50 1.36
85+ 756 19 1,158 16.4 10.5 25.7 753 35 1,729 20.2 14.5 28.2 0.81 0.44 1.46
Total 3,203 106 7,008 15.1 12.5 18.3 3,203 156 11,205 13.9 11.9 16.3 1.09 0.84 1.40
Females
50-69 587 21 1,939 10.8 7.1 16.6 587 10 2,974 3.4 1.8 6.2 3.22 1.45 7.66
70-74 768 34 2,284 14.9 10.6 20.8 768 24 3,635 6.6 4.4 9.8 2.26 1.30 3.97
75-79 1,402 57 3,540 16.1 12.4 20.9 1,404 55 5,708 9.6 7.4 12.6 1.67 1.13 2.47
80-84 1,781 64 3,888 16.5 12.9 21.0 1,782 84 6,325 13.3 10.7 16.4 1.24 0.88 1.74
85+ 2,212 79 4,024 19.6 15.7 24.5 2,209 110 5,827 18.9 15.7 22.8 1.04 0.77 1.40
Total 6,750 255 15,674 16.3 14.4 18.4 6,750 283 24,469 11.6 10.3 13.0 1.41 1.18 1.67
* Per 1,000 person-years
AD = Alzheimer's disease
Study Designs (Cont’d)
25
Endpoint Validation Results (Stroke)
Item no. Clinical review (profile + comments) N Cases PPV
Clinical assessment
1 Yes 92
3 No 3
4 Subtotal with clinical assessment 95 96.8
GP assessment
5 Yes 83
6 No 12
7 Subtotal with GP review 95 87.4
GP notes
8 Yes 86
10 No 9
11 Subtotal with GP notes 95 90.5
Study Designs (Cont’d)
26
Proof of concept – this kind of complex linkage is feasible and could be basis
of larger scale linkage (if needed)
Excellent means of validating quality of electronic data
Data quality: critical issue for regulators and payors
eHCD + physician text comments + de novo data collection (CRF)
27
Introduction: from a regulatory and payor perspective
Increasing need for database studies
Rationale
Approaches to database study design/analysis
1. Database study programs in Europe
2. Hybrid study designs
3. Data mining (e.g., treatment patterns and patient profiling)
4. Drug utilisation (label) studies in patient care databases
Data Mining/Exploratory Analysis
A Simple Example: Treatment Patterns
Look no hypothesis!
29
Data Mining/Exploratory Data Analysis
29
What is data mining?
• The process of discovering interesting patterns and knowledge from large
amounts of data (Han et al. 2011)
• Hypothesis and model free approach
• Hypothesis generating
• Discovered patterns can be then investigated by classical analytical techniques
Why data mining?
• Increasing volume of data
• Data from disconnected sources
Where is data mining currently used?
• Widely used in other fields of industry
• Discovery of Higgs Boson
• Financial and banking
• Genomics
• Online security
Source: Han, Jiawei; Kamber, Micheline; Data Mining: Concepts and Techniques: Elsevier Science, United
States, 2011
30 30
Psoriasis (PsO)
Psoriasis is an immune mediated chronic skin disease
Psoriatic Arthritis (PsA)
Chronic inflammatory arthritis associated with psoriasis
Data source is Clinical Practice Research Database (CPRD)
• Development phase analysis results shown
• Randomly selected sub sample
• For analytical development at this stage
Background
Descriptive Treatment Histories for PsO Patients
31
50 Randomly-selected PsO Treatment Sequences
PsO Treatment History Frequency Plot L
eg
en
d
Descriptive Treatment Histories for PsA Patients
32 32
Leg
en
d
50 Randomly-selected PsA Treatment Sequences
PsA Treatment History Frequency Plot
Cluster
Analysis
33
PsA Treatment History Clustering
PsO Patients Treatment History Clustering
PsA Treatment History Clustering
34
PsO Patients Treatment History Clustering
Three
groups
Cluster
Analysis
Treatment Typologies for PsA
Mean Time on Different Treatments
36
Further Analysis of PsA Patients with a Type 3 Treatment
History
37
Application of machine-learning techniques
Are there particular patient characteristics that predict type 2 membership
How well can we predict them from characteristics other than treatment
Primary predictors (possible)
Age at diagnosis >60
Smoking status
Time since diagnosis
Others...
Further Analysis of PsA Patients with a Type 3 Treatment
History (Cont’d)
38
Analysis can be augmented and developed
More data
Free text from general practitioner (GP) notes
May lead to recognition of a new patient group with specific needs
May lead to an already-recognised patient group
39
New and Evolving Data Sources
Drug Utilisation (Label) Studies
Drug Utilisation (Label) Studies
40
Modafinil approved for use in the UK in 2002; and in the US in 1998
Modafanil
Indicated for daytime sleepiness associated with narcolepsy, obstructive sleep
apnea/hypopnea syndrome and chronic shift work sleep disorders
In January 2011 CHMP concluded
Based on review of evidence (real world evidence not included)
For obstructive sleep apnoea, shift-work sleep disorder and idiopathic
hypersomnia effectiveness were not sufficient to out weigh risks, and that
therefore the benefit-risk balance was negative
Indication was therefore narrowed further
See EMA URL: http://www.ema.europa.eu/docs/en_GB/document_library/Referrals_
document/Modafinil_31/WC500099177.pdf
Source: European Medicines Agency. Questions and answers on the review of medicines containing Modafinil . 27 January 2011. Avaiable at:
http://www.ema.europa.eu/docs/en_GB/document_library/Referrals_document/Modafinil_31/WC500099177.pdf. Accessed 30January 2013.
Online Patient Platforms
41
Online patient platforms,
as a repository for patient-
reported outcomes, provide
an opportunity to create new
methods to study the effect
of drugs after they have
reached the market, as well
as, contributing in multiple-
study programmes together
with more conventional
data sources such as
Clinical Practice Research
Datalink.
Drug Utilisation (Label) Studies in eHCD and Patient Care
Databases
Study examining off-label use 21 January 2011*
PatientsLikeMe.com – Modafanil
* Source: Patients like me. 2013. Available at: patientslikeme.com. Accessed 30 January 2013;
Frost, Y, et al. Patient-reported outcomes as a source of evidence in off-label prescribing: analysis of data from PatientsLikeMe. J Med Internet
Res. 2011 Jan 21;13(1)e6
Found substantial off-label use
<1% of 1755 patients taking Modafinil reported taking each drug for purposes
approved by the FDA
Where using for fatigue associated with depression, multiple sclerosis, attention
deficit/hyperactivity disorder, mood, schizophrenia, myotonic dystrophy, Parkinson’s
disease and seasonal affective disorder
Patients subjectively reported effectiveness for off-label use as either higher
than or comparable to approved indications!
This study has also been validated through a parallel study using CPRD
as a data source (unpublished)
Drug Utilisation (Label) Studies
43
PatientsLikeMe.com – Modafanil
Indicated use for Modafanil was further restricted by CHMP 2011, due to safety
review
In the same year, a study was published indicating in the US, 99% of Modafinil
use, was off-label. Similar findings have been made for the UK in an as yet
unpublished CPRD study
Patient Perspectives
Drug regulators should also lead the way in developing a consensus
on what is considered a tolerable level of risk. Experts in other areas of risk
regulation, such as the regulation of carcinogenic residues in food or of
nuclear power, have long recognized that calls for zero tolerance of risk are
untenable and have instead established a consensus definition of ‘negligible’
and ‘tolerable’ risk levels. There is evidence that patients are willing to accept
a finite level of risk for a given benefit…We believe that public health would
be better served if patients’ views of acceptable risk levels were
incorporated into benefit–risk assessments.
Source: Eichler H-G, Abadie E, Raine J, Salmonson T. Safe drugs and the cost of good intentions.
N Engl J Med. 2009 r 2;360(14):1378–80. ’
‘
44
Summary
Introduction: From a regulatory and payor perspective
Increasing need for database studies (regulatory, payor, value demonstration,
positioning drug in market, treatment switching and patient-perspective studies)
Rationale
Approaches to database study design/analysis 1. Database study programmes in Europe
2. Study designs
3. Data mining (e.g., treatment patterns and patient profiling)
4. New and evolving data sources: Drug Utilisation (label) studies
46
Discussion/Q&A
Contact
47
Pamela Murray, EU Strategy and Client Development, United BioSource Corporation (UBC)
Direct: +44 (0)20 8834 9559
Dr David Neasham, EU Director, Epidemiology and Database Analytics, UBC
Dr Andrew Cox, Senior Research Associate, UBC