Incorporating Risk-Based MonitoringStrategies: Challengesin Implementation

Sherri Hubby, Director, U.S. Quality Assurance

.

26th AnnualEuroMeeting

25-27 March 2014ACV, Vienna

Austria

DisclaimerThe views and opinions expressed in the following PowerPoint slides are those of the individual presenter and should not be attributed to Drug Information Association, Inc. (“DIA”), its directors, officers, employees, volunteers, members, chapters, councils, Special Interest Area Communities or affiliates, or any organization with which the presenter is employed or affiliated.

These PowerPoint slides are the intellectual property of the individual presenter and are protected under the copyright laws of the United States of America and other countries. Used by permission. All rights reserved. Drug Information Association, DIA and DIA logo are registered trademarks or trademarks of Drug Information Association Inc. All other trademarks are the property of their respective owners.

Applying Quality RiskManagement to clinical trials“A systematic process for the assessment, control, communication and review of risks to the quality of the drug (medicinal product across the product lifecycle” (ICH Q-9)

Questions to raise on assessing quality:• What might go wrong?

• What is the likelihood (probability) it will go wrong?

• What are the consequences?

3

Goals of Risk-Based Monitoring

• Provides strategy for on-site and remote monitoring

• Monitor the right data (Identified Risks)

• Addresses the risks of the study (Risk Mitigation Plan)

• Allows better quality of data per ALCOA principles –Accurate, Legible, Contemporaneous, Original and Attributable.

• Allows quick identification and issue escalation in real time

• Results in better utilization of resources

4

What are the Considerations forRisk-Based Monitoring?• Monitoring experience to apply a metrics driven approach to centralized

monitoring?

• Experience/relationship in working with the investigator site?

• Is the Investigator site trained in GCP ICH?

• Does the investigator have sufficient experience in Therapeutic area and/or compound under study

• Experience in the patient population under study

• Stage of the study

• Critical Primary Efficacy, Outcome data and Safety Endpoints

• Statistical tools available monitor

• Sponsor/CROs experience in coordinated team approach

• Quantity of data – size, duration, number of sites – centralized monitoring tools may help

• Safety of the investigational product

5

Comparison of Traditional versus Adaptive Risk-Based Monitoring

6

Traditional Monitoring Risk-Based Monitoring

Standard -conduct pre-study visit

On-site pre-study visits based on risks: No pre-study on-site visit if site utilized within the last 6

months and clean compliance history Phone interviews with site personnel could be used in

lieu of on-site visit if site properly qualified

Standard: Conduct face to face InvestigatorMeeting and on-site initiation visit

On-site initiation visit based on site qualification: In the compound or therapeutic area Experience with other sponsors with similar

protocols or programs Timing of their last site qualification visit Whether changes have occurred since last

assessment Site indicators which may affect quality: high

turnover, PI availability and responsiveness

Comparison of Traditional versusAdaptive Risk-Based Monitoring

7

Traditional Monitoring Risk-Based MonitoringOn-Site Monitoring is the Standard 100% of ICFs, Inclusion/

Exclusion Criteria, AEs, Labs, EKGs, IMP

Regulatory Binder as time permits

Focused on site-level tasks Focused on one site and one

variable at a time Query resolution per patient No emphasis placed on review

of primary and secondary efficacy data points as compared to review of other data

Centralized + On-site monitoring models: 100% ICFs for all subjects Utilizes statistical monitoring

concepts o 100% review of primary efficacy

data and SAEs for first 2 subjectso 20% sampling for SAEs and

outcome data o Targeted monitoring for sites

with significant compliance issues

Remote review of labs, EKGs Receives data for centralized review

remotely as it is being generated and entered to spot issues early

Comparison of Traditional versusAdaptive Risk-Based Monitoring

8

Traditional Monitoring Risk-Based MonitoringReview of Source:100% Source Data Verification; driven by: Monitor identifying errors at

site Quality and timeliness of

monitoring Issues reflected in the

monitoring reports and follow-up letters to the sites

Highly dependent upon escalation system for reporting to EC and regulators

Review of Source:Data driven by Pre-identification of metrics at start of study and by measuring site compliance regarding: Number of protocol deviations Unusual trends in data regarding

AEs/SAEs and safety signals High number of queries Quality triggers – data of poor

quality and/or untimely corrections

Lack of PI oversight/involvement Use of cross-functional teams –

Comparison of Traditional versus Adaptive Risk-Based Monitoring

9

Traditional Monitoring Risk-Based Monitoring

Rigid, on-site monitoring visits Scheduled at 4 – 8

week intervals All sites start out

with the same plan for frequency of monitoring

Monitoring visits triggered by risk-specified criteria: Based on key risk metrics and risk scores used to

trigger a visit such as… Data quality trending issues identified: training required across sites data integrity issues unusually high or low screen failure rates lab issues indicating sample integrity issues high number of edit checks with data quality

issues edit checks with high number of manual queries

What Types of Systems Fit theCentralized Monitoring Model?According to the MHRA:

• Eligibility checks

• Rates of Recruitment, Withdrawals, and losses to follow-ups by sites

• Checks for missing or invalid data

• Checks that dose adjustments, investigation and management of events are consistent with the protocol

• Calendar Checks

• Checks for unusual data patterns

• Assessment of AEs and Toxicity Reporting Rates

• CRFs completed by authorized persons

• External verification

Reference: MRC/DH/MHRA Joint Project: Version: 10th October 2011

10

Challenges – What types of plans do Ineed for risk-based monitoring?

Four (4) Plans you should consider:

• Quality Monitoring Plan as a part of the Clinical Monitoring Plan

• Study Level plans

• Site Level Plan

• Program Level Plan

11

Considerations for Quality Plans

Include Risk-mitigation strategy

• Include metrics to address identified risks

• Define issue escalation triggers

• Include tolerance limits and corrective action required when limits are exceeded

• Include responsibility actions and timelines associated with identified risks

• Describe tracking of corrective and preventative action plans

12

Considerations for Risk-basedQuality PlanIncludes plan for on-site Monitoring• Identify Paper versus Electronic source requirements

• ID tasks to be performed and how tasks will be performed

Includes Remote and Centralized Monitoring • Define how significant protocol issues will be tracked - deviations

and violations

• Defines how different data is to be reviewed:

– Source Data Verification (SDV)

– Remote check of source data quality indicators: against protocol compliance, quality and timeliness of source completion, site delegation, training compliance

13

Considerations for Study Plans:

Includes all aspects of study focusing on:

Criteria for site selection

-Suitability of site (training, experience and assessment regarding site’s ability to conduct the study according to the protocol)

Significant data for review

-Primary efficacy

-Safety endpoint data

Questions to ask:

What data will be reviewed centrally as opposed to on-site

Direction on what is required to conduct the site close-out visits

Triggers for assessment of serious and on-going non-compliance issues

14

Considerations for ProgramLevel PlansAddresses overall quality and includes risks that are known or anticipated :

• ID of risks and Specifications of Measurement (Before the study –Planning and Design)

• Oversight and Execution of Monitoring

• Define DM and Statistical Analysis

• Data contained in the CSR

15

Considerations for Site LevelPlansInclude Information regarding the experience of investigator site including:

• Therapeutic experience

• Experience in similar programs or protocols

• Previous regulatory history and personnel experience in working with site

• Staff turnover

• PI availability and involvement

• Potential conflicts of interest or bias

16

Writing the Risk Mitigation Plan– Example 1

17

Risk Identified Assessment of Risk (Probability (P) and Consequences (C ) of Occurring and Overall risk evaluation/result (R)

Consequence if risk not managed

Risk Management Strategies

P C R

Loss of data at site

Low Medium

High Risk

-Possible permanent loss of data if captured electronically-Temporary loss of data causing study/data delays-Delay in payment to site and CRO/Sponsor deliverables may be affected

-Ensure adequate pre-study and qualification visits cover review of computer systems to ensure that systems are validated and have appropriate backups and audit trails-Site procedure cover Disaster Recovery Plans

Writing the Risk Mitigation Plan – Example #2

18

Risk Identified Assessment of Risk (Probability (P) and Consequences (C ) of Occurring and Overall risk evaluation/ result (R)

Consequence if risk not managed

Risk Management Strategies

P C R

Fraudulent data at site

Low Medium

High Risk

-Reporting to IRB/ Ethics/Regulators -Study may need to be repeated-Regulatory action for site, CRO and Sponsor possible including penalties and rejection of submission

-Ensure PI/staff are adequately trained on protocol and supported by relevant training and experience-Ensure site has GCP training and other controls in place to prevent and detect fraud

In summary, Risk-BasedMonitoring:• May not fit all trials

• Assessed before study start → lifecycle

• Represents one quality control tool

• Statistical approach recommended

• Risk-Based plans tailored to trial

• Examples provided on setting up risk-based plans – quality, site-level, program-level and risk mitigation plan

19

References:• FDA, Guidance for Industry: Oversight of Clinical Investigations- A Risk-

Based Approach to Monitoring; August 2013

• TransCelerate Position Paper: Risk-Based Monitoring Methodology 2013

• EMA. Reflection paper on risk-based quality management in clinical trials (EMA/INS/GCP/397194/2011)

• MRC/DH/MHRA Joint Project: Risk-adapted Approaches to the Management of Clinical Trials 10th October 2011

• FDA: Q9 Quality Risk Management ICH – Guidance for Industry: Q9 Quality Risk Management

• FDA Guidance for Industry; Computerized systems used in clinical trials, May 2007

20