Tammy FinniganNov 06, 2012

Objectives

This presentation will aim to:

• Provide a brief overview of the guidance that is driving the change in how the industry approaches monitoring

• Define risk based monitoring• Examine the process and data required to generate risk profiles for

investigational sites• Demonstrate how CTMS and EDC analytics can be used to identify

and manage changing risks during study conduct• Consider the changing role of the monitor

2

ICHICHICHICH----GCPGCPGCPGCP

“protect the safety and well being of human subjects and the quality and integrity of data”

Guidance from regulators:Guidance from regulators:Guidance from regulators:Guidance from regulators:

“most effective way “ to monitor a clinical trial was to “maintain personal contact between the monitor and the investigator throughout the clinical investigation”

Industry interpretationIndustry interpretationIndustry interpretationIndustry interpretation:

Site qualification

Regular monitoring visits (4-8wk intervals)

100% data verification

The role of the monitorThe role of the monitorThe role of the monitorThe role of the monitor

CFR and EU CTDirCFR and EU CTDirCFR and EU CTDirCFR and EU CTDir

“Sponsors of clinical investigations are required to provide oversight to ensure adequate protection of the rights, welfare and safety of human subjects and the quality and integrity of the data”

LegislationLegislationLegislationLegislation

Background

3

Risk-based approaches to monitoring, focusing on critical data elements, more likely to:

• ensure subject protectionensure subject protectionensure subject protectionensure subject protection

• overall study qualityoverall study qualityoverall study qualityoverall study quality

Will permit sponsors to monitor the conduct of monitor the conduct of monitor the conduct of monitor the conduct of clinical trials more efficiently and effectivelyclinical trials more efficiently and effectivelyclinical trials more efficiently and effectivelyclinical trials more efficiently and effectivelythan the current approach of routine visits to all clinical sites and 100% data verification

Interesting point:Interesting point:Interesting point:Interesting point:

Recent survey by an EDC vendor on >2000 protocols demonstrated that < 3% of the clinical study data was changed due to onsite monitoring

Growing consensusGrowing consensusGrowing consensusGrowing consensus

ReReReRe----visited ICHvisited ICHvisited ICHvisited ICH----E6E6E6E6

Flexibility in how trials are monitored

Advice to sponsors:Advice to sponsors:Advice to sponsors:Advice to sponsors:

Consider “the objective, purpose, design, complexity, blinding, size and endpoints of a trial” in determining the extent and nature of monitoring

Guidance specifically provides for the possibility of reduced, or even no* onsite monitoring and address data quality and site performance through centralized methods

*Guidance is clear only appropriate in exceptional circumstances

Recent Industry GuidanceRecent Industry GuidanceRecent Industry GuidanceRecent Industry Guidance

Change in industry guidance

4

Risk Based Monitoring

• Identify likely sources of error

• Impact of error

• Likelihood of error

• Ability to detect error

Protocol Protocol Protocol Protocol Risk Risk Risk Risk

AssessmentAssessmentAssessmentAssessment

• Critical data

• Processes identified in the risk assessment

• On site and centralized activities

Monitoring Monitoring Monitoring Monitoring PlanPlanPlanPlan

• Investigator experience

• Historical performance

• Standard of care in country

Study Site Study Site Study Site Study Site Risk Risk Risk Risk

AssessmentAssessmentAssessmentAssessment

• On-going data analysis

• Identify non-compliance

• Identify outliers

Proactive Proactive Proactive Proactive Risk Mgmt.Risk Mgmt.Risk Mgmt.Risk Mgmt.

ConsiderationsConsiderationsConsiderationsConsiderations• Type of data being

collected• Specific activities

required for data collection

• Data critical to reliability of study findings

• Safety concerns• Subject protection

concerns

ConsiderationsConsiderationsConsiderationsConsiderations• Complexity of study

design• Types of end points• Stage of study (taper

monitoring)• Standard of medical

care• Type of monitoring

activities (frequency, intensity, targeted, random, onsite/central)

ConsiderationsConsiderationsConsiderationsConsiderations• Outcome of protocol risk

assessment and monitoring plan

• Site quality assessment• Increased areas of risk

for the site • Additional area of risk

for the site • Site specific monitoring

plan

ConsiderationsConsiderationsConsiderationsConsiderations• Aggregate data analysis

and review outlying sites• Analyse site

characteristics and correlate with poor performance and non-compliance trends

• Adjust monitoring activities based on the analysis

5

Site risk profile

• Not within the context of a specific study

• Based on historical performance data across studies

• Set risk threshold for key performance indicators

• On-going quality management and development of investigator and site relationship

Study site risk profile

• Based on protocol risk assessment and protocol monitoring plan

• Cross reference protocol risk assessment against Site risk profile

• Include outcome of site qualification visit as applicable

• Identify areas of additional risk for the site

Study site monitoring plan

• Modify the protocol monitoring plan to accommodate additional site specific risks

• Study risks should always be addressed in the study site monitoring plan

• Additional focus or intensity of monitoring activities may be planned for individual sites to mitigate specific risks

Site Risk Profiles

6

Key Performance Indicators

The site risk profile should provide a continuous picture of the site The site risk profile should provide a continuous picture of the site The site risk profile should provide a continuous picture of the site The site risk profile should provide a continuous picture of the site

F

Subject SafetySubject SafetySubject SafetySubject Safety

---- Protocol violationsProtocol violationsProtocol violationsProtocol violations---- Screen failure ratesScreen failure ratesScreen failure ratesScreen failure rates---- Drop out ratesDrop out ratesDrop out ratesDrop out rates---- Compliance with safety Compliance with safety Compliance with safety Compliance with safety and ethics reporting and ethics reporting and ethics reporting and ethics reporting timelinestimelinestimelinestimelines

Data QualityData QualityData QualityData Quality

---- Data query rateData query rateData query rateData query rate---- % of missing data% of missing data% of missing data% of missing data---- Protocol violations and Protocol violations and Protocol violations and Protocol violations and deviationsdeviationsdeviationsdeviations---- Source data verification Source data verification Source data verification Source data verification issuesissuesissuesissues

PerformancePerformancePerformancePerformance

---- Data reporting timeData reporting timeData reporting timeData reporting time---- Query response timeQuery response timeQuery response timeQuery response time---- Critical site issuesCritical site issuesCritical site issuesCritical site issues---- Issue resolution timeIssue resolution timeIssue resolution timeIssue resolution time---- Recruitment ratesRecruitment ratesRecruitment ratesRecruitment rates---- GCP trainingGCP trainingGCP trainingGCP training

7

Proactive Risk Management

Study relatedNon-study

related

8

Improved Improved Improved Improved site and site and site and site and study study study study

performanceperformanceperformanceperformance

Solely data monitoring

• Statistical analysis

• Exception reporting

• Data violations

• Missing data

• Data aggregation

• Root cause analysis

• Trend analysis

Feedback loop into Feedback loop into Feedback loop into Feedback loop into protocol and site risk protocol and site risk protocol and site risk protocol and site risk assessmentsassessmentsassessmentsassessments

Centralized MonitoringCentralized MonitoringCentralized MonitoringCentralized Monitoring

Inexperienced sites or site staff i.e. new to clinical trials, new to therapy area

Sponsor has no/little previous experience with the site

Training

Maintaining and improving investigator relationships

When physical verification is required i.e. drug accountability

When is On Site When is On Site When is On Site When is On Site Monitoring Effective?Monitoring Effective?Monitoring Effective?Monitoring Effective?

Recent survey by an EDC vendor on >2000 protocols demonstrated that < 3% of the clinical study data was changed due to onsite monitoring

With advances in remote data capture systems, safety/PVG systems and internet access across the globe, is onsite monitoring effective?

On Site MonitoringOn Site MonitoringOn Site MonitoringOn Site Monitoring

Centralized vs. On Site Monitoring

9

Statistical MethodsStatistical MethodsStatistical MethodsStatistical Methods

Protocol specific rules

Parametric algorithms

(probability rules)

Fraud and bias detection

Exception reporting

Monitor Data QualityMonitor Data QualityMonitor Data QualityMonitor Data Quality

Missing data

Inconsistent data

Potential protocol deviations

Data outliers

Aggregate Data Aggregate Data Aggregate Data Aggregate Data AnalysisAnalysisAnalysisAnalysis

Aggregate data across study sites

Complete trend analysis and determine ‘expectedness’

Review sites in relation to other sites i.e. protocol distribution

Identify outliers

Analyse Site Analyse Site Analyse Site Analyse Site CharacteristicsCharacteristicsCharacteristicsCharacteristics

Data anomalies

Higher frequency of errors

Higher protocol violations

Delays in data reporting

Study Related Risk Management

During the study, off site, centralized monitoring activities and data analysis can be used to identify risk:

10

Aggregate Data Analysis

Review site data in relation to other study sites:Protocol ViolationsProtocol ViolationsProtocol ViolationsProtocol Violations

PV bubble chart from study site data analysis dashboard to show protocol site distribution of protocol violations, highlight outlier sites (with hover over displaying # by category i.e. eligibility, informed consent, IMP)

- In this example there are sites that display disproportionate number of violations compared with other study sites

- Root cause analysis- Drill down to the violation type, are there specific risk factors for subject

safety and/or data integrity?- Mitigation – additional site training visit

11

Aggregate Data Analysis

Screen Failure RatesScreen Failure RatesScreen Failure RatesScreen Failure Rates

SF bubble chart from study site data analysis dashboard shows protocol site distribution of screen failure rates, highlight outlier sites (hover over displaying # by reason i.e. withdrew consent, abnormal labs)

- Are there sites that display disproportionate number of screen failures compared with other study sites?

- Root cause analysis- Drill down to the screen failure reasons, are there specific risk factors for

subject safety?- What practices are the site following?- What mitigation needs to be put in place?

- Are there high rates of specific screen failure reasons across sites and countires?- Root cause analysis

- Feed back into the protocol risk assessment- Have we detected something unexpected?- Do we need to change our risk assessment and monitoring plans?

12

Aggregate Data Analysis

Review study data to identify trends:

Protocol DropoutsProtocol DropoutsProtocol DropoutsProtocol Dropouts

Early withdrawal scatter diagram from study subject data dashboard show sprotocol distribution of early withdrawal reasons, and rates by country

- Trend analysis- Is this the identification of additional risks to subjects and/or study

procedures- Assess against protocol risk assessment- Do we need to adjust our monitoring and quality plans- Do we need to conduct additional training- Is there a need for country specific amendments

13

Site Characteristics

Review site data to identify sites with characteristics correlated with poor performance and non-compliance:

Study Site Risk ProfileStudy Site Risk ProfileStudy Site Risk ProfileStudy Site Risk ProfileRadar diagram from study site risk dashboard, to show site risk characteristics in relation to study ‘normal’ rate

- The site is compared with the protocol average for critical risk factors to determine if there are:

• A higher number of data errors • A lower number of subject dropouts • Longer lag time entering data

14

Proactive Risk Management

Review aggregate data Review aggregate data Review aggregate data Review aggregate data dashboardsdashboardsdashboardsdashboards

Identify new risks (study and/or Identify new risks (study and/or Identify new risks (study and/or Identify new risks (study and/or study site)study site)study site)study site)

Follow escalation process Follow escalation process Follow escalation process Follow escalation process identified in monitoring planidentified in monitoring planidentified in monitoring planidentified in monitoring plan

Determine appropriate mitigation, Determine appropriate mitigation, Determine appropriate mitigation, Determine appropriate mitigation, update to risk assessmentsupdate to risk assessmentsupdate to risk assessmentsupdate to risk assessments

15

Risk Mitigation

Study levelStudy levelStudy levelStudy levelUnexpected trends identified in the protocol data may result in the need to change monitoring activities across the study e.g.• Data previously deemed to be non-critical may need to be monitored more closely• Trends in subjective end point analysis may result in more onsite monitoring

activities• Trends in protocol violations may point to the need for additional training of site

staff

16

Risk Mitigation

Site levelSite levelSite levelSite levelUnexpected site characteristics identified in the study site data may result in the need to modify the monitoring activities for the individual site e.g.

• More intense data monitoring if there is an inordinate number of data errors in the sub-set that is monitored for all sites

• Targeted onsite monitoring visits• Tapered monitoring visits i.e. additional site visits in the early stages

of a study if high number of eligibility violations or screen failures, address training

The continuous picture of the site should not be forgotten during the study• Site profile should be monitored for change • As this is based on cross study performance, the site may move

above/below the identified thresholds• Study teams should be informed

17

Site Quality ManagerSite Quality ManagerSite Quality ManagerSite Quality Manager

Create site risk profile

Assess site against study risk profile

Create study site monitoring plan

Site ManagerSite ManagerSite ManagerSite Manager

Site relationship management

Site training

Onsite data verification

Data (Quality) ManagerData (Quality) ManagerData (Quality) ManagerData (Quality) Manager

Data monitoring and analysis

Root cause and trend analysis

Ongoing study risk management

Changing Roles

The role of the monitor as currently defined, is going to change. Consider the following model:

18

Benefits of risk based monitoring

Real time identification of risk will allow sponsors and CROs to:• Make informed decisions • Be proactive in managing potential impact to subject safety

and data integrity

On-going management of site risk profiles will ultimately improve site and study performance

19

Questions

20

Presenter’s Biography and Contact details

21

Tammy Finnigan, COO, Triumph Consultancy ServicesTammy Finnigan, COO, Triumph Consultancy ServicesTammy Finnigan, COO, Triumph Consultancy ServicesTammy Finnigan, COO, Triumph Consultancy Services

ttttammy.finnigan@triumphconsultancy.co.ukammy.finnigan@triumphconsultancy.co.ukammy.finnigan@triumphconsultancy.co.ukammy.finnigan@triumphconsultancy.co.uk

Tammy’s entire career has been focused on clinical research, having worked in project management and clinical operations for 10 years, with both large Pharma and CRO businesses prior to joining Triumph. Her experience both in monitoring, and managing clinical trials made her a significant hire for Triumph in 2007. Tammy’s experience, passion and eye for quality saw her promoted to Head of EU Operations within her first year, and in 2011 she was appointed COO to take over global operations responsibility.