realizing the operational benefits of utilizing a …...adequacy of cra resourcing. 3%: 11%. 86%:...
TRANSCRIPT
REALIZING THE OPERATIONAL BENEFITSOF UTILIZING A THIRD-PARTY QUALITY
OVERSIGHT PROGRAMCASE STUDY
JOHN R. WILSON, JR., PH.D.SENIOR VICE PRESIDENT
OCTOBER 16, 2014EXL PHARMA CLINICAL QUALITY OVERSIGHT FORUM
WHY DID THE CASE EXIST?
• Global pharmaceutical firm needed the following• Assurance that their trial was being conducted
appropriately• Actionable data and information with an eye toward
early process improvement• Documentation of appropriate oversight of the CRO
• Why did they use a third party?• Sponsor didn’t have appropriate global resources• Sponsor didn’t have a Quality Oversight Plan• Sponsor knew what they didn’t know
2
DESCRIPTION OF CASE
• Global pharmaceutical company• Top 5 CRO• Phase III trial• 400 sites, 2500 subjects• 35 countries• Project start (planning and training): March 2013• Site visits and assessments: April 2013-January 2014• Inspection readiness: February 2014-July 2014
3
A WORD ABOUT FLOW…
• Let’s start with what was of optimal importance to the sponsor
• Then let’s discuss the process used to get there
4
ACTIONABLE DATA AND INFORMATION(THINK HOLISTICALLY)
Data and Information
High Level Overview of
Issues
Issues by Country Issues by Site Issues by CRA
5
HIGH LEVEL OVERVIEW OF ISSUES
Key Risks Identified by Co-Monitors: Cumulative Dashboard Red Yellow Green
Adequacy of CRA resourcing 3% 11% 86%
CRA knowledge and understanding of study product, therapeutic area, project goals, objectives, timelines 4% 9% 87%
CRA organization and efficiency - 7% 93%CRA adherence to the Clinical Management Plan 7% 25% 68%
CRA communication style appropriate and has established positive working relationships with site personnel 4% - 96%
CRA ability to identify problems and recommend solutions - 10% 90%CRA management of investigational product issues 5% 6% 89%CRA oversight and management of maintenance of site personnel blinding status 5% 2% 93%Adequacy of PI and site staff qualifications and training 9% 23% 68%Adequacy of site facilities 6% 18% 76%Site compliance with protocol 15% 39% 46%Site compliance with GCP 24% 33% 43%Site compliance with sponsor/IRB/EC safety reporting procedures 5% 13% 82%Site compliance with data entry and query resolution timelines 7% 16% 77%
Site compliance with investigational product management procedures 9% 18% 73%
Adequacy of PI oversight 4% 18% 78%
6
SCORES BY COUNTRY
The percentages in the pie charts below reflect yellow and red ratings as a percentage of total ratings by country, broken out by region. For example, 59% of Georgia’s ratings were yellow and red.
Australia, 25%
India, 54%
South Korea, 26%
Thailand, 25%
by Asia Pacific Country
Georgia, 59%
Romania, 30%
Spain, 29%
Germany, 29%
Estonia, 28%
Russia, 27%
Latvia, 27%
Poland, 27%
Hungary, 22%
Ukraine, 17%
Bulgaria, 15%
Moldova, 15%
Israel, 13%Croatia, 7%
Serbia, 6%
Lithuania, 0%Slovenia, 0%
by European Country
In this example, the majority of the issues occurred in Georgia, India, and Romania.
7
SCORES BY SITE
15
8
30
1315
8
41
0
5
10
15
20
25
30
35
100% 90% 80% 70% 60% 50% 40% 30%
Percentage Green Key Risk Factor Scores
Tota
l Num
ber o
f Site
s
Percentage of Green Key Risk Factor Scores by Site
As shown in the example above, 66 sites of the 94 sites that were assessed had at least 70% green scores on the key risk factors, with 15 of those sites at 100% green.
8
SCORES BY CRA
CRA 1 and CRA 2 had the greatest percentage of red and yellow scores. In this example, the issues that contributed to CRA 1 and CRA 2’s ratings were discrepancies in ISF, inadequate documentation, limited site staff experience, and numerous protocol deviations.
9
EARLY PROCESS IMPROVEMENT
• Assessors noted several issues that were evaluated for early process improvement (EPI) opportunities to mitigate risk and save time and/or cost for both current and future programs
• Specific examples included:• 100% source data verification in a RBM trial• Unacceptable freezer storage• Lack of adequate PI involvement
• High value to sponsor
10
RISK BASED MONITORING
CRAs conducting 100% Source Data Verification (SDV)
Discrepancy between CRO and sponsor monitoring manuals regarding SDV
Immediate CAPA
Develop training guides for future studies that provide step by step instructions on SDV
Implement into future studies to save time and cost
Issue Noted EPI Opportunity
11
INAPPROPRIATE EQUIPMENT
Unacceptable storage freezer was being used to store bacterial isolates
Temperature monitoring was not being conducted for the freezers
Immediately contacted other sites in the region to ensure the correct type of storage freezer was used to store bacterial isolates
Prevented loss of isolates due to inadequate storage
Issue Noted EPI Opportunity
12
PI INVOLVEMENT
A common trend of inadequate Pi involvement was detected early in the program.
Lack of PI involvement often led to other issues such as inexperienced study staff
For both current and future programs, retraining of the PI and study staff, or audits focused on PI involvement
Mitigated risk for the program
Issue Noted EPI Opportunity
13
HOW WERE THE DATA AND INFORMATIONDERIVED AND REPORTED?
Developed a comprehensive Program Plan• Risk Assessment• Assessment Plan
Organized a team of highly trained quality oversight assessors• Recruited from SCRA ranks, plus mentoring/management experience• Extensive project-specific training
Conducted Quality Oversight Visits• Data Gathering tools• Quality Oversight report templates
Provided data analytics and real-time reporting• Dashboards and reporting
14
COMPREHENSIVE PROGRAM PLAN
Analogous to a CMP for Quality Oversight
Cornerstone of project logistics
Flexible in terms of focus
Collaborative effort with sponsor
15
HIGHLY TRAINED ASSESSORS
• Global network assessors were approved by the sponsor, each with the following minimum education and professional qualifications:• Minimum eight years of monitoring experience, with
experience as a trainer, mentor, leader of CRAs, or GCP compliance responsibilities
• Qualified rating from a subject matter expert assessment, including GCP, monitoring practice, and quality oversight knowledge
• Based primarily in same country as sites visited• Same region as contingency
16
ASSESSMENT VISIT REPORT
• Assessment reports provided narrative descriptions regarding KPI/KQI observations – What is the meaning and
significance?– Isolated human error?
• Assessment reports also included CAPA recommendations when appropriate
QUALITY ANALYSIS AND REPORTING: SCORECARD ASSESSMENT TOOL
• Scorecard focused on key quality indicators, e.g.,– Timely reporting of SAEs– IP accountability– Protocol adherence
• Every question was scored Red, Yellow, or Green either on-site or centrally
QUALITY ANALYSIS AND REPORTING: QUALITY OVERSIGHT DATABASE
• Scorecards were entered into proprietary database
• Customized according to sponsor output requirements
• Tools and scoring provided powerful focus on key study performance parameters
HIGH LEVEL OVERVIEW OF ISSUES
Key Risks Identified by Co-Monitors: Cumulative Dashboard Red Yellow Green
Adequacy of CRA resourcing 3% 11% 86%
CRA knowledge and understanding of study product, therapeutic area, project goals, objectives, timelines 4% 9% 87%
CRA organization and efficiency - 7% 93%CRA adherence to the Clinical Management Plan 7% 25% 68%
CRA communication style appropriate and has established positive working relationships with site personnel 4% - 96%
CRA ability to identify problems and recommend solutions - 10% 90%CRA management of investigational product issues 5% 6% 89%CRA oversight and management of maintenance of site personnel blinding status 5% 2% 93%Adequacy of PI and site staff qualifications and training 9% 23% 68%Adequacy of site facilities 6% 18% 76%Site compliance with protocol 15% 39% 46%Site compliance with GCP 24% 33% 43%Site compliance with sponsor/IRB/EC safety reporting procedures 5% 13% 82%Site compliance with data entry and query resolution timelines 7% 16% 77%
Site compliance with investigational product management procedures 9% 18% 73%
Adequacy of PI oversight 4% 18% 78%
20
CONCLUSIONS
• Global pharmaceutical firm needed an objective third party for quality oversight• Large Phase III trial in 35 countries
• Primary deliverable was actionable data and information• Work performed via
• Comprehensive Program Plan• Highly trained assessors• Quality Oversight Visits• Data analytics and real-time reporting
• Significant value delivered in terms of • Early process improvement• Meeting regulatory agency expectations for oversight of vendors
