2nd exl quality oversight conf szpindor in process vendor audit
DESCRIPTION
Conducting audits of GCP service providers during the conduct of clinical studies once there are outputs, data, and deliverables to review.TRANSCRIPT
Our purpose
We enable people with life-altering conditions to lead better lives1
2nd exl Quality Oversight ofClinical Vendors Conference
A Case Study – In Process Audit Strategy
Stan Szpindor, M.S. Director, R&D Quality Assurance
To be as brave as the people we help
Agenda
• Traditional Qualification Audit• Assumptions and Limitations of a Qualification Audit• Paradigm Shift and Related Framework• In-Process Audit Strategy• Audit Fusion – Vendor Audit to Process Audit – A Case
Study• Supporting Quality Beyond the Audit
To be as brave as the people we help
2
To be as brave as the people we help
Disclaimer
As a component of this presentation is a “Case Study”, the names, facts, accounts, and outcomes have been altered to protect the confidentiality of the firms and personnel involved.
To be as brave as the people we help
3
To be as brave as the people we help
Traditional Vendor Qualification Audit
• Audit Plan Drafted from SOW
• Two-day Onsite Audit
• Scope Includes:
• Quality Management System
• Inspection History
• Review of Select SOPs
• Interview of Personnel
• Training Program
• Personnel Qualifications
• Infrastructure (physical, IT)
Debrief with Sponsor Team
QA Qualification Audit
Note: data covers timeframe of 1/1/2003 through 12/31/2010
Transactional Vendor Selected
CAPAs
Comprehensive Audit Report
“Green Light” to Proceed (or Not)
To be as brave as the people we help
Assumptions and Limitations of a Qualification Audit
• Transactional Vendor
• Little to No Previous Experience with Vendor• No Pre-conceived Intentions of Future Work• Cost and Timing Selection Drivers
• Narrow Audit Focus• Based on Service Area Needs of One Study or Program• Infrastructure Related Items Barely Touched Upon at Best• Training Curriculum and Requirements (not effectiveness)• Reading of SOPs and Discussion of Processes
To be as brave as the people we help
Assumptions and Limitations of a Qualification Audit
• Inability to Gauge Compliance and Efficiency
• Review of SOPs Can’t Predict Real Life Execution• Discussion of Quality Management System with No Real Ability
to Know it is Effective • No Data or Deliverables to Sample or Assess• “Dog & Pony Show” Please!
At the End of the Day, Can I truly Know with Confidence the Vendor WILL Deliver as Promised?
To be as brave as the people we helpWe continue to move forward at a faster pace. It’s a timing thing!
To be as brave as the people we help
Paradigm Shift and Related Framework
To be as brave as the people we help
Sourcing Strategy Alignment• Seek an integrated view of the sourcing
strategy across the study components and R&D functions
• Ensure framework for objective vendor selection and periodic reviews
• Consolidate base of providers from many Transactional to few Strategic
• Identify economies of scale and functional service provider (FSP) opportunities
Vendor Management
• Select metrics for each vendor from comprehensive menu based on applicability of services being performed
• Monitor metrics / KPIs / KQIs for vendors (in conjunction with functions and teams)
• Report executive level dashboard • Conduct In-process audits which gauge
deliverables, outputs, and commitments.
Vendor Evaluation• Shift to Functional Assessments from Pre-
Qualification Audits• Functions and Teams are accountable to
determine “fit for use” based on evaluation of vendor SOPs, personnel, and capacity against project specifications
• Provide standard approach and adaptable outcome capture tools for consistency across R&D functions and teams
Vendor Governance• Establish Relationship Management
Committee with cross-functional representation including QA
• Utilize Quality Agreements to ensure alignment between outsourcing strategy and quality expectations
• Regularly provide / discuss audit and inspection trends from studies
• Hold After-Action-Reviews to foster continuous improvement
To be as brave as the people we help
Anatomy of The Functional Assessment
• “Don’t Call Me Francis!”
• The Assessment Team• Consider availability and Subject Matter Expertise (SME) of your
Team/Functions• Consider availability of members of both your Team AND
members of other Teams that are using the same Vendor • Representatives should be aligned with requested outsourced
services and should be functional area expert or SME • Establish pool of “Core Away Team” personnel that are Subject
Matter Experts in Key Areas AND Trained / Experienced in conducting Functional Assessments
To be as brave as the people we help
The Functional Assessment
• Onsite Activities
• Confirm Firm’s Capabilities (project and beyond)
• Organization – how structured, recent changes
• Competencies – partners/sub-contracting, delivery sphere
• Personnel – qualifications, turnover, transition provisions
• Procedures - Do the Vendor’s SOPs meet or exceed Sponsor standards for your project?
• Management – communication channels, senior management involvement, issue escalation, workload management
• Functional Level Service Delivery – compatibility, alignment, systems/tools, QC robustness
• Reporting
• Customizable report containing “must items” as well as those relevant to Vendor and service type
• Notes a list of follow-up items
To be as brave as the people we help
10
To be as brave as the people we help
Benjamin Franklin
“Diligence is themother of good luck”
To be as brave as the people we help
In-Process Audit Strategy
Master Audit PlanMaster Audit Plan
Frequency Assignment: Annual Audit = Strategic Partners, High Risk Partners 2-Yr. Audit = Standard Level 3-Yr. Audit = Low Volume Partners, Low Risk Partners
Frequency Assignment: Annual Audit = Strategic Partners, High Risk Partners 2-Yr. Audit = Standard Level 3-Yr. Audit = Low Volume Partners, Low Risk Partners
GCP Risk Factors: Inspection History Audit History Global Exposure Key vs. Non-key Study Key Quality & Compliance Indicator Aggregates Quality Agreement Adherence
GCP Risk Factors: Inspection History Audit History Global Exposure Key vs. Non-key Study Key Quality & Compliance Indicator Aggregates Quality Agreement Adherence
To be as brave as the people we help
In-Process Audit Strategy
Master Audit PlanMaster Audit Plan
Business Risk Factors: Relationship Type – Strategic vs. Transactional Service Type – Full Service, Niche Service, FSP Volume of Work and Overall Spend Global Exposure Merger & Acquisition Activity Key Performance Indicator Aggregates Contract Adherence
Business Risk Factors: Relationship Type – Strategic vs. Transactional Service Type – Full Service, Niche Service, FSP Volume of Work and Overall Spend Global Exposure Merger & Acquisition Activity Key Performance Indicator Aggregates Contract Adherence
To be as brave as the people we help
In-Process Audit Strategy
Benchmarks, Benchmarks, Benchmarks!• Service Agreement / TORO / Statement of Work Compliance• Quality Agreement Compliance• SOP / Work Instruction Compliance • Functional Plan Compliance
Commitment Fulfillment• Audits
• Responses & CAPAs Process & Structural Improvements noted in Sponsor Audits and Select CRO Audits
• Vendor Scorecard• Inspections
• Responses & CAPAs to Agencies• Governance
• Outcomes of Relationship Committees, Study-level Issues Trackers
“Trust, but Verify”
To be as brave as the people we help
In-Process Audit Strategy
Outputs and Deliverables Follow the Process• Approval of Study Team Personnel
• Evidence of Qualification Review, Selection, and Approval• Evidence of ongoing GCP, SOP, Protocol, and Therapeutic
Training
• Monitoring Reports• Logistics: timing, format, review, approval, distribution, storage• Content: visit frequency, SDV approach, PI interaction, issue
identification/escalation/resolution
To be as brave as the people we help
In-Process Audit Strategy
Outputs and Deliverables Follow the Process• Clinical Database Lock Package
• Set up, Edit Specification Development, EDC Screen Design, Testing/Validation, Data Transfers, Data Cleaning, Safety Reconciliation, QC, Freeze/Lock
• Programming and Statistics• Blind Break, SAS Datasets, Program Development, Macro
Library and Controls, Mock Displays, Programming Outputs, Data Reviews, TLF Outputs, QC
To be as brave as the people we help
In-Process Audit Strategy
Risk Focus• Indicator Patterns and Anomalies
• Determination of systemic items, Outliers, Root-Cause
• SOP/Work Instruction Cohesiveness• M&A Impact at either Sponsor or Vendor• Clear Definition of which Processes in Effect at Any Point in
Time• SOP / QMS Improvement Initiatives
• Infrastructure (Baseline and Incremental Views)• Network Topography, Validation Program, Back-up/Recovery• Facility Security, Business Continuity
To be as brave as the people we help
Audit Fusion – Vendor Audit to Process Audit – Case Study
Background• Biometrics
• Vendor 1: FSP Audit Annually• Vendor 2: Biometrics Legacy Vendor Audited Biennially• Process Audit: Biometrics Function on Biennial Cycle
• A “Fused Audit” • Identification of a True End to End View Combining the Vendor
and Process Audits• Comprehensive Audit Strategy Plan with Modules for the
Process Audit and Two Vendor Audits• Focus on Expectation Setting, Oversight, Hand-offs, Process,
Actual Deliverables, Internal Review/Acceptance, Issue Resolution, and Feedback – Sponsor Oversight
To be as brave as the people we help
Process Audit Vendor Audit 1 (FSP) Vendor Audit 2
Management: Oversight (Functional & Communication Plans), Expectation Setting (Quality Agreement), Personnel
Management: Project & Relationship Management, Expectation Setting (Quality Agreement Adherence), Personnel
Management: Project Management (Functional Plan & SOW Adherence), Expectation Setting (Quality Agreement), Personnel
SAS Application: Access Control, Deployment Types, Validation Package, Add-on Tools, SAS Macro Library
SAS Application: Access Control, Deployment Types, Validation Package, Add-on Tools, SAS Macro Library (Sponsor & Study Specific)
SAS Application: Access Control, Deployment Types, Validation Package, Add-on Tools, SAS Macro Library (Study Specific), Plans for Upgrades, Migration, and Project Impact
Programming: Procedures, Data set Specifications, Data Conversions, Data Transfers, Blinded Data Review
Programming: Procedures, Data set Specifications, Data Conversions, Derived Variable Checks, Macro Development/ Testing/ Revision, Output QC & Validation, Data Transfers, Blinded Data-set Production
Programming: Procedures, Data set Specifications, Data Conversions, Derived Variable Checks, Macro Development/ Testing/ Revision, Output QC & Validation, Data Transfers, Blinded Data-set Production
Statistical Analysis:Procedures, Random. Code Mgt., Analysis Population Define, Data Pooling, TFL Creation & QC, Interim & Ad-hoc Analysis, TFL Version Control
Statistical Analysis:Procedures, Random. Code Mgt., Data Pooling, TFL Creation & QC, Interim & Ad-hoc Analysis, TFL Version Control
Statistical Analysis:Procedures, Data Pooling, TFL Creation & QC, Interim & Ad-hoc Analysis, TFL Version Control
To be as brave as the people we help
“Fused Audit” Approach
• Audit Planning More Strategic
• Fewer Audits Cost Saving
• Audit Intensity and Duration may increase
• Promotes Consistency Across like-audits
• Rotating Scope for Vendor.:
• Services A & B annually
• Service C every 3 years
• Infrastructure (physical, IT) biennially
• Ability to clearly see Total Picture
Debrief and Audit Report
Audit Yields/Benefits
Coordinated Multiple In-Process Vendor Auditsand Internal Process Audit
Verification and Effectiveness of Previous Commitments
CAPAs
Enhancement of Process, Expectations, and Relationship
To be as brave as the people we help
FrameworkIntegrated view of sourcing picture
to identify economies of scale opportunities
Expectations Collaborate to
set jointExpectations
through QualityAgreement, TORO,
and Plans
MetricsStandard Menu of
Meaningful Metrics that can be Aggregated
Relationship Management
Audits
In-process,Full View,CAPA &Issue
Resolution
Regular, Collaborative,
Forward Looking, Strategic
FunctionalAssessment
Operation teams accountable
to determine“fit for use”
Sourcing Strategy Alignment
Vendor Evaluation
VendorManagement
Quality
VendorGovernance
Commitment Fulfilment
Supporting Quality Beyond the Audit