Industry Efforts for Ensuring Effective
Oversight in Outsourced Clinical
Trials
Partnerships in Clinical Trials Conference Vienna, Austria
2
Mission: To accelerate the development of best practice approaches and industry standards for the proactive quality
management of outsourced trials
Lead Sponsors and CROs in optimizing their approaches to proactive quality management with an emphasis on bringing
them into greater alignment.
Avoca Quality Consortium
3
Avoca Quality Consortium Industry Participation to Date
4
Avoca Quality Consortium
5
Industry Efforts focused on Oversight
2013 Quality Consortium Leading Practices Focus: Development of Guidelines and Leading Practices for
Effective Oversight
Impetus: Common Challenges among Members • Higher level of scrutiny from regulatory authorities • Many companies do not have comprehensive, documented process
for oversight • Inefficiency in use of resources with some companies
6
Avoca Quality Consortium Research on Oversight
7%
59%
30%
4% Executive Management
Middle Management
Project Management
Other
20%
50%
28%
2% Executive Management
Middle Management
Project Management
Operational staff
Respondent characteristics
• Total of 476 respondents from 29 companies participated in the 2013 Avoca Quality Consortium Research
• 68% response rate
Consortium Sponsors:
Consortium CROs:
N=276 N=200
7
Levels of Satisfaction
Sponsor Satisfaction with Clinical Service Providers Work, Value Received, and Quality Delivered
2%
3%
2%
57%
37%
45%
22%
32%
30%
17%
23%
20%
2%
5%
3%
0% 20% 40% 60% 80% 100%
Satisfaction with work done
Satisfaction with value received
Satisfaction with quality delivered
5 - Very satisfied 4 3 - Neutral 2 1 - Very dissatisfied
N
246
239
245
8
Satisfaction with Quality Delivery
Sponsor: Overall, how satisfied are you with the quality delivered by your
Clinical Service Providers (including but not limited to CROs)?
2%
45%
30%
20%
3% 17%
70%
10% 3%
CRO: Overall, how satisfied are you with the quality that your company
delivers for its sponsors?
N=245* N=200*
5%
13%
8%
8%
15%
15%
40%
10%
3%
11%
27%
95%
100%
67%
77%
83%
64%
58%
20%
60%
71%
57%
66%
60%
20%
15%
15%
20%
20%
20%
14%
23%
16%
10%
8%
6%
7%
20%
10%
14%
17%
7%
2%
0% 20% 40% 60% 80% 100%
A
B
C
D
E
F
G
H
I
J
K
-All-
Very satisfied Somewhat satisfied Neither satisfied nor dissatisfied
Somewhat dissatisfied Very dissatisfied
*2013 Avoca Quality Consortium Assessment
9
N=249*
16%
76%
8% A higher standard than that to which in-house teams are held.
The same standard to which in-house teams are held.
A lower standard than that to which in-house teams are held.
Sponsors: With respect to expectations
surrounding quality, my company holds its CROs to…
CROs: In general, do you feel that the sponsors with
which you work hold your company to a higher standard of quality, the same
standard, or a lower standard than that to which they hold their own internal teams?
61%
36%
3% Sponsors hold us to higher quality standards
Sponsors hold us to the same quality standards
Sponsors hold us to lower quality standards
N=200*
Standards for Quality Delivered by CROs: Do sponsors hold CROs to a higher standard?
*2012 Avoca Quality Consortium Assessment
10
Root causes of quality issues in outsourced trials
● Sponsor's inability to provide specifics about requirements and lack of planning
● Continued push to meet timelines at all costs
● Sponsors’ decision-Making with impacts on quality (without CRO input)
Cited by CROs:
● Need for better capture and utilization of Lessons Learned
● Overall poor communication, particularly when it comes to escalation of issues
● Lack of clarity around expectations and definitions of quality
● Lack of proactivity
Other specific complaints included:
● The most common complaints had to do with resourcing.
╸ Under-resourcing at CROs
╸ Inexperience and lack of qualifications of CRO staff
╸ Inadequate training
╸ Staff turnover
Cited by Sponsors:
11
CRO Oversight: Qualitative Data from the CROs’ Perspective
● Open, honest, and proactive communication ╸ Clear, upfront communication of expectations
● Full transparency – on both sides ╸ Sponsor is fully aware of issues at all times
╸ Sponsor shares all knowledge of past trials
● Limit micromanagement: Balance between oversight and involvement, delegation and control
● Ensure there is a solid plan in place at start of project ╸ Roles and responsibilities defined
╸ Agreed definitions of standards
╸ Agreement on expectations
What best practices should sponsors adopt?
12
Challenges Overseeing CROs: Sponsor
● Avoiding excessive oversight and micromanagement while ensuring quality and meeting timelines
● Lack of standardization of oversight processes and procedures
╸ Variability with multiple CROs creates challenges due to CRO variations in SOPs and internal processes
● Budget management: balance of cost and quality
Within your company, what are the most significant challenges and issues that you face in overseeing CROs?
13
Consortium Research on Effective Oversight
● Efficiency in use of resources
● Definition of roles to minimize duplication of effort
● Micromanagement
● Documentation around roles and responsibilities
● Overseeing work at a strategic level rather than tactical
● Use of technology to promote efficient oversight practices
Research Pertaining to Oversight - Topics
14
Sponsor: My company is efficient in the use of resources
applied to the oversight of outsourced trials.
CRO: The sponsors with which I work are efficient in the use of resources
applied to the oversight of outsourced trials.
N=207 N=128
5%
28%
30%
32%
6% 5%
14%
41%
34%
6%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Efficiency in Use of Resources
15
Representative Verbatim Comment
“[Sponsors] repeat processes outsourced to us as the CRO...
then drive more resources to discuss and address mismatches at every step instead of just evaluating the quality of the final product.”
16
Sponsor: My company clearly defines the roles of internal
and CRO staff so as to minimize duplication of effort.
CRO: The sponsors with which I work clearly define the roles of internal
and CRO staff so as to minimize duplication of effort.
N=209 N=127
6%
34%
34%
18%
8% 4%
25%
34%
29%
8%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Defining Roles and Responsibilities
17
Sponsors The operational teams involved in overseeing our CROs consistently
micromanage to ensure that the quality of deliverables meets expectations.
CROs The sponsors with which I work
consistently micromanage to ensure that the quality of deliverables meets
expectations. 7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Prevalence of Micromanagement
21%
30% 34%
11% 4% 8%
46% 32%
11% 3%
N=219 N=157
18
Sponsors My company clearly documents CRO
oversight practices, roles, and responsibilities.
CROs The sponsors with which I work clearly
document CRO oversight practices, roles, and responsibilities.
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Documenting Roles
13%
42% 21%
20%
4% 5%
33%
34%
23%
5%
N=219 N=153
19
Sponsors I personally have received high-quality
deliverables from CROs, even when overseeing their work at a "strategic"
rather than detailed level.
CROs There is a direct positive relationship
between the intensity of sponsor oversight and the quality of CRO
deliverables. 7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Overseeing Work at a Strategic Level
5%
35%
34%
18%
8% 4%
22%
29%
29%
16%
N=203 N=158
20
Sponsor: The technology systems used by my company and its CRO partners promote efficient oversight practices.
CRO: The technology systems used by my company and its sponsor partners promote efficient oversight practices.
N=204 N=136
2%
19%
31% 32%
16% 2%
31%
33%
24%
10%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its sponsor partners promote efficient oversight
practices. The sponsors with which I work clearly define the roles of internal and CRO staff so as to minimize
duplication of effort. The sponsors with which I work are efficient in the
use of resources applied to the oversight of outsourced trials.
There is a direct positive relationship between the intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree
Technology and the Impact on Efficiency
Quality Consortium Leading Practice
Initiative Focused on Oversight
22
Consortium Process for Reviewing Member Documentation
Tailor Refine
Combine Supplement
Member Reviews
Edit
Adapt
Best Practices
Concepts
Descriptions Details
What/How Tools
Select
Read Interpret
Assess Extract
Discuss
Review Documents
Presentations
Interviews Meetings
Collect
23
And Researched Other External Sources
Internet Searches
Public and Consortia Information
Best Practices from Other Industries
Regulatory Guidance Documents
Other Institutional Materials
24
Governance/Organizational Construct
Technical/Project Oversight
Process Oversight
Communication
Metrics/Analytics/Technology
Roles/Responsibilities
Oversight Leadership Requirements
Proactive Risk/Opportunity Management
Consortium Guidelines and Tools for Operationalizing Proactive Approaches to Oversight
25
Governance / Organizational
Construct
Technical / Project Oversight
Process Oversight
Communication
Oversight Leadership
Requirements
Metrics / Analytics /
Technology
Roles / Responsibilities
Proactive Risk / Opportunity
Management
Define
Plan
Lead
Decide
Deliver
Measure
Proactive Quality
Management
Consortium Guidelines and Tools for Operationalizing Proactive Approaches to Oversight
Thank you!