1. THE AVOCA GROUP INAUGURAL QUALITY SUMMIT ESTABLISHING A BEST-PRACTICE APPROACH TO QUALITY MANAGEMENT AND CRO OVERSIGHTOperationalizing Best Practices in Proactive Quality Management

2. Never underestimate the power of a small group of committed people tochange the world. Indeed, it is the only thing that ever has.Margaret MeadThe Avoca Quality Summit, held May 1-2, 2012, in New Brunswick, NJ,was the official kick-off of The Avoca Groups Quality Consortium, acooperative effort that brings together quality, outsourcing, and operationalprofessionals from member pharma, biotech, and CRO organizationsto accelerate the development of a best-practices approach to qualitymanagement and CRO oversight. To date, the Avoca Groups QualityConsortium includes 27 members: 16 from pharma/biotech companies and11 contract research organizations (CROs) (see Table 1). The corporatesponsors of the Quality Consortium are Eli Lilly & Co. and Pfizer Inc.PharmaNet/i3 was the corporate sponsor of the Quality Summit. 3. TABLE OF CONTENTSCONSORTIUM MEMBERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1MISSIONS AND GOALS OF THE QUALITY CONSORTIUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1HISTORY OF THE CONSORTIUM: Opening Windows of Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . 2 2012 Avoca Consortium on Quality: Quality Delivered by CROs (Sponsor Data) . . . . . . . . . . . . . 2THE AVOCA QUALITY SUMMIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 SUMMIT DAY 1: First Steps Toward Creation of a Quality Agreement Template andQuality Metrics Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 SUMMIT DAY 2: Exploring Issues Around Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4ROUNDTABLE 1: Documenting Expectations for Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Does quality drive price up? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Defining quality and articulating expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Empowering individuals to reach for quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Is having standards practically achievable? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 The elephant in the room . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 The burning platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Summing it up - Roundtable 1: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12ROUNDTABLE 2: What are the Considerations when Implementing ComprehensiveMetrics Programs? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Measuring quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Quality is hard to define . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Pooling data to achieve a standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Controlling the perverse side of metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Deciding to share information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Summing it up - Roundtable 2: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16ROUNDTABLE 3: Ensuring High-Quality Oversight in Global Trials in Emerging Regions . . . . . . 17 Quality over cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Parallels to Moneyball . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Summing it up - Roundtable 3: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18ROUNDTABLE 4: What Does the Future Hold for Sponsor/CRO Strategic Partnerships? . . . . . . 19 Will developing more consistent and agreed-upon approaches to quality agreements and metricslead to more satisfaction and better performance? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 The implications for CROs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 CROs: Learn to come to the table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Which business models for sourcing are best for quality improvement? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Summing it up - Roundtable 4: Issues and suggested actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24NEXT STEPS: The Future of the Avoca Quality Consortium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4. CONSORTIUM MEMBERS Pharma, Biotech, and Device CompaniesClinical Service Providers Alexion Pharmaceuticals Chiltern International Amgen Covance Astellas Pharma Harrison Clinical Research AstraZeneca INC Research Biogen Idec ICON Bristol-Myers Squibb PAREXEL International Cerexa PharmaNet/i3 Cubist Pharmaceuticals PRA International Eli Lilly & Co. Quintiles GlaxoSmithKline RPS Grnenthal Theorem Clinical Research ImClone Systems Janssen Research & Development Pfizer Purdue Pharma RocheMISSIONS AND GOALS OF THE QUALITY CONSORTIUMConsortium members have a common mission: (1) to optimize their individual companies approaches toquality management within a short time frame, through collaboration with other Consortium membersand The Avoca Group, and (2) to effect change overall in the industry by creating a new paradigm andstandard for managing the quality of outsourced trials and programs.The Quality Consortiums short-term goal is to create standards for Quality Management, with a focuson Quality Agreements and Quality Metrics. The groups long-term goal is to develop a new paradigm inthe approach to Quality Management and partnering with CROs to ensure high quality and mitigate riskwhile introducing efficiencies and potential cost savings. Members of the Consortium will work togetherto establish mutually agreed-upon Quality Management standards in order to strengthen partnerships andreliability throughout the industry.The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.1 5. HISTORY OF THE CONSORTIUM: Opening Windows of OpportunitiesThe Avoca Groups 2011 Report on Quality and CRO Oversight identified a number of thought-provoking issues and gaps between sponsors and CROs in the area of quality. The data from 2011also identified areas of opportunity, particularly in the development of proactive approaches to qualitymanagement. Avoca formed the Quality Consortium to address challenging areas head-on and todevelop more consistent approaches to the proactive management of the quality of outsourced trials.According to Patricia Leuchten, Avocas President and CEO, We know that sponsor companies arestriving to become more efficient in the oversight of CROs and to reduce the duplication of effort whilefocusing on maintaining very high quality. 2012 Avoca Consortium on Quality: Quality Delivered by CROs (Sponsor Data) Over the last 3 years, the quality delivered by our CROs has been*:N = 167 7% Better than the quality delivered by our in-house teams. The same as the quality delivered by our 34% 59% in-house (sponsor) teams. Worse than the quality delivered by our in-house teams.*Data from the 2012 Avoca Consortium on QualityEliminating duplication of effort requires collaboration on a higher level, and Avoca sees its role as acatalyst for moving things forward. While there is no one-size-fits-all formula for oversight that can beapplied to all organizations, Ms. Leuchten believed the time was right for the formation of a consortiumto work together to bridge gaps and to develop mutually agreed upon standards for quality.Avoca is using its consultants, subject matter experts, and partners such as the Metrics ChampionConsortium (MCC) to move things forward quickly. The Consortium and Summit are opportunitiesto improve outcomes through collaboration in Quality Management and metrics, Ms. Leuchten says.It is a forum to share industry best practices and establish new standards, and it represents a significantopportunity - because we have the industrys leaders together.The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.2 6. THE AVOCA QUALITY SUMMITConsortium members attending the Quality Summit engaged in interactive discussions and collaboratedon how best to implement robust quality programs in their organizations to improve the quality andoutcomes of outsourced clinical trials. The Summit was the first of what will be several opportunitiesto collaborate on the implementation of new standards. In short, it provided a forum to find proactive,rather than reactive, approaches to the fundamental challenges the industry is facing.SUMMIT DAY 1: First Steps Toward Creation of a Quality Agreement Template andQuality Metrics Document.On Day 1 of the Quality Summit, 50 sponsor pharma/biotech members of the Avoca QualityConsortium reviewed new data from Avocas 2012 Quality Consortium Assessment Sponsor Survey anddiscussed the creation of Quality Agreement template/standard that applies to many outsourcing models.A draft template that was created by Avoca based on a compilation of sponsor companies individualQuality Agreement templates was reviewed and discussed. (CRO members of the Consortium are now inthe process of reviewing and providing comments on the second draft of this template.)Consortium members working on the Quality Agreement template/standard decided that a morestreamlined template was needed to make it easier to move toward an industry standard. It wasdetermined that functional details should be omitted from the standard template since this languagetypically appears in other types of documentation such as monitoring and data management plans. Thelanguage and text specific to functional areas will be preserved in a separate repository of information sothat members have access to the specific language if they need it. Thus, there will be references to thesedetails in the Quality Agreement.Consortium members also worked on the creation of a Quality Metrics document, creating a list of 12-15essential metrics focused on measuring quality.In advance of the Quality Summit, sponsor Consortium members were asked to submit their qualitymetrics to Avoca. These were reviewed and consolidated prior to the Summit. We were provided withcycle time and performance metrics and some relationship metrics, noted Lisa McKay, Avocas SeniorDirector of Relationship Management Programs. We also saw a great variation in what types of metricspeople are using, so it was obvious that there is a real need to develop standards and really assess andaddress how the industry is defining quality metrics.Cycle time metrics and performance metrics not directly related to monitoring of quality were taken outof the final mix, leaving 116 metrics that were provided to sponsor members of the Quality Consortium asa starting point for their work.The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.3 7. The 116 metrics fell into 3 categories: Quality of operations in outsourced trials (or in-sourced trials); Oversight of investigative sites; Quality of response to findings.Using these 116 metrics, each individual was asked to select the top 12-15 that would become essentialto measuring quality in clinical trials. There is still work to be done, Ms. McKay said, but we did see agreat deal of consensus on some aspects. Avocas next step is to get input from CRO members and thento work closely with the MCC to integrate the quality metrics into MCCs system of organizing metricsby work streams.The Quality Metrics document is still a work in progress, but the top 3 metrics that emerged at the end ofDay 1 were:1. Number/percentage of repeat audit findings or issues after corrective and preventive actions (CAPAs)have been implemented;2. Percentage of personnel passing Good Clinical Practice (GCP) compliance and CRO/sponsorStandard Operating Procedure (SOP) testing prior to being assigned to the project;3. Percentage of significant quality issues as defined in the Quality Management Plan (QMP), escalatedto the sponsor within a defined timeline.SUMMIT DAY 2: Exploring Issues Around Quality.Day 2 of the Quality Summit, open to Consortium members and non-members, featured fourroundtables focused on 1) documenting expectations for quality, 2) measuring quality, 3) exploring howto ensure high-quality and proper oversight in emerging regions, and 4) strategic alliances and howpartnering has an impact on quality.The message permeating the days roundtables and presentations was the need for collaboration andcooperation at the highest possible levels of the industry. Weve got a collective problem that impactsour jobs, our abilities to get medicines to patients, said Jeffrey Kasher, PhD, Vice President and COO ofGlobal Medical R&D at Lilly. We as individuals and individual organizations cant solve this problem-its bigger than any one of us. But Im convinced that if we work collectively, we can solve the problemand move the needle forward. Dr. Kasher echoed what Patricia Leuchten noted earlier about Avocasgoal in launching the Consortium and Summit: It starts with a group of people driving change; thenthe rest of the folks come along. We have to drive change. If were not ready to do that, then the cures forAlzheimers disease and cancer are going to come from some other new industry, not ours. The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.4 8. As if to underscore that very criticism, in his introductory remarks, Jeffrey Kasher asserted, As anindustry, we like to come together and talk about problems but not act. He urged everyone to make thatstatement untrue.Jeffrey McMullen, President and Chief Executive Officer of PharmaNet/i3, and John Hubbard, PhD, FCP,Senior Vice President and Worldwide Head Development Operations of Pfizer Inc., also spoke forcefullyin their introductory remarks about the need to work together to change the industry so that it meets thechallenges facing it. Were all confronted with increased complexity of clinical trials, increased regulatoryscrutiny, increased globalization of trials-real challenges we all need to address successfully, Mr. McMullensaid. Were in this together. Were all professionals. We share a common desire to produce the highestpossible quality in the work that we do and in the products that we ultimately bring to market.Whatever side of the equation youre on, pharmaceutical/biotech or CRO, there is incredible pressure. Theneed for collaboration between sponsors and providers has never been greater. This industry has been atarget for just about everybody, John Hubbard noted. We will not be successful if this is just a biotechor just a pharma initiative. It has to be an industry initiative that includes our partners. This has to be acollaboration.Jeffrey Kasher used starker terms to explain why the time to bridge differences, look for common ground,and harmonize to get to some common solutions is now: We have a big problem: the industry as weknow it and love it is not viable anymore. We are facing death. That is just a reality. We are too expensive,too slow, and were not bringing forward medicines that add enough value to society. While theres nomagic bullet to turn that around, he said there are several key levers that can be used to retain the industryor transform the industry going forward. Quality is one of those levers because focusing on quality canincrease speed, drive down costs, and increase the value of the products/medications companies areworking on.The fundamental importance of developing definitions and standards for quality was underscored in allthe roundtable discussions held at the Quality Summit. Quality is clearly on the agenda and maybe evenin the crosshairs of the FDA and other regulatory bodies, said Jeff McMullen. It is the responsibilityof our management team to identify the objectives around quality, to build employee commitment to it,to train our employees and investigators relative to that commitment, and, most importantly, to employcontinuous process improvement initiatives. His mission is to maintain a culture of quality. Butdefining quality is much easier said than done. Quality includes the obvious tangibles, such as things thatwe measure around deliverables, said Mr. McMullen. But there are also the intangibles, which I franklythink is whats behind some of the data Avoca presented with respect to different perceptions aboutquality reported by CROs versus Pharma/biotech.Its all part of what I call the Universe of Quality-both tangible and intangible, Mr. McMullenexplained.Getting to quality as a culture is a journey, Jeff Kasher acknowledged. At Lilly, there is a robust qualitysystem in place: integrated standards, business processes, clear governance, management oversight, and arobust organization, which ensures the maintenance of the quality system. Lilly has also put a big focus The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.5 9. on oversight of outsourced work. Weve got documentation, quality agreements, clear metrics, leadingindicatorsand we are trying to work with our partners to ensure the work is in place, weve got cleargovernance, and a means of escalation of how all of that work gets done, Dr. Kasher said.Whats needed now is consistency of quality standards across our organizations and industry, he said.The only way that is going to come to pass is if work is shared and discussed with CROs. And wecant just talk about it. We need to come together and say, This is how were going to do things: QualityAgreements are going to be like this across the industry, and here are the metrics were going to track asan industry. I like to use the phrase, different is different, better is better; dont confuse the two, said Dr. Kasher.We have been fond of everybody doing everything a little bit differently. But at the end of the day, is thatreally better? Most of the time, the answer is no.John Hubbard reiterated the need for establishing a Quality Agreement in the very beginning of thestudy design process. As much as we try to standardize processes, were still dealing with variabilityacross teams, he said. We always hear oncology is different, or immunology is different. Every teamfeels theyre different. Sitting down with each of those teams and defining that Quality Agreementprospectively with the alliance partner helps to reduce the noise and also the feeling that were not gettingthe quality that we want. The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.6 10. ROUNDTABLE 1: Documenting Expectations for QualityDoes quality drive price up?The tension between cost and quality was addressed by panelists in the first Roundtable. Theres hugepressure to get products to market quickly, and concern over loss of exclusivity, which creates a tensionthat unfortunately sometimes favors speed over quality, said John Hubbard. Ive made clear in myorganization that quality will not be sacrificed for anything else. Thats a good thing, and the right answer,but it creates constant tension.Tension is caused by a lack of understanding about what the real costs are and what it really takesto deliver a project. With the advent of more outsourcing, these costs are finally entering everyonesconsciousness. A big problem has been the industrys tendency to commoditize things that are notcommodities, said Dr. Hubbard. We want to negotiate good rates for activities, but we have tounderstand how the complexity of a study will drive up costs for certain activities even though the ratesmay be low. Frequency of monitoring, for example, should be driven by whats required for the protocoland the enrollment rate. Take the conversation out of the cost realm and put it more into the operationaldelivery realm, which is where I think it belongs, he concluded. Then its a debate around whats the bestapproach. Maybe theres an approach that will still deliver quality at a lower cost, but its not driven bycost, per se.2011 Avoca Research: Causes of Issues with QualityDo you believe that there is a correlation between the demand for decreased prices by sponsors and an overall lowering of quality by CROs?* CROsSponsors 100 2% 17% 24%80 Yes6040%No74%40 Dont know2043% *Data from the 2011 Avoca Industry Research:0Ensuring Quality in Outsourced Clinical Trials N 9384The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.7 11. Defining quality and articulating expectations.Re-work emerged as another explanation for the gap between sponsors and CROs satisfaction levels.Jorge Guerra, MD, Senior Vice President of Global Clinical Development Operations at Biogen Idec,pointed out, Re-work has a huge cost; if we could just build in quality up front and do a much better joband eliminate re-work, we could increase quality and satisfaction. The issue of re-work goes directly tothe issue of defining quality and establishing clear expectations. Were going into these relationships allagreeing that we want to produce a high-quality product, noted Jeffrey McMullen; but we dont definequality. We have these expectations that are not defined, and somewhere along the way we find out thatthe CRO is going in a direction thats not in line with the expectation of the sponsor. This is where re-work is required and inefficiency results. Standards, definitions, expectations, and metrics used effectivelywill improve quality and control costs, Mr. McMullen believes.Expectations differ from team to team. We can be working with the same CRO and its still confusing.If people dont know what theyre shooting for and what the expectation is, theyre going to miss, JeffreyKasher said. If were consistent and do things to a consistent standard in a repeated manner, quality willbe higher and the work will be done better and more efficiently, which results in speed, reduced cost, andbetter quality.Paula Brown Stafford, President of Clinical Development at Quintiles, agreed that different customersand sponsors have a somewhat different take on quality. That is why its necessary to first define whatquality means. If we define it as a standard, it would help everyone, she said. Next, we have to measurehow to define quality. Its not enough just to put in an integrated quality plan-we have to measure to thatplan on a consistent basis across the teams.A lack of consistent standards adds to the risk of failure, said Jeff McMullen. If youre working with 75different clients and doing things 75 different ways, its a challenge to get everyone properly oriented tothe processes they need to conduct. If we can get industry standards across the industry around quality,well minimize the need to address problems with the incredible amounts of training and oversight weall do. The more we can standardize, the more opportunity we have to lower costs because of greaterefficiency.We have examples where you have a highly complex therapeutic area, a highly complex protocol, andpeople are expecting perfection on a bargain-basement budget but arent prepared to make the tradeoffsin other areas to assure the quality, John Hubbard asserted. Those discussions need to happen very early.In the past they were just assumed, and thats when you have a misalignment between the sponsor and theCRO.Year after year our data show that the industry as a whole seems to have trouble articulatingexpectations, Avocas Patricia Leuchten noted. This is also true internally, within sponsor organizations;there is trouble just getting agreement internally on what the expectations should be. Is there an issuewithin both sponsors and CRO organizations concerning the staff s ability to know how to get clarityaround expectations, communicate those expectations, and understand that when things change, there hasto be another conversation?The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.8 12. Its human behavior, ventured Jeffrey Kasher. Everybodys project is the most important project and everyproject is slightly different. If youre on a team, your focus is your molecule, your study, your expectationsand demands, and in a large organization you have that happening in every team. For that reason, itsimportant to look at the portfolio level and agree on standards that meet needs across the portfolio. Itsthe only way to get the efficiency and overarching speed of movement that we need.Problems arise from a mismatch of expectations-whether thats within a team or across a portfolio,noted Harris Koffer, President and CEO of RPS, Inc. Expectations should be identical; there shouldbe no difference between partnersinternal or external. Expectations must be aligned up front, makingsure that the roles and responsibilities and accountabilities are clearly understood between the parties.While standardizing the Quality Agreement is part of the tool, I think that you have to look at thewhole process, he said. Quality Agreements define expectations around things that can be measured atan outcomes stage, as opposed to looking at the processes, starting with study design. But many of thechallenges the industry faces begin at the scientific study design stage, not in the field.Empowering individuals to reach for quality.How do companies empower individuals to reach for quality? This question was inspired by Jorge Guerrasquestion, Why does a CRO accept running a poorly designed protocol? The best thing is if someone atthe CRO says they cant run this poorly designed protocol, because it makes everyone rethink and startagain with building quality into the protocol. The sites that tell us the study cant be done are the best sitesto work with! They know what theyre doing.For this scenario to happen, however, there has to be a culture change in the industry, said John Hubbard.CROs have to feel empowered and supported to hit the red button. At the end of the day, at the studyteam level, if the individual feels pressure to go faster and feels like he or she might have to cut cornersto hit the date, they will have to believe they are empowered to say theres a risk of having a quality issuedownstream if things arent stopped. Thats the cultural change that has to happen to the individual,noted Dr. Hubbard. All of us have quality built into our goals, our systems, but the point of failure is stillthat one person on a study team whos being hammered by a project manager or clinicians or someoneto hit the date. Leadership has to be ready to delay a timeline, or a first patient visit, if its going topotentially have an impact on quality. Once people see that behavior was rewarded, a culture change canbegin toward a focus on quality. It was discussed that CROs have to have the ability to push back onsponsors and not be replaced just because they are not in agreement with a timeline or approach. Weneed to do a better job with our rewards systems, suggested Dr. Guerra, incentivizing behaviors thatpromote quality.Paula Brown Stafford directed some self-criticism to the CROs for their lack of proactive behavior withrespect to Quality Agreements. We providers may have failed somewhat because we have waited forpharma to ask us for Quality Agreements, and we have built a standard that we have been using. But howoften when were going into a new relationship do we proactively say, Can we put a Quality Agreementin place? CROs can do themselves a favor while the industry is moving toward those standards to at leastuse their own standards, to proactively ask for that Quality Agreement. Its too late once youre in the The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.9 13. middle of a program, a trial, or an FSP relationship because then youre only reacting when youre alreadyin crisis mode.Is having standards practically achievable?Unfortunately, the history in the industry is that companies are not adept at coming together to reachbroad agreement. But several Roundtable participants asked why should each company have a separate setof SOPs? Why not have one uniform set of SOPs for all of the functions?The big fallacy in our industry is that this is proprietary, John Hubbard asserted. While nobody sharesSOPs because theyre considered proprietary, in fact most of the work is driven by GCP and ICH,he said. There may be variations on the themes, but the basics are all the same. Standardization willmake it easier for the CROs to deliver better value because they do not have to recreate things from thedifferences that we have across the industry. I think everyone is motivated to do this, Dr. Hubbard said.Were doomed as an industry if we dont get this fixed. Its not a sustainable model. Costs are continuingto go up and we have to find a better way to be more efficient and reduce unnecessary complexity.Harris Koffer agreed that theres no reason for SOPs to be proprietary. Frankly, theres nothingproprietary about investigator selection either, other than the fact that companies dont want to share thatinformation, he said. Because we dont share it, it takes longer to start trials and longer to finish trials,and that will be the case until companies get together at the senior level and say there will be one set ofSOPs across the industry.We suffer in this industry from our inability to get standards-data standards or quality standards. Wewill only move forward if we take the time to have the discussions and get industry-wide agreement onstandards, said Jeffrey Kasher. Yes, we have relationships inside companies, but now were working ina more networked world, in partnership with many CROs, and we often dont take the time up frontto define what the fundamentals are. Its the changing rules, constantly changing landscape, that causesproblems. If we can define things and adhere to them, well make progress, he said.This Summit is an opportunity for the industry to get ahead of the regulators and agree on somethingthat is going to become an expectation in the near future from every agency around the world, said JorgeGuerra. Everything starts with a small group, he said, Thats why we are here today.The elephant in the room.One of the obstacles to achieving a high level of cooperation was brought into focus by Paula BrownStafford. The elephant in the room is that sponsors compete on products, but CROs compete on ourpeople, our processes, and our systems. I do believe that we have to standardize at some level to ensurequality, but you have to think about time, cost, and quality being what we compete on. How far do wecome toward one set of SOPs for the industry whereby theres no advantage between us? At the sametime, we do want a healthy industry; we want competition between ourselves to become better partnerswith you. The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.10 14. Theres another way of looking at that competitive advantage, however, said Harris Koffer. The advantageis not from the SOP; its how you implement and apply it around those functions, the experience of yourpeople, how well theyre trained, how well they execute around the SOPs. The hurdle we have to get overis the failed notion that the nonstrategic aspects of the business-not the innovative aspects of designand how you commercialize the productbut the bread-and-butter execution of these important butnonstrategic elements of the business are still viewed as proprietary.The concern about compromising individual companies competitive advantages by sharing SOPs andother formerly proprietary materials is not an isolated one. When we started talking to companies aboutthe Quality Consortium, some of the executives from pharmaceutical companies said theyd spent yearsdeveloping some of these processes and had put enormous time, energy, and resources into them, saidPatricia Leuchten. They asked why they should hand them over to the Avoca Quality Consortium so thatothers can be better. There are some hurdles to get over in terms of moving toward industry standards,and it is incumbent on us to choose what are the high priority activities where everyone benefits-sponsorsand CROs, she noted.There are people in the industry that dont get it, said Jeffrey Kasher. This movement to collaborate andcreate industry standards hasnt happened yet because, quite simply, everyone has been able to do thingsdifferently and hold their cards to their chests. Because we could afford to, he said. Nowadays, we cant.Ive seen a definite change in the last 2-3 years, where its still too much talking versus doing, but peopleare finally coming together and putting their cards on the table.An audience member, Dr. Francisco Harrison (founder of Harrison Clinical Research), expresseddisappointment that the industry tends to commodify quality. Quality is a result, he said. I totally agreewith Jeffrey Kasher: if we dont change this system, we are going to go down like the Titanic! We have toremember that on the other side of the table is not the sponsor or the CRO. On the other side of the tablethere is disease, death, and people who are suffering from the consequences of disease. This is why we haveto have quality: for the patient. If we understand this message and come to the same side of the table, wewill solve the problem.The burning platform.Productivity has declined to one-fifth or one-sixth of what it was 15 years ago, according to Harris Koffer.That is not a sustainable platform, its a burning platform, he said. The end result of a burning platformis that the patient doesnt get the drug in a timely manner, but the term burning platform also suggeststhe unsustainability of an industry that has had productivity decline at the rate it has. When you think ofhow much training goes on because each company does it a different way and because each company has adifferent way of operating, and how monitors are trained for sometimes three months, think of how muchtime, effort, cost, in that one area alone could be saved, Dr. Koffer concluded.Responding to burning platform imagery, Jeffrey Kasher ventured the thought that the industry iswearing very well-insulated shoes. For an industry that is pretty close to deathand I dont think thatsmuch of an exaggeration-most people arent feeling the heat, he said. Look at the economics of thebusiness. Are enough people going to realize that?The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.11 15. Summing it up - Roundtable 1: Issues and suggested actions.In Roundtable 1, many issues were identified as industry-wide problems that require creative solutions: The constant tension between quality and cost can favor speed over quality. There is a lack of understanding about the true costs of high-quality clinical trials. Too much re-work is requiredlargely the result of poor communication, a lack of clarity, and failure to define quality. There is a lack of consistent standards. Many communication problems are the result of poor clarity of expectations. The present industry business model is not sustainable, as is demonstrated by declining productivity. The business model relies on a failed notion that nonstrategic aspects of the business are proprietary.An important part of the solution for many of these issues is the governance of quality and ensuring thatstandards are implemented as part of this governance. Specific suggestions included: Define and build in quality from the very beginningstarting with protocol design. CROs should be involved earlier in the process and given more input to the process. Greater clarity and specificity about communicating expectations. Standards, definitions, expectations, and metrics used effectively will improve quality and control costs. When changes occur, have new conversations. Avoid re-work by doing it right the first timea more likely scenario if quality was built into the protocol from the start. Look at the portfolio level to realize greater speed and efficiencies. Build quality into every individuals goals and objectives, at every level of the company. Then measure it and reward it. Empower individuals to feel they can hit the red button when somethings not righta true culture change. The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.12 16. ROUNDTABLE 2: What are the Considerations when ImplementingComprehensive Metrics Programs?Measuring quality.Jonathan Lee, VP of Development Operations at Cerexa, said that because he works with smallercompanies its typically hard to get traction with senior management in terms of setting up metrics. Animportant consideration is figuring out what the maturity level of a company is and whether they areready to engage in metrics. Cerexa aims to breed metrics into their system by negotiating metrics upfront. There are 14 metrics written into every CRO contract, he said. Our CROs sit in protocol designmeetings. We pay them separately for feasibility. We only dictate the timeline and number of subjectsand ask where in the world we should go to do the study. Feasibility is codified in the contract and thereare bonuses and penalties associated with the work. Each month, those 14 metrics are reviewed.Hazel Collie, VP of Global Clinical Quality Assurance and Compliance at Grnenthal GmbH, alsoworks with small CROs. In the past, weve done some metrics exercises, and realized it didnt tell usmuch. Because were small, we have months with no audit reports, no throughput, so thats a real issuefor us. We want to measure, but what does it tell us? Can we compare what we do in-house to what theCROs do? Were getting there but were not there yet.At Amgen, however, which completes about 275 studies a year across different therapeutic areas andcountries, robust metrics are very important. Dr. Elspeth Carnan, Amgens Executive Director of GlobalClinical Site Management, explained that because of the scope of their work, 11 different providersare used. The metrics really have to mean something, she said. Any effort to create a Quality Metricsindustry-wide document will require a tremendous effort to narrow down the 116 metrics Avocaconsolidated to a few that mean something. We have to be able to ensure we can measure them and haveappropriate targets set, said Dr. Carnan. The metrics have to be easily measured, easily transmitted in oursystems, and timely.Providing the CROs perspective, Angelika Tillmann, Clinical Research VP of Global Clinical QualityAssurance and Compliance at Theorem Clinical Research, said there are different aspects to consider.Because you will have to have metrics directed toward the study, they will be different for each sponsor,she said. We are still struggling to find metrics that mean something. Its easy to check certain metricsoff the list, Ms. Tillmann acknowledged, when they relate to performance rather than quality.Quality is hard to define.Why has the industry struggled to create quality metrics? Its simply easier to define speed or performanceor cost metrics, but quality is hard to define. Thats why its been challenging, but now were coming tothe point where we are starting to see some consistency across some of the definitions and were gettingmore data around errors and training, said Dr. Carnan. The Avoca Group Inaugural Quality Summit:Establishing a Best-Practice Approach to QualityManagement and CRO Oversight.13 17. We monitor audit reports. We build quality metrics into our score cards (Key Performance Indicators)and our expectations are clearly articulated to our providers, said Dr. Carnan. They are told that wemeasure these metrics and monitor them; we expect action plans if we go off track against these metrics.We dive deep into causality of issues, as far as audits findings, because its important to have a deeperunderstanding of what goes wrong. Amgen wants to reduce that type of audit finding and they holdpeople accountable at all levels of the company.At Theorem, risk-assessment meetings are conducted to examine the risk of the protocol, and to get thesponsor involved early on to make them aware that previous experience in similar studies showed that aparticular aspect of the study has certain risks attached. We all need to think in these terms, of drawingcloser together and the boundaries disappear a bit, said Ms. Tillmann.Quality metrics are particularly difficult to use as leading indicators. Cerexa uses a composite metric thattheyve created; the subject has to be evaluable. There are 6 criteria. We set it as a key in our service levelagreements. For individual subjects it is a lagging indicator, but it becomes a leading one as the subjectgoes through the study, Jon Lee explained.Amgen uses audit reports as a lagging indicator. Audit reports reflect after-the-case. If you see trends, itcan be used for risk evaluation. You have to be able to gather enough data to know what are the triggers tocomplex issues that are going to lead to more errors, Dr. Elspeth Carnan said.2012 Avoca Consortium on Quality: Use of Quality Metrics (Sponsor Data)In general, do you feel that the quality metrics you use are adequate in design (i.e., in terms of scopeprovide meaningful(sufficient number reflection of actual of quality metrics)?quality delivered)? 10013%12%80Yes 32%36%60Sometimes40No55%52%20 *Data from the 2012 Avoca Consortium on Quality 0 N183181The Avoca Group Inaugural Quality Summit: Establishing a Best-Practice Approach to Quality Management and CRO Oversight.14 18. 2012 Avoca Consortium on Quality: Use of Quality Metrics (Sponsor Data) Do the teams with which you work use quality metrics to evaluate the performance of clinical service providerswith respect to the quality of services and deliverables?*NB 97% 3% 30C 91%9% 11A 90% 10% 21G 90% 10% 20D88% 12%34 I 82% 18%17N 73%27%11Company:J70%30% 10F69%31% 16H61% 39%31E 57%43%23K 56%44%25M47% 53%17L 44% 56% 9 All 74%26% 275 0 2040 60 80 100YesNo *p