Slide 1

The ISEAL Assurance Code ISEAL Conference June 10 2011 Patrick Mallet, ISEAL Credibility Director Paddy Doherty, ISEAL Code Development Manager Slide 2 Initiating the Consultation Scope agreed by Stakeholder Council Based on 4 month consultation process Prioritization of issues through survey, key person interviews Background Research Carried out by Richard Bradley quotes throughout Background information and issues to consider Consultation Process Steering and Technical Committees will meet later in June Your first opportunity to provide input to Code content First draft by September, approval in June, 2012 Slide 3 Consultation Findings In Brief Slide 4 Slide 5 Proposed Scope Include some or all of the following issues Auditor Competence - screening, training, qualification, calibration and monitoring Audit implementation minimum requirements for good practice + guidance notes to ISO 17065, 17021 Transparency additional requirements (beyond ISO) where needed transparency can reduce the need for excessive rigour Standard quality consistent interpretation of standards Accessibility deals with the challenges of cost and access and will include innovative options Complementary to ISO standards (17011, 17065, 17021) Requirements apply to scheme-owner; & CBs and ABs where appropriate Slide 6 Issues to consider for the Assurance Code Some standards schemes require compliance (and are satisfied) with ISO standards while others do not One-size does not fit all defining a cascade of assurance options appropriate for the scheme and its stage of development ISO standards are good at management systems for consistency, competency, and impartiality, but do not cover the soft issues important to ISEAL members Who is responsible for which activity is not always clear eg: training auditors, monitoring CBs Certification can deliver additional benefits besides the assurance (thus changing the cost/benefit ratio) Slide 7 Findings from the Research Interviewees for this project identified common issues Few could define a minimum level of certainty that they wanted an assurance program to deliver. All agreed that some form of risk assessment was required in a sampling program to focus audit attention on higher risk activity. While many used a formula to determine sampling numbers (e.g. 10% or square root), none had a statistical explanation as to why they used that formula. Most used judgmental sampling rather than statistical sampling. The cost of each program having its own different audit management systems was significant, and there was interest in how collaboration could take place. Slide 8 More Findings from the Research ISEAL members are taking innovative approaches which could be used as models to develop materials for the Assurance Code especially if a cascade of verification requirements was developed Most schemes do not have measures of overall performance, and do not understand sampling or risk assessment processes well The role of technology in certification is increasing, and the pace of change is accelerating. The Assurance Code needs to be able to accommodate these changes, and could be used to hasten or restrict them Slide 9 External Trends and Advances Strong trend amongst other programs (e.g. ISO, food safety) to place increased emphasis on personnel competency and potential for personnel certification. IFOAMs participatory guarantee system encourages self and/or peer assessment. Successful when local stakeholders are fully involved. GlobalG.A.P. operates a certification integrity program, in which staff repeat both accreditation and certification audits to compare results checks if outcomes achieved and calibrates accreditors and certifiers. European product conformity system (the CE mark) has a number of levels of possible assurance, to which products assigned due to risks of failure. Audit technology is rapidly developing, and the Assurance Code should take those changes into account. Slide 10 Choice of Assurance Models What level of risk is acceptable? Slide 11 Risk-based Approaches Certification as a risk management programme Audit risk is the risk that the audit will not provide an accurate conclusion as to client conformity Expressed by multiplying three factors: Control risk the risk that the client does not know that their system is non-conforming Inherent risk risks associated with the client, the industry or culture Detection risk the risk that the audit will miss non-conformities if they exist Slide 12 Sampling Sampling is inherent in certification but may not be explicit Sampling used in choice of who to audit, how frequently and what to audit focusing auditing on higher risk activity Most systems use judgmental or non-statistical sampling Limitations on conclusions that can be drawn Sampling within an audit can be performed in differing ways: Representative, at random (acceptance sampling) Focused on finding problems to be corrected (corrective sampling) Sampling the important issues to protect scheme (protective sampling) Preventing client from predicting sample, thereby lowering audit risk (preventive sampling) Slide 13 Risk and Sampling Options Many programs require CBs to perform a risk analysis, with little or no instruction as to how this should be performed or what evidence of analysis is required The Assurance Code could set out a standardised risk assessment program to be followed by certification scheme owners and by CBs. This could include methods for identifying hazards and risk analysis, and may include sections on identification and selection of risk controls, and on monitoring of effectiveness. Should the Assurance Code define how a risk assessment is to be performed and assign responsibility for performing it? Slide 14 Audit Performance Audit performance, and hence the credibility of assurance, is the sum of CB management and auditor competency. As many have commented, our approach to auditor competency is weak our concentration is on CB systems. Perhaps a rebalancing is required? Now Later Slide 15 Auditor Competence Personnel competencies: Can describe qualifications required, or Can describe what an individual must be able to do (outcomes) Latter approach is recognised as being more reliable and is being more widely adopted Those evaluating personnel competency can follow ISO17024, a standard for personnel certification bodies Increasing numbers of schemes are using established certifiers such as IRCA or RABQSA for this purpose Possible benefits for ISEAL members to adopt a cohesive approach to certification of personnel to avoid duplication, allow people to work across programmes and reduce costs Slide 16 Auditor Competence Options As well as setting competency requirements, an Assurance Code could consider minimum requirements for auditor experience as an auditor, and for auditor supervision and continuing professional development If the Assurance Code includes personnel certification requirements, other system requirements may be able to be lessened An Assurance Code could set out a generic process for competency evaluation. If it did so, it should consider basing processes on ISO 17024 requirements It may be efficient to have a central (common) registration / accreditation programme for auditors Slide 17 Audit Implementation Options Audit software auditors use templates that ask questions based on inputted information (RA Tourism is pleased with this approach) Common requirements for audit systems: software, reporting frameworks Common methodology for risk-assessment & sampling Certification scheme owners should consider their strategic objectives before deciding on which sampling strategy to follow during audits. An Assurance Code could set out examples of sampling strategies to be followed for differing types of objectives Slide 18 Audit Technologies Audit technologies have been developing rapidly, enabling the following Workflows built into software - checklists change based on responses Options to select descriptions of how the client achieving conformity, beyond yes / no Logic rules ensure complete audits and identify inconsistencies Information on risk used to change audit frequency or intensity New reporting tools, combined with faster hardware, increase ability to extract information from data Operating costs and response times are lowered Use of mobile phones for data transfer allows relatively low cost, almost ubiquitous access Slide 19 Accessibility A cascade of increasing verification requirements could have appeal. The Assurance Code could describe the verification requirements needed at each level within the cascade eg: depending on x, you are required to: - Comply with ISO Standards; or, - Second-party certification combined with selective auditing or, - Self declaration combined with peer review and risk-based sampled third-party audits; or -Another level of assurance (eg: certification of persons) Risk assessment to reduce frequency of audits select the control option that gives best control at reasonable cost Slide 20 Costs and Accessibility Simplistic financial model for a CB with 300 certificates Assumes surplus remains constant at 5% Cost reduced or size increasedReduction byFee drops by Accreditation fees50%5.25% 100%10.5% Auditor salaries, audit time or number of audits 20%7% 50%17.5% Drop both accreditation fees and auditor salaries, audit time or number of audits 20%9% 40%18.25% Increase CAB size50%5.5% 100%17% 400%25.75% Slide 21 Costs and Accessibility Implications of the model: To create a 10% drop in fees charged to clients, one of the following would be needed: Accreditation fees would need to drop to zero Audit salaries, frequency or duration would need to drop by 30% Accreditation fees and audit salary costs or audit frequency or audit duration (or a combination of the last three) would all need to drop by 23% CB volume would need to increase by 65% Simplest method of lowering fees with no impact on credibility is increased throughput in each CB Slide 22 Standard quality options: Good practice in crafting standards that provide for consistent interpretation Requirements for guidance and support to auditors to ensure consistent application of the standard Slide 23 Transparency Alternate assurance systems should include requirements for transparency (beyond what ISO standards require) Public client list Public list of de-certifications Certification scheme owners using 17065 as a base for their programs could consider aligning how they present CB requirements to match the layout and format of 17065 Current accreditation processes focus on a limited number of issues related to systems, competency and organisational behaviours. The Assurance Code may need to consider whether it should widen the ABs brief to include whether or not strategic objectives, including outcomes, are met Slide 24 Issues to Resolve The greatest challenge will be the discussion of how sure do we want to be? once this is known, choice of assurance models becomes easier Some stakeholders are demanding more rigour in assurance while others feel the costs outweigh the benefits how to reconcile? What is the balance in the Code outputs between requirements and guidance? Technology and knowledge could be combined to have a scheme run without traditional CBs (certification of auditors) Guidance for capacity-building (delivery of knowledge) in the audit (adding value to assurance to change the cost/benefit ratio)