company confidential registration management committee 1 completing independent assessments robert...

Company Confidential

Registration Management Committee

1

Completing Independent Assessments

Robert Flaharty & John HoranJanuary 17, 2013

OP Assessor Workshop San Diego, CA – 17 Jan 2013

OP Assessor Workshop, San Diego, CA - 17 January 2013


2

Independent or Supplemental OP Assessments…

• “Avoid them……..?”– Too much work, unsure of the process, etc.

• Embrace them…….!– The oversight process was designed to be more

than the Accreditation Board oversight.

– Member companies have a responsibility in the oversight process.

Which one best shows your preference?



3

Levels and Types of Oversight…

Oversight of SMS

Conducted by the IAQG OPMT

Oversight of CBMC’s

Conducted by the SMS

Oversight of AB’s

Office Assessments

Witness Assessments

Oversight of CB’s

Office Assessment

Witness Assessment

Oversight of AAB’s

Office Assessment

Oversight of TPAB’s

Office Assessment



Oversight of SMS

Conducted by the IAQG OPMT

Oversight of CBMC’s

Conducted by the SMS

Oversight of AB’s

Office Assessments

Witness Assessments

Oversight of CB’s

Office Assessment

Witness Assessment

Oversight of AAB’s

Office Assessment

Oversight of TPAB’s

Office Assessment

4

Levels and Types of Oversight…

Our focus in this presentation…



5

Focusing on Oversight of CB’s…

Oversight of CB’s

Office Assessment

Witness Assessment

• Each member company is required to complete 4 oversight assessments/activities per year.

• All assessment activities shall be planned and scheduled with all affected parties, prior to conduct.

• Whenever possible, OP assessors shall team with ABs to conduct joint team assessments.

• IAQG member companies may conduct additional oversight, above the minimum 9104-2 requirements and outside the SMS shared oversight schedule.



6

Focusing on Oversight of CB’s…

Oversight of CB’s

Office Assessment

Witness Assessment

• Each member company is required to complete 4 oversight assessments/activities per year.

• All assessment activities shall be planned and scheduled with all affected parties, prior to conduct.

• Whenever possible, OP assessors shall team with ABs to conduct joint team assessments.

• IAQG member companies may conduct additional oversight, above the minimum 9104-2 requirements and outside the SMS shared oversight schedule.

Sometimes, circumstances don’t allow this to happen or an IAQG

Member company may choose to Conduct a Supplemental IAQG Member Oversight assessment



7

Conducting Independent Office or Witness Assessments…

• AS9104-2 Defines the Requirements for Oversight of Aerospace Quality Management System Registration/Certification Programs

• Includes the various check sheets, declaration forms, records and reports in Appendix A through K.

Standard Work (Job Aid) for planning and completion of Independent Assessments are available upon request.



8

Independent CB Office Assessment…

Standard Operating SheetOPERATION

Independent OP Certification Body OFFICE Assessment

11/1/2012

Reference AS9104-2

Num ACTIVITY PROCEDURE KEY POINTS

1

Plan and schedule assessment activity

Work with the CB to establish a plan and schedule for the assessment activity.

Document and communicate the Assessment Schedule to the Certification Body (CB)

Document and Communicate the Assessment Schedule to the SMS Oversight Surveillance Sub-team Chair.

Define the Who, What, When, Where, Why and How of the assessment.

2

Identification of materials required in advance of the office assessment

Communicate to the Certification Body any requirements that you may have for information in advance of the assessment to assist you in preparation.

Such information may include:• Client List and Number of Certifications by Country• Number of Transfer certificates by Standard• Number of Multi-site certificates by Standard

• List of Auditors (include country, standard, and status) • List of Technical Experts• List of Contract Reviewers by Standard• List of Certification Decision Makers by Standard• List of Staff and position• List of Board and/or Committee Members (as applicable)



9


3

Conduct the assessment

Conduct the Certification Body Office Assessment using the CB Office Assessment Check Sheet, as guidance (see AS9104-2 Appendix F).

The CB office assessment shall include a review of CB activity to ensure conformance with the requirements of 9104 and applicable industry standards.

The assessment shall include a review of the latest CB oversight office assessment reports, including actions taken.

Conduct Opening Meeting with the Certification Body representatives.

Typical Opening meetings address:- Purpose and scope of the assessment- Introductions- Review of the Checklist to be utilized- Documentation that will be completed and submitted- Management process for Corrective Actions- Confidentiality and Conflict of Interest- Complaint Resolution Process- Records generated from this Assessment- Questions from the Certification Body

Conduct Closing Meeting with the Certification Body representatives.

Typical Closing Meetings address:- Voice appreciation for the professionalism and dedication process- Review the purpose and scope of the assessment- Review of the areas assessed- Strengths and Opportunities identified- Review of any Nonconformity Records generated- Expectations regarding Corrective Actions (content/timing)- Documentation that will be completed and submitted- Reinforce the Confidentiality and Conflict of Interest- Reinforce the Complaint Resolution Process- Records generated from this Assessment- Questions from the Certification Body



10


4

Document any nonconformities

Document any nonconformities identified during the assessment are to be recorded on an Oversight Nonconformity Record (see AS9104-2 Appendix J).

Provide the Certification Bodies "organizational representative" with copies of any resulting Nonconformity Records

Ensure that required timing for Plan Due Date and Dates of Evidence Due are within the required timeframes per AS9104-2- Plan due within 30 days of the NCR date- Minor NCR - Evidence due within 90 days of NCR date- Major NCR - Evidence due within 60 days of NCR date

5

Reporting The results of the assessment shall be recorded on an Oversight Assessment Summary Report (see AS9104-2 Appendix K) by the OP assessor.

The completed Oversight Assessment Summary Report - Appendix K is to be submitted along with any resulting Nonconformity Records (see AS9104-2 Appendix J) to the SMS Oversight Surveillance Sub-team Chair.

Current Oversight Surveillance Sub-team Chair is:Robert [email protected](316) 523-9260

6

Manage any Nonconformities

The assessment team leader ensures that acceptable corrective actions for all identified nonconformities are addressed in a timely manner.

Unresolved issues shall be elevated to the SMS, in accordance with the process defined in section 11 of AS9104-2.



11


7

Verify Evidence of Implementation and Close Nonconformity Records

Upon successful closure of any Nonconformity Records, submit closed Nonconformity Records to the Certification Body and the SMS Oversight Surveillance Sub-team Chair.

Ensure that Sections 1 and 2 of the Nonconformity Record are completed as applicable.

8

Maintain Records All forms, check sheets, reports, and completed appendices required by this standard shall be considered a record.

The responsible party that conducts approval and assessment activities, in accordance with this standard, shall maintain records for a minimum period of six years.

Ensure that records are maintained in a safe environment to protect the confidentiality of any information contained in these records.

Steps 4-8 are the same for OFFICE or WITNESS assessments



12

Key Points…Office Assessments

• CB office assessments shall include a review of CB activity to ensure conformance with the requirements of 9104 and applicable industry standards.

• The assessment shall include a review of the latest CB oversight office assessment reports, including actions taken.

• OP assessors are responsible to ensure they have been approved by their respective IAQG member company and submit a completed declaration form to the SMS, prior to conducting an assessment.

• OP assessors are expected to declare any potential conflict of interest prior to acceptance of an oversight assignment.



13

Independent CB Witness AssessmentStandard Operating Sheet

OPERATIONIndependent OP Certification Body WITNESS Assessment

11/1/2012

Reference AS9104-2

Num ACTIVITY PROCEDURE KEY POINTS

1

Plan and schedule assessment activity

Work with the CB to understand upcoming audit schedule and work with the Certification Body to identify a client audit that will not pose any potential conflict of interest.

Document and communicate the Assessment Schedule to the Certification Body (CB)

Document and Communicate the Assessment Schedule to the SMS Oversight Surveillance Sub-team Chair.

Remember:The objective of the CB witness assessment is to assess: Team competence. Conformity to established requirements. The effectiveness of the assessment program. Conformity of the assessment team with the CB procedures and processes.

2

Identification of special accommodations required before and after the witness assessment

Communicate to the Certification Body any requirements that you may have for information and /or planning in advance of the assessment to assist you in preparation.

Notify the CB that the OP Assessor requires an opening and closing meeting between the OP Assessor and the Certification Body representatives and without client involvement.

In all cases, it is critical that the OP Assessor not influence the audit between the Certification Body and their client.

Typical communication to the CB may read as follows:OP Assessor requires an opening and closing meeting as described below.

Opening Meeting: A private meeting with the CB's audit team prior to the CB's opening meeting with the organization. One-half hour should be allocated.

Closing Meeting: A private meeting with the CB's audit team after the CB's closing meeting with the organization. An hour and one-half should be allocated. The CB's audit team is required to allocate sufficient time prior to their departure for the OP Assessors closing meeting.

CB's management is welcomed to join the closing meeting; it is the responsibility of the CB to coordinate this with their audit team and the organization (if applicable).



14

Independent CB Witness Assessment

3

Conduct the assessment

Conduct the Certification Body Witness Assessment using the AB/CB Witness Assessment Check Sheet (see AS9104-2 Appendix G).

In all cases, it is critical that the OP Assessor not influence the audit between the Certification Body and their client.

The OP assessor shall not influence the AB or CB audit team, during the audit conduct portion of the witness assessment. The role when witnessing is one of observation only. Feedback to the assessment team shall occur upon conclusion of the client’s assessment. The OP assessor must ensure that confidentiality between the AB/CB or CB/Client is maintained (e.g., no out-brief in front of client).

Conduct Opening Meeting with the Certification Body representatives.

No in-brief or out-brief in front of the Certification Bodies client

Typical Opening meetings address:- Purpose and scope of the assessment- Introductions- Review of the Checklist to be utilized- Review the role of the OP Assessor and the urgency of not influencing the audit

- Documentation that will be completed and submitted- Management process for Corrective Actions- Confidentiality and Conflict of Interest- Complaint Resolution Process- Questions from the Certification Body

Conduct Closing Meeting with the Certification Body representatives.

No in-brief or out-brief in front of the Certification Bodies client

Typical Closing Meetings address:- Voice appreciation for the professionalism and dedication process

- Review the purpose and scope of the assessment- Review of the areas assessed- Strengths and Opportunities identified- Review of any Nonconformity Records generated- Expectations regarding Corrective Actions - Documentation that will be completed and submitted- Reinforce the Confidentiality and Conflict of Interest- Reinforce the Complaint Resolution Process- Questions from the Certification Body



15


4

Document any nonconformities

Document any nonconformities identified during the assessment are to be recorded on an Oversight Nonconformity Record (see AS9104-2 Appendix J).

Provide the Certification Bodies "organizational representative" with copies of any resulting Nonconformity Records

Ensure that required timing for Plan Due Date and Dates of Evidence Due are within the required timeframes per AS9104-2- Plan due within 30 days of the NCR date- Minor NCR - Evidence due within 90 days of NCR date- Major NCR - Evidence due within 60 days of NCR date

5

Reporting The results of the assessment shall be recorded on an Oversight Assessment Summary Report (see AS9104-2 Appendix K) by the OP assessor.

The completed Oversight Assessment Summary Report - Appendix K is to be submitted along with any resulting Nonconformity Records (see AS9104-2 Appendix J) to the SMS Oversight Surveillance Sub-team Chair.

Current Oversight Surveillance Sub-team Chair is:Robert [email protected](316) 523-9260

6

Manage any Nonconformities

The assessment team leader ensures that acceptable corrective actions for all identified nonconformities are addressed in a timely manner.

Unresolved issues shall be elevated to the SMS, in accordance with the process defined in section 11 of AS9104-2.



16


7

Verify Evidence of Implementation and Close Nonconformity Records

Upon successful closure of any Nonconformity Records, submit closed Nonconformity Records to the Certification Body and the SMS Oversight Surveillance Sub-team Chair.

Ensure that Sections 1 and 2 of the Nonconformity Record are completed as applicable.

8

Maintain Records All forms, check sheets, reports, and completed appendices required by this standard shall be considered a record.

The responsible party that conducts approval and assessment activities, in accordance with this standard, shall maintain records for a minimum period of six years.

Ensure that records are maintained in a safe environment to protect the confidentiality of any information contained in these records.

Steps 4-8 are the same for OFFICE or WITNESS assessments



17

Key Points…Witness Assessments

• The objective of any AB/CB witness assessment is to assess:– Team competence.

– Conformity to established requirements.

– The effectiveness of the assessment program.

– Conformity of the assessment team with the AB/CB procedures and processes.

• The OP assessor shall not influence the AB or CB audit team, during the audit conduct portion of the witness assessment.– Not auditing the client,

we’re assessing the audit team.

– Do not interject into the audit.

– All feedback to the audit team is conducted after the audit closing meeting and always in private (away from the client)



Supplemental Assessments

• Embrace the process.• Understand that our actions impact the

credibility of the process.• Be professional, friendly and courteous.• Understand the various standards and

requirements.• Coordinate all logistics with ANAB or

the CB as appropriate.• Understand your roles and

responsibilities in Office vs. Witness Assessments.

• Complete and submit all required forms and submit in a timely manner.

• Manage any resulting Corrective Actions as appropriate.

• Don’t avoid the process.

• Don’t allow poor communication with ANAB or the CB impact the oversight process.

• Don’t guess, resources are available if you need to “phone a friend”

• Don’t interfere, or otherwise influence, witness audits between the CB and their client.

• Don’t delay in completing and submitting required forms.

• Don’t forget the roles and responsibilities that we have to provide oversight to this process.

Supplemental Assessments



19

Questions?

company confidential registration management committee 1 completing independent assessments robert...

Documents

oversight process

op assessor workshop

additional oversight

assessment schedule

sms shared oversight

accreditation board

op assessors

iaqg member company