ISO 9001:2000 General Requirements – Monitoring, Measurement and Improvement

Reviewer Checks Reviewer Checks

Planning the Process Audit

Define key inputs and resources for each of the processes to be reviewed. Use process documentation, flowcharts or SIPOC diagrams to identify these.

Determine the processes/activities to be included in the scope of the audit. The scope may be provided as part of an audit schedule or plan, or it may need to be developed by the audit program manager. List the process(es) below:Processes Document No.

Internal Audits Corrective & Preventive Action Customer Satisfaction Monitoring Analysis of Data Continual Improvement

Process Key Inputs and Resources

Based on your review, add, delete, or modify the questions on this checklist to focus your evaluation and to incorporate local requirements and inputs. Recognize that most of the requirements that must be verified will be called out in local procedures and instructions.

Consider the following information relating to the processes to be examined.

Previous audit results related to the selected processes Current process metrics relating to the selected processes Significant corrective actions or complaints relating to the

selected processes Auditee management

Other (list) _____________________________________

Corrective Actions and Current/Recent problemsList any problems, CARs, or items from previous audits that should be evaluated or verified during this audit.Problems noted for later examination:

Review the core process metrics relating monitoring, measurement and improvement.Metric Recent Performance Trend

1.

2.

3.

4.

5.Objectives and/or targets to review:

1.

2.

2003 Joe Kausek & Associates 1

Internal Audit Program

Interview the Audit Program Manager (APM). Do we have a procedure for internal audits? Ask to see the procedure. Note the numbers and revision date for later confirmation of document control.

Ask the (APM) for a copy of the audit schedule. Select several audits as test objects for the remainder of this evaluation.

Can you walk me through our process for scheduling, planning, conducting and reporting audits? Follow along in the procedure. Pay particular attention to/or ask about the following:

How is the schedule of internal audits determined? Is it dynamic? Does it reflect status and importance of the areas being reviewed?

Are audits formed around processes, as opposed to elements? Are all core processes reflected? Are supporting processes reflected? Although not a stated requirement of the standard, audits should be process based. Generate an OFI if not.

Training and AwarenessHow are auditors trained? Have they been provided with training on the standard used by the organization? Have they been trained in the process approach? Have they accomplished any internal qualification requirements? Do they audit their own work?

Review the audit procedure to identify specific roles, responsibilities and authorities of auditors. Pay attention to responsibilities for planning, reporting and recommending corrective action. Interview several auditors about these responsibilities. Are they aware of them?

Are the audits being accomplished roughly per the schedule?

ISO 9001 8.2.2

Procedure No. Rev. Date

List audit reports sampled below.

Report Process Independent Qualified No. Based? Auditors? Auditors? Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Auditor Name Evidence of Awareness of Training? Responsibilities? Yes No Yes No

Yes No Yes No

Yes No Yes No

Yes No Yes No

Yes No Yes NoNo. Audits Scheduled to Date No. Completed

Overall evaluation: Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated.

2003 Joe Kausek & Associates 2

Audit PerformanceExamine the audit reports selected as samples. Are the audits thorough? Do they consider inputs and resources into the process as well as results? Do they consider process effectiveness? Is there evidence that auditors target specific areas where there are problems or poor performance? Note your results in the right-hand column.

Are audit findings reported? Are the write-ups clear? Are they provided to area management?

Is timely follow up provided to audit findings? This is a critical verification. Review the corrective actions issued as a result of the audits. Look for overdue, outstanding corrective actions. Check no for any audits where there are outstanding, overdue corrective actions.

Are the audits effective? Compare internal audit findings to the findings of external audits (customers, registrar, etc.). Did external audits find systematic weaknesses that the internal audits did not?

ISO 9001 8.2.2

List audit reports sampled below.

Report Clear Performance Timely No. Findings? Focused? Followup?

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

Yes No Yes No Yes No

EffectivenessExternal Audit Date No. of Systematic No. of These And Significant Identified by Findings Prior Internal Audits1. ______ ______

2. ______ ______

3. ______ ______Overall evaluation:

Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated.

2003 Joe Kausek & Associates 3

Corrective & Preventive Action

Interview the Corrective Action Coordinator. Ask him or her:Do we have a procedure(s) for Corrective and Preventive Action? Ask to see the procedures. Note the numbers and revision date for later confirmation of document control.

Ask the Coordinator for the action log. Select several CARs as test objects for the remainder of this evaluation.

Can you walk me through our process for generating and following up on corrective action? Follow along in the procedure. Pay particular attention to/or ask about the following:

Review the corrective action files. Is there evidence that corrective actions are being used effectively? Are corrective actions being used to drive improvement beyond audit findings? Generate a finding if CARs are only generated by audits.

Is the corrective action process effective in preventing recurrence of problems? Review the last 6 – 12 months of corrective actions. Is there a pattern of repeated occurrence of problems (same description of problem, same root cause)?

Are corrective actions being responded to in a timely fashion? Look for overdue, outstanding corrective actions. Is there follow up or new action/dates assigned?

Are there examples of preventive actions? Is the program being used to prevent problems?

Is there evidence that error proofing is a tool in both corrective and preventive action? Note this is not a requirement for ISO 9001, but should be a goal. Consider an OFI if no evidence of error proofing is evident.

ISO 9001 8.5

Procedure No. Rev. Date

Corrective Actions Evaluated

CAR Root Cause Timely Action No. Determined? Taken? Recurrence? _________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

_________ Yes No Yes No Yes No

Examples of Preventive Action

Overall evaluation: Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated.

2003 Joe Kausek & Associates 4

Customer Satisfaction Monitoring

Ask the auditee what methods and measures are used to monitor customer perceptions of how well we meet or met their requirements. Check off those below that are collected and analyzed.

Direct measures (list) based on customer feedback (performance report cards, etc.).

Direct measures based on surveys, trailer cards and other solicitations of customer satisfaction.

Indirect measures of satisfaction (awards, recognitions).

Indirect measures of customer dissatisfaction (complaints, returned product, problem reports, ppm quality, etc).

Are the above sources of information systematically analyzed and reported?

Are these measures trended over time?

Are these measures broken out by customer and/or product?

Have any objectives or targets been assigned to customer satisfaction performance?

Review the performance based on the measures used and objectives/targets set. Is the performance satisfactory? Is it improving?

For areas where performance is not satisfactory or is trending in the wrong direction, have actions to correct the performance been identified? Are they specific? Have responsibilities and timeframes been established?

Reference ISO 9001 8.2.1

The focus of this evaluation is verification that all sources of information relating to customer satisfaction are collected and considered during analysis of data for corrective, preventive and continual improvement actions. Note that customer satisfaction metrics are also reviewed during the Management responsibility audit. This review, with the management representative or other person who owns the process, is more comprehensive.

Procedure No. __________________ Rev. _________Overall evaluation:

Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated.

Metric Performance Trend

1. Good Poor Good Poor Action Yes No

2. Good Poor Good Poor Action Yes No

3. Good Poor Good Poor Action Yes No

4. Good Poor Good Poor Action Yes No

5. Good Poor Good Poor Action Yes No

6. Good Poor Good Poor Action Yes No

7. Good Poor Good Poor Action Yes No Comments and Findings

2003 Joe Kausek & Associates 5

Analysis of Data

Ask the auditee, beyond customer satisfaction monitoring (reviewed in the last set of questions) what other sources of information do we use to tell us where we may need to take action to improve our QMS? Verify the following sources of information are used:

Product conformity (ppm, 1st pass yield, etc.)

Process performance (metrics stated in quality system procedures, diagrams, etc.)

Supplier quality performance

Supplier delivery performance

Opportunities for preventive action (e.g. FMEA, preventive action requests)

Other (list) ___________________________

Other (list) ___________________________

Is the information above analyzed and trended?

Review the trends and performance information. Are any not performing well or trending in the wrong direction?

Is there evidence of actions being taken in response to the metrics, especially for those where performance is lacking?

Is the information systematically and routinely summarized and reported to the top management team? When/how often? Note that as a minimum it should be reported during management review.

This evaluation checks to see that all of the various sources of information relating to QMS performance are systematically collected, analyzed, and used to support action.

Procedure No. __________________ Rev. ___________Overall evaluation:

Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated.

Metrics with poor performance Reported? Action?

1. Yes No Yes No

2. Yes No Yes No

3. Yes No Yes No

4. Yes No Yes No

5. Yes No Yes No

6. Yes No Yes No

Examples of proactive (i.e. preventive) action to improve the system based on analysis of data:

Comments and Findings

2003 Joe Kausek & Associates 6

Continual Improvement

This checklist must be completed by the organization, based on the continual improvement strategies and methods used in the organization. Some of these methods may include: Six Sigma Lean Value Analysis Kaizen Poka-Yoke

Evaluation items would include how projects are selected, approved, conducted, reported and documented. Especially important would be evaluation of the benefits gained and whether the gains were maintained over time.

Also important is the process used to collect and analyze data to identify improvement opportunities.

Consider a separate checklist for each strategy/method employed. Use any local procedures or instructions where they exist. If documented procedures do not exist for these programs then interview program champions to determine core requirements.

ISO 9001 8.5

The organization will be effecting continual improvement, by definition, if it complies with the other requirements of the standard. This evaluation looks at other improvement programs being used to drive performance. The auditor is seeking to verify that these programs, whatever they may be, are being used in a systematic fashion to ensure maximum results. The auditor is looking to assess:1. What is the program?2. Is it defined? Is it formalized? Is it systematic?3. Is the program being followed/used as intended?4. Are results being obtained?

Procedure No. _________________ Rev. ______

Overall evaluation: Conforms. No deficiencies identified. Opportunity for improvement (OFI). Detail below. Best Practice (BP). Detail below. Be specific. Nonconformity (NC). Provide details below. Reference the

specific requirement violated. Program Formalized? Followed? Results?

1. Yes No Yes No Yes No

2. Yes No Yes No Yes No

3. Yes No Yes No Yes No

4. Yes No Yes No Yes No

Comments and Findings

2003 Joe Kausek & Associates 7