practical auditing rev. 0

PRACTICAL AUDITING

Continual Improvement Program

for Internal Quality Auditors

By Sid Calayag

Agenda

• PDCA and Process Approach

• Interview/Question Technique

• Notes, Check List & Cheat Sheet

• Report Writing

• Toughest ISO 9001:2000 Clauses

• How to Audit Difficult Clauses

• How to Audit Undocumented Process

• Auditor’s Code of Ethics

2Auditor Training rev 0 022009

4

Process Based Approach

MANAGEMENT

RESPONSIBILITY

RESOURCE MANAGEMENT

MEASUREMENT, ANALYSIS &

IMPROVEMENT

PROCESSINPUT OUTPUT

C

U

S

T

O

M

E

R

P

DC

A

PRODUCT REALIZATION

Satisfaction

C

U

S

T

O

M

E

R

Requirements

THE PDCA CYCLE

PLAN

CHECK

STUDY

DO

IMPLEMENT

VERIFY

ACT

EVALUATE

VALIDATE

Interview and Question Technique

• Interview

• Reason

• Steps

• Question Techniques

• Closed-ended Question

• Open-ended question

• Show-me Question


Interview Reason

• Supplements the documented process

• Determines the actual defined process

• Principal way of obtaining information

• Allows auditee to explain work

practices

• Ascertains understanding and

commitment

Auditor Training rev 0 022009 7

Interview Steps -1

• Interview persons at their workplace

• Conduct interviews during normal

hours

• Put person at ease (lower anxiety

level)

• Explain your purpose (what you want)

• Ask about their job (question; observe)

• Verify responses (confirm

understanding)


Interview Steps - 2

• Check the facts (use other sources)

• Record evidence (notes on checklist)

• Make tentative conclusion (no secrets)

• Give opportunity to discuss other

topics

• Thank for their time and cooperation


Summary on Interview

• Remember, it is an interview, not

an interrogation!

• Investigate a claim; accept an

admission.


Questioning Techniques - 1

• Ask question and then actively listen

• Rely primarily on open-ended questions

• Avoid closed question (except to confirm)

• Ask for explanations and examples

• Rephrase your question for clarification

• Restate answer for your understanding

• Keep neutral; don't disagree or interrupt


Questioning Techniques - 2

• Ask "suppose" or "what if" questions

• Find basic flaws with simple questions

• Ask the blunt question about quality

• Nod in agreement to maintain dialog

• Use silence for expanded responses

• Observe unguarded facial expressions

• Learn from remarks of nearby people


Close and Open-ended Question

• Open-ended question can be used in

determining the actual process during

the interview.

• Close-ended question can be used to

conclude the result of the interview.


Sample

Summary on Questioning Techniques

• STOP TALKING - LISTEN

Do not ask closed questions unless to conclude item

Maintain a 20% talking : 80% listening ratio

• USE THE SIX HONEST SERVING MEN

“I keep six honest serving men

They taught me all I knew

Their names were WHAT and WHY and WHEN

And HOW and WHERE and WHO”

(The Elephant Child - Kipling)

• SHOW ME

Notes, Check List & Cheat List

• Taking Notes in an Audit

• Check List

• Cheat List


Taking Notes in an Audit

• One use of a checklist is as a repository for your

notes

• Take brief notes on what you have read,

heard, and seen

• Capture specific references

• record what people are telling you about their

practices

• Some of your notes will be used immediately

for your next line of questioning


Advantages on Using Check List

Checklists, if developed and used properly:

• Promote planning for the assigned audit

• Ensure a consistent audit approach

• Act as a sampling plan and time manager

• Serve as a memory aid and confidence builder

• Provide a repository for notes on evidence


Drawback on Using Check List

Drawback

• May result in poor coverage.

• Restrict interview questions

• May cause the auditor to use an outdated

tool if not updated according to the new

standard


Summary on Check List

• A checklist should guide auditors through the

system flow from quality policy, to objectives,

to processes, to measurements, to results, to

actions, and eventually to continual

improvement.

• Auditors should use the checklist as a planning

tool for their assignment and be willing to

pursue other areas of investigation.


Cheat Sheet

• A "cheat sheet" is a concise set of notes used for

quick reference

• Job notes may not accurately describe the tasks, in

conflict with written instructions, or unapproved by

management.

ISO 9001:2000, clause 4.2.3 states that "Documents required

by the quality management system shall be controlled." So, if

cheat sheets are needed by employees to carry out their

activities, these would be viewed as documents that must be

controlled.


Report Writing


Writing Nonconformity Reports

• Conformity,

• Conformance, or

• Compliance?


Exercise No. 1

How well do you know the new ISO 9001:2000 standard? Can you identify the

clauses for these requirements?

1. Reviewing the system at planned intervals

2. Identifying the status of product measurements

3. Maintaining process equipment

4. Handling, packaging, and storing products

5. Preventing the "recurrence" of nonconformities

6. Maintaining evidence of conformity of acceptance criteria

7. Ensuring requirements are complete and unambiguous

8. Identifying the control of outsourced processes

9. Planning for design review, verification, and validation

10. Including a quality manual in the documentation


answer

Toughest Requirements

• 4.1 General Requirements

• 5.1 Management Commitment

• 5.4.1 Quality Objectives

• 5.4.2 Quality Management System

Planning (vs. 7.1 Planning of Product

Realization)

• 7.3.1 Design and Development Planning


Toughest Requirements

• 7.5.2 Validation of Processes for

Production and Service Provision

• 8.2.1 Customer Satisfaction

• 8.5.1 Continual Improvement

• 8.5.3 Preventive Action


Most Common Audit Findings

* Internal Audit Findings

• 1. Customer satisfaction data and assessment (8.2.1 . . .

• 2. Documentation gaps (4.2.3 . . .

• 3. Continual improvement process (8.5.1 . . .

• 4. Objectives not measurable (5.4.1 . . .

• 5. Collection and analysis of data (8.4 . . .

• 6-7. Top management commitment and responsibility (5.4 . . .

• 6-7. Record keeping gaps (4.2.4 . . .

• 8-9. Competency requirements (6.2.2 . . .

• 8-9. Effective control of processes (4.1 . . .* Ref. : <http://standardsgroup.asq.org/news/psi/IMS06-2002E-Implementing_ISO_9001-BD.pdf>


http://standardsgroup.asq.org/news/psi/IMS06-2002E-Implementing_ISO_9001-BD.pdf







..\..\..\..\Downloads\iso\Survey on Implementing.pdf

Most Common Audit Findings

* Registrar Audit Findings

• 1. Documentation gaps

• 2-3. Objectives not measurable

• 2-3. Top management commitment and responsibility

• 4. Continual improvement process

• 5-6. Customer satisfaction data and assessment

• 5-6. Effective control of processes

• 7-8. Collection and analysis of data

• 7-8. Record keeping gaps

• 9. Competency requirements* Ref. : <http://standardsgroup.asq.org/news/psi/IMS06-2002E-Implementing_ISO_9001-BD.pdf>









..\..\..\..\Downloads\iso\Survey on Implementing.pdf

Comparison of Rankings


Internal Audit Findings

• 1. Customer satisfaction data and

assessment



• 4. Objectives not measurable

• 5. Collection and analysis of data

• 6-7. Top management commitment and

responsibility


• 8-9. Competency requirements


Registrar Audit Findings


• 2-3. Objectives not measurable

• 2-3. Top management commitment

and responsibility


• 5-6. Customer satisfaction data and

assessment


• 7-8. Collection and analysis of data


• 9. Competency requirements

How to Audit Difficult Clauses

4.1 General Requirements

Clause 4.1 covers the requirement for your

organization to set up a quality management system

and broadly defines the associated activities. These

activities are described in greater detail in the

remainder of the standard. And, when you audit

these other clauses, you are in essence auditing

clause 4.1.


How to Audit ISO 9001:2000, Clause 4.1

• By recognizing its linkages to the clauses in the

remainder of the standard.

• Audit those other areas well and you are in effect

auditing clause 4.1.



5.1 Management Commitment

Look for evidence on how top managers

ensure their commitment is well known

throughout the organization and records

that show how they are keeping their

promise



5.4.1 Quality Objective

Are You Setting SMART Quality Objectives?

• Is it specific?

• Is it measurable

• Is it achievable

• Is it relevant

• Is it timed?



5.4.2 Quality Management System Planning

(vs. 7.1 Planning of Product Realization)

Organizations must decide how to monitor, measure,

and analyze their processes, as well as, be ready to

implement the actions necessary to achieve planned

results and continually improve the processes. Even

outsourced processes are included in the planning.

Determine how this is done using process approach.



7.0 Product Realization

• 7.1 Planning of Product Realization


• 7.5.2 Validation of Processes for Production

and Service Provision

Determine how the quality plan is developed.



8.0 Measurement, analysis and Improvement

• 8.2.1 Customer Satisfaction

• 8.5.1 Continual Improvement

• 8.5.3 Preventive Action


Auditing Undocumented Process

Auditing a Process that is Undocumented

Refer to the guide provided to you

separately


Summary on Audit Practice

Audits examine compliance from three perspectives:

1. Documents (or definitions) that indicate the process is adequate

2. Records that show the process is implemented

(being practiced)

3. Results that prove the process is effective (objectives are met)

By using the process approach in auditing, you will be

able to gather all the evidence that you need in

auditing for any clause that you may encounter in the field.


EXERCISE No. 2


The purpose of the exercise is to provide

practice in assessing evidence in an objective

manner..


Workshop

Mock-up audit of Engineering Department

• 5.4.1 Quality Objectives


Auditor Conduct

• Be critical

• Be side-tracked

• Argue

• Swear

• Be late

• Be sarcastic

• Compare

• Pass opinions

• Apportion blame

AN AUDITOR SHOULD NOT

Code of Conduct for Auditors

Purpose

To communicate the integrity, objectivity,

confidentiality, and competence expected of

internal auditors, as well as, to provide a means

for them to pledge their commitment to these

principles.



Integrity

The integrity of internal auditors establishes trust

and provides the basis for relying on their

judgment. As an internal auditor, I pledge to:

1. Perform my work with honesty, accuracy,

fairness, and responsibility.

2. Not engage in activities that might discredit

the audit profession or my organization.



Objectivity

Internal auditors must be objective in gathering, evaluating,

and communicating information about the activities being

examined. They must make a balanced and impartial

assessment of all the relevant facts and not be unduly

influenced by their interests, or those of others, in making

judgments. As an internal auditor, I pledge to:

3. Not join in any activity or relationship that may affect my

unbiased assessment.

4. Not accept anything that may impair, or appear to impair,

my judgment.

5. Disclose all the material facts to avoid any distortion of my

audit report.



Confidentiality

Internal auditors must respect the value and ownership of the

information they receive and not disclose it without the

appropriate authority (unless obligated for legal or

professional reasons). As an internal auditor, I pledge to:

6. Be prudent in the use and protection of the information

acquired during my audit duties.

7. Not use the information for personal gain or in any way

detrimental to the organization.



Competence

Internal auditors must apply their knowledge, skills, and experience in

the performance of their assessment duties. As an internal auditor, I

pledge to:

8. Engage only in audits where I possess the needed knowledge, skills,

and experience.

9. Perform audits in accordance with the procedures and practices of

my organizations.

10. Continually improve my proficiency and the quality and value of

my audit services.

11. Assist other auditors under my supervision to develop their audit

management skills.

12. Report any complaints regarding my performance and address

them to avoid recurrence.


Addendum

• Exercise No. 3 – take home exercise

• ISO 9000 : 2005 –

• ISO 9001: 2008 –

• Sample Question for Top Management


Exercise No. 3

This is an exercise to see how

nonconformities are reported

This is a take-home exercise. The

answer shall be submitted later.


ISO 9000:2005

• It makes no changes to the basic

principles of quality management stated

in ISO 9000:2000

• It is essentially a tidying-up exercise to

ensure consistency within ISO Standards

• It will probably have little, or no real

impact on our quality system


ISO 9001:2008

• ISO 9001:2000 is due for an update in

2008.

• It is currently at Draft International

Standard (DIS) stage

• Changes to the standard are very small

• It will replace ISO 9001:2000
