fundamentals of auditing ptc

FUNDAMENTALS OF AUDITING

Carroll Sams

Reference:

FUNDAMENTALS of

QUALITY AUDITING

By B. Scott Parsowith

ContentsChapter 1 Overview of Quality Auditing 1

Chapter 2 Audit Definitions and Applications 9

Chapter 3 Phase 1--Audit Initiation and

Preparation 26

Chapter 4 Phase 2--Performance of an Audit:

On-Site Evaluation 40

Chapter 5 Phases 3 & 4--Reporting and Closure

Phases 55

Chapter 6 Establishing the Framework for an

Audit Program 64

Chapter 7 The First Seven Elements of a Quality

System 69

Chapter 8 The Second Seven Elements of a

Quality System 78

Chapter 9 The Final Seven Elements of a

Quality System 87

Chapter 10 Statistical Process Control 96

Chapter 11 Statistical Sampling for Auditing 102

Chapter 1Overview of Quality Auditing

1

Audit Overview

• Auditing is an art and a science.

• Audits are snapshots in time.

• Audits should not be confrontations.

• Auditors investigate, verify, and confirm.

• The audit should be used as a tool for continual improvement of the quality management system.

2

Auditor Knowledge

• Quality management system requirements

• Product specifications

• Process procedures

• Investigation techniques

• Sampling techniques

• Communication techniques

3

Auditor Traits

• Observant

• Honest

• Investigative

• Questioning

• Through

• Communicative

• Adaptable

• Cooperative

• Polite

• Professional

• Persistent 4

Two Types of Quality Standards

• System standards– Examples

• ANSI/ISO/ASQ Q9000 standards

• QS-9000 automotive standards• ISO 14000 environmental standards

• Product standards (specifications)– Examples

• Chemical and physical property requirements

• Dimensional tolerances

5

Review and Verify Documentation, Implementation, and Effectiveness of

• Policy manuals

• Procedure manuals

• Work instructions, drawings, etc.

• General quality management system requirements

• Individual product specifications

6

Audits Can

• Enhance internal quality management systems

• Be part of continual quality improvement efforts

• Help establish trusting relationships internally and externally

• Find problems before someone else does!

7

Audits Are Not

• Acceptance of product

• Punch lists

• Public hangings

• Fault Finding

• Ill-prepared

• Antagonistic

• Biased

• Rambling

8

Chapter 2Audit Definitions and Applications

9

What Is an Audit?• ANSI/ISO/ASQC Q10011-1-

1994: “A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives.”

• Quality Applications:– System audit– Process audit– Product audit 10

III

III

Institute of Internal Auditors

• Definition from IIA– “Internal auditing is an

independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control and governance processes.”

11

Additional Audit Terms• Compliance audit

– Compliance to standards– Review of processes– Review of data

• Operational audit– Time studies– Work studies

• Readiness review by internal management– ANSI/ISO/ASQ Q9000 system

• Management review– Processes, procedures, & controls

• Survey– Comprehensive evaluation 12

System Audit Definition

“ A documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the quality management system are appropriate and have been developed, documented, and effectively implemented in accordance and in conjunction with specified requirements.”

13

Types of Audits

• Internal audits– First-party audits

• Management system audits• Self-assessments

• External audits– Second-party audits

• Sister company audits• Supplier audits

– Third-party [extrinsic] audits• ABS Registrar audits

• OSHA Safety audits• KPMG Accounting audits

14

Types of Audits(continued)

• Survey– Comprehensive evaluation of

facilities, resources, economic stability, personnel, capability

• System audits– Verification that a quality

management system exists and is being followed

• Process audits– Verifies procedures exist, correct, &

are being followed under standard, rushed/adverse conditions

• Product audits– Verifies inspector capabilities and

correct use of specifications 15

Hierarchy of Audits

Survey

System audit

Process audit

Product audit

16

Characteristics of Audits

• Are independent of doer– Performed by a person not

responsible for production or services in area being audited

• Have measurement criteria– Specifications (quality

management system and product requirements)

– Attribute data (good/bad)

– Variables data (actual measurements)

17

Characteristics of Audits(continued)

• Are performed by competent personnel– Trained audit team members

• Are cost effective– Prevention of problems

• Are samples of the system– Isolate one small moment in time.– Represent the total population

• Must be sure the sampling procedure is correct.– Avoid making incorrect

assumptions 18

Two Types of Sample Process Errors

• Type I error (alpha error)– We assume something is

unacceptable when it is actually acceptable

• Producer’s Risk

• Type II error (beta error)– We assume something is

acceptable when it is actually unacceptable

• Consumer’s Risk

• Sample consideration– Size, amount, location, & errors

19

Use of Audits

• Continual improvement efforts– Internal and external

• Benchmarking

• Value-added suppliers

• Certification– Internal (quality management

system)

– External (suppliers)

Note: Do not use audits alone to determine acceptance of product.

20

Why Audit?

• Competition– External--We want low-cost, high-

value suppliers– Internal--We want to verify our

systems are adequate/effective

• Regulation– Imposed by customers

– Self-imposed– General quality standards or

internal quality standards

• Self-preservation to audit ourselves before someone else does– OSHA 21

Audits Improve Performance if They

• Meet management needs– Provide feedback on areas

needing improvement

• Provide a forward look in time– Can predict where process

controls or product measurements are needed

• Measure effectiveness and compliance to internal and external policy and contract requirements

22

Who Is the Customer?

• There are three customers (clients) we must satisfy on each audit.– Auditee

• Person being audited (internal or external)

– Boss or manager• Give authority to perform internal

audit

– Buyer of the service (client)• Buyer of service on external audit

23

What Managers Do• Responsible for entire system

and its various processes

• Continual process improvement– Plan– Do– Check (Study)

• audits address the “check” function of the cycle by checking and monitoring a system

– ActPlan

DoCheck

Act

24

Audit Administration

• Formal audit procedures– Initiation and preparation 25%

• Areas to be audited

• Team composition and training• Notification of auditee• Checklist development• Transportation

– On-site audit performance 50%

– Reporting and closure 25%• Audit report• Audit records• Tracking and closure

25

Chapter 3Phase 1 -- Audit Initiation and Preparation

26

Audit Initiation Phase

• Audit initiated by client– Develop audit plan

• Lead auditor (audit manager) and client (approves)

• Time allocated for audit

– Client• Determine need for audit• The purpose of audit

• The type of audit

• The objectives of audit

– Lead auditor• Scope of the audit

27

The Preparation Phase

• Items to be covered in the audit plan, which must be defined:– Objective [purpose, manuals,

procedures, work instructions]

– Scope [facility, product, process, system elements, departments]

– Resources [trained auditors, technical expertise, team size]

• Length of audit =

< 500 employees: 1-3 days

500-1000 employees: 4-5 days

– Initial contacts

– Checklists– Past history 28

Lead Auditor Responsibilities

• Prepare the audit plan– 50% of work

• Select the audit team– Based on plan

• Brief the audit team– Safety issues

– Brief auditee

• Submit a report– 2 weeks maximum

29

Desirable Auditor Characteristics

• Knowledge– Quality, production, engineering

& procurement– requirements– techniques

• Open-mindedness

• Patience

• Interest

• Tenacity

• Professional attitude/integrity

• Good listening skills30

Desirable Auditor Characteristics [continued]• Inquisitiveness

• Good verbal/written skills

• Analytical skills

• Honesty

• Ethics

• Diplomacy

• Good planning skills

• Experience

• Objectivity

• Empathy31

Auditor Musts

• Knowledge of:– Verification techniques

• Show me…

• Objective evidence

– Standards– Sampling techniques– Human nature

• Four faces of Frank

– Time management• You are in control

32

Undesirable Auditor Characteristics

• Argumentative

• Opinionated

• Lazy

• Easily influenced

• Inflexible

• Impulsive--jump to conclusions

• Gullible

• Uncommunicative

• Devious

• Poor at planning

• Unprofessional 33

Contact the Auditee• Informal contact

– Date

– Purpose– Scope

• Formal contact by letter/memo– Time and date of audit– Purpose– Scope– Activities to be audited– Interfacing organizations– Applicable quality requirements– Identification of team members– Preliminary schedule 34

Perform Desk Audit

• Review quality system documentation of auditee– Policy Manual/Procedures

• Determine what control systems are in place– What facts are available?

• What is the product?– Become familiar with product

• Review prior audits

• Investigate exemptions or waivers

• Review performance history35

Develop Checklists that Provide:

• Structure– Define before audit/use team

• Required coverage– Not designed to stifle creativity.

• A communication document– A prompt to your next question.

• A place to record data– Objective evidence is required for

anything less than satisfactory.

• Help in time management– Pace of the audit

36

Auditors--Points to Consider

• Auditors can use:– Canned checklists– Yes/no checklists [attributes]

– Rating checklists [variables]

• Things to consider when developing checklists:– What is process during normal

operations?• Rushed conditions?• When there is a problem?

37

Summary of the Preparation Phase

• Define the audit objective

• Establish the audit scope

• Allocate resources

• Contact the auditee

• Develop checklists

• Review history

• Understand the process and control systems

38

Results of the Preparation Phase

• Audit plan– auditee, purpose, scope, activities

to be audited, team, standards

• Checklist questions

• Items and reference location of each requirement

• Initial evaluation– Based on documentation received

• Desk audit• Past audit results

• Plan for action– Areas needing attention/audited 39

Chapter 4Phase 2--Performance of an Audit: On-Site Evaluation

40

The Performance Phase

• Hold opening meeting

• Conduct on-site audit

• Review controls of quality management system

• Verify that the system is working– Is it documented, implemented,

and effective?

• Share information

41

The Opening Meeting

• Must have:– Entire audit team present– Auditee’s plant manager/staff

• Requires two-way communication– Trade introductions– Restate objectives– Form initial impressions– State any areas of concern

– Settle logistics; space, lunch, etc.

• Should take no more than 30 minutes 42

The Opening Meeting [continued]

• If needed:– Allow 15 minute maximum for

auditee to present quality management system enhancements

• Distribute the checklist(s)

• Define the audit schedule

• Be on time

43

On-Site Evaluation

• Auditors can be compared to florists:– Collects evidence [flowers]

– Puts it on worksheets [baskets]– Arranges the data [flower

arrangement]– Delivers the report [delivers

flowers in baskets]

44

Causes of Quality Issues

• Lack of top management support

• Lack of organization

• Lack of training

• Lack of discipline

• Lack of resources

• Lack of time

• Lack of teamwork

• Lack of knowledge

• Lack of consistency– Look for symptoms of these

issues 45

Evaluating Information• Formal [documented] control

systems vs. Informal [understood/verbal] control systems– Does top management believe in

quality and are these beliefs communicated throughout the organization?

– Is the control system used by management?

– Is the control system adequate?• Are key variables identified?

– Is the control system working?• How is it monitored? 46

Verification Methods

• Three primary methods used:– Tracing

– Corroboration

– Sampling [discussed in Chapter 11]

A Contract Random

HorizontalVertical

47

Tracing Techniques

• Techniques of tracing– Contract -- Series of audits

performed on a contract as the contract progresses [2nd party]

– Random -- Review what is taking place at time of audit [3rd party]

– Horizontal• Backward -- From the end of

process [1st, 2nd, 3rd party]

• Forward -- From the beginning of process [1st, 2nd, 3rd party]

• Middle -- Backward or forward from some critical point in process [1st, 2nd, 3rd party]

48

Tracing Techniques [continued]

• Vertical -- Reviews management layers for elements such as quality goals and quality communications

• Departmental -- Reviews numerous quality elements within a department

• Element -- Reviews a quality element in various departments

49

Corroboration

• A statement is not a fact until it is corroborated by someone else or is verified by a document.– The facts must agree:

• Two different auditors• Two different records• Two different interviews

– Combination of the above

• Tie conclusions to what is tangible– Qualitative [some documents reviewed

did not have…]

– Quantitative [out of 50 documents reviewed, 5 were found to have…]

• Forms, records, procedures50

Sources of Data Collection• Physical evidence

– Reinspect, retest, recheck

– Documentation and records

• Sensory observations– Rusty parts

– Nonconforming parts stored with good parts

– Observation of tasks being performed

• Comparisons and relationships– Patterns and trends

• Interviews and questions– Valuable source of information

– No place for sexism or biases 51

Effective Interviews

1.Put person at ease.– Your presence is threatening.

2. Explain your purpose.– Demonstrate competence.

3. Use proper questioning techniques.– Ask open-ended questions.– Use “pregnant pause”.– Ask why five times/five w’s.

4. Analyze and verify what is said.– Verify a claim.– Write notes out loud/no secrets.

5. Explain your next step.– Conclude cordially. 52

Brief the Auditee During the Audit

• Short and to the point– Discuss verifications and

concerns– About 5-10 minutes– In each area as you progress

through the facility

53

Auditor Team Meetings

• Informal– End of each day with audit team

• Auditor sharing– Facts, conclusions, problems

• Replanning– Changes to the audit schedule

• Reporting– Preliminary audit results

• Status of the audit or areas covered– Areas completed, areas to be

examined, concerns & issues 54

Chapter 5Phases 3 and 4--Reporting and Closure Phases

55

Reporting Phase

• Informal report– Exit meeting--60 minutes

• Formal report– Written & distributed within two

weeks after the audit

“Clear and precise reports”

• Statements of noncompliance contain– Requirement– Location

– Evidence– Finding(s) 56

The Exit Meeting

• Same people as opening meeting

• Lead Auditor presents summary

• Disclaimer about sampling

• Lead auditor reads findings

• Explain follow-up and corrective action response process.

57

Findings• Observation

– Write what you were doing.• “While auditing Purchasing I found

drawings for new suppliers were not being marked on form…”

• Attribution– Restate system’s deficiencies

relating to the standard or written documentation of company.

• “This is a finding against Procedure P01-01 which requires all new drawings sent to suppliers…”

• Explanation– Explain why this is a finding

• “The drawing must be recorded to ensure supplier has latest version.”

58

Degree of Severity of Findings

• The report should order findings by their importance.– Major finding--conflict with

contractual or procedural requirements. [6 maximum]

– Minor finding--Isolated observations in conflict with contractual or procedural requirements. [6 maximum]

– Concerns--Observations that may not be part of the required quality management system, but are worthy of further consideration.

59

Content of the Formal Audit Report

• Formal audit report from audit team to client should include:– Purpose and scope of audit

– Participant’s names– Background information– Summary of the results

– Identified weaknesses in order of importance

– Date and signature of lead auditor

60

The Closure Phase

• Auditee has 30-45 days to respond with plan of action.

• Auditee’s corrective action evaluated.

• Formally close corrected items.

• Items requiring on-site verification are reaudited.

• Letter of closure issued.

• All documentation filed.

61

Retention of Audit Documents

• “Working papers” should be maintained for at least one audit cycle for follow-up audits.

• Follow the required Quality Record Procedure requirements.

62

Pitfalls to Avoid

• Inadequate planning/preparation

• Inadequate communication prior to audit

• Lack of clearly defined scope

• Inadequate knowledge of standards

• Improperly trained auditors

• Failure to reevaluate implemented corrective actions

63

Chapter 6Establishing the Framework for an Audit Program

64

Establishing the Framework for an Audit Program• Top management gives authority

to establish audit function– Policy Q17-01

– Procedure P17-01

• Audit administrator is assigned– Management Representative

• Purpose and scope established– Minimum compliance purposes– Continuous quality purpose

65

General Principles to Administering an Audit

• Objectivity– Independence– Attack problem, not people

– Avoid conflict– Verify data

• Confidentiality– Information should not be shared

by auditor from company to company

66

Audit Staff

• Auditors are selected on the basis of:– Experience

– Knowledge– Communication skills– Personal traits

– Judgement

67

Audit Records Retention

• Permanent records– Notification letter– Audit plan

– Pre-assessment questionnaire– On-site audit/evaluation– Working papers– Audit report letter– Closure letter

– Auditor training/qualification

• Nonpermanent records– Notes, supporting documents

• Audit records kept 3 years 68

Chapter 7The First Seven Elements of a Quality Management System

69

The First Seven Elements of a Quality Management System

1. Management responsibility

2. Quality system

3. Contract review

4. Design control

5. Document control

6. Purchasing

7. Customer-supplied product

70

1--Management Responsibility [checklist]

• Management Responsibility

• Quality policy

• Planning

• Responsibility, authority & communication

• Management review

• Resource management

• Infrastructure

• Work environment

• Customer satisfaction

• Improvement 71

2--Quality System [checklist]

• General requirements

• Documentation requirements

• Quality Manual

• Quality management system planning

I quality manual

II procedures

III work instructions

IV Forms, tags, labels72

3--Contract Review [checklist]

• Customer focus– Interested parties

• Customer-related processes– Determine mutually acceptable

processes

• Review of requirements related to the product

• Customer communication– Product information– Amendments– Customer complaints/feedback

73

4--Design Control [checklist]

• Design and development– Planning– Inputs

– Outputs– Review– Verification– Validation

• Control of design and development changes

74

5--Document & Data Control [checklist]

• Control of documents– Approval– Review and update

– Current revision status– Latest versions at points of use– Documents remain legible and

retrievable – External documents are controlled– Obsolete documents are identified

75

6--Purchasing [checklist]

• Purchasing– Purchasing process– Purchasing information

– Verification of purchased product

76

7--Control of Customer Supplied Product [checklist]• Customer property

– Ingredients or components– Packaging materials

– Customer material in storage– Transport to a third party– Customer intellectual property

77

Chapter 8The Second Seven Elements of a Quality Management System

78

The Second Seven Elements of a Quality Management System

8. Product identification and

traceability

9. Process control

10. Inspection and testing

11. Inspection, measuring, and

test equipment

12. Inspection and test status

13. Control of nonconforming

product

14. Corrective & preventive action79

8--Product Identification and Traceability [checklist]• Identification and traceability

– Drawings– Documents

– Bar codes– Computer– Tagging– Labeling– Handling/storage

• Segregation/quality status

• Quantities

• Records of traceability

• Shipment records 80

9--Process Control [checklist]

• Product realization– Planning of product realization

• Production and service provision– Control of production and service

provision– Does not include servicing

– Validation of special processes

• Monitoring and measurement of processes– Reaction time– Yield 81

10--Inspection and Testing [checklist]• Monitoring and measurement of

product [from receiving inspection to product delivery]– Receiving inspection and testing

• Lot sampling plans• Skip lot

• Certifications

– In-process inspection and testing• First piece setup• Process audits

– Final inspection and test• 100% inspection• Lot sampling• Product audits

82

11--Inspection, Measuring, and Test Equipment [checklist]

• Control of measuring and monitoring devices– Calibration procedures

– Calibrated at defined intervals– Identified and safeguarded– Protected from damage– Personnel trained– Maintain records– Control environment– Confirm computer software

83

12--Inspection and Test Status [checklist]

• Identification and traceability– Status of material maintained

throughout process• Paper records

• Computer records• Product tags• Bar codes• Traceability of product with releases

– Identification of responsible personnel for conformance decisions

• Records of product release

84

13--Control of Nonconforming Product [checklist]

• Identification [NCMR]– Marking, tagging, computer

• Segregation– Physical, records, computer

• Disposition– Evaluation by authorized personnel

• Accept with repair by concession• Accept without repair by concession• Rework to specifications• Return to supplier [vendor]• Regraded for alternate application

• Scrap85

14--Corrective and Preventive action [checklist]• Identification of problem

– Causes• Common or special

• Repeat or isolated• Root cause determination• Evaluation of impact

– Costs

– Performance

– Safety

– Customer satisfaction

• Implementation of corrective actions [prevent recurrence]

• Implementation of preventive actions [prevent occurrence] 86

Chapter 9The Final Seven Elements of a Quality Management System

87

The Final Seven Elements of a Quality Management System

15. Handling, storage, packaging, preservation, and delivery

16. Quality records

17. Internal quality audits

18. Training of personnel

19. Servicing

20. Statistical techniques

21. Quality cost analysis

88

15--Handling, Storage, Packaging, Preservation, and Delivery

• Handling– Prevent damage/deterioration

– Maintain identification

• Storage– Maintain identification/traceability

• Packaging– Prevent damage/maintain identification

• Preservation– Protect from deterioration

• Delivery– Protection of quality after inspection

• Feedback system 89

16--Control of Quality Records• Examples of quality records:

– Management review– Employee education, skills,

training, and experience– Production criteria– Design records– Supplier evaluations and actions– Special process validations– Customer property damage

– Calibration and verification– Internal audits– Authorizing release of product

– Nonconforming product– Corrective/preventive action 90

17--Internal Quality Audits• Scheduling

– Frequency, dates, time involved

• Objective– Type of audits

• Scope– Location, element, audit team

• Procedures– Training program, checklists

• Implementation– On-site evaluation

• Reporting– Corrective actions

• Follow-up 91

18--Training of Personnel• Training needs planned/identified

– Certifications for specific employees

• Training is provided– Competency based on qualifications

• Employee training is verified– Qualification/certification on the

basis of;• Education, training, experience, and

evaluation of skills verified

• Human factors are considered– Inspire continuous improvement

– Be knowledgeable about quality– Provide a good work environment

92

19--Servicing

• Field servicing of products or services to meet requirements– Performed to written procedures

• [P19-01]

• Liability– Warranty

• Claims/traceability

– Ability to recall• Records, designs, specifications,

process controls, testing results

• Safety– Risks

• [MSDS] 93

20--Statistical and Analytical Techniques

• Process control techniques– Flowcharts– Control charts

– Trend charts– Histograms– Pareto charts– Cause/effect diagram– Inspection plans

• Assessment tools for applications– Benchmarking– FMEA – Reliability predictions 94

21--Quality Cost Analysis• Standard cost system

– Standard cost vs actual cost

• Cost of nonconformance– Value lost to company

• Quality cost system– Prevention--Blue $

• QIP process

– Appraisal--Blue $• Inspection, calibration, testing

– Internal failure--Red $• Scrap, rework

– External failure--Red $• Returns, warranty claims

95

Chapter 10Statistical Process Control

96

UCL

LCL

X

Key Process and Product Variables

• Process map [flowchart]– Identification of:

• Quality control points

• Location of activities• Who is responsible for activities

– Comparison to external and internal customer requirements [quality plan]

– Process and inspection data feedback to control system

Start or end

Go/no go

Task

97

Variability of Processes and Products

• Causes of variability in processes– Man, machine, material, methods,

mother nature

• Evaluation of the causes of variability– Special causes– Common causes

Process

MethodMaterial

MachineMan

Mother Nature

98

Statistical Control charts and Process Stability

• Statistical limits [based on past data]– Minimum 20 samples

• Statistical control– Common cause variation

• Data points outside limits– Out of control

• Data points exhibiting patterns– Runs, trends, 7 consecutive

• Determination of special causes or process changes– Notes 99

Types of Data Collection

• Variables Control Charts– Measurement

• X-bar, R control chart– Samples are averaged, range plotted

• Individual, moving range [Xmr]– One unit measured, range plotted

• Attributes Control Charts– Count

• Nonconforming products– P charts

– NP charts

• Nonconformities per unit– C charts

– U charts 100

Signals vs Noise

• Out of control signals or patterns– Data points beyond the control

limits

– Data points in a run [7 or more points above or below central line]

– Data points in a trend [7 or more points going up or down and crossing the central line]

– Data points in a periodic configuration

– Data points stratified 101

Chapter 11Statistical Sampling for Auditing

102

Aspects of Sampling

• Sampling obtains information from a part of the total population or field

• Sampling is used in:– Accounting– Business– Industry– Quality auditing

102

Methods Used Must Suit Needs of the Audit

• Types of approaches– Estimation sampling

• Provides the auditor with an estimate of the frequency of occurrences of an event in the field and considers sampling error

• Desired sample precision is based on risk and consequences of making an error

– Inventory accuracy vs airplane engine defects

– Discovery sampling• It is used to find one nonconformance

within a given field size

– Acceptance sampling• Is based on a continuous process and is

not suggested for use in quality audits103

Sample Selection

• Establish what is to be sampled depending on the scope of the audit– Critical work instructions

– Highly stressed, high-volume, high precision areas

– Contributory factors that make the process successful

– Time allocated

104

How to Select an Unbiased Sample

• Equal chance of selection

• After process is running consistently

• Selection of predetermined number of random samples– Use of random number tables– Use of calculator

– Use of a book

105

Deck of cards to show biased sample