fda medical devices: auditing the gmps the objects most feared … very few situations get the same...

FDA Medical Devices: Auditing the GMPs

The Objects Most Feared …

• Very few situations get the same automatic negative reaction that the dreaded QA guy carrying a clipboard will usually receive…

• Why?

Internal Auditors

• Usually part-time: the QA Department “borrows” auditors for a few days when needed on monthly, quarterly, or annual schedule

• Can be full-time in larger organizations: generally combined with inspection or other QA/QC duties

• Sometimes outsourced: consultant is hired to perform all or some of the audits

You should say YES if asked…

• Increases your value to the company

• Adds to your resume

• Good way to get to know other areas of the company

• You will learn to see the “Big Picture”

• QA Managers get lonely

The “Big Picture”

• Sometimes we audit the Trees…

• And sometimes we audit the Forest.

• What’s the difference?

CustomerInterface

ProposalGeneration

Order Fulfillment

ResourceValidation

Outside Resources

Customers

ProjectImplementation

ProductImprovement

Delivered Products

Sales/ Contract Review

Pre-Salesefforts Design

Review

SupplyManagement

Current& FutureNeeds

Continuous Improvement

• The Three Laws of Statistical Thinking

• Plan/Do/Check/Act Cycles

Three Laws of Statistical Thinking

• All Work is comprised of interrelated processes

• All Processes contain variation

• Success is derived from understanding and controlling variation

Quality Assurance

• Everything we do in QA should be designed to further the understanding and reduce the variation in our work processes.

• Big honkin’ binders of procedures are not written to please the auditors.

• Why do we write procedures?

Procedures

Procedures are written for training and thus variation reduction in our

processes…

Why do we audit?

RFP

TE009

Auditing

Auditing is performed to uncover the

difference between what we say we do

and what we actually do…

Auditing

…for both production and management

processes.

We want Continuous Improvement in all

processes.

QA Explanation

• Say What You Do• Do What You Say• Act on the Differences

PDCA Cycles

PLAN DO

CHECKACT

DO

ACT

PLAN CHECK

DO

ACT

Set objectives and build processes that meet customer expectations

CHECK

Perform your processes

ACT

PLAN CHECK

DO

ACT

PLAN

Monitor and measure the processes against policies, objectives and procedures

DO

Take actions based on what you’ve learned about the processes

PLAN CHECK

• Why is the “Systems Approach” important?

• How would you explain the Three Laws of Statistical Thinking to someone else?

• Auditing belongs to what part of PDCA?

Review

FDA Inspection Results

Most Common GMP Deficiencies by System Most Common GMP Deficiencies by System ––Domestic Inspections for 2004/5Domestic Inspections for 2004/5

Quality

47%

Materials

6%

Facilities & Equipment

8%

Packaging and Labeling

0%

Production

11%

Laboratory

19%

Quality: 47%Laboratory: 19%Production: 11%Facilities/Equipment: 8%Materials: 6%Packaging/ Labeling: 0%

FDA-483 Citations

1 211.22(d) The responsibilities & procedures applicable to the QC unit are not in writing or fully followed.

Quality

2 211.100(b) Written production & process control procedures are not followed in the execution of production & process control functions &/or documented at the time of performance.

Production

3 211.25(a) GMP training is not conducted on a continuing basis &/or with sufficient frequency to assure employees remain familiar with cGMP requirements applicable to them.

Quality

3 211.25(a) Employees are not given training in the particular operations they perform as part of their function, cGMPs, &/or written procedures required by cGMPs.

Quality

21 CFR PrimaryReference Deficiency System

FDA-483 Citations cont.

4 211.110(a) Control procedures are not established which monitor the output &/or validate the performance of those manufacturing processes that may be responsible for causing variability in the characteristics of IP material & the drug product.

Quality

5 211.188 Batch production & control records are not prepared for each batch of drug product produced &/or do not include complete information relating to the production & control of each batch.

Production

21CFR Reference Deficiency Primary System

FDA-483 Citations cont.

6 211.160(b) Lab. controls do not include the establishment of

scientifically sound & appropriate specs., standards, sampling plans, &/or test procedures designed to assure that components, containers, closures, in-process materials, labeling &/or drug products conform to appropriate standards of identity, strength, quality & purity.

Laboratory

7 211.165(a) Testing & release of drug products for distribution do not include appropriate laboratory determination of satisfactory conformance to the final specs. &/or identity & strength of each AI prior to release.

Laboratory

8 211.100(a) There are no written procedures for production & process controls designed to assure that the drug products have the identity, strength, quality, & purity they purport or are represented to possess.

Quality

21 CFR Primary Reference Deficiency System

Before the Inspection• Are procedures in place? FDA can request copies

of high-level documents and procedures: the organization is not required to provide.

• Do the procedures meet the requirements?• Review records• Pre-notification for quality system audits when

company has a clean record, but surprise inspections are always possible

FDA Inspection/ Audit4 Major Subsystems

• Management Controls• Design Controls• Corrective & Preventive Action (CAPA)

– Medical Device Reporting

– Corrections and Removals

– Medical Device Tracking

• Production & Process Controls– Sterilization Controls

FDA Inspectional Objectives4 Major Subsystems

• Management Controls (7 Objectives)• Design Controls (15 Objectives)• Corrective & Preventive Action (10 Objectives)

– Medical Device Reporting (4 Objectives)

– Corrections and Removals (3 Objectives)

– Medical Device Tracking (3 Objectives)

• Production & Process Controls (6 Objectives)– Sterilization Controls (5 Objectives)

First Major Subsystem:Management Controls

• Interview of the Management Representative– Does management

have a good understanding of what is required under this section?

Management ControlsThe purpose of the management control subsystem is to provide adequate resources for device design, manufacturing, quality assurance, distribution, installation, and servicing activities; assure the quality system is functioning properly; monitor the quality system; and make necessary adjustments. A quality system that has been implemented effectively and is monitored to identify and address problems is more likely to produce devices that function as intended. A primary purpose of the inspection is to determine whether management with executive responsibility ensures that an adequate and effective quality system has been established (defined, documented and implemented) at the firm. Because of this, each inspection should begin and end with an evaluation of this subsystem.

ISO 13485; 5.5.2; Management Representative CFR Title 21; Part 820; Subpart B; Section 820.20

Management Responsibility

• The Code of Federal Regulations Part 820 is referred to as the Quality System Regulation (QSR)

• It is “harmonized” with ISO 13485, meaning that they cover the same requirements and share much of the same language.

• ISO, in common usage, refers to the International Organization for Standardization.

• The organization registers to the Standard, and complies with the law…

ISO 13485; 5.5.2; Management Representative CFR Title 21; Part 820; Subpart B; Section 820.20

Management Responsibility

Top Management shall appoint a Representative with R&A that includes:

• Ensuring QMS processes are established and maintained

• Reporting to Top Management on performance and improvement needs

• Ensuring the promotion of regulatory and customer requirements throughout the Organization

Inspectional Objective #1

Verify that a quality policy, management review and quality audit procedures, quality plan, and quality system procedures and instructions have been defined and documented.

4.2 The Document System4.2 The Document System

Quality Manual: Why?

Procedures: Who, What, Where, When?

Work Instructions: How?

Records: Prove It!

Quality Policy and Objectives


Verify that a quality policy and objectives have been implemented.

What is the difference between documented and implemented to an auditor?

4.2 The Document System4.2 The Document System

Quality Manual: Why?

Procedures: Who, What, Where, When?

Work Instructions: How?

Records: Prove It!

Quality Policy and Objectives

ISO 13485; 5.3; Quality Policy CFR 820.20 (a)

Top Management must establish a written policy appropriate to the organization that includes:

• a statement of commitment to comply with the regulatory requirements and to maintain the effectiveness of the quality management system and

• provides a framework for establishing and reviewing the quality objectives

• Management must communicate the policy throughout the organization and continually review the policy for its suitability

Inspectional Objective #3Review the firm's established organizational structure to confirm that it includes provisions for responsibilities, authorities and necessary resources.


Confirm that a management representative has been appointed. Evaluate the purview of the management representative.


Verify that management reviews, including a review of the suitability and effectiveness of the quality system, are being conducted

ISO 13485: 5.6.1 Management Review

At planned intervals, Top Management shall review the QMS for suitability and effectiveness, which shall include assessment of opportunities for improvement and the need for changes to the QMS including the Quality Policy and Objectives.Records shall be maintained.

Requirements are established for Review of Inputs and Outputs including…

5.6.2 Review Input

• Results of Audits• Customer Feedback• Process Performance/

Product Conformity• Preventive/Corrective

Actions• Follow ups to Action Items• Changes affecting the QMS• Recommendations for

Improvement• Changes to regulations

Planned Intervals…• Monthly Project

Reviews• Quarterly or Annual

System Reviews• Annual Procedure and

Manual Reviews• Other?What’s appropriate?

5.6.3 Review Output

• Improvements needed to maintain effectiveness of the QMS and its processes

• Improvement of product relative to the customer

• Resource Needs

8.2.2 Internal Audit

The organization shall conduct internal audits at planned intervals to determine whether the QMS…– Conforms to planned

arrangements (7.1)– Requirements of the

standard– Requirements of the

organization

8.2.2 Internal Audit

The organization shall conduct internal audits at planned intervals to determine whether the QMS…

– Is effectively implemented and maintained

8.2.2 Internal Audit: PLAN

An audit program shall be planned, taking into consideration the status and importance of the processes and areas to be audited, as well as the results of previous audits. The audit criteria, scope, frequency and methods shall be defined. Selection of auditors and conduct of audits shall ensure objectivity and impartiality of the audit process. Auditors shall not audit their own work.

ISO 8.2.2 Auditors shall not audit their own work.

CFR 820.22Quality audits shall be conducted by individuals who do not have direct responsibility for the matters being audited.

Can the Manager of Product Engineering be an internal auditor?

8.2.2 Internal AuditDO

The responsibilities and requirements for planning and conducting audits, and for reporting results and maintaining records shall be defined in a documented procedure.

The management responsible for the area being audited shall ensure that actions are taken without undue delay to eliminate detected nonconformities and their causes.

8.2.2 Internal AuditACT

Actions must be taken or planned ASAP after the audit finding or observation is reported… Corrective Action


Evaluate whether management with executive responsibility ensures that an adequate and effective quality system has been established and maintained.

Review

• Why does FDA look at the high-level documents before beginning the inspection?

• Why is the Quality Policy required?

• Explain Management Review.

Major Subsystem 2:Design Controls

• In 1990 FDA published a study showing that 44% of device failures leading to recalls (1983-1989) were due to design failures that could have been prevented.

• For software failures, that number was over 90%.

Design Controls

The purpose of the design control subsystem is to control the design process to assure that devices meet user needs, intended uses, and specified requirements. Attention to design and development planning, identifying design inputs, developing design outputs, verifying that design outputs meet design inputs, validating the design, controlling design changes, reviewing design results, transferring the design to production, and compiling a design history file help assure that resulting designs will meet user needs, intended uses and requirements.

Inspectional Objective #1&2

1. Select a single design project.

2. For the design project selected, verify that design control procedures that address the requirements of Section 820.30 of the regulation have been defined and documented.


• Review the design plan for the selected project to understand the layout of the design and development activities including assigned responsibilities and interfaces.


• Verify that the design outputs that are essential for the proper functioning of the device were identified.

• The total finished design output consists of the device, its packaging and labeling, and the device master record.

CFR 820.120: Device Labeling

• Procedures must be written

• Labels must be legible and remain attached

• Information must be validated and released by signature

• Storage and handling procedures to prevent mix-ups…


• Confirm that acceptance criteria were established prior to the performance of verification and validation activities.

• What is the difference between these terms?


• Confirm that changes were controlled including validation or where appropriate verification.

7.3.5 Design and Development VerificationVerification shall be performed with planned arrangements (7.3.1) to ensure that the design and development have met the input requirements. Records shall be maintained.

7.3.6 Design and Development ValidationValidation shall be performed with planned arrangements to ensure that the resulting product have met the requirements for the application or intended use. Validation will be performed prior to delivery. Records shall be maintained.

7.3.7 Design and Development ChangesChanges shall be identified and records maintained. Changes shall be reviewed, verified and validated, and approved. Review shall include evaluation of effects on completed work.

Review

• T/F: Design problems with software in devices are a minor concern to the FDA

• T/F: Product acceptance criteria are established by the verification and validation processes

• T/F: Validation studies are completed years after the product is on the market

Major Subsystem #3:Corrective & Preventive Action

• Requires procedure(s)• Problem Analysis• Investigation• Identify actions• Verify & Validate planned

actions• Maintain documentation changes• Notify responsible parties• Submit to management review• Document entire process

CFR 820.100

Corrective & Preventive ActionsThe purpose of the corrective and preventive action subsystem is to collect information, analyze information, identify and investigate product and quality problems, and take appropriate and effective corrective and/or preventive action to prevent their recurrence. Verifying or validating corrective and preventive actions, communicating corrective and preventive action activities to responsible people, providing relevant information for management review, and documenting these activities are essential in dealing effectively with product and quality problems, preventing their recurrence, and preventing or minimizing device failures.

Inspectional Objective #2#2. Determine if appropriate sources

of product and quality problems have been identified. Confirm that data from these sources are analyzed to identify … corrective action.

#3. Determine if sources of product and quality information that may show unfavorable trends have been identified. Confirm that data from these sources are analyzed to identify …preventive action.

Inspectional Objective #6• Determine if failure investigation

procedures are followed. Determine if the degree to which a quality problem or nonconforming product is investigated is commensurate with the significance and risk of the nonconformity. Determine if failure investigations are conducted to determine root cause (where possible). Verify that there is control for preventing distribution of nonconforming product.

CFR 820.90: Nonconforming Product

• NCP must be controlled including– Identification

– Documentation

– Evaluation

– Segregation

– Disposition

• NCP can be authorized for use provided – Documentation of the

review

– Justification for release

– Signature of person authorizing the release


• Determine if information regarding nonconforming product and quality problems and corrective and preventive actions has been properly disseminated, including dissemination for management review.

3 Subsystems to CAPA Subsystem

• Medical Device Reporting: Confirm that the firm follows their procedures and they are effective in identifying MDR reportable deaths, serious injuries and malfunctions.

• Corrections & Removals: Determine if corrections or removals of a device were initiated by the manufacturer.

• Medical Device Tracking: Determine if the firm manufactures or imports a tracked device.

Review

• What must be done with Nonconforming product?

• What is the difference between corrective and preventive actions?

• What is required to release nonconforming product?

Major Subsystem #4Production & Process Controls

• Develop, conduct, control and monitor production processes

• Environment, personnel, contamination, buildings, and equipment monitoring and changes.

• Calibration and process validation issues…

CFR 820.70

Production & Process Controls

The purpose of the production and process control subsystem is to manufacture products that meet specifications. Developing processes that are adequate to produce devices that meet specifications, validating (or fully verifying the results of) those processes, and monitoring and controlling the processes are all steps that help assure the result will be devices that meet specifications.


• Select a process for review based on: a. CAPA indicators of process problems; b. Use of the process for manufacturing higher risk devices; c. Degree of risk of the process to cause device failures; d. The firm’s lack of familiarity and experience with the process; e. Use of the process in manufacturing multiple devices; f. Variety in process technologies and profile classes; g. Processes not covered during previous inspections; h. Any other appropriate criterion as dictated by the assignment


• Verify that personnel have been appropriately qualified to implement validated processes or appropriately trained to implement processes which yield results that can be fully verified.

Sterilization Process ControlsSubsystem of Production and Process Controls

• Inspectional Objective #3: If review of the Device History Records (including process control and monitoring records, acceptance activity records, etc.) reveals that the sterilization process is outside the firm’s tolerance for operating or performance parameters: – a. Determine whether the nonconformances were

handled appropriately; and – b. Review the equipment adjustment, calibration and

maintenance

• Sampling Plan for Records

• The Audit Process– Plan

– Do

– Check

– Act

Part II: Auditing

Luck of the Auditor:Sampling

Audits are often “samples.” We can’t look at every record or inspect every device. Formal sampling plans provide statistical assurance provided that every item has equal chance of being selected and that item is chosen randomly.

Sampling Plan for Records

Assumptions:

Record not “critical” 10 % Defective is my threshold of pain 90 % Detection desirable


Simplicity Rules!

Zero Acceptance Sampling (“Fail” on single nonconformance)

Constant Sample Size Randomize by taking every nth record


# Records Sx Size Reject on(Lot Size)

50 10 1 defect

100 10 1 defect

10,000 10 1 defect


# Records Sx Size Reject on Probability(Lot Size) Of Correct

Rejection

50 10 1 defect 81%

100 10 1 defect 80%

10,000 10 1 defect 78%


# Records Sx Size Reject on Probability(Lot Size) Of Correct

Rejection

50 20 1 defect 91%

100 20 1 defect 90%

10,000 20 1 defect 86%

The Audit Process

Planning

Doing

Checking

Acting on What We’ve Learned

Planning the Audit

Audit TypeAuditor SelectionPreparationChecklists

Systems Audit

Process Audit

ProductAudit

Systems Audit

Are systems requirements

(manuals, policies, standards, regulations)

being met?

Process Audit

Are Process requirements

(inputs, outputs, procedures, plans, methods)

being met?

Product (or Service)Audit

Are attribute requirements

(specifications)

being met?

Other Common Terms

Desk Audit

First, Second, and Third-party audit

Internal, customer, and external audit

Dock Audit

Gap Analysis

Auditor SelectionYou may have a conflict of interest if:

• The procedure or process was developed by you• A close friend works in the area• You currently work for the department• There is an interpersonal problem with someone in area• You are a previous employee of the area• You have previous relationships with people in area

Remember: The only thing that the ISO standard prohibits is the auditing of your own work… and the law prohibits?

Audit Preparation

Who? What? Where? When?

Establish the Scope: Why?What are we trying to learn? Verify compliance against a spec? Or to verify that contract requirements are met? Against a standard? Train new auditors? Program assessment/ gap analysis. Close previous findings or concerns?

Are we auditing a department? A whole facility? Multiple shifts? Travel arrangements? Team? Other resources needed?

Audit Preparation

Set Up the audit

• Complete paperwork required by the organization.• Contact the auditee to arrange times, dates, and communicate

the scope. Audits are never surprise inspections. (except for the FDA…)

• Arrange for advance document transfer to you (if necessary.)• Prepare checklists for the audit.

Read the Requirement with Care

Shall, must, will…

Should, might, can, may…

An auditor should never make up rules that they think you should comply with.

4.2.3 Document ControlA procedure shall be established to ensure…

– Approval of documents prior to use– Review, update, and re-approve documents– Identification of changes and revision status– Relevant documents are available at the points of use– Documents remain legible and identifiable– External documents are identified and controlled– The prevention of the use of obsolete documents

Who does annual reviews? Who stamps old procedures “OBSOLETE” and puts in a file? Who has a document master list?

Checklists

Canned checklists

• provide a useful starting place,

• give structure to a new program,

BUT• do not provide flexibility,

• And will not necessarily reflect your needs

Performing the Audit

Audit MeetingsPhysical EvidenceOpen-ended LanguageInterview Questions

Opening Meeting

Degree of formality will vary by the company and culture:

• Introductions• Thank/greet your “host”• Review plans, scope, standards• Workout any access/security/safety and escort issues• Review audit methods/reports/schedule• Request resources such as telephone/maps/documents etc.• Confirm meeting schedules: morning/closing/exit meetings

Ongoing Meetings

Keep the auditee informed:

• Progress: what is finished• Findings/observations/problems• Changes or additional concerns• New document requests• Review audit methods, reports, or schedule

Physical EvidenceDocuments and records:

• note references on audit checklists• copy if problem detected

Physical Examination:• if you looked or counted it, record it

Observations• Pay attention to the area• be inquisitive

Interviews• talk to the people who do the process

Other Concerns

If you observe unethical or unsafe activities, record and report them…

Corroborate whenever possible…

Put the interviewees at ease…

Always explain what you are doing to the auditee…

Open-ended LanguageTimely, readily, promptly, periodic, without undue delay

The organization shall ensure control over such processes

Top management must ensure the QMS is suitable…

Exercise care with customer property…

The organization shall preserve conformity of the product…

Open-ended LanguageWe are used to thinking of these as Quality issues; but what do they mean to us as auditors?

A poor auditor will use these as “gotcha’s.”

The auditor should challenge the interviewee/auditee toexplain their interpretation:

• Have you worked out what “exercise care” means to you?• Have you defined periodic in the current context?

Interviews

Don’t ask:

“Did you receive training in this task?” “Show me the work instruction.”“Where are the criteria for this job?”“Who is responsible for adjustments to this process?”

These will be interpreted as accusatory by some people.The formulated response in their heads to all of these is,“Why, am I doing it wrong?”

Ask Instead:

“How do you know what to do?” (Tells you if there is a plan, procedure, training…)

“Show me how you do it?”(Verifies training, competency, knowledge…)

“How is it checked? How do you know it’s right?”(Reveals acceptance criteria, QC plans…

“If it’s not right, what do you do? What is next action?”(evidence of corrective actions, follow up…

CheckingAnalysisClassificationReport

Analysis

Is there enough data to confirm your observation(s)?

Is there contradictory evidence?

One time? Repetitive problem?

Is it a critical process step? What is the consequence?

Classification

Nonconformity: Minor or major violationFinding: Systemic problems Observation: Might become a problemImprovement point/opportunity: could be betterDefect: minor violationConcern: Might become a problemIssue: Might become a problem

Positive practice: Something done well

Report

SWOT analysis:

Strengths: You can show directness, objectivity…Weaknesses: You can demonstrate indecisiveness, etc… Opportunities: Communication/improvement opportunityThreats: Audits reviewed by upper management

Be direct. Cite evidence. Be willing to discuss issues. Stay within scope. A well written finding answers

Who, What, Where, When and Why

Other Issues

Do not discuss auditee information outside of the proper channels…

Be willing to follow up as necessary; your judgment went into the decisions…

Speak your truth quietly and clearly…

Don’t take ownership of the problems…

Acting on What We’ve LearnedFollow the corrective action process for your own audits: conduct a formal review, but…

Could I have done something better?Did I overlook something?Did I cause problems by stepping out of scope?

Discuss audit procedures/forms/techniques with Management Representative or QA Manager: anything that could have gone better? Something out of scope that concerned you? Additional resource or training needs?

Review

• Pay attention to the words in the standards and procedures. • Document everything that you look at. Make copies of

problematic records to back up your findings• Auditors who are “out to get ‘em” are not doing it right.• Ask questions. Be honest.

Part III

Audit

Simulation

Auditing is simplebut not easy…

The best way to learn quality system auditing is to perform it!

Since we’re not in a company with ready-made procedures to audit against, we’re going to try a little “game.”

Everyone will get a turn at the various roles…