quality control and quality assurance ppt, 5.05mb

1

HIV Drug Resistance Training

Module 11:

Quality Control and Quality Assurance

2

Topics

Quality Assurance vs. Quality Control Quality Assurance Before, During, and After

Testing Quality Control EQA

3

Objectives

Identify requirements for QC/QA. Identify types of quality controls that must

be in place and the rationale for each. Given individual lab needs, develop a plan

for setting up an internal QC program. Identify types of quality assurance

programs and their purpose. Given individual lab needs, develop a plan

for setting up an external QA program.

4

quality assurance vs. quality controlWhat is the difference between QA and QC?

5

Terms

Quality Control Quality Assurance Quality Assessment

6

Definitions

Term DefinitionQuality Control

The measures that must be included during each assay to verify that the test is working properly.

Quality Assurance

The overall program that ensures that the final results reported by the laboratory are accurate

Quality Assessment

A means to determine the quality of the results generated by the laboratory. It is usually an external evaluation of the laboratory's performance. Quality assessment is a challenge to the effectiveness of the QA and QC programs.

7

Quality Assurance vs. Quality Control

Quality Assurance Quality Control

Definition

Activities to ensure process are adequate for a system to achieve its objectives

Activities to evaluate a product or work result

Examples

Establish standard procedures for sample collection

Define criteria for acceptable samples

Analyze known QC sample to determine if a test is valid

Decide if a sample is acceptable for testing

8

Quality Control is Part of Quality Assurance

"The aim of quality control is simply to ensure that the results generated by the test are correct. However, quality assurance is concerned with much more: that the right test is carried out on the right specimen, and that the right result and right interpretation is delivered to the right person at the right time.”

Quality Assurance

9

QA or QC?

HIV-negative plasma control SOP writing Personnel records Phylogenetic analysis DBS condition inspection Instrument calibration Proficiency testing New operator training Assay validation Lot release testing

10

The Quality Assurance Cycle

• Data and Lab Management

• Safety• Customer Service

11

Discussion

What is the difference between QA and QC?

12

quality assurance before, during, and after testingWhat are some examples of errors that occur before, during, and after testing, and what can we do about them?

13

The Quality Assurance Cycle



14

Pretesting



15

Preventing and Detecting Errors Before Testing

Lab personnel training and certification Sample criteria set and used Specimen labeling, collection and transport Specimen storage and processing Check storage and room temperature Lab set up Reagent inventory and expiration dates SOPs for all related procedures Preventive maintenance and equipment

checks External Quality Assessment (proficiency

testing)

16

During Testing



17

Preventing and Detecting ErrorsDuring Testing

Conducting test according to SOPs Quality Control (QC) (in-process controls) Safety precautions Prepare reagents Correctly interpreting test results Time management and planning

18

Post-testing



19

Preventing and Detecting Errors After Testing

Re-check patient/client identifier Write legibly Clean up and dispose of contaminated

waste Complete quality control review

– Positive and negative control results– Contamination check (phylogeny)– Other QA checking (e.g. SQUAT)

Transcribing, reporting and interpreting results

Critical review of results and interpretation Record Maintenance

20

SQUAT: Sequence Quality Assessment Tool

QA tool developed for WHO by Dr. Rami Kantor and colleagues, Brown University

Open-source and freely available To be used by genotyping labs before

sending sequences to national HIVDR working groups for surveillance and monitoring activities– Labs should perform initial investigations and

take corrective actions if required Sequences can be assessed by WHO for

external quality assurance as part of ongoing accreditation process

21

SQUAT Functions

Sequence alignment HIV PR and/or RT sequence identification Mutation identification and generation of

export file for uploading to database Flagging of problematic sequences

– ambiguous bases (mixtures)– stop codons– insertions/deletions– frameshifts – genetic distance anomalies– atypical and ambiguous amino acids

22

Quality Assurance is Everywhere!



23

Find It, Then Fix It

Use a problem-solving team. Investigate root causes. Develop and implement appropriate

corrective actions. Examine effectiveness. Record all actions and findings.

24

Discussion

Why do errors occur? What are some common errors that might

occur with HIVDR Genotyping? Where is QA applied in a HIVDR testing

site? What are some steps to take before,

during, and after testing to assure the quality of results?

Describe the impact that errors will have on public health.

25

Quality Control

What do we need to do to set up a quality control program for our genotyping?

26

What is quality control (QC)?

Monitoring the test quality itself Ensuring that the assay is working

appropriately and accurate test results are reported with confidence.

Prevention Detection Correction

27

QC: Two Levels/Two Types

Levels– Testing of samples with known results to verify

the procedure is working properly– Interpreting the presence or absence of

controls. Types

– Internal quality control – External quality control

28

Internal QC

Internal QC samples with known reactivity are included with the following HIVDR test kits.– TruGene– ViroSeq

For in-house assays, prepare and qualify your own internal QC samples

Test the IQC samples as you would test patient specimens.

Include positive and negative controls in every run.

Know what to do in case of unexpected results

29

Genotyping Assay Procedures

Patient virus

RNA Extraction

cDNA synthesis (RT)

cDNA amplification (PCR)

Electrophoresis

Data Analysis (base calling)

Report (sequence, mutations, interpretation)

Sequencing reactions

Positive and negative controls

Positive control

30

Guidelines for HIVDR Genotyping Internal QC Policies

Negative controls– HIV-negative plasma, at extraction step– Water, at RT step and at PCR step

• If any negative control shows a detectable PCR product, all samples must be discarded

– Investigate to locate source of contamination Positive controls

– HIV-positive plasma, at extraction step– HIV DNA at PCR step (low copy number)

• If positive control does not produce a PCR product, investigate to determine cause; handling of products from test samples may depend on outcome

– HIV DNA at sequencing step SOPs required

31

External QC

External QC samples do not come with the test kit and are provided by an external source – Regional reference laboratory– Commercial supplier– Another laboratory that has prepared validated

quality control in-house samples Test the EQC samples as you would test

patient specimens.

32

Internal or External Control Failure

If QCs fail, repeat the run. If problems or errors occur, then corrective

actions should be taken before results distribution.

33

Planning Your Internal QC Program

Incoming samples Incoming reagents and kits Methodology or instrument comparison

34

Incoming Samples

Upon arrival, all incoming sample shipments should be evaluated for proper collection, transport and storage

35

Incoming Reagents and Kits

Evaluate all incoming reagents and kits upon arrival.

Perform lot-to-lot comparisons.– Assay previously tested patient samples or

controls on both the previous lot and the new lot and compare the results.

– When a new lot has been evaluated and ‘passed,’ label container “Ready to Use.”

36

Lot-to-lot Comparison for Critical Reagents

Lot-to-lot comparison for critical reagents:– For commercial kits:

• Perform parallel run with new vs. existing kits• Define acceptance criteria for reagent lot to lot

comparison– For home-brew assays:

• Perform parallel run with new vs. existing lots• New lot of primers, RT-PCR enzymes, sequencing

reagents Source of specimens

– Positive controls– Previously tested samples– Old proficiency panels

37

Critical Reagent Quality Control

Extraction buffers

Enzymes (RT, DNA polymerase)

Primers

Capillary matrix polymer

Reaction buffers

RNA Purification

RT-PCR

Electrophoresis

Sequencing reactions

Results

38

Methodology or Instrument Comparison

For tests that have multiple instruments available in the laboratory for use, compare instruments.

Results should be comparable. Compare and document results routinely

(depending on lab throughput).

= ?

39

Small Group Discussion

1. What do you need to do to set up a QC program for genotyping? What components should be included?

2. If a RT-PCR negative control becomes positive, what should you do?

3. How can you perform lot-to-lot comparisons in your lab ?

4. What is critical to include in a QC SOP?5. Once you are done with a lot-to-lot

comparison and obtained satisfactory results, what should you do for the next step?

40

EQA

What is EQA and why is it important?Which EQA Methods should we employ?• Proficiency Testing• On-Site Evaluation• Re-testing

41

A Systems Approach to Laboratory Quality

Organization

Stock Managemen

t

SOP, Documents & Records

Process Improvemen

t

Personnel

Occurrence Managemen

t

Specimen Managemen

t

Equipment

Data Managemen

t

Assessment

Safety & Waste

Management

Quality Control

42

External Quality Assessment (EQA):Definition and Purpose

Objective assessment of a test site’s operations and performance by an external agency or personnel

Allows comparison of performance and results among different test labs

Provides early warning for systematic problems associated with kits or operations

Provides objective evidence of testing quality

Indicates areas that need improvement Identifies training needs

43

Management Responsibilities

Determine policies for EQA (WHO, WHAT, WHEN, HOW)

Assign responsibility Establish & maintain a system and

schedule Receive EQA results and support corrective

action measures Manage corrective action efforts Monitor and maintain records Investigate deficiencies Communicate outcomes

44

Testing Personnel’s Responsibilities: Overview

Participate in the EQA program Take corrective actions Maintain EQA records Communicate outcomes to supervisors

45

EQA Methods

46

What is Proficiency Testing?

Panels of specimens are sent to multiple test labs by reference laboratory/EQA provider

Test labs perform tests and report results

Results indicate quality of personnel performance and test lab operations

Results are often compared across several testing labs

47

Proficiency Panels for HIVDR EQA

VQA (NIH-supported, Rush University, Chicago1)

TAQAS (TREAT Asia2) ENVA/QCMD3

ANRS France DigitalPT/Accutest Acrometrix Others?

1. Huang et al. J Clin Microbiol.43(8):3963-70 Aug 20052. Land et al. Journal of Virological Methods 159(2): 185-193; Aug 20093. Pandit et al. J Clin Virol. 43(4): 401-6; Dec 2008

48

What is On-site Evaluation?

Periodic site visits to perform systematic assessment of lab practices– Focuses on how the lab

monitors its operations and ensures testing quality

– Provides information for internal process improvement

49

What is On-site Evaluation? – Cont’d

Also referred to as audits, assessments, or supervisory visits

Learn “where we are” Part of every lab quality

system Measures gaps or

deficiency Collect information for:

– Planning & implementation– Monitoring– Continuous improvement

50

What is Re-testing?

The process by which a random selection of specimens are collected from the routine workload at the test lab and sent to the reference laboratory for validation

Used to detect errors

51

EQA Should Lead to Corrective Actions

52

Sample of Corrective Action Form

53

Small Group Discussion

1. What are the methods in a EQA program?2. What type of EQA is most appropriate in

our own situation?3. What is the potential of internal bias in

self-evaluation of quality?

54

How To Implement EQA

55

On-Site Evaluation Process

56

On-Site Evaluation Checklist

57

Tester Responsibilities: Ensuring a Productive Site Visit

Before Visit:– Record keeping is essential. Get organized– Confirm date of visit– Review written policies and procedures– Conduct internal assessment in preparation of

site visit During Visit

– Participate in visits – cooperate– Ask questions of site assessors

After Visit– Take corrective actions, where necessary

Remember – visits are instructive, not punitive

58

On-site Evaluation: Pre-Evaluation Preparation

Assign Responsibility– Laboratory management – Quality Manager

Use Integrated team approach Determine who will conduct on-site

evaluations– Select auditors with necessary skills:

• Attention to detail• Ability to communicate effectively• Diplomacy

– Provide appropriate training Schedule site visits

59

On-site Evaluation: Pre-Evaluation Preparation

Determine site visit frequency:– Established sites - twice per year/once per year

For efficiency, cluster evaluations geographically

Schedule in advance– Announced or unannounced visits

60

On-site Evaluation: Entrance Interview

The entrance interview sets the tone for the entire visit

Provide overview of process in terms of what will be done– Review of facility– Record review– Observation– Interview with testing staff– Use of proficiency panel– Exit interview

61

On-site Evaluation: Information Gathering

Observe physical layout of the lab Evaluate testing operations

– Specimen collection– Observation of test performance– Quality control– Inventory of kits– Record-keeping

Use quality systems checklist Conduct in an instructional, not punitive,

manner.

62

On-site Evaluation: Outcome Assessment

Evidence ofimplementation

& maintenance of quality system?

• Ensure visit is sufficient to objectively document evidence

• Expand visit or observation

• Reference deficiencies

• Conclude visit

• No deficiencies

YES NO

63

On-Site Evaluation: Exit Conference

Review findings with supervisory and testing staff

Make positive statements first – acknowledge staff cooperation and support

Address negative findings – allow test site to discuss findings and provide additional information

Provide instructions and timeframe for submitting plan for correcting problems

64

On-site Evaluation: Reporting

Include information:– Lab Name & Location– Date of Visit– Assessment Team Members– Major Findings– Recommendations for corrective actions

Submit completed checklist and report to relevant authorities

65

Minimal Package for QC/QA

WHO-accredited DR labs should be enrolled in at least one EQA program

Perform self-assessment at least once a year

Internal/external controls must be included in each assay run and the results recorded.– Trending over time can be a very useful quality

indicator Clearly define valid testing run criteria in

related SOPs

66

Discussion

What is EQA and why is it important? Which EQA Methods should we employ?

– Proficiency Testing– On-Site Evaluation– Re-testing

67

Reflection

What are our responsibilities in preventing, detecting, and correcting errors before, during, and after testing?

68

Summary

Identify requirements for QC/QA. Identify types of quality controls that must

be in place and the rationale for each. Given individual lab needs, develop a plan

for setting up an internal QC program. Identify types of quality assurance

programs and their purpose. Given individual lab needs, develop a plan

for setting up an external QA program.