Validation of EUCAST methods by Pathology QLD

Narelle GEORGESupervising Scientist, Microbiology

Central Laboratory, Herston Hospitals Complex

Overview

§ Background to Pathology QLD§ Organisation of Microbiological Testing

§ CLSI change to EUCAST – Validation strategy/plan§ What needs to be validated§ Automated AST methods (VItek2)§ Disc testing method

§ Methods used for validation of § Vitek2 EUCSAT AES configuration file§ EUCAST discs and EUCAST zone diameter interpretation

§ Staff training and competency assessment

Pathology QLD Laboratories

• Metropolitan Tertiary (METRO)

• Group Co-ordinating (GCL)

• District Regional

Roll out across 26/34 Labs

Metropolitan Tertiary (METRO)Full range MIC and disc testing for routine and uncommon microbes with extended range of

antimicrobial susceptibility MIC testing at Central(Automated Vitek2, Disc Diffusion, ETest MIC)

Group Co-ordinating (GCL)Full range MIC and disc testing capability for routine

clinical isolates with referral for extended testing(Automated Vitek2, Disc Diffusion, Limited ETest MIC)

District RegionalLimited testing with referral to

GCL for routine testing of a variety of different clinical isolates

(Disc Diffusion only)

District RegionalLimited testing for a variety of

different clinical isolates with referral to GCL for wider range of testing

(Automated Vitek2, Disc Diffusion)

Prelude to validation?• Decide HOW you intend to roll-out EUCAST across

multiple laboratories with both manual and automated methods Ø What is the Implementation process?Ø Who will be involved in the Implementation team

• Senior management need to decideØ How to manage§ Organisms calibrated in current CLSI based database – not

calibrated in EUCAST§ Antibiotics calibrated in current CLSI based interpretations – not

calibrated in EUCAST§ Inadequate MIC range of antimicrobials in current Vitek2 AST

panels vs range required for EUCASTØ Options

§ Retain CLSI interpretations (M45-A2, M100-S24)§ Test an alternative antibiotic or§ Do not test

What is the validation strategy?Validation Strategy • Integral part of any new method implementation

• NATA/ISO/TGA requirements (verification not validation)

• Needs to be cost effective (restricted budgets, staff and time for validation activities

• Centralise for maximum efficiency (possible with statewide standardised methods)

• Identify areas where centralised management is not applicable (i.e. staff training and competency testing) – ? standardise training and competency assessment

protocols and training records

• Identify what needs to be validated and who will be responsible

What next?Validation plan • Series of plans

– Individual validation plan for each area that requires validation

• Include the SCOPE of the validation– What is being validated and why

• Specify the QC organisms to be used• Specify the number of tests to be performed• Specify the test method/s• Document the performance criteria to be assessed• Assign the acceptance criteria

– QC within specified range and one repeat allowed to achieve 100% correlation with standard

• Who will perform the testing – Specify individual roles– Variable skill set (bench scientist, quality, clinicians)

• Set the timeframe

Management Essentials

Review the validation plan and approve

Be available to make the “tough decisions” when required

Review and approve the final Method Validation/Implementation Report

Senior Management

Director of Microbiology

Impact of Changing Method from CLSI to EUCAST

Vitek2 Users• Minimal impact to workflow• New BPs & interpretations

set within instruments• Need to standardise the

configuration of software and rollout across multiple instruments

• Less antibiotics reported• Less computerised changes

Disc Testing• New disc concentrations• New zone diameters• New media for XV/PN• Extensive update of

documentation• New computer codes for

data entry or QC (other reading templates)

• Multi-skilled regional scientists

What needs to be validated?• Vitek2

– New MIC breakpoints– MIC range for non EUCAST calibrated organism:antibiotic

combinations• Disc Testing

– New medium for Fastidious organisms§ Mueller Hinton Fastidious Agar with 5% horse blood + NAD

– New disc concentrations (lower than CLSI)§ Vancomycin 5, Cefotaxime 5

– New zone diameter interpretations§ No intermediate zones§ <= for S categories, > for R categories

– Staff competency in EUCAST test method

VALIDATION OF AUTOMATED MIC TESTING USING VITEK2

Organisms calibrated – Vitek2

CLSI EUCASTEnterobacteriaceae P P

Pseudomonas species P P

Staphylococcus aureus / CoNS P P

Enterococcus P P

Streptococcus pneumoniae P P

Vibrio cholerae / Aeromonas Burkholderia

P

Non-fermenting GNB P

95% workload

USE CLSI M45-A2 or CLSI M100-S24

Antibiotics Tested not calibrated by EUCAST

Staphylococcus aureus Nitrofurantoin(urines)

EUCAST BPs applied (MIC distributions

Pseudomonas aeruginosa Norfloxacin EUCAST statesInappropriate for Organism Use CLSI MIC interpretation

Stenotrophomonas maltophilia Ticarcillinclavulanate

EUCAST has no calibrationUse CLSI MICs

Stenotrophomonas maltophilia Ceftazidime EUCAST has no calibrationUse CLSI MICs

Acinetobacter species Amp, Aug, TCCPip-Taz, 3GC,cefepime

EUCAST states testing unreliable. Use CLSI MIC interpretation

Enterobacteriaceae Cefazolin Use CLSI MICs

Enterobacteriaceae ESBL Screen Not recommended by EUCAST

Still test report ifpositive, if neg Add comment

Problem Antimicrobial MIC ranges

AST-P612

AntimicrobialVitek 2 Panel CLSI EUCAST

CodeConcentratio

ns S R S R

Mupirocin MUP 1 n/c n/c 1 256

Rifampicin RA 0.25,0.5,2 1 4 0.06 0.5

Set as CLSI breakpoints

Problem Antimicrobial MIC ranges

AST-N256

CLSI EUCASTCode Concentrations S R S R

Cefazolin CZ 4,16,64 2 8 nil nil

Norfloxacin NOR 1,8,32 4 16 0.5 1

Concentrations not low enough

Do not report except for urine

Lots of decisions that need to made before the Vitek2 validation work now begins!

Setting the Vitek2 to EUCAST

What do you need?– New AES configuration file to interpret MIC values in

accordance with EUCAST breakpoints

Setting the Vitek2 to EUCAST – NEW AES File• CLSI and EUCAST breakpoints are “hard-wired” in Vitek2 AES

configuration• Pathology QLD has a copy of the CLSI file that is modified to use

our own “user defined breakpoints” (No Intermediate category)• Process will be

– Copy the Global European file within Vitek2§ Use the Natural Resistance file not the Phenotype file§ Using the new copy, enter current EUCAST MIC breakpoints

and CLSI interpretations for non-calibrated antibiotics to be routinely reported

– Modify this to include§ Current EUCAST MIC breakpoints§ Add CLSI breakpoints for non EUCAST calibrated

anitibioitics/organisms§ Modify any Intermediate interpretation to Resistant

(Pathology QLD requirement only)

Validation of New EUCAST File• Manual checking cannot be avoided

– Print a hard copy of the AES Breakpoint file– Assign 2 members of the validation team to check the file

against EUCAST MIC Breakpoints Standard document– Correct file and print– Sign and date checked file– Maintain this paper copy for the validation file

AND• Check Instrument application of AES file

– Check that the instrument is applying the correct EUCAST breakpoints for interpretation of MIC results

– HOW?

Range of Organisms Staphylococcus aureus MSSA Enterobacter cloacae

Staphylococcus aureus nmMRSA Proteus mirabilis

Staphylococcus aureus MRSA Providencia species

Staphylococcus epidermidis Morganella morganii

Enterococcus faecalis Serratia marcescens

Enterococcus faecium Salmonella species

Enterococcus faecium vanB Pseudomonas aeruginosa (S)

Enterococcus faecium vanA Pseudomonas aeruginosa (CRP)

Escherichia coli Burkholderia cepacia

Escherichia coli ESBL Stenotrophomonas maltophilia

Klebsiella pneumoniae Acinetobacter baumannii

Klebsiella pneumoniae ESBL Acinetobacter lwoffi

Klebsiella oxytoca Achromobacter (other non fermenters)

Range of Organism Phenotypes Staphylococcus aureus MSSA Enterobacter cloacae

Staphylococcus aureus nmMRSA Proteus mirabilis

Staphylococcus aureus MRSA Providencia species

Staphylococcus epidermidis Morganella morganii

Enterococcus faecalis Serratia marcescens

Enterococcus faecium Salmonella species (ampC)

Enterococcus faecium vanB Pseudomonas aeruginosa (S)

Enterococcus faecium vanA Pseudomonas aeruginosa (CRP)

Escherichia coli Burkholderia cepacia

Escherichia coli ESBL Stenotrophomonas maltophilia

Klebsiella pneumoniae Acinetobacter baumannii

Klebsiella pneumoniae ESBL Acinetobacter lwoffi

Klebsiella oxytoca Achromobacter (other non fermenter)

How to Manage Vitek2 File - Statewide

• Generate new EUCAST file within the Vitek2 instrument located in the Central Laboratory

• Export a copy of this file on USB drive• Forward file or USB to all Vitek2 laboratories

within Pathology QLD• Provide instructions for uploading of new file

(copy current instrument file first to act as backup)• File loaded prior to GO LIVE date• On day of changeover, simply select the new file

and save.

Vitek2 – Validation Not Required

• Routine Vitek2 Quality Control for 4 weeks– QC organisms for CLSI and EUCAST identical for the

range of organisms tested within the Vitek2 system– QC testing is based on the MIC determined NOT the

interpretation (i.e. EUCAST or CLSI)• Interface between V2 and LIS

– Validation not required if only interpreted results are transmitted (i.e. S or R)

– Verification of data transfer may be required if MIC values as well as interpreted results transmitted

– For interface verification, use QC organisms as test isolates for Dummy patient ID set up in LIS

Don’t forget

• Check your BioART rules– Particularly those using MIC values to prompt for

additional testing• EUCAST breakpoint interpretation standard

changes annually– Need to update AES configuration file regularly – Audit regularly as part of laboratory quality system

VALIDATION OF DISC DIFFUSION TESTING

What needs to be validated?• Validation required for

– New fastidious test media MHF– New (low potency) discs– Ability to apply the EUCAST

method and – New QC organism (H influenzae

NCTC 8468)

• Centralise validation for efficiency & cost effectiveness

• Provide validation data to other laboratories

• Verify inter-laboratory performance

Validation of Disc Testing – New medium• New medium MHF agar• Commercial or local manufacture• Local manufacture (Central laboratory, PQ)

– NATA Biological validation required– Multiple lots to be manufactured on different days (3 lots)– Daily testing of QC organisms on each lot (10 days)– Shelf life validations§ pH, water content (w/v), QC performance§ Weekly for 12 weeks § Each individual lot tested

• Commercial supply– Routine QC testing on different lots required– How manage this for state?

Validation of Commercial vs In House MHF

• Locally produced MHF media validated against commercially produced MHF agar

• Non-metro PQ laboratories use all commercially produced media

• Test protocols– 3 scientists set up all QC organisms on MH and MHF weekly for

5 weeks– 1 scientist continues to set up all QC organisms on MH and MHF

weekly for 4 weeks

• Data compared using excel• Acceptance parameters – 100% within published

EUCAST zone diameter range for both media types

Media validation for Enterococcus faecalis ATCC 29212 & ampicillin on In-house and commercial media

Media validation for Streptococcus pneumoniae ATCC 49616 & oxacillin on In-house and commercial media

Tips and Tricks • Local manufacture of MHA and

MHF aim to use the same MH agar base

• Haemophilus influenzae• Fuzzy zones to SXT

• Medium formulation (OXOID)• CXM zones too small

• Age of culture not medium

• Streptococcus pneumoniae• Indistinct Oxacillin zone edge

resulting in zone reading problem • Best Medium formulation (MAST)

but poor growth of Enterococcus

ALL MH AGAR BASES are NOT the SAME

Validation of New AST procedure• Be practical- validation needs to be efficient and cost effective for

the laboratory• Currently apply CLSI recommendation for validation of AST methods• Require minimum of 4 weeks testing and multiple staff• Comparison to involve both media types (MHEU and MHF agars),

new disc concentrations, new QC organismUtilising at least 5 scientists at Central to set up over 5 consecutive days for 4 weeks

• Use resource staff for inter-laboratory performance (5 scientists – 5 days)

• Capture data using Excel program allows graphical comparison of data to demonstrate– Reproducibility of testing – Staff competency– Determination of test variables (e.g SD values)for ongoing routine QC of disc tests

Test ParametersStaph aureusATCC 29213

MH EU PEN 1, FOX 30, ERY 15, DA 2, CN 10, RD 5, FD10, CIP5, C30, SXT 25

Escherichia coliATCC 25922

MH EU AMP 10, AMC 30, CN10, CTX 5, SXT 25, TOB 10, AK30, CAZ 10, CIP 5, MEM 10, TIM 85, TAZ 60, F 100

Ps aeruginosaATCC 27863

MH EU CN10, TOB 10, AK 30, TIM 85, TAZ 60, MEM 10, CAZ 10, CIP 5,

Enterococcus faecalisATCC 29212

MH-F AMP 2, VA 5, TEC 30, CN 30, F100

Strep pneumoniaeATCC 49619

MH-F OX 1, ERY 15, VA 5, C30, SXT 25, TET 30

Haem influenzaeATCC 8468

MH-F AMP2 AMC 3, CTX 5, CIP 5, SXT 25, C30

Enterococci Validation Data - OP1

8

10

12

14

16

18

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Operator

Zone

(mm

) VA5 Zone

VA5 Min

VA5 Max

VA5 Target

EUCAST method and vancomycin 5 disc validation data over 4 weeks for Operator No 1

What about Staff Training and Competency Assessment?

Training and Competency Assessment• Cannot be centralised but staff training

and competency assessment can be standardised

• Dependent on standardised methodsDisc vs Automated MIC

§ Minimal for automated AST § More intensive manual training for disc

testingCLSI vs EUCAST

§ Only minimal changes for most organisms

§ Major changes with fastidious organisms (Haemophilus influenzae)

• Dependent on skill set of scientific staff– Muliskilled scientists in regional

laboratories (disc testing)

The Process• Train “resource staff” within GCLs• Forward QC organisms and test

unknowns for hands-on training of resource staff (Metro/GCL)

• Develop training file for recordingof test results for assessment

• Photograph test plates and forward with zone diameter results to Central laboratory for assessment of technique and accuracy in application of the EUCAST methods

• Resource Staff training records signed off by Central Trainers

One Narelle can be everywhere!

Send in photographs of your plate set-up

Results for 2 Resource Staff Members Guess the problem here?

Results for 2 Resource Staff Members Guess the problem here?

WRONG MEDIUM

MHSB (CLSI)>24 HR SUBCULTURE

Pathology QLD EUCAST Training Summary• Large organisations

• Utilise resource personnel• Train the Trainer process• Resource staff to train

individual staff within labs or district (GCL)

• Training methods• “Hands on” set up (QC)• Unknown isolate

• Competency• Assess media set up• Final written exam with

85% pass mark• Resource Documents

• USB for each Resource person with lectures, documents, reference documents, interpretation tables etc

In Summary

• For large organisations, centralise validation to ensure cost effective utilisation of resources but standardised methods required

• Validation of disc diffusion testing more labour intensive due to new medium and different disc potencies

• Staff training and competency testing needs to be managed locally

• Multi-skilled Team effort required

Validation of new methods can feel like this .…..

But with a little thoughtful planning and effort, the results can be rewarding …….

Acknowledgements• Pathology QLD EUCAST Implementation Team

– Dr Sally Appleton, Haakon Bergh, Greg Flohr, Michael Caffery, Cathy Engler

• Dr Graeme Nimmo, Director of Microbiology, Pathology QLD

• Members of the Pathology QLD, Microbiology Discipline Working Party

• EUCAST Resource Staff, Microbiology, Pathology QLD – Russell Enbom, Richard Lord, Karen Griffiths, David Stranger,

Sharon Dal-Cin, Jennine Hay, Mila Golmayo• EUCAST “Helpers”

– Dr Claire Heney, Katrina Lawrence, Anna Jones, Ron Songuan and staff in Central Laboratory, Herston Hospitals Complex.