validation of alternative methods - · pdf fileafnor microval usa aoac international official...
TRANSCRIPT
Validation of Alternative Methods Food Labs in Crystal Ball – Future Challenges in Food Analysis, 21-22 May 2015 AOAC EUROPE - NMKL - NORDVAL INTERNATIONAL SYMPOSIUM 2015
Dr Gail Betts Campden BRI, Chipping Campden, Gloucestershire, GL55 6LD, UK [email protected]
Why am I interested in Method Validation? • I work in the Microbiology Department at Campden BRI in UK
• We hold ISO 17025 accreditation for a range of microbiological tests and test for a large number of clients
• We have 2400 member companies in 75 countries who rely on us for advice. Particularly on testing.
• We help companies develop and implement new test methods • We are an expert laboratory for validation bodies such as
NordVal, MicroVal & AOAC RI (30 validations)
Validation of Alternative Methods -
• There are currently many different test methods available, – for example in a review of rapid methods we found over 70
methods available for Salmonella!
• So how does a laboratory decide which method to use…..In many cases the validation status is the most crucial factor of choice
• This talk will examine – What is method validation – Why it is important to validate – How is validation done (according to ISO 16140) – Things to look out for when doing a validation
What is method validation?
‘validation’ establishment of the performance
characteristics of a method and provision of objective evidence that the performance requirements for a specified intended use are fulfilled
EN ISO 16140-1 Microbiology of food and animal feed - Method validation Part 1: Vocabulary
How is method Validation done? ISO 16140 – part 2
Why is method validation important?
• European Food Law : Food Business Operators (FBOs) are obliged to comply with the microbiological criteria set out in Commission Regulation 2073/2005
www.fsai.ie/uploadedFiles/Consol_Reg2073_2005.pdf
CRITERIA •Food category •Microorganism •Sampling plan •Limits •Method to be used •Stage of production where criterion applies •Action to be taken if results unsatisfactory
2073/2005- Alternative methods equivalency Article 5—2073/2005 • The use of alternative methods is acceptable when the
methods are validated against the reference method in Annex 1 and if a proprietary method, certified by a third party in accordance with the protocol set out in EN/ISO 16140 or other internationally accepted similar protocols
• If the FBO wishes to use analytical methods other than those validated and certified as described in paragraph 3 the methods shall be validated according to internationally accepted protocols and their use authorised by the competent authority
Third Party Validation Systems No single validation system that satisfies users
throughout the world – although there are many ‘harmonisation talks on-going
Europe NordVal AFNOR MicroVal USA AOAC International Official Methods Program AOAC Research Institute Performance Tested Methods
Program
• ISO Standard (EN ISO 16140): Microbiology of food and animal feeding stuffs- Protocol for the validation of alternative methods
EN ISO 16140
SCOPE •This part of ISO 16140 specifies the general principle and the technical protocol for the validation of alternative, mostly proprietary, methods in the field of microbiological analysis of food, animal feed, and environmental and primary production stage samples.’
•‘Validation studies according to this standard are intended to be performed by organisations involved in method validation
ISO 16140:2013 : Quantitative Methods Method Comparison • 5 food categories for a ‘broad range of foods’ Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples Accuracy profile • 3 contamination levels per category with 2 samples at
each level with 5 replicates = 150 samples Inclusivity/Exclusivity (not applicable to total count
determinations) • 50 target strains - 30 non target strains Inter-laboratory Study • Min - 8 sets of lab results • 1 food, 3 levels, 2 replicates (min)
Food Categories (ISO 16140)
15 food categories e.g. • Meat –raw, RTE, RTRH • Poultry – raw, RTE, RTRH • Fish - raw, RTE, RTRH • Produce, fresh, processed, dried • Dairy – raw and heat processed • Chocolate/bakery/confectionary • Multi-components foods • Pet food/Animal feeds • Environmental and Primary production
ISO 16140‘new’ version : Quantitative Methods Method Comparison : Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples
Category Types Number of samples Dairy products Dairy desserts e.g. chilled custard, trifle 5
Soft cheese 5 Hard cheese e.g. cheddar 5
Fruits and vegetables Seasonings e.g. spices 5 Sprouts e.g. mung beans 5 Leafy greens e.g. parsley, lettuce 5
Raw poultry and meats Fresh chicken cuts 5 Fresh mince 5 Frozen patties 5
Ready to eat foods Ready to eat poultry e.g. turkey fillet, chicken sausage 5 Cooked fish products e.g. prawns 5 Cooked meat e.g. ham 5
Multi component foods Composite foods with raw ingredients e.g. sandwiches, pasta salads.
5
Mayonnaise based salads 5 Cooked chilled foods e.g. rice products 5
ISO 16140 ‘new’ version : Quantitative Methods
Method Comparison : Accuracy profile • 3 contamination levels per category with 2 samples at
each level with 5 replicates = 150 samples
-0,60
-0,40
-0,20
0,00
0,20
0,40
0,60
0,00 2,00 4,00 6,00Bia
s
Reference Median
Fresh produce
Bias
β-ETI
AL = +/- 0.5
Category Types Item Level Test portion
Dairy products
Dairy desserts
Chilled custard
Low:100cf/g 1-5 Batch1 1-5 Batch2
Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2
High : 10,000cfu/g 1-5 Batch1 1-5 Batch2
Fruits and vegetables
Leafy greens e.g. parsley, lettuce
Parsley
Low: 50cf/g 1-5 Batch1 1-5 Batch2
Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2
High : 50,000cfu/g 1-5 Batch1 1-5 Batch2
Raw poultry and meats
Fresh beef Fresh steak Low: 50cf/g 1-5 Batch1 1-5 Batch2
Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2
High : 50,000cfu/g 1-5 Batch1 1-5 Batch2
Ready to eat foods
Cooked fish products e.g. prawns
Fish pate Low: 50cf/g 1-5 Batch1 1-5 Batch2
Medium : 100cfu/g 1-5 Batch1 1-5 Batch2
High : 1000cfu/g 1-5 Batch1 1-5 Batch2
Multi component foods
Composite foods with raw ingredients
Pasta salad Low 500cf/g 1-5 Batch1 1-5 Batch2
Medium : 5000cfu/g 1-5 Batch1 1-5 Batch2
High : 50,000cfu/g 1-5 Batch1 1-5 Batch2
ISO 16140 : Qualitative Methods overview Stage 1: Comparison study • 5 food categories • 60 samples per category (~ 50% positive) Inclusivity/Exclusivity • 50 target organisms
(100 for Salmonella) • 30 non target organisms • specificity, sensitivity and relative accuracy Relative Level of Detection Stage 2: Inter-laboratory Study • 10 sets of lab results • 1 food, 3 levels, 8 replicates
Responses Reference Method Positive (R+)
Reference Method Negative (R-)
Alternative method positive (A+)
+/+ positive agreement (PA)
-/+ positive deviation (PD)
(R-/A+) Alternative method negative (A-)
+/- negative deviation (ND)
(A-/R+)
-/- negative agreement (NA)
NordVal AOAC RI MicroVal AFNOR Technical rules
√ √
√ √
Based on ISO 16140
√ √ √ √
Food Claims Broad range Specific foods
Broad range
Broad range
ILS √ × √ √ Protocol written by
Expert lab Technical consultants
Expert lab Expert lab
Acceptance criteria
In NordVal protocol
Not defined ISO 16140:2013
Not defined
Certificates renewed
2 years 1 year 4 years 4 years
Be aware that
• Validation of water methods is not covered in ISO 16140
• Separate standard covers waters – ISO 17994
• Different experimental design and different statistics.
Compact Dry ETC for water Campden BRI & NordVal 2014
• Combined experimental design and statistics of ISO 16140 and ISO 17994
• (i) Campden BRI as the expert lab did typical 5 products x 5 replicates x 5 contamination levels
• (ii) 8 collaborative labs tested 3 levels x 2 replicates • ISO 16140 statistics was done on linearity and
accuracy data from (i) • ISO 17994 statistics was done on whole data set
from all the labs
Compact Dry ETC for water
Validation of confirmation methods
• Is currently not covered in ISO16140 but part 6 will cover this
• ISO16140-6 'Microbiology of the food chain - Method validation - Part 6: Protocol for the validation of alternative methods for microbiological confirmation and typing'
• AOAC : Appendix J – does include Confirmatory Identification methods
Things to look out for – expert laboratory
• Even though all accreditation schemes are based on ISO 16140- each one has its own technical protocol so make sure you get the most up to date copy before you start writing a protocol
• Manage the expectations of the kit manufacturer if they are not familiar with the validation time-frame. It takes on average 12 months for a validation to be completed
• If doing a joint validation, start discussions with both schemes as early as possible to design a single acceptable protocol
Things to look out for : kit manufacturer • Make sure the kit is finalised before the validation
starts : do not use a validation as a method development exercise
• Do not change test method or kit instructions during a validation. This could cause delays or extra costs or could invalidate the study
• Be familiar with the scheme rules as these will highlight the manufacturers responsibility
• If existing validation was done under ISO16140 :2004, then you may need to do further studies when it is due for renewal to meet ISO16140 :2013
Things to look out for : end user • If testing under legislation and wanting to use a proprietary
method – Don’t just look at a Third party website and use any ‘validated’ for
the organism of interest.
• Check the validation certificate and report and ensure that the scope of the validation covers both the food types and organisms you wish to test for. – Issues- examples
• There is at least one method in a certification bodies list of validated methods that is noted not to detect non-motile Salmonella– but is validated and certified to detect motile Salmonella.
• Always check the food groups you wish to test have been included as a part of the validation – ‘all foods’ or specific matrix claim
• Always verify a validated methods use within your laboratory
Final thoughts........Importance of Validation to an end-user
“An official validation based on ISO 16140…is a must for our laboratories…. the food categories are very important for us…therefore we work with suppliers to validate our matrices…As for inclusivity test, because of some historical data, we are interested to have some specific strains on the list”
Further information on third party schemes:
Please get in touch.... [email protected]
NordVal : www.nmkl.org
MicroVal: www.microval.org
AFNOR: www.afnor-validation.com
AOAC: www.aoac.org