use of microbial risk assessment in decision-making david vose consultancy 24400 les lèches...
TRANSCRIPT
Use of Microbial Risk Assessment in Decision-Making
David Vose Consultancy
24400 Les Lèches
Dordogne
France
www.risk-modelling.com
David Vose's secretary David Vose
Slide show on: www.risk-modelling.com/firstmicrobial.htm
Note the 2 ‘l’s !
Slide 2 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
IntroductionApplying CODEX guidelines in reality
DifficultiesOther ways of thinking
Experience with microbial modellingSome survey resultsThe Dutch experienceSome US experience
Modelling challengesComparison of some complete modelsReviewing a model in context
Slide 3 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Microbial risk assessment is a scientifically-based process consisting of four steps:
1. Hazard Identification The identification of known or potential health affects associated with a particular agent;
2. Exposure Assessment The qualitative and/or quantitative evaluation of the degree of intake likely to occur;
3. Hazard Characterization The quantitative and/or qualitative evaluation of the nature of the adverse effects associated with biological, chemical and physical agents that may be present in food… For biological agents… a dose-response assessment should be performed if the data is available;
4. Risk Characterization Integration of Hazard Identification, Hazard Characterization and Exposure Assessment into an estimation of the adverse effects likely to occur in a given population, including attendant uncertainties.
Codex Alimentarius CommissionFAO/WHO (1995)
Slide 4 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
OIE experience
OIE produced guidelines for animal import risk assessments (for the management of disease spread)Now in its second editionGuidelines were offered as a way to help member (including developing) countries understand how to perform a r.a.First Ed. guidelines were used too literally, both by analysts and lawyers, and found to be often impractical or irrelevant to the risk questionLesson: keep guidelines non-specific, encourage understanding rather than prescribing a formulaic approachPopular interpretation of CODEX guidelines suffer similarly
Slide 5 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
(1) Risk analysis uses observations about what we know to make predictions about what we don’t know. Risk analysis is a fundamentally science-based process that strives to reflect the realities of Nature in order to provide useful information for decisions about managing risks. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks...
Society for Risk AnalysisPrinciples for Risk Analysis
Slide 6 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
(2) Risk analysis seeks to integrate knowledge about the fundamental physical, biological, social, cultural, and economic processes that determine human, environmental, and technological responses to a diverse set of circumstances. Because decisions about risks are usually needed when knowledge is incomplete, risk analysts rely on informed judgment and on models reflecting plausible interpretations of the realities of Nature. We do this with a commitment to assess and disclose the basis of our judgments and the uncertainties in our knowledge.
Society for Risk AnalysisPrinciples for Risk Analysis
Slide 7 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Current modelling
Microbial QRA is a developing scienceWe’re making a lot of progress, but it is still in infancy
Mostly producing ‘farm-to-fork’Models the whole system but very poorly
Not designed to model any decision question well
Often relies on poor data, surrogates, and guessesAlmost never is a decision question posed beforehandAssessors have probably over-sold QRA’s usefulnessManagers have expected too much
Slide 8 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
F2F Achilles’ HeelsVery little data available, system being modelled is hugely complex!
Uncertainty, variability, inter-individual variablilityTake too long to complete, too easy to make mistakesF2F considers only pathogen on the food source
E.g. not E.coli produced during life of animal, appearing in water, vegetables, farmers’ exposure
Predictive microbiology still unreliableBroth data doesn’t translate well to food (usually overestimate, but some data – Tamplin, USDA – shows lag period can be shorter, e.g. E.coli in ground beef, Listeria in processed hams)Models often not based on physical/biological ideas, so we don’t learnAttenuation may not be death, and ignores reactivation of bacteria
D-R models inadequateDon’t describe variability observedP(ill|dose, infected) = P(ill|infected)?Feeding trial data don’t match epi data – can hugely underestimate the risk
Little cost-benefit analysis effort madeIncluding actions affecting several risk issues
Requires enormous resources – impractical for many countries
Slide 9 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
The lessons learnt from risk analysis experiences:1. Risk management has not always been an integral part of risk
analysis so far;
2. Risk managers should be trained to understand risk assessment, and risk assessors should be trained to explain their work;
3. Available data are often of limited use for risk assessment and communication of data needs between risk assessors, food scientists and risk managers is a critical issue;
4. The risk manager questions usually require rapid results, whereas (farm-to-fork) risk assessment projects require several years to complete. Solving this conflict requires open communication;
5. Uncertainty is often large.
Dutch observations on past QRAHavelaar, Jansen (2002)
Slide 10 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Our surveyInternet based, voluntary participation, 39 valid responses
Do you consider that risk analysis has brought about the following improvement to government decision-making?
0%
10%
20%
30%
40%
50%
60%
Fairness Rationality Consistency Transparency
Very much
Much
Some
Little
Very little
Slide 11 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
What factors jeopardise the value of an assessment?
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Insufficient human resources tocomplete the assessment
Insufficient time to complete theassessment
Insufficient data to support therisk assessment
Insufficient in-house expertise inthe ares
Insufficient general scientificknowledge of the area
Always
Usually
50:50
Seldom
Never
Slide 12 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
What factors jeopardise the appropriate implementation of a risk management decisions?
0%
10%
20%
30%
40%
50%
60%
Politics Other issues take precedence Legal restrictions Insufficient resources toimplement the action
Risk assessment toocomplicated
Risk assessment not acceptedas valid
Insufficient time
Always
Usually
50:50
Seldom
Never
Slide 13 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Do decision makers:
0%
10%
20%
30%
40%
50%
60%
70%
9) Assign sufficientresources and time to
complete the riskassessment?
10) Encourage assessorsto suggest alternative
approaches to theassessment?
11) Require theassessment to be simplif iedfor ease of understanding,at the expense of technical
accuracy?
12) Make the resultsavailable only if it suits their
purposes?
13) Encourage assessorsto produce an assessmentto support a predetermined
position?
14) Involve risk assessorsin the decision-making?
15) Allow /expect the riskassessors to make the
decision?
Always
Usually
50:50
Seldom
Never
Do decision makers:
0%
10%
20%
30%
40%
50%
60%
70%
1) Understand and usethe results of a risk
assessment
2) Encourageinvolvement of
stakeholders andexternal expertise?
3) Encourageassessors to explain
w hat may and may notbe possible to achieve?
4) Put a lot of emphasison receiving commentsat the planning stage of
a risk assessment
5) Involve riskassessors in planninghow to communicatethe risk assessment
results?
6) Involve stakeholdersafter completion of the
assessment of theappropriate risk
management action totake?
7) Expect too much of arisk assessment?
8) Expect too little of arisk assessment?
Always
Usually
50:50
Seldom
Never
Slide 14 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Completion times of some farm-to-fork QRAs
Jan-96 Jan-97 Jan-98 Jan-99 Jan-00 Jan-01 Jan-02
US FSIS SE
USDA Vibrio
FDA Listeria
FSIS E. Coli
CVM Campy
Harvard BSE Final report
Draft report
Being revised
Draft report
Final report
Final report
Slide 15 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
S. enteritidis only: mean outbreak attack rates (plus 90% confidence intervals)
0%
20%
40%
60%
80%
100%
1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10
Dose
Inte
rnat
iona
l out
brea
k at
tack
rat
e
Normal; Normal food
Normal; Fatty food
Susceptible; Normal Food
Susceptible; Fatty food
All participants
Naive participants
S. typhirurium only: mean outbreak attack rates (plus 90% confidence intervals)
0%
20%
40%
60%
80%
100%
1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10
Dose
Inte
rnat
iona
l out
brea
k at
tack
rat
e
Normal; Normal food
Normal; Fatty food
Susceptible; Normal Food
Susceptible; Fatty food
All participants
Naive participants
Salmonella dose-responseEpi and feeding trial comparison
All Salmonella spp.: Mean outbreak attack rates (plus 90% confidence intervals)
0%
20%
40%
60%
80%
100%
1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10
Dose
Inte
rnat
iona
l out
brea
k at
tack
rat
e
Normal; Normal food
Normal; Fatty food
Susceptible; Normal Food
Susceptible; Fatty food
All participants
Naive participants
Review by Amir Fazil in FAO/WHO (2001)
D-R mathematical models review by Haas (2002)
Slide 16 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
“Although the goal was to make the model comprehensive, it has some important limitations. It is a static model and does not incorporate possible changes in SE over time as either host, environment or agent factor change. For many variables, data were limited or nonexistent. Some obvious sources of contamination, such as food handlers, restaurant environment, or other possible sites of contamination on or in the egg (such as the yolk), were not included. And, as complex as the model is, it still represents a simplistic view of the entire farm-to-table continuum. Finally, the model does not yet separate our uncertainty from the inherent variability of the system. Much more work is needed to address this, and all other, limitations.”
USDA-FSIS-FDA Salmonella Enteritidis
Slide 17 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
USDA-FSIS-FDA Salmonella Enteritidis
Original model impetus was to evaluate effect of refrigeration temp from laying to retail on food safety
Empirically must have little affect since it only deals with a few days in the life of an egg
No cost-benefit attached
Now being redone to focus on level of performance required for shell, and liquid egg pasteurisation
i.e. much more decision focused
Slide 18 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
FDA Listeria risk assessment
No specific decision questions attached
Attempted to look at relative importance of a large list of Listeria-carrying foods
Given the data available, perhaps the only method possible to estimate which food types contribute the greatest risk
So a good QRA application
Slide 19 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Remedies: focusing on decisionsConsider what is known about the risk problem, and data available immediately or within acceptable time frame
Use epi data as much as possibleCollect more epi data (e.g. Japan, Denmark)
Consider what analysis could be done with this knowledgei.e. a risk-based reasoned argument for evaluating particular actions
Estimate the possible magnitude of benefit for a risk actionNote that it may not be possible to evaluate all actions
Perform a cost-benefit analysis on these actionsPerform a Value of Information analysis
Determines whether it is worth collecting more data before making a decision
Consider strategy to validate whether predicted improvement occursTrain data producers to supply maximally useful data
E.g. microbiologists taken more than one cfu from a plateMore inter-agency unity
E.g. Farm (APHIS) Slaughter (FSIS) Retail (FDA)
Slide 20 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Make it as simple as possible: example
Risk: Human illness from SE in eggsA shell-egg selection system proposed that will reduce by 30% the number of contaminated eggs going to marketCurrently, 30,000 people a year suffer from SE from eggsWhat will be the reduction in cases if the new system is implemented?
Reduction in cases = 30%*30,000 = 9,000 people/year
No need for models of D-R, bacterial growth, handling, etc.
Vulnerability to assumptions smaller than from using F2F model
Slide 21 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Campylobacter Risk Management and Assessment
Dutch proposal
The main objectives of the project are to advise on the effectiveness and efficiency of measures aimed at reducing campylobacteriosis in the Dutch population. The two key questions are:
1.What are the most important routes (quantifiable?)?
2.Which (sets of) measures can be taken to reduce the exposure to Campylobacter, what is their expected efficiency and societal support?
An example of the way forwardHavelaar, Jansen 2002
Slide 22 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
The target of the assessment is not limited to estimating the possible reduction in disease incidence but to evaluate both costs and benefits of possible interventions and to access their acceptance by stakeholders.
Interventions with low social support will require more effort to uphold, which increases their costs and reduces their efficacy.
The way forward – cont.
Slide 23 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Danish Vet Service Salmonella QRA“A Bayesian Approach to Quantify the Contribution of Animal-food Sources to
Human Salmonellosis” - Hald, Vose, Koupeev (2002)
0 200 400 600 800 1,000 1,200 1,400
Unknown source
Beef
Ducks
Turkeys
Imported beef
Broilers
Imported pork
Imported poultry
Pork
Outbreak
Travel
Eggs
Estimated number of cases
97.5% percentile
Mean
2.5% percentile
Estimated number of cases of human salmonellosis in Denmark in 1999 according to source
Model ranks food sources by risk. Easily updateable with each year’s data. Bayesian update improves estimate and checks validity of assumptions.
Slide 24 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Fluoroquinolone-resistant
Campylobacter risk assessment
Section 1Campylobacter culture
confirmed cases observablein US population
Section 2Total number of
Campylobacter infections inyear in US population
Section 3Number of those with
Fluoroquinolone-resistancefrom chickens and
administeredFluoroquinolone
Section 4Number of Fluoroquinolone
resistant Campylobactercontaminated chicken
carcasses consumed in year
Section 5Using the model to manage risk.
Measuring the level of risk.Controlling the risk.
Model:
Contaminated carcasses after slaughter plant * probability = affected people
Slide 25 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Broiler house
Transport
Slaughter house Hanging Scalding
Defeathering Evisceration
Washing Chilling
Export Chicken parts Whole chickens
Chilled Frozen Import
Catering Cross contamination
Heat treatment
Retail
Consumer Cross contamination
Heat treatment
Dose response
Further Processing
Risk estimation
Slaughterhouse model
Consumer model
Example of Farm-to-Fork
model
Campylobacter in poultry
Draft report 2001
Institute of Food Safety and Toxicology
Division of Microbiological Safety
Danish Veterinay and Food Administration
Behaves the same way as CVM model if prevalence is reduced
Slide 27 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Reviewing a risk assessment
Risk assessment should be decision focusedIt is not appropriate to review a risk assessment independently from the question(s) the assessment is addressing
Eg because a point is moot if the decision is insensitive to the argument
It uses science but is not itself scientific research
So we have to go with the best we’ve got
Slide 28 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
Finally - risk assessors should gain hands-on experience to ensure their models reflect the
real world
Slide 29 David Vose Consultancy Ltd
www.risk-modelling.comMicrobial risk analysis in
decision making
References
• Haas, C.N., (2002), Conditional Dose-Response Relationships for Micro-organisms: Development and Applications. Risk Analysis 22 (3): 455-464.
• Havelaar, H. and J. Jansen, (2002), Practical Experience in the Netherlands with quantitative microbiological risk assessment and its use in food safety policy. Draft paper, RIVM, Bilthoven, The Netherlands.
• Hope, B.K., et al. , (2002), An overview of the Salmonella Enteritidis Risk Assessment for Shell Eggs and Egg Products. Risk Analysis 22 (3): 455-464.
• Joint FAO/WHO Expert Consultation on the Application of Risk Analysis to Food Standards Issues (Joint FAO/WHO, 1995).
• Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. (2001), FAO headquarters, Rome.
• Teunis, P.F.M. and A.H.Havelaar, (2001), The Beta-Poisson Dose-Response Model Is Not a Single-Hit Model. Risk Analysis 20 (4): 513-520.