how to assess your fsms (food safety management system)

Food Quality and Safety

Diagnosis of your food safety

management system

performance : output

Pieternel Luning & Willem Marcelis (WUR),

Liesbeth Jacxsens (UGent)


Topics

• Introduction

• Food Safety Management System (FSMS)

diagnostic instrument

• Food Safety Performance Indicators (SPI)

diagnose

• Example of case company X


Introduction

Safe products

Food safety assurance requirements

PRP, CODEX, GMP,HACCP, BRC, ISO22000, SQF, etc………

Company specific Food Safety Management System

Stakeholders

Government, retailers, branch organisations, sector boards, etc

Dynamic environment

Socio-demographic changes, technological developments,

emerging pathogens, global sourcing, etc


This slide illustrates that companies have to operate in a dynamic environment, and

that stakeholders (that are government, retailers, branch organisations, etc) put

assurance demands on the company’s FSMS.

They require that the set-up of the company’s FSMS is according to e.g. PRP, HACCP

guidelines and or BRC standard, and or ISO22000 standard, etc.

So the stakeholders affect the company’s FSMS set up by demanding the

implementation of certain QA guidelines/standards.

Keep in mind however, that each company has a UNIQUE FSMS which is a

translation of all the QA requirements into their own specific system


Food Safety Management System

Food Safety Management System

Stakeholder

requirements

Food safety assurance activities

to control safety management system and to provide

evidence and confidence to stakeholders about

meeting safety requirements

System requirements feedback

Assurance

on

Product

Safety

Food safety control activities

aim at keeping product properties, production

processes, and human processes between certain

acceptable tolerances

Product

safety


This slide is to show the companies that there are actually to outputs of their

FSMS.

One output is the realisation of safe products, which is the result of all actual

control activities as executed by equipment and people during daily operation

under the given contextual situation.

The other output is assurance of product safety, which means being able to give

the assurance that product as made under the company’s conditions are safe.

Control activities contribute to the first output whereas assurance (indirectly affects

the first output) but results in assurance.

So it does not mean that if a company is not/scarcely elaborating assurance

activities that products are not safe….but it induces a risk!!!


Topics

• Introduction




diagnose



FSMS diagnostic instrument :

general picture

Product

characteristics

Process

characteristics

Organisational

characteristics

Environmental

characteristics

Product

safety

feedbackSystem requirements

Food safety assurance

Food safety control

Preventive measures design

Intervention processes design

Monitoring system design

Operation control strategies

Setting system requirements

Validation

Verification

Documentation and record keeping

Assurance

on product

safety


FSMS Diagnostic Instrument :

control activities

CORE CONTROL ACTIVITIES

Preventive measures design

•Sophistication hygienic design equipment & facilities

•Specificity of sanitation program

•Extent personal hygiene requirements

•Raw material control, etc….

Intervention processes design

•Adequacy intervention equipment

•Specificity maintenance program

•….etc

Monitoring system design

•Appropriateness CCP analysis

•Adequacy analytical equipment

•…..etc

Operation control strategies

•Appropriateness and compliance to procedures

•Actual performance cooling, intervention, measuring equipment ,..etc


FSMS Diagnostic Instrument :

assurance activities

Food safety control system

CORE ASSURANCE ACTIVITIESDefining system set-up

•Sophistication translating external requirements

•Extent of systematic use of feedback information

Validation

•Sophistication validation

preventive measures

•etc

Verification

•Extent of verification of people

related performance

•Extent of verification equipment

related performance

Documentation and record-keeping


This slide shows the core assurance activities. The primary objectives of assurance

are to control the system, check its effectiveness, check whether it works in practice

as designed, and adapt the system when changes are necessary. All in order to

provide evidence and confidence to stakeholders that your system is able to comply

with the safety and assurance requirements.

An important assurance activity is defining the specific system set up for the company

(based on stakeholder requirements and internal feedback information about the

FSMS performance).

Validation is aimed ate checking effectiveness in advance, whereas verification is a

check afterwards (when to check if the system is functioning properly in practice).

Documentation and record keeping support above activities and play a major role in

providing evidence and confidence to stakeholders.


Example of grid

Indicator: Specificity cleaning and disinfection program

Mechanism

Complete, tailored

sanitation program

with appropriate

cleaning agents,

supported with

appropriate user

instructions better

prevents

contamination

positive contribution

to food safety

Level 1

•Incomplete program

equal for all

•Equipment & facilities.

•Common cleaning

agents not specific

for production system.

•Cleaning instructions

derived from

information on label

or company

experience

Level 2

•Complete programme

equal for all equipment

and facilities.

•Cleaning agents

selected based on

advices of suppliers;

•specific agents for

typical product

applications.

•Idem for instructions

Level 3

•Complete programs,

tailored different

equipment/facilities

•Cleaning agents

•specifically modified

•and tested on

effectiveness for

specific food

production system.

•Instructions based

on test results.


Criteria to differentiate FSMS

(=FSC + FSA) levelsControl activities

• Level 3 high i.e. scientifically underpinned accurate, complete, stable, predictable, tailored

• Level 2 medium, i.e. Best practice knowledge, sometimes variable, not fully predictable, generic information

• Level 1 low i.e. lack of scientific evidence, use company experience, history, variable, unknown, unpredictable, common materials, equipment

Assurance activities

• Level 3 high i.e. systematic, pro-active, scientific evidence, independent, experimental trials, well-documented

• Level 2 medium, i.e. regular, reactive, expert knowledge/opinion, internal independent, no actual testing, restricted documentation

• Level 1 low i.e. ad hoc, reactive, history/ company knowledge, dependent, no testing, no documentation


Product characteristics

Organisational characteristics

• Lack of technical workforce

• Variability in workforce composition

• Insufficiency operators competence

• Lack of commitment

• Deficiency of employee involvement

• Absence of formalisation

• Insufficiency information support

systems

Process characteristics

Environmental characteristics

Food safety assurance

system

Food safety control

systemProduct

safety

Assuranc

e on

product

safety

CONTEXTUAL FACTORS FOOD SAFETY MANAGEMENT SYSTEM OUTCOME


Criteria to differentiate context situations

• Situation 1 : not critical, not vulnerable

• Situation 2 : potentially critical, potentially vulnerable

• Situation 3 : highly critical, highly vulnerable

• Situation 1: high ability, advanced, specific

• Situation 2:constrained ability, restricted advanced, specific

• Situation 3: low ability, not advanced not specific

•Situation 1: low dependency, not vulnerable

•Situation 2: restricted dependency, potentially vulnerable

•Situation 3: highly dependent, highly vulnerable

Product/process Organisational Environmental



interpretation of results

• Addresses:

– Core control activities

– Core assurance activities

– The contextual factors of company

• Assesses

– at which level control & assurance activities are executed (0,1,2,3, simple sophisticated)

– in which risky contextual situation the FSMS has to operate (1,2,3 not demanding very demanding and riskfull)

• Aggregates

– Insights in relationships between context, activities and food safety performance


• Output of the assessment gives indication of:

– Insights in relationships between context, activities

and food safety performance

– At which level the specific control and assurance

activities of food safety control and assurance are

now

– How to go – what is needed for the higher level in

control and assurance activities

– Middle or long term improvements can be identified

• NOT:

– Judgement tool no audit is performed


interpretation of results


Interpretation results of contextual factors

• The contextual situations are represented in spiderweb diagrams

A more coloured spiderweb diagram is associated with a more demanding and risky contextual situation

A more risky contextual situation is expected to result more easily in food safety problems, which will put higher demands on the FSMS.

• To obtain an overall picture of the contextual situation, overall scores have been assigned For this purpose the mean scores for a certain set of characteristics are calculated by dividing the sum of situation numbers by the number of situation assessments for this specific set of characteristics (see the legend of a spiderweb diagram).

• The overall score has been assigned by interpreting the mean score of all contextual characteristics as:– Overall score 1: if mean score of characteristics is 1 - 1.2

– Overall score 1-2: if mean score of characteristics is 1.3-1.7

– Overall score 2: if mean score of characteristics is 1.8 - 2.2

– Overall score 2-3: if mean score of characteristics is 2.3 - 2.7

– Overall score 3: if mean score of characteristics is 2.8 - 3.0


Interpretation results of Food Safety Control and

Assurance Activities• The levels of the core safety control and assurance activities are

represented in spiderweb diagrams

The spiderweb diagrams represent the detailed levels of the core safety control activities and the core safety assurance activities

A more coloured spiderweb diagram is associated with a higher/more sophisticated level of control activities and assurance activities.

• To obtain an overall picture of the FSMS activities, overall scores have been assigned For this purpose the mean scores of levels for a certain set of activities were calculated by dividing the sum of activity levels by the number of activities of this specific set.

• The overall score has been assigned by interpreting the mean score:– Overall score 1: if mean score of activities is 0 - 1.2

– Overall score 1-2: if mean of score of activities is 1.3-1.7

– Overall score 2: if mean score of activities is 1.8 - 2.2

– Overall score 2-3: if mean score of activities is 2.3 - 2.7



Assumption behind diagnosis

• High risks products and/or processes (= higher overall score in contextual factors) put higher demands on FSMS to achieve safety requirements

• Less supporting organisational conditions and/or higher chain dependency (= higher overall score in contextual factors) result in higher impact on FSMS to achieve safety requirements

Which FSMS levels is necessary?

It depends !!


Assumption of output

Context FSMS FS

3

1

2

1

2

3

1

2

3

3=most dangerous 3=highest level 3=best performance


Topics

• Introduction




diagnose



Food safety performance indicators

• The microbiological food safety output of a companies Food Safety Management System can be measured via microbiological analysis of raw materials, intermediated products and final food products

• But the food safety output can as well be measured via Food Safety Performance Indicators (SPI)

• Indicators are measurable, objective, quantitative measures of key system elements performance

• They indicate the extent upto which a certain Food Safety Management System meets the needs and expectations towards microbiological food safety

• They can be applied for self-evaluation or for benchmarking

• They are applied in this work to measure the food safety output



For each indicator, a food safety level is attributed :

• Level 0 : not conducted, not present, not appropriate

• Level 1 : low performance of food safety problems

regarding food safety can be expected and are not

under control by the current food safety management

system

• Level 2 : medium performance of food safety problems

regarding food safety can be expected and are under

control by the current food safety management system

• Level 3 : high performance of food safety no problems

regarding food safety can be expected



• To obtain an overall picture of the food safety performance, overall scores have been assigned.

• For this purpose the mean scores of levels for a certain set of food safety performance indicators need to be calculated by dividing the sum of performance levels by the number of indicators of this specific set.

• The overall score has been assigned by interpreting the mean score:– Overall score 1: if mean score of activities is 0 - 1.2

– Overall score 1-2: if mean of score of activities is 1.3-1.7


– Overall score 2-3: if mean score of activities is 2.3 - 2.7



Assumption behind Food Safety

Performance Indicators diagnosis

• More sophisticated FSMS would be better

able to realise products with lower

contamination levels and less deviation in

contamination loads

Different FSMS level/context

situations

may result in good FS !!



Context FSMS FS

3

1

2

1

2

3

1

2

3



Topics

• Introduction




diagnose




Context FSMS FS

3

1

2

1

2

3

1

2

3



Profiles contextual situation

company XProduct and process characteristics

0

1

2

3Risk level raw materials

Risk level most criticalproduct groups

Safety contribution ofpackaging concept

Intervention steps

Level of production

processchanges

Rate of product / process

design changes

1-2 Environmental characteristics

0

1

2

3

Safety contribution in chainposition

Power in supplier

relationship

Authority in customerrelationship

Strictness

of legislativerequirements

2

Organizational characteristics

0

1

2

3Technical workforce

Variability workforcecomposition

Operators' competence

Management commitmentEmployee involvement

Formalisation

Supporting information

systems

2-3

• Overall context score:

2


Profiles of core control activities

company XPreventive measures design

0

1

2

3

Product specific measures

Hygienic design

Critical cooling facilitiesSanitation program

Personal hygienerequirements

1-2 Monitoring system design

0

1

2

3

CCP analysis

Standards and tolerancesassessment

Analytical methods

Measuring equipmentCalibration program

Sampling design

Corrective actions

2

Operation of FSCS

0

1

2

3

Appropriateness ofprocedures

Compliance to

procedures

Actual hygienicperformance of equipment

and facilities

Actual cooling capacity

Actual process

capabilityof intervention process

Actual performance of

measuring equipment

Actual analytical

equipment performance

2Intervention process design

0

1

2

3

Intervention equipment

Maintenance programIntervention methods

0

Overall score control activities 1-2


Profile of core assurance activities

company X

Food safety assurance system

0

1

2

3

Translating externalrequirements

Use of feedback

information

Validating

preventive measures

Validating

intervention systems

Validating monitoring system

Verifying people related

performance

Verifying equipment andmethods related

performance

Documentation and recordkeeping system

Overall score

assurance activities

1-2

System req.: 1-2

Validation: 1

Verification: 2-3

Documentation: 2


Assigning overall scores to mean scores

of control and assurance activity levels

• Overall score 1 if mean score of activities 0 - 1.2

• Overall score 1-2 if mean of score of activities 1.3-

1.7

• Overall score 2 if mean score of activities 1.8 - 2.2

• Overall score 2-3 if mean score of activities 2.3 - 2.7


• To calculate the overall score for FSMS one should

take all original scores as basis!!!



of contextual situations


• Overall score 1-2 if mean of score of activities 1.3-

1.7




• To calculate the overall score for context one should

take all original scores as basis!!!



company x

Overall score 2 (because average of the score is 2)



of food safety performance indicators


• Overall score 1-2 if mean of score of activities 1.3-1.7




• To calculate the overall score for FSMS one should take all original scores as basis!!!


Overall results company X

Context FSMS FS

3

1

2

1

2

3

1

2

3


1-2


Conclusion : overall results company X

• Assumption for this company : In context 2 we expect FSMS level 2 and FS level 3

• so, FS level lower than expected probably due to FSMS level being lower than 2

• With FS level 2 there is a potential risk on microbiological food safety

• When FS level perceived as unacceptable by the company, measures could be discussed in FSMS activities (towards level 2/3) and or in the context (towards level 1)

• Possible measures/interventions can be discussed from a detailed analysis of the spider-webs


Detailed discussion FSMS interventions

for company X• Score 0: Product specific measures; intervention strategies

• Score 1: Extent of personal hygiene requirements; corrective actions; appropriateness of procedures; use of feed back information and validation

• Score 2: Sophistication hygienic design, specificity sanitation program, compliance to procedures, sampling design and tolerances design (important for control raw materials)

• Resulting in discussions about:

-preventing raw material contamination by product specific measures and or raw material control (including sampling design, tolerances design of monitoring system)

-possible techological intervention strategies (because now score 0)

-personal hygiene, hygienic design, hygiene control, and cleaning and disinfection program

-use of information for feedback and corrective actions

-adequacy of procedures and instructions and compliance to procedures

-validation of preventive measures and monitoring system


Detailed discussion context interventions

for company X• Score 3: Risk level of raw materials and products; lack of

technological staff; lack of information systems; lack of formalisation/procedures; lack of management commitment; strictness of legislative requirements

• This results in discussions about:

-reducing the risk level of raw materials (i.e.. Reducing initial load and variation in initial load) in collaboration with suppliers

-redesigning their information system (to support decision making)

-the quality management system with its procedures (introduce some extent of formalisation)

-people in quality staff positions (technological staff, people responsible for FSMS activities), enhance competences/skills (e.g. by training)

-management commitment


Thanks for your attention

and

for contributing to our research

2 2 3

how to assess your fsms (food safety management system)

Business

assurance of product

food quality

safety diagnosis

specific system

fsms performance

actual control activities

system setup sophistication

companys fsms