via confidential reporting systems rae soc hf group emsg conference, 10 th october 2007 dr mike...
TRANSCRIPT
Identifying Risk via via Confidential Reporting Confidential Reporting
SystemsSystems
RAe Soc HF Group EMSG RAe Soc HF Group EMSG Conference, 10Conference, 10thth October 2007 October 2007
Dr Mike Rejman Dr Mike Rejman Director, CIRASDirector, CIRAS
Recent Career 1988 - present Recent Career 1988 - present
Head of Human Factors Unit, UK Army Air Corps Head of Human Factors Unit, UK Army Air Corps (accident investigator, set up their confidential (accident investigator, set up their confidential reporting system) reporting system)
Principal Consultant, DERA, QinetiQ,Principal Consultant, DERA, QinetiQ,
Assistant Director of Patient Safety at National Assistant Director of Patient Safety at National Patient Safety Agency (accident investigation Patient Safety Agency (accident investigation training, reporting system, risk assessment) training, reporting system, risk assessment)
Director of CIRAS, the confidential reporting Director of CIRAS, the confidential reporting system for the UK railwayssystem for the UK railways
Understanding the ProblemUnderstanding the Problem• ~ 80% of accidents are attributable to human factors 80% of accidents are attributable to human factors
issues, at the individual level, the organisational level, issues, at the individual level, the organisational level, or more commonly bothor more commonly both
• This is probably a conservative figure, and is This is probably a conservative figure, and is irrespective of domainirrespective of domain
• To manage this we need to identify and understand the To manage this we need to identify and understand the risksrisks
• Without this understanding we can’t put appropriate Without this understanding we can’t put appropriate remedial action in placeremedial action in place
Reason’s Swiss Cheese ModelOrganisation and processes- Deficiencies “Latent failures” - precursors, pre-
existing enabling conditions
“Active failures” (skill, rule and knowledge errors)
Multiple Defences Accident or incident
The Accident Iceberg (Heinrich, Bird, and others)
1
?
?
?
accidentsaccidents
serious incidentsserious incidents
incidents incidents
near misses near misses
& concerns& concerns
often often unreportedunreported
Prior Indicators of RiskPrior Indicators of Risk
• Herald of Free Enterprise, 1987Herald of Free Enterprise, 1987
– Numerous accounts of previous sailings with the Numerous accounts of previous sailings with the bow doors openbow doors open
– Economic pressuresEconomic pressures to spend to spend
less turnaround time in port less turnaround time in port
Prior Indicators of RiskPrior Indicators of Risk
• Challenger Space ShuttleChallenger Space Shuttle
– Ample previous evidence of seals shrinking Ample previous evidence of seals shrinking in cold temperaturesin cold temperatures
– Voiced by some engineersVoiced by some engineers
– Political pressure to launchPolitical pressure to launch
Prior Indicators of RiskPrior Indicators of Risk
• Kings Cross Fire, 1987Kings Cross Fire, 1987
– Numerous fires, bundles of rubbish and Numerous fires, bundles of rubbish and wooden escalators with greased tracks, some wooden escalators with greased tracks, some records but nothing done records but nothing done
– Complacency and no system to evaluate risk Complacency and no system to evaluate risk
Prior Indicators of RiskPrior Indicators of Risk
• Hillsborough, 1989Hillsborough, 1989
– Two Police Officers on horseback prevented Two Police Officers on horseback prevented similar incident the previous year by blocking similar incident the previous year by blocking access to the same terrace areaaccess to the same terrace area
– No organisational memoryNo organisational memory
Investigating Accidents with the Army Air Investigating Accidents with the Army Air CorpsCorps
• Experience while conducting investigations Experience while conducting investigations indicated that the workforce held lots of indicated that the workforce held lots of important information that could have been important information that could have been used to prevent the accidentused to prevent the accident
• The culture didn’t encourage reporting and The culture didn’t encourage reporting and there was no ‘safe’ route for this information there was no ‘safe’ route for this information to be passed on and assessedto be passed on and assessed
AAC views on confidential reportingAAC views on confidential reporting
• (Some) Senior Command (Some) Senior Command – Blame & punishmentBlame & punishment– Would subvert the chain of commandWould subvert the chain of command
• Others Others – System requiredSystem required– RAF-run Condor system inappropriateRAF-run Condor system inappropriate
AAC maintenance issues AAC maintenance issues
• Gazelle maintenance team changeGazelle maintenance team change– habit, slip habit, slip
• Lynx tail rotor gear box problem and Lynx tail rotor gear box problem and cannibalisationcannibalisation– culture, knowledge error, perceptual problem, culture, knowledge error, perceptual problem,
assumptions (some of these issues could have assumptions (some of these issues could have emerged via a confidential reporting system)emerged via a confidential reporting system)
The situation in the NHSThe situation in the NHS
Prof Sir Liam Donaldson, CMO EnglandProf Sir Liam Donaldson, CMO England
• ‘‘Organisation with a Memory’ (2000)Organisation with a Memory’ (2000)
• National Patient Safety Agency formed (2001)National Patient Safety Agency formed (2001)
• NHS Staff views – e.g. Primary CareNHS Staff views – e.g. Primary Care– PharmacistsPharmacists– GPsGPs– DentistsDentists
Patient safety risk
0
2
4
6
8
10
12
14
16
18
% of acute admissions
Australia 16.6%
England 10.8%
Denmark 9%
New Zealand 12.9%
Canada 7.5%
Japan 11%
France 14.5%
The situation in the NHSThe situation in the NHS
• The NHS is a The NHS is a labour-intensive industry labour-intensive industry – leading to a large number of human leading to a large number of human
interactionsinteractions– increasing the risks of decision-making errorsincreasing the risks of decision-making errors– and increasing the risk of communication and increasing the risk of communication
errors errors
The national reporting and learning system The national reporting and learning system
• Positive:- the system now has over Positive:- the system now has over well over one million reports well over one million reports
• Negative:- however the quality of Negative:- however the quality of much of the data is not high much of the data is not high
• The system was originally set up to The system was originally set up to be anonymous, rather than be anonymous, rather than confidential confidential
NHS exampleNHS example
• ~ I30 types of infusion pump available ~ I30 types of infusion pump available • Study revealed that 47 different types Study revealed that 47 different types
were present in one Trustwere present in one Trust• And 6 different types were found on And 6 different types were found on
one wardone ward• Leading to high level of performance Leading to high level of performance
errors errors
NHS examples NHS examples
• Removal of the wrong kidneyRemoval of the wrong kidney
• Chemotherapy drug vincristine Chemotherapy drug vincristine delivered intrathecally instead of delivered intrathecally instead of intravenouslyintravenously
Rail Confidential Incident and Analysis Rail Confidential Incident and Analysis System System
• Pilot study in Scotland 1996Pilot study in Scotland 1996• Rolled out nationally after Rolled out nationally after
Ladbroke Grove accident and Ladbroke Grove accident and Cullen Inquiry 2000Cullen Inquiry 2000
• 3 regional offices run by 3 regional offices run by contractors, now one in-house contractors, now one in-house team in Londonteam in London
• In 10 years of operation there In 10 years of operation there has never been a breach of the has never been a breach of the confidentiality guarantee for confidentiality guarantee for reportersreporters
UK Rail UK Rail
• Is one of the safest Is one of the safest forms of transport forms of transport both for travellers both for travellers and staffand staff
• But it could be even But it could be even safer as the safer as the accident in Cumbria accident in Cumbria in February showedin February showed
Forms of non-compliance across all sectors (2004 - 2006)
51.0%
31.4%
17.6%
Intentional rule violation
Best practice ignored
Rule violation
CIRAS reports on non-compliance CIRAS reports on non-compliance with rules with rules
Non-compliance by sectorNon-compliance by sector Forms of non-compliance by sector (2004 - 2006)
0
1
2
3
4
5
6
7
8
9
10
NR Subcontractor TOC LUL
Sector
No
. o
f re
po
rts
Intentional rule violation
Rule violation
Best practice ignored
Causal factors for each sectorCausal factors for each sector
SectorSector Cause 1 Cause 1 Cause 2Cause 2 Cause 3Cause 3
Network RailNetwork Rail Performance Performance before safetybefore safety
46.7%46.7% Poor Poor planningplanning
20.0%20.0% Poor Poor managementmanagement
13.3%13.3%
SubcontractSubcontract- or to NR- or to NR
Inadequate Inadequate trainingtraining
33.3%33.3% Staff Staff shortagesshortages
22.2%22.2% Performance Performance before safetybefore safety
22.2%22.2%
TOCTOC Performance Performance before safetybefore safety
28.0%28.0% OtherOther 22.2%22.2% Inadequate Inadequate trainingtraining
16.7%16.7%
LULLUL Performance Performance before safetybefore safety
55.6%55.6% Staff Staff shortagesshortages
22.2%22.2% Poor Poor managementmanagement
22.2%22.2%
27
Quotes from reportersQuotes from reporters
“ We’ve refused to do the job on the grounds of safety … …. and we’ve been threatened with disciplinary action...”
“I’ve been told that if I didn’t want to do the job [on safety
grounds] then I shouldn’t
bother coming to work
tomorrow”
How can we best identify risks?How can we best identify risks?
• Reactive/retrospectiveReactive/retrospective – Accidents and incident investigation Accidents and incident investigation – Root Cause Analysis (what, who, how, Root Cause Analysis (what, who, how, WHYWHY))– Reporting systems: incidentsReporting systems: incidents
• But we need to move people from ‘fire-fighting’ But we need to move people from ‘fire-fighting’ the last error to trying to prevent the next onethe last error to trying to prevent the next one
• Proactive/prospective Proactive/prospective – Confidential Reporting systems: near misses, and safety Confidential Reporting systems: near misses, and safety
concernsconcerns– Hazard identification, and prospective risk assessments Hazard identification, and prospective risk assessments
Barriers to ReportingBarriers to Reporting• Fear (including concerns re confidentiality)Fear (including concerns re confidentiality)
– blame culture, job loss, commercial issues blame culture, job loss, commercial issues • PracticalitiesPracticalities
– time to reporttime to report– complex formscomplex forms
• Ignorance Ignorance – what to report (definitions e.g. ‘near miss’)what to report (definitions e.g. ‘near miss’)
• Apathy Apathy – lack of perceived benefit to individual vs lack of perceived benefit to individual vs
potential cost potential cost – lack of faith in the system to change things lack of faith in the system to change things – lack of any feedback (‘black hole’ syndrome)lack of any feedback (‘black hole’ syndrome)
Reporting rates and triangulating riskReporting rates and triangulating risk
• For these reasons, incidents, near-misses & For these reasons, incidents, near-misses & safety concerns safety concerns will always be under-reportedwill always be under-reported whatever the systemwhatever the system
• But there are ways to increase reporting, by But there are ways to increase reporting, by targeting specific issues, and seeking other targeting specific issues, and seeking other ways to triangulate risk including surveys ways to triangulate risk including surveys
Reporting rates and triangulating riskReporting rates and triangulating risk
• More reports = good news, not bad news. With More reports = good news, not bad news. With more information to analyse we can act as an more information to analyse we can act as an Early Warning SystemEarly Warning System
• The industry must therefore encourage and The industry must therefore encourage and reward reporting, not penalise itreward reporting, not penalise it
• CIRAS has recently moved beyond the model of CIRAS has recently moved beyond the model of the ‘passive post-box’ and is seeking to be more the ‘passive post-box’ and is seeking to be more ‘proactive’ in engaging industry groups e.g. with ‘proactive’ in engaging industry groups e.g. with workshops and surveys workshops and surveys
Confidential Reporting Systems Confidential Reporting Systems
In an ideal world we wouldn’t need In an ideal world we wouldn’t need confidential reporting systems. Staff confidential reporting systems. Staff would be happy to volunteer information would be happy to volunteer information about errors they made, or safety about errors they made, or safety concerns they had, without fear of blame concerns they had, without fear of blame or victimisation, and management would or victimisation, and management would willingly address all the issues raised by willingly address all the issues raised by their staff………… their staff…………
Confidential Reporting Systems Confidential Reporting Systems
But we don’t live in an ideal world ! But we don’t live in an ideal world !
Consequently in virtually all safety-critical Consequently in virtually all safety-critical industries, here and abroad, it has been industries, here and abroad, it has been necessary to incorporate confidential necessary to incorporate confidential reporting systems as part of the suite of reporting systems as part of the suite of reporting systems available and operating reporting systems available and operating within safety management systemswithin safety management systems
Identifying Risk by Identifying Risk by Donald RumsfeldDonald Rumsfeld
As we know,As we know,
there are known knowns.there are known knowns.
There are things we know we know. There are things we know we know.
We also know there are known unknowns. We also know there are known unknowns.
That is to say That is to say
We know there are some things we do not know.We know there are some things we do not know.
But there are also unknown unknowns, But there are also unknown unknowns,
The ones we don’t know we don’t know. The ones we don’t know we don’t know.
Confidential reporting systemsConfidential reporting systems
• Are there to access all these Are there to access all these categories categories
• Are an indispensable part of the Are an indispensable part of the process of identifying risk in any process of identifying risk in any safety-critical industry safety-critical industry
• Should be an integral part of any Should be an integral part of any safety management systemsafety management system
[email protected]@ciras.org.uk