arms - the enabler to understanding risk (chirp mar12).ppt · microsoft powerpoint - arms - the...
TRANSCRIPT
3/27/2012
1
ARMS
CHIRP Maintenance Meeting
28th March 2012
Andrew Rose – Llanbury Consulting
The enabler to
understanding Risk
Llanbury Consulting‘RISK’ – The big picture
ARMS – Aviation Risk Management
Solutions working group
The Mission of the ARMS Working Group is to
produce useful and cohesive Operational Risk
Assessment methods for airlines and other aviation
organisations and to clarify the related Risk
Management processes.
Formed in 2007 (Jari Nisula – Airbus, Andrew Rose – British Airways)
Llanbury Consulting‘RISK’ – The Big Picture
ARMS Background
Jun 07 – Oct 09 Safety practitioners from over 20 aviation organisations
3/27/2012
2
Llanbury Consulting‘RISK’ – The Big Picture
ARMS Participants
ARMS Working Group 2009
4
1
2500
500
100
100 500
10020
50
10
202 4
1
2500
500
100
100 500
10020
50
10
202
Investigations
Data Analysis-Frequencies
-Trends
-Identification of Safety
Issues
Hazard
Analysis
Safety
Performance
Monitoring
All collected
safety data
-Categorized
-ERC values
Safety Issue Risk
Assessment“SIRA”
Safety
Issues
Actions to
reduce risk
Safety
Assessments
Event Risk Classification (ERC)
All Data
Safety
Events
Safety
Issues
Scenarios
The ARMS Concept
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
3
Llanbury Consulting‘RISK’ – The Big Picture
What’s in it for me?
Effective risk management requires us to
monitor the performance of our system
ARMS techniques enable the best use of
the data and knowledge to enable the
optimum risk management decisions
Possibilities…
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
4
Llanbury Consulting‘RISK’ – The Big Picture
G-VXLA G-VXLB G-VXLC G-VXLD
G-VXLE G-VXLF G-VXLG G-VXLH
G-VXLG
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
5
ARMS Working Group 2009
4
1
2500
500
100
100 500
10020
50
10
202 4
1
2500
500
100
100 500
10020
50
10
202
Investigations
Data Analysis-Frequencies
-Trends
-Identification of Safety
Issues
Hazard
Analysis
Safety
Performance
Monitoring
All collected
safety data
-Categorized
-ERC values
Safety Issue Risk
Assessment“SIRA”
Safety
Issues
Actions to
reduce risk
Safety
Assessments
Event Risk Classification (ERC)
All Data
Safety
Events
Safety
Issues
Scenarios
The ARMS Concept
Llanbury Consulting‘RISK’ – The Big Picture
Llanbury Consulting‘RISK’ – The Big Picture
Event Risk Classification
This activity is about measuring performance and
risk in the system based on historical events:
Risk classification – NOT Risk Assessment
Severity of the
potential accident
outcome
Probability of the
event resulting in that
outcome*+
= Risk
* Probability of occurrence is derived from the data
3/27/2012
6
Catastrophic accident with multiple fatalities >8
1-8 fatalities, multiple serious injuries, major
damage/loss to the aircraft
Minor injuries, minor damage to aircraft
No potential damage or injury could occur
If this event had escalated into an accident, what would have been the most credible
accident outcome?
ARMS Event Risk Classification (ERC) Framework
What is the probability of this event progressing to that credible accident outcome?
Llanbury Consulting‘RISK’ – The Big Picture
1
0.01
2000
1000
100
20 200
10010
2
1
100.1
Catastrophic accident with multiple fatalities >8
1-8 fatalities, multiple serious injuries, major
damage/loss to the aircraft
Minor injuries, minor damage to aircraft
No potential damage or injury could occur
If this event had escalated into an accident, what would have been the most credible
accident outcome?
The shared Aviation Risk Space
What is the probability of this event progressing to that credible accident outcome?
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
7
Llanbury Consulting‘RISK’ – The Big Picture
ERC – Severity Question
The severity question has to be based on the
credible accident outcome and not some
intermediary point.Why?
Because this is a measurement exercise for
risk across the aviation system so you have
to measure to the same point
Intermediary outcomes (such as
‘unairworthy aircraft’) are only separated
from accidents by probabilities and these
are accounted for in the next question
Llanbury Consulting‘RISK’ – The Big Picture
ERC – Probability Question
It is very subjective based on our personal
experience so the ARMS approach suggested
adopting a barrier principle:
Probability is a difficult question for us
What was the effectiveness of the remaining barriers
between this event and the accident scenario?Effective / Limited / Minimal / Not Effective
Barriers are an easier concept for us to
judge but this approach is still subjective
and perhaps over simplistic with regards to
the effectiveness of barriers.So …
3/27/2012
8
Llanbury Consulting‘RISK’ – The Big Picture
ERC – Structured Barrier Models
Significant work is on-going with the UK CAA
(involving airlines, EASA, FAA, EC, etc.) to
develop a structured barrier model approach to
answer the probability question:
What barriers stopped this event progressing?
What other barriers could also have stopped
this event progressing?
How reliable are those barriers?
It is a whole presentation on its own but in brief it is
evaluating standardised approaches to developing and
using models for different scenarios to determine:
Common Risk Classification Framework
1
0.01
2000
1000
100
20 200
10010
2
1
100.1
Catastrophic accident with multiple fatalities >8
1-8 fatalities, multiple serious injuries, major
damage/loss to the aircraft
Minor injuries, minor damage to aircraft
No potential damage or injury could occur
If this event had escalated into an accident, what would have been the most credible
accident outcome?
Common Risk Classification Framework
0.01
20002
1 1000
0.1 100
What are the barriers in this scenario?How important is each barrier?What is the status of the barriers in this case?
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
9
Llanbury Consulting‘RISK’ – The Big Picture
Framework approach to the ERC
It makes the semi-automated risk classification of
masses of system generated data a possibility…
- Reporting data is a rich source but narrow in its scope
- System generated data lacks detail but has wide coverage
- Combined they become a powerful source
of system performance knowledge
As an aviation system we over rely on employee reporting
data as our key (and separate) window on performance
"The whole is greater
than the sum of its
parts." - Aristotle
ARMS Working Group 2009
4
1
2500
500
100
100 500
10020
50
10
202 4
1
2500
500
100
100 500
10020
50
10
202
Investigations
Data Analysis-Frequencies
-Trends
-Identification of Safety
Issues
Hazard
Analysis
Safety
Performance
Monitoring
All collected
safety data
-Categorized
-ERC values
Safety Issue Risk
Assessment“SIRA”
Safety
Issues
Actions to
reduce risk
Safety
Assessments
Event Risk Classification (ERC)
All Data
Safety
Events
Safety
Issues
Scenarios
The ARMS Concept
Llanbury Consulting‘RISK’ – The Big Picture
3/27/2012
10
2009 Page 19
Undesirable
operational
stateTriggering EVENT
Maintenance
error
Flight ops hazard
Hazard on
ground
ATC hazard
Weather
hazard
ACCIDENT OUTCOME
Catastrophic
accident (e.g.
mid air collision)
Major accident
(e.g. overrun)
Minor safety
occurrence (e.g.
turbulence
bruises)
Negligible
PREVENTPREVENT AVOIDAVOID RECOVERRECOVER((MINIMISEMINIMISE
LOSSES)LOSSES)
1. FREQUENCY
OF Triggering EVENT
2. EFFECTIVENESS
OF AVOIDANCE
BARRIERS
3. EFFECTIVENESS
OF RECOVERY
BARRIERS
4. ACCIDENT
SEVERITY
Safety Issue Risk Assessment
Technical
hazard
Undesirable
operational
state
Triggering EVENT
ACCIDENT OUTCOME
Unrecoverable engine failures resulting in catastrophic accident
Loss of control due ash damage to airframe or sensors
Midair collision as a result of ash avoidance and recovery maneuvers
Injuries & damage sustained during emergency diversion & landing
Example SIRA for operations into areas
of forecast volcanic ash
Flight into area of forecast volcanic ash
Inadvertent encounter with higher than
expected density of volcanic ashPredict
Plan
Operate
MEL ATC
Engine mgmt
Flt Crew SOPs
Recover
3/27/2012
11
Expanded Volcanic Ash Model- One off encounter
KLM UK example
3/27/2012
12
ARMS - Jari Nisula 2010
ARMS SIRA Excel application
• Analysing the
risk factors
and barriers
3
3.1 Triggering event 3.2 Undesirable Operational State 3.3 Accident Outcome
4 Describe the barriers
4.1 To avoid the UOS 4.2 To recover before the Accident
5 Risk Assessment
The estimated frequency of
the triggering event (per
flight sectors) is:
The barriers will fail in AVOIDING
the UOS...
The barriers will fail in
RECOVERING the situation before
the ACCIDENT...
The accident severity would be...
About every 100000 sectors Once every 10 times Once every 10 times Major
1.E-05 1.E-01 1.E-01
UOS frequency: Mean Accident frequency:
1.E-06 1.E-07
6 Result 1.E+00
6.1 Resulting risk class Secure 1.E+00
Comments on actions:
Secure
Analysis of potential Accident Scenario
• Needed figures can be obtained
from the event
database or
estimated
Llanbury Consulting‘RISK’ – The Big Picture
Conclusion
ARMS offers powerful tools in the cohesive
understanding of risk across the aviation
system:
ERC and its associated barrier framework provides a
basis for the common measurement of risk
performance across aviation
So reach for the possibilities of a cohesive
picture of risk performance for maintenance
within the whole aviation context…
3/27/2012
13
Questions
Contact:
Andrew Rose – [email protected]
Questions and Further Information
?
Llanbury Consulting‘RISK’ – The Big Picture
ARMS link –
http://www.skybrary.aero/index.php/ARMS_Methodology_for_Risk_Assessment