research & education challenges in risk analysis & risk...
TRANSCRIPT
![Page 1: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/1.jpg)
Research & Education Challenges
in
Risk Analysis & Risk Management
Improved Understanding of Risk Management
Type Matching Risks, Risk Analysis & Risk Response
Robert G. Ross, Captain, USCG (Retired
DHS Science and Technology Directorate
Chair, Security and Defense Specialty Group, Society for Risk Analysis
Maritime Risk Symposium 2011
Rutgers University
9 November, 2011
![Page 2: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/2.jpg)
The views presented here are those of
the presenter and are not to be taken as
necessarily reflecting the official views
of the Department of Homeland
Security or any other agency of the
federal government
![Page 3: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/3.jpg)
Risk Management is Not Meeting Expectations
Observed – Risk Management is failing in the face of 21st Century Threats and Hazards
Hazards – Terrorism, Climate Change, Global Supply Chain Disruption, others
Evidence – Financial System Meltdown
– Deepwater Horizon
– “The Failure of Risk Management” by Douglas Hubbard
![Page 4: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/4.jpg)
Diagnosis– Cause in three parts
1. Managers/Risk Managers who don‟t understand risk management
2. Risk Analysts who don‟t understand risk management
3. Analytic approaches and risk responses that are ill-suited to the risks to which they are applied, esp. true for newly emergent, newly recognized risks
![Page 5: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/5.jpg)
Recommended Treatment
1. Risk Managers and Risk Analysts both need a better, more complete understanding of risk management
2. Analytic methods and risk responses must be compatible with fundamental characteristics of the risk in question – we especially need new approaches better suited to complex and complex adaptive systems
![Page 6: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/6.jpg)
Recommended Treatment
1. Risk Managers and Risk Analysts both need a better, more complete understanding of risk management
2. Analytic methods and risk responses must be compatible with fundamental characteristics of risk in question – we especially need new approaches better suited to complex adaptive systems
![Page 7: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/7.jpg)
Background
1981 – Kaplan & Garrick‟s Risk Assessment Triplet
• What can happen?
• How likely is it that it will happen?
• If it does happen, what are the consequences?
Kaplan S, Garrick B. J. “On the Quantitative Definition of Risk”
Risk Analysis, 1981: Vol. 1 No. 1
![Page 8: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/8.jpg)
Background
1991 – Haimes‟ “Total Risk Management” Triplet
• What can be done and what options are available?
• What are their associated trade-offs in terms of all costs, benefits and risks?
• What are the impacts of current management decisions on future options?
Haimes Y. Y. “Total Risk Management”
Risk Analysis, 1991: Vol. 11 No. 2
![Page 9: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/9.jpg)
Background
2009 – Haimes suggests adding 4th RA Question to Kaplan & Garrick‟s original triplet
• Over what time frame?
Haimes, Y. Y., “”On the Complex Definition of Risk:
A Systems-Based Approach” Risk Analysis, 2009: Vol. 29, No. 12
![Page 10: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/10.jpg)
Define the
Context
Identify
Potential Risk
Assess
Potential Risk
Develop Alternative
Courses of Action
Decide and
Implement
Evaluate Alternative
Courses of Action
Evaluate and
Monitor
COMMUNICATIONS
The Total Risk Management Cycle
Figure 1
![Page 11: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/11.jpg)
The 5 Question Triplets in Risk Management
1. Risk Context 1-1. What are my risk management responsibilities? * 1-2. What is my risk management environment? *
1-3. What outcomes and objectives am I expected to achieve? *
2. Risk Assessment 2-1. What can happen? *
2-2. How likely is it that it will happen? *
2-3. If it does happen, what are the consequences? *
3. Risk Response 3-1. What could I do about it? *
3-2. What should I do about it? *
3-3. What am I going to do about it? *
4. Risk & Response Monitoring & Evaluation 4-1. How well is my chosen course of action working? *
4-2. Has anything changed that requires altering my existing
risk management measures? *
4-3. Are there current trends and/or potential future developments
that could require altering my existing risk management measures? *
5. Risk Communication 5-1. What risk information needs to be communicated? *
5-2. Between whom does it need to be communicated? *
5-3. How can necessary risk information be most effectively communicated? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 12: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/12.jpg)
Define the
Context
Identify
Potential Risk
Assess
Potential Risk
Develop Alternative
Courses of Action
Decide and
Implement
Evaluate Alternative
Courses of Action
Evaluate and
Monitor
COMMUNICATIONS
The Total Risk Management Cycle
Figure 2
1-1, 1-2, 1-3
2-2, 2-3
2-1
3-1
3-2
3-3
4-1, 4-2, 4-3
5-1, 5-2, 5-3
![Page 13: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/13.jpg)
Risk Context
1-1. What are my risk management responsibilities? * What is the nature of the risk(s) for which I am responsible?
What is the scope of my risk?
1-2. What is my risk management environment? *
1-3. What outcomes and objectives am I expected to
achieve? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 14: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/14.jpg)
Risk Assessment
2-1. What can happen? *
2-2. How likely is it that it will happen? *
2-3. If it does happen, what are the consequences? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 15: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/15.jpg)
Risk Response
3-1. What could I do about it? *
What can be done and what options are available?
3-2. What should I do about it? * What are their associated trade-offs in terms of all costs,
benefits and risks?
What are the impacts of current management decisions on future options?
3-3. What am I going to do about it? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 16: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/16.jpg)
Risk & Response Monitoring & Evaluation
4-1. How well is my chosen course of action working? *
4-2. Has anything changed that requires altering my
existing risk management measures? *
4-3. Are there current trends and/or potential future
developments that could require altering my
existing risk management measures? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 17: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/17.jpg)
Risk Communication
5-1. What risk information needs to be communicated? *
5-2. Between whom does it need to be communicated? *
5-3. How can necessary risk information be most
effectively communicated? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 18: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/18.jpg)
Recommended Treatment
1. Risk Managers and Risk Analysts both need a better, more complete understanding of risk management
2. Analytic methods and risk responses must be compatible with fundamental characteristics of risk in question – we especially need new approaches better suited to complex adaptive systems
![Page 19: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/19.jpg)
2 Propositions and a Question
P1 - Risk Management includes Risk Identification, Risk
Assessment, other Risk Analyses, chosing Risk Management
Strategies & specific Interventions, and Risk Communications.
P2 - To be effective, these elements of Risk Management must be
appropriate to the fundamental characteristics of the risk in
question.
Q – Can risks be usefully typed by fundamental characteristics to aid
in selecting analytic methods and risk management strategies?
![Page 20: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/20.jpg)
(1) Infectious and degenerative diseases
(2) natural catastrophes
(3)failure of large technological systems
(4) discrete, small-scale hazards
(5)low-level, delayed-effect hazards
(6) sociopolitical disruptions
William W. Lowrance
“The Nature of Risk,” in Societal Risk Assessment: How Safe is Safe Enough?
Richard C. Schwing and Walter A. Albers, Jr., eds.
(Plenum Press, New York and London, 1980). pp. 5-17.
Six Classes of Hazards
Risk Typing by Hazard
![Page 21: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/21.jpg)
Risk Typing by Weight & Color of Tail Feathers
THE FOURTH QUADRANT: A MAP OF THE LIMITS
OF STATISTICS
Nassim Nicholas Taleb, Edge , 15 Sept 2008 http://www.edge.org/3rd_culture/taleb08/taleb08_index.html
![Page 22: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/22.jpg)
Complexity ***
induced
Epistemological
Agency Staff
External Experts
Cognitive
Risk Problem
Type of Discourse
Actors
Type of Conflict
Probabilistic Risk
Modelling
Remedy
Agency Staff
External Experts
Stakeholders – Industry
– Directly affected
groups
Uncertainty
induced
Reflective
Risk Problem
Type of Discourse
Actors
Cognitive
Evaluative
Type of Conflict
Risk Balancing
Necessary
+Probabilistic
Risk Modelling
Remedy
Agency Staff
External Experts
Stakeholders – Industry
– Directly affected
groups
– General public
Ambiguity
induced
Participative
Risk Problem
Type of Discourse
Actors
Cognitive
Evaluative
Normative
Type of Conflict
Risk Trade-off
Analysis & Delib-
eration necessary
+Risk Balancing
+Probabilistic
Risk Modelling
Remedy
Simple
Instrumental
Risk Problem
Type of Discourse
Agency Staff
Actors
Statistical Risk
Analysis
Remedy
Risk Typing by Decision Support Needs & Modes
International Risk Governance Council (IRGC): White Paper on Risk Governance.
Towards an Integrative Framework. Author: Ortwin Renn (Geneva 2005)
Available at www.irgc.org under publications.
*** “Complexity” used here to
mean “complicated but
understandable and bounded”
– not in the CAS sense
IRGC Risk Management Escalator
![Page 23: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/23.jpg)
Typing Risk to Facilitate Analysis and Action
![Page 24: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/24.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
![Page 25: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/25.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
Stable Risk
• Neither the hazard nor the systemic context in which the hazard
resides change in direct response to risk management actions
• Hazards and their systemic contexts change relatively slowly
• Cause-effect pairs tightly coupled, isolable
![Page 26: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/26.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
Stable Risk
• Neither the hazard nor the systemic context in which the hazard
resides change in direct response to risk management actions
• Hazards and their systemic contexts change relatively slowly
• Cause-effect pairs tightly coupled, isolable
Dynamic Risk
• The hazard and/or the systemic context change, either in direct
response to risk management actions or spontaneously and
unpredictably…. or both
• Hazards and/or systemic context can change very quickly
• Cause-effect pairs neither tightly coupled nor isolable
![Page 27: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/27.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
Stable Risk
• Neither the hazard nor the systemic context in which the hazard
resides change in direct response to risk management actions
• Hazards and their systemic contexts change relatively slowly
• Cause-effect pairs tightly coupled, isolable
• Second distinction – Easily Discerned vs. Difficult to Discern
Dynamic Risk
• The hazard and/or the systemic context change, either in direct
response to risk management actions or spontaneously and
unpredictably…. or both
• Hazards and/or systemic context can change very quickly
• Cause-effect pairs neither tightly coupled nor isolable
![Page 28: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/28.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
Stable Risk
• Neither the hazard nor the systemic context in which the hazard
resides change in direct response to risk management actions
• Hazards and their systemic contexts change relatively slowly
• Cause-effect pairs tightly coupled, isolable
• Second distinction – Easily Discerned vs. Difficult to Discern
Dynamic Risk
• The hazard and/or the systemic context change, either in direct
response to risk management actions or spontaneously and
unpredictably…. or both
• Hazards and/or systemic context can change very quickly
• Cause-effect pairs neither tightly coupled nor isolable
• Second distinction – “Natural” vs. Adversarial
![Page 29: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/29.jpg)
Proposed Risk Typology
Type 1 – Stable Easily Discerned Risk
Type 2 – Stable Difficult to Discern Risk
Type 3 – Dynamic Natural Risk (includes human error)
Type 4 – Dynamic Adversarial Risk
![Page 30: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/30.jpg)
Type 1 Risks – Stable Easily Discerned
Example – Marine Steam Boilers (1807 to 1852)
• What can happen? Exploding Boilers
• How likely is it to happen? Very Likely
• What are the consequences? Bad! Really, really bad!!!
Sophisticated Risk Assessment Not Required!
Scientific Analysis of Cause, Effect and Interventions Was Required
Solution Set – Primarily Prevention based on
• Science & Engineering – first ever federal grant for scientific research
• Standards– design, licensing, inspections, periodic testing
• Law – creation of first federal public welfare (safety) regulatory agency
• Based on analysis of cause (engineering & operational) & interventions
• Well-suited to “Fix and Forget” mentality (but frequently requires enforcement)
Kaplan & Garrick‟s
Risk Assessment
Triplet
![Page 31: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/31.jpg)
Type 1 Risks – Stable Easily Discerned
Example – Marine Steam Boilers (1807 to 1852)
• What can happen? Exploding Boilers
• How likely is it to happen? Very Likely
• What are the consequences? Bad! Really, really bad!!!
Sophisticated Risk Assessment Not Required!
Scientific Analysis of Cause, Effect and Interventions Was Required
Solution Set – Primarily Prevention based on
• Science & Engineering – first ever federal grant for scientific research
• Standards– design, licensing, inspections, periodic testing
• Law – creation of first federal public welfare (safety) regulatory agency
• Based on analysis of cause (engineering & operational) & interventions
• Well-suited to “Fix and Forget” mentality (but frequently requires enforcement)
Kaplan & Garrick‟s
Risk Assessment
Triplet
![Page 32: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/32.jpg)
Type 1 Risks – Stable Easily Discerned
Example – Marine Steam Boilers (1807 to 1852)
• What can happen? Exploding Boilers
• How likely is it to happen? Very Likely
• What are the consequences? Bad! Really, really bad!!!
Sophisticated Risk Assessment Not Required!
Scientific Analysis of Cause, Effect and Interventions Was Required
Solution Set – Primarily Prevention based on
• Science & Engineering – first ever federal grant for scientific research
• Standards– design, licensing, inspections, periodic testing
• Law – creation of first federal public welfare (safety) regulatory agency
• Based on analysis of cause (engineering & operational) & interventions
• Well-suited to “Fix and Forget” mentality (but frequently requires enforcement)
Kaplan & Garrick‟s
Risk Assessment
Triplet
![Page 33: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/33.jpg)
Type 1 Risks – Stable Easily Discerned
Example – Marine Steam Boilers (1807 to 1852)
• What can happen? Exploding Boilers
• How likely is it to happen? Very Likely
• What are the consequences? Bad! Really, really bad!!!
Sophisticated Risk Assessment Not Required!
Scientific Analysis of Cause, Effect and Interventions Was Required
Solution Set – Primarily Prevention based on
• Science & Engineering – first ever federal grant for scientific research
• Standards– design, licensing, inspections, periodic testing
• Law – creation of first federal public welfare (safety) regulatory agency
• Based on analysis of cause (engineering & operational) & interventions
• Well-suited to “Fix and Forget” mentality (but frequently requires enforcement)
Kaplan & Garrick‟s
Risk Assessment
Triplet
![Page 34: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/34.jpg)
Type 1 Risks – Stable Easily Discerned
Example – Marine Steam Boilers (1807 to 1852)
• What can happen? Exploding Boilers
• How likely is it to happen? Very Likely
• What are the consequences? Bad! Really, really bad!!!
Sophisticated Risk Assessment Not Required!
Scientific Analysis of Cause, Effect and Interventions Was Required
Solution Set – Primarily Prevention based on
• Science & Engineering – first ever federal grant for scientific research
• Standards– design, licensing, inspections, periodic testing
• Law – creation of first federal public welfare (safety) regulatory agency
• Based on analysis of cause (engineering & operational) & interventions
• Well-suited to “Fix and Forget” mentality (but frequently requires enforcement)
Kaplan & Garrick‟s
Risk Assessment
Triplet
![Page 35: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/35.jpg)
Type 2 Risks – Stable Difficult to Discern
Why difficult to discern?
• Inability to directly discern answers (gambling, actuarial/insurance)
• Scale and complicated nature of engineered systems (nuclear power)
• Latency between cause and effect (carcinogenic chemicals at work)
• Low signal to noise ratio in cause and effect (pollution impacts)
Sophisticated Risk Assessment/Analysis Absolutely Necessary
• Statistics, Probabilistic Risk Analysis, Epidemiologic Studies, Modeling and other
methods to answer K&G‟s Risk Assessment Triplet
Scientific Analysis of Cause, Effect and Interventions Also Required
Solution Set very similar to Type 1 solution set - standards,
regulations, engineering controls, enforcement, “Fix and Forget,”
plus Consequence Mitigation, insurance …
![Page 36: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/36.jpg)
Type 2 Risks – Stable Difficult to Discern
Why difficult to discern?
• Inability to directly discern answers (gambling, actuarial/insurance)
• Scale and complicated nature of engineered systems (nuclear power)
• Latency between cause and effect (carcinogenic chemicals at work)
• Low signal to noise ratio in cause and effect (pollution impacts)
Sophisticated Risk Assessment/Analysis Absolutely Necessary
• Statistics, Probabilistic Risk Analysis, Epidemiologic Studies, Modeling and other
methods to answer K&G‟s Risk Assessment Triplet
Scientific Analysis of Cause, Effect and Interventions Also Required
Solution Set very similar to Type 1 solution set - standards,
regulations, engineering controls, enforcement, “Fix and Forget,”
plus Consequence Mitigation, insurance …
![Page 37: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/37.jpg)
Type 2 Risks – Stable Difficult to Discern
Why difficult to discern?
• Inability to directly discern answers (gambling, actuarial/insurance)
• Scale and complicated nature of engineered systems (nuclear power)
• Latency between cause and effect (carcinogenic chemicals at work)
• Low signal to noise ratio in cause and effect (pollution impacts)
Sophisticated Risk Assessment/Analysis Absolutely Necessary
• Statistics, Probabilistic Risk Analysis, Epidemiologic Studies, Modeling and other
methods to answer K&G‟s Risk Assessment Triplet
Scientific Analysis of Cause, Effect and Interventions Also Required
Solution Set very similar to Type 1 solution set - standards,
regulations, engineering controls, enforcement, “Fix and Forget,”
plus Consequence Mitigation, insurance …
![Page 38: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/38.jpg)
Type 2 Risks – Stable Difficult to Discern
Why difficult to discern?
• Inability to directly discern answers (gambling, actuarial/insurance)
• Scale and complicated nature of engineered systems (nuclear power)
• Latency between cause and effect (carcinogenic chemicals at work)
• Low signal to noise ratio in cause and effect (pollution impacts)
Sophisticated Risk Assessment/Analysis Absolutely Necessary
• Statistics, Probabilistic Risk Analysis, Epidemiologic Studies, Modeling and other
methods to answer K&G‟s Risk Assessment Triplet
Scientific Analysis of Cause, Effect and Interventions Also Required
Solution Set very similar to Type 1 solution set - standards,
regulations, engineering controls, enforcement, “Fix and Forget,”
plus Consequence Mitigation, insurance …
![Page 39: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/39.jpg)
Type 2 Risks – Stable Difficult to Discern
Why difficult to discern?
• Inability to directly discern answers (gambling, actuarial/insurance)
• Scale and complicated nature of engineered systems (nuclear power)
• Latency between cause and effect (carcinogenic chemicals at work)
• Low signal to noise ratio in cause and effect (pollution impacts)
Sophisticated Risk Assessment/Analysis Absolutely Necessary
• Statistics, Probabilistic Risk Analysis, Epidemiologic Studies, Modeling and other
methods to answer K&G‟s Risk Assessment Triplet
Scientific Analysis of Cause, Effect and Interventions Also Required
Solution Set very similar to Type 1 solution set - standards,
regulations, engineering controls, enforcement, “Fix and Forget”…
![Page 40: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/40.jpg)
Example – Vessel Traffic in Ports
• Fixed physical parameters – bridges, wharves, channels
• Dynamic physical parameters – wind, water depth, current, visibility, etc.
• Dynamic, variable mix of vessels – types, locations, courses & speeds, human
operators – general patterns but also near-infinite variety
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Vessel Traffic Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – very
• What are the consequences? Scenario dependent – minor to 7K+ dead
• Meaningful PRA extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem –
Requires Constant Attention – No “Fix and Forget”
Type 3 Risks – Dynamic “Natural” Risk
![Page 41: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/41.jpg)
Example – Vessel Traffic in Ports
• Fixed physical parameters – bridges, wharves, channels
• Dynamic physical parameters – wind, water depth, current, visibility, etc.
• Dynamic, variable mix of vessels – types, locations, courses & speeds, human
operators – general patterns but also near-infinite variety
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Vessel Traffic Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – very
• What are the consequences? Scenario dependent – minor to 7K+ dead
• Loose cause and effect linkages
• Meaningful PRA extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem –
Requires Constant Attention – No “Fix and Forget”
Type 3 Risks – Dynamic “Natural” Risk
![Page 42: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/42.jpg)
Example – Vessel Traffic in Ports
• Fixed physical parameters – bridges, wharves, channels
• Dynamic physical parameters – wind, water depth, current, visibility, etc.
• Dynamic, variable mix of vessels – types, locations, courses & speeds, human
operators – general patterns but also near-infinite variety
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Vessel Traffic Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – very
• What are the consequences? Scenario dependent – minor to 7K+ dead
• Loose cause and effect linkages
• Meaningful PRA extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem –
Requires Constant Attention – No “Fix and Forget”
Type 3 Risks – Dynamic “Natural” Risk
![Page 43: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/43.jpg)
Example – Terrorism
• Numerous possibilities -- attackers, attack modes & near-infinite target list
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Terrorism Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – unknown
• What are the consequences? Scenario dependent – minor to catastrophic
• Meaningful PRA, especially at tactical level, extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem
• Risk Reduction Measures, if known to adversary, can be bypassed or overcome,
or even exploited if unanticipated vulnerabilities are created
• Strategic vs. Tactical Mismatch
Type 4 Risks – Dynamic Adversarial Risk
![Page 44: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/44.jpg)
Example – Terrorism
• Numerous possibilities -- attackers, attack modes & near-infinite target list
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Terrorism Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – unknown
• What are the consequences? Scenario dependent – minor to catastrophic
• Cause and effect linkages are deliberate/chosen, not statistical or stochastic
• Meaningful PRA, especially at tactical level, extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem
• Risk Reduction Measures, if known to adversary, can be bypassed or overcome,
or even exploited if unanticipated vulnerabilities are created
• Strategic vs. Tactical Mismatch
Type 4 Risks – Dynamic Adversarial Risk
![Page 45: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/45.jpg)
Example – Terrorism
• Numerous possibilities -- attackers, attack modes & near-infinite target list
• Very Complex Adaptive System (nested/overlapping systems of systems)
Assessment of Terrorism Risk
• What can happen? Many different scenarios – near-infinite variety
• How likely are they to happen? Individually – not very, Collectively – unknown
• What are the consequences? Scenario dependent – minor to catastrophic
• Cause and effect linkages are deliberate/chosen, not statistical or stochastic
• Meaningful PRA, especially at tactical level, extremely difficult to impossible
Difficult Risk Management Context
• Multiple “Risk Managers” – Multiple Tools – Ongoing, ever-changing problem
• Risk Reduction Measures, if known to adversary, can be bypassed or overcome,
or even exploited if unanticipated vulnerabilities are created
• Strategic vs. Tactical Mismatch
Type 4 Risks – Dynamic Adversarial Risk
![Page 46: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/46.jpg)
Proposed Risk Typology
Type 1 – Stable Easily Discerned Risk
Type 2 – Stable Difficult to Discern Risk
Type 3 – Dynamic Natural Risk
Type 4 – Dynamic Adversarial Risk
Each type possesses fundamentally different characteristics
Each type requires fundamentally different approaches to Risk
Assessment, Risk Analysis and Risk Management
Strategies & Interventions
![Page 47: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/47.jpg)
Mixed Risk Types A given risk context or source can exhibit multiple risk types
• Example: Marine Boilers
– Type 1 – Engineering Deficiencies
– Type 3 – Human Error – inadequate maintenance, operator error
– Type 4 – Misconduct – gagging relief valves to boost pressure & speed
– Type 2 – Asbestos – used in insulation, pipe lagging
• Example: Biological Threats
– Type 1 – Traditional “Normal” Diseases
– Type 2 – Emergent Zoonotic Disease – e.g., “Flying Pig Flu”
– Type 3 – Human Error – e.g., accidental lab release
– Type 3 – Drug Resistant Bacteria – e.g., MRSA, bacillus gonnakillus
– Type 4 – Biological Attack – e.g., 2001 anthrax, synthetic smallpox
![Page 48: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/48.jpg)
Type 3 & 4 Risks – Complex Adaptive Systems
Complex Adaptive Systems • Many interdependent components
• Behaviors and interdependencies unknown, possibly variable
• Non-linear stimulus-response relationships, also possibly variable
Type 2 Analytic Methods Potentially Useful but Always Inadequate
Type 1 & 2 Interventions Necessary but not Sufficient
• Continuous attention & adaptation required - “Fix and Forget” guaranteed to fail
Suggestions for New Approaches • Look at the system rather than at individual elements or factors
• Focus on understanding system dynamics rather than predicting specific events
or outcomes
• Intervene to affect component behaviors and system responses to inputs and
changes rather than to prevent specific events or outcomes
![Page 49: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/49.jpg)
Type 3 & 4 Risks – Complex Adaptive Systems
Complex Adaptive Systems • Many interdependent components
• Behaviors and interdependencies unknown, possibly variable
• Non-linear stimulus-response relationships, also possibly variable
Type 2 Analytic Methods Potentially Useful but Always Inadequate
Type 1 & 2 Interventions Necessary but not Sufficient
• Continuous attention & adaptation required - “Fix and Forget” guaranteed to fail
Suggestions for New Approaches • Look at the system rather than at individual elements or factors
• Focus on understanding system dynamics rather than predicting specific events
or outcomes
• Intervene to affect component behaviors and system responses to inputs and
changes rather than to prevent specific events or outcomes
![Page 50: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/50.jpg)
Type 3 & 4 Risks – Complex Adaptive Systems
Complex Adaptive Systems • Many interdependent components
• Behaviors and interdependencies unknown, possibly variable
• Non-linear stimulus-response relationships, also possibly variable
Type 2 Analytic Methods Potentially Useful but Always Inadequate
Type 1 & 2 Interventions Probably Necessary but Never Sufficient
• Continuous attention & adaptation required - “Fix and Forget” guaranteed to fail
Suggestions for New Approaches • Look at the system rather than at individual elements or factors
• Focus on understanding system dynamics rather than predicting specific events
or outcomes
• Intervene to affect component behaviors and system responses to inputs and
changes rather than to prevent specific events or outcomes
![Page 51: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/51.jpg)
Type 3 & 4 Risks – Complex Adaptive Systems
Complex Adaptive Systems • Many interdependent components
• Behaviors and interdependencies unknown, possibly variable
• Non-linear stimulus-response relationships, also possibly variable
Type 2 Analytic Methods Potentially Useful but Always Inadequate
Type 1 & 2 Interventions Probably Necessary but Never Sufficient
• Continuous attention & adaptation required - “Fix and Forget” guaranteed to fail
Suggestions for New Approaches • Look at the system rather than at individual elements or factors
• Focus on understanding system dynamics rather than predicting specific events
or outcomes
• Intervene to affect component behaviors and system responses to inputs and
changes rather than to prevent specific events or outcomes
![Page 52: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/52.jpg)
Lessons from High Reliability Organizations
99.999999+% Success Catastrophic Failure & Public Outrage
These situations demands High Reliability Organizations (HROs)
HROs exhibit:
– Preoccupation with failure
– Reluctance to simplify interpretations
– Sensitivity to operations
– Commitment to resilience
– Deference to expertise
Reward rather than punish problem identification & reporting
HROs results from organizational culture & real behavior, not from
slogans on the walls or analyses done to satisfy a checklist
![Page 53: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/53.jpg)
Research Needs in CAS Risk Management
New Ways of Conceptualizing CAS Risk & CAS Risk Management
Analytic Methods Appropriate for CAS Risks
Risk Management Strategies Appropriate for CAS Risks
Risk Interventions Appropriate for specific CAS Risks
Decision-Making Processes Appropriate for when RM
Responsibilities are Shared
How to achieve “HRO” results in environments with high uncertainty,
dynamic risks, multiple risk managers and stakeholders with
competing agendas, some of which may be malicious
![Page 54: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/54.jpg)
54
"Not everything that can be counted counts, and not everything that
counts can be counted."
- Albert Einstein (1879-1955)
![Page 55: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/55.jpg)
Questions?
![Page 56: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/56.jpg)
National Academy of Science
If you don't know where
, “Rarely is there a single „right‟ risk analysis tool, method or model to provide „correct‟ analysis to support decision making…” ad will get you there
Committee to Review the Department of Homeland Security's Approach to Risk Analysis.
Review of the Department of Homeland Security's Approach to Risk Analysis. Washington DC: National Academies Press, 2010.
e Cheshire Cat
![Page 57: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/57.jpg)
More from the NAS Report
For terrorism risk analysis, neither threats nor consequences are well
characterized by data.
…terrorism involves an open rather than a closed system…
Terrorists observe and respond to defenses and to changing political
conditions…
…it will rarely be possible to develop statistically valid estimates of attack
frequencies (threat) or success probabilities (vulnerability)…
…better methods need to be found for incorporating the intentional nature of
terrorist attacks into risk analyses…
![Page 58: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/58.jpg)
A Note of Caution on PRA
• “…it is simply not possible to validate predictive models of rare events that have not occurred, and unvalidated models cannot be relied upon.”
• “…distinction between models for probabilistic risk assessment on long time scales…vs. specific point prediction of individual rare events.”
• Models for prediction vs. models for insight
Source – “Rare Events”; JASON (DOD Advisory Group); October 2009
http://www.fas.org/irp/agency/dod/jason/rare.pdf
![Page 59: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/59.jpg)
Words of Wisdom
“Unlike the position that exists in the physical sciences, in…
disciplines that deal with essentially complex phenomena, the
aspects of the events to be accounted for about which we can
get quantitative data are necessarily limited and may not
include the important ones. While in the physical sciences it is
generally assumed… that any important factor which
determines the observed events will itself be directly
observable and measurable...in…complex phenomena…which
depend on the actions of many individuals, all the
circumstances which will determine the outcome of a
process…will hardly ever be fully known or measurable.”
![Page 60: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/60.jpg)
Words of Wisdom (cont’d.)
“It is an approach which has come to be described as the
"scientistic" attitude - an attitude which, as I defined it some
thirty years ago, "is decidedly unscientific in the true sense of
the word, since it involves a mechanical and uncritical
application of habits of thought to fields different from those in
which they have been formed.””
Friedrich August von Hayek First Nobel Laureate in Economics
“The Pretence of Knowledge”
Lecture to the memory of Alfred Nobel
December 11, 1974
![Page 61: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/61.jpg)
Risk Management Strategies/Responses
You can
• Accept Risk
• Avoid Risk
• Transfer Risk
• Reduce Risk
![Page 62: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/62.jpg)
Risk Management Strategies/Responses
You can
• Accept Risk
• Avoid Risk
• Transfer Risk
• Reduce Risk
Substitute one risk for another
![Page 63: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/63.jpg)
Challenging Orthodoxy
“Risk analysis is broadly defined to include risk
assessment, risk characterization, risk
communication, risk management, and policy
relating to risk…,”
Society for Risk Analysis Vision Statement
![Page 64: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/64.jpg)
Challenging Orthodoxy
“Risk analysis is broadly defined to include risk
assessment, risk characterization, risk
communication, risk management, and policy
relating to risk…,”
Society for Risk Analysis Vision Statement
![Page 65: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/65.jpg)
Challenging Orthodoxy
“Risk analysis is broadly defined to include risk
assessment, risk characterization, risk
communication, risk management, and policy
relating to risk…,”
Society for Risk Analysis Vision Statement
Risk Management is the superior construct, not Risk
Analysis
Risk Analysis is one of several supporting subordinate
components of Risk Management
![Page 66: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/66.jpg)
Challenging Orthodoxy
“Risk analysis is broadly defined to include risk
assessment, risk characterization, risk
communication, risk management, and policy
relating to risk…,”
Society for Risk Analysis Vision Statement
Risk Management is the superior construct, not Risk
Analysis
Risk Analysis is only one of several subordinate
supporting components of Risk Management
![Page 67: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/67.jpg)
Background Various Risk Management Cycles/Frameworks –
GAO, IRGC, DHS, numerous others
Assessment Sphere:
Generation of Knowledge
Management Sphere:
Decision on & Implementation of
Actions
Risk Characterisation
• Risk Profile
• Judgement of the
Seriousness of Risk
• Conclusions & Risk
Reduction Options
Risk Evaluation
• Judging Tolerability
& Acceptability
• Need for Risk
Reduction Measures
Tolerability & Acceptability Judgement
Pre-Assessment:
• Problem Framing
• Early Warning
• Screening
• Determination of Scientific Conventions
Pre-Assessment
Risk Appraisal:
Risk Assessment
• Hazard Identification & Estimation
• Exposure & Vulnerability Assessment
• Risk Estimation
Concern Assessment
• Risk Perceptions
• Social Concerns
• Socio-Economic Impacts
Risk AppraisalRisk Management
Implementation
• Option Realisation
• Monitoring & Control
• Feedback from Risk Mgmt. Practice
Decision Making
• Option Identification & Generation
• Option Assessment
• Option Evaluation & Selection
Risk Management
Communication
Define the Context
Identify Potential Risk
Assess and Analyze Risk
Develop Alternatives
Decide and Implement
Evaluate and Monitor
Communication
![Page 68: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/68.jpg)
Risk Context
1-1. What are my risk management responsibilities? * What is the nature of the risk(s) for which I am responsible?
What is the scope of my risk?
1-2. What is my risk management environment? *
1-3. What outcomes and objectives am I expected to
achieve? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 69: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/69.jpg)
Consequence
Likelihood
Must
Consider
Action
Should
Consider
Action
Could
Consider Action Don’t
Worry
Risk C
Risk B
Risk A
High C/Low L
Single Incident Impact
High L/Low C
Cumulative Impact
![Page 70: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/70.jpg)
Risk Response
3-1. What could I do about it? *
3-2. What should I do about it? * What are their associated trade-offs in terms of all costs,
benefits and risks?
What are the impacts of current management decisions on future options?
3-3. What am I going to do about it? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 71: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/71.jpg)
Scope
GLOBAL Thinning of
the ozone layer
X
LOCAL Recession in a country Genocide
PERSONAL Your car is stolen Death
ENDURABLE TERMINAL
Intensity Six risk categories
Existential Risks Analyzing Human Extinction Scenarios and Related Hazards
Nick Bostrom
Faculty of Philosophy, Oxford University
Journal of Evolution and Technology, Vol. 9, March 2002
Risk Typing by Nature and Scale of Consequence
![Page 72: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/72.jpg)
Research & Education Challenges
in
Risk Analysis & Risk Management
Improved Understanding of Risk Management
Type Matching Risks, Risk Analysis & Risk Response
Robert G. Ross, Captain, USCG (Retired
DHS Science and Technology Directorate
Chair, Security and Defense Specialty Group, Society for Risk Analysis
Maritime Risk Symposium 2011
Rutgers University
9 November, 2011
![Page 73: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/73.jpg)
Risk Management is Not Meeting Expectations
Observed – Risk Management is failing in the face of 21st Century Threats and Hazards
Hazards – Terrorism, Climate Change, Global Supply Chain Disruption, others
Evidence – Financial System Meltdown
– Deepwater Horizon
– “The Failure of Risk Management” by Douglas Hubbard
![Page 74: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/74.jpg)
Diagnosis– Cause in three parts
1. Managers/Risk Managers who don‟t understand risk management
2. Risk Analysts who don‟t understand risk management
3. Analytic approaches and risk responses that are ill-suited to the risks to which they are applied, esp. true for newly emergent, newly recognized risks
![Page 75: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/75.jpg)
Recommended Treatment
1. Risk Managers and Risk Analysts both need a better, more complete understanding of risk management
2. Analytic methods and risk responses must be compatible with fundamental characteristics of the risk in question – we especially need new approaches better suited to complex and complex adaptive systems
![Page 76: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/76.jpg)
The 5 Question Triplets in Risk Management
1. Risk Context
1-1. What are my risk management responsibilities? * 1-2. What is my risk management environment? *
1-3. What outcomes and objectives am I expected to achieve? *
2. Risk Assessment 2-1. What can happen? *
2-2. How likely is it that it will happen? *
2-3. If it does happen, what are the consequences? *
3. Risk Response 3-1. What could I do about it? *
3-2. What should I do about it? *
3-3. What am I going to do about it? *
4. Risk & Response Monitoring & Evaluation 4-1. How well is my chosen course of action working? *
4-2. Has anything changed that requires altering my existing
risk management measures? *
4-3. Are there current trends and/or potential future developments
that could require altering my existing risk management measures? *
5. Risk Communication 5-1. What risk information needs to be communicated? *
5-2. Between whom does it need to be communicated? *
5-3. How can necessary risk information be most effectively communicated? *
* “And when?” or “Over what timeframe?” should be added when appropriate
![Page 77: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/77.jpg)
Define the
Context
Identify
Potential Risk
Assess
Potential Risk
Develop Alternative
Courses of Action
Decide and
Implement
Evaluate Alternative
Courses of Action
Evaluate and
Monitor
COMMUNICATIONS
The Total Risk Management Cycle
Figure 2
1-1, 1-2, 1-3
2-2, 2-3
2-1
3-1
3-2
3-3
4-1, 4-2, 4-3
5-1, 5-2, 5-3
![Page 78: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/78.jpg)
Typing Risk to Facilitate Analysis and Action
First Distinction – Stable vs. Dynamic
Stable Risk
• Neither the hazard nor the systemic context in which the hazard
resides change in direct response to risk management actions
• Hazards and their systemic contexts change relatively slowly
• Cause-effect pairs tightly coupled, isolable
• Second distinction – Easily Discerned vs. Difficult to Discern
Dynamic Risk
• The hazard and/or the systemic context change, either in direct
response to risk management actions or spontaneously and
unpredictably…. or both
• Hazards and/or systemic context can change very quickly
• Cause-effect pairs neither tightly coupled nor isolable
• Second distinction – “Natural” vs. Adversarial
![Page 79: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/79.jpg)
Proposed Risk Typology
Type 1 – Stable Easily Discerned Risk
Type 2 – Stable Difficult to Discern Risk
Type 3 – Dynamic Natural Risk
Type 4 – Dynamic Adversarial Risk
Each type possesses fundamentally different characteristics
Each type requires fundamentally different approaches to Risk
Assessment, Risk Analysis and Risk Management
Strategies & Interventions
![Page 80: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/80.jpg)
Type 3 & 4 Risks – Complex Adaptive Systems
Complex Adaptive Systems • Many interdependent components
• Behaviors and interdependencies unknown, possibly variable
• Non-linear stimulus-response relationships, also possibly variable
Type 2 Analytic Methods Potentially Useful but Always Inadequate
Type 1 & 2 Interventions Probably Necessary but Never Sufficient
• Continuous attention & adaptation required - “Fix and Forget” guaranteed to fail
Suggestions for New Approaches • Look at the system rather than at individual elements or factors
• Focus on understanding system dynamics rather than predicting specific events
or outcomes
• Intervene to affect component behaviors and system responses to inputs and
changes rather than to prevent specific events or outcomes
![Page 81: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/81.jpg)
Lessons from High Reliability Organizations
99.999999+% Success Catastrophic Failure & Public Outrage
These situations demands High Reliability Organizations (HROs)
HROs exhibit:
– Preoccupation with failure
– Reluctance to simplify interpretations
– Sensitivity to operations
– Commitment to resilience
– Deference to expertise
Reward rather than punish problem identification & reporting
HROs results from organizational culture & real behavior, not from
slogans on the walls or analyses done to satisfy a checklist
![Page 82: Research & Education Challenges in Risk Analysis & Risk ...dimacs.rutgers.edu/Workshops/Maritime/Slides/Total_Risk...Research & Education Challenges in Risk Analysis & Risk Management](https://reader036.vdocument.in/reader036/viewer/2022071301/60a1d3e6aa47903ff82a07c2/html5/thumbnails/82.jpg)
Research Needs in CAS Risk Management
New Ways of Conceptualizing CAS Risk & CAS Risk Management
Analytic Methods Appropriate for CAS Risks
Risk Management Strategies Appropriate for CAS Risks
Risk Interventions Appropriate for specific CAS Risks
Decision-Making Processes Appropriate for when RM
Responsibilities are Shared
How to achieve “HRO” results in environments with high uncertainty,
dynamic risks, multiple risk managers and stakeholders with
competing agendas, some of which may be malicious