risk management and the agile approach...- detailed - specific iso 14971 3.1: mfr "shall...
TRANSCRIPT
3/31/2019
1
Risk Management and the Agile Approach
Brian Shoemaker, Ph.D.
ShoeBar Associates
SDMD Global 2019, 11.-14. February 2019
Who I am
SDMD Global 2019 11.-14. Feb. 2019
2
� Originally an analytical chemist
� 15 y in clinical diagnostics (immunoassay): analytical support → assay development → instrument software
validation
� 6 y as SW quality manager (5 in clinical trial related
SW)
� 13 y as independent validation consultant to FDA-
regulated companies – mostly medical device
� Active in: software validation, Part 11 evaluation,
software quality systems, auditing, training
3/31/2019
2
Acknowledgement
SDMD Global 2019 11.-14. Feb. 2019
3
Special thanks to Pat Baird, who has provided valuable material for this presentation, not only from his own workshops on risk management, but also by furnishing background on the planned updates to ISO 14971.
Thesis
SDMD Global 2019 11.-14. Feb. 2019
4
In the changing world, medical device risk management more than ever needs the flexibility of an Agile approach.
3/31/2019
3
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
5
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
Safety Incidents Make News (1)
SDMD Global 2019 11.-14. Feb. 2019
6
July 10, 2006:
concrete ceiling
panel in one of the
tunnels leading to
Boston Logan
Airport collapsed,
killing an auto
passenger and
seriously injuring
the driver. The
tunnel was closed
for almost a year.
http://archive.boston.com/news/specials/big_dig_ceiling_collapse/
3/31/2019
4
Safety Incidents Make News (2)
SDMD Global 2019 11.-14. Feb. 2019
7
The same blowout that caused the explosion also caused a massive offshore
oil spill in the Gulf of Mexico, considered the largest accidental marine oil spill
in the world, and the largest environmental disaster in U.S. history.
April 20, 2010: explosion
and fire aboard the
Deepwater Horizon offshore
drilling rig resulted in the
sinking of the Deepwater
Horizon and the deaths of
11 workers; 17 others were
injured.
https://slate.com/technology/2016/09/bp-is-to-blame-for-deepwater-horizon-but-its-mistake-was-actually-years-of-small-mistakes.html
Safety Incidents Make News (3)
SDMD Global 2019 11.-14. Feb. 2019
8
May 12, 2015: Amtrak
Northeast Regional train
from Washington, D.C.
bound for New York City
derailed and wrecked on
the Northeast Corridor in
the Port Richmond
neighborhood of
Philadelphia, Pennsylvania.
Of 238 passengers and 5
crew on board, 8 were
killed and over 200 injured,
11 critically.
https://www.nytimes.com/2017/05/12/us/amtrak-derailment-crash-philadelphia.html
3/31/2019
5
Safety Incidents Make News (4)
SDMD Global 2019 11.-14. Feb. 2019
9
September 13, 2018: Excessive pressure in natural gas lines owned by
Columbia Gas caused a series of explosions and fires to occur in as many
as 40 homes, with over 80 individual fires, in the Merrimack Valley,
Massachusetts towns of Lawrence, Andover, and North Andover.
https://www.usatoday.com/Feature/money/2018/09/15/massachusetts-explosions-
nisource-columbia-gas-linked-previous-blasts/1302447002/
Medical Devices Aren’t Exempt
SDMD Global 2019 11.-14. Feb. 2019
10
Headline:
Medical Devices
Harm Patients Worldwide As Governments
Fail On Safety
November 25, 2018: A global investigation reveals the rising human
toll of lax controls and testing standards pushed by a booming industry.
https://www.icij.org/investigations/implant-files/medical-devices-harm-patients-worldwide-as-governments-fail-on-safety/
3/31/2019
6
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
11
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
Risk Management Is Central
SDMD Global 2019 11.-14. Feb. 2019
12
Quality System: ISO 13485 / 21 CFR 820
IEC 60601-1Medical Elect Eqpt – Basic Safety, Essential Performance
IEC 62304Medical Device SW
- Lifecycle
ISO 14971Med Dev Risk Management
IEC 62366Med Dev –
Application of Usability
Engineering
Risk Mgmt
3/31/2019
7
Risk Management in Brief
SDMD Global 2019 11.-14. Feb. 2019
13
1. What are you trying to do?
2. What can go wrong?
3. What are you going to do about it?
4. Did it work?
Source: Pat Baird; "Incorporating Risk Management into (software) Design & Development"; IQPC's 18th Software Design for Medical Devices; 10/27/2014
TIR 80002-1: Iterate!
SDMD Global 2019 11.-14. Feb. 2019
14
IEC 62304SW lifecyclerisk based
ISO 14971Med dev risk mgmt
IEC TIR80002-1SW risk mgmt
SW concerns, examples
… Iteration of RISK MANAGEMENT activities and coordination with SYSTEM design activities throughout the software LIFE-CYCLE … is advantageous to address any further HAZARDS as soon as they become apparent. This implies iteration within the implementation of the RISK CONTROL measure.
3/31/2019
8
Refine both features and mitigations
SDMD Global 2019 11.-14. Feb. 2019
15
Requirements
Requirements Hazards
Requirements
+ Mitigations
Early in project
- Preliminary
- High-level
- Approximate
Late in project
- Refined
- Detailed
- Specific
ISO 14971 §3.1: Mfr "shall
establish, document and
maintain throughout the
life-cycle an ongoing
process“ for analyzing,
evaluating, and controlling
risks.
Include RM in Emerging Documentation
SDMD Global 2019 11.-14. Feb. 2019
16
SRS
•Feature 1
•Feature 2
•Feature 3
•Feature 4
•Feature 5
•Feature 6
•Feature 7
V&V
SDS
Product
Hazards & Mitigations
3/31/2019
9
RM Lasts Through the Product Life
SDMD Global 2019 11.-14. Feb. 2019
17
Intended
Use Inherent risks, irrespective of design
Concept
Rqmts
Design
CodeTest
Demo
Design decisions introduce new hazards, new risk controls
RM matures, doesn’t freeze
Postmarket: customers use and misuse product!
Continue monitoring even after sales/service stop
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
18
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
3/31/2019
10
62304: Manage RISK in Development
SDMD Global 2019 11.-14. Feb. 2019
19
Activities within ISO 13485 / 14971Customer
Needs
Customer
Needs Satisfied
SYSTEM development ACTIVITIES (including RISK MANAGEMENT)
7 Software RISK MANAGEMENT
8 Software CONFIGURATION MANAGEMENT
9 Software problem resolution
5.1SW
DevelPlanning
5.2SW
RqmtsAnalysis
5.3SW
Architectdesign
5.4SW
Detaileddesign
5.5SW Unit
Implem& verif
5.6SW
Integrn,Int Tstg
5.7 SW
SystemTesting
5.8
SWRelease
62304: Manage Risk in Maintenance
SDMD Global 2019 11.-14. Feb. 2019
20
Activities within ISO 13485 / 14971Maintenance
Request
Request
Satisfied
SYSTEM maintenance ACTIVITIES (including RISK MANAGEMENT)
7 Software RISK MANAGEMENT
8 Software CONFIGURATION MANAGEMENT
9 Software problem resolution
6.1Estab
SW MaintPlan
6.2Prob &
modificnanalysis
5.3SW
Architectdesign
5.4SW
Detaileddesign
5.5SW Unit
Implem& verif
5.6SW
Integrn,Int Tstg
5.7 SW
SystemTesting
5.8SW
Release
6.3 Modification Implementation
3/31/2019
11
Discussion: Examples?
SDMD Global 2019 11.-14. Feb. 2019
21
Class A: No injury or damage to health is possible
Class B: Non-serious injury is possible
Class C: Death or serious injury is possible
A risk management process – compliant with ISO 14971
- is required throughout development and maintenance
… BUT the type and extent of required activities /
documentation depends on the safety classification.
That is, you must tailor your use of 62304 based on
safety classification.
Agile must be tailored, too
SDMD Global 2019 11.-14. Feb. 2019
22
Credit: Ahmed Sidky, “The Agile Mindset”, available at http://www.softed.com/assets/Uploads/Resources/Agile/The-Agile-Mindset-Ahmed-Sidky.pdf
3/31/2019
12
Risk Management Fits in Agile Stories
SDMD Global 2019 11.-14. Feb. 2019
23
What to build
Estimate
Architecture
Risk Plans
Test Approach
QA Approach
Agile Story
All these evolve as a side-effect when the voices of Customer and Engineering bring a Story to maturity.
How about this?
7th SDMD Europe 21.-24. Feb. 2017
24
Hazard Mitigation:“As a caregiver,I want to ensure that therapy will stop if short, open circuit, or high impedance is detected,to avoid harming the patient.”
A “negative story” is perfectly valid for the development backlog.
3/31/2019
13
Risk Mgmt Fits in Agile Process
SDMD Global 2019 11.-14. Feb. 2019
25
DR Deploy
Each iteration has design, dev, test, demo ( )
Each demo an incremental design review
Consider risk issues in each demo / design review
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
26
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
3/31/2019
14
Much to learn from system failures
SDMD Global 2019 11.-14. Feb. 2019
27
Key issues:• Complex systems: unlike linear (e.g.
assembly line), many different parts can interact, often in unpredictable ways. (Ex: nuclear power plants, chemical plants, modern autos, computerized stock trading systems, social media)
• Tight coupling: failure or change in one part of a system directly affects other parts, often quickly. (nuclear power plants, modern [computerized] auto systems)
The Danger Zone
SDMD Global 2019 11.-14. Feb. 2019
28
Complexity
Tig
ht
Co
up
ling
Nuclear power plants
Marine Transport
Assembly line production
Post offices
Dams
Chemical plants
R&D firms
Universities
Redrawn from Clearfield & Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It
3/31/2019
15
Move Out of the Danger Zone
SDMD Global 2019 11.-14. Feb. 2019
29
• Break the system into manageable parts (cook turkey
and stuffing in portions, not all at once)
• Provide more visibility into internals of the system
(indicators, meaningful error messages)
• Conduct "premortem" analysis - note the similarity to
FTA
• Make major changes in small increments
• Record accidents and near-misses, and assess them for
root causes (ASRS; MAUDE)
• Have diverse, cross-functional teams making decisions
Taken from Clearfield & Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
30
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
3/31/2019
16
ISO 14971 – updates coming
SDMD Global 2019 11.-14. Feb. 2019
31
Changes are a response to
• lower risk tolerance
• media coverage of incidents
• attention to security and privacy
• some cases of fraud
Jos van Vroonhoven, “Risk Management – Revision of ISO 14971 and Critical Issues for the Future,” presented at
AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20, London UK
Planned Updates
SDMD Global 2019 11.-14. Feb. 2019
32
• Definitions: HARM changing; BENEFIT added
• REASONABLY FORESEEABLE MISUSE distinguished from
USE ERROR
• New Clause 2 on normative references (per ISO-IEC Directives)
• Corrections of minor (editorial) errors and inaccuracies
• More emphasis on benefits and benefit-risk balance
• Explanation of risks related to usability, security and privacy
• Clarification of requirements for overall residual risk evaluation
• Clarification of requirements for post-production phase
• Additional guidance (in companion report ISO TR 24971)
Jos van Vroonhoven, Risk Management – Revision of ISO 14971 and Critical Issues for the Future, presented at
AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20, London UK
3/31/2019
17
Risk Management and Agile
SDMD Global 2019 11.-14. Feb. 2019
33
Public attention is focusing on risk management – in all fields
Risk work is inherently iterative
62304 requires us to "tailor" our approach
Reduce risk: decouple, reduce complexity
ISO 14971: broadened definitions; more risk/benefit emphasis
Flexibility of Agile: better risk management
Risk – part of the Agile Mindset
SDMD Global 2019 11.-14. Feb. 2019
34
NOT this:
But this:
3/31/2019
18
Tailor to consider risks
SDMD Global 2019 11.-14. Feb. 2019
35
Credit: Ahmed Sidky, “The Agile Mindset”
With risk
management!
New Risk Information All the Time
SDMD Global 2019 11.-14. Feb. 2019
36
3/31/2019
19
References
7th SDMD Europe 21.-24. Feb. 2017
37
Pat Baird; "Incorporating Risk Management into (software) Design & Development"; presented at IQPC's 18th Software Design for Medical Devices; 27-Oct-2014.
Chris Clearfield and András Tilcsik, Meltdown: Why Our Systems Fail and What We Can Do About It, New York, Penguin Press, 2018.
Ahmed Sidky, “The Agile Mindset”, available at http://www.softed.com/assets/Uploads/Resources/Agile/The-Agile-Mindset-Ahmed-Sidky.pdf
Jos van Vroonhoven, “Risk Management – Revision of ISO 14971 and Critical Issues for the Future,” presented at AAMI/BSI International Conference on Medical Device Standards and Regulations, June 19-20 2018, London UK
ANSI/AAMI/IEC 62304:2006, Medical device software – Software life cycle processes, 17-Jul-2006 (with ed. 1 revisions, Jun-2015).
ANSI/AAMI/IEC TIR80002-1:2009, Medical device software - Part 1: Guidance on the application of ISO 14971 to medical device software, 26-Oct-2009.
Contact information
7th SDMD Europe 21.-24. Feb. 2017
38
Brian Shoemaker, Ph.D.
Principal Consultant, ShoeBar Associates
199 Needham St, Dedham MA 02026 USA
+1 781-929-5927
http://www.shoebarassoc.com