agile for software as a medical device

O R T H O G O N A L

orthogonal.io

A G I L E I N A Q M S

Prepared for Bayer, October 2016

ABOUT ME

2 Agile in a QMS orthogonal.io

Bernhard Kappe

CHIEF EXECUTIVE OFFICER, ORTHOGONAL

Bernhard is an expert in lean innovation and the

application of agile methods in regulated

industries. He is the author of the eBook Agile in

an FDA Regulated Environment and a frequent

conference speaker on wireless medical devices

and mHealth. He works with Orthogonal clients

on connected care strategy and at the application

of lean and agile methods.

http://pathfindersoftware.com/agile-in-an-fda-regulated-environment/

orthogonal.ioAgile in a QMS3

CONNECTED CARE TECHNOLOGIES

MEDICAL INDUSTRY CLIENTS


PRESENTATION AGENDA


Introduction

Agile in 5 Minutes

Agile for Medical Devices

How to be Agile in a QMS

Automation

How to be Agile in a QMS (for Non Developers)

Agile Implementation Mistakes

Resources

A G I L E I N F I V E

M I N U T E S

A Quick recap of the why and what of Agile


CLASSIC WATERFALL

7

Works great when

problems and

solutions are well

known!

THE COST OF CHANGE IN

WATERFALL

8

ITERATIVE METHODS

Fast Feedback Loops to Reduce Uncertainty and

Efficiently Manage Change:

• OODA Loops

• Shewhart/Deming Cycles (PDCA)

• Six Sigma (DMAIC)

• Toyota Production System

• Lean Manufacturing

• Lean Startup

• Lean UX


AGILE APPROACH


• Break functionality into the smallest units of customer value: the user

story.

• Organize the stories into a prioritized list called a backlog

• Organize development efforts into short (one to two week0 efforts called

iterations or sprints.

• The goal of each iteration: Demonstrate working software

• Take feedback and improve the process through retrospectives and

the software through refactoring.

• Self Organizing Teams

BEHAVIOR DRIVEN DEVELOPMENT

(BDD)


“Behavior-driven development is an

“outside-in” methodology. It starts at

the outside by identifying business

outcomes, and then drills down into

the feature set that will achieve those

outcomes. Each feature is captured as

a “story”, which defines the scope of

the feature along with its acceptance

criteria.”

• —Dan North

Title (one line describing the story)

Narrative:

As a [role]

I want [feature]

So that [benefit]

Acceptance Criteria: (presented as Scenarios)

Scenario 1: Title

Given [context]

And [some more context]...

When [event]

Then [outcome]

And [another outcome]...

Scenario 2: ...

AGILE ITERATIONS

12

• Story Baking – Detailed specification of user stories, including acceptance criteria and ties to FURPS+.

• Story Points – Unit of estimation measuring complexity and size.

• Iteration Planning Meeting – a negotiation between the team and the product owner about what the team will do during the next

Iteration. The product owner and all team members agree on a set of Iteration goals, which are used to determine which product

backlog items to commit from the uncommitted backlog to the Iteration. Often new backlog items are defined during the meeting.

• Daily Standup Meeting - a 10-minute daily meeting for the team

• Pair Programming – two programmers work together on one user story.

• Test Driven Development (TDD) – repetition of a very short development cycle: first the developer writes a failing automated test case

that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to

acceptable standards.

• Continuous Integration – code is checked into a continuous integration environment/build server, which runs automated tests (unit,

integration, static and dynamic analysis, etc.)

• Demo/Design Review - On a per iteration basis, functional software is reviewed by stakeholders, including source code review

• Retrospective - what went well and what to improve in the next Iteration.

AGILE CADENCE

THE COST OF CHANGE CURVE

REVISITED

14

THE COST OF CHANGE CURVE

REVISITED

15

A G I L E F O R

M E D I C A L

D E V I C E S

Why we need Agile and why it’s taken so long to adopt it


CHANGE AND UNCERTAINTY

• New Product Development

• Complex Systems

• System Interdependencies

• More Users

• Technology Changes

• Market Changes

• Regulatory Changes



MEDICAL DEVICE CHALLENGES

The Target

Business Value/

Real-World

Outcomes

Patient Safety

Process

Efficiency

Regulatory

Compliance

BARRIERS TO AGILE

• Agile evolved in software, and Medical Devices used to not have much

software.

• Regulations, standards and guidance documents (CFR 21 Part 820, ISO

13485, IEC 62304) were mostly defined before agile, and are written in a

waterfall way.

• Medical Device regulations and standards have additional requirements

(risk management, design controls, verification and validation) that add

complexity to Agile.


THE FDA DESCRIBES RATHER THAN

PRESCRIBES

“The quality system requirements do not dictate the types of design process that a manufacturer must use.

Manufacturers should use processes best suited to their needs. However, whatever the processes may be,

it is important that the design controls are applied in an appropriate manner.”

FDA Design Control Guidance for Medical Device Manufacturers, p. 8

“Based on the intended use and the safety risk associated with the software to be developed, the software

developer should determine the specific approach, the combination of techniques to be used, and the level

of effort to be applied. While this guidance does not recommend any specific life cycle model or any

specific technique or method, it does recommend that software validation and verification activities be

conducted throughout the entire software life cycle.”

General Principles of Software Validation (GPSV), p. 1


RESOURCES: AAMI TIR45:2012

• Provides recommendations for complying

with international standards and FDA

guidance documents when using agile

practices to develop medical device

software.

• Agile brings value to medical device software: continuous focus on safety, risk management and control, impact analyses, . . .

• Designs coherent (locally constant) and highly cohesive (vs. tightly coupled)

• Recognized as a standard by the FDA in 2013

21

H O W T O B E

A G I L E I N A Q M S

Mapping Agile to IEC 62304 and CFR 21 Part 820


IEC 62304: MEDICAL DEVICE

SOFTWARE LIFECYCLE

PROCESSES

23

8. Software Configuration Management

9. Software Problem Resolution

7. Software Risk Management

5. Software Development Process

5.2 SW

Requirements

Analysis

5.3 SW

Architectural

Design

5.4 SW

Detailed

Design

5.7 SW

System

Testing

5.8 SW

Release

5.1 SW

Development

Planning

5.5 SW Unit

Implement.

& Verification

5.6 SW

Integration &

Integration

Testing

FDA DESIGN CONTROLS GUIDANCE

Footer Text25

V-MODEL: BENT WATERFALL OR

AGILE?


User Requirements

System

Requirements

Specification

Software Detailed

Design

User Acceptance

Testing

System &

Integration

Testing

Unit Testing

Quality Assurance / Risk Control • Relationship between

artifacts are time-

independent

• Finish to finish parallelism

(vs. start to finish)

• Design Inputs ↔ Design

Outputs

• Synchronize Approvals

VERIFICATION DRIVEN AGILE


Verification Test

• Avoids different interpretations of

requirements and specification that are

typical in waterfall development

• This avoids “verification hell” that can double

the length and cost of a project.

• Allows for continued feedback from end

users on working software – further reducing

risk

R I S K

M A N A G E M E N T

I N A G I L E

It’s Iterative


THE ITERATIVE NATURE OF

RISK


“Risk can be introduced throughout the product lifecycle, and risks that

become apparent at one point in the life-cycle can be managed by

action taken at a completely different point in the life cycle.”

ISO 14971: 2007 Medical devices -- Application of

risk management to medical devices


ITERATIVE RISK MANAGEMENT

• Iterative Risk Analysis Takes inputs from and

provides outputs to Human Factors and

Development

• Risk Mitigations are reflected in new stories.

• Risk Mitigations are verified and validated

• The Iterative Risk Management cadence can

be different than the development cadence

HUMAN FACTORS

ENGINEERING


• Lean and agile cadence ships insights and/or UI each iteration

• User experience architecture maps and assesses human

interaction models and requirements

• Usability studies conducted with real users to identify and assess

the effectiveness of vital system interactions

• Interactive prototypes to simulate interactive functions of the

software

• User interface design of software interface using color, animation,

text and graphics

• Result:

– User Engagement

– Real-World Outcomes

A U T O M A T I O N

The key to maintaining quality at speed


TESTING AUTOMATION

• Full Code Coverage: Unit, Integration, Acceptance Testing built into continuous

integration environment.

• White-box and black-box testing for all elements of the stack,

from hardware to firmware to web services

• Result:

– Maintain Velocity on Larger, More Complex Projects

– Faster Parallel Development of Hardware and Software

– Faster, More Efficient Maintenance


TDD+ BDD + TESTING AUTOMATION

34

From this:

… to this

AGILE DESIGN CONTROLS


H O W T O B E

A G I L E I N A Q M S( F O R N O N -

D E V E L O P E R S )

It’s not Animal Farm


PIGS AND CHICKENS

37


Agile development is

wasted If none of the

related processes can

iteratively provide and

receive input.

Risk

Management

Human

Factors

Verification &

Validation

Post Market

Surveillance

Product

Management

Document

Managment

PRODUCTION PROCESS


PUTTING IT ALL TOGETHER


Business Value/

Real-World

Outcomes

Patient Safety

Process

Efficiency

Regulatory

Compliance

Better usability,

adherence/compliance = Improved

Engagement and Outcomes

Iterative Risk Management = Safer

Products and Reduced

Regulatory Risk

Faster Development

Streamlined Maintenance and

Manufacturing

Platform Development/Reusability

A G I L E

I M P L E M E N TAT I O N

M I S T A K E S

Common mistakes we’ve seen, and how to avoid them.


orthogonal.ioCompany Overview42Area Finding (Gap) Best Practice Consequences of Gap Recommended Approach Dependencies

Quality Management Development Methodology

and Quality Management

System (QMS) are Tightly

Coupled

Loose Coupling of Quality

Management and Development

Approach.

Changes in development methodology as

needed by individual projects may result in

changes and re-approvals of parts of the

QMS which takes time and consumes

significant organizational resources

Allow for different development methodologies by

stating in QMS that any methodology based on a

standard and that meets relevant regulations is

allowed as long as it is approved by Quality.

Consider templated approach at SDP level.

Approvers of QMS (Executive

management, Quality,

Regulatory)

Risk Management Risk Management is not

iterative and integrated into

agile process

QMS states that Risk Management

Plan determines how risk control is

performed throughout the product

lifecycle.

Risk Management that is not iterative and

integrated into agile process may result in an

"Agile Falls" lifecycle where risk "in-stream"

risk analysis and mitigation suffers and risk

management activities become a bottleneck

for other project activities to continue

QMS does not specify risk management

methodology, only that risk control activities will be

performed throughout the project lifecycle as

required by relevant regulations. Risk

Management Plan outlines risk control activities

and their rationales on a per project basis.

QMS Risk Management

Process.

Verification and Validation Verification and Validation

(VnV) is not iterative and

integrated into agile process

QMS states that Validation Plan

determines how VnV is performed

throughout the product lifecycle

VnV that is not iterative and integrated into

agile process may result in an "Agile Falls"

lifecycle where verification testing fails to

indicate when an issue appears and VnV

activities become a bottleneck for other

project activities to continue.

QMS includes a VnV process definition that is

iterative and can be integrated into agile

processes.

QMS Verification And

Validation Process.

Requirements Management Requirements PRDs do not

incorporate Agile User

Stories

Include "story points" in User Stories

as user requirements

User requirements may miss important user

requirements

Include "story points" in User Stories as user

requirements. Also consider using Human Factors

Engineering and Stakeholder (e.g. clinician and

patient) flow analyses as a way of logically

grouping User Stories.

QMS Requirements

Management Process

Requirements Management Inadequate Software Tools

for Requirements

Management and

Traceability

Analyze and determine which tools

for Requirements Management and

Traceability would best fit the

organization.

Poorly specified requirements, inadequate

traceability; inability to analyze logically

categorize requirements

Follow recommendation for which tools to use for

Requirements Management and Traceability that

would best fit the organization

QMS

Testing Automation No unit or integration tests Implement a "test first" approach to

Unit Tests when specifying low level

requirements (i.e. how would this

requirement be tested?).

Poorly specified and tested requirements. Automate unit and integration testing in a

Continuous Integration Environment

QMS Test Plan, Tools

Validation

Testing Automation No Continuous Integration

Environment

Specify interface requirements and

sub-system integration tests during

system development.

Poorly specified and tested requirements. Automate Continuous Integration Environment;

justify when able to do so.


Validation

Testing Automation No Acceptance tests Specify user acceptance tests when

constructing User Requirements.

Poorly specified and tested requirements. Automate acceptance testing (as reasonably

possible)


Validation, Continuous

Integration EnvironmentAcceptance test automation

#1: TIGHT COUPLING

Finding: Development Methodology and Quality Management System (QMS) are Tightly Coupled

Consequences: Constraints on Implementing Agile. Changes in development methodology as needed by

individual projects may result in changes and re-approvals of parts of the QMS which takes time and

consumes significant organizational resources.

Best Practice: Loose Coupling of Quality Management and Development Approach.

Recommended Approach: Allow for different development methodologies by stating in QMS that any

methodology based on a standard and that meets relevant regulations is allowed as long as it is approved

by Quality. Consider templated approach at SDP level.

Dependencies: Approvers of QMS (Executive management, Quality, Regulatory)


#2: WATERFALL RISK MANAGEMENT

Finding: Risk Management is not iterative and integrated into agile process

Consequences: Risk Management that is not iterative and integrated into agile process may result in an "Agile

Falls" lifecycle where risk "in-stream" risk analysis and mitigation suffers and risk management activities become a

bottleneck for other project activities to continue

Best Practice: QMS states that Risk Management Plan determines how risk control is performed throughout the

product lifecycle.

Recommended Approach: QMS does not specify risk management methodology, only that risk control activities

will be performed throughout the project lifecycle as required by relevant regulations. Risk Management Plan

outlines risk control activities and their rationales on a per project basis.

Dependencies: QMS Risk Management Process.


#3: V&V NOT INTEGRATED

Finding: Verification and Validation (V&V) is not iterative and integrated into agile process

Consequences: V&V that is not iterative and integrated into agile process may result in an "Agile Falls"

lifecycle where verification testing fails to indicate when an issue appears and V&V activities become a

bottleneck for other project activities to continue.

Best Practice: QMS states that Validation Plan determines how V&V is performed throughout the product

lifecycle.

Recommended Approach: QMS includes a V&V process definition that is iterative and can be integrated

into agile processes.

Dependencies: QMS Verification And Validation Process


#4: NO USER STORIES IN THE

PRDFinding: Product Requirement Documents do not incorporate Agile User Stories

Consequences: Requirements may miss important user requirements, and require additional translation to

agile-usable requirements

Best Practice: Include story definitions in User Stories as user requirements

Recommended Approach: Include story definitions in User Stories as user requirements. Also consider

using Human Factors Engineering and Stakeholder (e.g. clinician and patient) flow analyses as a way of

logically grouping User Stories.

Dependencies: QMS Requirements Management Process


#5: NO AGILE RM TOOLS

Finding: Inadequate Software Tools for Requirements Management and Traceability

Consequences: Poorly specified requirements, inadequate traceability; inability to analyze logically

categorize requirements, lots of extra work.

Best Practice: Analyze and determine which tools for Requirements Management and Traceability would

best fit the organization.

Recommended Approach: Follow recommendation for which tools to use for Requirements Management

and Traceability that would best fit the organization

Dependencies: QMS, Tools Validation


#6: NO TDD OR BDD

Finding: No or insufficient unit, integration and acceptance test coverage.

Consequences: Poorly specified and tested requirements. Bugs, more laborious maintenance

Best Practice: Implement a "test first" approach to Unit Tests, Integration Tests and Acceptance tests

when specifying requirements (i.e. how would this requirement be tested?).

Recommended Approach: TDD and BDD, Automate unit, integration and acceptance testing in a

Continuous Integration Environment

Dependencies: QMS Test Plan, Tools Validation


#7: NO PAIRING

Finding: No Developer Pairing

Consequences: Slower development, greater dependency on individual developers (lower “bus number”),

no built-in design review, less ability to scale projects up and down, higher training costs

Best Practice: Implement Promiscuous Pairing.

Recommended Approach: Training and Mentoring by developers experienced with pairing.

Dependencies: Developers who can pair


R E S O U R C E S

A selection of connected care solutions Orthogonal has

developed for the medical industry.


RESOURCES: AAMI TIR45:2012

• Provides recommendations for complying

with international standards and FDA

guidance documents when using agile

practices to develop medical device

software.

• Agile brings value to medical device software: continuous focus on safety, risk management and control, impact analyses, . . .

• Designs coherent (locally constant) and highly cohesive (vs. tightly coupled)

• Recognized as a standard by the FDA in 2013

51

RESOURCES: ORTHOGONAL

EBOOK• Addresses the shortcomings of waterfall

development specifically in regulatory

environments.

• How agile development meets the

safety, reliability and regulatory needs of

the medical device and diagnostics

industry.

• How agile development can help ensure

delivery of successful software.

52

Q U E S T I O N S ?

Bernhard Kappe

[email protected]


orthogonal.io

AGILE ITERATION AND DOCUMENTATION