regulatory landscape for ai-powered algorithms in digital … · 2019. 11. 1. · mastering the...

Proprietary & Confidential

Regulatory Landscape for AI-powered algorithms in Digital Pathology

1

20191030Esther Abels, VP RACA and strategic BD


Disclosures and Disclaimers

2

Esther Abels is:

• Full-time employee of PathAI

• Secretary and Board of Directors member of Digital Pathology Association (DPA)

• Chair of DPA regulatory and standardization taskforce

• Co-Founder of Digital Pathology Alliance

Opinions expressed in this presentation are solely those of the author or presenters, and do not necessarily reflect those of PathAI.

The information presented herein is not specific to any product of PathAI or their intended uses.

The information contained herein does not constitute, and should not be construed as, any promotion of PathAI products or company policies.


33

III

2005

887.7160056


Agenda

4

Background

What is needed

Regulatory Landscape


Agenda

5

Background

What is needed



6

Out of ~34,000 cases 3,000 errors or missed diagnoses


“Most medical treatments are designed for the "average patient" as a "one-size-fits-all-approach," which may be successful for some patients but not for others.

Precision medicine, sometimes known as "personalized medicine" is an innovative approach to tailoring disease prevention and treatment that takes into account differences in people's genes, environments, and lifestyles.

The goal of precision medicine is to target the right treatments to the right patients at the right time.” -FDA

Precision Medicine Initiative

The success of Precision Medicine depends on having accurate, reproducible, and clinically useful companion diagnostic tests to identify patients who can benefit from targeted therapies.


Definition of a CDx vs CoDxCompanion diagnostic (IVD-CDx)

• The companion diagnostic result is essential for the safe and effective use of the corresponding drug

• The companion diagnostic result is a yes/no answer to identify patients for treatment with the corresponding drug

• The drug label requires that a companion diagnostic is used to determine if the patient is eligible for treatment

•

Complementary diagnostic (IVD)

• The complementary diagnostic result provides significant information about the use of the corresponding drug

• The complementary diagnostic result guides physicians for informed patient dialogue about expected treatment benefit with the corresponding drug

• The drug label does not require complementary diagnostic testing

The use of an IVD companion diagnostic device with a therapeutic product is stipulated in the instructions for use in the labeling of both the diagnostic device and the corresponding therapeutic product, including the labeling of any generic equivalents of the therapeutic product

8


9

• First WSI systems received FDA 510(k) Clearance 2017 and 2019

• Different offerings from archiving to full solutions including algorithms

• By 2025 Digital Market is $887.7 million with 12% CAGR

Digital Pathology Market


10

• An emerging technology providing an image-based environment:• managing and interpreting the information • sharing data around the world

• Innovation and collaboration across specialties to develop new medical discoveries that will improve health

• New drug and biomarker discovery depends on tissue pathology, technological advances and applications could accelerate biomarker discovery and CDx

• Streamlining drug development process:• discovery, preclinical, clinical and post market trials• enabling communication on crucial findings in tissue based

toxicity and efficacy studies

What is Digital Pathology?


What is the concept

Computational pathology as an approach to diagnosis that incorporates multiple sources of data (e.g., pathology, radiology, clinical, molecular and lab operations); uses mathematical models to generate diagnostic inferences; and presents clinically actionable knowledge to customers. This vision goes beyond an informatics-centric view and leverages the core competency of pathology and the ability to effectively communicate clinically actionable knowledge

Source: A. Beck - https://mlhc17mit.github.io/slides/lecture6.pdf

An Emerging Definition David N. Louis, MD; Arch Pathol Lab Med—Vol 138, September 2014


Agenda

12

Background

What is needed



What is the issue?

• Tissue

• Terminology

• Harmonization, Standardization

• Ethics

• Regulations

• System interoperability

• Technology

• Economics

• …13


14

• Image Analysis Algorithm

• CADs

• Deep learning

• Artificial Intelligence

• Data Mining

• Big Data

• Neural Network

Terminology


User WorkFlow

Black box

Adaptive15

23

Considerations for algorithmsRealize that spell check and grammar check is a form of AI. Simply identifying that you’ve used a word more often than othersnot get in the way


16

24

Ethics in AI


Silver et al, 2016. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), pp.484-489.

Esteva et al, 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), pp.115-118.

17

Proof for machine learning


https://camelyon17.grand-challenge.org/results/

http://tupac.tue-image.nl/node/62https://www2.warwick.ac.uk/fac/sci/dcs/research/tia/glascontest/results/

18

Only natural to think of applications in pathology


• Advantages:• Improve patient care,

• Reduce errors and adverse events, and

• Encourage innovation

• Where does it start and where does it end?• Workflow?

• Within departments, hospitals? Across hospitals?

• EMR, Image sharing?

• Billing, insurance companies?

19

The need of interoperability?

Image Acquistion

Transfer Storage

Open system

Dispatching Analytics xxx

Applications

LIS, EMR

DICOM storage(Bulk)Display


20

FDA - interoperability


(19) ‘interoperability’ is the ability of two or more devices, including software, from the same manufacturer or from different manufacturers, to:

a) exchange information and use the information that has been exchanged for the correct execution of a specified function without changing the content of the data, and/or

b) communicate with each other, and/or

c) work together as intended;

13.5 Devices that are intended to be operated together with other devices or products shall be designed&manufactured in such a way that the interoperability and compatibility are reliable & safe.

Annex VIII Classification rules

1.2: If the device in question is intended to be used in combination with another device, the classification rules shall apply separately to each of the devices

1.4: Software, which drives a device or influences the use of a device, shall fall within the same class as the device. If the software is independent of any other device, it shall be classified in its own right.

IVDR - interoperability


Regulations for interoperability

22

FDA/AAMI summit 2012: Seven Clarion Themes1. Standardize to achieve success2. Align incentives, expectations, roles, and responsibilities3. Drive patient safety with a systems approach to design

and implementation4. Focus on human behavior first 5. Improve regulatory clarity6. Streamline clinical workflow to improve return on

investment7. Remove barriers with shared, continuous learning


What does it mean?

• FDA emphasizes two goals:• Protecting patient safety• Promoting transparency

• Manufacturers:• Intended Use and Anticipated users• Design control, Manage risks, Label, Test

• Users:• Labeling to understand how to use their connected device seamlessly

with other technology• Intended Use to understand the flow of data between devices• Minimal requirements / Specifications

23


Agenda

24

Background

What is needed



Clear benefits for AIs

• Improve workflow, quality, safety, efficiency

Considerations:

• Scanner• Proprietary file formats vs common file formats

• Intended and indications for use• Support tool vs automation / change in patient care / dependency on tool• Multiple indications regulated to populations

• Training data• What data are used to train and how to address bias when developing training

sets• Continuous learning vs locked down

Regulatory landscape


Whitepaper

• Image Analysis Algorithm

• CADs

• Deep learning

• Artificial Intelligence

• Data Mining

• Big Data

• Neural Network

26


FDA’s position paper

• Recommendations for working groups represent each indication area and include members from different associations in these working groups

27

Overarching observation:

• The pathologist is the ultimate special control


Considerations

How to assess risk and what kinds of studies are needed to substantiate the claims that will be made:

• What would be an acceptable and effective way to regulate AI?

• What would be the definition of "substantial change" that would trigger notification to the FDA?

• How to evaluate AIs and algorithms in an initial standard, and how to conduct testing to this standard to determine what would be a "clinically relevant change" to an AI?

• How much can be borrowed from the radiology regulatory area to use for regulation of AI in DP?

• How to assess risk with AI in DP device?

• What kinds of studies are needed to substantiate the claims that will be made? What is the general principal approach?

• What kind of use cases should be considered?

• What are potential regulatory special controls?

28


Considerations

• What is the Gold Standard? • Manual Read is currently the FDA’s standard

• Image to image

• How do we define the reference?

• Is stand-alone software validation sufficient?• Accuracy: Clinical Validation

• Precision: inter site and instrument precision (scanner, viewing, algorithm)

29


What is available?

30

The complexity of medical device software, together with the increasing connectedness of systems, results in emergent behaviors not usually seen in hardware medical devices. Challenges: - Medical device software might behave differently when deployed

to different hardware platforms.- Often an update made available by the manufacturer is left to the

user of the medical device software to install.- Due to its non-physical nature (key differentiation), medical device

software may be duplicated in numerous copies and widely spread, often outside the control of the manufacturer.

- Furthermore, there are lifecycle aspects of medical device software that pose additional challenges. For instance:

- Have rapid development cycles,- Introduce frequent changes to their software, and- Deliver updates by mass and rapid distribution

Software as a Medical Device SaMD is defined as software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device


What is available?

31

21st Century Cures Act amended the Federal Food, Drug, and Cosmetic Act (FD&C Act), which excludes certain software functions from the definition of device, i.e. software functions that meet all of the following four criteria: 1. Not intended to acquire, process, or analyze a medical image or a signal

from an in vitro diagnostic device or a pattern or signal from a signal acquisition system;

2. Intended for the purpose of displaying, analyzing, or printing medical information about a patient or other medical information (such as peer-reviewed clinical studies and clinical practice guidelines);

3. Intended for the purpose of supporting or providing recommendations to a health care professional about prevention, diagnosis, or treatment of a disease or condition; and

4. Intended for the purpose of enabling such health care professional to independently review the basis for such recommendations that such software presents so that it is not the intent that such health care professional rely primarily on any of such recommendations to make a clinical diagnosis or treatment decision regarding an individual patient.


What is available?

32

• Reduce inefficiencies,• Improve access,• Reduce costs,• Increase quality, and,• Make medicine more

personalized for patients.

Many medical devices now have the ability to connect to and communicate with other devices or systems


What is available?

33

Withdrawn: Guidance for the Submission of Premarket Notifications for Medical Image Management Devices

Not subject to FDA regulatory requirements: software functions that are solely intended to transfer, store, convert formats, and display medical device data and results, including medical images, waveforms, signals, or other clinical information.

Software functions that analyze or interpret medical device data in addition to transferring, storing, converting formats, or displaying clinical laboratory test or other device data and results remain subject to FDA’s regulatory oversight


What is available?

• FDA uses RWD and RWE to monitor PM safety, AEs and regulatory decisions

• Developers use for innovation

• Can we use it in Active Post Market Follow up?

• Continuous learning or release of a new device?

34


What is available?

35

Overview of Priority Areas


So all we need is…

36

• So, all we need is:• Stakeholders to collaborate

• Clarity and consensus on terminology, especially on intended use• Feature detection, classification, quantitation, Diagnostics

• Is the algorithm interactive (risks of user error)

• What is the intended use (patient management)

• How to use the result (confirm)

• Interoperability Standardization Harmonization

• Algorithms and technical testing to replace the burdensome performance studies

TPA Analytical Clinical


Resources used for this presentation

37

• https://www.fda.gov/regulatory-information/search-fda-guidance-documents/vitro-companion-diagnostic-devices

• https://www.fda.gov/media/109618/download

• https://onlinelibrary.wiley.com/doi/full/10.1002/path.5331

• https://www.fda.gov/medical-devices/digital-health/digital-health-software-precertification-pre-cert-program

• https://www.fda.gov/medical-devices/digital-health/software-medical-device-samd

• http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf


• https://www.fda.gov/medical-devices/digital-health





• http://www.fdalawblog.net/2019/02/congress-floats-discussion-draft-of-ivd-legislation/

• https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence

• https://www.fda.gov/medical-devices/science-and-research-medical-devices/cdrh-regulatory-science-priorities

• https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/vitro-companion-diagnostic-devices

https://www.fda.gov/media/109618/download

https://onlinelibrary.wiley.com/doi/full/10.1002/path.5331

https://www.fdahttps/www.fda.gov/medical-devices/digital-health/digital-health-software-precertification-pre-cert-program

https://www.fda.gov/medical-devices/digital-health/software-medical-device-samd

http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf


https://www.fda.gov/medical-devices/digital-health





http://www.fdalawblog.net/2019/02/congress-floats-discussion-draft-of-ivd-legislation/

https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence

https://www.fda.gov/medical-devices/science-and-research-medical-devices/cdrh-regulatory-science-priorities

https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device


38

regulatory landscape for ai-powered algorithms in digital … · 2019. 11. 1. · mastering the...

Documents