NICE Scientific advice &

Diagnostics assessment programme

September 23, 2014

How to generate evidence to support value claims for diagnostics

Dr Grace Jennings and Dr Sarah Byron

What is NICE?

•prevent, diagnose and treat disease and ill health in most effective ways

•reduce inequalities and variation

•ensure quality and value for money for the NHS

An independent institute that identifies how to:

Accountabilityfor

reasonableness

Accountabilityfor

reasonableness

NICE’s Procedural Principles

A NICE Process

Decision Committee

Public Consultation

Decision

StakeholderPerspectives

Evidence submissions

Independent review

Input from topic experts

DIFFICULT CHOICES FORDECISION MAKERS

HTA

Scarce resourcesScarce resources

Product Market

Evidence-based way of guiding the efficient allocation of health care resources

Health Technology Assessment (HTA)

Evidence

Policy-making

Benefits of engaging with NICE

How NICE makes decisions

Two key questions asked by NICE

• How well does the technology work compared to established practice in the National Health Service (NHS)?

• How much does this course of action cost compared to established practice in the NHS?

Decision-making at NICE

Value Proposition

Impact on health system

resources

Impact on health system

resources

Incremental benefit for patients

Incremental benefit for patients

Value varies depending on your perspective

Perspective

NICE takes the perspective of the National Health Service (NHS) and

Personal Social Services (PSS)

Understand the perspective of your decision maker and know what question they want to answer

Defining the clinical question: Components

• Population• Intervention• Comparator• Outcomes

PICO

Population Usually the patients indicated in the marketing authorisation

Intervention Technology to be appraised

Comparators Established practice in the NHS

Outcomes Outcomes which have an impact on: - survival - health related quality of life (HRQoL)

Putting your case together

Navigating NICE

Product developers may be interested in ensuring their clinical development studies and other plans generate the evidence which is relevant to NICE

Why seek scientific advice?

IIncrease the likelihood that clinical trials and other research ncrease the likelihood that clinical trials and other research activities undertaken meet NICE evidence requirementsactivities undertaken meet NICE evidence requirements

Typical issues raised for advice

Clinical Trial Programme• Study population• Comparators• Acceptability of endpoints, surrogate endpoints • Trial design• Appropriateness of health-related quality of life and other Patient

Reported Outcomes• Positioning in the clinical pathway

Economic Evaluation• Plans for using specific economic models• Sources of data , observational studies, analyses

Additional issues raised in previous diagnostics advice projects

• Study design• Diagnostic cohort design, case-control study, cluster randomisation trial• Sensitivity and specificity• Diagnostic accuracy• Stratification by risk factor• Clinician blinding

• Outcomes relevant to NICE• Role of new diagnostic in existing treatment pathway• Assessment of cost-effectiveness• Model structure and input assumptions

Key stages of advice processes

Advice for developers of screening tests

• Screening not part of NICE remit• Screening and diagnostic tests often similar• Diagnostic test may be used as screening test in treatment pathway

• National Screening Centre• Part of Public Health England

• Areas of screening needs explored• Models developed by external academic group• New initiative – advice from NICE Scientific Advice in conjunction

with NSC for screening tests

Questions?

Medical Technologies

EvaluationProgramme

(MTEP)

DiagnosticsAssessment Programme

(DAP)

Technology Appraisals

(TA)

Programmes at NICE

Topic SelectionSTEP 1

The Company submits a notification form to Medical Technologies Evaluation Programme that details:

•Product description• Patient population• Current management and comparator(s)• Claimed patient benefit • Claimed healthcare system benefit• Claimed sustainability benefit • Costs• Patient safety

Notifying a product to NICE

STEP 2.

NICE produces a briefing note for MTAC which decides whether the technology is suitable for the evaluation at NICE

STEP 3.

If selected, MTAC routes the technology to the appropriate Programme.

Process takes ~10 weeks

Questions about the process and eligibility? Contact MTEP:

medtech@nice.org.uk

MTEP vs DAP

DAP and MTEP encourage further research into promising technologies

• Specialist programme to undertake complex assessments of diagnostic technologies

• Decision making by independent Diagnostics Advisory Committee

• Assessment of single or multiple technologies

• No formal manufacturer submission required

• Systematic review of evidence and modelling to estimate outcome benefits and cost effectiveness is undertaken as part of the assessment

Diagnostics Assessment Programme

Overview of assessment process

Two key questions asked by NICE

• How well does the technology work compared to established practice in the National Health Service (NHS)?

• How much does this course of action cost compared to established practice in the NHS?

Scoping Single technology notified and referred to DAP

DAP technical lead •Diagnostic pathway•Care pathway•Alternative technologies•Relevant population(s) •Costs•Outcomes •Potential equality issues •Potential implementation barriers

Scoping workshop

Registered stakeholders

ASG meeting

Specialist Committee membersStanding DAC member

Draft scope

Revised scope

Final scope

Assessment of single or multiple technologies

1. How is the condition

managed in the NHS?

3. What does my product deliver?

2. Where does my product fit in the care pathway?

Understanding benefits: diagnostics

False negative?

False positive?

•The treatment pathway or the range of pathways must be understood for the value of the diagnostic to be assessed

•Test side effects should be included

Patient benefits rarely arise from the diagnostic directly – they come mainly from treatments informed by the

diagnostic

Understanding benefits: diagnostics

Assessment

Regulator

• Product safety: laboratory testing with clinical trial data for devices with greatest risk

• Evidence of efficacy: does the device meet its intended purpose? (not necessarily from comparative studies)

HTA•Evidence on clinical effectiveness (compared to established practice): trials, evidence synthesis

•Evidence on cost effectiveness: trials, modelling

•Evidence on relative safety/adverse events

What data do you need?

Trueman P et al 2011

• The ideal evidence would be a good quality ‘end-to-end’ study – follows patients from testing, through treatment, to final outcomes• Typically not available for diagnostics • Search for data on test accuracy, direct outcomes from the test,

indirect health outcomes from the test result, and costs• Identified evidence can then be combined through a linked

evidence approach

Diagnosticaccuracy

Impact onoutcomes

Impact on treatmentdecisions

Evidence considerations

Evidence hierarchy

Study design

• Outcomes: patient focussed outcomes are particularly important, as opposed to intermediate or surrogate outcomes

• e.g. a reduction in tumour size will be given less weight than evidence about clinical benefit such as improved survival or quality of life

• Size: Studies with larger numbers of patients will usually be preferred as estimates of benefits and harms will be more accurate

• Duration: Studies should have sufficient follow up to capture final outcomes where possible

• e.g. very important for prognostic tests

Diagnostic tests: Outcomes data

IIdeally comparative ‘end-to-end’ clinical studies including the test and subsequent treatments should be conducted

Test side effects should be included

Identify studies on the effectiveness of those subsequent treatments

Use a systematic approach to identifying relevant studies

Not possible

Diagnostic tests: Outcomes data

• Measurements of test accuracy are necessary:

Condition as determined by “Gold Standard”

Condition positive

Condition negative

Test outcome

Test outcome positive

True Positive

False Positive

PPV

Test outcome negative

False negative

True Negative

NPV

Sensitivity Specificity

Diagnostic tests: Outcomes data

Cut off points

Example:SonoVue (sulphur hexafluoride microbubbles)Contrast agent for contrast-enhanced ultrasound imaging of the liver

Characterising incidentally detected

focal liver lesions

Detection of potential liver metastases

Characterising focal liver lesions

(cirrhosis)

No end-to-end studies available

High quality accuracy data – SonoVue vs CT and MRI

Relevant evidence on care pathway and outcomes

Characterising incidentally detected

focal liver lesions

Detection of potential liver metastases

Characterising focal liver lesions

(cirrhosis)

Cost effective

Adoption recommendation

Slightly less cost effective than CT and MRI

Adoption recommendations

where CT and MRI not

appropriate+

Research recommendations

to explore potential broader

applicability

Decision making

Recommendations

• Adoption recommendations

• Research recommendations

• Not recommended

Guidance development

• Decision making in presence of uncertainty• Public consultation can change decision making• Clarity in recommendations on indicationo Rule-in / rule-out / diagnosis / monitoringo SettingSupported by evidence, minimise risk of indication creep and

inappropriate use of tests that may lead to misdiagnosis Cost-effective use of NHS resources

• ‘Committee considerations’ describe uncertainties and rationale behind decision-making.

Diagnostic Guidance (DG5)

SonoVue (sulphur hexafluoride microbubbles) – contrast agent for contrast-enhanced ultrasound imaging of the liver

Contrast-enhanced ultrasound with SonoVue is recommended for

characterising incidentally detected focal liver lesions in adults in whom an

unenhanced ultrasound scan is inconclusive. An unenhanced ultrasound

scan in which a focal liver lesion is detected, but not characterised, is

defined as inconclusive.

Examples of recommendations

Recommendations

• Adoption recommendations

• Implementation support

• Health Technologies Adoption Programme

• Research recommendations

• NICE research commissioning

• Not recommended

Technology Appraisal Programme

Diagnostics Assessment Programme

Appraisals of new and existing medicines and treatments

Specialist programme to undertake complex assessments of diagnostic technologies

Topics from department of health Topics from manufacturers or clinical sponsors

Assesses single or multiple technologies

Assesses single or multiple technologies

STA - Manufacturer submission

MTA – Manufacturer submission and evidence gathered by academic group

No formal manufacturer submission

Structured information requestEvidence gathered by academic group

Evidence assessed by independent external group

Evidence assessed by independent external group

STA – 35 weeksMTA – 62 weeks

62 weeks

NICE: Companion diagnostics

NICE: Companion diagnostics

• Alternative technologies• Timing is an important factoro Appraisal programme coordinates publication of guidance with market

authorisation of drug o CDx may be developed simultaneously with drugo CDx can come to market years after drug approval or may already be

availableo Rapid development of CDx and supporting technologies

• In January 2013, NICE published update to methods guide• Costs of CDx testing incorporated into evaluation of clinical and

cost effectiveness

• Sensitivity analysis to assess impact of CDx cost on cost effectiveness of pharmaceutical

• Diagnostic accuracy can be examined and incorporated in cost effectiveness analysis

• Potential issues of alternative CDx can be highlighted in guidance without assessment of evidence

Technology Appraisals: Companion diagnostics

Example of CDx in TA programme

• TA 208 Trastuzumab for HER2-positive metastatic gastric cancer• MA included testing with fluorescence in situ hybridisation (FISH) then revised to include silver

in situ hybridisation (SISH)• Timing of MA meant that only FISH was included in NICE appraisal

• Trial used parallel testing strategy• Sequential testing strategy in manufacturer’s model

• Only ICH2 positive received FISH test• ERG scenario analyses for both sequential and parallel testing strategies

• Sequential ICER £66,982 per QALY• Parallel ICER £71,637 per QALY due to increased incremental costs

• Committee concluded that sequential testing was most appropriate for people with metastatic gastric cancer

Example of CDx in DAP programme

• EGFR-TK mutation testing in adults with locally advanced or metastatic non-small-cell lung cancer

• Evidenceo Two tests used in clinical trialso Three tests had accuracy data o Linked to clinical trial data

o Remaining tests had no trial or accuracy data

o Included a survey of labs providing EGFR-TK testing o test characteristics and costs

o Data from an EGFR-TK national external quality assurance scheme study

EGFR testing - Recommendations• 5 tests recommended but insufficient evidence to make

recommendations for others

• Key issues:

• Test validation

• Competent execution

• Participation in external quality assurance scheme

• Research recommendation

• Studies comparing different EGFR-TK mutation methods that link to patient outcomes

• Many assumptions in assessment

Diagnostics guidance (http://www.nice.org.uk/dg9)

Ways of assessing alternative CDx:

• Only assess CDx used in clinical trial • include general commentary on use of alternative CDx tests • Where appropriate, undertake sensitivity analysis on

diagnostic accuracy and cost to understand the importance of CDx in relation to patient outcome benefits and cost effectiveness

Technology Appraisal programme

• Assess each alternative CDx / treatment package separately • Separate clinical and cost effectiveness analyses Diagnostics Assessment programme

Summary

Health Technology Adoption Programme (HTAP)Main objectives:

•Increase the uptake of NICE recommended technologies

•Establish an agreed approach for NICE to develop effective partnerships with Academic Health Science Networks (AHSN) •To support the expansion of the Medical Technologies Industry in the UK by providing adoption advice to suppliers

Two approaches to adoption in the NHS:

57

• Site demonstrator projects for commonly used technologies

• Adoption projects for more complex, less commonly used technologies

Produce adoption support packs to accompany guidance produced in MTEP, DAP, TA and Quality Standards (QS).

Please contact Jae Long (jae.long@nice.org.uk), Assistant Project Manager for HTAP, for further details.

Key contacts

• NICE Scientific Advice• Richard Chivers (richard.chivers@nice.org.uk)• http://www.nice.org.uk/about/What-we-do/Scientific-advice• NICE/Medilinks seminar http://www.medilinkem.com/events/events-calendar/2014/10/21/nice-medtech-workshop

• NICE DAP• Sarah Byron (sarah.byron@nice.org.uk)• http://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-diagnostics-guidance

• NICE Medical Technologies Evaluation Programme• Lee Dobson (lee.dobson@nice.org.uk)• http://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-medical-technologies-evaluation-

programme • Health Technologies Adoption Programme

• Jae Long (jae.long@nice.org.uk)• http://www.nice.org.uk/About/What-we-do/Into-practice/Health-Technologies-Adoption-Programme

Any questions?