· web view2020. 11. 5. · sciences sector. as this process unfolds, it is important to clearly...

National Institute for Health and Care Excellence

The NICE methods of health technology evaluation: the case for change

Introduction1. Ensuring rapid access to clinically and cost-effective health technologies is

critically important to patients and their families, the NHS and the life sciences

industry. For more than 20 years, NICE’s health technology evaluations have

played a key role in supporting access to medicines, medical technologies and

diagnostics, guided by the NICE charter and NICE principles. As health

technologies rapidly advance, so too do methods of evaluation. To continue to

support the needs and objectives of all parts of the healthcare and life

sciences ecosystem, NICE must ensure that its methods remain cutting edge

and future proof.

2. Within that ecosystem, NICE guidance both informs shared decision making

and shapes how companies put forward their technologies. NICE has an

opportunity to continue supporting a productive relationship between the NHS

and the life sciences industries and to help align their ambitions with the

expectations that the public has of the NHS. In this way, NICE can help ensure

that the NHS gets the best value out of the products it uses while supporting

the life sciences industry in its journey through regulation, health technology

evaluation and patient access. Changes to NICE’s methods in isolation will not

overcome all the challenges for accessing new health technologies in the

NHS, in terms of how they are developed, evaluated and supported for

adoption. But they present an important opportunity to build on NICE’s success

to date, and further support patients, the life sciences industry and the NHS

now and in the future.

3. As the UK comes out of the Transition Period after having left the European

Union, it will develop a new regulatory and access environment with the life

National Institute for Health and Care ExcellenceThe NICE methods of health technology evaluation: the case for change 1 of 62Published 6 November 2020

https://www.nice.org.uk/about/who-we-are/our-principles

https://www.nice.org.uk/about/who-we-are/our-charter

sciences sector. As this process unfolds, it is important to clearly show how

NICE’s methods can support early patient access and encourage companies

to launch their products first in the UK.

4. NICE’s methods of health technology evaluation are a crucial tool that

underpins the robust, consistent and transparent decision making on which

NICE’s health technology guidance is based. The methods are pivotal in

supporting NICE’s objective to secure rapid and fair access to clinically and

cost-effective health technologies in the NHS. The continual evolution of

NICE’s methods over time is central to their success to date. A series of major

updates to NICE’s health technology evaluation methods (most recently in

2013), and other changes in between, have contributed to important

improvements in the methods over time. They have also ensured that NICE

has been able to adapt to emerging changes in healthcare and health

technology evaluation.

5. It is in this context that the current review of the methods of health technology

evaluation began. With important developments in science and technology

leading to rapid changes in healthcare and health technology evaluation, a

wide-reaching assessment of NICE’s health technology evaluation methods is

both appropriate and timely. The work formally started in July 2019, with broad

input from a wide range of stakeholders.

6. This paper should be read with the NICE Board paper of July 2019, which

describes the scope of the review and governance arrangements. The current

methods, on which the proposals for change build, can be found on the NICE

website, in the:

guide to the methods of technology appraisal 2013

medical technologies evaluation programme methods guide

highly specialised technologies guidance : the methods guide (PDF) can

be found under ‘useful documents’

diagnostics guidance : the programme manual (PDF) can be found

under ‘how we develop guidance’.


https://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-diagnostics-guidance

https://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-highly-specialised-technologies-guidance

https://www.nice.org.uk/process/pmg33/chapter/introduction

https://www.nice.org.uk/process/pmg9/chapter/foreword

https://www.nice.org.uk/news/article/nice-announces-details-of-health-technology-evaluation-methods-review

Why we are reviewing NICE’s methods7. Our overarching goal for considering any change to NICE’s health technology

evaluation methods is to improve the health of people using the NHS and

social care systems in the UK countries that use NICE guidance, and in doing

so support equitable access for people in greatest need and secure good

value to the NHS.

8. NICE’s health technology evaluation methods should be flexible, agile and

robust, to support rapid patient access to clinically and cost-effective health

technologies in the ever-changing health and care landscapes. They must be

evidence based and future proof. NICE’s health technology evaluations must

also carefully balance risks and benefits to patients and the health system.

This is particularly relevant in the context of regulatory changes and the

developing role of managed access as a key tool to mitigate risks to the NHS.

Ensuring that the methods are flexible and adaptable provides the tools

needed to accommodate faster process changes at NICE. This then supports

rapid transformation in healthcare. This is particularly important in response to

the COVID-19 pandemic, when decision making at speed is the norm. Flexible,

adaptable methods are also critical to ensuring that NICE’s evaluations remain

able to deal with innovative technologies as they emerge. This review

considers particular challenges for the latest health technology innovations

(such as advanced therapy medicinal products [ATMPs], histology-

independent cancer treatments and other emerging technologies), to ensure

that NICE can evaluate them fairly, efficiently and robustly. This will secure

rapid access to such valuable innovations. The cumulative effect of the

individual improvements across a wide breadth of methods issues will create a

more receptive environment for emerging health technologies.

9. Any changes to NICE’s health technology evaluation methods should:

support patients and the NHS in accessing clinically and cost-effective

health technologies


align with changing UK regulatory systems (particularly the Medicines

and Healthcare products Regulatory Agency [MHRA])

support the attractiveness of the UK as a first-launch country for

important and promising new health technologies, to ensure that people

can access them as early as possible

facilitate equitable consideration of commercial and managed access

flexibilities, and recognise and respond to experience with the Cancer

Drugs Fund and other commercial and managed access arrangements

enable the technological change and innovation that the UK wants to

encourage and support, by ensuring fair, robust and predictable

evaluations across new and existing technology types

sustain NICE’s reputation as a world leader in health technology

evaluation.

10.This consultation document considers methods for the evaluation of

medicines, medical technologies and diagnostics. It applies across all 4 of

NICE’s health technology evaluation programmes (technology appraisals,

highly specialised technologies, medical technologies and diagnostics), and

seeks to align them when possible and appropriate. This document does not

consider the structure or operation of the 4 programmes. That is, the

programmes will continue to operate as they currently do and with their current

remits, and only their methods are considered here.

11. It would be logical to expect that, when other parts of NICE consider health

technologies, they may adopt similar methods. However, formal changes to

NICE methods beyond technology appraisals, highly specialised technologies,

medical technologies and diagnostics are outside the scope of this review. Any

changes would need to be considered individually by the relevant other parts

of NICE. As this work continues, it will be valuable to explore the links between

NICE’s health technology evaluation and guidance ecosystems, and our other

programmes and activities.

12.This consultation takes into account all relevant factors affecting the methods

of health technology evaluation. Key considerations include:


The scientific, economic and methodological evidence that supports a

case for change in the methods.

NICE’s overall objectives and principles: the NICE charter and NICE

principles describe how NICE works and the principles that guide NICE

guidance development. They include commitments to patients, the life

sciences industry and the NHS.

The wider objectives and needs of patients, families and carers, the

NHS, government, the life sciences industry, and other stakeholders:

although an independent body, NICE does not operate in isolation and

must recognise the needs of all stakeholders.

Legislative and policy commitments, including the 2019 voluntary

scheme for branded medicines pricing and access (2019 Voluntary

Scheme). The review applies to all types of health technology, and so

is not limited to the scope of the 2019 Voluntary Scheme, but the

scheme is still an important consideration. It specifies that NICE’s usual

cost-effectiveness thresholds will not change. It also states that any

changes to the methods will respond to new types of innovation and be

consistent with improving the health gain achieved by spending on new

innovative medicines. This highlights that we should be aware of the

effect of any methods changes on healthcare spending and on how

healthcare resources are prioritised.

NICE’s ethical and legal duties to equalities and human rights: this

includes the crucial need to support fairness and equity, eliminate

unlawful discrimination and promote equality.

Approach to considering a change in methods13.The review of NICE’s health technology evaluation methods has 2 stages:

In the first stage, the evidence and considerations affecting these

methods have been reviewed, to establish whether there is a case for

changing them. This is the basis of this consultation document: the

evidence and case for change.


https://www.gov.uk/government/publications/voluntary-scheme-for-branded-medicines-pricing-and-access

https://www.gov.uk/government/publications/voluntary-scheme-for-branded-medicines-pricing-and-access



https://www.nice.org.uk/about/who-we-are/our-charter

In the second stage, NICE will consider responses from stakeholders to

those cases for change and consider the wider implications of any

amendments (including any financial effects). We will then develop a

structured decision-making framework and include the changes in an

updated programme manual. Once the second stage is complete, the

new methods will be used in NICE’s health technology evaluations.

Details of the steps and timelines for the 2 stages of the review are

available on the methods review webpage.

14.Future updates of the methods of health technology evaluation will use a more

modular and iterative approach, moving away from a cycle of updating every

4 to 6 years. We have already identified genomics, digital technologies and

antimicrobial resistance technologies as 3 topics that will be considered in

future phases of methods review. These topics will be prioritised as soon as

possible, but the precise timing depends on other activities and developments

across the healthcare system.

15.Please see the NICE Board paper of July 2019 for details of the governance

arrangements for the review. Details of the steering group, working group and

task and finish groups, including the task and finish group specifications, are

on the methods review webpage , and in the supporting documents. To date,

there have been 25 working group and steering group meetings, and the

10 task and finish groups have completed a total of 32 meetings involving

about 160 participants.

16.The methods review forms part of the NICE Connect programme, the NICE-

wide transformation programme.

Summary of potential areas for change17.This review comprises the work of many people in the task and finish groups,

the working group and steering group. The breadth and depth of the review,

and the involvement of so many different stakeholders with varying

perspectives, means that that there is not an overarching consensus on


https://www.nice.org.uk/about/who-we-are/nice-connect

https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation

https://www.nice.org.uk/news/article/nice-announces-details-of-health-technology-evaluation-methods-review

https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation

everything considered. This consultation therefore provides an essential step

in testing out potential changes with a wider audience.

18.The evidence that was considered in the review is summarised in appendix 1 ,

alongside the judgements NICE has made to come to these proposals. It is

informed by reports from the task and finish groups, prepared by NICE taking

into account input from external stakeholders. The reports do not necessarily

represent all the views of the individuals and organisations involved in the

groups but are informed by them as far as possible. The reports are provided

as supporting information.

Valuing the benefits of health technologies

19.Based on the review, there is a case to update how factors that affect NICE’s

decisions on health technologies (referred to as ‘modifiers’) are considered.

The proposed changes include incorporating severity of disease, health

inequalities, and a refined approach to uncertainty and innovative

technologies. There is also a case to change how NICE values costs and

health effects for health technologies in the future (through ‘discounting’), but

the wider policy and affordability implications of such a change go beyond the

reach of this review and will need to be considered separately before any

change could be implemented.

20.See the evidence and considerations on decision modifiers informing these

proposals in appendix 1, paragraphs 3 to 10 , and on discounting in

appendix 1, paragraphs 11 to 18 .

Proposals

a) The perspective that health benefits are of equal value (regardless of other

characteristics of the technology and people having those benefits), except in

exceptional circumstances, should be retained.

b) The current modifier for life-extending treatments at the end of life should be

removed.


c) A new modifier considering the severity of the disease should be included.

Based on previous work that NICE has done in this area, we anticipate that

this modifier can work quantitatively, in a similar way to the end of life modifier

(that is, applying a quality-adjusted life year [QALY] weight, up to a maximum

of 1.7). However, further work is needed.

d) Uncertainty should remain an important consideration in decision making. A

greater degree of uncertainty and risk should be accepted in defined

circumstances, including:

conditions for which it is recognised that evidence generation is

complex and difficult, such as rare diseases

innovative technologies

technologies that provide large benefits

when the uncertainty and risks can be monitored and controlled, such

as in a managed access arrangement.

e) There is a case to consider whether a technology will reduce health

inequalities as a modifier. Further work is needed to consider how this could

be defined (including what types and sources of inequality should be

considered), and how it could be implemented in evaluations.

f) The modifiers that are used should be as consistent as possible between

evaluation programmes (although how they are used may differ, and the

fundamental nature of the decision making and value for money frameworks

in different programmes will not change).

g) The evidence suggests there is a case to change the reference-case discount

rate to 1.5% per year for both costs and health effects (constant throughout

model time horizons). However, although there may be an evidential case for

change, this would create a number of policy and affordability challenges that

go beyond the reach of this review, including the impact, and dynamic and

distributional consequences across the health system, that will need to be

considered separately before any change could be implemented.


h) There is a case to reconsider the provisions for non-reference-case

discounting, taking into account the final reference-case discount rate and

other policy-level needs.

Understanding and improving the evidence base

21.We propose to update, refresh and clarify NICE’s methods for sourcing,

reviewing and presenting the evidence base for health technology evaluations.

In particular, there is a case to review the methods guidance for sourcing,

synthesising and presenting evidence, calculating the costs of introducing

health technologies (including those with multiple uses), considering health-

related quality of life, and characterising, quantifying and presenting

uncertainty.

22.See the evidence and considerations informing these proposals in appendix 1:

sources and synthesis of evidence: appendix 1, paragraphs 19 to 22

costs in health technology evaluations: appendix 1, paragraphs 23

to 27

health-related quality of life: appendix 1, paragraphs 28 to 31

understanding, characterising and presenting uncertainty: appendix 1,

paragraphs 32 to 39 .

Proposals

a) The methods guidance on sourcing, assessing and presenting evidence

should be refreshed and clarified, including:

no change to the general preference for randomised controlled trials

(RCTs), when feasible, to inform estimates of treatment effects

an emphasis on the role of a comprehensive evidence base, including

non-RCTs and real-world evidence, and the circumstances in which

different types of evidence have strengths or limitations

additional guidance on the use of RCT and non-RCT evidence,

assessment and reporting of study quality, risk of bias and

confounding, and presenting evidence.


b) The methods should describe more fully the circumstances in which ‘real-

world’ evidence (including evidence from observational studies, patient

registries, electronic health records and other sources beyond RCTs) may be

most valuable. They should also provide guidance on getting, analysing and

reporting real-world evidence. This type of evidence is an important topic, and

NICE health technology evaluations are ambitious in ensuring that we make

the most of this valuable resource.

c) Additional guidance on methods for synthesising evidence, based on current

methodological research and recommendations, should be included.

d) The methods should be updated to include additional guidance on the value

of, and methods for, analysing, synthesising and presenting qualitative

evidence and expert elicitation.

e) Evidence requirements for surrogate outcomes should be updated. This

should include information on different levels of evidence, expectations for

evidence of validation, and how to account for uncertainty, and should include

flexibility for surrogate outcomes in different evidence scenarios.

f) The methods should include a hierarchy of preferred sources of prices for

medicines that should be used when commercial arrangements (including

patient access schemes, commercial access arrangements and other

discounts) are in place.

g) The methods for including the costs of acquiring medical technologies,

devices and diagnostics should be clarified.

h) Non-reference-case analyses in which a particular cost is apportioned or

adjusted should be allowed in defined circumstances. These circumstances

may include when there is an established plan to change practice or service

delivery, when there is a formal arrangement with relevant stakeholders, or

when introducing the technology will have substantial, identifiable health

benefits not captured in health technology evaluations.


i) Unrelated healthcare costs should continue to be excluded from NICE’s

health technology evaluations.

j) Methods for using non-UK costing studies (including currency conversion) and

for adjusting cost data for inflation should be aligned with the methods used

by NICE’s Centre for Guidelines.

k) A hierarchy of preferred health-related quality-of-life methods for when the

reference-case measure (EQ-5D) is not available or is not appropriate should

be added.

l) NICE’s position statement on the use of the EQ-5D-5L value set for England

should be incorporated into the methods, changing the preferred method for

mapping from EQ-5D-5L to EQ-5D-3L to the tool developed by the NICE

Decision Support Unit.

m) For health technology evaluations that include children and young people,

NICE should recommend that health-related quality of life is measured using a

generic measure that has been shown to have good psychometric

performance in the relevant age group and reporting who has completed the

questionnaire. Where utility values are generated, the methods used should

be explained.

n) Further research is needed on measuring and valuing health-related quality of

life in children and young people, and in carers. NICE is actively engaged in

relevant research projects and will continue to identify and support further

work to address these issues.

o) When utility values are extrapolated over long time horizons, they should be

adjusted to reflect decreases in quality of life seen in the general population –

normally using a multiplicative approach – unless evidence is presented that

adjustment is not appropriate in an individual case.

p) The methods for selecting outcome measures should be improved by

specifying that outcome measures should be selected in consultation with

people with the condition or disease (a high-quality ‘core outcome set’ may


help with outcome selection). Patient-reported outcomes can capture

important aspects of conditions and interventions, and should be appropriately

validated and the methods used clearly reported.

q) Evaluations should include an overall assessment of uncertainty, including the

effects of different types of uncertainty, whether uncertainties have been

captured in analyses, and whether uncertainties can be addressed by

additional evidence.

r) Approaches to presenting and visualising uncertainty should continue to be

developed.

s) Probabilistic analyses should be used as the starting point for economic

analyses, including committees’ preferred analyses and scenarios, unless

there is clear justification not to.

t) Probabilistic analyses should include enough simulations to minimise the

effect of Monte Carlo error, and should use appropriate methods that account

for ordering and correlation in parameters.

u) Probabilistic univariate sensitivity analyses and threshold analyses may be

included as options, with their limitations noted in the methods guide.

v) Decision-making analyses must be based on plausible parameters and

assumptions. Analyses based on implausible values may be used for specific

purposes and must be clearly labelled.

w) Integrating structural uncertainty with parameter uncertainty in a combined

analysis (for example, by parameterising structural uncertainty) should be

encouraged as an option, but is not mandatory.

x) Methods for extrapolating beyond the available data should be clarified and

expanded on, including:

clarifying the plausible extrapolation scenarios that might be considered

acknowledging standard and flexible extrapolation models


explicitly requiring that plausibility of hazard functions and associated

uncertainty is considered

requiring that uncertainty associated with methods to adjust for

treatment switching is considered.

y) The expected value of perfect information should be routinely presented.

Structured decision making

23.The review identified a case to change how clinical and cost-effectiveness

analyses are presented and considered (including incremental and pairwise

analyses and subgroup analyses). We also propose to clarify decision making

for technologies that provide less health benefit at lower costs, and for

technologies that are not cost effective even if priced at £0, and to refine how

NICE’s recommendations on health technologies are presented. We also

reaffirm NICE’s commitment to equality and fairness in all NICE’s

recommendations.

24.See the evidence and considerations on structured decision making informing

these proposals in appendix 1, paragraphs 40 to 59 .

Proposals

a) Fully incremental analyses should remain the preferred approach for

economic analyses. Pairwise comparisons may be allowed when relevant and

justified.

b) Net benefit approaches may be particularly relevant when:

there are several interventions or comparators

the differences in costs or QALYs between comparators is small

there are subgroup considerations

technologies provide less health benefit at lower costs (that is, in the

south-west quadrant of the cost-effectiveness plane)

c) When net benefits are presented, net health benefits are preferred.


d) For technologies in the south-west quadrant, cost-effectiveness

considerations should take into account the usual cost-effectiveness levels of

£20,000 to £30,000 per QALY.

e) The methods for considering subgroups should be updated to include

presentation of absolute and relative treatment effects, and to improve the

methods for reviewing the credibility of subgroup effects.

f) If they consider it appropriate, committees may choose not to recommend a

technology for a particular subgroup (that is, to make an optimised

recommendation) even when the technology is found to be clinically and cost

effective for the whole population. Such a decision must be methodologically,

clinically and ethically appropriate, and must take into account the benefits

and harms of including or excluding a particular group.

g) In exceptional circumstances, non-reference-case analyses that omit or adjust

specific high-cost care may be presented for technologies that are otherwise

not cost effective at £0.

h) Approaches to presenting information to committees and to stakeholders (in

addition to uncertainty) should continue to be developed.

i) The wording of recommendations should be updated for consistency and

clarity, retaining flexibility for individual programmes.

j) When recommending technologies that are one of several similar options,

committees may specify what should be taken into account when choosing

between them, and this may include cost.

k) Equalities considerations must continue to be taken into account throughout

health technology evaluations, including when selecting and routing

technologies for evaluation.

l) Further work is needed to consider how equality issues associated with

implementing technologies in NHS practice might feed into evaluations, and to

explore potential equality considerations for qualitative evidence.


Challenging technologies, conditions and evaluations

25.Reviewing key challenges for technologies such as ATMPs, histology-

independent cancer treatments and other emerging technologies allowed us to

identify generally applicable methods improvements that are particularly

helpful for these challenging topics. We also explored the methods challenges

for rare diseases in detail.

26.See the evidence and considerations on challenging technologies, conditions

and evaluations informing these proposals in appendix 1, paragraphs 60 to 78 .

Proposals

a) Medical technologies evaluations should now consider, when relevant,

unpublished evidence and post-marketing surveillance data.

b) Evaluations in which there is uncertainty about long-term health benefits

should include scenario analyses that explore the effects of different

assumptions about long-term benefits. This might include threshold analysis

for the duration of treatment effects.

c) Cure-proportion modelling should be considered as an option.

d) For evaluations including significant service delivery effects, NICE should

seek information from relevant health and care system partners.

e) When basket trials are used, they should be appropriately designed and

analysed and include assessment of heterogeneity and allow borrowing

between baskets.

f) When clinical trials do not include a comparator group, several methods to

derive comparative evidence should be explored.

g) When heterogeneity between groups within a population is a concern, any

assumptions about homogeneity or heterogeneity and generalisability to

clinical practice must be clearly presented, tested and fully explored.


h) NICE should improve its processes for active horizon scanning for potential

methodological and implementation challenges associated with emerging

technologies and innovations.

Aligning methods across programmes

27.Aligning methods across NICE’s health technology evaluation programmes will

continue into the second stage of this review, as we develop a consolidated

structured decision-making framework and methods manual. At this stage, we

identify a case to align and improve NICE’s methods of cost comparison for

health technologies.

28.See the evidence and considerations on aligning methods across programmes

informing these proposals in appendix 1, paragraphs 79 to 83 .

Proposals

a) The methods should include a single, consistent approach for cost-

comparison analysis, which can be used in relevant circumstances in all

health technology evaluation programmes.

b) Consideration of similarity should include explicit discussion of clinical,

technological, biological, pharmacokinetic and/or economic plausibility.

c) Cost-comparison analyses should include relevant analyses of uncertainty,

appropriate to the analysis and associated uncertainty. This may include

sensitivity, scenario and probabilistic analyses.

Questions for consultation29.The views of stakeholders and the public on the proposals and cases for

change presented in this document are welcomed. Please comment on the

following questions:

Do the proposals and cases for change provide a suitable basis to

inform the final methods?


Do you have any comments or feedback on the methodological

evidence and considerations that have been taken into account, or

how the evidence has been interpreted?

Do you have any comments or feedback on how well the proposals

will achieve the aims of the review?

That is, to optimise NICE’s methods of health technology evaluation

to support the NHS’s ambition to provide high-quality care that offers

good value to patients and to the NHS

What are the potential effects of the proposed changes on patients and

their families, health technologies, the life sciences industry and the

NHS?

What are the potential benefits of the proposed cases for change?

Are there any risks that might arise from adopting the proposals? If

so, how might we try to reduce them?

Do you have any comments or feedback on how well the proposed

methods will support innovation for patients, science, society and the

life sciences industry?

What are the potential implications of the proposed changes for other

NICE guidance and advice, and for other NICE programmes and

activities?

Do the proposals create any equalities concerns, particularly for NICE’s

legal responsibilities and the important need to eliminate unlawful

discrimination and promote equality?

30.When making comments, you are reminded of the objectives of the review and

the boundaries of the current stage, as described earlier. In particular, this

document focuses specifically on the methods of health technology evaluation

(and not its processes or other related developments, which are considered

separately), and presents the evidence and case for change only (a finalised

methods framework will be developed in the next stage).


Assessing the impact31.The proposals and cases for change describe a broad range of incremental

improvements to NICE’s methods of health technology evaluation. Taken

together, the proposals have the potential to bolster the ability of NICE and its

evaluation committees to support introducing valuable innovative technologies,

bringing health benefits for patients and good value to the NHS. They aim to

ensure that the methods remain robust for advances in healthcare and

technology evaluation methods. They also seek to ensure that any risks to

patients and the health system are appropriately identified, considered and

managed. The proposals tackle some of the key challenges faced by

committees for some of the most difficult topics. In this way, they aim to ensure

that NICE’s evaluations continue to be robust and efficient. This is important

for supporting the NHS in securing fast access to clinically and cost-effective

health technologies for the benefit of patients across England.

32. It is too soon to identify and quantify the precise effect of the proposals on

individual technologies, conditions, wider healthcare budgets and how different

uses of healthcare resources are prioritised. This is because much depends on

the views of stakeholders expressed in consultation, and on how the proposals

might be implemented in practice. The effect of changes on healthcare

spending must be considered as part of this review, in part because of the

commitments in the 2019 Voluntary Scheme. We note that, although the 2019

Voluntary Scheme includes measures to mitigate the costs of branded

medicines, the effects on healthcare budgets are still important because of

healthcare displacement effects within financial years, long-term dynamic

effects (such as comparator prices), and effects on other health technologies.

In the context of fixed healthcare budgets, methodological changes are likely

to affect how different health technologies and healthcare resources are

prioritised. This will be explored in detail in the second stage of the review (see

section 13).

33.At this stage, we can identify which elements of the proposals would have a

particular effect. These include:


The modifiers proposals aim to improve how we value the benefits of

health technologies. It is anticipated that a modifier for the severity of

disease would be more broadly and fairly applicable than the previous

end of life criteria, which were most commonly restricted to cancer

technologies. The number of technologies that would receive a severity

modifier depends on how it is defined (and it should not be assumed

that this will necessarily be higher or lower than the number that

currently qualify for end of life; this is yet to be determined). However,

we anticipate that a severity modifier could be relevant to a broader

range of conditions, and would more accurately reflect society’s values.

Improvements to the description and characterisation of uncertainty,

and the clarifications on how it should be taken into account as a

modifier, intend to ensure that we can take a more robust and nuanced

approach to risk for new technologies. We should continue to be

confident that the technologies recommended provide valuable health

benefits, while allowing a more flexible approach in identified priority

areas. Refining our approach to uncertainty is expected to be

particularly beneficial in the context of developments in commercial and

managed access arrangements.

The potential change to the discount rate, if implemented, would seek

to reflect as accurately as possible the value of costs and health effects

in the future. The task and finish group was able to begin to estimate

the effect of such a change on health technology recommendations and

medicines pricing. It highlighted a particularly large effect on

technologies that have high upfront costs and long-term health benefits

such as ATMPs and other one-off treatments. It also highlighted that

such a change could have a substantial effect on healthcare budgets

and priorities, and therefore interacts with the policy and affordability

challenges that go beyond the reach of this review.

Incremental improvements in how evidence is collected and assessed,

including clinical effectiveness, health-related quality of life and costs,

should improve the quality of health technology evaluations across the

board. This supports the methodological robustness of NICE’s


evaluations, which is key to ensuring that patients can continue

accessing clinically and cost-effective technologies, while minimising

the risk of inappropriately displacing healthcare from other patients in

the NHS.

Taken together, the proposals in this methods review and the high

degree of flexibility in NICE’s existing methods and decision-making

approach aim to provide a broad package that ensures that NICE can

robustly and efficiently evaluate innovative technologies. Such

technologies include ATMPs, histology-independent cancer treatments,

technologies for rare disease and emerging innovations. The proposed

improvements include the recommendations triggered by specific

considerations of these particularly challenging topics, combined with

general methodological improvements from other elements of the

review. Of particular importance for these technologies are the

approaches to considering uncertainty, innovation, apportioning costs

and different types of evidence. These innovations are an important

priority for patients for whom they may provide valuable health benefits,

and for many other stakeholders (including the NHS Accelerated

Access Collaborative). The incremental improvements in evaluation

methods proposed in this review aim to ensure that these topics

continue to be robustly and efficiently evaluated and introduced in the

NHS.

34.Further assessment of the effect of the health technology evaluation methods

review is an important priority for the second stage of the project. This will

include, when possible, quantifying the effect of different recommendations

and the recommendations as a whole. This work has begun already, and will

continue throughout the next stage of the review.

Equality assessment

35.Equality issues were considered in detail by all of the task and finish groups.

Their findings are documented in the respective reports, and were taken into

account in developing the final proposals and cases for change. In most cases,


no equality issues associated with the methods were raised. When issues

were identified, adjustments to the proposals were incorporated. Of particular

note:

The equalities task and finish group focused specifically on equalities,

and made a series of recommendations.

The modifiers group noted particular equality considerations when

exploring modifiers for age (including children and young people) and

health inequalities.

It was highlighted that discounting has the potential to interact with

characteristics such as age and disability. Equalities concerns about

the current criteria for non-reference-case discounting were identified

and will be addressed in the proposed reconsideration of the methods

and provisions for non-reference-case discounting.

In the review of understanding and improving the evidence base,

potential concerns were raised about: EQ-5D in hearing disorders and

other disabilities; adjusting health-related quality of life over time;

unrelated healthcare costs; and uncertainty in rare diseases and other

disabilities.

The decision-making task and finish group highlighted equality

considerations relating to subgroups and technologies that are not cost

effective even if priced at £0.

36.We consider that there are no further equalities issues raised by the current

methods and cases for change. We welcome comments on any equalities

related issues and concerns in the consultation.


Appendix 1: Case for change1. This appendix describes the key considerations informing the case for change,

on which our proposals are based. It is informed by the work of the task and

finish groups, whose findings are documented in the reports that accompany

this document.

2. The task and finish groups reviewed in detail published evidence and

methodological literature, international approaches at other health technology

assessment bodies, and past experiences and challenges at NICE. They

discussed their findings with external stakeholders and considered their views

and perspectives to inform the final reports.

Valuing the benefits of health technologies

Decision modifiers

3. The clinical benefit and value for money offered by health technologies are

important considerations in NICE's health technology evaluations, but NICE’s

recommendations are not based on evidence of costs and benefits alone. We

must consider other factors when developing NICE guidance. Factors that

influence or change the considerations, decisions or recommendations of

NICE’s committees (either qualitatively or quantitatively) may be referred to as

'modifiers'. The evidence and case for changing the modifiers were explored

by the modifiers task and finish group.

4. The review considered in detail NICE’s current approach to modifiers and the

underlying normative principles, as described in NICE’s health technology

evaluation methods guidance and the principles that guide our work.

Importantly, we use a broad and flexible deliberative approach that ensures

NICE guidance takes into account all relevant factors and considerations in the

unique circumstances of each technology evaluation. In the usually preferred

analyses (the reference case), health benefits are considered to be of equal

value regardless of other characteristics of the technology and people having

those benefits. Also, specific considerations are defined that must be taken


into account for technologies that have higher cost-effectiveness estimates –

either in the range of £20,000 to £30,000 per QALY gained, or at higher cost-

effectiveness levels through an additional QALY weight (that is, departing from

the reference case). NICE guidance must take into account the broad balance

between the costs and benefits of recommending a health technology. This

must be considered in the context of health displaced elsewhere in the NHS by

additional spending. When an additional factor or modifier is taken into account

such that it is given additional weight or a higher incremental cost per QALY is

accepted, there is greater displacement of health elsewhere. Because of this,

and to protect people from losing out on clinically and cost-effective healthcare

across the NHS, additional modifiers must be applied cautiously. Modifiers

should be supported by evidence of societal value or a robust and reasoned

justification. In that context, we consider that there is no case to change the

fundamental principle that health benefits are of equal value regardless of

when or how they are gained, except in exceptional circumstances, when

additional weight may be given to specific, defined factors. The following

sections consider which modifiers should be formally taken into account and

defined in NICE’s health technology evaluation methods guidance as specific

considerations. Further work will be needed in the second stage of the review

to establish how these modifiers will work in practice, including whether they

are quantitative or qualitative considerations. In particular, we will establish

which modifiers should be considered routinely, within the normal cost-

effectiveness ranges, and which modifiers may be given an exceptional

additional (and quantified) weight. We will also establish how the selected

modifiers should be defined and implemented. The further work will consider

the effect of the selected modifiers framework on decision making, different

conditions and technologies, and the NHS.

5. In that context, we explore several current and potential modifiers in detail:

Severe and terminal conditions: The review found that there is

limited evidence that society places additional value for life-extending

treatments at the end of life. We therefore consider that there is a case


to remove the current end of life modifier. Conversely, the review found

greater evidence that society highly values health benefits in very

severe conditions. We consider that there is a case to introduce a

modifier that considers disease severity. Such a modifier would focus

on the severity of the condition with current treatment. So, it would

implicitly encompass concepts such as the burden of illness and the

degree of unmet need in a condition. Severity of disease and end of life

conditions overlap (both conceptually and in some practical measures).

Given this, and based on NICE’s previous work in this area (see

section 6), we anticipate that severity can work as a quantitative

modifier that operates similarly to the current end of life modifier that we

propose to remove (that is, applying a QALY weight, up to a maximum

of 1.7). However, further work is needed to confirm how such a modifier

should be defined and implemented.

Uncertainty and risk: It is important that NICE’s decisions take into

account the uncertainty associated with the evidence, and the resulting

risks to the health of patients having a technology and the health of

those elsewhere in the NHS (whose care may be displaced by an

uncertain technology). This is conceptually somewhat different to other

modifiers, in that it is not an attribute of a technology or condition that

society may value, but rather a feature of the evidence base.

Nevertheless, we consider that there is an important case to retain

uncertainty as a key factor that influences decision making. This is

particularly important in the context of developments around

commercial and managed access options, which provide critical tools to

address uncertainty and risk. There is also a case to introduce more

flexibility in interpreting uncertainty, to reflect and support important

priorities for the NHS. We propose that committees should have the

flexibility to accept greater uncertainty and risk in specific situations,

including:

Conditions for which it is recognised that generating evidence is

complex and difficult, such as rare diseases. Although the review

found limited evidence that society values health benefits for rare


diseases more highly, so there is no case for a specific modifier for

rarity, we consider it important to ensure that people with rare

diseases are not disadvantaged and that technologies for these

conditions are supported. So, we propose to adopt a more accepting

attitude to uncertainty in this context.

Innovative technologies: the review identified a clear case to refine

NICE’s approach to innovation as a modifier because of significant

difficulties in interpretating and applying the current methods. Again,

there is no case for a specific modifier (because there is no evidence

to show that society values health benefits from an innovative

technology more highly than equivalent benefits from a less

innovative technology). However, we must emphasise that

supporting innovation is important to NICE. NICE has statutory and

ethical duties to patients, the life sciences industry and the NHS, and

innovative technologies can provide valuable benefits for patients

and society. We therefore consider that it is appropriate to accept

greater risks for highly innovative technologies, to support patients in

accessing their valuable benefits.

Technologies that provide large benefits: it is appropriate that a

greater degree of risk can be accepted when the potential benefit is

higher. We will need to explore whether this consideration should

include large benefits to individuals, smaller benefits across a large

population, or both.

Technologies for which the uncertainty and risks can be monitored

and mitigated, for example through managed access arrangements.

In all cases, we must take into account the nature, scale and

consequences of the uncertainty and risks. Further work is needed to

define more precisely the situations in which greater uncertainty can be

accepted. In particular, a clear understanding and definition of

innovation is needed, and this must not overlap with the size of the

benefits to create ‘double counting’.


Health inequalities: The reviews by the modifiers and equalities task

and finish groups identified evidence that the UK population prioritises

seeking a fair distribution of health across society. The reviews

highlighted evidence that people are willing to generate less health

overall if the health is generated in disadvantaged groups, particularly

for socioeconomic disadvantage. This suggests that there may be a

case for a modifier that considers health inequalities. Also, NICE’s

statutory duties and stated principles support developing guidance that

gives particular benefits to the most disadvantaged. When a population

having a technology is disadvantaged in terms of health because they

have a severe health condition, it would be captured by a modifier for

severity of disease (as proposed in section 5). However, when a

technology can reduce health inequality associated with socioeconomic

status or other factors, there may be a case to take this into account as

a modifier. Further work is needed to explore how this could be defined

and implemented in a health technology evaluation, and in which

circumstances. This work would consider, for example, what types and

sources of inequality should be considered and how, and how a

modifier should be applied. A further issue that might be explored is

whether such a modifier should include technologies that directly

reduce inequalities (for example, by specifically targeting or providing

additional benefits for a disadvantaged group), and whether indirect

effects might be considered (for example, if a technology has uniform

benefits across groups but the condition disproportionally affects a

disadvantaged group). Any modifier would need to be implemented

such that it did not overlap with (and hence double-count) severity of

disease. It is not clear that there is sufficient evidence for a substantial

departure from the reference case (that is, to apply a quantified

modifier or weight), but this can be explored. We propose that there is a

case to include consideration of whether a technology will reduce

health inequalities as a modifier. Further work will be done in the

second stage of the review to consider how this could be defined and

implemented in evaluations.


6. It is acknowledged that severity of disease has previously been considered by

NICE, under the heading of burden of illness, in the consultation on value-

based assessment of health technologies. Although the proposals in that

consultation were not adopted, broad support was expressed for the

incorporation of burden of illness as a criterion. Consultees expressed a wide

range of opinions on the proposals, including the severity of disease

component and other proposals. Nevertheless, we are confident that the

evidence is sufficient to support a case for introducing a severity modifier in the

current review, and that it could be successfully implemented. Based on

NICE’s previous work in this area, it is likely that methods for measuring

severity, such as QALY shortfall, may be an option, and this will need to be

explored. We will also take into account concerns that were raised in the

value-based assessment consultation.

7. We consider that there is limited rationale for valuing a particular factor

depending on the NICE programme in which it is considered. Therefore, the

modifiers that are currently used in technology appraisal, diagnostic and

medical technology evaluations, and those for which there is a case for change

identified (see section 5), should be considered in all technology evaluations.

We consider the situation for the highly specialised technologies evaluation

programme to be different. This programme is built on different ethical and

normative principles. It uses a different decision-making framework, and

considers different value for money levels, compared with other health

technology evaluations. The key modifier that is currently used in highly

specialised technologies evaluations but not in other programmes is the

additional weight applied to treatments that provide large health benefits. The

review identified that there is limited evidence and rationale supporting this

modifier, beyond the view that a greater degree of risk may be accepted when

the potential benefits are large. Nevertheless, we do not consider there to be a

case to remove the magnitude of benefit modifier from the highly specialised

technologies programme because of its exceptional nature.


https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/methods-of-technology-appraisal-consultation

https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/methods-of-technology-appraisal-consultation

8. How the different modifiers operate (for example, whether they are quantified,

and if so in what way) will be established in the second stage of the methods

review. It is beyond the scope of the review to change the fundamental nature

of the decision-making and value for money frameworks that are used in

technology evaluations. So, the final modifiers will operate within the

frameworks as currently defined.

9. We consider that there are no further cases to substantively change the

modifiers that are included in NICE’s health technology evaluation methods

guidance. The review did not identify evidence that society places particular

additional value on technologies that are potentially curative (such as ATMPs).

We therefore consider that the evidence is currently too limited to support a

case to introduce a specific modifier for potentially curative treatments, but

highlight that this issue is also considered as part of non-reference-case

discounting (see sections 15 to 17). NICE understands there is broad interest

in potentially curative technologies including ATMPs, and a policy-level drive to

support them, so further research in this area should be commissioned. Also,

we do not propose to introduce a modifier based on age. Although the task

and finish group identified some evidence that suggests the UK population

considers different age groups differently, we conclude that this evidence has

important limitations and does not directly support a case for a modifier based

on age. Also, we note the conclusions from the NICE Citizen’s Council (2003)

and NICE’s previous social value judgements, which do not support giving

additional weight based on the age of the target population. On balance, we

conclude that the evidence is not sufficient to support a case for introducing a

modifier at this stage, but, again, further research would be valuable.

10.Across the board, the evidence explored in this review on societal value for

different factors and modifiers has several important limitations for our

purposes. Robustly identifying societal views in a way that directly informs

NICE’s decision-making framework is difficult. We have been able to establish

some cases for change to improve our consideration of key factors at this

stage. But there remains scope to reconsider our approach if relevant



evidence emerges. Such research would be highly valuable and should be

prioritised.

Discounting

11.NICE’s valuation of the benefits of health technologies also depends on how

benefits are valued over time. Discounting is an economic method used in

assessing benefits and costs that occur in different time periods. It reflects the

view that people generally prefer to receive benefits or goods now but pay for

them later. NICE currently specifies a preferred (reference case) discount rate

of 3.5% per year for both costs and health effects. It also allows alternative

(non-reference-case) discounting at 1.5% per year in defined circumstances.

12.The discounting task and finish group explored whether there was a case for

changing the approach to discounting. It considered 4 issues:

the reference-case discount rate

whether the discount rate should be the same or different for costs and

health effects

the approach to non-reference-case discounting

whether the discount rate should change during the time horizon of an

economic model.

13.The review identified a wide range of relevant evidence and considerations,

and generated a lot of discussions with wide ranging views. The key

challenges included the economic evidence on the discount rate, the

recommendations in HM Treasury’s Green Book and the associated rationale,

the effect of a change in the discount rate on medicines pricing, and interaction

with commitments such as the 2019 Voluntary Scheme and other associated

policy challenges. A change in discount rate would seek to reflect as

accurately as possible the value of costs and health effects in the future.

However, the review highlighted in particular the potential financial impact of

any change in discount rate, which would have knock-on effects on budgets

and priorities across the health system. It highlighted a particularly large effect


on technologies that have high upfront costs and long-term health benefits,

such as ATMPs and other one-off treatments.

14.We took into account all of this evidence and the perspectives of working

group members, and concluded that the best available evidence suggests

there is a case for changing the reference-case discount rate from 3.5% to

1.5%, for both costs and health effects. We consider that the argument that the

2% ‘wealth effect’ does not apply to health (as described in the Green Book) is

important. We further consider that, in the resource-constrained health system

in which we operate, costs and health effects should be treated the same.

However, although there is evidence to support the case for change, this

would create several policy and affordability challenges that go beyond the

reach of the methods review. These include the effect of the change on

healthcare costs, and dynamic and distributional consequences across the

health system. The interaction with the 2019 Voluntary Scheme is important,

although we also note that NICE evaluates many technologies that are not

within its remit. These policy and affordability challenges will need to be

considered separately.

15.Taking into account the conclusions on the reference-case discount rate, we

need to consider carefully the methods for non-reference-case discounting.

The review identified several important considerations. In particular, it found

limited support for using an alternative discount rate in specific circumstances,

because it identified no evidence or rationale to support a difference in time

preference for specific circumstances or technology types. The current non-

reference-case provision effectively gives ‘additional value’ to technologies that

restore people who would otherwise die or have a very severely impaired life

to full or near-full health sustained over a very long period. We note that

changing the discount rate is not necessarily the best way to include any

relevant additional value. These considerations might imply that there could be

a case to remove the provision for non-reference-case discounting in defined

circumstances.


16.We consider that if a reference-case discount rate of 1.5% can be

implemented, there is no case to keep provision for a lower rate in defined

circumstances. The considerations described in section 15 are relevant, and in

any case no justifiable rate below 1.5% has been identified. However, if a

change to the reference-case rate cannot be implemented, and the current

rate of 3.5% is retained, we do need to consider the role of a non-reference-

case exception. The finding that the discount rate can significantly affect how

we value costs and health benefits that occur in the future, particularly for

technologies such as ATMPs and other potential cures, is important. It

emphasises that a discount rate of 3.5% decreases the value placed on

longer-term benefits, compared with 1.5%, and so affects the apparent cost-

effectiveness of such technologies. NICE understands there is a significant

policy-level drive to support curative and potentially curative technologies

including ATMPs. We acknowledge that discounting is not necessarily the best

way to reflect societal value or policy-level support (for example, modifiers and

commercial mechanisms might also be options), but it is nevertheless a tool

within the current methods. Having concluded that the evidence is currently too

limited to support a specific modifier for curative treatments (see section 9), the

policy-level drive is sufficient to advocate caution in considering the case to

change the current non-reference case exception. However, the criticisms of

the current criteria for using the non-reference-case provision are important.

We note in particular the concern that the criteria are rarely applied because

they set a high bar for the technologies they otherwise appear to support, by

requiring there be no significant irrecoverable costs and implying a high degree

of certainty is needed. We also note important ethical and equalities concerns.

These criticisms show that if the non-reference-case provision is kept then, as

a minimum, the criteria need to be refined (retaining non-reference-case

discounting without specific, defined criteria is unlikely to be successful). This

is likely to be challenging, and will require a careful understanding of the

technologies and attributes that need to be supported and why. Any change to

the criteria must first-and-foremost provide clarity for committees and

stakeholders and address the ethical and equalities concerns. In order to have

an appreciable effect on particular types of health technology, any change


would most likely need to increase the number of technologies for which the

non-reference-case rate is applied. This would have cost implications, which

may overlap with the policy and affordability challenges identified for the

reference-case rate.

17.Overall, we conclude that there is a case to reconsider the provisions for non-

reference-case discounting. This will need to take into account the final

reference-case discount rate, the policy-level need to support particular types

of technologies or circumstances, limitations of the current criteria for non-

reference-case discounting, and the effects and any accompanying policy and

affordability challenges of any change. Further exploration is needed in the

second stage of the methods review.

18.We consider that there is no case to introduce a change in the discount rate

during the time horizon of an economic model. Based on the review, we note

that this would have a minimal effect on model results. However, it would

create additional complexity and need NICE and stakeholders to put in

unjustified extra effort.

Understanding and improving the evidence base

Sources and synthesis of evidence

19.The sources and synthesis of evidence task and finish group completed a

detailed and comprehensive review of how clinical and economic evidence

should be identified, presented and critically appraised in NICE health

technology evaluations. It took into account different sources of evidence,

methods for assessing the quality, strengths and limitations of the evidence,

and methods for combining (or ‘synthesising’) evidence to inform an

evaluation. Its findings are presented in the report in the supporting

documents, and for brevity are only briefly summarised here. In general, we

propose to follow the recommendations in that report.

20.The key theme running through this element of the methods review is to

reaffirm NICE’s commitment to using evidence that is high quality, fit for


purpose and appropriately critiqued throughout all health technology

evaluations. Evidence must be clearly and transparently presented and

thoroughly assessed, and consideration given to its strengths and limitations.

We recognise the value of different sources and types of evidence in different

circumstances and for different purposes. We also recognise the need to

ensure that we have the flexibility to give a fair and valid assessment of any

technology. All of these measures ensure that NICE guidance remains

fundamentally evidence based.

21.The review identified several cases for change.

Sourcing, assessing and presenting evidence: The review identified

a case for a broad refresh and clarification of how the roles of different

sources of evidence are described, and how the evidence is assessed

and presented. There is no case to depart from the current strong

preference for using high-quality RCTs to inform estimates of treatment

effects. However, it is important to emphasise the role of a

comprehensive evidence base (including non-RCTs), while also

acknowledging that there are circumstances in which RCTs may be

unethical, unfeasible or have limitations and giving additional guidance

on the choice and use of non-RCT evidence. Non-RCT evidence may

have a particular role in complementing and supplementing RCT

evidence, particularly for inputs into clinical evaluations and economic

models beyond the estimates of relative efficacy (for which it is already

commonly used). Guidance on the use of RCT and non-RCT evidence

should consider its strengths and limitations, consistency and

heterogeneity. There is a case to bolster the methods used to assess

and report the quality of studies and risks of bias and confounding.

There is also a case to improve evidence presentation in line with

international standards.

‘Real-world’ evidence: This includes evidence from observational

studies, patient registries, electronic health records and other sources

beyond RCTs. It is particularly important in the current health and


technology landscape. NICE is ambitious in ensuring that we make the

most of this valuable resource in health technology evaluations. The

review identified a case for changing the methods, to help optimise the

use of real-world evidence. We propose to describe more fully the

circumstances in which real-world evidence may be most valuable,

including:

providing information on the generalisability of trial evidence in

clinical practice

supplementing and adding to clinical trial evidence with evidence

from a real-world setting, particularly when trial evidence is limited

including in the longer term

describing populations and patient groups, clinical practice, the

natural history of disease and resource use

describing the experiences of people with particular conditions and

having particular treatments.

The review also identified a case to provide guidance on getting,

analysing and reporting real-world evidence. This topic links with

ongoing work at NICE on increasing use of health and social care data

in guidance development. This work will deliver on NICE’s ambition to

make better use of data and analytics in its decision making, as

outlined in the statement of intent and in the further information about

our plans for a data and analytics methods and standards programme.

In particular, it will provide clear guidance on expectations of data

quality, research governance and analytical methods when generating

evidence from observational and non-randomised data.

Synthesising evidence: The review identified extensive information

and recommendations on how evidence should be synthesised,

including in technical support documents published by the NICE

Decision Support Unit. Ensuring that high-quality methodological

guidance is followed should help ensure that evidence syntheses

presented in health technology evaluations are robust and well


https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-guidelines/how-we-develop-nice-guidelines/data-and-analytics-statement-of-intent

https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-guidelines/how-we-develop-nice-guidelines/data-and-analytics-statement-of-intent

conducted. The review identified a case to provide additional guidance

on methods for:

population adjustment

bias adjustment

ordered category data and trials involving both discontinuation and

efficacy outcomes

joint synthesis of structurally related outcomes

using prior distributions (and particularly using informative priors in

appropriate circumstances)

assessing the proportional hazards assumption and associated

methods when it is and is not met

synthesising diagnostic accuracy evidence.

Qualitative evidence: There is a case to update the methods to

include clearer recognition of the range of circumstances in which

qualitative evidence may be valuable. We also propose to give

additional guidance on analysing, synthesising and presenting

qualitative evidence.

Expert elicitation: Similarly, there is a case to recognise when and

how expert elicitation evidence, either qualitative or quantitative, may

be valuable. We should also acknowledge the limitations of this type of

evidence. To ensure that this evidence is of the highest possible quality

when it is used, there is a case to give preferred methods, focusing on

structured methods for quantitative elicitation and clear, transparent

reporting.

Surrogate outcomes: Noting that the use of surrogate outcomes is

expected to become increasingly common, the review identified a case

to make our requirements for surrogate outcome evidence more

explicit. In particular, we should describe the levels of evidence, our

expectations for evidence of validation, and how to properly account for

uncertainty in surrogate relationships. Although we seek the highest

quality evidence possible for surrogate outcomes, the methods should


include a degree of flexibility to ensure that they are suitable for a wide

range of scenarios.

22.We will consider how the cases for change should be incorporated into NICE’s

published health technology evaluation methods guidance in the second stage

of the methods review. We need to consider to what extent it is necessary and

appropriate to formally reference technical support documents and other

published literature within the methods guide. We will also need to consider

whether guidance can be placed in submission templates, and how to ensure

consistency across NICE (for example, in references to real-world evidence

and quality assessment tools).

Costs in health technology evaluations

23.A fundamental methodological principle of health technology evaluations is

that they must reflect as accurately as possible the costs associated with

introducing and using the technology under evaluation. When costs are

variable or uncertain, the variations and uncertainties should be considered,

just as they are for any other aspect of the evaluation.

24.One of the first considerations is the acquisition cost of the technology being

evaluated and its comparators. We propose a hierarchy for the sources of

prices for medicines that should be used when commercial arrangements are

in place (including for generic and biosimilar medicines). This is to ensure that

the most accurate price possible is used, and the analysis is as transparent as

possible. For medical devices and diagnostics, it is beyond the scope of the

methods review to make changes to the commercial arrangements that are

available. Nevertheless, there is a case for clarifying what costs should be

included in evaluations and how. New submission templates will be developed.

In all cases, evaluations should accurately reflect the cost to the NHS and

personal social services of acquiring the technology, and any associated

variation and uncertainty.

25.The review explored circumstances in which there may be a case to consider

whether the full cost of a technology should be included in the evaluation, or


whether some of its costs may be apportioned between other technologies,

indications or evaluations. This particularly applies to:

high-cost medical devices that have other uses beyond the evaluated

indication (either now or in the future)

diagnostic tests that could identify multiple markers (in particular gene

panel and genomic tests)

technologies that need substantial service redesign to introduce them

(which may affect other technologies or people not having treatment

with a technology).

We propose that the fundamental methodological principle should inform

NICE’s normally preferred analyses. That is, in the reference case, the full

additional costs associated with introducing a technology should be included.

However, there is also a case to introduce flexibility to allow non-reference-

case analyses in which only a fraction of a particular cost is included (that is,

the particular cost is apportioned between different stakeholders, technologies

or indications), in defined circumstances. The justification for this approach is

to reflect the dynamics of the health system, and to fairly recognise when

benefits of health technologies fall outside health technology evaluations. It

does not aim to specifically reward or incentivise particular innovations

because innovation is captured elsewhere within NICE’s methods (see, for

example, section 5) and beyond. As a result, non-reference-case analyses

might be allowed when, for example:

there is an established plan to change practice or service delivery in the

NHS, for example, when a genomic test is currently not done but is

likely to be covered by a new routine funding arrangement soon

there is a formal arrangement with relevant stakeholders (including

NHS England when relevant) that the full costs should not be attributed

to the new technology, for example, when NHS England agrees that a

certain infrastructure cost should be shared between organisations, or

introducing the new technology will lead to substantial, identifiable

health benefits that are not captured in health technology evaluations,


for example, when the technology is accompanied by service

improvements that improve health for people not covered by any


Further work is needed to define those circumstances in which non-reference-

case analyses would be appropriate. Further work is also needed to establish

how costs should be apportioned and how the relevant cost to include the

analysis should be established in different circumstances for medical

technologies (for example, new or existing high-cost devices), medicines and

diagnostics. These will be explored in the second stage of the methods

review.

26.NICE's health technology evaluations currently exclude healthcare costs that

are considered unrelated to the condition or technology in the evaluation. For

example, in an evaluation of a cancer treatment, the costs of treating

hypertension would be viewed as unrelated (assuming that hypertension is not

an adverse effect of the treatment) and so would be excluded. The review

explored arguments for and against this approach and also explored the

methods available to incorporate unrelated healthcare costs if considered

appropriate. It explored key issues around completeness and accuracy of the

economic analysis, consistency with how health effects are considered, the

ethics and fairness of including or excluding these costs, and practical issues

in doing so. Importantly, unrelated healthcare costs are genuine costs that are

incurred by the NHS and personal social services, and any increase in such

costs (for example, through life extension by a particular technology) would

displace healthcare elsewhere in the NHS. There is therefore an economic

argument that unrelated healthcare costs should be included. However, there

are still several critical outstanding research questions and concerns. These

include important questions about distributional effects and ethical concerns

(for example, that this may have unintended or unfair consequences for certain

groups, particularly based on age), and practical challenges of robustly

accounting for unrelated healthcare costs. This is an active area of research,

but is as yet unresolved. We consider that these concerns are sufficiently


serious to advocate caution, so there is no case for change at this stage (that

is, unrelated healthcare costs should not be included in evaluations). Further

research in this area is important, and the case for change should be

reconsidered as research progresses.

27.The review identified further improvements to the methods for estimating costs

and resource use in health technology evaluations. We propose that methods

for using non-UK costing studies (including currency conversion) and for

adjusting older cost data for inflation should be aligned with the methods used

by NICE’s Centre for Guidelines. No changes to the methods for using

microcosting studies, expert elicitation or formal carer costs are needed.

Health-related quality of life

28.NICE’s preferred measure of health-related quality of life is the EQ-5D health

questionnaire. The review concluded that the evidence suggests that EQ-5D

works well for most diseases and conditions, and we consider there is no case

to change this preference. However, we recognise that there are

circumstances in which challenges with using EQ-5D need further

consideration.

In some circumstances, EQ-5D is not appropriate or is not available.

We have identified a case for change to provide additional clarity and

guidance. We propose to include a hierarchy (as described in the task

and finish group report) that draws together these situations and

outlines the evidence requirements and potential approaches.

We note the particular challenge in collecting EQ-5D data or proving

that it is an inappropriate measure for rare diseases. There is no

evidence that EQ-5D is systematically inappropriate for rare diseases,

so it is correct that we need specific evidence of inappropriateness if

EQ-5D is not to be used. However, we consider that it is appropriate to

include a degree of flexibility in the evidence requirements to account

for these challenges.

An additional challenge for EQ-5D is the choice between the 3-level

and 5-level questionnaires (EQ-5D-3L and EQ-5D-5L). NICE’s position


https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/technology-appraisal-guidance/eq-5d-5l

statement on use of the EQ - 5D - 5L value set for England says that the

EQ-5D-5L questionnaire may be used to collect quality-of-life data but

that the UK EQ-5D-3L value set should be used. Most of this position

statement will be incorporated directly into the updated methods.

However, there is a case for changing the preferred method for

mapping from EQ-5D-5L questionnaire data to the EQ-5D-3L value set.

The review identified that an alternative tool developed by NICE’s

Decision Support Unit (DSU) has additional functionality and is

informed by a larger dataset compared with the previously preferred

tool (by van Hout et al. 2012). We propose that the preferred method

for mapping from EQ-5D-5L to EQ-5D-3L should be using the DSU’s

tool, based on the dataset collected by Hernández Alava and

colleagues (the ‘EEPRU’ dataset).

29.The review identified 2 important areas of significant difficulty: health-related

quality of life in children and young people, and in carers. These topics are

separate and have different challenges, but the common outcome is a critical

need for further research.

Health-related quality of life in children and young people: Changes to the methods are proposed to improve the quality of

evidence for health-related quality of life in these groups. Substantial

research is needed to explore the appropriateness of different

measures of health-related quality of life in children and young people,

and to identify appropriate valuation methods. Several research

projects are under way, and NICE is actively engaged in supporting

and considering this work.

Carer health-related quality of life: There is no case to change the

current option to include carer health-related quality of life when

relevant, but further research is needed to clarify when and how this

can be done. This research must consider the effects of displacement

and opportunity cost, that is, healthcare displaced by a new technology


https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/technology-appraisal-guidance/eq-5d-5l

would itself have effects on carers and family members, which must be

taken into account.

In both cases, it will not be possible to complete the further research needed

within the timeframe of this methods review. We will prioritise reviewing and

reconsidering our methods in this area in subsequent phases of methods

review, as the research progresses.

30.When health-related quality of life is extrapolated over a long time horizon in

an economic model, it may need to be adjusted over time (for example, for the

effect of age or comorbidities). The review concluded that this is appropriate,

and identified a need to define a preferred approach (with the option to provide

evidence that adjustment is not appropriate in individual cases). Exploratory

analyses confirmed that this technique does not disproportionately affect older

people. We consider that it is a proportionate means of achieving the

legitimate aim of accurately modelling health-related quality of life.

31.We propose a further change in the health technology evaluation methods to

improve the quality of outcome-measure selection and reporting. Outcome

measures in studies should be selected in consultation with people with the

condition or disease. A high-quality ‘core outcome set’, developed with input

from these people, may help with outcome selection. Patient-reported

outcomes can capture important aspects of conditions and interventions.

Patient-reported outcome measures should be appropriately validated, and the

methods used to collect the data should be clearly reported.

Understanding, characterising and presenting uncertainty

32.As well as exploring how uncertainty is considered as a modifier in decision

making (see section 5), the review also explored whether there was a case to

update NICE’s methods for understanding, characterising and presenting

uncertainty in health technology evaluations. This was done by the exploring

uncertainty task and finish group. The overriding objective of this element of

the review was to ensure that uncertainty is fully and robustly characterised

(both quantitatively and qualitatively) to clearly identify the nature, size and


effect of uncertainties across evaluations. This is crucial for decision making,

particularly when doing earlier evaluations for innovative technologies. It also

aligns with the need to clarify NICE’s approach to managing uncertainty, to

brief committees on the types of uncertainty, and to make sure that they focus

on areas of uncertainty that have the most significant effect (referred to in the

2019 Voluntary Scheme).

33.Fundamental to the success of this objective is that uncertainty is clearly

presented to decision makers and stakeholders. This applies to all forms and

sources of uncertainty, across both clinical effectiveness and economic

analyses. There is a case to include an overall assessment of uncertainty in

health technology evaluations. This should describe key components

including: the nature and effects of different types of uncertainty; whether it is

captured in quantitative analyses; and to what extent it may be resolved,

reduced or mitigated by additional evidence or expert input. The second stage

of the methods review will give further details of how this assessment will be

implemented. The current review also identified several different options for

presenting and visualising uncertainty, including net benefit rankings,

categorisations of uncertainty and evidence generation over time. Each of

these options had some limitations and generated debate. Overall, there

remains a case to continue developing and improving NICE’s approaches to

presenting and visualising uncertainty in health technology evaluations.

However, this does not need to be restricted to a formal methods review.

34.One of the key components of uncertainty is around the data and inputs that

inform the economic model. This is referred to as parameter uncertainty. The

review identified a case to reaffirm and bolster the importance of probabilistic

analysis as a tool to address parameter uncertainty. Probabilistic analysis

should be used as the starting point for analysis, for committees’ preferred and

decision-making cost-effectiveness estimates, and for scenario analyses,

unless clearly justified. This provides a more accurate estimate of the

incremental cost-effectiveness ratio (ICER), particularly when the model is

non-linear (that is, when the results of deterministic and probabilistic analyses


are not similar), and captures the spread of the estimate. Greater emphasis on

the use of probabilistic analyses strengthens the need to ensure that

probabilistic analyses are well carried out. The review identified 2 cases for

change to improve this: analyses should include enough simulations to

minimise the effect of Monte Carlo error, and should use appropriate methods

that account for ordering and correlation in parameters.

35.The review also explored tools that can help identify and explore the

uncertainties and parameters that may be particularly important for health

technology evaluation committees to focus on:

Univariate sensitivity analyses remain a valuable tool, particularly for

identifying parameters to which the cost effectiveness is particularly

sensitive and for guiding the committee discussions. Although there are

known limitations of univariate analyses (particularly for closely

correlated parameters), there is no case to depart from their use

entirely. The review identified a case to allow probabilistic univariate

analyses as an option, to overcome some of the limitations of

deterministic approach. But the case was not sufficient to make this a

routine, mandatory requirement at this stage.

Threshold analyses are also a valuable tool for exploring individual

parameters and uncertainties. There is a case to include threshold

analyses as an option in the methods, with appropriate commentary to

note their strengths and limitations. Importantly, threshold analyses

should not be used to identify subgroups or cut-offs for restricting a

recommendation.

36.When exploring uncertainty in an economic model, it is important to take into

account the plausibility of the parameters and assumptions that are being

used. It is perhaps self-evident that committee decisions must be based on

plausible inputs and assumptions that are consistent with the evidence.

Nevertheless, there is sometimes value in exploring implausible values to test

the function of the model or show relevant features of an analysis (for

example, to show that a particular uncertainty is irrelevant because it does not


affect the results even when taken to implausible extremes). There is a case

for change to ensure that the methods have sufficient flexibility to allow such

analyses, but also to clearly label such analyses with their purpose and

emphasise that they are not suitable for decision making. The plausibility of

parameters and assumptions is routinely explored by committees with relevant

experts, and the review noted that it may be desirable to expand this to more

formal elicitation approaches. The practicality of including this step in the

process is beyond the remit of the health technology evaluation methods

review.

37.The structure and assumptions in an economic model can significantly affect

its results. So, the associated uncertainties are an important routine

consideration in health technology evaluations. The review identified a case to

supplement the description of how model concepts and structures should be

established, to ensure that this is robust and transparent. The review also

explored advanced techniques to incorporate structural uncertainties along

with parameter uncertainties in a single analysis, by parameterising the

structural uncertainties. This would be highly desirable. However, the

challenges and limitations of the available methods are potentially important.

We therefore consider that there is a case to include these techniques as

options in the methods guidance, and to encourage their use. However, the

case has not yet sufficiently been made for it to be a mandatory part of routine

decision making. Further research and consideration of these techniques

would be welcomed.

38.A frequent source of significant uncertainty is in extrapolating beyond the

available data, for example, over time. The review identified several cases to

improve and expand on the methods guidance for extrapolation, by:

clarifying the plausible extrapolation scenarios that might be considered

acknowledging standard and flexible extrapolation models

explicitly requiring consideration of the plausibility of hazard functions

and uncertainty.


39.Value of information analysis is a potentially important tool in assessing

uncertainty, with particular potential for recommendations associated with

additional evidence collection. This is an area of considerable interest, and the

review highlighted substantial variation in opinion between different groups. On

balance, we conclude that, given the importance of fully characterising and

assessing uncertainty in NICE’s health technology evaluations, there is a clear

case to embrace methods that support this whenever possible. However, there

remain significant challenges associated with value of information methods,

including their technical application and how the results inform decision making

in different circumstances. This is pertinent in the context of managed access

and evidence generation, which is a dynamic situation and is beyond the remit

of the methods review to influence. Therefore, we propose that there is a case

to introduce wider use of the expected value of perfect information (or other

equivalent concepts, such as payer uncertainty burden) as an additional tool to

help committees understand the likelihood and consequences of decision error

for parameter uncertainty. This will need minimal additional resources to

implement, but provides additional information for committees to inform their

overall deliberative view of the technology and the uncertainties. Further work

is needed, in the second stage of this review, to explore in more detail how the

expected value of perfect information is factored into decision making.

However, there is no case to introduce further use of other value of information

methods in routine decision making at this stage. There is a need to consider

this further as approaches to managed access evolve.

Structured decision making

40.The decision-making task and finish group reviewed a range of considerations

for how evidence is presented and considered to reach health technology

evaluation recommendations. It was accompanied by the equalities task and

finish group, which focused on issues of equity, fairness and equality and

considered whether there were any cases to change the methods to capture

these issues in recommendations. The proposals from these reviews, along

with those from the other elements of the methods review, will be brought


together into a consolidated decision-making framework during the second

stage of the methods review.

41.The review explored 2 key aspects of how cost-effectiveness analyses should

be presented and considered in decision making.

Incremental and pairwise comparisons: The review identified no

case to change NICE’s normal preference for fully incremental

economic analyses. This finding was supported by the same view from

the sources and synthesis of evidence task and finish group. However,

the review also identified a case to allow pairwise comparisons in

economic analyses when relevant and justified, for example, when the

technology is expected to specifically displace individual comparators.

Net benefit approaches: The methods currently allow presenting net

health or net monetary benefits alongside incremental cost-

effectiveness ratios. The review found that this may be particularly

helpful when there are several interventions or comparators, and when

there are small differences in costs or QALYs between the technologies

being compared. Net benefits may also be helpful in presenting

uncertainty (highlighted in the review of ATMPs and histology-

independent cancer treatments; see sections 63 and 70) and for some

subgroup considerations (see section 44). Although not a fundamental

change in methods, there is a case to emphasise and expand the use

of net benefit approaches. When net benefits are included, net health

benefits are preferred over net monetary benefits. Although the 2 are

equivalent, including a preference for net health benefits supports

consistency and more intuitively reflects the objectives of health


42.The current methods of health technology evaluation do not include guidance

on how to consider technologies that provide less benefit at a lower cost than

their comparator (the south-west quadrant of the cost-effectiveness plane).

Such technologies are important. They offer additional options for patients and

clinicians. They also have the potential to improve overall health outcomes in


the NHS if they free up sufficient resources that can be reinvested. We

recognise a potential concern that supporting decisions purely based on cost

savings may be clinically inappropriate and give worse outcomes for individual

patients. But we emphasise that this is not the purpose of a health technology

evaluation. In all cases, if a technology is recommended, it is recommended as

an option, so that it is made available if a patient and their clinician consider it

appropriate. We therefore propose that there is no case to change the overall

approach in this circumstance, but only to clarify how decisions should be

made. The review identified that decisions on whether sufficient resources are

freed up should take into account the same cost-effectiveness levels that are

used for cost-increasing technologies, that is, £20,000 to £30,000 per QALY.

Further work is needed to establish how any modifiers might apply in this

setting, and how this might be considered if it were to occur in a highly

specialised technology evaluation. The review also noted that net benefit

approaches may be particularly informative for technologies in the south-west

quadrant.

Subgroups

43.NICE’s recommendations often consider the clinical and cost effectiveness of

technologies in defined subgroups of the population, and there is substantial

methodological guidance to ensure that this is done robustly. The review

identified cases for minor changes to support good conduct of subgroup

analyses, including presenting absolute and relative treatment effects and

reviewing the credibility of subgroup effects.

44.A more substantive challenge was the role of subgroups when the technology

is found to be clinically and cost effective in a whole population (for example,

the population covered by the medicine’s marketing authorisation). This may

depend on more fundamental questions of the role of NICE’s

recommendations about supporting access and ensuring optimal use of NHS

resources. NICE must take into account both of these responsibilities. It is

always important to try to secure access to technologies for the widest

possible population, and also to ensure cost-ineffective uses of technologies


are not supported to avoid health being displaced elsewhere in the NHS.

Therefore, we conclude that committees must be able to not recommend a

technology in a particular subgroup (that is, to exclude a subgroup from the

recommendation), even if the technology is clinically and cost effective in the

whole population, if they consider it appropriate. In common with any subgroup

or optimised recommendation, any such decision must be based on an

appropriate consideration of subgroups, in line with the current methods

guidance, to ensure that the decision is clinically justifiable, methodologically

robust, ethical, and lawful under equalities legislation. The committee should

be particularly aware of the benefits and harms (to individuals and to the NHS

as a whole) of including or excluding a given subgroup. Net benefits

approaches may be informative in this situation. If considering excluding a

subgroup, the committee must be convinced the harm to the NHS of including

it is sufficiently great to justify this decision.

45. In exploring methodological issues for evaluating technologies that sit at

different points in care pathways, the review identified that this is essentially a

particular type of subgroup consideration. We therefore propose that the same

principles described for subgroups apply to technologies in care pathways, and

there is no further case for change.

Not cost effective at £0

46.Rarely, some health technologies that are clinically effective are found not to

be cost effective even if priced at £0. This is a substantial area of debate, and

proved particularly difficult for the methods review. The task and finish group

and working group did not reach a consensus on the extent to which this is a

problem that needs a solution or is just a feature of a small number of

economic analyses. Nevertheless, it was clear that further exploration was

needed.

47.The review explored in detail the circumstances in which this situation might

occur, and 3 themes were identified:


If it occurs from a direct, intrinsic consequence of the technology being

evaluated, such as administration costs or costs for a high-cost

complication of the technology, it is difficult to establish a case for

changing the usual approach. It is likely that the technology is

unfortunately not cost effective because the benefits it provides are

insufficient to justify the costs to the NHS.

If it occurs because the technology is used with another branded

technology, it is first-and-foremost a commercial issue, so is outside the

remit of the methods review.

If it occurs because the technology is used with, or leads to later use of,

high-cost healthcare that the NHS has already decided to provide, it

becomes challenging and there is a potential case for change. NICE

technology evaluations would not seek to support or encourage non-

cost-effective practice in the NHS. However, it may not be within the

reach of an evaluation committee to reconsider a funding decision that

has been made by the NHS. Perhaps of most concern in this situation

is that people may be unfairly disadvantaged in accessing new

technologies because of other treatments they are having or will have,

which may create equalities issues.

48.Overall, we conclude that there is no case to change NICE’s usual approach

for health technology evaluations. That is, in the reference case, the full costs

and health effects associated with introducing a new technology must be

considered. However, in extreme, exceptional circumstances, there is a case

to depart from the reference case for technologies that are not cost effective at

£0, by removing or adjusting specific high-cost care. This would not apply

when commercial considerations are relevant (for example, for combinations of

branded medicines). Also, it would not apply when this situation occurs as a

direct, intrinsic consequence of the technology being evaluated (such as

administration costs, adverse effects or complications). It would apply when

the high-cost care is a primary driver of the cost-effectiveness results, and only

if this care is something that the NHS has decided to routinely offer.

Committees should be particularly aware of this situation when it is associated


with a disability and there is the potential for people to be unfairly

disadvantaged.

49.Further work is needed to refine the situations and circumstances in which the

non-reference-case approach should apply. This will include considering

whether technologies that are not cost effective at exceptionally low prices

should be included and, if so, how. Further work is also needed to define how

the non-reference-case analyses should be done. This should take into

account the need to ensure that both the costs and health effects of the high-

cost care are appropriately considered and adjusted.

Presenting information and recommendations

50.There is an important need for NICE to continually review and improve how

information is presented to committees and to stakeholders. Although the

current review did not identify any specific cases for change that need to be

implemented through this methods review, there remains a case to continue to

develop and improve NICE’s approaches to presenting information as part of

health technology evaluations. This overlaps with the aims of the NICE

Connect programme.

51.One of the key pieces of information that we present is the recommendations.

There is a case to update how we word health technology recommendations,

simplifying them and ensuring consistency between programmes, to help

stakeholder understanding. Any update needs to allow sufficient flexibility for

each programme to provide the information needed by its audience. When

recommending that evidence is generated, we should align with NICE’s

research recommendations manual. Evidence generation in managed access

is a rapidly evolving area, and further work is likely to be needed as this

proceeds.

52.The review considered how recommendations should be made when there are

several similar options. There is a simple case to align approaches between

programmes, but a more complex discussion is needed to explore how we

should approach this situation. We need to balance the need to support the


central role of patients and clinicians in identifying and choosing the most

appropriate option with the need to support the most cost-effective use of NHS

resources. Overall, we consider that committees should have the option to

specify in their recommendations what considerations should be taken into

account when choosing between similar options, if they consider it appropriate.

These considerations may include costs. However, this should not be

mandatory for committees, and recommendations should not require that the

cheapest option must always be chosen. Further work is needed to identify

how to implement this in practice.

53.The review also explored ways that NICE could improve how it supports

people to take part in health technology evaluations. This is important, but is

outside the remit of the current methods review. It is therefore not discussed

further here, but will be taken forward in other projects.

Value judgements and decision-making frameworks

54.The process of developing an updated decision-making framework sits in the

second stage of the methods review. At this stage, the review explored

whether there was a case to follow a specific structured framework. It found

that there was no case to change to a fully quantitative version of multicriteria

decision analysis, and that the current deliberative decision-making approach

should be kept.

Equality considerations

55.Equality, equity and fairness are central to all health technology evaluations at

NICE. The role of the equalities task and finish group was to consider the case

for general changes to NICE’s health technology evaluation methods to bolster

and support equalities considerations. The equalities implications of individual

components of the methods and any accompanying cases for change were

explored individually by the other task and finish groups, and are summarised

in the equality assessment section of this document.

56.Among the key findings from the review was a reaffirmation of the importance

of considering equalities across the related ongoing development projects.


This includes the process review, topic selection process update and the

review of the topic selection criteria for the highly specialised technologies

programme. These should take into account approaches at other health

technology assessment bodies.

57.The review emphasised that equalities considerations must be taken into

account when deciding whether a health technology should be selected for

evaluation. For medicines, the 2019 Voluntary Scheme’s commitments to

expand NICE’s assessments restricts the scope for equalities implications.

However, when assessing significant new indications and considering

rationales not to do an assessment, the equalities implications must be

considered. For medical technologies, equalities should continue to be a

routine consideration for all topic selection and routing decisions.

58.The review identified 2 further potential cases for change that need further

exploration:

There is a potential case to explore further how equality issues

associated with implementing technologies in NHS practice might feed

into evaluations. This includes, for example, equity of access to the

technology in practice.

There is a potential case to explore equality considerations around

methods for qualitative evidence.

59.There is no case for changing NICE’s use of QALYs in health technology

evaluations on the grounds of equity. QALYs have been criticised, but the

review identified that proposed alternatives to QALYs also have limitations and

have too little evidence to support a change. Committees should consider

equity-related limitations of QALYs when relevant.

Challenging technologies, conditions and evaluations

60.The technology-specific issues task and finish group explored methodological

issues and adjustments that might occur because of particularly challenging

technologies. It focused in particular on medical devices and how they are


affected by new regulations, ATMPs, histology-independent cancer treatments,

and future technologies. It also considered draft findings from other task and

finish groups in the context of these technologies, to ‘stress test’ their

proposals for these specific challenging topics. In this section, we also explore

the challenges associated with rare diseases, and describe the considerations

given to rare diseases across the 10 task and finish groups.

Medical device regulations

61.The review of new medical device regulations (including the European Union

regulations for medical devices [MDR]) found minimal case for changing

NICE’s evaluation methods. It found that, in general, NICE’s evidence

requirements are more comprehensive than those specified in the regulations.

This is appropriate given the needs of NICE health technology evaluations.

The review did explore the role of evidence that has not been published in

peer-reviewed journals in medical technologies evaluations. The European

Union regulations for medical devices places greater emphasis on unpublished

evidence and post-marketing surveillance data that may be informative for an

evaluation. Other NICE health technology evaluations give little emphasis to

whether data have been peer reviewed because of the role of the academic

review and committee evaluation. There is therefore a case to consider

unpublished and post-marketing data in medical technologies evaluations.

ATMPs

62.Comprising gene therapies, cell therapies and tissue-engineered products,

ATMPs have the potential to provide substantial health benefits, including

potentially curative options for serious conditions. As a result, ATMPs are often

considered to be an important and high-priority group of technologies. NICE

has recommended almost all of the ATMPs that it has evaluated to date

(although many of the recommendations are optimised). However, the

evaluations have often been challenging, which suggests that there may be

potential for improvement in the context of the current high success rate.


63.The review identified several prominent challenges for ATMPs that may affect

their evaluation:

There are often important uncertainties, particularly in long-term

outcomes.

The technologies have high upfront costs (which cannot be recovered).

They are particularly affected by factors such as discounting, service

redesign costs and value not captured in QALYs.

Although these issues are common for ATMPs, they are not unique to this

technology type. There is therefore no case to make specific provision for

ATMPs in the methods, but there are some cases for making general changes

that are of particular relevance to ATMPs:

Evaluations in which there is uncertainty about long-term benefits

should include scenario analyses exploring the effects of different

assumptions about long-term benefits, potentially including threshold

analysis for treatment effect duration.

Cure-proportion modelling (a survival modelling method based on a

proportion of patients being ‘cured’ of the condition) should be

considered as an option.

Net health benefits approaches may help in presenting uncertainty in

this context (highlighted in the recommendations from the decision-

making task and finish group; see section 41).

For evaluations including significant service delivery effects, NICE

should seek information from relevant system partners (including, when

relevant, the Accelerated Access Collaborative). There is no further

case for changing NICE’s approach to service delivery costs in health

technology evaluations beyond the considerations made for

apportioning costs (see section 25).

64.Although they offer potentially important benefits for patients, the high, upfront,

often one-off costs of ATMPs can create challenges for affordability and

implementation in the NHS. This does not affect NICE’s assessment of the


value of the health technology, so there is no case for changing the evaluation

methods. Many of the options for mitigating this challenge are commercial in

nature, and so are outside the remit of this review.

Histology-independent cancer treatments

65.Histology-independent cancer treatments are an exciting development, with

the potential to substantially change how cancer is considered and treated.

They are another high-priority area for many stakeholders, including the NHS

Accelerate Access Collaborative, and pose significant methodological

challenges for health technology evaluation. The first 2 histology-independent

cancer treatments were successfully appraised by NICE in 2020. However, it is

still important to consider the methodological challenges as part of this

methods review.

66. In common with ATMPs, there is no case to make specific provisions in the

methods guidance for histology-independent cancer treatments. Rather, any

changes these technologies trigger should be incorporated as generally

applicable methods. People taking part in future evaluations of histology-

independent cancer treatments may benefit from the extensive exploration of

the specific methods challenges for these technology types in the report by the

universities of York and Sheffield, and the accompanying review by the task

and finish group.

67.The review highlighted particular challenges in establishing estimates of

relative treatment effectiveness in the absence of a comparator in a clinical

trial, and when the validity of available historical control data is uncertain.

These challenges can certainly occur for histology-independent cancer drugs,

and also in many other settings. We propose cases for change to recommend

that when basket trials (a type of clinical trial in which a technology is studied

in several different groups, usually with the same biological or genetic marker)

are used, they should be appropriately designed and analysed. When trials are

presented without a comparator, several methods to derive the comparator

data should be explored. These cases are broadly applicable and may be

helpful in a variety of challenging evaluations.


68.Another important theme from the review was heterogeneity across tumour

types and, related to this, generalisability from clinical trials to clinical practice.

Again, challenges of heterogeneity, generalisability and unrepresented groups

are common to many evaluations, although to a different degree. Overall, it is

desirable to treat histology-independent indications as a single indication (and

not as a collection of tumour-specific subgroups). However, in practice, it will

be difficult to justify not considering different tumour types to a significant

extent in the evaluation. Any assumptions about homogeneity, heterogeneity

and generalisability to clinical practice must be clearly presented, tested and

fully explored. The review notes the potential value of Bayesian hierarchical

models in this context, although we consider there is no need to refer to this

specific method in NICE’s general health technology evaluation methods

guidance.

69.Depending on its view of heterogeneity and generalisability, a committee may

consider whether to recommend a technology for the whole indication covered

by the marketing authorisation or for a specific (set of) subgroups. This will

include considering tumour histologies that are not represented in the clinical

trials. In this context, the committee must take into account the appropriate use

of subgroups, as set out in the current methods guidance and explored in

sections 43 to 45. Committees should be particularly aware of their roles in

seeking the broadest possible access for clinically and cost-effective

technologies and in optimising the use of NHS resources. They should also

consider the benefits and harms (to individuals and to the NHS as a whole) of

including or excluding a given subgroup. The review suggested defining

subgroup analyses at the scoping stage, which is desirable if possible.

However, it is inappropriate and unfeasible to make a definitive decision on

subgroups at the scoping stage because of the limited evidence and critique

available. Overall, there is no case for changing the general methods of health

technology evaluation for subgroup considerations.

70.There were 3 further areas with important challenges, but no further case for

change was made:


Costs of genetic testing: this was explored as part of the consideration

of apportioning costs (see section 25). No further case for change is

made.

Model structure: we recognise the challenges with model structures

that are commonly used in cancer (particularly partitioned survival

models), but would not normally make general recommendations about

specific model structures. The proposals for model conceptualisation

(see section 37) and using surrogate outcomes (see section 21) are

relevant here.

Uncertainty: this is explored in detail in previous sections. We note the

potential value of net benefit approaches in quantifying uncertainty and

informing subgroup considerations, which further supports the proposal

to expand the use of net benefits approaches (see section 41).

71.The review explored how to apply NICE’s current considerations for life-

extending treatments at the end of life for histology-independent cancer

treatments. Strictly speaking, this is no longer relevant because of the proposal

to remove the end of life considerations (see section 5). Nevertheless, the

same question will remain for any modifier that is in place, particularly for

quantitative modifiers. In the next stage of the review, when considering how

modifiers should be implemented, we will consider specific issues for applying

modifiers in circumstances such as histology-independent cancer treatments.

Other emerging technologies

72.Given that a key objective of the review is to ensure that the methods are fit for

the future, there is a need to review potential methodological challenges for

emerging new technologies.

73.Considering near-term developments first, the task and finish group reviewed

technologies that are expected in the coming months and years. It identified

several relevant methods considerations. Taking into account other proposals

in this review, no further cases for change are identified.


74.Longer term, it is important that NICE’s health technology evaluation methods

allow us to remain flexible and responsive to emerging changes. NICE

previously committed to making future methods updates more dynamic by

allowing more regular, modular updates. The methods review identified a need

to improve NICE’s active horizon scanning for upcoming methods challenges

for new technologies and innovations. This will allow us to proactively consider

and respond to those challenges, and avoid unnecessary delays to patient

access. Although not strictly a methodological change, it is appropriate to

document this case for change in the current review.

75.Several of the proposals in this review described in previous sections have

significant importance for ATMPs, histology-independent cancer treatments

and other emerging technologies. These include:

improved description and characterisation of uncertainty

clarifying the approach to uncertainty, innovation and other factors as

modifiers

apportioning the costs of diagnostic tests and service design

improving how evidence is presented and explored, including sourcing

and synthesising evidence and estimating health-related quality of life.

Taken together, the proposals in this methods review and the high degree of

flexibility in NICE’s methods and decision-making approach provide a broad

package that will ensure NICE can robustly and efficiently evaluate innovative

technologies in the future, including ATMPs and histology-independent cancer

treatments. These technologies will inevitably remain challenging topics to

evaluate. However, the proposals ensure that there are no outstanding

methodological barriers to fast access for patients to clinically and cost-

effective innovative technologies.

Rare diseases

76.Methodological issues for rare diseases span the breadth of this methods

review, and so were explored by all of the task and finish groups. Two groups

were tasked with exploring a specific question about rare diseases:


Modifiers: this group considered whether there was a case to include a

specific modifier for rarity in decision making. We propose that there is

a case to accept a greater degree of uncertainty in evaluations for rare

diseases (see section 5).

Health-related quality of life: this group considered methods for

measuring health-related quality of life in people with rare diseases

(and, more generally, for rare health states). The proposed hierarchy of

evidence sources will be helpful for many evaluations in rare diseases.

Also, including flexibility in the evidence requirements to show whether

EQ-5D is appropriate ensures rare diseases are not blocked by an

inappropriate evidence barrier (see section 28).

77.Rare diseases have particular challenges in collecting evidence. So, the

review of methods for sourcing and synthesising evidence considered ways to

ensure that NICE’s health technology evaluation methods are suitable for rare

diseases. Although NICE must maintain its rigorous assessment of evidence

across rare diseases, this need not be seen as a criticism or a barrier to

accessing new technologies. Interpreting the evidence and the accompanying

critique will always be proportionate and appropriate to the context. The review

identified areas in which additional flexibility is needed for rare diseases, which

are included in the proposals. It noted that qualitative evidence, expert

elicitation and surrogate outcomes may be particularly relevant in rare

diseases, so additional clarity and guidance in the methods may be beneficial.

Similarly, the review acknowledged the challenges in quantifying uncertainty in

rare diseases, for example, in doing full probabilistic analyses. It aimed to

ensure that there is sufficient flexibility in the methods to accommodate rare

diseases in a full and robust assessment of uncertainty.

78.When implementing the methods proposals, there is a case to avoid including

specific provisions for rare diseases as much as possible. This is to ensure

that the methods apply to any evaluation. If it is necessary to make specific

provision (for example, in considering uncertainty and risk), it will be important

to carefully define what is meant by a rare disease. This must take into


account the changing understanding and classification of disease, and the

increasing targeting of treatments to smaller groups within larger disease

areas (often referred to as precision medicine).

Aligning methods across programmes

79.One of the key objectives of the methods review is to align the methods across

the 4 health technology evaluation programmes as much as possible. We

acknowledge that there are some relevant differences between programmes

and technology types that may need different approaches. However, most of

the methodological principles are common and so can be aligned. Most of the

alignment will be done in the second stage of the review (that is, after the

current consultation), as part of developing a consolidated programme manual.

80.A key focus of alignment for this stage of the review is on methods for

comparing costs of technologies separately from considering their health

benefits. This is most relevant to the medical technologies evaluation

programme and cost-comparison methods in the technology appraisals

programme. Currently, the methods use several different terms and methods

for similar approaches, so there is an immediate case for change to simplify

and align the methods guidance. The review was done by the alignment of

cost-comparison methods task and finish group. Importantly, this review is not

able to consider fundamental changes to the types of analysis that are

available in each evaluation programme, nor the way in which topics are

selected or routed to programmes.

81.The review identified a case to clarify the terminology used to describe

different types of analysis. The task and finish group proposed defining

3 methodological approaches: cost-comparison analysis, extended cost-

comparison analysis and cost–utility analysis. We propose that, in practice, it is

not necessary to differentiate between a ‘standard’ and ‘extended’ cost

comparison. We therefore propose a single approach for cost-comparison

analysis that can be used, when relevant, across all 4 technology evaluation

programmes. In a cost-comparison analysis, the costs associated with the

technology are compared with those of the relevant comparator. Health effects


(for example, clinical outcomes and QALYs) are not considered in this

analysis, but elsewhere in the evaluation. If there are relevant differences in

health or process outcomes that affect resource use, the associated costs

should be included in the analysis (although substantial differences in health

outcomes may not be consistent with a claim of clinical similarity). We

acknowledge that the term ‘cost-comparison analysis’ differs from terms more

commonly used in health economic literature (notably ‘cost minimisation

analysis’). This is appropriate because it is conceptually different. Cost

minimisation starts by assuming clinical equivalence, whereas NICE’s cost

comparisons in health technology evaluation make no clinical assumptions and

simply focus on costs (with clinical considerations explored elsewhere in the

evaluation).

82.Evaluations that include cost-comparison analyses also often consider

evidence of similarity, equivalence and non-inferiority. It is helpful to note the

distinction between these terms because they are not interchangeable.

Equivalence and non-inferiority have specific, defined, statistical meanings.

Similarity, in the context of a NICE evaluation, refers to a pragmatic, practical

consideration based on all of the evidence available at the time, and its

strengths, limitations and uncertainties. Based on the review of evidence, we

identify a case to improve how we take into account plausibility in considering

similarity.

83.There is no case to change the methods for doing cost-comparison analyses

because the review found the current health technology evaluation methods

guidance to be sufficient. It is helpful to emphasise that, for all cost-comparison

analyses, appropriate analyses of uncertainty should be done. These analyses

should be proportionate and consistent with the nature of the analysis and the

potential uncertainties. They might include sensitivity analyses, scenario

analyses and probabilistic analyses. Probabilistic analyses are particularly

relevant when there are multiple sources of uncertainty that may interact.


· web view2020. 11. 5. · sciences sector. as this process unfolds, it is important to clearly...

Documents