Utilising rank and DCE data to value Utilising rank and DCE data to value health status on the ‘QALY’ scale health status on the ‘QALY’ scale using conventional and Bayesian using conventional and Bayesian

methodsmethods

John Brazier and Theresa Cain John Brazier and Theresa Cain

with Aki Tsuchiya and Yaling Yangwith Aki Tsuchiya and Yaling Yang

Health Economics and Decision Science, ScHARR, Health Economics and Decision Science, ScHARR,

University of Sheffield, UKUniversity of Sheffield, UK

Prepared for the CHEBS Focus FortnightPrepared for the CHEBS Focus Fortnight

OutlineOutline

Concerns with current cardinal methods for valuing Concerns with current cardinal methods for valuing health stateshealth states

Problems in using ordinal data Problems in using ordinal data

Application of rank and DCE methods to valuing Application of rank and DCE methods to valuing Asthma health states using conventional methodsAsthma health states using conventional methods

Application of Bayesian methods to analysing DCE Application of Bayesian methods to analysing DCE datadata

Implications for research and policy Implications for research and policy

Problems with cardinal Problems with cardinal methods for valuing health methods for valuing health

statesstates

TTO and SG seen to be cognitively TTO and SG seen to be cognitively complex tasks that may be too difficult complex tasks that may be too difficult for some (e.g children, very elderly)for some (e.g children, very elderly)

TTO values contaminated by time TTO values contaminated by time preference, standard gamble by risk preference, standard gamble by risk attitude and rating scales by end point attitude and rating scales by end point bias (among other things)bias (among other things)

Role for ordinal methods (rank and Role for ordinal methods (rank and discrete choice)discrete choice)

Ordinal tasks: Ordinal tasks: Ranking and discrete choice Ranking and discrete choice

experimentsexperiments Ranking respondents asked to order Ranking respondents asked to order

a set of health states from best to a set of health states from best to worst - traditionally used as a warm worst - traditionally used as a warm up exercise prior to VAS/SG/TTO up exercise prior to VAS/SG/TTO based preference elicitationbased preference elicitation

Discrete choice experiments (DCE) - Discrete choice experiments (DCE) - typically asks respondents to choose typically asks respondents to choose between two health states (A and B)between two health states (A and B)

Problems with using Problems with using ordinal data to value ordinal data to value health for QALYshealth for QALYs DCE and rank models estimate a DCE and rank models estimate a

latent health state utility value, latent health state utility value, but with arbitrary anchorsbut with arbitrary anchors

QALYs require health states to be QALYs require health states to be valued on the full health (one) valued on the full health (one) and being dead (zero) scaleand being dead (zero) scale

Key problem is linking results of Key problem is linking results of DCEs to the full health-dead scaleDCEs to the full health-dead scale

Previous work using Previous work using ordinal dataordinal data

RankingRanking Early application of Thurstone’s method by Kind (1982)Early application of Thurstone’s method by Kind (1982) Use of conditional logit on rank data by Salomon Use of conditional logit on rank data by Salomon

(2003) on EQ-5D and McCabe et al (2005) on SF-6D (2003) on EQ-5D and McCabe et al (2005) on SF-6D and HUI2 – some successand HUI2 – some success

DCEDCE DCE applications in health economics mainly DCE applications in health economics mainly

concerned with relative weight of different attributes of concerned with relative weight of different attributes of health care rather than to valuing health per sehealth care rather than to valuing health per se

DCE considered unsuitable for assessing cost DCE considered unsuitable for assessing cost effectiveness (because utility scale is not comparable effectiveness (because utility scale is not comparable between studies)between studies)

Past attempts to apply Past attempts to apply DCE to valuing HRQoLDCE to valuing HRQoL Hakim and Pathak (1999) applied DCE to valuing EQ-Hakim and Pathak (1999) applied DCE to valuing EQ-

5D states5D states- used ‘pick one’ from 12 choice sets (each containing 3 used ‘pick one’ from 12 choice sets (each containing 3

states plus dead)states plus dead)- Exploratory and did not produce weightsExploratory and did not produce weights

McKenzie et al (2001) estimated weights for asthma McKenzie et al (2001) estimated weights for asthma symptomssymptoms

- no link to full health-dead scaleno link to full health-dead scale

Viney et al (2004) included attributes for HRQoL and Viney et al (2004) included attributes for HRQoL and survival – but did not estimate health state values survival – but did not estimate health state values

Alternative Alternative approaches to using approaches to using DCEDCE The latent utility scale needs to be anchored on The latent utility scale needs to be anchored on

the full health-dead scale and there are a the full health-dead scale and there are a number of different ways: number of different ways:

Value PITS state externally by TTO/SG (Ratcliffe Value PITS state externally by TTO/SG (Ratcliffe and Brazier, 2005)and Brazier, 2005)

Include a dead state in the pair wise choice Include a dead state in the pair wise choice set* set*

Using the question ‘is this a state worth living’ Using the question ‘is this a state worth living’ in the best-worst scaling method (Flynn et al, in the best-worst scaling method (Flynn et al, 2005)2005)

* Method used in this study* Method used in this study

Background to AQLQ studyBackground to AQLQ study

Asthma Quality of Life Questionnaire Asthma Quality of Life Questionnaire (AQLQ) developed by Professor Juniper is a (AQLQ) developed by Professor Juniper is a condition specific measure with 32 condition specific measure with 32 questions with 7 levels each covering 4 questions with 7 levels each covering 4 dimensionsdimensions

A simplified health state classification was A simplified health state classification was developed from the AQL-5D based on a developed from the AQL-5D based on a sample of items on 5 domains: concern, sample of items on 5 domains: concern, breathlessness, pollution and environment, breathlessness, pollution and environment, sleep and activitysleep and activity

AQL-5DAQL-5D

Feel concerned about having asthmaFeel concerned about having asthma [1]None of the time [2]A little or hardly any of the time [3]Some of the time [1]None of the time [2]A little or hardly any of the time [3]Some of the time [4]Most of the time [5] All of the time [4]Most of the time [5] All of the time

Feel short of breath as a result of asthmaFeel short of breath as a result of asthma [1]None of the time [2]A little or hardly any of the time [3]Some of the time [1]None of the time [2]A little or hardly any of the time [3]Some of the time [4]Most of the time [5] All of the time [4]Most of the time [5] All of the time

Experience asthma as a result of air pollutionExperience asthma as a result of air pollution [1]None of the time [2]A little or hardly any of the time [3]Some of the time [1]None of the time [2]A little or hardly any of the time [3]Some of the time [4]Most of the time [5] All of the time [4]Most of the time [5] All of the time Asthma interferes with getting a good night’s sleepAsthma interferes with getting a good night’s sleep [1]None of the time [2]A little or hardly any of the time [3]Some of the time [1]None of the time [2]A little or hardly any of the time [3]Some of the time [4]Most of the time [5] All of the time [4]Most of the time [5] All of the time

Overall, the activities I have done have been limitedOverall, the activities I have done have been limited [1] Not at all[1] Not at all [2] A little [3] Moderate or some [2] A little [3] Moderate or some [4] Extremely or very [4] Extremely or very [5] Totally [5] Totally

Health state Health state 3234532345

Feel concerned about having asthma Feel concerned about having asthma some some of the time [3]of the time [3]

Feel short of breath as a result of asthma Feel short of breath as a result of asthma a little or hardly a little or hardly anyany of the time [2] of the time [2]

Experience asthma symptoms as a result of air pollution Experience asthma symptoms as a result of air pollution some some of the time [3]of the time [3]

Asthma interferes with getting a good night’s sleep Asthma interferes with getting a good night’s sleep mostmost of of the time [4]the time [4]

Overall, Overall, totallytotally limited with all the activities done [5] limited with all the activities done [5]

Valuation survey: Valuation survey: sampling and interviewsampling and interview

Representative sample of adult general population Representative sample of adult general population invited to participateinvited to participate

At the interview:At the interview: Ranked health states from best to worst (7 AQLQ Ranked health states from best to worst (7 AQLQ

health states, full health (i.e. best AQLQ state), the health states, full health (i.e. best AQLQ state), the worst AQLQ state and immediate death)worst AQLQ state and immediate death)

Time trade-off (York MVH variant) of 8 AQLQ health Time trade-off (York MVH variant) of 8 AQLQ health states against shorter time in full healthstates against shorter time in full health

100 health states valued in this way100 health states valued in this way

Methods: postal follow-upMethods: postal follow-up

Approx 4 weeks after interview respondents Approx 4 weeks after interview respondents received DCE questionnaire in postreceived DCE questionnaire in post

Optimal statistical design for DCE based upon level Optimal statistical design for DCE based upon level balance, orthogonality and minimum overlap was balance, orthogonality and minimum overlap was produced by programme in SAS (Huber and produced by programme in SAS (Huber and Zwerina, 1996)Zwerina, 1996)

12 pair wise comparisons were produced and 12 pair wise comparisons were produced and randomly allocated to two versions of questionnaire randomly allocated to two versions of questionnaire with 6 choices in eachwith 6 choices in each

Two additional pairs presented to respondents Two additional pairs presented to respondents containing with AQL-5D states vs. dead. containing with AQL-5D states vs. dead.

Discrete choice Discrete choice questionquestion

Health State A Health State B Feel concerned about having asthma none of the time. Feel short of breath as a result of asthma none of the time. Experience asthma symptoms as a result of air pollution none of the time. Asthma interferes with getting a good night's sleep all of the time. Overall, a little limitation in every activity done.

Feel concerned about having asthma all of the time. Feel short of breath as a result of asthma a little of hardly any of the time. Experience asthma symptoms as a result of air pollution most of the time. Asthma interferes with getting a good night's sleep a little or hardly any of the time. Overall, moderate or some limitation in any activity done.

Which health state do you think is better? (please tick one box only)

A B

Statistical model for rank Statistical model for rank and DCE dataand DCE data

General model:General model: µµijij = = f(f(ß’xß’xij ij + + ΦDΦD++uuijij))

Where µij is the latent utility function of respondent i for state j

x is a vector of dummy explanatory variables for each level of each dimension of the classification. For example, x32 denotes dimension α=3, level λ = 2. denotes dimension α=3, level λ = 2.

   D is a dummy variable for the state of being dead D is a dummy variable for the state of being dead

which takes the value 1 for being dead or otherwise which takes the value 1 for being dead or otherwise zero. zero.

Modelling health state valuesModelling health state valuesModelling:Modelling: TTO: individual level model (random effects)TTO: individual level model (random effects) DCE: random effects probit modelDCE: random effects probit model Ranking: rank ordered logit modelRanking: rank ordered logit model

Rescaling:Rescaling:

Re-scale by dividing Re-scale by dividing ß ß coefficients on each coefficients on each dimension level by the coefficient for being dead. dimension level by the coefficient for being dead.

These rescaled coefficients provide predictions for These rescaled coefficients provide predictions for health state values on the same scale as TTO health state values on the same scale as TTO valuations although the predicted values for health valuations although the predicted values for health states may not necessarily be the same as those states may not necessarily be the same as those obtained using the TTO technique.obtained using the TTO technique.

Results of valuation surveyResults of valuation survey

Rank/TTO interview:Rank/TTO interview: 308 respondents (response rate 40% )308 respondents (response rate 40% ) Representative in terms of gender, age, education Representative in terms of gender, age, education 2455 TTO valuations across 100 health states2455 TTO valuations across 100 health states

DCEDCE 168 returned questionnaires (response rate 55%)168 returned questionnaires (response rate 55%) 1336 pair wise comparisons1336 pair wise comparisons

Results - impact of dimension level on TTO Results - impact of dimension level on TTO scores (Individual level Random Effects scores (Individual level Random Effects model with main effects)model with main effects)

Concern2 -0.047* Concern2 -0.047* Concern3 -0.064* Concern3 -0.064* Concern4 -0.074*Concern4 -0.074* Concern5 -0.095*Concern5 -0.095*

Breath2 -0.024Breath2 -0.024 Breath3 -0.045* Breath3 -0.045* Breath4 -0.107*Breath4 -0.107* Breath5 -0.116*Breath5 -0.116*

* statistically significant in 0.05 * statistically significant in 0.05 levellevel

Dependent variable: TTO values Dependent variable: TTO values MAE = 0.051MAE = 0.051

Pollution2 -0.017Pollution2 -0.017 Pollution3 -0.028Pollution3 -0.028 Pollution4 -0.063*Pollution4 -0.063* Pollution5 -0.099*Pollution5 -0.099* Sleep2 -0.013Sleep2 -0.013 Sleep3 -0.029Sleep3 -0.029 Sleep4 -0.054*Sleep4 -0.054* Sleep5 -0.069*Sleep5 -0.069*

Activity2 -0.029Activity2 -0.029 Activity3 -0.044*Activity3 -0.044* Activity4 -0.139*Activity4 -0.139* Activity5 -0.164*Activity5 -0.164*

Comparison of Comparison of ßsßs

-0.40

-0.35

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

Dimension level

De

cre

me

nts

TTO

Rank

DCE

Concern Breath Pollution Sleep Activity

Spearman rank Correlations Spearman rank Correlations (n=100)(n=100)

TTO predTTO pred Rank Rank predpred

DCE predDCE pred

Rank Rank pred.pred.

.918.918

DCE DCE predpred

.901.901 .885.885

TTO TTO ObserveObservedd

0.7900.790 .688.688 .770.770

Predicted health state Predicted health state valuationsvaluations

0

0.2

0.4

0.6

0.8

1

0.0 0.2 0.4 0.6 0.8 1.0

Observed mean TTO values

Pre

dic

ted

he

alt

h s

tate

va

lue

s

TTO predictions y=0.73x+0.2

Rank predictions y=0.66x+0.2

DCE predictions y=1.21x-0.2

Comparisons of modelsComparisons of modelsTTO RETTO RE RankRank DCE DCE

warmwarmDCE DCE coldcold

Negative Negative ßsßs 20/2020/20 20/2020/20 15/2015/20 19/2019/20

InconsistencieInconsistenciess

00 00 33 11

MAE/MADMAE/MAD 0.0510.051 0.0650.065 .09.09 0.120.12

>0.05>0.05 2222 3030 3131 4040

Mean Mean error/differenerror/differencece

0.0150.015 0.010.01 0.030.03 0.10.1

Scale rangeScale range 0.457-0.457-1.001.00

0.405-0.405-1.001.00

0.172-0.172-1.001.00

0.121-0.121-1.001.00

Overall comparisonOverall comparison

TTO model predicts observed TTO TTO model predicts observed TTO values best (lowest MAE)values best (lowest MAE)

Rank model predicts observed TTO Rank model predicts observed TTO values nearly as well as TTO modelvalues nearly as well as TTO model

DCE model is associated with DCE model is associated with largest difference from observed largest difference from observed TTO values and seems to have a TTO values and seems to have a steeper gradient (i.e. more steeper gradient (i.e. more extreme values) extreme values)

Research questionsResearch questions

1. Is DCE really easier than TTO/SG or VAS?1. Is DCE really easier than TTO/SG or VAS?

2. Does DCE produce different estimates from 2. Does DCE produce different estimates from TTO and SG?TTO and SG?

3. Theoretical basis for using DCE rather than 3. Theoretical basis for using DCE rather than conventional TTO or SGconventional TTO or SG

4. Basic DCE design issues4. Basic DCE design issues

5. Analysis – mixed logit or Bayesian models5. Analysis – mixed logit or Bayesian models

6. Does the dead dummy solve the problem?6. Does the dead dummy solve the problem?

Does including dead solve Does including dead solve the problem?the problem?

A more natural solution is to include survival as an A more natural solution is to include survival as an attribute – but this has a multiplicative relationship to attribute – but this has a multiplicative relationship to QoL and so would require a far larger designQoL and so would require a far larger design

Using ‘dead’ requires the ‘pits’ health state of the Using ‘dead’ requires the ‘pits’ health state of the classification to be considered worse than dead by some classification to be considered worse than dead by some respondents – so not suitable for milder classificationsrespondents – so not suitable for milder classifications

What about those who do not think any state is worse What about those who do not think any state is worse than dead (85% in this sample)?than dead (85% in this sample)?

For those who do not think any state is worse than dead, For those who do not think any state is worse than dead, then their data tells us nothing about their strength of then their data tells us nothing about their strength of preference for QoL compared to quantity of lifepreference for QoL compared to quantity of life

Are the 85% all none traders? SF-6D (67%), HUI3 (33%) Are the 85% all none traders? SF-6D (67%), HUI3 (33%) and EQ-5D (14%)and EQ-5D (14%)