report prepared by dr melanie calvert & mr derek kyte, pro ... · report prepared by dr melanie...

Institute Advanced Studies Workshop: BestPractice for Patient Reported Outcomes (PROs) in Randomised Clinical Trials

23rd July 2013

Report prepared by Dr Melanie Calvert & Mr Derek Kyte, PRO Research Group, University of Birmingham, UK

For further information please contact: Dr Melanie Calvert University of Birmingham, UK mail: [email protected]

1

ttp://www.birmingham.ac.uk/patient‐reported‐outcomes h

2

Executive Summary Patient reported outcomes (PROs) are increasingly assessed in clinical trials, but PRO trial design and data collection are often suboptimal and the results poorly reported. The University of Birmingham PRO Research Group led an Institute of Advanced Studies Workshop on 2nd‐3rd July 2013. The workshop aimed to disseminate the latest research from the group and its international collaborators, to promote and consider best practices for PRO trial design, dentify research priorities and develop collaborations to obtain funding to isupport these activities. The l i fo lowing research priorit es were identified during the workshop:

1. Determining optimal methods of patient involvement in outcome selection.

2. Establishing the optimal methods for incorporating PROs in Core Outcome Sets.

3. Development of a checklist for protocol writers, detailing recommended n PRO‐specific items for inclusion i trial protocols, with supporting online

tools. ent of ‘PRO 4. The development of consensus guidance on the managem

Alerts’ within trials.

5. Development of PRO data collection/management guidelines. tating clinician and 6. Development of methods and tools aimed at facili

omains. patient understanding of PROs.

7. Research into standardisation of PRO tools/d8. Development of a UK‐based PRO item bank.

The UoB PRO Research Group in collaboration with partner organisations are committed to addressing these priorities and to providing training and education o facilitate best practice for PROs in clinical trials. Details of the PRO Research roup and our international collaborators can be found on our website: tG http://www.birmingham.ac.uk/patient-reported-outcomes

3

Executive Summary .......................................................................................................2 Background ....................................................................................................................4 Acknowledgements........................................................................................................4 Workshop Aims: ............................................................................................................5 W ...........................................................................................................5 orkshop formatSe ..............................................................................6 ssion One: PRO SelectionKey questions:

..............................................................................6

.........................................oup discussions:

..............................................................................6 Summary of Gr

..............................................................................7 Se ..............................................................................7 ssion Two: PRO Trial DesignKey Questions:

..............................................................................7

..................................oup discussions:Summary of Gr

..............................................................................9 Se ..............................................................................9 ssion Three: PRO Trial ConductKey questions:

..............................................................................9

...............................oup discussions:Summary of Gr

............................................................................10 Se ts to inform clinical practicession Four: Use of PRO trial resul

.............................oup discussions:

..............................10 Key questions:

............................................................................10 Summary of Gr

............................................................................11 Se clinical perspectivession Five: Research priorities – a

.........................................................................................................11

...........................................11 Key questions:Summary of Group discussions:

Research priorities .......................................................................................................12 Next Steps ....................................................................................................................12 Appendix 1: Attendees:................................................................................................13 Appendix 2: Workshop Programme ............................................................................15 Ap tationspendix 3: Copy of Presen

.................................................................................65

.............................................................................17 Ap group discussionspendix 4: Notes made during

............................66

.....................................................65 Session 1: PRO Selection .......

............................68 Session 2: PRO Trial Design.......................................................

..............ractice

Session 3: PRO Trial Conduct ......................................Session 4: Use of PRO trial results to inform clinical p ............................69 Session 5: Research priorities: a clinical perspective .........................................71

Appendix 5: Voting Slides...........................................................................................72

4

Background

Patient reported outcomes (PROs) are increasingly assessed in clinical trials and provide 'the patient voice' in evidence on treatment effectiveness. PROs commonly evaluate health‐related quality of life, symptoms such as pain or fatigue, health utility, adherence and patient satisfaction and may be used to influence clinical care and decision‐making, predict long‐term outcomes and inform health policy. Despite this, processes relating to PRO trial design and data collection are commonly suboptimal and the results are often poorly reported. These practices devalue important patient‐centred data, diluting the impact of PRO results in the clinical setting. Researchers at the University of Birmingham are leading a programme of work, which aims to enhance the design, implementation, analysis and reporting of PROs in trials, to improve the way that results are used to inform clinical care. This two‐day workshop aimed to bring together leading experts in PRO trials methodology with key personnel within the Birmingham Clinical Trials Units, UHB clinical leads, researchers involved in PRO data collection and patient representatives, who will collectively work on a strategy to improve standards of PRO trial measurement and reporting:

Locally, through collaboration, academic detailing and other knowledge transfer initiatives.

Internationally, through high‐impact publications and via collaboration with the International Society of Quality of Life Research (ISOQOL) on an international knowledge transfer process.

Acknowledgements

The Workshop was funded by the Institute for Advanced Studies (IAS), University of Birmingham and was organised by Dr Melanie Calvert and Mr Derek Kyte (PRO Research Group), with Sue Gilligan and Sarah Myring (IAS). Dr Melanie Calvert is member of the MRC Midland Hub for Trials Methodology Research, University of Birmingham, UK (Medical Research Council Grant ID G0800808). Derek Kyte is supported by a National Institute for Health Research School for Primary Care Research PhD studentship. The views expressed in the report reflect individuals opinions voiced at the workshop and may not reflect funder views. We thank the workshop participants for their valuable contributions and our speakers for stimulating lively debate (a full list of attendees and the meeting programme with the list of speakers is provided in Appendix 1 and 2 respectively). We thank Helen Duffy and Karen Biddle for assistance in preparing this report.

5

Workshop Aims:

To disseminate key findings of recent advances in PRO methodology for trials with members of clinical trials units and other key stakeholders across the UK, including clinicians and patient representatives.

Generate and prioritise key research/logistical questions surrounding the future identification, dissemination and implementation of best‐practice for PROs in randomised clinical trials and develop a resulting publication/funding/knowledge transfer strategy.

Form working groups tasked with addressing aspects of this strategy, tive. both at a local level and from an international perspec

To consider developing a theme proposal for the IAS.

ts: Benefi

Patient engagement, local & international knowledge translation, publications, collaborative grants.

t: Impac

Output from the workshop will help to improve the quality of PRO information collected from trials, thus ensuring that the clinical care of patients is informed by robust results.

Workshop format The list of attending delegates is provided in Appendix 1 and the workshop programme is provided in Appendix 2. Each session began with presentations from key stakeholders introducing current challenges and opportunities of using PROs in clinical trials. Following the presentations (Appendix 3), delegates formed break‐out groups to discuss research needs and propose research priorities (notes of these discussions are provided in Appendix 4). During day 2 of the workshop, delegates were asked to vote on research priorities (Appendix 5).

6

Session One: PRO Selection

Key questions: g e y iHow do we select PRO domains (e. . fatigu /anxiet ) and quest onnaires

that are valid, acceptable to patients and will inform clinical care? What is the difference between a patient centred and patient reported

outcome? How do we ensure that patient centred outcomes are included in clinical

trials? What other issues should we consider? How can we address these issues?

Summary of Group discussions: Methods to select PROs/ domains. Group discussions focused on identification of appropriate PRO domains associated with the research hypothesis. Delegates highlighted the need for standard methodology to aid selection of PROs with patient input. The need for ongoing validation of existing PROs in different populations was recognised. Inclusion of PROs in Core Outcome sets (COS) The groups recognised a need to develop optimal methods for incorporation of ROs into COS. The group questioned whether PROs in a COS could be used for ultiple purposes ‐ in both trials and routine practice (as an audit tool).

Pm Optimal methods of patient involvement in outcome selection & trial design. Patient involvement in trial design and selection of outcomes should be routine. Researchers need to work closely with patients to ensure that they fully understand their role. Group discussions focused on methods to recruit representative patients including those from ‘hard to reach groups’. Increased ollaboration between patients and researchers at all stages of the study may elp maximise patient benefit. ch Carer Outcomes he group felt that increased assessment of carer outcomes may be warranted in ome trials. Currently the impact of caring may not be assessed. Ts Feedback to patients on trial progress. The group recognized an increased need to provide feedback to trial participants. There is a need to develop and refine optimal methods for providing useful feedback, which could include newsletters, emails, posters, websites, phone apps. Training tools for PRO selection. The group identified a clear need for training on the optimal methods of PRO Selection. Development of methods and training for researchers (and patients) was seen as a priority area for future research.

7

Session Two: PRO Trial Design

Key Questions: What PRO‐related components should be included in the trial protocol or

other trial documentation such as SOPs? rial What are the key areas that need further work to inform optimal PRO t

design? ow do we ensure that trials adopt best‐practice in PRO trial protocol

onfiguration? Ideas to address this? Tools? Education? Hc

Summary of Group discussions:

Considerations for a PRO Protocol Checklist Groups considered a number of items which they felt should be considered for inclusion in a protocol checklist including: a clear rationale for PRO assessment with patient input, identification of the PRO as a primary or secondary endpoint, specification of valid questionnaires (with supporting references), domains linked to the study hypotheses, timing of assessments (measured at appropriate clinical time points – appropriate to the patient journey), a pre‐specified analysis plan (i.e. specific domains, global score) and plans for dealing with missing data (e.g. imputation), and assessing clinical significance. The groups discussed at length, practical issues regarding administration of PROs including ensuring patient privacy and dealing with language barriers. Delegates felt that: (1) further practical guidance on PRO administration should be provided in trial documentation, (2) SOPs should include guidance on how to deal with concerning PRO scores – for example, advocating that the research team contact the patient’s GP in response to such scores, and (3) trial documentation should also specify how feedback should be provided to patients on the progress of the trial. Delegates recognised the need to first decide which domains to measure, based on the study hypotheses, before selecting the most appropriate tool/measure. Consideration should be given to timing of assessment (i.e. before or after clinical consultation/treatment) – this should be specified. Groups generally felt that administration in advance of a consultation may have methodological benefits (patient views not influenced by clinical encounter) and those questionnaires may be more complete. The protocol should specify whether the patients are allowed to receive help (carer/nurse/doctor), e.g. for patients with visual or co‐ordination problems. The groups considered sources of potential bias when patients may ‘want to please/think there’s a right answer’. This ‘noise’ is likely to be balanced across trial arms, but should be considered at the design stage. Justifications for PRO methods, with appropriate references, should be provided in the protocol. Many practical issues associated with PRO trials, such as selecting questionnaires and time‐points for assessment could be considered during pilot work with patient input. Missing data can be a major problem and is a potential source of bias. The groups considered ways to minimise missing data. They felt that trials should pre‐specify in the protocol: approved methods to avoid or minimise missing

8

data, how trial personnel should report reasons for any missing data ncountered and how they should deal with missing data. e PRO Protocol Checklist Tools It was recognized that the development of a checklist for protocol writers, detailing recommended PRO‐specific items for inclusion trial protocols (with supporting online tools) would improve PRO trial design and implementation, and could incorporate both ‘core’ essential items and desirable items. The groups felt that this checklist (similar to CONSORT), if required for publication of protocols, would be helpful. Endorsement of use by major funding bodies would help facilitate uptake. The need for education/training for researchers and peer reviewers was recognized. A ‘stick approach’ was felt to be necessary – e.g. requirements by funders.

9

Session Three: PRO Trial Conduct Key questions:

Is experience of PRO alerts widespread? How best should they be ce we can draw upon?

be included in protocol managed? Is there existing guidan

What information for data collection staff should

guidance/trial training and why? How do we encourage uptake of new guidance?

Do patients understand why we assess PROs in trials? What PRO‐specific tion information should be included in the patient informa

documentation? Are/should PROs be discussed in consent interview?

Summary of Group discussions: PRO Alert Consensus & Guidance Experience of PRO Alerts varied amongst delegates. The groups were in general agreement that predetermined thresholds should be identified and a plan for dealing with PRO Alerts documented in the protocol or SOPs. There is no consensus on optimal methods for dealing with alerts and this was recognized as a research priority. To be effective, alert management needs to occur in a timely way (electronic data capture was discussed). The group discussed data capture of arising actions. Delegates discussed potential methodological challenges of the clinical team viewing PRO data (e.g. risk of unblinding). Data managers need guidance on how to manage unrequested additional data provided on uestionnaires or as letters. It was felt that PRO Alert management should be isk‐related. qr PRO Data Collection/Management Guidelines and Training A range of training needs were identified by delegates including: the value of PRO data, communicating with patients regarding PROs, how to minimize and report missing data, guidance on how to deal with PRO Alerts/concerning data. elegates felt that this could be incorporated into GCP/site level training. Online raining may be useful. Dt PROs in the Patient Information Sheet Delegates felt that in general patient information sheets are too long, it was felt to be a good idea to use ‘light’ versions of PIS. Concerns were expressed around the expectation gap between the benefit that a patient perceives that they will get from being involved in a trial, the requirements on the researcher to ensure that patient info sheets state clearly that there is not expected to be a direct benefit and reality (which may lie somewhere in between). Groups felt that further work is needed to consistently improve the presentation of PISs (length, accessibility, other methods DVD, multimedia, apps). Groups queried whether patients understand why we assess PROs in trials? It is unclear what PRO‐specific information should be included in the patient information. Do patients want/need this information (either in the information sheet or in the consent interview)? Further research is needed in this area and more generally on optimal methods of appropriate patient information provision.

10

Session Four: Use of PRO trial results to inform clinical practice

Key questions: What are the key challenges for reporting PROs?

How can we facilitate improved reporting? Tools, education, other KT activities?

What are the barriers to the use of PRO trial results to inform clinical care and how can we address these?

Summary of Group discussions: Research into standardisation of PROMs/reporting. Group discussions focused on the need for endorsement and uptake of the CONSORT PRO Extension. Ensuring that journals and funders adopted the extension, and that online tools were developed to support trialists, was felt to be important. Delegates agreed that PROs should be reported in the main publication. Discussions highlighted the need for high quality trial design to facilitate the reporting of meaningful results. Clinicians were keen to see standardised use of measures to facilitate reporting and interpretation. Some concerns were expressed that PRO data takes too long to analyse and thus is not ncorporated into the main publication. This was not felt to be a legitimate oncern since programming can be done using dummy data in advance. ic Methods to facilitate clinician/patient understanding of PROs. Clinicians may need training on how to interpret and communicate PROs to patients. Staff not involved in research may find the data particularly difficult to interpret. Local training could be facilitated by the Birmingham Region Research and Training Collaborative and could provide training to: (a) researchers on how to bring PRO correctly into the trial design/protocol, and (b) to site staff, to ensure appropriate procedures are followed when using PROs (e.g. timing/location of questionnaires, using translators, using persons to read out questionnaires). It was generally recognised by the group that dissemination of PRO findings to patients is not done well – if at all. Web‐based tools/Podcasts or patients may be useful. Again, these discussions highlighted the need for ncreased patient input from study design to dissemination. fi Methods to facilitate shared decision making in relation to PROs. The group felt that greater understanding of PRO data could help facilitate shared decision making. It was recognized that the presentation of PRO results can be particularly challenging. Tools/software to support clinicians should be considered. Patient lay summaries, to accompany trial publications, may be eneficial. Patients should be consulted and could help identify the optimal ontent and layout. bc

11

S

ession Five: Research priorities – a clinical perspective

Key questions: Selection of instruments (e.g. for rare diseases/ systemic vs localized

effects/ generic‐e.g. EQ‐5D vs. disease specific)

The ability of a clinical community to have a preferred PRO tool in trials ina certain area and having a central repository for use across several trials.

a significant and s this.

Use of PROs for those who do not read or speak English ‐ important minority of our population ‐ and how to addres

Questionnaire fatigue (links to selection of instruments)

Summary of Group discussions: Selection of instrument Instrument selection should be hypothesis driven and valid for the population of interest. Group discussion focused on limiting the number of questionnaires used, to ensure consistent, interpretable results. Use of both disease specific and generic questionnaires was considered, although some concerns were expressed regarding patient burden. The balance between patient burden and useful PRO data should be considered. An item‐bank approach may be helpful and may llow comparison across different clinical settings. Data repositories for PRO ata warrant further consideration. ad Assessing PROs in Diverse Populations Group discussions focused on the need to have valid questionnaires appropriate for different cultures, which should be considered at the design stage and costed appropriately. Perceptions of symptoms, such as pain or depression, may differ across cultures. Any translation needs to be validated and interviewers need raining. The use of audio system/speech‐enabled systems for illiterate or blind articipants should be considered. tp Integration of PROs in the Institute for Translational medicine PROs were identified as a strategic theme which will be included in the ITM trategy.

s

Ensuring Best Practices for PRO Assessment in Trials The group welcomed the workshop as an opportunity to learn about the latest research from the PRO Research Group, other UoB academics and international collaborators. The workshop has provided a valuable platform to share best practice, build on existing collaborations and develop new collaboration, for example, PROs will be included as a theme within the Institute for Translational Medicine at Birmingham. The PRO Research Group will work to foster partnerships with key stakeholders including the HRA, research funders such as the NIHR and CTUs.

12

Research priorities he following research priorities were identified by anonymous voting on day 2 f t w p : To

he orkshop (Ap endix 5)

1. Determining optimal methods of patient involvement in outcome selection.

2. Establishing the optimal methods for incorporating PROs in Core Outcome Sets.

3. Development of a checklist for protocol writers, detailing recommended n PRO‐specific items for inclusion i trial protocols, with supporting online

tools. ent of ‘PRO 4. The development of consensus guidance on the managem

Alerts’ within trials.

5. Development of PRO data collection/management guidelines. tating clinician and 6. Development of methods and tools aimed at facili

omains. patient understanding of PROs.

7. Research into standardisation of PRO tools/d8. Development of a UK‐based PRO item bank.

Next Steps The UoB PRO Research Group in collaboration with partner organisations are committed to addressing these priorities and to providing training and education to facilitate best practice for PROs in clinical trials. The latest details of the PRO esearch Group activities, outputs, grants, publications, our international ollaborators and project partners can be found on our website: Rc http://www.birmingham.ac.uk/patient-reported-outcomes

Appendix 1: Attendees:

Attendee List: Institute of Advanced Studies Workshop - 2nd/3rd July 2012

Best-Practice for Patient Reported Outcomes (PROs) in Randomised Clinical Trials

Title First Name

Surname

Role & Organisation Email

Dr Hareth Al-Janabi Birmingham Fellow, School of Health & Population Sciences

[email protected]

Dr Cassie Aldridge Clinical Research Officer – Protocol Development Service, UHB

[email protected]

Prof. Jane Blazeby Professor of Surgery, University of Bristol Director of the MRC ConDucT Hub for Trials Methodology Research

[email protected]

Prof. Mike Brundage Professor of Oncology & Director, Division of Cancer Care and Epidemiology, Cancer Research Institute, Queen's University, Ontario

[email protected]

Dr Jen Burr Senior Clinical Research Fellow, University of St Andrews

[email protected]. ku

Dr Melanie Calvert

Reader in Epidemiology, Outcomes Research Lead MRC Midland Hub Trials Methodology Research

[email protected]

Dr Clara Day Consultant Nephrologist, UHB [email protected]

Dr Paul Cockwell

Consultant Nephrologist, UHB [email protected]

Dr Alastair Denniston Hon Senior Lecturer in Ophthalmology, Consultant Ophthalmologist

[email protected]

Prof. Heather Draper

Professor of Biomedical Ethics

[email protected]

Ms Helen Duffy Project Officer, PRO research group

[email protected]

Prof. Khaled Ismail Professor of Obstetrics & Gynaecology, School of Clinical & Experimental Medicine,

Consultant Obstetrician & Gynaecologist, Birmingham Womens’ Hospital

[email protected]

13

http://www.birmingham.ac.uk/research/activity/ias/workshops/2013/best-practice-for-PROs-in-randomised-clinical-trials.aspx

mailto:[email protected]






14

Title First Name

Surname

Role & Organisation Email

Dr Natalie Ives Assistant Director of BCTU, Senior Statistician, School of Cancer Sciences

[email protected]

Prof. Paulus Kirchhof Chair in Cardiovascular Medicine

[email protected]

Mr Derek Kyte

Doctoral Researcher - NIHR School for Primary Care Research

[email protected]

Ms Frances Lloyd Trust Lead Research Nurse, SWBH NHS Trust

[email protected]

Ms Rhiannon Macefield Research Associate, University of Bristol

[email protected]

Dr Hardev Pall

Senior Lecturer in Neurology, Consultant Neurologist UHB

[email protected]

Dr Anna Philips Reader in Behavioural Medicine

[email protected]

Dr Saaeha Rauz Clinical Senior Lecturer/Consultant Ophthalmologist, School of Immunity & Infection

[email protected]

Sr Sarah Reavenall Burns Research Nurse [email protected]

Dr Andrea Roalfe Senior Lecturer in Medical Statistics, Primary Care Clinical Sciences

[email protected]

Mrs Lesley Roberts Patient representative [email protected]

Dr Alison Rushton Senior Lecturer Physiotherapy

[email protected]

Ms Sue

Southworth Research Nurse, Birmingham & Midland Eye Centre

[email protected]

Dr Olga Tucker Consultant surgeon [email protected]

Dr Wilma van Riel Clinical Trials Quality Assurance Manager, College of Medical & Dental Sciences

[email protected]

Ms Maria Von Hildebrand

Patient and Public Stakeholder Engagement Manager

[email protected]

Sr Heather Willis Trauma Research Nurse [email protected]












Appendix 2: Workshop Programme

15

Day 1 - Tuesday 2nd July 2013

Start Item Lead discussant

09:00 Registration

09:20 Meeting commencement Chair: Melanie Calvert Welcome from Pro-vice-chancellor Professor Malcolm Press, director of the IAS

09:30 Aims of the workshop and introductions Melanie Calvert

10:00 Session 1: PRO Selection • What patients/carers want from PROs [LR;MvH] • Evidence-based PRO selection (Core Outcome Sets) [JB] • Where next? Key points for discussion [JB] • Small group discussion [JB; MC] • Feedback/whole group discussion

Jane Blazeby Lesley Roberts Maria von Hildebrand

11:45 COFFEE BREAK

12:00 Session 2: PRO Trial Design • A trialists’ perspective on the quality of PRO protocol components [MK] • PRO Trial Protocol Design [MC/MB]

Melanie Calvert Madeleine King Michael Brundage Jane Blazeby

12:30-13:30 LUNCH

13:30 Session 2: PRO Trial Design cont… • Where next? Key points for discussion [MB] • Small group discussion [MC; MB; JB] • Feedback/whole group discussion

Melanie Calvert Michael Brundage Jane Blazeby

14:45 COFFEE BREAK

15:00 Session 3: PRO Trial Conduct • Challenges in assessing PROs in trials [DK] • Ethical Considerations [HD] • Where next? Key points for discussion [MC] • Small group discussion [DK; HD; MC] • Feedback/whole group discussion

Derek Kyte & Heather Draper Melanie Calvert

16:45 Round-up of the day Melanie Calvert

17:00 Drinks reception

18:00 MEETING CLOSE

Day 2 - Wednesday 3rd July 2013

Start Item Lead discussant

08:30 Meeting commencement Chair: Melanie Calvert

08:30 Priorities from Day 1/ Aims for day 2 Melanie Calvert

09:30 Session 4: Use of PRO trial results to inform clinical practice • CONSORT PRO extension [MB] • Using PRO trial results in practice [MB] • Where next? Key points for discussion [JB]

Michael Brundage Jane Blazeby Melanie Calvert

10:30 COFFEE BREAK

10.50 Session 4: Use of PRO trial results to inform clinical practice, cont… • Small group discussion [MB; JB; MC] • Feedback/whole group discussion

Michael Brundage Jane Blazeby Melanie Calvert

12:00-13:00 LUNCH

13:00 Session 5: Research priorities: a clinical perspective The Institute of Translational Medicine [PC] Priority areas for PRO research- clinicians perspective [PC; PK; CD; AD; HP] •Where next? Key points for discussion [PC] • Small group discussion [PC; PK; AD] • Feedback/whole group discussion

Paul Cockwell Paulus Kirchhof Clara Day Alastair Denniston Hardev Pall

14:30 COFFEE BREAK

14:50 Session 6: Moving PRO research forwards • Priorities from day 2 • Summary of priorities arising from the workshop • Where next? whole group discussion

Melanie Calvert Michael Brundage Jane Blazeby

15:50 Closing remarks Melanie Calvert

16:00 MEETING CLOSE

16

17

Appendix 3: Copy of Presentations

Session 1: PRO Selection

PRO Research Group

What do Patients and Carers want from PROs?

Maria von Hildebrand and

Lesley Roberts

University Of Birmingham Workshop

July 2nd‐ 3rd 2013

MvH email: [email protected], LR Email:[email protected]

Introduction

• Maria has worked in patient and public involvement as parent, carer, campaigner senior manager in DH, and now working freelance in health advocacy.

• PPI Work has covered acute, chronic, primary, secondary, tertiary, health, social care, and research.

• Most recent publication: Co‐authored report with Simon Denegri on Patient and Public Survey Response to UK Clinical Trials Gateway http://www.nihr.ac.uk/files/Publications/UKCTG%20Report_Jan%202013.pdf

[email protected]


From research to clinical practice

•What is the purpose of research?

•In 2009 Iain Chalmers and Paul Glaziou estimated that 85% of research is wasteful or inefficient with deficiencies in four main areas:

Is the research question relevant?

Are the design methods appropriate?

Is the full report accessible?

Is it unbiased and clinically meaningful?Source: The Lancet Volume 381, Issue 9864, Page 347 2nd Feb 2013

The public and patients assume all of this to be true, because they trust in the experts.

Research is being politically exploited as an industry for economic growth due to availability of patient data in the NHS, making the above areas even more important to avoid exploiting goodwill of patients and public especially those who are vulnerable.


Relevance of PRO to patient?

• At which point in the patient journey, (not clinical pathway)and stage of illness, is a patient presented with opportunity to take part in research that is relevant to them?

• ‘Only 28% of patients surveyed had accessed a clinical trial’ Source UKCTG Patient and Public Survey Report S Denegri, M von Hildebrand Jan 2013.

• Having been granted participation in a trial at what point do patients become aware of PRO questionnaire, its purpose, number of times it needs to be filled out, and its relevance to them as an individual?

• What difference do they think it will make to their quality of life and does this match with the communication they receive when consenting to take part?

• If parents/carers are to be included in study, will they be made aware of the importance of their participation in the study?


What do patients/carers want

• Clarity of purpose about RCT and relevance to individual taking part.

• Knowledge that the PRO will be a useful and reliable tool to individual and future patients.

• Understanding that PRO will measure their quality of life, at the beginning, during and end of trial.

• Ability to fill in PRO and complete it at every required stage. and importance of this requirement.

• Ease of functionality, length, language, jargon, no duplication!

• Pledge that having taken part in the RCT they will be able to read published findings detailing the outcome of research in regards to clinical practice and patient benefit.

MvH email: [email protected], LR email: [email protected]

18

Over to Lesley

• Lesley is currently a member of –

• The National Board of Involve. On the Advisory Group producing “Plain English” advice for DH.

• NCRI PPI committee “The Hub”.

• The Haematological Oncology Clinical Study Group and the Myeloma sub‐group.

• The NCRI Consumer Liaison Group.

• The Clinical Translational Radiography and Radiotherapy Programme.

• On line peer reviewer for RfPB and PGfAR.

• The West Midlands North CLRN.

[email protected]


19

Jane M Blazeby Professor of Surgery, Honorary Consultant Upper Gastrointestinal Surgeon, Director MRC Trials Methodology Hub, Director Surgical Trials Centre, University of Bristol & University Hospitals Bristol NHS Foundation Trust

Evidence based PRO selection for core outcome sets

Overview

• Problems with patient reported outcomes and core outcome sets as a solution

• How to select PROs in core outcome sets• Future work needed

Patient reported outcomes

• Many PROMs, with scales & items assessing indistinct domains

• Impossible to synthesise, compare like with like

• Enormous risk for outcome reporting bias

HRQL Questionnaire Number of studies (n=26)

EORTC QLQ-C30/CR38 17

SF-36 4

FACT-C 1

EQ-5D 1

HADS 1

STAI 1

mCOP-QoL-Ostomy 1

FIQL 1

Spitzer’s QLI 1

CRC-QoL 1

Different PRO measures Outcome reporting bias

20

Quality of life > 20 scales/items

• EORTC QLQ-C30 and QLQ-OES18• Dermatology QoL Index

Solutions

• Core PRO sets, • Could also be used as core disclosure

sets for clinicians to discuss with patients

Advantages of core outcome sets

• Increases consistency across trials

• Maximise potential for trial to contribute to systematic reviews of these key outcomes

• Much more likely to measure appropriate outcomes

• Major reduction in selective reporting 11

Core Outcome Measures in Effectiveness Trials

http://www.comet-initiative.org/home/

@COMETinitiative

21

“Core outcome set: agreed set of outcomes

reported as a minimum in all clinical trials of a

specific clinical condition.”

What does this mean for PROs?

Core outcome set Identify core domains and then…

Creating order out of chaos

• Identify and deconstruct all PROMs (valid and not valid)

• Re-categorise into domains (experts)• Use Delphi methods to prioritise domains

Overview

• Problems with patient reported outcomes and core outcome sets as a solution

• How to select PROs in core outcome sets• Future work needed

Once selected what to measure, think about how

• ‘What’ to measure

• ‘How’ to measure (validity, reliability, feasibility)

- PROMIS, http://proqolid.org/, COSMIN, TREAT-NMD

www.comet-initiative.org

Twitter: @COMETinitiative

22

Session 2: Objectives, hypotheses, questionnaires and endpoints

QOL Protocol Checklist Workshop

Session 2: PRO Trial Design

PRO Research Group

Session 2 – PRO Trial DesignA trialists’ perspective on the

quality of PRO protocol componentsProf Madeleine King

& Ms Rebecca Merciecer‐Bebber

2 day Workshop: Best‐Practice for PROs in RCTs2‐3 July 2013 – Birmingham, UK

QOL Office, University of Sydney•Established in 2005•Funded by Cancer Australia•Provide QOL/PRO advice and services to the network of 13 Australasian Cancer Clinical Trials Group (CTGs)

Overview

• What we have learned

– things that should be included in protocols

– PROtocol Checklist

• Results of our pilot study

– Things that are not currently included in protocols

Importance of the protocol

• Procedures for good conduct of the trial– success or failure of a trial may depend on how well the protocol was designed and written

– all relevant requirements so the study can be implemented uniformly by all sites and staff

– detailed and clearly worded

• These general points apply just as much to PRO assessment as to any other aspect of a trial

SPIRIT 2013 Chan et al.

Rationale for PROs• Why measure PROs in the trial?

– What’s known / not known, why it matters

• Which PROs variables are considered relevant and why?– Acute v late effects

– Benefits v harms

– Primary v secondary PRO endpoints

– Important to anticipate those issues/domains that are most likely to be affected by the intervention, differentially between trial arms

• Proximal to disease/treatment – symptoms/side‐effects

• Distal – may include core HRQOL domains (physical, emotional, role, social functioning) and/or global assessment of HRQOL

23



Choosing the most appropriate PROM(s)

• Decide on which PROs matter first, then choose the best questionnaires to measure them– ‘don’t put the cart before the horse’– ‘the right tool for the job’– FDA: ‘fit for purpose’

• Protocol should include:– Justification of choice ‐ which PROM(s) and why– Citations for validity, reliability, responsiveness,

interpretability– Which specific symptoms, side‐effects, HR‐QoL domains

covered– Number of items (questions)– Approximate time to complete (rule of thumb: 10 sec/item)

• copies of questionnaires in appendix

When, where and how PRO will be assessed

• timing of PRO assessments– acute and late effects– acceptable time windows

• mode of PRO data collection– in person; in clinic or at home; by telephone, mail or

online

• Also include this info in informed consent procedure

Statistical considerations• Scoring

– how questionnaire items are scored into summary scales

• direction of scale, e.g. higher score = more symptoms

• Minimal important difference (MID)

– Clinical importance vs statistical significance

• Responder definition (optional)

– Palliative context

– Improvement of >=MCID for specified period

Statistical considerations (cont.)

• PRO assessment schedule will result in numerous PRO variables– Early/late symptoms/side‐effects/function

• Which are the primary/secondary PRO endpoints/timepoints– For primary endpoints, corresponding hypotheses– Size and direction of expected effects

• Sample size/power– if a PRO is the primary endpoint, then required sample size

– if PROs are secondary, then the power of the trial sample size in relation to the PRO hypothesis/es

Missing data

• Prevention is better than cure– Emphasise the importance of compliance, methods for enhancing it

– Also cover in site manuals for trial co‐ordinators, research nurses, etc

• Some missing data cannot be avoided– How missing data are to be dealt with in analysis– How relevant ancillary data will be collected (e.g. ECOG‐PS, Karnovsky, Spitzer clinician rated QOL) and incorporated into analyses (e.g. used in mulitple imputation)

• Key issues addressed in protocol, detailed statistical analysis plan developed later

QOL Office

PROtocol Checklist

available @ QOL Office Website

http://www.pocog.org.au

+ PROtocol Development Workshops

24



Is such a checklist needed? Pilot Study

Aim: Assess “PRO‐completeness” of Aust CTG protocols for Phase III RCT

Methods:

•13 protocols (convenience sample)

•37 checklist items, one point per item included

Results:

•Mean (SD) “completeness” score = 11.8 (4.2)

•Min = 3, max = 28 (75%)

Is such a checklist needed? Yes!Selected results from the 13 protocols assessed

Only 3 Described methods for handling missing data.

Only 2 Described the constructs used to evaluate the intervention (most just referred vaguely to “HRQOL” or “QOL”Specified acceptable time windows for each assessment.

State the Inclusion and exclusion criteria for PRO endpoint(s) and analyses.

Only 1 Stated a PRO-specific hypothesis.

Specified the timeframe of interest and rationale for this.

Included a standardised form for recording reasons for missed PRO assessments..Stated the sample size and power requirements.

None Named the PRO sub-study investigator/coordinator.

Specified how PRO was to be assessed – pencil & paper, online, etc.

Stated and justified minimal important difference (interpretation).

Next steps• Development of a definitive PRO protocol checklist

– ISOQOL Task Force: Best practices for PROs in Randomized Clinical Trials

• Oversight Executive: Brundage, Calvert, Blazeby, Revicki

• Aust CTGs– Grant under review

– Rebecca embarking on a PhD

– Extend our pilot to a large sample of existing protocols, link to quality of PRO reporting (CONSORT‐PRO guidelines)

– Evaluate the effectiveness of the PROtocol Checklist + training workshops for increasing the completeness of PRO components of Aust CTG protocols (long‐term, prospective project)

Thank you from Sydney


PRO Research Group

Research update: Evaluation of patient reported outcomes in clinical trials: systematic review of trial protocols

University of Birmingham UK

Investigators: Calvert M (PI), Kyte D, Draper H, Ives J, Gheorghe A, Duffy H, Brundage M, King M, Mercieca-Bebber R, Blazeby J.

25



What’s the problem?Hypothesis: Information on PRO hypotheses, data collection and analysis in trial protocols is suboptimal.

Suboptimal trial design:• Unethical

• Inefficient use of funding

• Lack of guidance for trial staff‒ Potential ethical tension‒ Non-standardised (inappropriate?) practices

(Session 3)

• Poor reporting? (Session 4)

• Poor quality evidence to inform patient care.

Evaluation of patient reported outcomes in clinical trials

Eur J Cancer. 1997 Jan;33(1):20-8.Quality of life assessment in clinical trials--guidelines and a checklist for protocol writers: the U.K. Medical Research Council experience. MRC Cancer Trials Office.Fayers PM, Hopwood P, Harvey A, Girling DJ, Machin D, Stephens R.

Madeleine King, PoCoG

Review current guidance on PRO assessment in trials.

Establish the current quality of trial protocols in relation to the PRO information they provide.

NETSCC HTA Protocol 02-10-02The primary outcome measure will be complete ulcer healing at 12 weeks. Other secondary outcome measures that will be included in the analysis are healing at six months and one year, recurrence at six months and one year, EQ-5D and SF-36 health related quality of life questionnaires, and the McGill pain questionnaire.

NETSCC HTA Protocol 01-74-03Questionnaires on pain, satisfaction and HRQOLIn the absence of a widely used disease specific measure, we will analyse the responses to four separate measures, all of which are completed in private and returned by post: (Rand) SF36, EuroQol EQ5D, CWIS and a 10cm visual analogue scale for pain. [A satisfaction questionnaire will be used at the end of the study]. These are all assessment tools that have been used without problem in this patient population in previous studies.

www.hta.ac.uk/project/

Next Steps:

International Consensus on Guidance for PRO protocol writers

Taskforce led by Calvert, King, Brundage, Blazeby, Kyte

PRO Research Group

26




Topics for group discussion:

1.What PRO-related components should be included in the trial protocol or other trial documentation such as SOPs?

2.What are the key areas that need further work to inform optimal PRO trial design?

3.How do we ensure that trials adopt best-practice in PRO trial protocol configuration? Ideas to address this? Tools? Education?

27

Session 3: PRO Trial Conduct !!

PRO Research Group

Challenges in Administering PROs in trialsDerek Kyte - PRO Research Group, University of Birmingham UK

Doctoral Research supervised by: Calvert M, Draper H, Ives J Collaborators: C Liles, Gheorghe A, Keeley T

School for Primary Care Research

Funded by the National Institute for Health Research - School for Primary Care Research

Acknowledgements

BACKGROUND• Key researcher concerns:

Inconsistencies

Uncertainty

Lack of Guidance

Additional Info

‘Concerning’ Data

METHODS

- One primary care NHS trust

- Two secondary care NHS trusts

- Two clinical trials units

• 26 semi-structured interviews

RESULTS

“If they go into our control group… they’re not gonna have any personal contact, then [the questionnaire] will go back out in the post with just a

letter, sort of, saying, ‘Oh, you accidentally missed out this one,’ and... hopefully they’ll return it back… if they’re put into an exercise, erm,

intervention, then they do get seen by [a research facilitator]. I’ll ask her to take the questionnaire with her… she’ll help them to fill it in.” [participant 25]

Inconsistency: inconsistent standards in administration of QOL measurement, both between, and within, trials, which appeared to risk the introduction of bias.

Theme 1

RESULTS

PRO Alerts

Theme 2Dealing with ‘concerning’ data: staff reported uncertainty when encountering participant self-reports of problems that potentially required intervention.

“worrying levels of psychological distress or physical symptoms that may require an immediate response.”

28

RESULTS

“A couple of times where measurable data… that the doctor had done, showed yeah, maybe a bit of a slight wobble but everything is okay. Whereas... according to their quality of life questionnaire, they will sort of say... it's all awful and it's all really getting worse… that was noted and the patient was called back in early.” [participant 17]

“One consultant called me back in because the patient had left [the HRQL questionnaire] with him… and he’d looked at it and was concerned at what he was

seeing, and saying, ‘That’s not the patient that presented to me. We need to call them back in and we need to... talk to them again’.” [participant 15]


RESULTS

Co-intervention Bias


bias caused by ‘any intervention other than the experimental manoeuvre that alters the frequency of a

trial’s outcome of interest’ (Sackett, 2011)

RESULTS

“You might feel a bit torn between… just being somebody who’s making sure all the data’s complete… you’ve got your complete datasets… and you’re also paying

attention that somebody that needs help... is gonna get some support.” [participant 25]

“I have been in that position and I have been told that the study comes first… at the end of the day… that’s not how it works… I am [a] registered nurse… I have

to act upon that as well.” [participant 8]

Theme 3 Burden: data collection could be associated with emotional and/or ethical burden for some trial staff.

RESULTS

“I don’t think there's an overall clarity about… using quality of life measures... [if the patient writes] additional information [on their questionnaire], how does that get recorded, how does that get fed back to the team? What happens… if you are concerned about somebody? What's the process? What level should we get

involved?... these sorts of issues aren't really covered anywhere.” [participant 20]

Theme 4 Lack of training: frequent lack of QOL-specific guidelines and training for trial staff.

RESULTS

“We could have done with the training on the relevance of quality of life much, much earlier on... I think that would have helped explain the role of quality of life to

patients as well, erm, because the feedback you get is… ‘I can’t understand why they’re asking me this,’… so, I think it’s very important that you educate the

interviewer so that they can explain.” [participant 4]

Theme 4 Lack of training: frequent lack of QOL-specific guidelines and training for trial staff.

RESULTS

Theme 4

Theme 3

Theme 2

Theme 1 Inconsistency

Researcher Burden

Lack of Guidance

PRO Alerts

29

CONCLUSIONS

Systematic Review

Qualitative Study

National survey

Protocol Review

PIS Review

Data quality

Co-intervention bias

Pt Understanding

Inconsistency

Researcher Burden

Lack of Guidance

PRO Alerts

CONCLUSIONS

Systematic Review

Qualitative Study

National survey

Protocol Review

PIS Review

Training

Guidelines

Inconsistency

Researcher Burden

Lack of Guidance

PRO Alerts

WHAT IS OWED TO PRO RESEARCH PARTICIPANTS?

Professor Heather [email protected]

• Researchers have ethical obligations to research participants• Consent, confidentiality, balance of harms & benefits

etc.• Generally assumed that participant should not be

foreseeably disadvantaged as result of participation• E.g. trials against SAT rather than placebo

• If harms foreseeable, steps taken to minimise• E.g. monitoring

• Treating clinician responsible for monitoring & treating

• Participant expectations - assume have reported

• Researchers with dual roles have to put patients first

Solution 1:Do nothing

Solution 2:Clear PIS

with disclaimer/

helpline

Solution 3:PRO alerts

and proactive response

Data lost to study

Data

collected

PRO research merely records, not intervention

• If so, then needs similar monitoring to somatic interventions

• With alerts in place if distress results

Completing a PRO questionnaire is an intervention?

30

Session 3: PRO Trial Conduct !!

Topics for group discussion:

1.  Is experience of PRO alerts widespread? How best should they be managed? Is there existing guidance we can draw upon?

2.  What information for data collection staff should be included in protocol guidance/trial training and why?

3.  How do we encourage uptake of new guidance?

4.  Do patients understand why we assess PROs in trials? What PRO-specific information should be included in the patient information documentation? Are/should PROs be discussed in consent interview?

31

Cancer Care and Epidemiology

Patterns of Reporting PRO Results from Randomized

Clinical Trials: Implications, Impact, and

Challenges

MD Brundage and ColleaguesMarch 2013

Objectives

• Current patterns of reporting PROs in RCTs of biomedical interventions

• Illustrate knowledge translation issues from a clinical perspective

• To review consensus on a CONSORT extension guidance for reporting PROs in clinical trial with implications for further research

Objectives



Objectives



• To review consensus on a CONSORT extension guidance for reporting PROs in clinical trial with implications for further research

HQL Measurement and Epidemiology

Clinical Trials

GuidelineDevelopment

Population StudiesProgram EvaluationPolicy Formulation

Patientand

Practitioner

The Adoption of Health-Related Quality of Life Innovations

32


Clinical Trials



Patientand

Practitioner



Clinical Trials



Patientand

Practitioner


Back to our Research Question

I’m interested in comparing two treatments:“R” and “M”

I want to evaluate how each treatment affects patients’ quality of life.

NCIC CTG PR.3/MRC PR07/SWOG JPR3:Study Scheme

Initial PSA Level: < 20 vs 20‐50 vs > 50 μg/L Hormonal Therapy: orchiectomy vs LHRH analogue+ anti androgen Method of lymph node staging: clinical vs radiological vs surgical Gleason Score: < 8 vs 8‐10 Prior hormonal therapy: yes vs no Centre

Continuous Androgen Deprivation Therapy

+ Radiotherapy


T3/T4 N0/NXor

T2 and PSA > 40 μg/Lor

T2 and PSA > 20 μg/L and GS: 8‐10

NCIC CTG PR.3/MRC PR07/SWOG JPR3:Study Scheme

Initial PSA Level: < 20 vs 20‐50 vs > 50 μg/L Hormonal Therapy: orchiectomy vs LHRH analogue+ anti androgen Method of lymph node staging: clinical vs radiological vs surgical Gleason Score: < 8 vs 8‐10 Prior hormonal therapy: yes vs no Centre


+ Radiotherapy


Locally Advanced Prostate Cancer

Planned Treatment

Androgen Deprivation Therapy

• Bilateral Orchiectomy

or

• LHRH agonist

– Antiandrogen for 2 weeks, optional to continue

Radiotherapy

• 45 Gy/25 F/5 weeks to pelvis

• 20‐24 Gy/10‐12 F/2‐2.5 weeks to prostate

• If treating physician felt patient inappropriate for whole pelvis then RT given to prostate only

33

Locally Advanced Prostate Cancer1990s

Canadian and UK surveys of clinicians revealed substantial uncertainty about the role of radiotherapy“These men all have metastatic disease; adding radiotherapy to hormones is unnecessary and unkind”

Baseline Characteristics

Characteristic ADT Alone ADT+RT

Median Age 69.7 years 69.7 years

T Category

< T2c

T3/T4

11%

89%

10%

88%

Gleason Score

< 7

8‐10

81%

18%

81%

18%

PSA ng/ml

<20

20‐50

>50

37%

38%

25%

36%

38%

26%

15

Final Analysis ‐ overall survival

HR = 0.70 (95% C.I. 0.57 to 0.85, P = 0.0003)

10 yr OS 55%

10 yr OS 49%

+RT

Per

cent

age

0.0

0.2

0.4

0.6

0.8

1.0

Time (Years) # At Risk(ADT)

# At Risk(ADT + RT)

0602603

2571558

4498505

6353381

8185208

107785

122832

ADT ADT + RT

16

Final Analysis: Cumulative Incidence Probability for Disease‐Specific Survival

DSS HR=0.46 (95% CI 0.34-0.61)

Estim

ated

Cum

ulat

ive

Inci

denc

e

0

20

40

60

80

100

Time (Years)0 2 4 6 8 10 12

Death related to disease ADT ADT + RT

0

10

20

30

40

50

60

70

80

90

100

0 6 12 18 24 30 36

17

Quality of Life:Bowel Domain (EORTC QLQ)

Me

an

Sym

pto

m S

core

s

Bowel and Rectum

ADT onlyADT + Radiation

Months

Few

er S

ym

pto

ms

18

Quality of Life:Bowel Domain (EORTC QLQ)

Me

an

Sym

pto

m S

core

s

0

10

20

30

40

0 6 12 18 24 30 36

0

10

20

30

0 6 12 18 24 30 36

Bowel and Rectum


Months

Few

er S

ym

pto

ms

Diarrhea


34

Proportion of Patients Worsening

• Patients deteriorating by 10 points or more at any point up to 3 years

0

10

20

30

40

50

60

70

80

90

100

ADT only ADT + XRT

Not worsened

Worsened

Rectal Symptoms

P < 0.01

0

10

20

30

40

50

0 6 12 18 24 30 36

20

Quality of Life:Urinary Domain (FACT‐P)

Me

an

Sym

pto

m S

core

s


Months

Few

er S

ym

pto

ms

Proportion of Patients changing

• Patients changing by 10 points or more at any point up to 3 years

0

10

20

30

40

50

60

70

80

90

100

ADT only ADT + XRT

Improved

Neither

Worsened

Urinary Symptoms

P > 0.05 0

10

20

30

40

50

60

70

80

90

100

Baseline 1 2 3 4 5

Time from Baseline in Years

PH

YS

Sc

ore

22

Quality of Life:Physical Domain (EORTC)


35

Objectives

1. Has the quality of RCT PRO reporting improved?

2. In what ways do RCTs report PRO findings?

Screening Articles for Review and Abstraction

ONCOLOGY papers’02 - ‘08

N=998

Non-ONCOLOGY papers’02 - ‘08N=4866

Total794

Random sample prior to screening

413381

Random sample prior to screening

Eligibility:

- Completed Phase III RCT of intervention with HRQL endpoint - Full publication was available in English- Primary report or expanded secondary analysis - Not an aggregate / meta-analysis / review

Disease or Cancer Site Studied

Disease/Condition:

Cardiovascular 15%Gastrointestinal 9%Gynaecological 5%Neurological 5%Psychiatric 9%Arthritis/Orthopaedic 8%Respiratory 10%Other 39%

Cancer Site:

Breast 22%Lung 15%Colorectal 10%Mixed 12%Prostate 7%Gynaecological 7%Other 27%

Initial ReportSupplementary

88%12%

84%16%

Non-Oncology (N=413) Oncology (N=381)

0 20 40 60 80 100

Rationale Providedfor Instrument

Evidence forInstrument Validity

Statement on WhoCompletedInstrument

HRQL Hypothesis

HRQL Sample SizeCalculation

Flow Chart

Missing DataAddressed

Discussion ofFindings

Percent of Trials

All Trials(N=794)

36

0 20 40 60 80 100




HRQL Hypothesis


Flow Chart



Percent of Trials

HRQL SecondaryOutcome (N=592)

HRQL PrimaryOutcome (N=202)

0 20 40 60 80 100




HRQL Hypothesis


Flow Chart



Percent of Trials

HRQL in MainRCT (N=684)

HRQL in SecondaryReport (N=110)

0 20 40 60 80 100




HRQL Hypothesis


Flow Chart



Percent of Trials

Oncology Setting (N=381)

Non-OncologySetting (N=413)

0 20 40 60 80 100



HRQL Hypothesis

Sample SizeCalculation



Percent of Trials

2002 (N=97)

2003 (N=104)

2004 (N=160)

2005 (N=136)

2006 (N=100)

2007 (N=99)

2008 (N=98)

• Important variation in quality of HRQL reporting in RCTs remains

• Secondary reports help, but only apply to about 15% of studies

• This can’t be good.

0 20 40 60 80 100

Mean Scores Provided

Mean Change Scoresfrom Baseline

Provided

Response Scores(Proportions

improved/deteriorated)

Graph(s) used forHRQL Data

Presentation

Percent of Trials

All Trials(N=794)

37

0 20 40 60 80 100

Mean Scores Provided

Mean Change Scoresfrom Baseline

Provided

Response Scores(Proportions

improved/deteriorated)

Graph(s) used forHRQL Data

Presentation

Percent of Trials

HRQL Primary Outcome (N=202)

Supplementary HRQLReport (N=101)

All Trials(N=794)


Clinical Trials



Patientand

Practitioner


• Appropriate instrument

• Bias resulting from missing data (missing not at random)

• Type I statistical error resulting from multiple testing

• Others


Clinical Trials



Patientand

Practitioner


• Oncologists’ attitudes, knowledge, understanding and skills relating to clinical trial HRQL data

• Intrinsic and extrinsic barriers to effective uptake

• Factors that enable uptake

• Oncologists’ preferred learning experiences and educational needs

• 30 academic oncologists participated – 2 Centres

• Structured semi-qualitative interviews

• Quantitative component

38


•A recurring theme was that HRQL data “help patients, and empower patients to make decisions based on their values” and can “level the playing field between physician and patient”.


• Many participants were equivocal about validity of HRQL measurement

“Many trial reports do not give enough detail. I do not know enough about the scales and it is hard to gauge sketchy, patchy information”.

0%

20%

40%

60%

80%

100%

Strongly Agree Agree Neither Agreenor Disagree

Disagree StronglyDisagree

I have a good understanding of the concept of quality of life

0%

20%

40%

60%

80%

100%



The measurement of quality of life in clinical trials isgenerally valid

0%

20%

40%

60%

80%

100%



I feel comfortable interpreting quality of life data as it isreported in the clinical trial literature

0%

20%

40%

60%

80%

100%



I feel a need to improve or increase my use of clinical trialquality of life data in my clinical practice

39

First Order Barriers: (Intrinsic to the Individual)

• Lack of knowledge of the HRQL measurement tools

•Lack of confidence in ability to interpret the published data

•Skepticism regarding current measurement techniques

Second Order Barriers: (Extrinsic to the Individual)

Mostly related to trial results:

• Questionable quality of the data

• Lack of standardization in presentation of findings

• Validity of clinically meaningful change in HRQL

Second Order Barriers: (Extrinsic to the Individual)

Mostly related to trial context:

• Lack of data that is relevant to specific populations

• Non-generalizability of the data

• Lack of time to access secondary publications


Clinical Trials



Patientand

Practitioner


Enablers

*Clinician understanding of the potential value of HRQL data

*Clinician willingness to use the data if available and if relevant to their patient populations

*Clinician willingness to learn more

40

Some Background onCONSORT

(Consolidated Standards for Reporting Trials)http://www.consort-statement.org

The CONSORT group:

• Proven ability to reach consensus on reporting guidelines• Includes editors of key journals • Includes clinical trialists and other stakeholders• Promotes successful implementation of guidelines / “buy-in”

A core group of reporting guidelines has been developed and should be used to report any RCT.

More on CONSORT Guidelines……




Other more specific guidelines are contained in what are termed ‘CONSORT Extensions’.


Why Use this Approach?

1. To reach the right number of right people quickly.

A CONSORT extension would provide guidance for editors, reviewers, and authors on meeting the appropriate standards for HRQL reporting

2. It works.

Existing CONSORT guidelines have been widely adopted. The website has had over 100,000 hits in the 6 months following the latest CONSORT 2010 statement in March.

3. It is an inherent component of our KT strategy.

Our project plans to stream a number of its KT activities through the CONSORT / EQUATOR Network.

Recommended Process for Developing a Health Research Reporting Guideline

(Adapted from D. Moher et al., 2010)

1. Initial steps2.Pre-meeting activities 3.The face-to-face consensus meeting itself4.Post-meeting activities5.Post-publication activities

41

Initial steps

• Identify the need for a guideline √• Review the literature √• Identify previous relevant guidance √• Seek relevant evidence on the quality of

reporting in published research articles √• Identify key information related to the

potential sources of bias in such studies √

Pre-meeting activities

• Obtain funding for the guideline initiative √(MRC-funding obtained, supplementary CIHR-funding requested to allow 30 international participants at the face-to-face meeting)

• Identify participants √( Core team of ISOQOL members form the planning committee. Broad stakeholder group to be invited to face-to-face meeting)

Pre-meeting activities currently in process:

• Conduct a Delphi exercise• Generate a list of items for consideration at the face-

to-face meeting • Prepare for the face-to-face meeting

Acknowledgements

We thank our funders and collaborators:

Funders: Medical Research Council, Canadian Institutes for Health Research

Collaborators:• International Society for Quality of Life Research

• CONSORT Executive & EQUATOR Network

• MRC Midland and ConDuCT Hubs for Trials Methodology Research

• University of Birmingham, UK, Queens University, Canada, University of Bristol, UK, University of Ottawa, Canada

• Survey participants including: ISPOR, MRC HTMR, NIHR Research design service, SCT, European Clinical Trials Units, Journal Editors, Policy Makers

• Meeting participants

Modified Delphi Approach

What does the literature say? Survey

Items include:

• Study population

• Instrument characteristics

• Study hypotheses

• Data collection procedures

• Statistical analyses

• Study results

• Quality and monitoring

• Discussion & interpretation

• Other

42

Survey of Key StakeholdersSurvey Response

Stakeholder n

International Society for Quality of Life Research 161

International Society for Pharmacoeconomics and Outcomes Research

82

Society for Clinical Trials 63

European Clinical Trials Units 58(UK (n=43); Italy (n=2); Switzerland(n=12); Denmark (n=1)

Research DesignService

28

Medical Research Council (MRC) Hubs for TrialsMethodology Research

18

Cochrane Patient‐Reported OutcomesMethods Group

8

Policy makers 4

Development of Proposed Reporting standards

• Survey results were debated by the ISOQOL Reporting Guidelines Task Force

• Feedback was obtained at the ISOQOL Annual Conference, Denver, October 2011 and used to inform proposed standards for this meeting.


An example: 'HRQL should be identified as an outcome in the abstract"

0

20

40

60

80

100

"Essential"

"Desirable"

"Optional"

"Rarely Neces."

When primary outcome

0

20

40

60

80

100

"Essential"

"Desirable"

"Optional"

"Rarely Neces."

When secondary outcome


An example: "HRQL should be identified as an outcome in the abstract"

0

10

20

30

40

50

60

70

80

90

100

When primary outcome

"Optional"

"Desirable"

"Essential"

0

10

20

30

40

50

60

70

80

90

100

When secondary

outcome

"Optional"

"Desirable"

"Essential"

0102030405060708090

100

ISO

QOL

ISPOR

SC TRIA

LS

CTUs

RDS/MRC H

ubs/

Cochra

ne

Polic

y

% Desirable when Primary Outcome

% Essential when Primary Outcome

0102030405060708090

100

ISO

QOL

ISPOR

SC TRIA

LS

CTUs

RDS/MRC H

ubs/

Cochra

ne

Polic

y

% Desirable when Secondary Outcome

% Essential when Secondary Outcome

N= 161 (ISOQOL), 82 (ISPOR), 63 (SC Trials), 58 (CTUs), 54 (RDS/MRC Hubs/Cochrane), 4 (Policy)


"HRQL should be identified as an outcome in the abstract"

43


Survey results were debated by the ISOQOL Reporting Guidelines Task Force & synthesised into 3 groups of reporting standards:

Feedback was obtained at the ISOQOL Annual Conference, Denver, October 2011 and used to inform proposed standards for this meeting.

Development of ISOQOL Proposed Reporting standards

Summary

Key Points

• Scope: PROs to include HRQL, symptoms, multi‐attribute utility measures, patient satisfaction and adherence.

• Focus of meeting is on trial reporting.

• Reporting of PROs as primary vs. secondary outcomes

• Primary publication may be the sole source of PRO data.

• Modified Delphi Approach has been used to inform the reporting standards proposed today.

The face-to-face consensus meeting itself

Activities to take place at a meeting in November 2011:

• Present and discuss results of pre-meeting activities and relevant evidence

• Discuss the rationale for including items in the checklist of reporting standards

• Develop: flow diagram, KT strategy, strategy for producing documents

44

What was the tastiest thing you had this morning for breakfast?

1. Cereal

2. Toast

3. Croissants

4. Eggs and bacon

5. Cheese

6. Coffee

7. Cigarette

8. Beer

9. Nothing

No additional guidance needed CONSORT sub‐optimal for PROs

PRO‐specific standards needed

Primary vs. secondary outcome

Few true “Extensions” required

Detail required

All secondary outcomes

Primary / “important” secondary

Elaborations also appropriate

Parsimony required

CONSORT Reporting Standard

2010 Guidance

• Completely defined pre‐specified primary and secondary outcome measures, including how and when they were assessed

CONSORT Reporting Standard Category:

METHODS: OUTCOMES

2010 Guidance

• Specific objectives or hypotheses


INTRODUCTION, BACKGROUND AND OBJECTIVES

Proposed Standard:

3. The PRO hypothesis should be stated and should specify the relevant PRO domain(s) if applicable.



45


1 2 3 4

56%

0%4%

41%1. Include ‐ all trials with PRO

2. Include ‐ when PRO is primary outcome

3. Unsure

4. Exclude

2b. Specific objectives or hypotheses

1 2 3

96%

0%4%

1. Agree

2. Disagree

3. Further discussion

Propose: The PRO hypothesis should be stated and should specify the relevant PRO domain(s) if applicable.

Example

Potential survival benefit needs to be weighed against the burden of treatment. For this reason, HRQOL…was included as a secondary end point in the EORTC 18991 study …

The protocol hypothesized that there would be a difference in global HRQOL scale between both arms, showing worse HRQOL in the PEG‐IFN‐α‐2b arm. The remaining HRQOL variables were then examined on an exploratory basis.’


TITLE & ABSTRACT

Proposed Standard:

1. The PRO should be identified as an outcome in the abstract




1 2 3 4

43%

0%4%



3. Unsure

4. Exclude

46

1b. Structured summary of trial design methods, results and conclusions

1 2 3

92%

4%4%

1. Agree

2. Disagree


Propose: The PRO should be identified as an outcome in the abstract if protocol‐specified.


METHODS: OUTCOMES

2010 Guidance

• Completely defined pre‐specified primary and secondary outcome measures, including how and when they were assessed

• Any changes to trial outcomes after the trial commenced, with reasons


METHODS: OUTCOMES

Proposed Standard:

7. The rationale for choice of the PRO instrument used should be provided.

7. The rationale for choice of the PRO instrument used should beprovided.


7. The rationale for choice of the PRO instrument used should beprovided.

1 2 3 4

58%

0%4%



3. Unsure

4. Exclude

2a. Scientific background explanation and rationale

1 2 3

88%

0%

12%

1. Agree

2. Disagree


Propose: The relevant background and rationale for the protocol‐specified PRO should be included.

47


METHODS: STATISTICAL METHODS

Proposed Standard:

16. Statistical approaches for dealing with missing PRO data should be explicitly stated, and the extent of missing data should be discussed.

16. Statistical approaches for dealing with missing PRO data should be explicitly stated, and the extent of missing data should be discussed.


16. Statistical approaches for dealing with missing data should be explicitly stated, and the extent of missing data should be stated.

1 2 3 4

68%

0%0%

32%

1. Include ‐ all trials with PRO


3. Unsure

4. Exclude

12b. Methods for additional analyses, such as sub group analyses and adjusted analyses

1 2 3

93%

0%7%

1. Agree

2. Disagree


Propose: Statistical approaches for dealing with missing data should be explicitly stated.


METHODS: STATISTICAL METHODS

Proposed Standard:

17. The manner in which multiple comparisons have been addressed should be provided.

48




1 2 3 4

74%

0%

11%15%

1. Include ‐ all trials with PRO


3. Unsure

4. Exclude


RESULTS: OUTCOMES AND ESTIMATION

Proposed Standard:

23. The proportion of patients achieving pre‐defined responder definitions should be provided where relevant.

23. The proportion of patients achieving pre‐defined responder definitions should be provided where relevant

.


23. The proportion of patients achieving pre‐defined responder definitions should be provided where relevant.

1 2 3 4

44%

8%

16%



3. Unsure

4. Exclude


TITLE & ABSTRACT

2010 Guidance

1a. Identification as a randomised trial in the title.

1b. Structured summary of trial design, methods, results & conclusions.

49

7a. How sample size was determined

1 2 3

96%

0%4%

1. Agree

2. Disagree


Propose: No extension

22. Interpretation consistent with results, balancing benefits and harms and considering other relevant evidence

1 2 3

69%

8%

23%

1. Agree

2. Disagree


Propose: The clinical significance of the PRO findings should be discussed.


Clinical Trials



Patientand

Practitioner


CONSORT


Clinical Trials



Patientand

Practitioner


CONSORT

COSMINSPIRIT

0%

20%

40%

60%

80%

100%



I feel comfortable interpreting quality of life data as it isreported in the clinical trial literature

Purpose & Objective

Attitudes

Perceived knowledge

Experience

Perceived barriers

50

We asked oncologists if they……

• find patient‐reported outcomes useful

• have positive (or negative) attitudes towards them

• use them in their clinical practice

• face barriers that prevent from using them

Methods

• Study participants

• Inclusion criteria

1) Be a practicing oncologist/ hematologist

2) Be a member of the NCIC Clinical Trials Group

3) Be involved in disease site/ scientific committee

Results

• Demographics

• Males (62%) Females (38%)

• Between 1‐20 years of medical practice (64%)

Results

• Clinical role and research exposure

• 85% spend more than half their time in clinics

• 74% work in both curative and palliative settings

• 96% have previously participated in an RCT‐ 87% have had exposure to patient‐reported outcomes in clinical trials

Results

• Perspectives on patient‐reported outcomes

• 66% have a good or very good understanding of the concept of patient‐reported outcomes

• 68% find patient‐reported outcomes useful or very useful

Results

Respondents (%)

51

Results

• Barriers to using patient‐reported outcomes in their clinical practice

#1 Concerns about the generalizability of results#2 Time

#3 Lack of understanding of results

Upcoming results

Australasian Cancer Clinical Trials Network

(Australia and New Zealand)

National Institute for Health Research Cancer Research

Network(United Kingdom)

NCIC Clinical Trials Group(Canada)

Proportion of Patients changing

• Patients changing by 10 points or more at any point up to 3 years

0

10

20

30

40

50

60

70

80

90

100

ADT only ADT + XRT

Improved

Neither

Worsened

Urinary Symptoms

P > 0.05


Clinical Trials



Patientand

Practitioner


Overall Quality of Life

0

20

40

60

80

100

0 12 24

Time (Months)

Glo

bal Q

OL

Treatment "A"

Treatment "B"

Bet

ter

QO

L Overall QOL: Response at Six Months

0

20

40

60

80

100

Treatment "A" Treatment "B"

Pro

port

ion

of P

atie

nts Improved

Deteriorated

0

1020

30

4050

60

7080

90

Prefers LineGraph

Prefers BarGraph

Finds BothUseful

Adjuvant Setting

Per

cen

t o

f P

arti

cip

ants

0102030405060708090

100

Prefers LineGraph

Prefers BarGraph

Finds BothUseful

Adjuvant Setting

Palliative Setting

Per

cen

t o

f P

arti

cip

ants

52

Future Directions:

•Promote the CONSORT Pro extension Facilitate its use

•Explore best ways to communicate data to patient and clinician consumers

•Improve clinical trial process

• Protocol completeness

• Clinical trial execution

•Integrate with new technologies – e.g. PROMIS adaptive methods

54

Birmingham Institute of Translational Medicine

• World class clinical facility

• Large catchment area

• Clinical trials infrastructure

• Clinical academic expertise which allows delivery of world class translational medicine programme

The largest catchment area in Europe

Birmingham Biomedical Campus and the Institute of Translational Medicine

Broad work areas

• Bioclinical stratification of patient cohorts

• Bioclinical stratification of patients enrolled in clinical trials

• Early phase trials

• Phase II and Phase II clinical trials

• Informatics

• Medical device development and evaluation

• in vitro diagnostic development and evaluation

• industry‐academic partnership

• Patient reported outcomes

Specialty groups

• Audiology/ENT• Cardiology Critical Care• GI Medicine• Liver Disease• Opthalmology• Paediatrics• Rheumatology• Renal Medicine and Diabetes Medicine• Respiratory Medicine

55

Theme groups (non‐oncology)

• Auto‐immune and inflammatory diseases• Chronic Diseases• Acute disease and trauma • Transplantation• Rare Diseases • Paediatric and Transition medicine• In vitro diagnostics• Advanced Imaging• Horizon scanning• Periodontal Health• Patient Reported Outcomes and QOL

56

Neurology Trials and PROHardev Pall

Clinical and Experimental Medicine

Parkinson disease, mortality and DBS• Desire Ngoga, Rosalind Mitchell, Jamilla Kausar, James Hodson, Anwen

Harries, Hardev Pall. Deep brain stimulation improves survival in severe Parkinson’s disease. J Neurol Neurosurg Psychiat In Press 2013

• Cohort study over 10 yr follow up

• Controls – patients who were eligible for surgery but declined

• Primary endpoint – mortality

• Significant difference in mortality and rates of admission to residential care.

• Not randomised

• No patient reported outcomes

• Other studies have reported improved QoL

New Research Question?• Is survival improved by deep brain stimulation of the

subthalamic nucleus in Parkinson’s disease?

• Is this good quality improved survival?

Possible trial design?• Randomise patients to immediate DBS surgery or surgery

delayed till all medical avenues exhausted

• 10 year follow up after DBS to:

• Determine if survival is longer in those operated on at an earlier stage

• Assess ongoing QoL in the two groups

What PRO’s?• EQ5D

• PDQ39 – mobility, ADL, Emotional well-being, sense of stigma, social support, cognition, communication, bodily discomfort and a summary index

• DBS has motor effects – cognition may be helped least or possibly impaired. Does that mean we selectively pick this?

• Carer outcomes

• Driving

• Holidays

• Interaction with families

• Employment

57

Haemodialysis

• Three times a week for four hours

• Significant time travelling and waiting

• Doesn’t give normal renal function

• 15% die per year....

Improving survival in dialysis

When 100 patients were asked if they would consider more frequent haemodialysis:

94% would if increased energy levels

57% would if improved sleep quality

19% would if improved survival by up to 3 years

Ramkumar et al. Hemodial Int 2005 9 281‐295

Current measures

• KDQOL: 15 pages. – Short version 36 questions over 5 pages

• Depression: BDI / HADS

• POS‐S Renal

• EQ5D

• Dialysis recovery time

• PREM

58

Uveitis: inflammation in the eyeUveitis: inflammation in the eye

Visual lossVisual lossUVEITISUVEITIS DamageDamageActivityActivityPatient

experience

Patient experience

Only one FDA recognised outcome for UveitisVitreous haze: clinical assessment vs photograph

4+ 3+ 2+ 1+ 0.5+ 0

Outcome: Disease ActivityOutcome: Disease Activity

Nussenblatt RB, Palestine AG, Chan CC, Roberge F. Standardization of vitreal inflammatory activity in intermediatand posterior uveitis. Ophthalmology. 1985;92(4):467‐71

Uveitis: inflammation in the eyeUveitis: inflammation in the eye

Visual lossVisual lossUVEITISUVEITIS DamageDamageActivityActivityPatient

experience

Patient experience

KEY DOMAINS

Outcome: Patient ReportedOutcome: Patient Reported

Body Function/Structure (Impairment)

‘loss or abnormality in body structure or physiological function’ (WHO)

Activity limitation

‘difficulties an individual may have to executing activities’ (WHO)

Participation restriction

‘problems an individual may experience in involvement in a life situation’ (WHO)

Satisfaction

Not defined within the WHO ICF

Body Function/Struct

ure


ure

ActivityActivity

ParticipationParticipation

SatisfactionSatisfaction

Che Hamzah J et al. Choosing appropriate patient‐reported outcomes instrument for glaucoma research: asystematic review of vision instruments. Qual Life Res. 2011;20:1141‐58.

KEY DOMAINS VISION‐SPECIFIC PROs



ure


ure

ActivityActivity



Vision Impairment Measure


Visual Disability Measure

Visual Disability Measure

Vision Status Measure


Vision Satisfaction Measure


KEY DOMAINS NEI‐VFQ25



ure


ure

ActivityActivity



9 Qs9 Qs

1 Qs1 Qs



13 Qs13 Qs

2 Qs2 Qs

59




ure


ure

ActivityActivity





VCM‐1VCM‐1

NEI‐VFQ25NEI‐VFQ25



Gardiner AM BJO 2002Murphy CC BJO 2005

Gardiner AM BJO 2002Murphy CC BJO 2005

Schiffman RM Arch Ophth 2001

Schiffman RM Arch Ophth 2001




ure


ure

ActivityActivity





VCM‐1VCM‐1

NEI‐VFQ25NEI‐VFQ25



ENDURE/INSUREENDURE/INSURE

MUSTLUMINATEVISUAL

ENDURE/INSURE

MUSTLUMINATEVISUAL

ENDURE/INSURE

60

PRO work shop Birmingham Session 5 Research priorities: a clinical perspectiveProfessor Paulus KirchhofChair in Cardiovascular MedicineUniversity of Birmingham Centre for Cardiovascular Sciences andSWBH NHS Trust, Birmingham, UKDepartment of Cardiovascular Medicine, University of Münster, [email protected]

WESTFÄLISCHEWILHELMS-UNIVERSITÄTMÜNSTER

• Why bother?

• The “EHRA score” as a simple PRO tool in atrial fibrillation

• PRO in cardiovascular guidelines?

• PRO in clinical trials in AF: two examples

• Why bother?




Personalised cardiovascular medicine

• The medical profession (and, by extension, health care workers) aim to prolong life and to reduce suffering.

• Cardiovascular diseases are still the most common cause of death and chronic illness in the world.

• Evidence-based medicine was essential to reduce the burden of cardiovascular diseases in Europe and world wide in the last 2 decades.

• Prevention (which may reduce PROs) can help to further reduce this burden to some extent.

• Personalised approaches to therapy supported by evidence are needed to improve the situation further.

• Why bother?




The “EHRA score” for AF Symptoms

Proposal for a new symptom classification scheme

Having noticed that there is no simple, easily applicable, yet AF-specific score to measure symptoms in AF patients, the panel agreed on an AF symptoms score

This classification relates not to the type of AF (to be determined by the physician), but exclusively to the patient-reported symptoms

The panel of experts suggests the following score to describe AF-related symptoms referred to as the EHRA score

1st AFNET/EHRA consensus document. Kirchhof P et al. Eur Heart J 28, 2803-2817 / Europace 9, 1006-23 (2007)

61

AF-related symptoms: EHRA score

1st AFNET/EHRA consensus document. Kirchhof P et al. Eur Heart J 28, 2803-2817 / Europace 9, 1006-23 (2007)

The „EHRA score“ assesses symptoms during AF

In addition, symptom frequency is classified (rare – occasional – frequent)

• Why bother?




Principles of Antiarrhythmic Drug Therapy to Maintain Sinus Rhythm

1. Treatment is motivated by attempts to reduce AF-related symptoms

2. Efficacy of antiarrhythmic drugs to maintain sinus rhythm is modest

3. Clinically successful antiarrhythmic drug therapy may reduce rather than eliminate recurrence of AF

4. If one antiarrhythmic drug ‘fails’, a clinically acceptable response may be achieved with another agent

5. Drug-induced proarrhythmia or extra-cardiac side-effects are frequent

6. Safety rather than efficacy considerations should primarily guide the choice of antiarrhythmic agent

ESC guidelines for the management of AFCamm AJ, et al. Eur Heart J 31: 2369-2429. (2010)

Rhythm Control therapy in AF

2012 focussed update of the ESC guidelines for the management of AFCamm AJ, et al. Eur Heart J, published on line 25 August 2012

• Why bother?




Flec-SL AFNET-3 Design

Standard long-term therapy6 months flecainide

Control groupNo antiarrhythmic drug therapy

Pharmacological reversal of electrical remodeling 4 weeks flecainide

cardioversion, randomization

6 months daily transtelephonic ECG monitoring

inclusion:- persistent AF- Indication for cardioversionpretreatment: - 48 h treatment with flecainide

Kirchhof P, et al. Am Heart J.150: 899 (2005)

Prospective, randomised, open, blinded outcome design (PROBE)

Primary outcome: time to persistent AF or death

Two hierarchichal analysis steps:

1. superiority of flecainide vs. no therapy (4 weeks FU, initial patients)

2. noninferiority of short-term vs. long-term therapy (all patients)

62

Flec-SL AFNET 3 CONSORT chart

Kirchhof P, et al. Lancet 380: 238-246. (2012)

Flec-SL AFNET 3: Primary outcome


635 patients, mean age 64 years, flecainide 4 weeks vs long-term therapyPrimary outcome: time to persistent atrial fibrillation or death, monitored by telemetric ECG

Short-term and long-term therapy were more effective than no therapyShort-term therapy conveyed 80% of the effect of long-term therapy

Flec-SL AFNET 3 Main secondary outcomes

• number of recurrent AF episodes prior to persistent AF not different between groups: 9 (2-13) – 14 (6-23) – 13 (6-23)

• no difference in major adverse outcomes(control 1 (1.2%), short-term 9 (3.3%), long-term 10 (3.6%))

death 0 – 0 – 0

resuscitation / VT: 0 – 2 – 1

syncope: 0 – 2 – 4

stroke/TIA: 0 – 3 – 3

major bleed: 1 – 2 – 2

• improved quality of life at the end of FU:Karnowsky (base 8.3 (1.2)): 8.8 (8.5-9.1)* – 8.9 (8.7-9.0)* – 9 (8.9-9.1)*

SF 12 physical (base 39 (11)): 45 (42-48)* – 44 (42-46)* – 44 (42-45)*

SF 12 mental (base 48 (10)): 49 (47-52) – 51 (50-53)* – 50 (49-52)*

* indicate p<0.05 versus baseline


EAST – AFNET 4 Hypothesis:

Early comprehensive rhythm control therapy can prevent AF-related major complications (stroke,

death, heart failure) compared to usual care

Early treatment of Atrial fibrillation for Stroke prevention Trial

www.easttrial.org

NCT 01288352, ISRCTN04708680

EAST – AFNET 4 trial flow chart

Patients at risk for cardiovascular events(≈ CHADSVASc score ≥ 2*)

and havingrecent onset atrial fibrillation (≤ 1 year duration or first documented by ECG)

Early Rhythm Controlanticoagulation, rate control and either antiarrhythmic drug therapy orpulmonary vein isolation (PVI)

in case of recurrent AF:Re-PVI, adaptation of antiarrhythmic drug therapy

ECG monitoring of therapy

Usual Careanticoagulation, rate control, supplemented by rhythm control only in symptomatic patients on optimal rate control therapy

outpatient FU at 12, 24, 36 moths (both study groups)therapy of underlying heart disease (both study groups)blind assessment of primary outcomes (both study groups)

Pre-Study Screening Study Procedures

Ran

do

mis

atio

n

*Detailed inclusion criteria: One of the following: age > 75 years or prior stroke / TIAORTwo of the following: age > 65 years; female sex; arterial hypertension;diabetes mellitus; previous myocardial infarction, CABG or PCI; stable heart failure (NYHA II or LVEF<50%); left ventricular hypertrophy (>15 mm wall thickness); chronic kidney disease (MDRD stage III - IV); peripheral artery disease.

Kirchhof P, et al. Am H J; in press (2013)

77 / 2810 patients enrolled

ROs:

HRA score

Q-5D, SF-12

Ontreal Cognitive Assessment

minute walk test

EAST – AFNET 4 trial: May 2013

www.easttrial.org

63

Summary: PRO research priorities

• PROs have a huge potential in an era of cost-contained medicine that demands patient-centered and personalised practice of medicine.

• PROs are “soft outcomes” in the context of trials

• difficult to measure and monitor

• likely to remain secondary outcomes in most trials

• Credibility: Reliability, validity (internal, external), bias?

• Standardisation (beyond EQ-5D L / SF-12)

• Role of disease-specific instruments?

64

65

Appendix 4: Notes made during group discussions


Key questions:

valid, acceptable to • How do we select PRO domains (e.g. fatigue/anxiety) and questionnaires that are

patients and will inform clinical care?

ome? ials? • What is the difference between a patient centred and patient reported outcHow do we ensure that patient centred outcomes are included in clinical tr

w can we address these issues? •• What other issues should we consider? Ho

roup Discussion 1 ‐ summary of key points G Tabl 1 e

1. who is answering the questionnaires ‐ is it the patients ? 2. best practice issues – lots of good work going on but lack of dissemination.

lidated etc and

3. clinical audit issues re dh funding dependent on specific audit measures that not va

ted? ignores research that’s been done.

nd are people interes enefit) ‐ next stages?

4. What are we doing re measuring outcomes for carers? A5. me sets – (in terms of patient b6.

How you use CORE outco

bridging the gap between research and patient benefit.

include patient’s earlier. balancing act of collaboration between all key stakeholders.

group presentation to patients on the trial you’re planning to run and then getting patients to email you afterwards about what they understood about the trial.

Table 2 • Patient information sheets – too long, good idea to use ‘light’ versions of PIS; we would recommend that NIHR talks to their ethics committees to make it acceptable to have short versions; alternative to have an ‘Executive Summary’ at the start

• Concern expressed around the expectation gap between the benefit that a patient perceives that they will get from being involved in a trial, the requirements on the researcher to ensure that patient info sheets state clearly that there is not expected to be a direct benefit and reality (which may lie somewhere in between)

• Within renal, there is an agreed data set that have to be returned for audit purposes and which can be used as a core outcome set; recently DoH have introduced an EQ5D,time for recovery from dialysis, POSS‐S‐Renal (symptom score – which has not been validated in these cohorts)

Table 3

w your plans and designs • Concise patient information sheets

atients to peer revie • Involving patients with every step – getting p

• Recruitment process and appointment of patient representative

mic background etc • Remit that patients have for giving opinions

ono es? • Recognising your patient group – socioec

iv e • How local are your patient representat

rs in your centr gy for PROMS • Advertising using poste

olo • Standardised method

• Feedback to patients….

ers…email? iceboard in opd • Newslett

• Poster / notWebsite

Phone apps •• Table4 k with ethics committees to • Improve presentation of PIS (length, accessibility, other methods dvd?) – wor

encourage consistency

• What are the preferred/optimal methods for incorporating PROs into COS’s • How does one label what a domain is? • Methodology needed to ensure a fully representative patient group (hard to reach, BMEG’s, those with depression etc)

• Wording of questionnaires to aid interpretation (audio options for those who face literacy challenges)

66

• Should questionnaires be tweaked in some way for use with• Note: online or web‐based translation services (real‐time?)

proxies?


Key questions:

ld be included in the trial protocol or other trial documentation such • What PRO‐related components shou

as SOPs?

• Language barriers – considerations • What are the key areas that need further work to inform optimal PRO trial design? How do we ensure that trials adopt best‐practice in PRO trial protocol configuration? Ideas to address •this? Tools? Education?

Group Discussion 2 ‐ summary of key points Table1 PROTOCOL

- - Depends what the trial is

- Which domains are you measuring

- Which questionnaires used Which direction is the measure of change – high scores / low scores

- measured at appropriate clinical time points and these be standardized to Timing of assessments –

- journey of treatment…lots of discussion around this point…

- m the start) Mode of administration Rationale for measurement (how was

-

this chosen, ie were patients involved fro- obal scores How the PRO is being analyzed ie specific domains of interest or gl

How the results will be disseminated - vering the intervention? Is the PRO data used by the clinicians deli

- SOPS

Guidance re pt concerns / contacting GP - Communicatio

LANGUASimplicity

n plan re updating pts.on progress of trial GE BARRIERS / translation

- languages, eg American questionnairess some of lots of questionnaire’s not validated in different

- the language different than in UK eg feeling blue Cost implications for using translated measures

- tures Cultural concepts differ eg pain different in different cul

- KEY AREAS NEEDING FURTHER WORK

Patient / carers involvement (as co‐designers in study) - yet applied Issue re funding, ie how do you pay for patient involvement in the design stages as not

- for funding… Development grants available to put together funding bids available from RfPB PGAR Access to pts if you’re not a clinician so can utilize clinical colleagues or clinical research nurses

- Accessing the right people and information , Sharing best practice, building collaborations etc – linical research networks

-

need to access your cACTICE

- funding conditional BEST PR

Table2 S eci inp your trial design: fy

• Domains first based on what you want to measure, and then choose the

tool/measure that you want to use

• Time windows; inc pre or post intervention (eg dialysis), pre or post doctor

• consultation – generally are done on arrival

Specify whether the patients allowed to have help (carer/nurse/doctor) to fill them in eg for vision, co‐ordination problems. NB issues around bias/pts want to please/think there’s a right answer. This ‘noise’ may be balanced between treatment arms.

w to deal with missing data, sample size • Statistics – planned analysis (primary, secondary), ho

Issues aro practice need to be considered in terms of und best• Design • Delivery

• Recommend to use the BRTC network (Birmingham region Research and Training Collaborative) to provide training to a) researchers on how to bring PRO correctly into the trial design/protocol and b)to

67

site staff to ensure appropriate procedures are followed when e.g. using patient questionnaires (e.g.

timing/location of questionnaires, using translators, using persons to read out questionnaires)

• Challenges of having training around questionnaires being both disease specific assessments and generic. an English, but

pulation. • Issues around language barriers –most questionnaires not validated in languages other thif one omits non‐English speakers than the trial results may not be applicable for your po

audio system/speech‐enabled IT for those who are illiterate/blind. • Consider usage of an Ensuring best practice • Responsibility of Sponsor/CTU/CRF • Have an equivalent standard to CONSORT that is required for publication warding Bodies (try to get a standard shared between • Checklists which are published by the Grant A

MRC/NIHR) as a minimum standard Educating researchers/grant panel members

Published best practice recommendations (preferably open access) •• Table3 • Clarify if relatives can assist patients to answer questionnaires – some family members may answer instead of the patient!

lowing the patient to answer • How can we read questions to patients whilst maintaining privacy and al

s work with language barriers honestly

• SOP – can questions be read. How do

es thi • Identify scoring manual for questionnaire

• Specify time points of questionnaire

• “core” requirements as opposed to “minimum” requirements

s • Who will look at questionnaire results…will they be acted on clinically or only for study analysis • Pilot – ask previous trial patients when they think the time points should be to answer questionnaire • Identify which domains really matter to patients. Select questionnaires appropriately and document reliability / validity

the first few pages completed more fully / with more thought than later • Length of questionnaires…are

pages in the questionnaire.

• Does the above affect quality • Are questionnaires answered before or after clinical assessment time / afterwards patient may be keen to get off • Egpatients are waiting to be seen so questionnaires pass

home.

• Also clinical assessment may change patient perspective

• Commercial vs university sponsored trials • Layout of questionnaire…being clear with plenty of spaces to ensure more complete data entry the • Check complete data entry at the time and let patient know if they have missed fields to give them

er etc. opportunity to complete

• Comment why questionnaire not fully completed in crf….not well enough / language barriAim for good quality data…not just large quantity/recruitment

Conflict of interest for Research Nurses collecting data if they cannot act on their findings •• Table4 • Funding application: justification, details of tool • In protocol: What domain(s) measures, what tool, details of measurement plan (time points, pre‐

randomisation?, in relation to clinic, postal questionnaire – justify what you are going to do and do it uniformly, and be explicit about what you are doing), and data analysis plan, with references

• How to prevent missing data what is the best way, do we know (if so how best to collect, how to remind r missing data, + how to deal with missing data + analyzing missing

e trial results) etc) + Method for recording reason fo

data + plan for reporting. Publication plan (PRO data alongsid

es • Pre‐specify key secondary outcom

• Specify the minimally important difference, justify, reference • ? Cite validation refs in protocol? • Justification for the tool (should we introduce PPI’s/potential trial participants at this stage to select the right outcome for them?)

• Could we look continual validation of a PROM in an interactive way from trial to trial (so the tool becomes lidate more accurate, useful?) – possibly should be a requirement laid down by funders – you must re‐va

the PROM (during or following the trial) to inform the next trial • We should encourage publication of the PRO data alongside the main trial results, funders should encourage this from the outset

• How do we balance out having fewer q’s (not burdening pts, not collected extra info) against more comprehensive measure which might capture unexpected data

68

• Is the answer to selection adaptive trial design, start wider then narrow down options as the trial progresses

Funding panels should pay more attention to PRO elements, **should be a PRO expert on the panel** and •more time to discuss PRO‐specific issues

Need publication/funding (stick) system to force people to conform to a SPIRIT/ CONSORT‐like extension.

Session 3: PRO Trial Conduct

Key questions:

• Is experience of PRO alerts widespread? How best should they be managed? Is there existing guidance we

can draw upon?

• What information for data collection staff should be included in protocol guidance/trial training and why?• How do we encourage uptake of new guidance? Do patients understand why we assess PROs in trials? What PRO‐specific information should be included

Are/should PROs be discussed in consent interview? •in the patient information documentation?

roup Discussion 3 ‐ summary of key points G Ta1. ble1 No,

ation given is shared with the team or GP?

Management – any informDoes it change how pt’s respond if they know info is being shared with team

co se… V. little existing guidance

2. mmu training / what to do when concerning issues ari1. Make it part of GCP training which is a requirement for all researchers

nicating with pts

- induction packs - clinical trials network part of site initiation to provide training

4. Where is pt told that research will learn from their responses but not necessarily benefit – PIS… -

Ta e2 1. Experience of PRO alerts is not widespread; it is felt they are a remote and a rare event. In certain

there

bl

clinical trials there are certain parameters that could be predicted to deteriorate and for thoseshould be a contingency plan put in place to deal with them as a certain threshold is reached.

2. There are many studies where PRO data collection is independent of clinical data collection – completely anonymous. If data collected in person, the collectors should be aware of threshold levels nd act or this should be red flagged. Question whether there is anything that is discussed in the visit hat the patient would like to be raised with their clinician. at

Table3 Even if reported what action is taken?

nParameters written i to protocol as with other measures? High and low measures and also for a step increase/ decrease? f information is not relevant then don’t collect. If it is important enough to collect then it ought to be Iresponded to. • What information for data collection staff should be included in protocol guidance/trial training and Guidance on limits for data entry time period as for RCTs.

n into protocol then action to be taken on collection of data taken whether

why?

If built in parameters writtelectronic data or not. eIs the PRO data monitored?

• How do we encourage uptake of new guidance?

re agreement about what this guidance sh • ould be? Is the

• Qualitative work initially with research bodies. • Stick • Do pat at PRO‐specific information should be included in the p Os be discussed in consent interview?

ients understand why we assess PROs in trials? Whtation? Are/should PR

atient information documen

Why would they? Clinicians find this very difficult.

Is this all overcomplicated?

Could this be free text and qualitative rather than quantitative manageable data Tools and tool choice has become very complex

Interpretation of QoL data very complex and poorly understood by researchers/ clinicians.

69

Information to be included: who will read, what will be acted upon, why it is being collected, how results will be reported.

Table4 Phone?

?

Unknown how muchWhat’s responsibility – to patient carer?

patient. ty to carer?

Suicide riskMask flipping off – carer duInform GP?

. ncerned.

Guidance regarding carersPatient – act if reason to be coImplications for blinding.

written /letters ronic data collection.

Guidance for Different from elect

. pain.

Letter to GPDehydration/Autonomy Risk related. SAE life threatening. Pilot guidance Paper based/electronically. Patients and thresholds Methods for PIS – multimedia/patient groups interactive with clinician.

Session 4: Use of PRO trial results to inform clinical practice

Key esqu tions:

1. What are the key challenges for reporting PROs? 2. How can we facilitate improved reporting? Tools, education, other KT activities? 3. What are the barriers to the use of PRO trial results to inform clinical care and how can we address

these? Group Discussion 4 ‐ summary of key points Table 1 er than just add on to meet funding • Ensure PROs used appropriately in clinical trials to begin with rath

requirements

• If reporting on clear hypotheses it becomes much easier to report • Bear in mind differences between clinical and statistical significance • What do the results actually mean in terms of scales, scores, differences and how this applies to thepatient?

eople without condition so what does it all • Trials don’t compare disease qol to normal age qol for p

actually mean?

• PROMIS working on getting normalized qols for people

• Clinician understanding of data – to aid collaborative / shared decision making for patient treatment • Aren’t always RCTs in disease areas – often more cohort studies so less simple choices SF36, useful for clinicians to have pop’n norms • Many tools do have validated noms to compare against eg

reported to compare against but not necessarily for patients

• Pts need the proportionate figures i.e. 1 in 10 will benefit ness – • Events eg harmful side effect easier to present proportionately, however domains such as tiredharder to present (quantify) in this way?

• Basic training for clinicians should include higher level understanding / analysis of results and communicating these to pts.

it • Clinicians who aren’t involved in research don’t necessarily access this information on a regular basis sobecomes much harder for them to interpret findings…

urther work • There are user guides for clinicians to understand research, but not written by clinicians…f

required on this

• Publish online guide to accompany the paper for clinicians for them to interpret the data… • Dissemination to patients is not done or not done well and then communicate this to patients • We need different tools / approaches to combine data from trials

eg web based decision making / podcasts etc…

• Patient lay summaries should be included with trial publications • Patients are looking for far more simplistic information than graphs and data required for reporting and clinicians

70

• We need to get the measurements and scales right as difficult to compare scales against each other when different measures are used so need for more standardized measures and scales – will improve

interpretation.T ble 2 • Cha lena

l ges of • Qualitative data – but recognize that there is a strong quantitative limit. • Time to analyse – but not necessarily so (eg Example of a trial analysis already done with a test

of

data set and then just insert the real trial data at lock, and have the analysis done within a couple hours).

• Higher amounts of missing data – how to impute and report; this may reflect problems in design or capture techniques; too high a burden in terms of multiple time points/unnecessary

uestionnaires; issues of data queries taking time). (po

• Fac itadomains/q

il te• Ch cklist

e

based on • interactive checklist for authors

your entries regarding trial design • encourage reviewers to use the checklist

• Encouraging submission of original data • Commitment to publish (eg NIHR commit to this already, but could this become general

requirement of all registered trials eg either by registries or by funding bodies); can patient voice be used to promote this (ie ‘don’t waste my time collecting data that you don’t disseminate’)

edominantly captures the key • Improve the selection/design of the PROMs we use to ensure only/prtcomes (although may need space for exploratory variables). ou

• Bar ier

• Method of delivery needs consideration eg paper based vs electronic r s lly • Most clinicians would like a single measure that is standard for their whole speciality – not usuapossible

• Jargon – need to make sure that the results can be communicated in a way that is meaningful to patients; may be helped to have patients involved more in the design stage

a T ble 3

ean etc 1• Understanding exactly what the pro is telling you. What exactly does 30% m

• Quality of life meaning varies from person to person • Being statistically significant may not be clinically significant or vice‐versa Looking at sub‐sections for accurate reporting as well as the tool as a whole.

may relate to patient group – can make interpretation difficult • • Not all of a generic tool

2• Education and training • Encourage uptake of this Simple online training sessions for completing questionnaires.

plete questionnaires accurately in pre‐registration training • • Address “value” of knowing how to com

3• Hard to put cost against quality of life

• There can be differences in interpretation of results in relation to QOL • ‘soft data’ difficult for clinicians to interpret themselves • Surgeons are currently measured on their surgery figures not patient QOL etc. which could present a

o take on high risk patients even though it may have a conflict of interest. There is potential resistance tchance to increase QOL. It is difficult to capture quality of life as a target. •

4• The key is having accurately reported data in the first place • Quality of communication in consultation • Ensuring patients are aware that some reduced quality of life is part of the disease process initially and

may improve with time. Maximum interventions may already be in place, so other potential interventionsmay only be considered if symptoms persist.

• Follow up consultations with letter to the patient as well as the GP. This is an informative way of helping the patient to remember what was said and the opportunity to then follow it up with the GP as required. QOL could be added to this. – this has time constraints especially if ideally the patient letter should be worded differently to GP letter

Table 4 Generic route – disease specific issues. People argue disease specific gives more info to guide individual care – but can this be implemented.

71

n/countriesPolicy decisio .

Patient input up front?

Pilot studies. Focus group work at the end.Respositories

thodology PROs. measurement properties and patient views.

Submit meLook carefully of existing measures –

turally appropriateness. Software/ ore on culM

Constructs. Ask patients

ht 1. Challenge of reporting PROs

The key is to get the hypothesis right, and then this allows reporting etc and interpretation rigtility data which if not done properly (and Also problems with interpreting EQ5D scores and u

2. reported properly) this can lead to misleading decisions

3. Need better peer reviewing and editing of journals Look at the educational literature to understand risk communication Need easily accessible data for clinicians to use for everyday conditions

5. Work out how to measure and what sorting out what is core – keep them together in one paper 4.

Session 5: Research priorities: a clinical perspective

Key considerations:

1. Selection of instruments (e.g. for rare diseases/ systemic vs localized effects/ generic‐e.g. EQ‐5Dvs. disease specific)

2. The ability as a community (within one specialty such as renal) to have a preferred PRO tool in trials in a certain area (eg haemodialysis) and having a central repository for use across several trials.

3. lish which is a significant and important Use of PRO in those who do not read or indeed speak Engminority of our population and how to address this.

4. Questionnaire fatigue (links to selection of instruments) 5. Use of PROs in routine clinical care vs. trials. – Suggest we limit our discussions on this. Perhaps topic

for a future workshop?

roup Discussion 5 ‐ summary of key points G Table 1 s • Think need both generic measures and symptom specific, so generic validated measures eg eq5d for croscultural comparisons. Specific should be hypothesis driven

is there any evidence • how many are too many questions eg 4 qr’s for a trial = 80+ questions (qr fatigue),

on this...no of questions, qrs, layout etc...

• should we be able to choose independent domains from validated questionnaires? r single items? so agreed PROs used • PRO bank repositary in specialty areas eg chronic disease, renal, or fo

across trials, deposited in bank and ethical approval to access data from the bank?

• Difference between language and culture in terms of questionnaires. • Clinically difficult to exclude a large number of your cohort because you know the measures etc aren’t applicable...try to address this, eg focus group to discuss depression symptoms with South Asians, lots of work to do...

Table 2

• Selection of instruments

• Use of generic vs disease‐specific • Defining what you want to measure first and then look for tools that capture the domain ifferent • Depends on the context – look at clinical trial stratification eg could have different PROs for ddiseases/subtypes of disease or a single PRO with pre‐specified analysis of selected domains

al societies/royal colleges/NICE/FDA ‐ may be able to provide t will depend on the domains being considered

• The community – internationrecommendations although i

• Translations to be validated

72

Appendix 5: Voting Slides

Best-Practice for Patient Reported Outcomes in Clinical TrialsInstitute of Advanced Studies Workshop

PRO Research Group


Session 1: PRO SelectionPlease vote for your top priority area for research from the following

options:

5%

14%

5%

36%

27%

14%1. Methods to select PRO domains.

2. Development of COS.3. Optimal methods of

patient involvement in outcome selection.

4. Guidance on measuring carer/proxy outcomes.

5. Feedback to patients on trial progress.

6. Development of training tools for PRO selection.

Session 1: PRO SelectionPlease vote for your second priority area for research from the following

options:

23%

9%

0%

27%

23%







Session 1: PRO SelectionPlease vote for your third priority area for research from the following

options:

30%

17%

4%

17%

9%








73

Research priority: PRO Protocol checklist?

1 2

0%

100%1. Yes2. No

Research priority: PRO Protocol checklist tools?

1 2

5%

95%1. Yes2. No

Session 3: PRO Trial Conduct

Research priority: PRO Alert Consensus & Guidance

1 2

13%

88%1. Yes2. No

Research priority: PRO Data Collection/Management Guidelines and Training

1 2

8%

92%

1. Yes2. No

Research priority: PIS – content and mode of delivery

1 2

4%

96%1. Yes2. No

74

Sessions 2 & 3

Please vote for your priority area for research from the following options:

0%

17%

9%

13%

61%1. PRO Protocol Checklist.2. PRO Protocol Checklist Tools.3. PRO Alert Consensus and guidance.4. PRO Data Collection/Management Guidelines

and Training.5. PIS Content and mode of delivery.

Sessions 2 & 3Please vote for your second priority area for research from the following

options:

0%

29%

33%

17%



Sessions 2 & 3Please vote for your third priority area for research from the following

options:

39%

35%

9%

13%



Session 4: Use of PRO trial results to inform clinical practice

Session 4: Use of PRO trial results to inform clinical practice. Please vote for one priority area for research from the following options:

0%

19%

44%

38%1. Research into

standardisation of tools.

2. Methods to facilitate clinician/patient understanding of PROs.

3. Methods to facilitate shared decision making in relation to PROs.

4. Optimal presentation of lay summaries of PRO results.

Session 5: Research Priorities: a clinical perspective

75

Session 5: Research Priorities: a clinical perspectivePlease vote for one priority area for research from the following options:

12%

53%

29%

6%1. What generic?2. Research into questionnaire

fatigue – layout media, length.3. PRO item bank approach –

validated set of items to be selected by focus group.

4. Culturally appropriate items.

76

report prepared by dr melanie calvert & mr derek kyte, pro ... · report prepared by dr melanie...

Documents