cadth 2015 d7 presentation 2015 14 apr15

The Campaign for McMaster UniversityThe Campaign for McMaster University

Strengthening Public and Patient Engagement in Health Technology Assessment:

A Framework for Involving Patients and the Public in Health Quality Ontario's Evidence-Review Process

Julia Abelson, PhD

Frank Wagner, MA

Deirdre DeJean, PhD

Stephen Petersen, MSc

Sarah Boesveld (PhD cand.)

Francois-Pierre Gauvin, PhD

Mita Giacomini, PhD

John Lavis, PhD

CADTH Conference 2015

Saskatoon, SK

April 14th, 2015

Acknowledgements

Funders Canadian Institutes of Health Research

Evidence-Informed Healthcare Renewal Policy Analysis grant

Government of Ontario Ministry of Health and Long-Term Care Health System Research Fund

grant “Harnessing Evidence and Values for Health System Excellence”

Collaborators Health Quality Ontario Public Engagement Subcommittee of the Ontario Health

Technology Advisory Committee (OHTAC) McMaster Health Forum


Background & Objectives

Increasing attention to involving patients and members of the public in HTA agencies worldwide

Efforts to produce conceptually robust, evidence-informed frameworks to guide public and patient engagement (PPE) in HTA have been rare

In particular, lack of attention to matching PPE goals and approaches to specific HTA stages


Background & Objectives (2) Context:

Ontario’s evidence-review process for health technologies (Health Quality Ontario (HQO), Ontario Health Technology Advisory Committee (OHTAC)) – non-drug HTA

Traditional stakeholder consultation & passive communication and consultation with patients and members of the public (interest in doing more)

Collaboration between researchers and HQO-OHTAC Public Engagement Subcommittee (Jan 2013 – July 2014)

Parallel activities (review of Decision Determinants framework; strategic emphasis on PPE by HQO leadership)

Description and results of framework development process


Key Elements of the Framework

Articulate the underlying principles, values and goals for public and patient engagement in HTA

Establish a common language to support public and patient engagement efforts

Describe a flexible menu of approaches that can be used depending on the goal and phase of the evidence-review process

Identify and develop measures of engagement and its outcomes that can inform adjustments over time

Final Report (released April 8, 2015)

Subcommittee membership

OHTAC members External members HQO/EDSFrank Wagner (Chair) Julia Abelson, Deirdre DeJean &

Sarah Boesveld (McMaster/CIHR research grant)

Shamara Baidoobonso (EDS)

Pat Campbell Renata Axler (University of Toronto) Gaylene Pron (EDS)

Tony Easty Sally Bean (Sunnybrook Health Sciences)

Stephen Petersen (EDS)

Murray Krahn Tara Gomes (Ontario Drug Policy Research Network)

Heather Thomson (Strategic Partnerships)

Dorothy Pringle Sine Mackinnon (Patient & Public Engagement)

Shirlee Sharkey

Charles Wright

Methods

Committee deliberations informed by: website review of 53 HTA organizations across

34 countries stated goals, approaches and experiences with PPE

in HTA

review of published literature (PPE in HTA) reviews, empirical & conceptual studies (1990-2013)

expert consultations (international & local) stakeholder dialogue (May 2014)


Conceptual Foundation for the Framework

Goals for Public & Patient Engagement

Democratic accountability & legitimacy

To achieve more informed, transparent, accountable and legitimate decisions about health technologies

ScientificTo promote a more robust and comprehensive approach to HTA that incorporates social and ethical values, as well as patients’ problems, lived experiences, outcomes and preferences

Instrumental To enhance the quality of outputs across all stages of the HTA process

DevelopmentalTo increase public understanding of health technologies and HTA, and strengthening the public’s and patients’ competence and capacity to contribute to health technology policy issues

Adapted from Abelson et al. 2007. Bringing the public into health technology assessment and coverage policy decisions. Health Policy; 82(1):37-50.

Who to engage with?

Patients, families, caregivers/providers Individuals with experiential knowledge about living with an illness or condition who can provide valuable perspectives about the intended or unintended consequences of current or future health technologies

Broader publics Individuals who can contribute broad social values regarding the efficiency or fairness of a technology but who may not have specific experience with a particular technology, disease or condition

Stakeholder groups Groups with organized interests in the technology, program or service, including its funding and delivery arrangements (e.g., advocacy groups, industry, provider organizations)

Facey et al. 2010. Patients’ perspectives in health technology assessment: A route to robust evidence and fair deliberation. IJTAHC; 26(3):334-40.

Interact

PARTICIPATION

Collaborate

Involve

CONSULTATION

Gather Information

Inform

COMMUNICATION

Disseminate

How to engage?

Three categories of PE activity:

Adapted from: Rowe & Frewer (2005). A Typology of public engagement mechanisms. Science, Technology & Human Values; 30(2):251-90.

HQO-OHTAC evidence-review process(at the beginning of the committee’s work)

13

• HQO drafts brief overview of intervention

• OHTAC determines which interventions proceed to a full review

• HQO reviews evidence in consultation with:

- Clinical experts and expert

panels - Scientific partners - Industry - Government

• OHTAC drafts recommendations based on evidence-based review

• Draft review and recommendations are posted on the HQO/OHTAC websites for public and professional comment

• HQO reviews public and professional comment feedback

• OHTAC modifies recommendations as required

• HQO evidence-based review and OHTAC recommendations published on website and announced in e-bulletin

• OHTAC may request a field evaluation to assess effectiveness and cost-effectiveness of an intervention in the Ontario context

1 SCOPING 7 FIELD

EVALUATION6 POST REVIEW

ANDRECOMMENDATION

5 ASSESSMENT

OF COMMENTS

4 PROFESSIONAL

AND PUBLIC CONSULTATION

3 DRAFT OHTAC

RECOMMENDATIONS

2 EVIDENCE-

BASED ANALYSIS

Stakeholderconsultation

Passive consultation

Passive communication

Preliminary Framework for Strengthening Public & Patient Engagement

14

• HQO drafts brief overview of intervention

• OHTAC determines which interventions proceed to a full review

• HQO reviews evidence in consultation with:

- Clinical experts and expert

panels - Scientific partners - Industry - Government

• OHTAC drafts recommendations based on evidence-based review

• Draft review and recommendations are posted on the HQO/OHTAC websites for public and professional comment

• HQO reviews public and professional comment feedback

• OHTAC modifies recommendations as required

• HQO evidence-based review and OHTAC recommendations published on website and announced in e-bulletin

• OHTAC may request a field evaluation to assess effectiveness and cost-effectiveness of an intervention in the Ontario context

1 SCOPING 7 FIELD

EVALUATION6 POST REVIEW

ANDRECOMMENDATION

5 ASSESSMENT

OF COMMENTS

4 PROFESSIONAL

AND PUBLIC CONSULTATION

3 DRAFT OHTAC

RECOMMENDATIONS

2 EVIDENCE-

BASED ANALYSIS

Rationale for PE: Whom to engage with: How to engage:- improved knowledge & awareness - patients, families, providers - communication- higher quality decisions - broader publics - consultation- increased transparency, legitimacy - stakeholder groups - participation

Sources: published research, international practice, expert consultations

Evidence Review (Key messages)

15

Key messages from website scan and literature synthesis

General one-third to two-thirds of organizations reporting some PPE activity (increasing) very small number have comprehensive approach to PPE (all HTA stages)

Goals for PPE improving the quality of assessments and achieving accountable, legitimate

processes are regularly cited goals no explicit links drawn between the goals for PPE and specific HTA stage

PPE (Who, When & How) increasing efforts to engage the public and patients in early stages (e.g., topic

selection, prioritization and scoping stages); common PPE approaches: communication (web & some social media),

consultation (through solicitation of input, feedback on draft documents, focus groups) and direct participation (through membership on committees)

Key messages from review of stage-specific PPE in HTA (2)

PPE (Who, When & How) emphasis on incorporating patient values/perspectives through in-depth consultation, qualitative research & patient representation on advisory committees broader publics tend to be involved at board level or when broader societal considerations are explicitly sought

Evaluation minimal evaluation of the effectiveness or impacts of specific PE mechanisms primarily descriptive and case study driven; small number of rich ethnographic studies

Feasibility/Implementation considerations numerous cultural and organizational challenges major enabler is organizational commitment and resources (e.g., time and expertise) common challenges: how to seek ‘representative’ perspectives; how to balance broad public interest with narrower interests of organization reps

McMaster Health Forum Evidence Brief

FRAMEWORK & RECOMMENDATIONS

Key Elements of the Framework

Articulate the underlying principles, values and goals for public and patient engagement in HTA

Establish a common language to support public and patient engagement efforts

Describe a flexible menu of approaches that can be used depending on the goal and phase of the evidence-review process

Identify and develop measures of engagement and its outcomes that can inform adjustments over time

Building a Common Language for Public & Patient Engagement

MENU OF APPROACHES & TOOLS

Newly configured process to include new stage for PPE

Summary of Approaches

Topic Selection: Who and How

0 TOPIC SELECTION & PRIORITIZATION

Groups Mechanism Tools• General/interested public • Communication Website description of

topic selection process

• General public• Patients

• Patient organizations

• Consultation • Polling/surveys Horizon scanning (media

analysis, focus groups, surveys)

Stakeholder meetings/Delphi process (e.g., 2-3 years)

• Patients • Participation • Advisory committee representation

Scoping: Who and How

1 SCOPING (may include expert panel input)

Groups Mechanism Tools• Patient organizations • Consultation Invited submissions

(targeted and web)

• Patients • Consultation Social media analysis Interviews/focus groups

• Participation Committee representation (e.g., expert panel)

Evidence-Based Analysis: Who and How

2 EVIDENCE-BASED ANALYSIS (may include expert panel input & qualitative meta synthesis)

Groups Mechanism Tools• Patient organizations • Consultation Invited submissions

• Patients • Consultation Surveys Social media analysis Qualitative

research/synthesis

• Participation Committee representation (e.g., expert panel)

Draft OHTAC Recommendations: Who and How

3 DRAFT OHTAC RECOMMENDATIONS (process begins in Evidence Review stage)

Groups Mechanism Tools• Patient

organizations/patients• Consultation Expert panel consultation

with priority populations

• Broader publics/patients • Participation Explicit discussion of societal/patient values (OHTAC meeting)

Professional and Public Consultation: Who and How

4 PROFESSIONAL AND PUBLIC CONSULTATION

Groups Mechanism Tools• Unsolicited public• Patients/patient

organizations

• Communication • HQO website posting Social media profile

(selected reports) Lay review of summary

• Patient organizations • Consultation • Targeted mailings Solicited reviews of draft

report/face-to-face meetings with priority groups

• Patients• Broader publics

• Participation • Committee representation (e.g., expert panel)

• Citizen panels (for high priority topics triggered by DD framework)

Assessment of Comments: Who and How

5 ASSESSMENT OF COMMENTS

Groups Mechanism Tools• Authors of public

comment submissions• Communication Web posting of comments

and actions taken

• Authors of public comment submissions

• Consultation Face-to-face meetings to discuss and address concerns (selected)

Post Review and Recommendation: Who and How

6 POST REVIEW AND RECOMMENDATION

Groups Mechanism Tools• Broader publics

• Patients

• Communication • Multimedia dissemination of OHTAC report

Lay review of plain language summary

• Patient organizations Targeted dissemination to high priority groups

Report Recommendations

1. Increase the transparency of the evidence review process to facilitate a fuller understanding among interested patients, broader publics, and stakeholder groups of how topics are selected and referred to OHTAC

2. Increase the relevance and responsiveness of HQO-OHTAC’s work by undertaking a range of horizon-scanning and consultation activities to prospectively identify potential review topics and emerging issues of concern to patients and priority populations


3. Embed public and patient engagement in the scoping of all review topics with the following exceptions:

the topic under review concerns technologies with no direct patient interface

The focus of the review is exclusively on the technical aspects of the technology

4. Draw on a range of sources for incorporating societal and patient values into the evidence-based analysis stage. e.g.,

patient members on expert panels, consultations with individual patients and/or patient groups synthesis of primary qualitative research studies


5. Identify dedicated agenda time at monthly meetings for the explicit consideration of societal and patient values relevant to all evidence reviews

6. Enhance public and patient consultation process at the post-appraisal stage to develop increased awareness of its work and to encourage broader input on its draft recommendations

Measurement & Evaluation

Weak evidence base for PPE in HTA (and related fields)

Framework provides opportunity to embed evaluation into the initiation of new PPE activities informed by clearly articulated goals, populations and methods

Formative evaluation metrics (examples) are we meeting our goals for transparency and increased

awareness of HQO and OHTAC’s work? are patient and public members contributing meaningfully

and at relevant stages of the HTA process? (how) are the outputs from PPE activities informing OHTAC

& HQO decisions?

Implementation

Early stages Report adopted by HQO Board Presentations to HQO leadership and staff Alignment with HQO’s strategic focus on patient, caregiver

and public engagement

Need to prioritize and support areas of focus Stage by stage approach or early vs. later? Building awareness of Ontario’s evidence review process

among interested constituencies Ensure that training and supports for staff, OHTAC

members, patient and citizens are in place from the outset Ensure that evaluation metrics are developed early on

Future Work

How and when should PPE be triggered within the HQO-OHTAC process?

Which PPE approaches for which types of technologies, conditions or populations?

Link to Final Report

http://www.hqontario.ca/evidence/publications-and-ohtac-recommendations/other-reports/special-reports


cadth 2015 d7 presentation 2015 14 apr15

Health & Medicine