patient reported measures - agency for clinical innovation · - patient reported measures (prms) -...
TRANSCRIPT
![Page 1: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/1.jpg)
Patient Reported Measures
Tuesday 11th August 2015
Mel Tinsley | Program Manager | ACI
Outcomes that matter to patients
![Page 2: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/2.jpg)
Welcome and House keeping
#patientreportedmeasures Photography and consent Fire exits Bathrooms Breaks… coffee Phones
![Page 3: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/3.jpg)
Background
• In 2014, the NSW Government announced the NSW Integrated Care Strategy.
• The strategy includes the development of key state-wide enablers: - IT systems - Monitoring and Evaluation - Patient Reported Measures (PRMs) - Risk Stratification
• The Agency for Clinical Innovation was tasked with leading the development of Patient Reported Measures and Risk Stratification.
![Page 4: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/4.jpg)
Patient Reported Measures
AIM: Reduce burden to clinicians and patients whilst
adding value by:
Enabling patients to provide direct,
timely feedback about their health related
outcomes and experiences to drive
improvement and integration of health
care across NSW.
![Page 5: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/5.jpg)
4 pilot sites across NSW Health
Mid North Coast Northern Sydney Western NSW Western Sydney
![Page 6: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/6.jpg)
PRM Pilot overview
• Developing a comprehensive approach based on local need and circumstances including target populations.
• 4 pilot sites will implement PRMs (commencing Oct 15) by:
• Agreeing on core and optional PRMs questions/ tools. • Developing systems and processes to capture PRMs
from the target population. • Developing systems and processes to use PRMs results
to improve individual patient care, inform local service improvement, and evaluate the NSW Integrated Care Strategy.
![Page 7: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/7.jpg)
Timeline
![Page 8: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/8.jpg)
What's the difference? Interim Solution vs. October Pilot
Interim Solution: Currently being used by 3 demonstrator sites (May 2015-
May 2016) Collecting PROM & PREM for evaluation purposes only No Direct, timely feedback, no shared care planning
October Pilot Used by 4 pilots Collecting PROM & Condition specific for real time feedback
and shared care planning/decision making PREM used for service improvement Will be used to evaluate the Integrated Care Strategy
![Page 9: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/9.jpg)
Literature Review
![Page 10: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/10.jpg)
Question sets (PROMS http://www.nihpromis.org/)
![Page 11: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/11.jpg)
Question Sets (PREMS) No. Core/
Optional
Domains Questions 1 2 3 4 5 6 Comment from v.6
1 Core Collection/ administration process Who completed this questionnaire for the patient? Self/ Patient Carer/ Family
member
Healthcare
provider/ Staff
Interpreter Other Added interpreter
Timeframe for questions is: "In the last 6 months"
2 Core Respect for patients’ values,
preferences & expressed needs
How often did healthcare providers ask about things in your life that affect
your health?
Always Often Sometimes Rarely Never
3 Core Information, communication &
education
Were you involved as much as you wanted to be in decisions about your
care and treatment?
Always Often Sometimes Rarely Never New response
option and
rephrasing of 4 Core Involvement of family & friends;
transition & continuity
Were your family or carer involved as much as you wanted them to be in
decisions about your care and treatment?
Always Often Sometimes Rarely Never Not
applicable
New response
option and
rephrasing of 5 Core Coordination & integration of care Was there an ongoing care plan to address your health condition? Yes, definitely Yes, somewhat No Unsure/ Unknown Wording changed
for clarification and
consistency.
Added unsure/
unknown as a
6 Core Coordination & integration of care;
transition & continuity
When healthcare providers planned care or treatment with you, did it
happen?
Always Often Sometimes Rarely Never Wording changed
to planned care
with you.
7 Core Information, communication &
education
Did a healthcare provider talk with you about how you were supposed to
take your medicine?
Yes, definitely Yes, somewhat No Not applicable - I did
not have any new
medicine
8 Core Coordination & integration of care;
transition & continuity
How often did your healthcare providers work well together as a team? Always Often Sometimes Rarely Never Wording and
response options
changed to reflect
team-based
approach to care.
9 Core Information, communication &
education
How often did healthcare providers give you easy to understand
information about your health care?
Always Often Sometimes Rarely Never
10 Core Overall care rating How easy was it for you to manage your health care? Very easy Easy Neither easy nor
difficult
Difficult Very difficult Removed
duplication of "last 6
months".
11 Core Overall self-efficacy rating How confident were you that you could manage your health condition on
a regular basis?
Completely
confident
Very confident Moderately
confident
Not very confident Not at all confident Added "health" to
wording for
clarification.
OPTIONAL QUESTIONS
12 Optional Access; transition & continuity Was there a place you USUALLY went to when you needed health care? Yes No Changed to past
tense.
13 Optional Access; transition & continuity If yes, was it: (select the main place only) GP / family
doctor clinic
Medical
specialist clinic
Hospital emergency
department
Hospital outpatient
clinic
Community
health centre
Other
(specify___)
14 Optional Information, communication &
education
How often did your healthcare providers seem to know the important
information about your medical history?
Always Often Sometimes Rarely Never
15 Optional Coordination & integration of care;
Information, communication &
education
How often did your healthcare providers help you identify the most
important things for you to do for your health condition?
Always Often Sometimes Rarely Never
16 Optional Access; Coordination & integration
of care
Did you need services at home to help you take care of your health? Yes No
17 Optional Access; Coordination & integration
of care
If yes, how often did your healthcare providers help you get these
services?
Always Often Sometimes Rarely Never
18 Optional Coordination & integration of care;
transition & continuity
Is there one healthcare provider who coordinates your healthcare? Yes, definitely Yes, somewhat No Don’t know
19 Optional Coordination & integration of care Was the care you received for your health care condition complete? Yes, definitely Yes, somewhat No Don’t know Working changed to
reflect response
options.
20 Optional Information, communication &
education
When you asked questions about your health care, did you get answers
you could understand?
Always Often Sometimes Rarely Never I didn't ask
questions
21 Optional Overall self-efficacy rating How important do you think it is to be able to manage your health
conditions at home?
Very Important Important Somewhat
important
Not very important Not at all
important
I didn't ask
questions
Added as an
optional question
based on WSyd
feedback
![Page 12: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/12.jpg)
Question sets (Condition specific - http://www.ichom.org/)
What clinicians’ value
What patient’s value
ICHOM Standard Set for Coronary Artery Disease (2013)
![Page 13: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/13.jpg)
Question sets
![Page 14: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/14.jpg)
Patient Reported Measures – mapping
Mapping PRM journey across 4 pilot sites Reducing the burden Identifying Clinical Workflows What are you already using? Opportunities for PROM/PREM What do the reports look like Identifying opportunities Linkages
![Page 15: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/15.jpg)
Patient Reported Measures - visits
Mapping out processes Talking through question sets Discussing engagement strategies/co-design Identifying barriers and enablers Clinicians Managers Consumers
![Page 16: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/16.jpg)
Scenario
Patient Identified
PRMs collected Consultation Action Information
sharing Ongoing
monitoring
Risk stratification
GP/ED presentation
Contact Centre
Self/ carer/ clinician
Clinic/ home
Paper/ tablet/ phone/ web
Readministration
Alerts
Review and discuss PRMs
results
Develop care plan
Self-management
support
Referrals
Shared care plan
PCEHR
![Page 17: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/17.jpg)
Education and training
General Practice Primary Health Care Acute settings Clinicians/Managers/Executive Consumers
What are Patient
Reported Measures?
![Page 18: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/18.jpg)
IT Solution
Linda Murray – eHealth NSW
![Page 19: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/19.jpg)
NSW Health Integrated Care Program IT System/ Infrastructure for PRMs
Linda Murray, Program Manager, Integrated Care
eHealth NSW
Tuesday 11 August 2015
Patient Reported Measures Workshop
![Page 20: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/20.jpg)
• Recruited additional resources to support each of the Demonstrator sites with their delivery plans, providing relevant guidance and assessing technology options and system integration capabilities
• Working with ACI to support their ICT progress in respect to the Patient Reported Measures (PRMs) and;
• Shared Care Planning, which has been highlighted as another high priority area for technology enablement across the state*
• Patient identification and tracking is also being scoped for technology solutions, similar at first to the process used by CDM (flags/alerts in EMR)
• Information on Unique Health Identifiers is available on the eHealth Intranet http://intranet.hss.health.nsw.gov.au/about/ehealth/innovation-strategy-and-architecture/integrated-care/update
eHealth NSW has been working to support the IC Strategy in a number of ways, across several statewide enabler areas
![Page 21: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/21.jpg)
The Priority Areas
1
2
3
4
5
1. Shared Care Planning Support
2. Patient Reported Measures (PRM) Pilot
3. Patient Register and Tracking for Analytics
4. Secure Messaging*
5. Risk Stratification*
1
2
3
4
5
eHealth Enabler
Priorities
*Solution scoping and requirements to be further developed & validated
5
The PRM tool is one of the five core building blocks of IC Architecture and is a key priority area for eHealth NSW (Standalone with integration with patient record / shared care plan tools planned for the future)
![Page 22: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/22.jpg)
May June July August September October November December January February March April May June
Patient Reported Measures
(Pilot)Go Live
Update evaluation tool output (requirements / vender select)
Confirm costs
& funding
Vendor Contracting
Evaluation of Shared Care POC initiatives and plan ongoing adoptions
Develop PRM Integration Plan (e.g. clinical repository)
Pilot & Support PRM POC Project (ACI & 4 sites)Set-Up Infrastructure Test
Vendor T&C Review
Application Configuration: Set-Up Questions & User Roles
Identify Installation & Hosting Requirements
Confirm Hosting Design & Costs
DESIGN BUILD TEST RUN
ACI & eHealthACI eHealthLegend
Application Management, User Support & TrainingMobilise PRM Demo Sites & ACI Support
Resourcs Develop Use Cases & Training Materials
Engage vendors, confirm business requirements and visit Case Sites
Pre-Go Live Engagement and Planning (Pilot Sites / Target Question Sets)
Install Solution
Infrastructure and hosting set-up is on track and will be completed in time for the scheduled October ‘Go Live’ date
eHealth NSW will be available to support the ACI and pilot LHDs with
implementation relating to infrastructure and hosting as well as helping to
assess the pilot tool for further rollout and/or enhancements and/or system
integration e.g. with electronic records or other IC tools (SCP)
![Page 23: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/23.jpg)
Activity Outputs Support ACI to define business requirements
and evaluate candidate solutions for survey tool
Provide advice on technical aspects of the solution and assessment of applications
PRM Survey evaluation tool developed to support requirements and preferred tool selection
Question logic to be used is based on the PROMIS survey questions
Preferred application vendor selected (REDCAP)
(Completed June 2015) Infrastructure Services engaged and detailed
design / architecture / costing underway for solution hosting
Confirmation of design, costs, and delivery timeline (Completed July 2015)
Privacy Security Assessment Framework (PSAF) documentation drafted for security and privacy review
PSAF Stage 0/1 review Plan for security testing/penetration testing (Completed July 2015)
• Confirm budget, install hosting infrastructure, install solution
• Hosted PRM solution (Scheduled completion end August 2015)
• Test solution – including security/penetration testing
• Security / User Acceptance Testing Complete (Scheduled completion end September 2015)
• Support ACI (if required) during roll-out ‘Go Live’ in October 2015
eHealth NSW is engaged in a number of activities to facilitate successful rollout of REDCap to pilot sites
![Page 24: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/24.jpg)
Consumers
PRM Registration and Consent
PRM Answer Display
Presentation of Individual, Longitudinal and Aggregated results (graphs, statistics etc)
Multi-Channel PRM Questionnaire
Complete PRM Questionnaire
SelfManagement
Inform Clinical Decision-Making
Discuss PRM Results
PRM Data
- Question Lists- Answer Storage
- Task / Reminder Storage- Registration & Consent- Patient Demographics
PRM Management Services
- Question Selection (Generic PROM, Disease-Specific PROM, Generic PREM)
- Task Management- Alert Management
- Answer View Management- Rules Engine (frequency, reminders)
Inform Questionnaire Generation Alerts
RemindersAlerts and
Tasks
Define Rules
Key channels from the proposed end-state PRMs solution will be available in the pilot
![Page 25: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/25.jpg)
The eHealth NSW IC Team is fully resourced and available to assist with any infrastructure / hosting queries or issues
![Page 26: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/26.jpg)
The PRM Workstream is a key priority and a number of resources have been assigned to facilitate delivery
![Page 27: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/27.jpg)
Contacts Dr Michael Costello | eHealth NSW Director Innovation, Strategy and Architecture Tower B, Level 13, Zenith Centre, 821 Pacific Highway, Chatswood NSW 2067 Tel +61 2 8644 2259 | email: [email protected] Linda Murray | eHealth NSW Program Manager, Integrated Care Tower B, Level 13, Zenith Centre, 821 Pacific Highway, Chatswood NSW 2067 Tel +61 2 8907 1429 | email: [email protected] Rebecca Lissing | eHealth NSW Business Analyst, Integrated Care, Innovation, Strategy and Architecture Tower B, Level 13, Zenith Centre, 821 Pacific Highway, Chatswood NSW 2067 Tel +61 8907 1428 | email: [email protected]
![Page 28: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/28.jpg)
REDCap- Research Electronic Data Capture
REDCap was initially developed and deployed by the
Office of Research Informatics at Vanderbilt University Nashville, TN, USA
User-friendly Secure web-based application Cost-effective 1536 active institutional partners 92 countries
![Page 29: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/29.jpg)
REDCap
Surveys are designed by REDCap administrators and are available to users via web link or on the REDCap mobile app
Easy to use surveys Data entered is transmitted securely to the central servers maintained by eHealth Data is analysed and reports sent to nominated recipients.
![Page 30: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/30.jpg)
REDCap
Confidential – Restricted access
Suitable for PROMs – Longitudinal surveys
In built scheduler In built capacity to create
reports Requires minimal input
from providers
![Page 31: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/31.jpg)
PRM – United States America
Clinician buy in from the start is essential It is a long road…. Don’t underestimate the patient voice Research and evidence to demonstrate EBP No one size fits all approach Infrastructure Adding value whilst decreasing burden: Clinician and Patients Look at any PRM carefully Don’t need ethics – it is usual care What’s in it for me? The PRM Journey Language Ownership at every level DON’T IMPACT CLINICAL WORKFLOW
![Page 32: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/32.jpg)
Level 4, Sage Building 67 Albert Avenue, Chatswood NSW 2067 PO Box 699 Chatswood NSW 2057
T + 61 2 9464 4666 F + 61 2 9464 4728
www.aci.health.nsw.gov.au
Melissa Tinsley Program Manager – PRM
(02) 9464 4649 [email protected]
![Page 33: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/33.jpg)
Open Source Platform for Measuring Health Outcomes and a Learning Based System
S e a n M a c k e y, M D , P h D I m m e d i a t e P a s t P r e s i d e n t , A m e r i c a n A c a d e m y o f P a i n M e d i c i n e R e d l i c h P r o f e s s o r C h i e f , D i v i s i o n o f P a i n M e d i c i n e D i r e c t o r, S t a n f o r d S y s t e m s N e u r o s c i e n c e a n d P a i n L a b S t a n f o r d U n i v e r s i t y h t t p : / / p a i n c e n t e r. s t a n f o r d . e d u h t t p : / / s n a p l . s t a n f o r d . e d u s m a c k e y @ s t a n f o r d . e d u
![Page 34: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/34.jpg)
Disclosures – Funding Sources NIH Pain Consortium – Partial funding for CHOIR HHSN 271201200728P
National Center of Complementary and Alternative Medicine P01 AT006651
National Institutes of Drug Abuse (NIDA) K24 DA029262 T32 DA035165 R01DA035484
Redlich Pain Research Endowment Dodie and John Rosekrans Pain Research Endowment No industry conflicts
![Page 35: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/35.jpg)
Sandra with Complex Regional Pain Syndrome (CRPS)
“It’s that feeling, if you’re digging through the bottom of a cooler, and you just get that burning sensation because your arm is so cold,”
![Page 36: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/36.jpg)
How do you know whether you have made Sandra better?
How do you know when a certain treatment is better than another for a specific patient?
![Page 37: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/37.jpg)
The Systems Challenge and Complexity of Pain
From Ming Kao
HypervigilanceAvoidance
Pain-Related
FearPain
CatastrophizingDisuse
DisabilityPain
Experience
Spontaneous Discharge
InjuryInflammation
Receptor Sensitization
Convergence
DescendingInhibition
Receptor field expansion
Neg. AffectivityThreatening Info
BiomechanicalAlterations
VascularInfectiousTraumaic
AutoimmuneMetabolicIatrogenic
AnxietyDepression
Augmented Muscle ActivityInc. Stiffness
Dec. Stiffness
Atrophy
Compromised Muscle Activity
Reflex Inhibition
Impaired Proprioception
Modified Motor Planning
Cortical Reorganization
Spasms & Spindle
Discharge
BiomedicalPsychological
Biomechanical
Insomnia Fatigue
SleepSleep
Disorders
Opioid Use
Opioid Tolerance
Aberrant Opioid Use
Opioid-Induced
Hyperalgesia
Adverse Childhood
Event
Substance Use
Disorder
PTSD
Genetic Load
Cognitive RepraisalExpressive Suppresion
Acceptance
Solicitous Response
Punishing Response
Mindfulness
Anxiety Sensitivity
Resilience
Suicide
Locus of Control
Coping Skills
Non-opioid
Rx
Iatrogenesis
Surgeries
Somatization
Psychosis
![Page 38: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/38.jpg)
The Problem with Randomized Controlled Trials and Chronic Pain
10% of persons with chronic pain qualify for clinical trials
90% do not qualify!!!
![Page 39: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/39.jpg)
Barriers Against Translation and High Throughput Technologies
![Page 40: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/40.jpg)
Experiences With Pain – Institute of Medicine Report
• Affects 100 Million Americans • Indirect/direct medical expenses
US $560-$630 Billion/year • Pain can become a disease IOM Finding 2-2. More consistent data on
pain are needed. Bottom line: We need better data!!
Institute of Medicine – Relieving Pain In America 2011
![Page 41: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/41.jpg)
Institute of Medicine: Need for Patient Registries and Learning Health Systems
“There is a need for greater development and use of patient outcome registries that can support point-of-care treatment decision making, as well as for aggregation of large numbers of patients to enable assessment of the safety and effectiveness of therapies. “We seek the development of a learning health system in which science, informatics, incentives and culture are aligned for continuous improvement and innovation – with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral byproduct of the delivery experience”
![Page 42: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/42.jpg)
Medical Practice Pressures on Measuring Outcomes
http://www.healthit.gov/providers-professionals/how-attain-meaningful-use
![Page 43: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/43.jpg)
• Open source, open standard, highly flexible, and free health and treatment registry and platform for a learning health system
• Point of care decision making • Comparative effectiveness research • Longitudinal outcomes research • Large simple trial designs • Software based decision making • Comprehensive assessment of: Physical, psychological and social functioning and
global health
![Page 44: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/44.jpg)
CHOIR: Data Capture System Features • Easy to use data entry for patients,
staff and clinicians • Clinical workflow support e.g. notify
patient of survey URL prior to clinical appointment
• Data import support for automated data entry (e.g. EMR) for medications and other treatments, medical conditions, costs, etc.
• Point of care reporting to support clinical decision making
![Page 45: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/45.jpg)
• Short version – you can’t do it. • Computational complexity of modern patient
reported outcomes (PROs) are beyond what can be provided by traditional EMR.
• With modern PROs, software decision support, and development of learning based systems, expect rapid algorithm development and frequent code revisions.
• Better to off-load modern PRO processing/infrastructure to a separate system
• Also allows rapid development and implementation of features
Why don’t you just use (or your favorite EMR)?
![Page 46: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/46.jpg)
Stanford Pain Management Center
• Interdisciplinary, coordinated comprehensive approach to pain management
• Use of validated outcomes assuring optimal patient assessment and care
• Over 14,000 patient visits (2014) • 21 Physician Pain Faculty All Boarded in Pain Medicine
• Anesthesiology • Internal Medicine • Physiatry • Neurology • Addiction Medicine
• 4 Pain Psychologists Faculty • Pain Psychology training program
• Physical therapy, Nutrition, Biofeedback, Acupuncture • Strong connection and translation with pain research
group
![Page 47: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/47.jpg)
Initial and Follow-Up Surveys
• Initial Survey – 22 min • Demographics • Prior Treatments, Pain Beliefs • Interactive Body Map • PROMIS 9 domain measures:
• Pain Intensity, Pain Behavior, Pain Interference, Fatigue, Physical Function, Depression, Anxiety, Sleep Disturbance, Sleep Related Impairment
• Pain Catastrophizing Questionnaire (PCS) • Follow up Survey – 9 min
• Interactive Body Map • PROMIS 9 domain measures as above • PCS
![Page 48: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/48.jpg)
Started in 2004 as a part of the NIH Roadmap
“Psychometrically validated, dynamic system to measure PROs efficiently in study participants with a wide range of chronic diseases and demographic characteristics.”
Based on Item-Response Theory Free
![Page 49: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/49.jpg)
Physical Health
Social Health
Mental Health
Global Health
![Page 50: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/50.jpg)
Pain is a product of the brain!
Pain
![Page 51: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/51.jpg)
Pain is a product of the brain!
Pain
Cognition: attention, distraction, hypervigilance, catastrophising, re-appraisal, hypnotic suggestion
Context: beliefs, expectations, placebo, motivation
Mood: depression, anxiety, catastrophising, emotional context
Individual differences: genetics, gender, history of injury, atrophy
![Page 52: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/52.jpg)
PROMIS: Why the need for improved patient reported outcome (PRO) measures? Classical Testing Theory limitations Validation needed when applying to new patient population Missing data problem Ceiling and floor effects High patient burden Many cost $ Planned benefits of the PROMIS measures Responsiveness to differences across treatment groups. Improved performance where floor and ceiling effects are expected. Potential to reduce patient burden and costs Normative comparisons Item-response theory (IRT) combined with computerized adaptive testing (CAT) Highly efficient compared to classic testing theory (CTT)
![Page 53: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/53.jpg)
Computerized Adaptive Testing (CAT)
![Page 54: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/54.jpg)
Computerized Adaptive Testing (CAT) Applied to Health Care
Suppose our subject score is 73 on a 1 to 100 scale of Physical Function
0 100 73
No
t 1
N
ot
2
No
t 3
N
ot
4
Yes
73
N
ot
74
No
t 9
9
Classic Testing Theory
IRT with CAT
0 100 73
vs. 5
0?
vs. 7
0?
vs. 7
5?
vs. 7
3?
vs. 6
3?
Found!
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Physical Functioning Item Bank
Item1
Item2
Item3
Item4
Item5
Item6
Item7
Item8
Item9
Itemn
100500
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
![Page 55: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/55.jpg)
The PROMIS Metric: Comparing to the US Population
T Score Mean = 50 SD = 10 Referenced to the US general population
https://dhs.stanford.edu/spatial-humanities/comparing-population-density-and-wikipedia-density-on-gis-day/
![Page 56: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/56.jpg)
Current Status
• Stanford Pain Management Center in 2012 • Integrated smoothly using web and iPad/Android
devices • Staff involvement early. • Minimal complaints from staff or patients to date! • Approximately 8,000 unique patients and 40,000
longitudinal data assessments • Implemented local CAT engine – CHOIR CAT • Changed the culture and how we care for patients
– Like Sandra!
![Page 57: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/57.jpg)
CHOIR CAT: API-based CAT algorithms Incurs Significant Latency
Unavoidable Network Latency in Use of NW API
SystemSubject NW API
Start InitiatePrepare
assessment
Save assessment
Present Item
DoItem
ChooseItem
QueryItem
Present Item
Respond to Item
RecordItem
StoppingCondition
Record Item
Calculate Score
Return Score
Yes
RecordScore
StopAssessmentComplete
NoNetwork Latency
• Round-trip HTTPS traffic from Palo Alto, California to Chicago, Illinois incurs significant latency and degraded patient experience
• Exacerbated by the nature of CAT which requires re-calculation with each response
• Motivation for development of our own CAT algorithm • Multi-objective framework
that incorporates additional features from educational testing
![Page 58: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/58.jpg)
CHOIR CAT: Reduction in Patient Survey Burden
![Page 59: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/59.jpg)
![Page 60: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/60.jpg)
CHOIR as a Platform in Pain Research and Clinical Practice
•Generation of preliminary data •Dynamic studies of pain •Systems studies of pain •Comparative effectiveness •Large simple trials/pragmatic trials
•Recording individual patient data •Dynamic treatment of pain •Systems treatment of pain •Learning based systems of pain
Clinical Practice
Research
![Page 61: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/61.jpg)
Desipramine Low-dose Naltrexone
Fatigue
CHOIR: Using Dynamic Outcomes to Inform Care for Sandra
No change in Function!
![Page 62: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/62.jpg)
Health Education
Desipramine Low-dose Naltrexone
Fatigue
CHOIR: Using Dynamic Outcomes to Inform Care for Sandra
![Page 63: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/63.jpg)
CHOIR Computer-Assisted Documentation
![Page 64: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/64.jpg)
CHOIR Primary Care: Pain Management Toolkit
![Page 65: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/65.jpg)
CHOIR Primary Care: Opioid Taper Tool
![Page 66: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/66.jpg)
CHOIR Primary Care: Opioid Taper Tool
![Page 67: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/67.jpg)
Research-grade clinical data generation + Decision support
• Bedside research-grade diagnostic criteria that generates customized documentations for • Subjective findings • Objective findings • Impression and plan • Patient instructions
• Covering the most common and the most
complex diagnoses in Chronic Pain • CRPS • Fibromyalgia • Hypermobility • SEID (previously known as CFS) • 46 peripheral nerve entrapments • 431 non-headache, non-spine regional pain
syndromes
![Page 68: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/68.jpg)
Using CHOIR to generate “research quality” clinical data
69 Abernethy et al, Health Services Research, 2008
Data with equal quality of a clinical trial
Quality data can be used for: • Clinical trials
• Pilot data • Large simple
trial designs • Clinical decision
making • Improving quality
care and monitoring • Comparative
effectiveness research
![Page 69: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/69.jpg)
CHOIR’s ability to rapidly iterate and improve EMR code review/release process • A necessity given the wide
ranging critical roles of EMRs • Typically measured in months
• IOM released report on April 28, 2015 • Mackey sent to group at 3:14pm • CHOIR Provider new Core Metrics user
interface live on April 30, 2015 at 7:31am
• CHOIR: from concept to live beta in 40 hours
![Page 70: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/70.jpg)
Clinically-integrated registry enables studies of larger orders of magnitude
• 573 interventional studies for back pain in ClinicalTrials.Gov
• Subjects • Mean 651 (SD 10,447) • Median 100 (IQR 196)
• CHOIR • 5,306 subjects • 13,157 complete
longitudinal data points
![Page 71: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/71.jpg)
Patient characteristics: Stanford Pain Management Center
![Page 72: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/72.jpg)
Patient Characterization
N = 2,082 Patients
• Foot Pain: 30.5% of patients
• Foot pain not identified by clinic physicians or IOM report as a primary location of pain
![Page 73: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/73.jpg)
Low Back Pain Pelvic Pain Fibromyalgia CRPS Headache/Migraine
CHOIR and Overlapping Pain Conditions
![Page 74: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/74.jpg)
Dynamics of Patients’ Response to Treatment
Non-Responders
Responders
![Page 75: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/75.jpg)
CHOIR ProviderClinical Operations & Decision Support
-.406
Pain
Fatigue
Depression
Physical Function
Sleep Disturbance
.201
.253
.354
-.193
-.108
-.342
.345
.406
-.309
Pain
Fatigue
Depression
Pain Interference
Sleep Disturbance
.201
.253
.354
.360
.164
.315
.345
.406
-.309
Fatigue is: • Common in chronic pain • Understudied as a target of intervention • Likely a confluence of physical and psychological factors • A significant barrier to physical functioning, likely mediating effects of pain on physical dysfunction
Sturgeon, Darnall, Kao, & Mackey (Under review).
Physical and Psychological Correlates of Fatigue and Physical Function: A CHOIR Study
![Page 76: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/76.jpg)
CHOIR ProviderClinical Operations & Decision Support
77
-7
-6
-5
-4
-3
-2
-1
0
Effect of FCR – CHOIR Outcomes
![Page 77: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/77.jpg)
Big Outcomes Data Enables True Systems Biology Approach to Medicine
Gene Express
ion
Patient Reporte
d Outcom
es
![Page 78: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/78.jpg)
Resting State
Volumetry
ActivityTractography
Labs & Diagnostics
Documentation
Psychometrics
Whole Genome Seq
SNPtyping
Immunomics
Microbiomics
Metabolomics
Transcriptomics
Quantified Self
Experiential Sampling
Information
1,000,000,000
Billions
1,000,000
Millions
1,000
Thousands
1
Individuals
BedsideBytes BenchBedside
![Page 79: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/79.jpg)
Press-Ganey Patient Satisfaction and the Challenges of Chronic Pain
![Page 80: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/80.jpg)
“The Best Defense is a Good Offense”…
![Page 81: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/81.jpg)
System To Enhance Patient eXperience (STEPx)
Accecss & Scheduling
Admission & Check-in
Physician & Provider
Care Coordination
Outcomes
Nursing & AssistantNurses
Referring
Provider
Patient
Scheduler
Front Office
BackOffice
An unmet need • Comprehensive capture of patient
experience touchpoints • Concise item stems • Actionable results • Integrated into CHOIR • Open source and free
Covers, and extends, all the domains of existing patient satisfaction surveys, including:
• Press Ganey • Hospital Consumer Assessment of
Healthcare Providers and Systems (HCAHPS)
• TOPS
![Page 82: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/82.jpg)
83
Patients with Severe Pain, High Depression or High Pain Catastrophizing: Much Less Likely to Recommend
![Page 83: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/83.jpg)
84
Pain Center Optimization of Patient Experience (PCOPE)
Pain Clinic Optimization of Patient Experience (PCOPE)Phase 1: Model Building and Algorithm Development
CHOIRSTRIDE STEPXPain COPE NLP Engine
Ph
ase
1
Medical Co-morbidities
Psychosocial
Pain Characteristics
Overall
Admissions
Provider
Nursing
Care Coordination
Scheduling
Outcomes
Free-Text
Machine Learning
Algorithm Yelp Data
NLP Feature Extraction
Pain Clinic Optimization of Patient Experience (PCOPE). Phase 2: Risk Stratification and Targeted Care Coordination
CHOIR& STRIDE
SPEQPain COPE Clinic StaffCare Coordinator Social Work ProvidersPatient Patient Satisfaction
Fo
ste
rin
g L
HC
7 d
ay
s P
re-V
isit
Vis
itP
hase
Medical Co-morbidities
Psychosocial
Pain Characteristics
Overall
Admissions
Provider
Nursing
Care Coordination
Scheduling
Outcomes
Schedules NPV
Fills out CHOIR
Case Analysis
Machine Learning
Algorithm
High RiskPoor
Satisfaction
Notify Staff and Providers Identify
Resource Needs and Solutions
Pre-Visit Medical,
Psychology, PT
Needs
Preparations for Visit
Pre-visit Discussion
with Referring ProviderCare
CoordinationCare
Coordination
Optimize Scheduling
Predict
Presents for Visit
Coordinated Interdsiciplinary
Pain Management
Case Follow-Up
Identify Solutions for
Barriers
Post-Visit Assessmentof Low Risk
Patients
Care Coordination
Follow-Up & Issues
Plan Adaptation
Discussion with PCP and
Referring ProviderFeedback
Optimize scheduling
Poor Satisfaction
Real-time Patient
Experience Monitoring
High RiskPoor
SatisfactionYesNo
Assess
Care Coordination
Knowledge Base
Monthly Patient Experience M&M Conference
Continuous Improvement
Awarded Stanford Health Care Innovation Challenge grant (2015)
![Page 84: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/84.jpg)
Perioperative CHOIR Preoperative emotional distress highly predictive of: • Poorer surgical outcomes • Increased pain intensity and duration • Increased use and duration of postop
opioids • Increased length of stay
![Page 85: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/85.jpg)
The Present and Future
• Specific practice modules (e.g. Pain Psychology, Pain Physical Therapy) and conditions (e.g. Headache, Orofacial Pain, Pelvic Pain). Hospital module to connect clinic to hospitalized patients
• Clinical note generation (e.g. CHOIR Provider), condition “calculators” • Software based decision support
• Opioid risk stratification and documentation • Recommended referrals to psychology, physical therapy, etc • Targeted education and therapies
• Results integrated with EPIC/EMR • Currently under development by Univ Florida
• Patient Reported Outcomes On Demand • Means of dynamically ordering patient reported outcomes as needed
• Aggregation of data for national registry
![Page 86: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/86.jpg)
• Consolidation of data across sites within specialty
• Advanced CAT features • Genetics: Stanford GenePool • Activity monitors
• FitBit, Jawbone, Basis • Mobile device integration for daily experiential
sampling. Passive metadata. • Quantitative sensory testing • Automatic natural language processing • Consumer-facing front-end • Patient Satisfaction
The Present and Future
![Page 87: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/87.jpg)
Collaborations
• Expanding to multiple clinics/sites and expanding outside of pain • Stanford Children’s Hospital: Pediatric Pain
Center • Other Academic Pain Centers • Ortho CHOIR, GI CHOIR, etc
• Currently providing source code with minimal licensing restrictions.
• Crowdsourcing approach to software development • Donations accepted • Change health care nationally!!!
![Page 88: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/88.jpg)
![Page 89: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/89.jpg)
Trainee opportunities for CHOIR • Provides opportunities for trainees (e.g.
clinical fellows, postdocs, medical students, etc) to easily devise and conduct a research study.
• NIH NIDA T32 “Interdisciplinary Research Training in Pain and Substance Use Disorders” PD: Mackey
• More info at http://snapl.Stanford.edu
![Page 90: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/90.jpg)
Thanks to and Collaborators • NIH Pain Consortium • Redlich Pain Research Endowment • Stanford Center for Clinical Informatics
• Michael Halaas • Susan Weber • Garrick Olson • Teresa Pacht
• Stanford Systems Neuroscience and Pain Lab (SNAPL) • Northwestern/PROMIS
• Karon Cook, PhD • University of Florida
• Chris Harle, PhD • Medical College of Wisconsin
• Rob Hurley, MD, PhD • University of Pittsburgh
• Ajay Wasan, MD • Stanford-Packard Pediatric Pain Center
• Elliot Krane, MD, Sam Huestis, PhD • NIH Pain Consortium
• Richard Denisco, MD (NIDA) • David Thomas, PhD • Linda Porter, PhD
![Page 91: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/91.jpg)
![Page 92: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/92.jpg)
Psychometric properties of Stanford Pain Management Center population
Research
![Page 93: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/93.jpg)
Institute of Medicine: Need for Patient Registries and Learning Health Systems
![Page 94: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/94.jpg)
PROMIS: Why the need for improved patient reported outcome (PRO) measures? Classical Testing Theory limitations Validation needed when applying to new patient population Missing data problem Ceiling and floor effects High patient burden Many cost $ Planned benefits of the PROMIS measures Responsiveness to differences across treatment groups. Improved performance where floor and ceiling effects are expected. Potential to reduce patient burden and costs Normative comparisons Item-response theory (IRT) combined with computerized adaptive testing (CAT) Highly efficient compared to classic testing theory (CTT)
![Page 95: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/95.jpg)
CHOIR: Framework • Flexible platform to allow any survey or clinical
conditions • Headache CHOIR, Ortho CHOIR, GI CHOIR,
Primary Care CHOIR • Industry standard tools (Oracle database, Java,
Google Web Toolkit, open source libraries). No commercial vocabulary or proprietary libraries.
• NIH PROMIS computer adaptive testing (CAT) surveys and legacy instruments.
• Funded by a partnership with NIH (NIH Pain Consortium) and Stanford (Redlich Pain Endowment, Stanford Center for Clinical Informatics). • Public-private partnership
![Page 96: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/96.jpg)
• Short version – you can’t do it. • Computational complexity and demands of
modern patient reported outcomes (PROs) are beyond what can be provided by traditional EMR.
• Significant, unpredictable, uneven demand on computational space and time from modern PROs
• Modern PROs, software decision support, and development of learning based systems - need rapid algorithm development and frequent code revisions.
• Off-load modern PRO processing/infrastructure to a separate system
Why don’t you just use
(or your favorite EMR)?
![Page 97: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/97.jpg)
Taking the Systems Biology view of Pain Research
![Page 98: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/98.jpg)
104 Systems View of Chronic Pain Research
Ange
r D
epre
ssio
n An
xiet
y
Slee
p D
ist.
Slee
p Im
pair.
Fatig
ue
Pai
n B
ehav
ior
Pai
n In
terfe
re.
Phys
ical
Fn
![Page 99: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/99.jpg)
PRM Pilot sites overview
NSLHD WSLHD MNCLHD WNSWLHD
![Page 100: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/100.jpg)
Lyn Olivetti Service Development Manager Chronic & Complex Medicine, Rehabilitation & Aged Care, NSLHD
Christine Collins
Osteoporosis Re-fracture Prevention Project Officer, NSLHD
Musculoskeletal Coordinator, Sydney North Health Network
Musculoskeletal Integrated Care Initiative
Not present: Cynthia Stanton
Sydney North Health Network Dr Rodger Laurent
Chair, Back and Neck Pain Advisory Group, NSLHD and HOD, Rheumatology, RNSH
Matt Williams
Musculoskeletal Coordinator, RNSH (Osteoarthritis Chronic Care Program)
Prof Lyn March
Chair Osteoporosis Re-fracture Prevention Advisory Group, NSLHD
Prof David Hunter
Chair Osteoarthritis Chronic Care Program Advisory Group NSLHD
![Page 101: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/101.jpg)
Musculoskeletal problems typically account for around 18% of GP-Patient encounters 1
Common musculoskeletal conditions (osteoporosis, hip/knee osteoarthritis and acute low back pain) are associated with significant morbidity and a reduction in QOL
These patients rarely present to a care provider with a single condition, are a often likely to have multiple comorbidities
These patients face challenges in accessing appropriate intervention, and receiving coordinated care across primary and secondary health sectors
1. General Practice Activity in Australia 2012-2013. BEACH, General Practice Series No 33. Family Medicine Research Centre, University of Sydney
The Problem
![Page 102: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/102.jpg)
Despite evidence regarding best practice clinical management for each condition there is inconsistency in clinical management of these conditions in both primary and secondary care facilities
– Inappropriate and over-utilisation of some services (e.g. surgical procedures for joint replacement, clinical management of fractures, bed days for back pain).
– Patients may experience a fragmented journey; care may differ depending on their provider, rather than their clinical presentation
The Problem
![Page 103: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/103.jpg)
OBJECTIVE:
To improve timely access to appropriate evidence based care for residents of NSLHD with MSK conditions (OA, OP and back pain)
STRATEGY:
Concurrent initiatives in both primary care and hospital setting to improve management of MSK conditions (OA, OP and back pain)
The Challenge
![Page 104: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/104.jpg)
OACCP
Back/Neck Pain
ORP
Hospital Clinics
Community Services and
Programs
GP + Practice Nurse
![Page 105: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/105.jpg)
Expected Outcomes
Osteoporosis ORP
Osteoarthritis OACCP
Back Pain
Reduce incidence of subsequent fracture
Optimise pain management and
function
Optimise pain management and
function Optimise pre-surgical
condition (and surgical outcomes) and/or delay the need for
surgery Optimise management of co-morbidities
Improve quality of life Improve capacity for long-term self-management
Musculoskeletal Initiative in Primary Health
![Page 106: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/106.jpg)
Osteoarthritis Chronic Care Program (OACCP)
Eligibility = people on the surgical waiting list for hip or knee joint replacement surgery at a NSLHD Hospital
Program consists of 9 – 12 months of contact with a multidisciplinary team to address identified needs
Patient reported outcome measures collected at baseline and at each review
![Page 107: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/107.jpg)
Comparison of outcome measures at 12 and 26 & 52 week reviews index joint = knee (2012-
13)
Indicator
Initial
mean (SD)
n=401
Wk 12 change
in mean from
initial visit,
n=269
Wk 26 change
in mean from
initial visit,
n=196
Wk 52 change in
mean from
initial visit,
n=104
Body mass index* 30.8 (6.70) -0.4, p<0.001 -0.7, p<0.001
-0.9, p<0.001
Waist circumference
(cm)
100.9
(14.29) -1.4, p<0.001 -1.9, p<0.001
-2.7, p<0.001
Pain VAS (0-10) 4.5 (2.27) -1.1, p<0.001 -1.1, p<0.001
-1.5, p<0.001
Timed up and go (sec) 11.0 (4.90) -1.4, p<0.001 -1.8, p<0.001
-2.5, p<0.001
KOOS ADL (0- 100)#
48.0
(20.45) 6.8, p<0.001 8.4, p<0.001
9.0, p<0.001
KOOS Pain (0- 100)#
46.0
(19.42) 6.3, p<0.001 8.5, p<0.001
10.1, p<0.001
Six Minute Walk Test
(m)
409.1
(117.45) 35.2, p<0.001. 43.6, p<0.001
55.3, p<0.001
![Page 108: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/108.jpg)
Key performance indicators(Updated Oct 2014)
Number of people seen = 740
Removed from waiting list = 55 (16% from WL)
Accelerated to surgery = 30
![Page 109: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/109.jpg)
RNSH / Ryde
OACCP Service
![Page 110: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/110.jpg)
Northern Sydney region, General Practice
“OACCP”
![Page 111: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/111.jpg)
Enablers
ICT buy-in
– Local eMR2 developments
– Data linkage, analytics and visualisation tools
Advisory Group buy-in
Evaluation Plan linked to ACI MS Network and PHI project
![Page 112: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/112.jpg)
Patient Reported Measures Update – August 2015
Western Sydney Integrated Care Program
ACI Workshop l Aug 2015
Presenter:
Donna Sedgman, Primary Care Integration Manager
Western Sydney Integrated Care Demonstrator
![Page 113: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/113.jpg)
Model of Care Re-cap
Western Sydney Integrated Care Program
Key components of model: Disease cohorts – COPD, heart failure, coronary artery disease, diabetes Patients + GP practices registered for ICP Dynamic Shared Care Planning / PCMH Care Facilitation GP Support Line Specialty Rapid Access Specialty Stabilisation Service Building capacity in Primary Care/General Practice
ACI Workshop l Aug 2015
![Page 114: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/114.jpg)
Western Sydney Integrated Care Program
ACI Workshop l Aug 2015
Local teams – representative across primary + secondary
settings
WSICP PRM Task Group
Name Position
Donna Sedgman Integration Manager, Primary Care
Rowena Bellwood Integration Manager, Specialty Services
Marina Fulcher Primary Care Manager,
WSPHN (WentWest)
Mary Roberts Respiratory CNC, RACS WMH
Heena Puri Care Facilitator
Janis Patterson GP Liaison Nurse (Primary Care)
Ken + Marj Freeman Consumer Advisory
![Page 115: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/115.jpg)
Western Sydney Integrated Care Project
Change management strategies aimed at:
Identifying + building concept into appropriate positions
Assessment of specific barriers
Appropriateness in process design - partnering
Supported training (ACI/AFS)+ assistance with change management
Implementation Strategies + linkage to change
ACI Workshop l Aug 2015
![Page 116: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/116.jpg)
Western Sydney Integrated Care Project
ACI Workshop l Aug 2015
Mar2015
Initial Interim PRM Approach drafted + presented to IC Clinician Working Group.
May 2015
Interim PRM WSICP question set finalised.
Questions loaded onto tablets.
Initial training on tablets by AFS/ACI.
June 2015
PRM Focus Groups conducted with CDMP patients (ACI).
ACI PRM Presentation at GP Information Evening.
DRAFT Guideline developed for interim PRM strategy across primary + secondary settings.
July 2015
Consultation with primary care stakeholders.
DRAFT flowchart developed.
WS Integrated Care Program GO Live (27th July).
EOI by PCMH Practices to trial tablet devices + administer PRM’s.
Aug 2015 onwards
Agreement across Primary + Secondary Settings on interim PRM processes.
Follow-up training on tablet devices.
Tablet kiosks to be installed in identified PCMH practices.
Distribution of devices to key positions – Care Facilitators, CNC’s, CNS’s
Pilot PRM’s + Refine Approach.
PRM Timeline
![Page 117: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/117.jpg)
Western Sydney Integrated Care Project
Within IC Program:
Data Manager = ICP Business Analyst/Data Manager
Matching of patient identifiers with PRM identifiers and PC/GP identifiers (IHI)
Building and extraction of PRM reports
Reports to be tabled at ICP Operational Meeting (MDT) + ICP Consumer Advisory
Data for local evaluation (UWS)
Within LHD/across Primary Care + General practice settings:
Agreed Reports tabled at ICP Steering Committee in project/pilot phase
PRM KPI’s reported up to the MoH
Data Management + Reporting
ACI Workshop l Aug 2015
![Page 118: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/118.jpg)
Planned implementation
Western Sydney Integrated Care Project
Key principles: Patients to be provided written information about PRM’s – keep simple
Consent obtained (either as Registered ICP patient or Connecting Care)
Patient understanding of ‘who/what/where/why’ re: PRM data
PRM’s administered to both ICP registered patients and ICP candidates
PRM’s administered only once patient stabilised (in hospital setting)
PRM’s to be factored into business processes (clinic appointment or GP care planning appointment)
ACI Workshop l Aug 2015
![Page 119: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/119.jpg)
ACI Workshop l Aug 2015
2. WSICP flagged patients in general practice identified for PRM
Questionnaire by Practice Staff and/or Care Facilitator
1. PENCAT data extraction in general practice
and patient selection criteria in RASS/Inpatient
setting identifying patients suitability for WS Integrated Care Program (WSICP)
2.WSICP Flagged Patients in RASS/Inpatient setting identified for PRM
Questionnaire by Integrated Care (IC) CNC/CNS
3. Patient provided information(WSICP Patient brochure or
verbally) about the questionnaire by Care Facilitator, Practice Staff or
IC CNC/CNS
5. GP practices and Integrated Care Facilitators, CNC and CNS s with e-tablet availability to
provide instruction to patient in use of the e-tablet to complete the PRM questionnaire
Does the patient meet any of the following:1. Non-english speaking?2. Needs assistance with reading + responding to questionnaire?3. An enrolled Connecting Care patient (Non-IC Patient)
4. Refer patient to Connecting Care for scheduling + electronic adminis tration of
PRM +/- Telephone Healthcare Interpreter Service**
YES
4. Care Facilitator to decide on timing + administration options (as per below) for PRM adminis tration + schedule patient
(Baseline + then 6-monthly PRM s)
NO
PRM scheduled by Care Facilitator in conjunction with GP consultation
or Shared Care Planning appointment
PRM scheduled by Care Facilitator as part of follow-up/monitoring
post-RASS visit (phone call or visit to patient in
home)
PRM scheduled by Care Facilitator as part of follow-up appointment in Stabilisation Clinic or by Integrated
Care CNC/CNS as an inpatient consultation
OPTIONS
Home
GP Practice5. If GP practice/RASS Clinic/Inpatient ward
does not have an electronic PRM kiosk, paper-based PRM questionnaire to be
handed to patient with Reply-paid envelope
(Patient s individual health identifier [IHI] to be populated in the identifying field on
form)
Hospital
1
2
3
6. Patient may opt to fill out PRM questionnaire whils t at GP Practice/RASS
Clinic/Inpatient ward or at home
7. Patient or Practice staff, Care Facilitator or IC CNC/CNS return PRM questionnaire
via reply-paid mailed envelope
9. Once questionnaire electronically
submitted, PRM data uploaded via wireless
into a cloud-based central repository
6. Patient completes PRM Questionnaire with administering s taff ensuring that
questionnaire has patients IHI entered into the identifying field and that the survey is
submitted at the conclusion
e-Tablet/Kiosk Paper-based
8. Paper-based PRM Questionnaire received by Integrated Care Office and
entered by Integrated Care Administrative Staff into an e-tablet
WSICP Patient Reported Measures (PRM s) Process Flowchart -DRAFT ONLY
10. Individual patient data (identified by IHI) downloaded by Integrated Care Data
Manager weekly and formatted into a useful PRM Report
11. Individual Patient PRM Report to be scanned and formatted into a pdf document,
matched to the patient s IHI and uploaded into Linked eHR by IC Data Manager/Care
Facilitator
12. PRM Report readily identified in Linked eHR at time of next clinical review of the patient
either in the General Practice or RASS/Inpatient setting
PRM informs clinical
decision-making
13. Patient-Reported information confirmed with patient and utilised by
clinicians in the shared care and treatment planning process
Patient consented + enrolled in
WSICP
Patient refusal of PRM questionnaire*
Notes:*Where patient refuses PRM:Paper-based form - Document on form + return Questionnaire to Integrated Care Office.E-tablet - Indicate refusal in patient declined field on electronic survey and submit questionnaire as per outlined process.
**Referral to Connecting Care Service for PRM administration:Practice Staff, IC CNC/CNS to ring Connecting Care Contact Centre 1800 113 644 + provide patient details – Name, DOB, IHI and contact details.
![Page 120: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/120.jpg)
Patient Reported Measures in the Nambucca Shire
![Page 121: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/121.jpg)
Nambucca Valley Integrated Care (NVIC)
Patient Reported Measures (PRMs) in the Nambucca Shire will become the focus of a working group in the Nambucca Valley Integrated Care (NVIC) Initiative
![Page 122: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/122.jpg)
Why Nambucca Shire?
• Identified as focus for MNCLHD Integrated Care proposal – • Significant ageing population • Significant Aboriginal population • Socio-economic disadvantage and • Limited, fragmented services.
• Strong stakeholder support for an innovative initiative to better integrate patient focused care for residents of the Nambucca Shire
![Page 123: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/123.jpg)
What do we want to do?
• Work toward a holistic patient focus
• Replace a complex model of care with a collaborative of local health and community care providers centered on the patient and their needs
• Improve communication between all stakeholders
• Reduce avoidable hospital admissions and ED presentations, and
• Use health resources more effectively
![Page 124: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/124.jpg)
PRM Collected
Nambucca resident
attends ED triage 4/5
Assertive follow up
Risk stratification
Low risk of escalating
chronic disease
Patient Story collation and
analysis
Patient Story
Collection
High risk of escalating
chronic disease
PRN data
Develop and implement solutions
NVIC Patient Feedback Working Group
Care provided
![Page 125: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/125.jpg)
Patient Reported Measures Plan
Assertive follow-up of residents of Nambucca Shire presenting at Emergency Departments with triage 4/5
Assess if suitable for PRMs project
Work with care provider for follow-up PRMs
Work toward integrating with existing chronic disease follow-up programs
Use responses to inform the activities of the NVIC Initiative Patient Feedback Work Group.
![Page 126: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/126.jpg)
Next Steps Small trial of assertive follow-up
Convene NVIC Patient Feedback Working Group
PRMs education for MNCLHD including healthcare providers and Working Group
Work toward integrating with existing chronic and complex disease management programs
Use PRM data to improve and integrate health care through the NVIC Patient Feedback Working Group.
![Page 127: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/127.jpg)
Western NSW
Integrated Care Strategy
To transform existing services into an integrated Western NSW system of care that is tailored to the needs of our rural and remote communities,
improves access to care and health outcomes, focuses on closing the Aboriginal health gap
Better Care…………Better Health…………Better Value
![Page 128: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/128.jpg)
Local Demonstrator Sites Demonstrating redesign of delivery models across general practice, LHD primary & community health services, Aboriginal Medical Services, local rural hospital/MPS and specialist outpatient services
• 12,000 pop • GP led integrated
models of care focussed on managing chronic disease and mental health conditions with associated complex health needs.
• 4,900 pop • GP led
multidisciplinary model of care for older people with chronic and complex conditions
• Focus on at risk population
• 2,400 pop • HealthOne facility • GP led
multidisciplinary model of care for chronic and complex conditions
• New business funding models
• 8,777 pop • ACCHO • GP led
multidisciplinary model of care for chronic and complex conditions
• Minimise impact of diabetes on Aboriginal people in Dubbo
• ‘In reach’ care coordination via AHW
• Nurse Practitioner
![Page 129: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/129.jpg)
Local demonstrator sites – work to date • Development of GP-led multidisciplinary models of care to manage high risk
patients and those with chronic and complex illnesses (including diabetes, CVD, COPD, mental illness)
• Risk stratification at the demonstrator sites using agreed clinical markers to identify those patients who would benefit most from integrating their care.
• Implementing integrated models of care, tailored to the local needs of the population, with a focus on engaging all relevant care providers within a locality
• Recruited Care Navigators based in primary care to support and manage patients enrolled in the local strategies.
• 320 people enrolled in the local integrated care projects across the 5 localities.
• Implemented the electronic shared care planning tool cdmNet in the Local Demonstrator Sites to facilitate the sharing of clinical information between relevant care providers. Note: cdmNet proving challenging……..
• Commenced collection of patient reported measures for the enrolled cohort to determine evaluate the impact of integration on their experience/health.
![Page 130: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/130.jpg)
Patient Reported Measures - WNSW
• HREC approval received.
• Collecting all interim solution questions plus Aboriginality and a unique identifier.
• Commenced collection of PRMs via tablets in June 2015 with 74 PRMs collected to date (320 enrolled across the 5 local demonstrator sites).
• All PRMs collected in Primary Care as close to enrolment as possible and annually thereafter.
• Central point allocated responsibility for managing PRMs
![Page 131: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/131.jpg)
• More often than not patients need assistance completing the PRMs via the tablet.
• Primary care is reporting the process can be tedious and time consuming for staff, patients and carers.
• Feedback from patients/carers indicates that there are too many questions with patients often expressing confusion around some questions i.e. “How is that question different to the last question?”
• Need to determine how to provide timely relevant feedback from the PRMs to clinicians.
• Watching with interest in regards to future direction of PRMs.
Learnings
![Page 132: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/132.jpg)
Workshopping next steps - pilots
Pilot sites Prof Mackey Mel Tinsley
![Page 133: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/133.jpg)
Patient Reported Measures: Outcomes that
Matter to Patients
Question Sets and Processes
Professor Madeleine King Quality of Life Office University of Sydney
Agency for Clinical Innovation PRM Workshop
11 August 2015
![Page 134: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/134.jpg)
What are we trying to do?
• Integrated Health aims – integrated care, holistic, patient-centred, shared decision-
making – Use PRMs to inform patient-centred management of individual
patients (and facilitate shared decision-making?) – Improve patients’ self-reported experience of care, and of their
own general health and quality of life
• 4 very different settings • Evaluate improvement within and across all 4 sites • Measures – need to be fit for purpose
– 1. condition specific, for use in clinic – 2. global, PROMIS 10, for overall evaluation – Outcomes that matter to patients
![Page 135: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/135.jpg)
PRMs and Processes • Study design
• Evaluation – of what?
• PROMs? PREMs? Shared decision-making? Is care becoming more integrated? Is care becoming more patient-centred? Are patients more satisfied with their care? Do they feel they have gained in health and improved quality of life?
• A series of evaluations – need to articulate specific question for each one
• PROMs in patient management
– Stanford experience – NSW – sites select measures to match population – Need to fit with current clinic processes
• PRMs for global evaluation – PREMS – patient experience of integrated care – PROMIS-10 – global health outcomes and quality of life
![Page 136: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/136.jpg)
“Study design” = “Evaluation”?
• Overarching evaluation of …. << WHAT specifically? >>
– WHO to include
– WHEN to assess
– WHAT to measure
![Page 137: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/137.jpg)
Site Level: Evaluation plan and sample size Post-intervention Target Sample Size n=400
Pre and Post
Patient Level: who to include and when to assess - Eligibility criteria, exclusion criteria - What is “baseline”? - Chronic condition, ongoing management - Recruit at first (?) chronic care visit - Assessments:
- PROMIS10 - PREMs - Condition specific
![Page 138: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/138.jpg)
PRM Assessment Schedule
![Page 139: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/139.jpg)
Disease Symptoms - pain, fatigue - others differ with
site and stage of disease
Functioning & well-being
• Physical
• Role
• Emotional
• Social
• Cognitive
• Sexual, body image
• Spirituality
• Financial
Treatment - benefits & toxicity - differ with treatment
and dose
Proximal Effects
Causal variables
He
alth
pro
ble
m(s
) +
tre
atm
ent(
s)
Global QOL,
Wellbeing
&
Happiness
Other aspects of life
Finances, family,
job, safety,
security
How do health problems & treatments affect a person?
Process of care
• Satisfaction with health care/providers/information
• Preferences
• Inconvenience
Distal Effects Indicator variables
![Page 140: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/140.jpg)
PREMs
• Common set of ~13 items
• Being fine-tuned
![Page 141: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/141.jpg)
PROM for overall evaluation
• PROMIS-10
![Page 142: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/142.jpg)
![Page 143: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/143.jpg)
![Page 144: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/144.jpg)
Health problems caused by many health conditions
Pain_7 Fatigue_8
Health_1 PHYSICAL Physical Health_3 Physical Function_6 MENTAL Mental Health_4 incl
mood, cognitive fn Emotional problems_10
incl anx, depr, irrit SOCIAL Satisfaction w social
activ & relationships_5
Social roles & activities_9
Some treatments can cause
Fatigue_8
He
alth
pro
ble
m(s
) +
tre
atm
ent(
s)
QOL_2
Other aspects of life
Finances, family,
job, safety,
security
How do health problems & treatments affect a person? Where the 10 PROMIS Global items fit
![Page 145: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/145.jpg)
Scoring the PROMIS Global-10 2 summary scores:
• Global Physical Health = Physical health + Physical activities + pain + fatigue
• Global Mental Health = QOL + mental health + satisfaction w/ social activities & relationships + emotional problems
• Convert to t-scores via conversion tables, using US norms (mean 50, SD 10)
• May be useful as outcome measures in aggregate data for evaluation of pilot programs
![Page 146: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/146.jpg)
Individual items
• can be examined separately to provide specific information about perceptions of physical function, pain, fatigue, emotional distress, social health and general perceptions of health.
• ~ 2 mins to complete
• Separate items may be useful in managing individual patients – Identify patients at-risk of health-related problems
– Open the conversation about how to address problems
![Page 147: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/147.jpg)
Quality of Life
0
20
40
60
80
100
Symptoms
0 2 4 6 8 10
Data show that symptom burden does affect QOL QLQ-C30 data from n=346 mixed cancer pts
QLQ-QL= 87.1 - 10.5*sympt_index + 0.33*sympt_index^2.
163
QLQ-C30 Global QOL score
Higher is better
Number of Symptoms – higher is worse
![Page 148: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/148.jpg)
PROs for use in clinical practice
• Why bother?
• How?
– Measures
– Processes
![Page 149: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/149.jpg)
• 5 systematic reviews to date – Valderas et al. (2008)
– Frost et al. (2007; cancer-specific)
– Marshall et al. (2006)
– Espallargues et al. (2000)
– Greenhalgh et al. (1999)
Evidence about the effect if PRO assessment in clinical practice
![Page 150: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/150.jpg)
PRO assessment in clinical practice
What the evidence says: Improves: – communication + – awareness + Equivocal: – patient management +/- – satisfaction +/- – PROs / HRQoL +/-
Does NOT increase consultation time
![Page 151: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/151.jpg)
Site / disease-specific PROMS What to measure? Which PRM to use?
• Include health-related issues that
– matter to the target patient group
– health care providers are responsible for and can do something about
![Page 152: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/152.jpg)
Many PRMs to choose from!
H E L P !!! Too
many!
![Page 153: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/153.jpg)
PRM toolkit
Choose the best tool for the job
at hand
![Page 154: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/154.jpg)
What is the “best” tool?
• Includes health-related issues that – matter to the target patient group – health care providers are responsible for and can do
something about
• Feasible to complete
– time, cognitive & physical effort
• Psychometrically sound – Valid, reliable, responsive to change – Precise, accurate
![Page 155: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/155.jpg)
Patient group for Integrated Care
• Heterogeneous
• Various health conditions and treatments
• Many with co-morbidities
Each site:
• What are health care providers responsible for?
• What can do something about?
![Page 156: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/156.jpg)
Mode of Administration
• The PLACE of completion + HOW the patient completes the questionnaire
• Electronic is encouraged
– Established feasibility in oncology clinics – The only way to get real-time data capture and feedback for the
PROMs used in clinical management
• Paper & pen / phone as backup – OK for the PREMs and PROMIS-10
• Will this cause bias?
Rutherford, Costa, Mercieca-Bebber, Rice, Gabb, King. Does mode of administration of patient-reported outcome (PRO) does not cause: a meta-analysis Quality of Life Research – Accepted 10 Aug 2015
![Page 157: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/157.jpg)
MOA review methods
included 58 studies : • English language; • compared ≥2 administration modes of a
standardised PRO instrument; • adult samples; • reported sufficient data for inclusion in a
quantitative analysis.
48 of these studies provided sufficient data for meta-analysis
![Page 158: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/158.jpg)
MOA review RESULTS
• Forest plot of paper- and electronic-self-completion
• Plotted means on the right-hand side of the plot indicate paper/pen scores were lower than electronic (lower scores = better outcome).
• There was no systematic difference due to mode
• Effect size [95% CI] = 0.01 [-0.04, 0.05]
![Page 159: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/159.jpg)
MOA review conclusions
• Robust evidence that there is no bias between:
– paper vs electronic self-complete
– self-complete vs phone/interviewer-assisted MOA
• Self-complete paper and electronic MOA can be used interchangeably in clinic and home settings.
• Self- and assisted-completion produce equivalent scores overall, although heterogeneity may be induced by setting.
• These results support the use of mixed MOAs within a study, which may be a useful strategy for reducing missing PRO data.
![Page 160: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/160.jpg)
Patient Reported Measures: Outcomes that Matter to Patients
Question Sets and Processes - Discussion
![Page 161: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/161.jpg)
Pressing issues
• Questions sets for use in clinic – Each site, each health condition
– Share across sites providing IC for the same conditions?
• Developing IT systems to capture these, convert to report for clinician to review and use in clinical consultation – Problem areas?
– What to do about them? Evidence-based? Or keep it simpler at this stage?
![Page 162: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/162.jpg)
Patient Reported Measures: Outcomes that Matter to Patients
Question Sets and Processes - Discussion
![Page 163: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/163.jpg)
Next steps
![Page 164: Patient Reported Measures - Agency for Clinical Innovation · - Patient Reported Measures (PRMs) - Risk Stratification • The Agency for Clinical Innovation was tasked with leading](https://reader031.vdocument.in/reader031/viewer/2022041321/5e16d6275a91c21cbe0a67e4/html5/thumbnails/164.jpg)
Level 4, Sage Building 67 Albert Avenue, Chatswood NSW 2067 PO Box 699 Chatswood NSW 2057
T + 61 2 9464 4666 F + 61 2 9464 4728
www.aci.health.nsw.gov.au
Melissa Tinsley Program Manager – PRM
(02) 9464 4649 [email protected]