validation and re finementofapaininformation model from …...validation and...
TRANSCRIPT
Validation and Refinement of a Pain InformationModel from EHR Flowsheet DataBonnie L. Westra1 Steven G. Johnson1 Samira Ali2 Karen M. Bavuso3 Christopher A. Cruz4
Sarah Collins5 Meg Furukawa6 Mary L. Hook7 Anne LaFlamme8 Kay Lytle9 Lisiane Pruinelli1
Tari Rajchel10 Theresa (Tess) Settergren11 Kathryn F. Westman12 Luann Whittenburg13
1School of Nursing, University of Minnesota, Minneapolis,Minnesota, United States
2Adjunct Faculty, School of Nursing, Grand Canyon University,Phoenix, Arizona, United States
3Department of Clinical Informatics, Partners Healthcare System,Wellesley, Massachusetts, United States
4Department of Care Delivery BioClinical Informatics, KaiserPermanente, Oakland, California, United States
5Department of Medicine, Brigham and Women’s Hospital, PartnersHealthcare System, Harvard Medical School, Boston,Massachusetts, United States
6Department of Information Services & Solutions, University ofCalifornia Los Angeles Health, Los Angeles, California, United States
7Center for Nursing Practice and Research, Aurora Health Care,Milwaukee, Wisconsin, United States
8Department of Nursing, Fairview Health Services, Minneapolis,Minnesota, United States
9Department of Nursing & Duke Health Technology Solutions, DukeUniversity Health System, Durham, North Carolina, United States
10Department of Information Technology, North Memorial MedicalCenter, Robbinsdale, Minnesota, United States
11Department of Enterprise Information Services, Cedars-SinaiHealth System, Los Angeles, California, United States
12Department of Nursing, Allina Health, Minneapolis, Minnesota,United States
13 Informatics Consultant, Bumrungrad International, HealthInformatics/Health System Architecture, Bangkok, Thailand
Appl Clin Inform 2018;9:185–198.
Address for correspondence Bonnie L. Westra, PhD, RN, FAAN, FACMI,School of Nursing, University of Minnesota, 308 Harvard Street SE,Minneapolis, MN 55455, United States (e-mail: [email protected]).
Keywords
► pain management► nursing informatics► electronic health
records and systems► knowledge modeling
and representation► secondary use► efficiency
improvement
Abstract Background Secondary use of electronic health record (EHR) data can reduce costs ofresearch and quality reporting. However, EHR data must be consistent within andacross organizations. Flowsheet data provide a rich source of interprofessional data andrepresents a high volume of documentation; however, content is not standardized.Health care organizations design and implement customized content for different careareas creating duplicative data that is noncomparable. In a prior study, 10 informationmodels (IMs) were derived from an EHR that included 2.4 million patients. There was aneed to evaluate the generalizability of the models across organizations. The pain IMwas selected for evaluation and refinement because pain is a commonly occurringproblem associated with high costs for pain management.Objective The purpose of our study was to validate and further refine a pain IM fromEHR flowsheet data that standardizes pain concepts, definitions, and associated valuesets for assessments, goals, interventions, and outcomes.Methods A retrospective observational study was conducted using an iterativeconsensus-based approach to map, analyze, and evaluate data from 10 organizations.Results The aggregated metadata from the EHRs of 8 large health care organizationsand the design build in 2 additional organizations represented flowsheet data from 6.6million patients, 27 million encounters, and 683 million observations. The final pain IM
receivedSeptember 14, 2017accepted after revisionJanuary 15, 2018
Copyright © 2018 Schattauer DOI https://doi.org/10.1055/s-0038-1636508.ISSN 1869-0327.
Research Article 185
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Background and Significance
Thewidespread implementation of electronic health records(EHRs) provides health care organizations the opportunity tocapture, use, and share data for evaluation, benchmarking,quality improvement, and research to improve the effective-ness, efficiency, and outcomes of patient care. Secondary useand sharing, however, requires data to be represented usingrecognized terminologies and descriptors that are consis-tent, understood, and effectively formatted for comparison.These requirements suggest that concepts must be standar-dized, formally modeled, and mapped into the EHR foroptimal use. An “information model” (IM) is an organizedstructure to represent knowledge about a clinical conditionor concept including data elements, their relationships, andthe data standards that are independent of implementationin EHRs.1 IMs can be mapped to EHR data to identifysemantic similarities2 and, more importantly, to enableresearchers to understand and normalize differences whenthey occur to improve data sharing.
The American Recovery and Reinvestment Act of 2009(ARRA) provided incentives for creating a national healthcare technology infrastructure and accelerating the adoptionand meaningful use of enterprise-wide, vendor-based EHRsystems with a key focus on physician-based data capture.Vendors provide generic content and guide organizations inusing consensus-based approaches to configure the bulk oftheir system to meet the clinical requirements of the orga-nization. Much of the documentation, however, is capturedin flowsheet format, using nonstandardized semistructureddata in a matrix format for patient assessments, goals,problems, interventions, and outcomes of care. Limitedresources and rapid deployment timelines provide littletime for organizations to identify and adopt standardizedterminologies and use IMs to design flowsheets for futuredata sharing. Further, informaticians are required to choosefrom multiple terminologies3,4 with limited reference stan-dards to guide flowsheet builds. These conditions alloworganizations to continue to design and implement custo-mized content, creating flowsheet rows (unique identifica-tions [IDs]) for different care areas, e.g., intensive care units,emergency departments, or medical–surgical units,5 withvaried choice options for documentation in flowsheet rowswith the same or similar names. As organizations movebeyond deployment, it is time to reevaluate the extensiveclinical information captured in flowsheets and considerhow to optimize and manage data better in the future.Furthermore, analyzing existing content may inform thedevelopment of standardized terminologies and IMs for
representing the essential nursing and interprofessionalassessments and interventions to achieve best patientoutcomes.
Nursing informatics leaders have successfully utilizedmethods for developing generalizable domain-specific IMsbased on documentation artifacts captured in the EHR.6,7
These investigators used consensus-based, data-drivenmethods for analyzing EHR data elements embedded bylarge multisite health care systems to develop a skin inspec-tion and pressure ulcer IM for standardizing and codingconcepts. Both groups identified that existing EHR systemscontain heterogeneous data with limited interoperability.They recommended ongoing efforts to create common IMsbased on best evidence, clinical expertise, and standardizedterminology beyond skin and pressure ulcer prevention.
Similarly, Westra et al5 utilized EHR data to develop aReference Information Model for the concept of pain. Theseresearchers selected the concept of pain because it is acommonly occurring problem, assessed and managed byall professional nurses and those who specialize in painmanagement.8 About 126 million, or more than half (56%)of adults in the United States, reported some level of painwithin a 3-month period.9 The estimated total nationaleconomic cost (direct and indirect) attributed to pain in2010 ranged from$560 to $635million.10 The concept of painremains an important aspect of hospital-based patient care,with additional regulatory focus on conducting pain assess-ments consistent with age, condition, and ability to under-stand, with an increased focus on patient involvement andthe effective use of nonpharmacological interventions.11 ThePain Reference IMwas developed by extracting themetadatafrom a clinical data repository (CDR) of one large integratedhealth care system representing over 2.4 million patients.The validation of the Pain Reference IM with other healthcare organizations was needed to increase the generaliz-ability of the model.
Objective
The purpose of our study was to validate and refine a PainReference IM fromEHR flowsheet data that standardizes painconcepts, definitions, and associated value sets for assess-ments, goals, interventions, and outcomes..
Methods
This study is a retrospective observational study using aniterative consensus-based approach to map, analyze, andevaluate EHR pain data across several organizations to
has 30 concepts, 4 panels (classes), and 396 value set items. Results are built on LogicalObservation Identifiers Names and Codes (LOINC) pain assessment terms and extendthe need for additional terms to support interoperability.Conclusion The resulting pain IM is a consensus model based on actual EHRdocumentation in the participating health systems. The IM captures the mostimportant concepts related to pain.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.186
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
validate and refine the Pain Reference IM.5 A conveniencesample of nursing informatics researchers who were activein the Nursing Knowledge Big Data Science Initiative12 wereinvited to represent their organization as participants in thestudy. One researcher was a pain management specialist;others consulted pain experts in their organizations or painresources (i.e., pain society guidelines or studies).Theresearchers represented medium to large size multihospitalhealth care systems with the majority of the group using theEpic EHR (see ►Table 1). Of the 10 participating organiza-tions, 8 shared metadata for mapping their EHR to the PainReference IM and 2 additional organizations shared how theybuilt their systems since theywere just going live. The sharedmetadata included all flowsheet data, but only general painconcepts from inpatient and outpatient settings includingthe emergency department were analyzed for this project.Specialized cardiac/chest pain assessments were excluded asthe focus was on general pain.
Organizations were asked to extract metadata about theflowsheet documentation contained in their EHR. The meta-data consisted of a unique identifier for each flowsheet data
row representing assessments, interventions, goals, or out-comes; the internal description and name used to display theflowsheet row; the name of the template (data entry screen)that was used to collect the data (andwhich grouping of painconcepts within the screen); the number of observations,encounters, and patients; and the date of first and last uses.This metadata represented actual documentation by clini-cians at each organization.►Fig. 1 shows an example of howthe pain flowsheet data are documented and the relationshipto the metadata. Within each organization, the EHR datawere transferred to their Clarity relational database. AStructured Query Language (SQL) script was developedthat allowed each of the organizations to extract the meta-data in exactly the same manner. Based on organizations’resources for data extraction, there was variation in the timeframes selected and specific hospitals or practices includedin metadata extractions.
Each organization next mapped their metadata to theconcepts in the Pain Reference IM. This was accomplishedusing software (FloMap) that allowed the metadata to beimported from each organization. FloMap was developed by
Table 1 Data source for validation of the pain information model
Organization Organization type Data source Number of beds Datesrepresented bydata
Allina Health Hospitals, medical centers,clinics, rehabilitation, hospice,homecare, retail pharmacy
13 hospitals, 90þclinics
1,775 2005–2016
Aurora Health Care Private, not-for-profit, integratedhealth care system w/ 16 hospi-tals including behavioral health,rehab, and hospice
1 hospital; qua-ternary medicalcenter
710 CY 2016
Bumrungrad Interna-tional Hospitala
Hospital 1 hospital 580 2013–2016
Cedars Sinai Academic medical center andhealth system
1 hospital, 40clinics
886 2009–2016
Duke University HealthSystem
Health system 3 hospitals, 400clinics
1,512 2012–2016
Fairview Health Services Hospitals, academic health cen-ter, clinics, senior housing, retailpharmacy
7 hospitals,40 þ clinics
2,530 2011–2016
Kaiser Permanente Health system, hospitals, aca-demic hospitals (graduate medi-cal education), clinics,ambulatory care centers, acuterehab, inpatient psychiatry
Northern Califor-nia region only: 21hospitals, 233medical officebuildings, 203ambulatory carecenters
3,922 2005–2016
North Memorial MedicalCenter
Hospitals, specialty and primarycare clinics, home care, medicaltransportation
2 hospitals 355 2016
Partners Healthcarea Integrated health system 9 hospitals, manyclinics
2,825 2016
UCLA Health Health system 4 hospitals 861 2013–2016
Abbreviations: CY, calendar year; EHR, electronic health record; UCLA, University of California, Los Angeles.aOrganizations that provided information about their EHR build only.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 187
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
one of the researchers (S.J.) and is not currently publicallyavailable. A researcher from each organization used FloMapto search for pain-related flowsheet rows in their organiza-tion’s metadata and map them to the appropriate concept inthe Pain Reference IM. FloMap allows sophisticated search-ing using Boolean logic to make it easy to find local data thatmatched the pain concepts. Flowsheet rows related to exclu-sion criteria (i.e., cardiac/chest pain) or rows that had lessthan 10 observations were not mapped. ►Fig. 2A demon-strates the mapping process. This example shows howFloMap finds all flowsheet rows that contain “pain” andone of the additional terms. Users can see the value setswhich help to determine if the flowsheet rows represent asimilar concept. They then select the flowsheet rows thatmap to the concept and click on “Add items to concept.”After
these flowsheet rows are added to the concept, they aredisplayed and included in reports for comparison(see ►Fig. 2B).
After the local flowsheet dataweremapped to concepts inthe Pain Reference IM, the group met biweekly to evaluatethe concept mappings across all of the organizations. AFloMap-generated report was used by the group to makedecisions about which concepts to keep, combine, remove, oradd additional concepts. Concepts were retained when allresearchers agreed that the concepts represented essentialquestions for the majority of patients. One researcher was apain management specialist; others consulted pain expertsin their organizations or pain resources (i.e., pain societyguidelines or studies). There was discussion that there aresome differences in use based on the population such as age
Fig. 1 Example of documenting pain on flowsheets. The orange template is a screen view that shows the Adult Assessment which includesmultiple groups of related questions shown in light green on the left. The group called “Pain” shows examples of specific questions/flowsheetmeasures displayed to the clinician. The clinician selects answers from the value sets with actual documentation shown in blue fordocumentation that occurred at specific dates/times.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.188
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
(pediatric vs. adult), type of unit (e.g., intensive care unit vs. amedical–surgical unit), or the patient’s capability (e.g., abil-ity to verbalize pain). The group developed a definition anddiscussed the use of the concepts to help determine decisionsabout the concept and associated value sets. A value setrepresents the list of all possible values (answers) associatedwith a specific concept (question). Value set response countsvaried by concept and ranged from a few (3–4) to many(> 100). For example, the concept of “Body Site” had 507different response choices across the 10 organizations. Someresponse values were not useful (e.g., misspelled or incom-plete words like “a,” “ac,” “acu,” “acut” for a value choice of“acute”) or clearly inappropriate such as “…,” “/,” “ þ þ þ,”etc. After the inappropriate responseswere removed, severalconcepts with multiple diverse value sets remained forevaluation.
To support the group in evaluating diverse responsevalues, a FloMap “survey” feature was developed. The Flo-Map survey aggregated all of the response values for aparticular concept into a single list while retaining detailsabout which organizations used each choice. Researchersfrom each organization received a survey via email with 1 to2 concepts and a list of response values set choices for eachconcept from all of the organizations. The email contained asecure link to the survey. Survey participants were asked toselect values that were considered generalizable acrossorganizations even if their organization did not currentlyinclude that value. The results were then discussed at thebiweekly calls. Value set items that received 50% or more ofthe votes were automatically retained in the Pain ReferenceIM. Those items that received less than 30% were automati-
cally removed from themodel and those between 30 and 50%were discussed by the group. The group decided on thesethresholds to reduce the amount of discussion needed toreach consensus. The results were compared with painconcepts included in Logical Observation Identifiers Namesand Codes (LOINC). Some concepts were then renamed tomatch those in the Nursing Physiologic Assessment Panel inLOINC or other LOINC locations.
Results
The aggregate metadata from 8 large health care organiza-tions that contributed metadata represented flowsheet datafrom 6 million patients, 27 million encounters, and 683million observations. A high level diagram of the resultingpain IM concepts is shown in ►Fig. 3; the red font indicatesnew panels and concepts added. ►Table 2 shows a compar-ison of the original Pain Reference IM and final consensusregarding which concepts were retained with or withoutrevision, removed, or added. The new model consists of 30concepts grouped into 4 panels with 396 value set items. Thein-depth analysis revealed that 24 concepts were retained, 6added, and 59 removed compared with the concepts in theoriginal Pain Reference IM. Since some scales require copy-right permission to use, we retained only the scale score foreach of the pain scales for consistency. The SupplementalDigital Content (SCD) 1 includes a detailed list of the retainedconcepts, definitions, and their value sets. ►Table 3 lists theinformation for each concept that was part of the finalmodel: the minimum (Min) and maximum (Max) numberof flowsheet rows per organization mapped to a concept as
Fig. 2 (A) Example of Boolean searching FloMap for mapping flowsheet rows to “Factors that Aggravate Pain.” (B) Display of flowsheet measuresmapped to the concept of “Factors that Aggravate Pain.”
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 189
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
well as the average (Avg) number of flowsheet rows acrossorganizations. On average, organizations mapped 9 flow-sheet rows to a single concept in the model. In fact, oneorganization had 81 unique flowsheet rows for recording“Numeric Pain Rating 0–10 Score.” ►Table 3 also includesstatistics for the percent of organizations using a particularconcept, the number and percent of patients for which aconcept was documented, and the total number of observa-tions documented. Some concepts, such as “Numeric PainRating 0–10 Score” are documented on 100% of patients.Finally, ►Table 3 also includes the number of value setchoices for each concept in the original and final models.The number of items in a value set ranged from 4 items for“Pain Duration” to 91 items for “Body Site.” The concepts thatremained in themodel (not newlyadded) are documented onaverage for 16% of patients. ►Table 4 lists the 13 conceptsfrom the final pain IM that are currently mapped to LOINCand 17 new concepts needed in LOINC. Additionally, therewere pain concepts in LOINC that were not found in organi-zations’ data.
Discussion
The purpose of our study was to develop and refine a pain IMfrom EHR flowsheet data that standardizes pain concepts,definitions, and associated value sets for assessments, goals,interventions, and outcomes. The data-driven consensus
process among 10 organizations resulted in a considerablereduction of concepts, panels (classes), and value set itemscompared with the original Pain Reference IM whichincluded 84 concepts grouped into 14 panels and 599 valueset items.5 The new model consists of 30 concepts groupedinto 4 panelswith 396 value set items. The consensus processhelped eliminate concepts from the original model mainlydue to limited use across organizations and consistency inrepresenting pain assessment scales. However, some infre-quently occurring concepts were retained in the model asthey were used to simplify documentation, such as a one-item question to assess nonverbal pain indicators versus a 5to 9 item observational pain scale. We found that organiza-tions combined some concepts for ease of documentationsuch as body orientation which included value items fromboth body location qualifier and body laterality. The pain IMseparated body orientation into the two concepts to beconsistent with LOINC standards.
One of the strengths of our study was extracting allflowsheet rows related to pain and mapping them to thePain Reference IM. EHRs become unwieldy over time withmultiple people building a system and upgrades occurring.Mapping all semantically comparable flowsheet rows to aconcept demonstrated the redundancy in EHRs. While Harriset al7 used a similar consensus process for developing apressure ulcer model, our study goes beyond their process toinclude the ability to find andmap data throughout the EHR. A
Fig. 3 Concepts retained in the pain information model (IM) through a data-driven consensus process.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.190
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table
2Pa
inreferenc
eIM
conc
epts
withvalid
ationde
cision
s
Pain
Reference
IMConc
eptNam
eValidation
Dec
ision
Commen
tDefinition
Pain
Assessm
entPa
nel
New
Thispa
ininclud
esmultipleco
ncep
tsas
wella
spa
nelsforpa
inpa
ttern,
pain
loca
tion
,an
dpa
inassessmen
tscales.
Theus
eof
multipleco
ncep
tsforassessmen
tof
pain
Current
Pain
Retain
Patien
tiscu
rren
tlyex
perien
cing
pain
atthetimehe
/she
isaske
dthequ
estion
Pain
Type
Retain
Categ
orizationof
thetype
ofpa
inthat
helpsgu
idefurthe
rassessmen
tan
dtrea
tmen
tch
oice
s
Con
text
ofPa
inRa
ting
Retain
Thecircum
stan
cesthat
form
thesettingforan
even
tin
which
pain
isassessed
–qu
alifier
forwhe
npa
inisassessed
Pain
Qua
lity(narrative
)Re
tain
Preferredterm
asitisus
edin
LOINC-rem
ove“n
arrative
”as
therewas
ava
lueset
Words
patien
tsus
eto
describe
theirpa
in
Non
verbal
Pain
Indicators
Retain
Merge
dExpression
ofpa
inwiththisco
ncep
tBe
haviorale
xpressions
ofpa
in–Th
eseinclud
eno
ticeab
lech
anges
inbe
haviorsan
dfluc
tuations
inph
ysiologic
sign
s.
Pain
Exacerba
ting
Factors
FactorsTh
atAgg
rava
tePa
inRe
tain
Rena
med
tomatch
LOINC
Postures/m
ovem
ents/age
ntsthat
produ
ceor
intens
ifya
patien
t’spa
inan
dareused
toestablishtheseve
rity,irritab
ility,
andna
ture
oftheco
ndition
Pain
Alle
viatingFactors
FactorsTh
atRe
lieve
/Alle
viatePa
inRe
tain
Rena
med
tomatch
LOINC
Patien
trepo
rtingwha
trelie
vestheirpa
in,an
agen
tthat
alleviates/red
uces/relieve
spa
in
Pain
PatternPa
nel
Add
Thisisthena
meof
apa
nelw
hich
does
notha
veavalueset
Thepa
neln
amethat
includ
esthesp
eedof
onset,freq
uenc
y,du
ration
,an
dco
urse
ofpa
in
Spee
dof
Pain
Ons
etRe
tain
Nam
emod
ified
tomatch
LOINC
Therate
atwhich
pain
occu
rred
,be
fore
thevisit/en
coun
ter
Pain
Duration
Retain
Thisco
ncep
treq
uires2an
swers-a
numbe
rand
ach
oice
listfor
theun
itof
mea
sure
i.e.,2an
dda
ysORLO
INCha
squ
antity
Patien
trepo
rted
descriptionof
how
long
thepa
inoc
curs
during
apa
rticular
pain
episod
e.
Pain
Freq
uenc
yRe
tain
How
oftenpa
inoc
curs
Pain
Cou
rseprogression
(narrative
)Re
tain
Pain
Cou
rseispreferredov
erPa
inProgression
beca
use‘Pro-
gression
’im
pliesso
mething
that
isge
ttingworse
(e.g.,a
disease);‘Cou
rse’
canim
plyso
mething
gettingbe
tter
orworse.LO
INCha
sPa
intempo
ralp
attern
which
isno
minal
Thepa
tternof
pain
orco
urse
canim
plyso
mething
getting
better
orworse
Pain
Body
Loca
tion
Pane
lRe
tain
Thisisthena
meof
apa
nelsyn
onym
ouswithBo
dysite
iden
tification
pane
linLO
INC
Thepa
neln
amethat
includ
esbo
dysite,loc
ationqu
alifier,a
ndlateralityof
pain
BodySite
Loca
tion
Retain
Cha
nged
nameto
match
LOINC
Spec
ifican
diden
tified
anatom
ical
location
BodyLo
cation
Qua
lifier
Orien
tation
Retain
Split
thisco
ncep
tinto
body
loca
tion
qualifier
(orien
tation
)and
laterality.
3911
2–8Bo
dyLo
cationQua
lifier
Ana
tomical
loca
tion
description
BodyLaterality
Add
Thisco
ncep
twas
derive
dfrom
body
orientationba
sedon
LOINClaterality.
2022
8–3Ana
tomic
part
Laterality
Description
ofaside
ofabo
dy
(Con
tinue
d)
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 191
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table
2(Con
tinue
d)
Pain
Referenc
eIM
Con
ceptNam
eValidation
Dec
ision
Com
men
tDefinition
Pain
ScalePa
nel
Add
Thisisthena
meof
apa
nel
Theassessmen
ttool/metho
dus
edto
evalua
tethepa
tien
t’s
pain
basedon
agean
dab
ility
toself-repo
rt.
Che
cklistof
Non
verbal
Pain
Indicators(CNPI)Score
Add
CRIES
Score
Add
Critica
l-carePa
inObs
ervation
Tool
(CPO
T)Sc
ore
Add
FACES
(Won
g-Bak
er)Ra
ting
ScaleScore
Retain
Face
sPa
inSc
ale–Re
vised
(FPS
-RScale)
Score
Add
Pain
Intens
ityself-rep
orttool
(3þ
yrs),c
ulturally
sens
itivetool
(witho
uttears)
FLACCPa
inAssessm
entScore
Retain
Retained
only
score–remov
ed6ind
ividua
litemsfor
cons
istenc
y
Neo
natalInfan
tPa
inScale
(NIPS)
Score
Retain
Retained
only
score–remov
ed6individua
litemsfor
cons
istenc
y
Neo
natalP
ain,
Agitation&
Seda
tion
Scale(N
-PASS
)Score
Retain
Retained
only
score–remov
ed5individua
litemsfor
cons
istenc
y
Num
eric
Pain
Rating
Scale0–
10Score
Retain
Syno
nymou
swithNum
eric
Rating
Scale(0–1
0)Pa
tien
tsrate
theirp
ainon
asimplescalefrom
0to
10,w
here
0is’nopa
in’an
d10
is’w
orst
pain
imag
inab
le’
PAIN
Adv
ance
dDem
entia
(PAINAD)Sc
ore
Retain
Retained
only
score–remov
ed5individua
litemsfor
cons
istenc
y
Prem
atureInfant
Pain
Profi
le(PIPP)
Score
Add
RevisedFLACCPa
inAssess-
men
t(rFLACC)Score
Retain
Retained
only
score–remov
ed6individua
litemsfor
cons
istenc
y
Pain
GoalsPa
nel
Thisisthena
meof
apa
nel
Thepa
tien
t’sde
siredpa
ingo
al
Accep
tableCom
fort
Leve
l(num
eric)
Add
Rating
of0-1
0ba
sedon
wha
tthepa
tien
tstates
isacce
ptab
leforman
agingpa
in
Accep
tableCom
fort
Leve
l(nom
inal)
Retain
Narrative
descriptionof
anacce
ptab
lelevelo
fpa
instated
bythepa
tien
t
Pain
Interven
tion
sRe
tain
Anac
tion,
trea
tmen
t,or
proc
edureto
prev
entor
alleviatepa
indo
neforthepa
tien
t
Pain
Outco
meDescription
Resp
onse
toInterven
tion
Retain
Com
bine
dAccep
table
Com
fort
Levelfor
Acu
tePa
inwith
Resp
onse
toInterven
tion
Percep
tionof
patien
t’sresp
onse
tothepa
ininterven
tion
s
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.192
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table
2(Con
tinue
d)
Pain
Referenc
eIM
Con
ceptNam
eValidation
Dec
ision
Commen
tDefinition
Original
Model
Conc
epts
Remove
d
Pain
ScaleUsed
Remov
eRe
place
dwithapa
nelfor
pain
scalean
dthen
includ
edthe
scoreforea
chpa
inscale
PreferredPa
inAssessm
entTo
olRe
mov
eLimited
use
Pain
Descriptors
Remov
eMerged
withPa
inQua
lity,
overlapin
valuesets
Expression
ofPa
inRe
mov
eOve
rlap
invaluesets
withNonv
erbal
Pain
Indicators
Pain
Fluc
tuation
Remov
eSimila
rco
ncep
tto
pain
course
Pain
History
(narrative
)Re
mov
eRe
dun
dant
ofothe
rco
ncep
ts
Loca
tion
-multiplesites
Remov
eLimited
use
Pain
RiskFactorsPa
nel
Remov
eLimited
use–Th
isinclud
ed4co
ncep
tsof
trea
tmen
t,ph
ysio-
logic,
person
al,an
den
vironm
entalrelated
Physiological
Pain
IndicatorPa
nel
Remov
eLimited
Use
–Th
isinclud
ed4co
ncep
tindica
tors
–bloo
dpressure,he
artrate,respiration
,oxyge
nsaturation
Clin
ically
Alig
nedPa
inAssessm
ent
(CAPA
)Pa
nel
Remov
eLimited
Use
–Th
isinclud
ed5co
ncep
tsof
comfort,ch
ange
inpa
in,pa
inco
ntrol,func
tion
ing,
slee
p
Pain
/Toleran
ceforH
andling(N
ICU)
Pane
lRe
mov
eLimited
Use
–thisinclud
ed6co
ncep
ts–co
mfort,p
ositioning
,problem
siden
tified
,arou
salstate,calm
ingtech
nique
s,an
da
narrative
Resp
onse
toInterven
tion
Remov
eCom
bine
dinto
Pain
Outco
meDescription
Resp
onse
toPa
inEd
ucation
Remov
eEd
ucationwas
notinclud
edin
thePa
inIM
Pain
med
ication(freetext)
Remov
eTh
isispa
rtof
themed
icationad
ministrationreco
rd
Pain
med
icationda
telast
dose
Remov
eTh
isispa
rtof
themed
icationad
ministrationreco
rd
Elec
trod
ePlacem
ent(TEN
S)Re
mov
eTh
isisinclud
edin
pain
interven
tion
s
Abbrev
iations
:FLA
CCscore,Face,Leg
s,Activity,Cry,C
onso
labilityscore;
IM,informationmod
el;LOINC,Log
icalObservation
Iden
tifiersNam
esan
dCod
es;N
ICU,n
eona
talinten
sive
care
unit;T
ENS,
tran
scutan
eous
elec
trical
nervestim
ulation.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 193
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table
3Descriptive
statistics
andLO
INCco
deforva
lidated
conc
epts
inthefina
lpainIM
Con
ceptna
me
Flow
shee
trowsmap
ped
%Organ
iza-
tions
Patien
tsNumber
ofob
servations
#Valuesetitem
s
Min
Avg
Max
Number
%Original
Fina
l
Current
Pain
118
5575
869,38
013
28,284
,945
94
Pain
Type
712
3288
980,28
715
17,895
,687
115
Con
text
ofPa
inRa
ting
111
3038
557,99
89
16,296
,187
45
Pain
Qua
lity
45
625
187,31
33
2,08
2,82
130
41
Nonv
erbal
Pain
Indica
tors
17
1650
56,254
170
3,59
827
37
Pain
ExacerbatingFactors
13
438
139,17
92
1,24
8,67
926
38
Pain
Alle
viatingFactors
223
6138
40,658
14,06
3,02
228
37
Pain
PatternPa
nel
Spee
dof
Pain
Ons
et1
58
8883
1,08
113
12,355
,821
115
Pain
Duration
27
1988
2,80
6,58
243
29,006
,751
44
Pain
Freq
uenc
y1
1131
6326
2,09
44
2,61
0,20
37
5
Pain
Cou
rse
14
838
126,07
62
823,07
44
Pain
Loca
tion
Pane
l
Body
Site
628
104
883,74
9,89
457
46,098
,215
8491
Body
Loca
tion
Qua
lifier
614
3263
763,38
012
10,969
,940
NA
NA
Body
Laterality
00
00
�0
�NA
NA
Pain
ScalePa
nel
Che
cklistof
Nonv
erbal
Pain
Indicators(CNPI)Score
209
CRIESScore
206
Critical-carePa
inObs
erva
tion
Tool
(CPO
T)Score
505
FACES
(Won
g–Ba
ker)
Rating
ScaleScore
113
2738
73,525
11,80
6,27
06
6
Face
sPa
inScale–
Revised
(FPS
-RScale)
Score
306
6
FLACCPa
inAssessm
entScore
17
2088
277,68
24
4,67
0,83
76
6
Neo
natalInfan
tPa
inSc
ale
(NIPS)
Score
11
163
258,25
44
2,72
3,49
27
7
12
250
111,32
32
4,78
1,12
96
6
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.194
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table
3(Con
tinue
d)
Con
ceptna
me
Flow
shee
trowsmap
ped
%Organ
iza-
tions
Patien
tsNumber
ofob
servations
#Valuesetitem
s
Min
Avg
Max
Number
%Original
Fina
l
Neo
natalP
ain,
Agitation
&Se
dationSc
ale(N
-PASS
)Sco
re
Num
eric
Pain
Rating
0–10
Score
423
8110
06,56
1,15
010
015
3,34
0,44
811
11
PAIN
Advanc
edDem
entia
(PAINAD)Score
28
2038
34,380
189
7,54
65
5
Prem
atureInfant
Pain
Profile
(PIPP)
Score
208
RevisedFLACCPa
inAssess-
men
t(rFLACC)Score
22
213
1,66
40
47,532
66
Pain
Goa
lsPa
nel
Accep
tableCom
fort
Leve
l(num
eric)
13
1163
2,71
6,90
641
65,625
,525
1111
Accep
tableCom
fort
Leve
l(nom
inal)
14
775
388,19
96
4,01
7,35
614
5
Pain
Interven
tion
s1
1344
100
2,63
9,96
440
43,971
,596
6667
Pain
Outco
meDescription
14
950
115,76
92
2,18
0,85
35
5
Abbrev
iation
s:Avg
,ave
rage
;FLA
CCscore,
Face,Leg
s,Activity,Cry,C
onso
labilityscore;
IM,informationmode
l;LO
INC,Log
ical
Observation
Iden
tifiersNam
esan
dCod
es;M
ax,m
axim
um;M
in,m
inim
um;N
A,n
otap
plicab
le.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 195
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
Table 4 Comparison of validated pain information model with LOINC nursing physiological assessment panela
Concepts in pain IM and LOINC Nursing Physiological Assessment (n ¼ 13)
32419–4 Pain Quality
38209–3 Pain Exacerbating Factors
38210–1 Pain Alleviating Factors
38203–6 Speed of Pain Onset
38207–7 Pain Duration
38206–9 Pain Course
39111–0 Body Site
39112–8 Body Location Qualifiera
20228–3 Body Lateralitya
80316–3 Pain Scales
38221–8 FACES (Wong–Baker)a
38208–5 Pain Rating 0–10 Scale
38213–5 FLACC Pain Assessment
Concepts In LOINC Nursing Physiological Assessment not in pain Reference IM (n ¼ 6)
38201–0 Pain Onset [Date and Time] – Reported
38202–8 Pain Onset [Hours Ago] – Reported
38204–4 Pain Primary Location – Reported
38205–1 Pain Radiation
38211–9 Pain Initiating Event Narrative – Reported
80317–1 Pain Assessment [Interpretation]
New concepts not in the Nursing Physiological Assessment (n ¼ 17)
Current Pain
Pain Type
Context of Pain Rating
Nonverbal Pain Indicators
Pain Frequency
Checklist of Nonverbal Pain Indicators (CNPI) Score
CRIES Score
Critical-care Pain Observation Tool (CPOT) Score
Neonatal Pain, Agitation & Sedation Scale (N-PASS) Score
Neonatal Infant Pain Scale (NIPS) Score
PAIN Advanced Dementia (PAINAD) Score
Premature Infant Pain Scale (PIPP) Score
Faces Pain Scale – Revised (FPS-R Scale) Score
Revised FLACC Pain Assessment (rFLACC) Score
Acceptable Comfort Level (numeric)
Acceptable Comfort Level (nominal)
Pain Outcome Description
Abbreviations: FLACC score, Face, Legs, Activity, Cry, Consolability score; IM, information model; LOINC, Logical Observation Identifiers Names andCodes.aConcepts in LOINC but not in Nursing Physiological Assessment Panel.
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.196
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
custom query provides a method for extracting comparabledata for interoperability and cross-organization pain research.
Using real-world evidence from large data sets is anincreasing trend in research due to the potential cost-sav-ings.13However, if research was conducted evaluating a vitalsign such as patients’ pain and it used only one of themultiple flowsheet rows mapped to a concept like “NumericPain Rating 0–10 Score,” then the study’s effectiveness forevaluating medications or nursing pain interventions couldresult in false conclusions because the pain rating datacontained in the other flowsheet rows mapped to “NumericPain Rating 0–10 Score” would be missing. Implementationof an IM can reduce redundancy and increase the usefulnessof the data.
While it might be ideal to have a single pain scale, weretained 12 unique pain assessment scales which includeboth self-report and observational assessments. Nurses andother clinicians need to select the appropriate tool based onage, setting, and clinical condition. The pain assessmentscales identified address these wide variety of circum-stances. One essential point, however, is the importance ofconsistent use of the same scale over time to evaluatepatient’s progress.
Results of our study extend the concepts needed in LOINCfor interoperability.14 For example, there are some importantpain concepts that are missing from LOINC such as “CurrentPain,” “PainType,” and “Acceptable Comfort Level (numeric).”However, there are LOINC concepts not found in our study,such as “Pain Onset,”which is a date and time stamp. AnotherLOINC term that was not found in our organizations’datawas“Pain Primary Location.” This is likely due to the fact thatpatients can have multiple pain locations, each with its ownassessment, so it is not used in practice.
There are several future steps planned including addingstandard terminology mappings to the concepts, validatingthe pain IM with additional organizations and settings, andapplying the process to validate IMs for other clinical areas.The terminology standards include LOINC for assessmentsand some outcomes and Systematized Nomenclature ofMedicine–Clinical Terms (SNOMED CT) for value sets asso-ciated with assessments, problems, and interventions.14
Additional terms will need to be submitted to LOINC andSNOMED CT when codes do not exist. Once this work iscompleted, broad dissemination is needed. The NursingKnowledge Big Data Science Initiative is developing anopen source repository for sharing work such as the painIM.15 Additional research is needed in several areas. Valida-tion of the IM with a broader set of stakeholders includinghome care, hospice, long-term care, and others would bebeneficial. The IM could also be validated with multipleclinical experts specific to pain using a Delphi technique orother consensus approach. Further IMmapping is needed onother nurse-sensitive measures such as falls, catheter-asso-ciated urinary tract infections (CAUTI), and central lineassociated bloodstream infections (CLABSI). Once themodelshave been validated, research is needed on implementationof the models and the coded data elements in EHRs tounderstand what worked, what problems and issues were
uncovered, how documentation is impacted, and if anydifferences exist based on vendor solutions. Ultimately, wewant to know if the IMs and use of coded key data elementsincreases interoperability and our ability to enable large,multicenter research including comparative effectivenessresearch.
Our work has several limitations. A volunteer sample oforganizations participated, and thus, the model may not begeneralizable to all organizations. While this was a conve-nience sample, which can limit generalizability of findings,the geographic locations, population size, and variety ofpractices provides a foundation for a generalizable pain IMthat can be used to support research. The pain IM is abeginning and it is anticipated that it will evolve overtime. In particular, there are additional concepts neededfor cardiac services and pain clinics may have more specia-lized assessments and interventions. Therewas variability inthe data extraction approaches and criteria used at eachparticipating organization that could influence the results.No attempt was made to dictate how to implement the painIM in an EHR, and thus organizations need to determine thebest practice for doing this. While the researchers consultedtheir pain experts, amore conscious effort is needed in futurework to include domain experts. Another limitation is thatFloMap is not yet available publicly nor is the SQL script fordata extraction. If others are interested in its use, they cancontact S.J., one of the authors on this article.
Conclusion
The purpose of our research was to validate and refine a painIM by using a data-driven approach across multiple healthsystems. The resulting pain IM is a consensusmodel based onactual EHR documentation in the participating health sys-tems. The pain IM captures the most important conceptsrelated to pain.
Clinical Relevance Statement
Secondary use of EHR data must be standardized for com-parison within and across organizations. Our study resultedin 30 concepts, definitions, and associated value sets agreedupon by 10 organizations as useful for building or optimizingan EHR. Our methods also allowed agencies to map theirflowsheet data to these concepts to support future research.
Multiple Choice Question
Variation in flowsheet data are often due to
a. The content and guidelines provided by vendorsb. Professional guidelines that influence contentc. Limited resources and rapid deployment of EHRsd. Available guidelines from terminologies of how to build
EHRse. All of the above
Correct Answer: The correct answer is e, all of the above.Vendorsprovide generic content andguideorganizations in
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al. 197
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.
using consensus-based approaches to configure the bulk oftheir system to meet the clinical requirements of theorganization. Much of the documentation, however, iscaptured in flowsheet format, using nonstandardized,semistructured data in a matrix format for patient assess-ments, goals, problems, interventions, and outcomes ofcare. Limited resources and rapid deployment timelinesprovide little time for organizations to identify and adoptstandardized terminologies and use IMs to design flow-sheets for future data sharing. Further, the informaticiansare required to choose frommultiple terminologies3,4withlimited reference standards to guide flowsheet builds.
Protection of Human and Animal SubjectsThe data were considered “metadata” and representeddescriptions of how the organization’s EHR was designedand aggregated counts for frequency of use; no patient-identifiable data were included. Each participant con-sulted with their organization to determine whetherInstitutional Board Approval was needed. If InstitutionalReview Board (IRB) approval was required, it wasobtained prior to data extraction and transmission to asecure database at the University of Minnesota.
Conflict of InterestNone.
AcknowledgmentWe would like to acknowledge the organizations thatwere willing to share their data and committed their stafftime to collaborate on this project over an extendedperiod of time.
References1 GoossenW, Goossen-Baremans A, van der Zel M. Detailed clinical
models: a review. Healthc Inform Res 2010;16(04):201–214
2 Johnson SG, ByrneMD, Christie B, et al.Modelingflowsheet data forclinical research.AMIAJt Summits Transl SciProc2015;2015:77–81
3 Rutherford M a. Standardized nursing language: what does itmean for nursing practice? Online J Issues Nurs 2008;13(01):1–7
4 Westra BL, Delaney CW, Konicek D, Keenan G. Nursing standardsto support the electronic health record. Nurs Outlook 2008;56(05):258–266.e1
5 Westra BL, Christie B, Johnson SG, et al. Modeling flowsheet datato support secondary use. Comput Informatics Nurs 2017;35(09):452–458
6 Chow M, Beene M, O’Brien A, et al. A nursing information modelprocess for interoperability. J AmMed Inform Assoc 2015;22(03):608–614
7 Harris MR, Langford LH, Miller H, Hook M, Dykes PC, Matney SA.Harmonizing and extending standards from a domain-specific andbottom-upapproach:anexamplefromdevelopment throughuse inclinical applications. J AmMed InformAssoc 2015;22(03):545–552
8 American Nurses Association & the American Society for PainManagement. Pain Management Nursing: Scope and Standards ofPractice. 2nd ed. Silver Spring, MD: American Nurses Association;2016
9 Nahin RL. Estimates of pain prevalence and severity in adults:United States, 2012. J Pain 2015;16(08):769–780
10 Gaskin DJ, Richard P. The economic costs of pain in the UnitedStates. J Pain 2012;13(08):715–724
11 The Joint Commission. Joint Commission Enhances Pain Assess-ment and Management Requirements for Accredited Hospitals.The Joint Commission Perspectives 2017;37(7):1-3. Available at:https://www.jointcommission.org/assets/1/18/Joint_Commis-sion_Enhances_Pain_Assessment_and_Management_Require-ments_for_Accredited_Hospitals1.PDF
12 Delaney CW, Pruinelli L, Alexander S, Westra BL. 2016 NursingKnowledge Big Data Science Initiative. Comput InformNurs 2016;34(09):384–386
13 Jarow JP, LaVange L, Woodcock J. Multidimensional evidencegeneration and FDA regulatory decision making: defining andusing “real-world” data. JAMA 2017;318(08):703–704
14 Matney SA, Settergren TT, Carrington JM, Richesson RL, Sheide A,Westra BL. Standardizing physiologic assessment data to enablebig data analytics. West J Nurs Res 2016;39(01):63–77
15 Carter-Templeton H, Effken J, Weaver C, Cochran K, Androwich I,O’Brien A. Toward a central repository for sharing nursing infor-matics’ best practices. Comput Inform Nurs 2016;34(06):245–246
Applied Clinical Informatics Vol. 9 No. 1/2018
Pain Information Model Westra et al.198
Thi
s do
cum
ent w
as d
ownl
oade
d fo
r pe
rson
al u
se o
nly.
Una
utho
rized
dis
trib
utio
n is
str
ictly
pro
hibi
ted.