+ tweets about hospital quality: a mixed methods study felix greaves harkness fellow, harvard school...
TRANSCRIPT
![Page 1: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/1.jpg)
+
Tweets about hospital quality: A mixed methods study
Felix Greaves
Harkness Fellow, Harvard School of Public Health / Imperial College London
@felixgreaves
![Page 2: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/2.jpg)
+Co-authors / conflict of interest
Anthony Laverty
Daniel Cano-Ramirez
Stephen Pulman
Karo Moilanen
Ara Darzi
Christopher Millett
Dr Pulman and Dr Moilanen are founders of Theysay Ltd, a sentiment analysis spin off company from Oxford University
![Page 3: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/3.jpg)
+
![Page 4: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/4.jpg)
+
![Page 5: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/5.jpg)
+Social media use is normal
81% of UK population use the Internet
61% use a social networking site
Source: Oxford Internet Survey 2013
![Page 6: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/6.jpg)
+We love describing things on the internet
![Page 7: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/7.jpg)
+We compared ratings with traditional measures of quality
Vs.
Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS. Greaves F et al. BMJ Qual Saf. 2012;21(7):600-5.
![Page 8: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/8.jpg)
+Cloud of patient experience
Free Text
Greaves F et al. Harnessing the cloud of patient experience: using social media to detect poor quality healthcare. BMJ Qual Saf.2013 Mar;22(3):251-5.
![Page 9: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/9.jpg)
+ A new trend: ‘Big Data’ and social media analytics
![Page 10: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/10.jpg)
+Sentiment analysis
![Page 11: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/11.jpg)
+
(
![Page 12: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/12.jpg)
+
:(
![Page 13: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/13.jpg)
+
:)
![Page 14: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/14.jpg)
+Sentiment analysis of patientexperience is possible
Overall CleanlinessDignity• rude • dirty • rude• excellent • floor • told• hours • filthy • thank• pain • bed • friendly• communicatio
n • blood • attitude
Greaves F et al. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res. 2013 Nov 1;15(11):e239.
![Page 15: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/15.jpg)
+NHS Insight dashboard
![Page 16: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/16.jpg)
+Aims
To describe the frequency with which people are talking about hospitals on Twitter
To understand what proportion of tweets to hospitals are related to care quality
To examine whether there are associations between Twitter sentiment and other measures of care quality
![Page 17: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/17.jpg)
+Methods
Collected the Twitter names of all English NHS acute hospitals in April 2012
Collected all tweets ‘mentioning’ hospitals for 1 year
Qualitative analysis of 1000 random tweets
Automated sentiment analysis of all tweets Compared with hospital quality metrics
![Page 18: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/18.jpg)
+Results76 of 166 hospitals were on Twitter198,499 tweetsMean tweets per trust: 2647 Median: 796, range: 0 to 88169
0-99
200-
299
400-
499
600-
699
800-
899
1000
-109
9
1200
-129
9
1400
-149
9
1600
-169
9
1800
-189
9
2000
-209
9
2200
-229
9
2400
-249
9
2600
-269
9
2800
-289
9
3000
+
0
1
2
3
4
5
6
7
8
9
10
Number of tweets
Nu
mb
er
of
hosp
itals
![Page 19: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/19.jpg)
+Time of tweets
02
00
04
00
06
00
08
00
0F
req
ue
ncy
00 05 11 16 22Hour of day
![Page 20: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/20.jpg)
+Mean: 508 tweets per dayMedian: 405Range: 62 to 3601
![Page 21: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/21.jpg)
+Qualitative analysis –Most tweets aren't about quality
All tweets: 47% were positive, 47% were neutral and 5.6% were negative.
Tweets about quality: 77% were positive, 2% were neutral and 21% were negative .
Main theme % Minor theme % Sub theme %
Care Quality 11.4Patient centred 9.8
Staff 5.5Environment 1.1
Access 1.4Effective 2.9
Safe 0.4
Fundraising 38.7
About a patient in hospital 10.5Promotion 8.8
Organisational Information 7.1Health Information 4.4
Non-specific content 20.7
![Page 22: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/22.jpg)
+Interaction with staff
Home from [@named hospital] after a weeks stay...we feel blessed to have been cared for by such an amazing team. Thank you [named ward] x
At the [@named hospital] just had an operation on me foot. Outstanding care as usual, & the nurse has just made me a cracking cup of tea :-)
[@named hospital] [named ward] - Disgusted with your treatment of my mother. Will be making huge complaints.
![Page 23: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/23.jpg)
+Environment
Be nice if this room had been cleaned before we got it. Blood filled cap from an iv on the bedside cabinet, unflushed toilet [@named hospital]
Spent a night in [named hospital] with my son. Excellent care - spotlessly clean. Thank you [@named hospital]
Don't suppose there is any chance of full english [@namedhospital] Been here since 3 yesterday no hot food or drink #poor
![Page 24: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/24.jpg)
+Access/Timeliness
[@named hospital] Thanks for squeezing me in with orthoptist [named staff member] today. Great service just so sad that waiting list for [named surgeon] so long :-(
[@named hospital] where the waiting time is ridiculous waited 3hr yesterday, 3lots of bloods took, 2hrs so far today for a blood test again!
Waiting at [@named hospital]- appointment was over 2 hours ago. can we get takeout delivered??
![Page 25: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/25.jpg)
+Effectiveness / Safety
[@named hospital]Yes pls. Main concern now is the doctor overprescribing. We worked out the error but vulnerable patient might not
[@named hospital] Also looking at a scan from 2010 when u didn't get scanned until 2011 not good, wrong person, terrible, disgusting
![Page 26: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/26.jpg)
+Multimedia
[@named hospital] Cannot believe I have been served this for my 18 month old. Tastes disgusting and hardly nutritional
[permission obtained to use]
![Page 27: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/27.jpg)
+People can be rude
[@named hospital] [Named chief executive] should come down onto the wards n see what's really going on under her nose. I wish my nan was in [another hospital]
[@named hospital] Shit on floor wet sheets, visitors having to change bedding, shit in toilet, ignorant staff- [name ward]!! Stay away !
![Page 28: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/28.jpg)
+Twitter sentiment compared to traditional quality measures
Twitter Metric Conventional Measure Spearman Rho p value
Twitter sentiment Survey measure of “Overall, how would you rate the quality of care you received”
0.15 0.30
Twitter sentiment Risk adjusted mortality rate 0.15 0.24
Agreement 71%, Kappa 0.39
![Page 29: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/29.jpg)
+Limitations
• Only one way messages, not conversations
• Ability of sentiment analysis approach• Sarcasm / Irony• Culturally specific
• Vulnerable to external effects
![Page 30: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/30.jpg)
+
Like an angel
![Page 31: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/31.jpg)
+
Stank of urine
![Page 32: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/32.jpg)
+
Cup of tea
![Page 33: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/33.jpg)
+Tweets about the NHS are affected by external factors
![Page 34: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/34.jpg)
+
![Page 35: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/35.jpg)
+
![Page 36: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/36.jpg)
+Conclusions
Twitter is being used by patients to discuss care quality
But…it’s only a minority of the tweets
Signal at risk of being lost in the noise
Twitter may be more useful to drive quality improvement than understand comparative performance
Less about Big Data – more about small human stories
![Page 37: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/37.jpg)
+Wider lessons from social media
A powerful tool for spreading ideas
Allows crowdsourcing information
Flattens hierarchies
Allows new approaches to patient engagement
![Page 38: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/38.jpg)
+Caution…
Self selecting
Fickle
New ethical questions
Reality vs. hype
![Page 39: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/39.jpg)
+
![Page 41: + Tweets about hospital quality: A mixed methods study Felix Greaves Harkness Fellow, Harvard School of Public Health / Imperial College London fgreaves@hsph.harvard.edu](https://reader035.vdocument.in/reader035/viewer/2022062518/56649c955503460f949527b1/html5/thumbnails/41.jpg)
+What affects the public’s decision about where to go to hospital
-8
-6
-4
-2
0
2
4
6
If better than average
If worse than average
We
ek
s w
illin
g t
o w
ait