Public hospital quality report awareness:evidence from National and CalifornianInternet searches and social mediamentions, 2012

Marco D Huesch,1,2,3,4 Elizabeth Currid-Halkett,1 Jason N Doctor2,5

To cite: Huesch MD, Currid-Halkett E, Doctor JN. Publichospital quality reportawareness: evidence fromNational and CalifornianInternet searches and socialmedia mentions, 2012. BMJOpen 2014;4:e004417.doi:10.1136/bmjopen-2013-004417

▸ Additional material isavailable. To view please visitthe journal (http://dx.doi.org/10.1136/bmjopen-2013-004417).

Received 6 November 2013Revised 29 January 2014Accepted 14 February 2014

1USC Sol Price School ofPublic Policy, Los Angeles,California, USA2USC Schaeffer Center forHealth Policy and Economics,Los Angeles, California, USA3Department of Communityand Family Medicine, DukeUniversity School ofMedicine, Durham,North Carolina, USA4Duke University FuquaSchool of Business, HealthSector Management Area,Durham, North Carolina, USA5USC School of Pharmacy,Los Angeles, California, USA

Correspondence toDr Marco D Huesch;huesch@usc.edu

ABSTRACTObjectives: Publicly available hospital quality reportsseek to inform consumers of important healthcarequality and affordability attributes, and may informconsumer decision-making. To understand how muchconsumers search for such information online on oneInternet search engine, whether they mention suchinformation in social media and how positively theyview this information.Setting and design: A leading Internet search engine(Google) was the main focus of the study. GoogleTrends and Google Adwords keyword analyses wereperformed for national and Californian searchesbetween 1 August 2012 and 31 July 2013 forkeywords related to ‘top hospital’, best hospital’, and‘hospital quality’, as well as for six specific hospitalquality reports. Separately, a proprietary social mediamonitoring tool was used to investigate blog, forum,social media and traditional media mentions of, andsentiment towards, major public reports of hospitalquality in California in 2012.Primary outcome measures: (1) Counts ofsearches for keywords performed on Google; (2)counts of and (3) sentiment of mentions of publicreports on social media.Results: National Google search volume for 75hospital quality-related terms averaged 610 700searches per month with strong variation by keywordand by state. A commercial report (Healthgrades) wasmore commonly searched for nationally on Googlethan the federal government’s Hospital Compare, whichotherwise dominated quality-related search terms.Social media references in California to quality reportswere generally few, and commercially producedhospital quality reports were more widely mentionedthan state (Office of Statewide Healthcare Planning andDevelopment (OSHPD)), or non-profit(CalHospitalCompare) reports.Conclusions: Consumers are somewhat aware ofhospital quality based on Internet search activity andsocial media disclosures. Public stakeholders may beable to broaden their quality dissemination initiativesby advertising on Google or Twitter and using socialmedia interactively with consumers looking for relevantinformation.

INTRODUCTIONPublic reports of hospital quality are a nationalpublic health priority at the intersection ofseveral important movements in healthcare:quality improvement,1 outcome transparency2

and autonomous, patient-centred decision-making by informed individuals.3 Activatingawareness of, interest in, access to, and increas-ing the usefulness of existing public reports onhospital healthcare delivery performance canpositively impact public health.4

To achieve these aims, public health stake-holders must take account of the mechanismsand networks by which information diffusionoccurs through the new media platforms, par-ticularly the omnipresence of Internet searchengines (eg, Google), ‘Web 2.0’ (second gener-ation Internet-based tools such as Facebook, aleading social network) and social media (weblogs or blogs, microbloggers such as Twitter,forums, Internet chat boards and discussiongroups).5 The Pew group notes that one in fiveAmericans report that the Internet has helpedthem deal with at least one major life decisionincluding health-related decisions.6 Amongpeople who have a significant change in theirhealth this proportion reaches one in four.6

Strengths and limitations of this study

▪ Study’s novel focus on Internet searches andsocial media mentions to explore modern con-sumer approaches in gathering and sharing ofinformation relating to hospital quality.

▪ This study’s results were limited by its reliance onone Internet search engine and on proprietary ana-lytical tools.

▪ Although freely available to researchers, we haveno way of ensuring that Google’s searchvolumes accurately represent Internet searches.

▪ Similarly, our use of a well-accepted commercialsocial media monitoring technology relies onproprietary, unverifiable analytics.

Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417 1

Open Access Research

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

The use of online social media and social networks alsoallows consumers to directly reach out to and mobilisefriends, relatives and experts for help.6 More than 50% ofonline adults between the ages of 18 and 55 years usesocial networking sites.7 The use of social media is explic-able as part of individuals’ well-documented need forautonomy, competence and relatedness, intrinsic andextrinsic rewards and well-being perceptions.8

Despite these new and increasingly popular mechan-isms of information dissemination, little research hasfocused on how public health stakeholders can leveragesuch channels to complement existing strategies. Intheory, such channels could allow fine targeting ofsubsets of consumers with well-defined interests and thepresentation of relevant health-related information tothem in cost-effective ways. Such outreach could, forexample, allow timely matching of preferred providersto interested patients.It is increasingly clear that merely placing hospital

quality information on the Internet is insufficient.9

Consumers must become aware of the information, theyought to have confidence in the information whichneeds to become salient in their mind, allowing the infor-mation to become actionable. Media dissemination ofmedical information has been associated with changes inconsumer behaviour. A significant drop in vaginal birthsafter caesarean section (CS) followed reports of increasedrisk in the traditional media and on the Internet.10

Before the Internet age, there was a significant increasein faecal occult blood tests following the widespreadmedia attention to President Reagan’s colon cancer.11

However, the public (and referring physicians) appearto underuse online hospital quality reports. Forexample, the introduction of Hospital Compare,Medicare’s public reporting initiative, was not associatedwith reductions in mortality beyond existing trends foracute myocardial infarction.12 Similarly, public reportingof provider and hospital CS rates via an interactivewebsite in Virginia since 1996 has not controlled CS rategrowth—indeed the rate of CS deliveries has risen fasterin Virginia than in the USA as a whole.13

This shortfall between the promise of public reportsin informing and empowering consumers to choosehigher quality, safer or more preferred providers andtheir limited success in achieving such shifts in con-sumer choice in practice has been attributed to designand formatting,14 relevance and appropriateness.15

Others argue that methodological issues are import-ant,16 that some online behavioural interventions havebeen effective,17 or that approaches for reporting qualityinformation have been slowly evolving towards effectivestrategies that help consumers.18–21

However, an additional explanation may be that mostconsumers do not know that such information existsand/or that it could be helpful for them. Were this tobe true, then one might expect few searches for suchinformation on the Internet, and few mentions of suchinformation or its sources on social media.

To our knowledge, no previous study has usedInternet and social media search tools to understandwhether consumers are actually aware of publicly avail-able hospital quality reports, have searched for them,and have discussed these with others. Such tools couldallow public health researchers to understand whichentities produce and disseminate reports that havehigher awareness or have been searched for more often.While such awareness is a necessary step in driving con-sumer activation higher it is clearly not sufficient, andour study did not investigate whether consumer behav-iour changed as a result of searches or mentions ofquality.We used Google’s in-house search and Adword analy-

tics tools to understand the volume and types of hospitalquality and quality report searches occurring throughGoogle’s market-leading Internet search engine. Googlehas maintained an approximate two-thirds of the marketfor Internet search, ahead of Yahoo and Bing.22 We alsoused a proprietary social media monitoring tool to inves-tigate all instances in California in which any of sixmajor publicly available hospital quality reports werementioned on social and traditional media, and the atti-tude of users (their ‘sentiment’).Although consumer awareness of such information is

clearly necessary for it to inform consumer decision-making, however it is insufficient. Our study was notable to infer whether Internet search activity or socialmedia mentions were associated with the processing ofsuch information. We also did not analyse whether theinformation contained in the publicly available hospitalquality reports was reliable, consistent, relevant andactionable. Moreover, our study was fundamentallylimited by a focus on just one Internet search engine,Google. We ignore searches occurring on Yahoo andMicrosoft’s Bing, and also do not ascertain importantother channels of accessing such information. Forinstance, we do not know how much direct Internetaccess occurs from consumers to a provider website, orhow many individuals click through an advertisement onone site directly through to a particular hospital orpublic sector website on healthcare quality.Despite these limitations, the findings in this study

may still serve to inform public dissemination strategies.Beyond this, our presentation of advanced Internetsearch engine and social media analytical tools mayallow other researchers to add such analyses to theirown, unrelated studies.

METHODSInternet search volumesTo understand Internet search behaviour, we focused onthe market-leading search engine, Google. We used twoGoogle tools to understand Google search enginevolumes. Google’s business model depends on onlineusers seeing or ‘clicking through’ display advertisements.These advertisements appear adjacent to search results

2 Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

and next to content on third party websites. Relevantadvertisements are more likely to be viewed and clickedthrough, so Google’s revenue depends on accuratelymatching advertisement keywords with those containedin searches or in published content elsewhere.Google provides two research tools to investigate key-

words and search terms. Both tools are free to researchersafter the creation and registration of a user account. Wefirst surfaced the most common search terms in the lastyear relating to three so-called seed terms: “hospital quality”,“best hospital” and “top hospital” using Google Adwords’Keyword Planner.23 The degree of relationship is calcu-lated by Google using proprietary algorithms incorporat-ing similar user search and purchasing behaviour. Forexample, Google suggests that “patient safety” is a relatedkeyword to “hospital quality”. However, AdWords did notsurface “healthgrades” as a keyword related to our threeseed terms of “hospital quality”, “top hospitals” and “besthospitals”. Google AdWords’ Keyword Planning Tool isdesigned primarily to suggest keywords for advertisersseeking to create matches between their own onlinedisplay advertisements and others’ online content.Consequently, advertisers are barred from using a differententity’s trademarks in their own keyword list.24

From the suggested keywords we then narrowed downsearches based on an arbitrary threshold of 1000 searchesper month and by our determination of their specificity tohospital quality (see table 1). We then used the GoogleTrends tool to quantify relative search volume by term,and by state, over the 12-month period from.25

Social media disclosuresTo understand instances (‘mentions’) in which hospitalquality terms and hospital quality reports were discussedin social and traditional media, we used a proprietarysocial media monitoring tool (MAP) from Marketwire/Sysomos (http://www.sysomos.com). Sysomos has mainlycorporate customers which include Microsoft, Coca-Cola,

Google, Marriott, Nike and other leading consumergood companies.This tool was leased by us as a ‘software-as-a-service’,

and allowed us to remotely access a database maintainedby Sysomos to investigate hundreds of millions of currentand historical data sources from all publicly disclosedblog, forum, social and traditional media sources (text,image and video) originating in California between 1January 2012 and 20 December 2012 for the six publichospital quality reports listed below.Social media tool queries were constructed using a

Boolean structure to limit extraneous mentions (eg,reuters reporting on a hospital construction) and tocapture abbreviations (eg, USNWR). In addition to men-tions, several other commonly used marketing metricswere tracked by MAP.‘Reach’, the degree of authoritativeness of the online

content or user, was estimated by MAP using a proprietaryalgorithm. Reach is summarised for each mention as ascore between 0 and 10 based on its authority and influ-ence. This measure is based on a number of differentfactors as to assess the influence and reach of the mediasource. For example, if the mention is on a blog, themeasure incorporates posting frequency, number ofunique links into and out of the blog, and number offollow-up comments. ‘Impressions’ are an estimate of thenumber of total views of content seen and were similarlyestimated by Sysomos’ proprietary analytics.Finally, sentiment was analysed by MAP’s proprietary

natural language processing (NLP) software. This isagain carried out automatically by MAP using proprietaryalgorithms, which summarise the sentiment of a set ofsuch writings (eg, all social media writing regardingHealthgrades, say) by what proportion was positive, nega-tive or neutral. Sysomos claims an 85% overall match ratebetween its MAP classification algorithms compared togold standard manual (ie, human expert) categorisation.Formally, sentiment analysis is the use of NLP tools to

identify, extract, grade and categorise subjective

Table 1 Most popular Google searches for terms relating hospital quality, 1 August 2012 to 31 July 2013

Exact Google search term Monthly search volume Exact Google search term Monthly search volume

hospital compare 12 100 best hospitals 1900

doctor reviews 9900 top hospitals in the us 1600

best medical schools 9900 patient satisfaction 1600

doctor ratings 6600 hospital rankings 1600

core measures 2900 patient satisfaction survey 1300

rate doctors 2400 best med schools 1300

patient safety 2400 best cancer hospitals 1300

hospital ratings 2400 physician ratings 1000

customer satisfaction survey 2400 nursing home ratings 1000

best hospitals in the us 2400 best hospitals in the world 1000

us news best hospitals 1900

Monthly search volume averaged over 12 months preceding 31 July 2013. Twenty-one most popular national Google searches related to “tophospital”, “best hospital” and “hospital quality” seed terms as determined by Google Adwords Keyword planner, excluding terms with fewerthan 1000 monthly searches or terms not specifically or exclusively related to quality in the authors’ determination. See supplementarymaterial for list of 75 search words before exclusions.

Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417 3

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

information (eg, attitude of blogger to the contentbeing discussed, mood of the person sharing some infor-mation on social media) contained within the writing.NLP tools effectively allow computers to understandhuman language.26 Similar tools have been used byhealth services researchers to assess the effectiveness oflabelling adverse drug events,27 to mine patient medicalrecord free text for evidence of diabetes care28 and toascertain patient falls faster than incidence reports byapplying NLP on imaging requests.29

Publicly available hospital quality reportsPublicly available hospital quality reports in Californiaincluded the following six major sources. From the infor-mation and news industry we selected Thomson Reuters’Best Hospitals ranking (http://www.100tophospitals.com), US News & World Report’s ranking of top hospi-tals (http://health.usnews.com/best-hospitals) andHealthgrades, a large private supplier of national phys-ician and hospital quality information (http://www.healthgrades.com/).From government sources we used Medicare’s

Hospital Compare tool (http://www.medicare.gov/hospitalcompare/) and California’s state Office ofStatewide Healthcare Planning and Development(OSHPD) reports (http://www.oshpd.ca.gov). Lastly, weselected the non-profit CalHospitalCompare (http://www.calhospitalcompare.org). Hospital Compare,Thomson Reuters and US News hospital quality reportswere not specifically related to maternity care.

RESULTSInternet search volumesWe explored search volume for popular terms related to‘top hospital’, ‘best hospital’ and ‘hospital quality’. Wecollected the intersection of all 2095 suggested keywordsrelated to these three seed terms which had been

surfaced by Google AdWords Keyword Planning tools.23

We then excluded search terms with less than 1000monthly searches nationally (full list available onrequest).From the resulting 75 search terms comprising

610 700 searches nationally each month on average, wethen further excluded search terms relating to specific,named hospitals (eg, “dana farber cancer institute”),geographies (eg, “hospitals in los angeles”) as well assearch terms not obviously related to quality (eg, “hos-pital gowns”, “hospital beds”) or not exclusively relatedto quality (eg, “hospital”, “medicare”).This process yielded 21 search terms (table 1) compris-

ing a total of 97 700 searches nationally per month onaverage. We focused further analysis on the mostcommon search terms within this list. We used anothertool, Google Trends, to track national and Californianonline search activity on Google’s search engine forterms related to hospital quality and quality reportsbetween 10 August 2012 and 2 August 2013. GoogleTrends results are normalised by Google so that the peaksearch volume over the terms analysed is set to 100.25

In figure 1, we show national time trends for searchesfor the four most popular hospital-related terms as pertable 1. These were “hospital compare” (presumablyintended as a search for the Federal Government’sHospital Compare website), “hospital ratings”, “best hos-pitals in the us” and “us news best hospitals”.Nationally, searches for the Hospital Compare term

dominated those relating to the other search terms,except in mid-July, where the contemporaneous publish-ing of US News and World Report’s rankings occurred.Interestingly, there was also a transient spike in searchvolume for “hospital ratings” at the same time, althoughnot for the other two search terms.In unreported analyses we examined search volumes

for terms that focused specifically on physicians not hos-pitals, despite being algorithmically related to the

Figure 1 Google National

Search Trends for Hospital

Quality and Reports, 1 August

2012 to 31 July 2013. Note:

search volume data provided by

Google Trends for national

searches. Four terms represented

the four most common keywords

surfaced by Google AdWords

related to seed terms “hospital

quality”, “best hospitals” and “top

hospitals”. Transient search

increase in mid-July 2013 for

“US News Best Hospitals” and

“Hospital Ratings” coincides with

release of US News’ 2013–2014

Best Hospitals rankings.

4 Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

hospital-centric seed terms according to GoogleAdWords. We found that “doctor reviews” and “bestmedical schools” dominated “doctor ratings”, and thatnational searches for all three terms were carried outthroughout the year with greater variability than thehospital-related searches shown in figure 1. WithinCalifornia, there were insufficient searches for “doctorratings” to be reported by Google.We also compared search volume in figure 2 between

the leading not-for-profit publicly available hospitalquality reports (Hospital Compare) and the leading for-profit one (Healthgrades). We found that “healthgrades”and its mis-spelled version “health grades” were eachmore commonly searched for than “hospital compare”,and that search volumes varied substantially over time.We also observed a very large, transient pair of spikes insearch volume in mid-January 2013 for “healthgrades”and the mis-spelled variant which coincided with releaseof Healthgrades’ Top Hospitals for Outcomes list.We disaggregated national searches into state-level

searches, data not shown, and note in passing an unex-pectedly varied distribution geographically for manyterms, and these differences did not remain constantacross the search terms.

Social media disclosuresUsing the commercially available MAP software, weexamined a complementary aspect of consumer aware-ness and interest. MAP tracked mentions of any of thepublic quality reports in consumer blogs, in commentson forums, and in social media more generally. Intable 2, we summarise this measure of consumer activa-tion for each of the six reports in 2012, focusing only onconsumers self-identifying as originating in California.Generally, there were few direct mentions of any of

these reports on either social media or traditional media,with less than 1000 instances in each type of media. Inunreported analyses, the typical posters of such content

were healthcare providers, academics, public healthagencies and commercial entities and stakeholders in thehealthcare system. The estimated number of online userswho were exposed to these mentions in aggregate waslarge for users of Twitter (>75 000 individuals) and fortraditional media (>275 000) but small or not estimatedby MAP for the other social media platforms.Irrespective of social, online or traditional media type,

the three commercial suppliers of hospital quality infor-mation appeared to dominate consumer awarenessusing this research tool. The federal government’sHospital Compare lookup tool was a distant fourth inmentions across all platforms, but in turn overwhelminglydominated the state-level OSHPD and the non-profitCalHospitalCompare reports which were effectively absentfrom publicly disclosed social conversations.To classify the sentiment expressed towards the reports,

we used MAP’s proprietary sentiment analyses to gaugethe disposition of consumers to these reports. Twitter sen-timent is typically ‘neutral’, given the short length. Inblogs and traditional news media sources, reaction tocontent was generally highly positive for HospitalCompare, Healthgrades, US News and World Report’sTop Hospitals and Thomson Reuters’s Best Hospitalspublic quality reports. Given the absent mentions of theOSHPD and CalHospitalCompare hospital qualityreports, MAP was unable to determine sentiment.

DISCUSSIONOur exploratory study used on market-leading Internetsearch engine, Google and Google Trend search tools toshow substantial Internet search activity broadly relatingto hospital quality, exceeding half a million monthlysearches nationally with a more narrowly defined set ofnearly one hundred thousand monthly searches. To putthis into context, a high traffic health-related websitesuch as WebMD is searched for on Google more than 1.8million times per month (and additional inquiries are

Figure 2 Google National

Search Trends for “Hospital

Compare” and “Healthgrades”,

1 August 2012 to 31 July 2013.

Note: search volume data

provided by Google Trends for

national searches. Transient

search increase in mid-January

2013 for “Healthgrades” and

mis-spelled “Health Grades”

coincides with release of

Healthgrades’ Top Hospitals for

Outcomes list.

Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417 5

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

made directly by typing webmd.com into a browser, cir-cumventing a search engine). Our main finding was thatcommercially produced hospital quality reports weresearched for more commonly than government reports,although searches for HospitalCompare dominatedother health-related quality keywords. Unsurprisingly,state-based variation broadly reflected number of consu-mers but variation differed by keyword searched for onGoogle.We supplemented this analysis of awareness by examin-

ing social media activity using a different, social mediamonitoring research tool. This analysis revealed littlesocial media and social network references to hospitalquality reports generally, and the instances found tendedto be from providers, experts, regulators, academics andnot individual consumers. However, of the little activitydocumented, we again found that commercially pro-duced hospital quality reports were more often men-tioned than not-for-profit or government ones.Our study has several strengths. Our focus was both

nationally and on the racially and ethnically diverse,

large state of California which suggests that the resultsmay have external validity. Yet, our study also has severalimportant limitations.Most importantly, our study’s ascertainment of con-

sumer access to healthcare information is fundamentallylimited to just one firm operating in one such channel.Within the space of Internet search engines, our focus onGoogle excluded Microsoft’s Bing, Yahoo and othersmaller but still important Internet search engines.Moreover, we have little understanding of how Google’susers might differ from those of other search engines.Outside this space, our study was not designed to ascer-tain individual’s use of, for example, advertisements inweb articles (with direct clickthrough to a source ofquality information) or the direct use of their Internetbrowser (to directly enter an already known website suchas healthgrades.com). Thus, our study’s results in thespace of Internet search engines should be seen as alower bound only.A separate limitation is that our social media analyses

were procured through the use of MAP, an analytical

Table 2 Social and traditional media mentions and sentiment for hospital quality reports: California 1 January 2012 to 20

December 2012

Commercial hospital quality reports Federal State Non-profit

Media Metric

Thomson

Reuters best

hospitals Healthgrades

US News &

World Report

top hospitals

Hospital

compare OSHPD

Cal Hospital

compare

Social media Twitter Mentions 33 14 11 2 0 0

Impressions 29 007 31 849 15 751 2348 0 0

Sentiment + 3% 0% 0% 0% NA NA

∼ 97% 100% 100% 100% NA NA

− 0% 0% 0% 0% NA NA

Facebook Mentions 0 2 0 0 0 0

Google+ Mentions 29 24 67 23 0 0

Sentiment + 28% 13% 34% 13% NA NA

∼ 65% 83% 62% 78% NA NA

− 7% 4% 4% 9% NA NA

Forum Mentions 0 2 6 1 0 0

Impressions 0 2 6 1 0 0

Youtube Items 95 86 2232 21 4 0

Video Items 22 12 107 3 0 0

Blogs Mentions 674 19 22 23 2 0

Reach 4.0 4.8 3.0 6.0 1.5 0.0

Sentiment + 100% 80% 91% 80% 0% NA

∼ 0% 20% 9% 20% 100% NA

− 0% 0% 0% 0% 0% NA

Traditional media Mentions 512 318 156 87 13 0

Impressions 274 341 2009 957 1290 283 0

Sentiment + 97% 97% 98% 87% 100% NA

∼ 3% 2% 1% 13% 0% NA

− 0% 1% 1% 0% 0% NA

Mentions (specific instances of public hospital quality reports mentioned in media), reach (authority level score out of 10), impressions (totalindividuals exposed) and sentiment (content attribute) for the period 1 January 2012 through 20 December 2012, except for Facebook whichis limited to status updates in the 6 weeks prior to 20 December 2012. MAP query terms were {"Healthgrades" AND "quality"}, {"OSHPD"AND "quality"}, {("Calhospitalcompare" OR "Cal hospitalcompare" OR "Calhospital compare" OR "Cal hospital compare") AND "quality"},{("ThomsonReuters" OR "Thomson Reuters") AND "Hospital"}, {("Hospitalcompare" OR "Hospital compare") AND "quality"}, and {("US NewsWorld Report" OR "USNWR") AND "Hospital"}.OSHPD, Office of Statewide Healthcare Planning and Development.

6 Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

software-as-a-service from Marketwire/Sysomos a leadingsocial media monitoring technology provider. This serviceuses proprietary, non-transparent NLP algorithms andsearch methods. It is possible that company-specificdesign decisions and negotiated upstream access to socialmedia platforms may have affected the reported results.Similarly, our Internet search analyses were performed

using Google’s proprietary research tools, and we haveno ability to confirm that the data supplied by Googleaccurately or completely represent all searches. However,we have no reason to believe that the data were masked,distorted or otherwise manipulated by Google in waysthat might have biased the results. To reiterate, we havelittle understanding of how representative Google usersare of Internet search engine users in general, or evenof Internet users using direct entry of website addressesinto their browser or clicking through from ads directlyto quality information sites. Despite Google’s command-ing share of the market, estimated at two-thirds,22 futurework will address the extent of sample selection thatcould arise from focus on a single search engine. Finally,we have assumed without data that an individual search-ing for ‘healthgrades’ is interested in quality and search-ing for this reason. This assumption may be wrong, forexample, as Healthgrades also informs users as to closesthospital to a particular zipcode or geographical area.Without data on intent, our assumption may thereforelead us to overstate interest in quality.Many stakeholders have the objective to increase con-

sumer awareness of hospital quality reports, and use ofthe Internet to disseminate such reports is common,increasing and promising. Yet, the largest and mostrecent study showed an apparent limited effectiveness ofthe flagship Hospital Compare reports in reducing mor-tality for key conditions requiring hospital admission.12

This shortfall between the promise and the actual hasbeen attributed to many causes. In this study, our focuswas on whether consumers know that such informationexists and/or that it could be helpful for them.Our use of Internet and social media search tools in

this exploratory study suggests that consumers are some-what aware of publicly available hospital quality reports,have searched for them and have discussed these withothers.We believe that these results suggest that there may be

some lessons that could be learned by public stake-holders from the private sector. Such lessons couldfurther their objectives of the enhancement of quality,greater transparency and more empowered consumers.To bridge the gap between the promise and the actualityof hospital quality reports, marketing approaches andconsumer activation approaches such as widespreadadvertising might help ‘get the word out’. Despite sub-stantial focus group testing of report contents andformats at CalHospitalCompare,30 for example, ourstudy finds that their public reporting efforts appear notto have resonated with consumers after their initiallaunch in 2007.

Fine-tuning of existing public quality reports toenhance their visibility and usability may also help, ascould a greater understanding of the crowded field ofratings-producing entities and the limited attention spanof consumers. Understanding how consumers see, reactto and use the information from all players in this space—private and not for profit—could allow organisationsto benchmark their performance, tailor and distinguishtheir offering from others’ in the market.Many of these efforts may require substantial invest-

ments. We estimate that Healthgrades spends in excessof US$50 million per year in producing, hosting andadvertising its services. Replicating a similar approachwould be infeasible for a county, but might be possiblefor a larger state health department. Such entities couldadvertise relatively cheaply on Google’s search engine,paying for ‘clicks’ through to their own websites by indi-viduals who search for hospital quality-related terms.Many of the keywords in table 1 are priced at around

US$1 per click at Google and on other search engines:an advertiser only pays this amount if the online userclicks the advertisement and navigates through to theadvertiser’s website. For example, a budget can be set inadvance, limiting financial risk for a small agency.Similarly, cheap advertising can be purchased throughsponsored tweets on Twitter. Certainly, at the federal gov-ernment level such levels of expenditure are easily pos-sible and might assist in elevating awareness and use ofHospital Compare.Cheaper niche marketing efforts in building con-

sumer awareness could also include engagement strat-egies using social media instead of other forms oftraditional advertising or passive Internet website cre-ation. Examples of campaigns on social media that havesucceeded through identification of leads and directinteraction with online users span the education indus-try, entertainment, small business and many other ‘viralmarketing’ settings.31 Stakeholders could exploit theadvertising platforms on, for example, Facebook to serveadvertisements to finely targeted (through Facebook’sextensive private knowledge of their users interests andprivate disclosures) segments of individuals ‘in themarket’ for a particular health-related message.The use of social media is increasingly part of indivi-

duals’ well-documented need for autonomy, competenceand relatedness, intrinsic and extrinsic rewards, and well-being perceptions.8 In addition, by virtue of its opensource nature, social media enables much more efficientdiffusion of information, network ‘cascades’ and perme-ation of many more social network.32 In contrast, thetraditional highly passive view of online consumers indi-vidually receiving broadcast messages from a staticwebsite and acting on them has shown at best onlymodest evidence of success in practice.14 15

Finally, a focus on more narrow subsets of customers, con-ditions or geographies may also be the optimal use ofscarce public health resources. Focusing on the mostimportant set of ‘shoppable conditions’ rather than trying

Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417 7

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

to cover every area of healthcare quality may be a morecost-effective strategy. Examples of such conditions withhigh variation in quality include maternity care.33–37

Targeted engagement at women of childbearing age,women who are pregnant and their family and friendswould allow a focus on foreseeable delivery healthcareneeds.Both social media interaction (eg, through a tool like

Sysomos’ MAP which allows real time identification ofconsumers posting relevant content on social media)and Internet search-linked advertising (eg, throughGoogle’s AdWords which allow fine delineation of whatsort of individuals, where, etc, see advertisements) arefeasible tools to reach such niche groups. Such toolscould be fruitfully and potentially cheaply used toeducate mothers regarding which hospitals in her neigh-bourhood have consistently lower primary CS rates,higher vaginal birth after cesarean (VBAC) rates, greaterlactation consultation access or higher breastfeedingrates.

CONCLUSIONA strong alignment of interests in stimulating qualityimprovement, increasing accountability and improvingconsumer choices appears increasingly opportune forpublic health stakeholders. An increasing variety ofcheap technology tools allow stakeholders to assess anddrive consumer awareness of healthcare choices. Giventhe importance of hospital quality information topayors, providers, consumers and regulators, we are opti-mistic that the science of public reporting will continueto develop in the ways outlined here.

Acknowledgements The authors are grateful to Marketwire for advice andtechnical assistance in obtaining the social media data using their SysomosMarket Access Platform.

Contributors MDH collected and analysed the data and wrote the first draft ofthe manuscript. MDH, EC-H and JND conceived and designed the study andrevised the manuscript for critical intellectual content. All authors gave finalapproval of the version to be published. MDH serves as guarantor of the article.

Funding Funding was received by the Agency for Healthcare Research andQuality (R21 HS021868-01; PI: Huesch).

Competing interests MDH discloses support from grants to his institution bythe Agency for Healthcare Research and Quality (AHRQ) and Lockheed Martinand receiving payments for consulting to the Institute of Medicine, the ParklandCenter for Clinical Innovation and Precision Health Economics, a healthcareconsultancy with clients in the pharmaceutical and medical device industries.

Ethics approval The study was qualified for expedited review under 45 CFR46.110 (b)(2) and was approved by the Institutional Review Board of theUniversity of Southern California.

Provenance and peer review Not commissioned; externally peer reviewed.

Data sharing statement All data obtained during this study from GoogleSearch and from Marketwire are freely available to academic researchers uponrequest to the corresponding author at huesch@usc.edu.

Open Access This is an Open Access article distributed in accordance withthe Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license,which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, providedthe original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

REFERENCES1. National Quality Forum. NQF endorsed performance measures.

2011. http://www.qualityforum.org/Measures_List.aspx (accessed 20Dec 2012).

2. Agency for Healthcare Research and Quality. Health Care ReportCard Compendium. 2011. https://www.talkingquality.ahrq.gov/content/reportcard/search.aspx (accessed 20 Dec 2012).

3. National Committee for Quality Assurance. Physician and hospitalquality report card. 2011. http://reportcard.ncqa.org/phq/external/phqsearch.aspx (accessed 20 Dec 2012).

4. Hibbard J, Sofaer S. Best practices in public reporting no. 3: how tomaximize public awareness and use of comparative quality reportsthrough effective promotion and dissemination strategies. AHRQPublication No. 10-0082-EF, Rockville, MD: Agency for HealthcareResearch and Quality, May 2010. http://www.ahrq.gov/qual/pubrptguide3.htm (accessed 20 Dec 2012).

5. Rainie L, Horrigan J, Wellman B, et al. The strength of Internet ties.Pew Internet. 2006. http://www.pewinternet.org/Reports/2006/The-Strength-of-Internet-Ties/01-Summary-of-Findings.aspx(accessed 20 Dec 2012).

6. Fox S. Health, technology and communities of color. 2012. http://pewinternet.org/Commentary/2012/February/Health-Technology-Communities-of-Color.aspx (accessed 20 Dec 2012).

7. Pew Internet & American Life Project. Generations 2010: Trends inOnline Activities. 2010. http://www.pewinternet.org/Infographics/2010/Generations-2010-Heat-Map.aspx (accessed 13 Aug 2013).

8. Hoffman DL, Novak TP. Why do people use social media? Empiricalfindings and a new theoretical framework for social media goalpursuit. 2012. http://ssrn.com/abstract=1989586 or http://dx.doi.org/10.2139/ssrn.1989586 (accessed 13 Aug 2013).

9. Lefebvre C. 5 Fictions about social media for public health andhealthcare. 2011. http://socialmarketing.blogs.com/r_craiig_lefebvres_social/2011/02/5-fictions-about-social-media-for-public-health-and-healthcare.html(accessed 20 Dec 2012).

10. Price J, Simon K. Patient education and the impact of new medicalresearch. J Health Econ 2009;28:1166–74.

11. Brown ML, Potosky AL. The presidential effect: the public healthresponse to media coverage about Ronald Reagan’s colon cancerepisode. Public Opin Q 1990;54:317–29.

12. Ryan AM, Nallamothu BK, Dimick JB. Medicare’s public reportinginitiative on hospital quality had modest or no impact on mortalityfrom three key conditions. Health Aff 2012;31:3585–92.

13. Main EK, Morton CH, Hopkins D, et al. Cesarean deliveries,outcomes, and opportunities for change in California: toward a publicagenda for maternity care safety and quality. CMQCC White Paper.2011. http://www.cmqcc.org/resources/2079

14. Sinaiko A, Eastman D, Rosenthal ME. How report cards onphysicians, physician groups, and hospitals can have greater impacton consumer choices. Health Aff 2012;31:602–11.

15. Weaver JB, Mays D, Lindner G, et al. Profiling characteristics ofInternet medical information users. J Am Med Inform Assoc2009;16:714–22.

16. Friedberg M, Damberg CL. A five-point checklist to helpperformance reports incentivize improvement and effectively guidepatients. Health Aff 2012;31:612–18.

17. Cugelman B, Thelwall M, Dawes P. Online interventions for socialmarketing health behavior change campaigns: a meta-analysis ofpsychological architectures and adherence factors. J Med InternetRes 2011;13:e17.

18. Hibbard JH, Peters EM. Supporting informed consumer health caredecisions: data presentation approaches that facilitate the use ofinformation in choice. Annu Rev Public Health 2003;24:413–33.

19. Hibbard JH, Greene J, Daniel D. What is quality anyway?Performance reports that clearly communicate to consumers themeaning of quality of care. Med Care Res Rev 2010;67:275–93.

20. Peters EM, Dieckmann N, Dixon A, et al. Less is more in presentingquality information to consumers.Med Care Res Rev 2007;64:169–90.

21. Hibbard JH, Greene J, Sofaer S, et al. An experiment shows that awell-designed report on costs and quality can help consumerschoose high-value health care. Health Aff 2012;31:560–8.

22. Goodwin D. Search Engine Watch. http://searchenginewatch.com/article/2307115/Google-Fails-to-Gain-Search-Market-Share-Bing-Steals-From-Yahoo (accessed 28 Jan 2014).

23. Google Adwords. Keyword Tool. 2013. Registration needed. https://adwords.google.com/ko/KeywordPlanner/home (accessed 31 Jul 2013).

24. Google AdWords. Trademark Policy. 2013. https://support.google.com/adwordspolicy/answer/6118?hl=en (accessed 13 Aug 2013).

25. Google Trends. Search Tool. 2013. Registration needed. http://www.google.com/trends (accessed 31 Jul 2013).

8 Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from

26. Jha AK. The promise of electronic records: around the corner ordown the road? JAMA 2011;306:880–1.

27. Duke J, Friedlin J, Ryan P. A quantitative analysis of adverse eventsand “Overwarning” in drug labeling. Arch Intern Med 2011;171:941–54.

28. Turchin A, Kohane IS, Pendergrass ML. Identification of patientswith diabetes from the text of physician notes in the electronicmedical record. Diabetes Care 2005;28:1794–5.

29. Toyabe S. Detecting inpatient falls by using natural language processingof electronic medical records. BMC Health Serv Res 2012;12:448.

30. CalHospitalCompare. Frequently asked questions. 2012. http://www.calhospitalcompare.org/about-us/faqs.aspx (accessed 20 Dec 2012).

31. Sysomos. Resource library. 2013. http://www.sysomos.com/social-media/case-studies/ (Accessed 13 Aug 2013).

32. Newman M, Barabasi A, Watts DJ. The structure and dynamics ofnetworks. Princeton: Princeton University Press, 2011.

33. Gregory KD, Korst LM, Platt LD. Variation in elective primarycesarean delivery by patient and hospital factors. Am J ObstetGynecol 2001;184:1521–32.

34. California Maternal Quality Care Collaborative. Current priorityinitiatives. 2012. http://www.cmqcc.org/ (accessed 20 Dec2012).

35. Joynt J. Quality of care facts and figures. California FoundationHealth Care Almanac, 2011. http://www.chcf.org/publications/2011/10/quality-care-facts-figures (accessed 20 Dec 2012).

36. Huesch MD. Physician incentives, maternal choice andprimary pre-labor cesarean rates. Am J Public Health2011;101:1–7.

37. Agency for Healthcare Research and Quality. Perinatal caremeasure summary. 2012. http://www.qualitymeasures.ahrq.gov/content.aspx?id=34144 (accessed 20 Dec 2012).

Huesch MD, Currid-Halkett E, Doctor JN. BMJ Open 2014;4:e004417. doi:10.1136/bmjopen-2013-004417 9

Open Access

on July 26, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2013-004417 on 11 March 2014. D

ownloaded from