social media intelligence: measuring brand sentiment from online conversations
DESCRIPTION
Social Media Intelligence: Measuring Brand Sentiment from Online Conversations. David A. Schweidel Goizueta Business School Emory University October 2012. What’s Trending on Social Media?. Agenda. Social Dynamics in Social Media Behavior - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/1.jpg)
Social Media Intelligence: Measuring Brand Sentiment from Online Conversations
David A. SchweidelGoizueta Business School
Emory University
October 2012
![Page 2: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/2.jpg)
What’s Trending on Social Media?
![Page 3: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/3.jpg)
Agenda• Social Dynamics in Social Media Behavior
– Why do people post a product opinion? What influences their posting behavior? (Moe and Schweidel 2012)
– What is the impact of these social dynamics on product sales? (Moe and Trusov 2011)
• Social Media Intelligence (work in progress with W. Moe)– What factors influence social media metrics?– How can we adjust our metrics for different sources of
data?
![Page 4: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/4.jpg)
Why do people post?
• Opinion formation versus opinion expression (Berinsky 2005)
• Opinion formation– Pre/post purchase (Kuksov and Xie
2008)
– Customer satisfaction and word-of-mouth (Anderson and Sullivan 1993, Anderson 1998)
• Opinion expression– Opinion dynamics (Godes and Silva
2009, Li and Hitt 2008, Schlosser 2005, McAllister and Studlar 1991)
– Opinion polls and voter turnout (see for example McAllister and Studlar 1991)
Pre-Purchase EvaluationE[uij]
Purchase Decision and Product Experience
Post-Purchase EvaluationVij=f(uij, E[uij])
IncidenceDecision
EvaluationDecision
Posted Product Ratings
LATE
NT
EXPE
RIEN
CE M
ODE
LIN
CIDE
NCE
& E
VALU
ATIO
N M
ODE
LS
SELECTIONEFFECT
ADJUSTMENTEFFECT
![Page 5: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/5.jpg)
Selection Effects: What influences participation?
• Extremely dissatisfied customers are more likely to engage in offline word-of-mouth (Anderson 1998)
• Online word-of-mouth is predominantly positive (e.g. Chevalier and Mayzlin 2006, Dellarocas and Narayan 2006)
• Subject to the opinions of others– Bandwagon effects (McAllister and Studlar 1991, Marsh 1984)
– Underdog effects (Gartner 1976, Straffin 1977)
– Effect of consensus (Epstein and Strom 1981, Dubois 1983, Jackson 1983, Delli Carpini 1984, Sudman 1986)
![Page 6: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/6.jpg)
Adjustment effects:What influences posted ratings?
• Empirical evidence of opinion dynamics– Online opinions decline as product matures (Li and Hitt 2008)
– Online opinions decline with ordinality of rating (Godes and Silva 2009)
• Other behavioral explanations of opinion dynamics– “Experts” differentiate from the crowd by being more negative
(Schlosser 2005)
– “Multiple audience” effects when opinion variance is high (Fleming et al 1990)
![Page 7: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/7.jpg)
Modeling Overview
• Online product ratings for bath, fragrance and home retailer over 6 months in 2007 (sample of 200 products with 3681 ratings)
• Two component model structure (Ying, Feinberg and Wedel 2006):– Incidence (Probit)– Evaluation (Ordered Probit)
• Product utility links incidence and evaluation models (non-linear)
• Bandwagon versus differentiation effects– Covariates of ratings environment– Separate but correlated effects in each model component
• Product and individual heterogeneity
![Page 8: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/8.jpg)
Role of post-purchase evaluation (Vij)in rating incidence
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
01.5 2 2.5 3 3.5 4 4.5 5
Util
ity E
ffect
on
Post
ing
Inci
denc
e
Individual-Product Utility Value
![Page 9: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/9.jpg)
Classifying Opinion Contributors
• Groups based on frequency of posts (β0)
![Page 10: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/10.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Prop
ortio
n of
Pos
ters
Rating
% Activists % Low-Involvement
0
0.5
1
1.5
2
2.5
3
3.5
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Varia
nce
Aver
age
Rat
ing
Rating
Average Variance
Empirical Trends• Posted ratings
– Average rating decreases over time
– Variance increases over time
• Poster composition– Community-builders are over-
represented in the posting population.
– As forum evolves, participation from community-builders increases while that of LI and BW decreases.
![Page 11: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/11.jpg)
0
0.5
1
1.5
2
2.5
3
3.5
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Varia
nce
Aver
age
Rat
ing
Rating
Average Variance
0
0.5
1
1.5
2
2.5
3
3.5
3
3.2
3.4
3.6
3.8
4
4.2
4.4
4.6
4.8
5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Varia
nce
Aver
age
Rat
ing
Rating
Average Variance
Effect of Opinion Variance
• Customer bases have same mean but different variance in opinions
• With a polarized customer base, ratings exhibit:– Lower average with a significant decreasing trend– Greater variance
• Negative ratings do not necessarily signal a lower average opinion among customers
MEDIAN CUSTOMER BASE POLARIZED CUSTOMER BASE
![Page 12: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/12.jpg)
Conclusions: Individual-level Analysis• Empirical findings
– Heterogeneity in posting incidence and evaluation– Incidence and evaluation behavior are related and can result in a
systematic evolution of posting population
• General trends in the evolution of product forums:– Dominated by “activists””– Participation by activists tends to increase as forum evolves while
participation by low-involvement individuals tend to decrease
• Implications:– Ratings environment does not necessarily reflect the opinions of the
entire customer base or even the socially unbiased opinions of the posters
– Posting behavior is subject to venue effects…
![Page 13: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/13.jpg)
Posting Decisions
Opinion / Brand Evaluation
Do I post? What do I post?
Sentiment
Product
Where do I post?
Attribute
Venue Format
Domain
SOCIAL MEDIA METRICS
![Page 14: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/14.jpg)
The value of social media as a research tool
• Does it matter where (i.e., blogs, microblogs, forums, ratings/reviews, etc.) we listen?– Product/topic differences across venues?– Systematic sentiment differences across venues?– Venue specific trends and dynamics?
• How do social media metrics compare to other available measures?
![Page 15: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/15.jpg)
Social media monitoringIN PRACTICE
• Early warning system: Kraft removed transfats from Oreos in 2003 after monitoring blogs
• Customer feedback: Land of Nod monitors reviews to help with product modifications and redesigns
• Measuring sentiment: Social media listen platforms collect comments across venues
IN RESEARCH• Twitter to predict sock prices
(Bollen, Mao and Zeng 2011)
• Twitter to predict movie sales (Rui, Whinston and Winkler 2009)
• Discussion forums to predict TV ratings (Godes and Mayzlin 2004)
• Ratings and Reviews to predict sales (Chevalier and Mayzlin 2006)
![Page 16: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/16.jpg)
Does source of data matter?• Online venues (e.g., blogs, forums, social networks, micro-blogs) differ
in:– Extent of social interaction– Amount of information– Audience attracted– Focal product/attribute
• Venue is a choice– Consumers seek out brand communities (Muniz and O’Guinn 2001)
– Venue depends on posting motivation (Chen and Kirmani 2012)
• Social dynamics affects posting (Moe and Schweidel 2012, Moe and Trusov 2011)
![Page 17: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/17.jpg)
Research Objective• Assess online social media as a listening tool
• Disentangle the following factors that can systematically influence posted sentiment– Venue differences– Product and attribute differences– Within venue trends and dynamics
• Examine differences across different venue types– Sentiment– Product and attribute differences– Implications for social media monitoring and metrics
![Page 18: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/18.jpg)
Social Media Data• Provided by Converseon (leading online social
media listening platform and agency)• Sample of approximately 500 postings per month
pertaining to target brand• Comments manually coded for:
– Sentiment (positive, neutral, negative)– Venue and venue format– Focal product/attribute
• Categories: (1) enterprise software, (2) telecommunications, (3) credit card services, and (4) automobile manufacturing
![Page 19: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/19.jpg)
Data for Enterprise Software Brand• 140 products within the brand portfolio• 59 brand attributes (e.g., compatibility, price, service, etc.)• Social Media data spanned a 15 month period
– June 2009 – August 2010– 7565 posted comments– Across 800+ domains
Venue Format Illustrative Website Frequency Direct ExperienceDiscussion Forum forums.adobe.com 2728 93%
Micro-blog twitter.com 2333 37%Blog wordpress.com 2274 23%
Social Network linkedin.com 155 40%Mainstream News cnbc.com 36 3%
Social News digg.com 19 47%Wiki adobe.wikia.com 10 50%
Video vimeo.com 6 0%Review Sites epinions.com 4 25%
![Page 20: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/20.jpg)
Modeling Social Media Sentiment• Comments coded as “negative”, “neutral”,
or “positive”• Ordered probit regression
)Pr()Pr( 1)(
*)(
rivii
rivi Ury
1),(1),(*
iaipii VSU
venue-specific brand sentiment
product effect attribute effect
![Page 21: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/21.jpg)
What affects venue-specific brand sentiment?
• General brand impression (GBI)• Domain and venue effects (including
dynamics)
)(),()()()( itivividiti GBIVS
domain effect
venue-format effect
venue-specific dynamics
![Page 22: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/22.jpg)
Venue Attractiveness
• Model venue format as a choice made by the poster
• Multinomial logit model
effect of content on venue choice
product effect Attribute effect
![Page 23: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/23.jpg)
Model Comparisons
• Baseline model– Independent sentiment and venue decisions– Controlling for product, topic and domain effects
DICSentiment
hit rateVenue choice
hit rateBaseline 32588.0 0.454 0.316+ Sentiment link 30040.7 0.451 0.424+ Venue specific sentiment 29677.2 0.465 0.424+ Venue specific dynamics 29529.1 0.473 0.424
![Page 24: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/24.jpg)
Sentiment metrics vary depending on what you are measuring.
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
GBI
Observed AverageSentiment
GBIObs. Avg. Sentiment Blog Forum Microblogs
GBI 1
Obs. Avg. Sentiment -0.0346 1
Blog 0.678 0.496 1
Forum 0.00409 0.263 -0.0870 1
Microblogs 0.751 0.503 0.8196 -0.139 1
CORRELATIONS
![Page 25: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/25.jpg)
Sentiment Differencesacross Venues (v(i))
0
0.5
1
1.5
2
2.5
3
3.5
![Page 26: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/26.jpg)
Divergent Trendsacross Venues (v(i),t(i))
• Sentiment varies across venues• Venue-specific sentiment is subject to
venue-specific dynamics
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Blog
Forum
Micro-blog
* Blogs, forums and microblogs are the 3 most common venues
![Page 27: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/27.jpg)
Attribute Effects on Sentiment (a(i),1)
• Provides attribute-specific sentiment metrics• Empirical measures are problematic due to data sparsity• Correlation between model-based effects and observed attribute-sentiment
metrics = -.276
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
Appl
icati
on
Com
patib
ility
Bran
d Re
puta
tion
Docu
men
tatio
n/Su
ppor
t
Ease
of I
nsta
llatio
n
Har
dwar
e Co
mpa
tibilit
y
Mob
ile C
ompa
tibili
ty
Pric
e
Qua
lity
of P
rodu
ct
Secu
rity
Size
of C
ompa
ny
![Page 28: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/28.jpg)
Venue Attractiveness ResultsVenue Intercept (g)
Product/Attrib. Effects (l) Effect of GBI (k)
Blog 5.012 1.000 -1.610Forum 5.655 -2.713 5.002Mainstream Media 0.246 2.474 -2.632Microblogs 5.378 1.005 2.978Photoshare -4.509 -0.469 0.321Review site -1.229 -2.427 0.767Social network 2.349 1.771 1.086Social news aggregator 0.607 -0.058 1.848Video share -1.024 0.784 -1.105Wiki -- -- --
Posters with positive sentiments toward the brand are attracted to forums and microblogs.
Forums attract comments that focus on different products and attributes than microblogs.
![Page 29: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/29.jpg)
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8In
dust
ry N
ews
Corp
orat
e Pa
rtne
rshi
pM
arke
ting/
Cam
paig
nM
arke
ting
Clou
d Co
mpu
ting
Mic
roso
ft P
artn
ersh
ipM
arke
t Sha
reFu
ture
of V
irtua
lizati
onPr
ivat
e/In
tern
al C
loud
Infr
astr
uctu
re C
ost
Serv
er V
irtua
lizati
on T
echn
ical
Issu
eI/
O P
erfo
rman
ceCu
stom
izatio
nEa
se o
f…Re
sour
ce o
ptim
izatio
nG
raph
ics s
uppo
rtHa
rdw
are
Com
patib
ility
Appl
icati
on C
ompa
t.Pr
otoc
ol C
ompa
tibili
ty
Attribute effects on venue choice (a(i),2)
TOP 10 BOTTOM 10
* For products with >5 mentions only
![Page 30: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/30.jpg)
Predictive Value of GBI
• Offline brand tracking survey– Satisfaction surveys conducted from Nov 2009 to Aug 2010 in waves
(overlapping with last 10 months of social media data)– Approximately 100 surveys conducted per month– 7 questions re: overall sentiment toward brand
• Company stock price– Weekly and monthly closing prices for firm– Weekly and monthly closing S&P– June 2009 to September 2010 (extra month for lag)
![Page 31: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/31.jpg)
GBI vs. Offline Survey• Potential for GBI as a lead
indicator
• Correlation with survey– GBI(t) = .375 [.277,.469]*– GBI(t-1) = .875 [.824,.919]*– Avg sentiment = -.0346– Blogs = .678– Forums = .00410– Microblogs = .751
8.75
8.8
8.85
8.9
8.95
9
9.05
9.1
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
Surv
ey Se
ntim
ent
GBI
Sen
timen
t
GBI at t Survey
8.75
8.8
8.85
8.9
8.95
9
9.05
9.1
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
Surv
ey Se
ntim
ent
GBI
Sen
timen
t
GBI at t-1 Survey
![Page 32: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/32.jpg)
GBI and Stock Price(DV=monthly close)
Coeff StdErr p-val
Constant -69.045 34.044 0.070
S&P* 0.104 0.031 0.008
GBI(t) -16.695 10.324 0.137
GBI(t-1) 30.693 10.375 0.014
Adj R-sq .475
Posterior Means
* Closing price in month
Median Coeff
% p-val <.05
Constant -67.096 0.138
S&P* 0.102 1
GBI(t) -15.872 0.012
GBI(t-1) 29.921 0.9892
Adj R-sq .4635
Iteration level
![Page 33: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/33.jpg)
Observed Social Media Metrics(DV=monthly close)
Average Blogs Forums Microblogs
Coeff p-val Coeff p-val Coeff p-val Coeff p-val
Constant -44.550 30.776 0.178 -93.215 31.675 0.015 222.605 52.477
S&P** 0.073 0.026 0.018 0.071 0.021 0.007 -0.014 0.021
SM(t) 34.995 17.181 0.069 6.059 8.055 0.469 -42.767 10.838
SM(t-1) 5.078 18.008 0.784 18.267 8.289 0.052 -39.252 10.840
Adj R-sq 0.404 0.547 0.753 0.390* Closing price in month
![Page 34: Social Media Intelligence: Measuring Brand Sentiment from Online Conversations](https://reader035.vdocument.in/reader035/viewer/2022062400/568168a8550346895ddf44bd/html5/thumbnails/34.jpg)
Conclusions• Social media behavior varies across venue formats
Need to account for the source of SM data
• Potential to use social media as market researchAdjusted measure (GBI) can serve as lead indicator
• Implications for academic research that use social media measures and for practitioners who monitor social media sentiment
• Next steps– Additional data sets– Simulations of different GBI scenarios and resulting metrics