boston dataswap topic modeling by alice oh

Topic Models & Computational Social Science

October 17, 2013Alice [email protected]@seas.harvard.eduhttp://uilab.kaist.ac.kr/members/aliceoh/

Thursday, October 17, 2013

mailto:[email protected]




http://uilab.kaist.ac.kr/members/aliceoh/

http://uilab.kaist.ac.kr/members/aliceoh/

What is topic modeling?


Blei, Communications of the ACM, 2012


Motivation


Motivation

• What are the topics discussed in the article?

• Is the article related to

• household finances?

• price of gasoline?

• price of Apple stock?

• How would you build an automatic system for answering these questions?


http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?hp

nascar, races, track, raceway, race, cars, fuel, auto, racing

economic, slowdown, sales, recession, costs, spending, save

fans, spectators, sports, leagues, teams, competition6






fans, spectators, sports, leagues, teams, competition

Topics: multinomial over wordsThursday, October 17, 2013




Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013

http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?




Topics: multinomial over wordsTopic DistributionsThursday, October 17, 2013



Input to LDA

8


Input to LDA

8

http://www.nytimes.com/2010/08/09/sports/autoracing/09nascar.html?




Topics Discovered by LDA

nascar 0.12 spending 0.09 sports 0.12

races 0.10 economic 0.07 team 0.11

cars 0.10 recession 0.06 game 0.10

racing 0.09 save 0.05 player 0.10

track 0.08 money 0.05 athlete 0.09

speed 0.06 cut 0.04 win 0.07

... ... ...

money 0.002 speed 0.003 nascar 0.001

Topics: multinomial over vocabulary9


Graphical View

10


Graphical View

sales xxx slowdown recession cars races spending xxx save costs fuel

10

Observed


Graphical View

Topics

sales xxx slowdown recession cars races spending xxx save costs fuel

10

Observed




Topics: multinomial over words

Discovered

Topic Distributions

Discovered


Do you feel what I feel?Social Aspects of Emotions in Twitter Conversations

Suin Kim, JinYeong Bak, Alice OhICWSM 2012

11


Twitter conversation data

• Twitter conversation data: approx 220k dyads who “reply” to each other, 1,670k conversational chains (We now have about 5x this amount)

!"!

#!

$!

%!


Asking Research Questions

13


Asking Research Questions

Human emotion is typically studied as a within-person, one-direction, non-repetitive phenomenon; focus has traditionally been on how one individual feels in reaction to various stimuli at a certain point of time. But people recognize and inevitably react emotionally and otherwise to expressions of emotion of other people. We propose that organizational dyads and groups inhabit emotion cycles: Emotions of an individual influence the emotions, thoughts and behaviors of others; others’ reactions can then influence their future interactions with the individual expressing the original emotion, as well as that individual’s future emotions and behaviors. People can mimic the emotions of others, thereby extending the social presence of a specific emotion, but can also respond to others’ emotions, extending the range of emotions present.

14


Topic model with a twist

• Dirichlet forest prior (Andrzejewski et al.)

• Mixture of Dirichlet tree distribution

• Dirichlet tree: Generalization of Dirichlet distribution

• Knowledge is expressed using Must-link and Cannot-link primitives

• Must-link(love, sweetheart)

• Cannot-link(exciting, bored)

15DF-LDA


Topic model with a twist

• Dirichlet forest prior (Andrzejewski et al.)

• Mixture of Dirichlet tree distribution

• Dirichlet tree: Generalization of Dirichlet distribution

• Knowledge is expressed using Must-link and Cannot-link primitives

• Must-link(love, sweetheart)

• Cannot-link(exciting, bored)

15

qβ

η

DF-LDA


Domain knowledge in Dirichlet forest prior

16

Seed Words

anticipationhopewaitawaitinspirexcitborereadiexpectnervoucalmmotivpreparcertainanxiouoptimistforese

joyawesomamazwonderexcitgladfinebeautihighluckisuperperfectcompletspecialblesssafeproud

angershitbitchassmeandamnmadjealoupissannoiangriupsetmoronragescrewstuckirrit

surpriseamazwowwonderweirdluckidiffer

awkwardconfusholistrangshockodd

embarrassoverwhelmastoundastonish

fearscarestresshorrornervouterroralarmbehindpanicfearafraiddesperthreatentensterrififrightanxiou

sadnesssorribadawsadwronghurtbluedeadlostcrushweakdepressworslowterribllone

disgustsickwrongevilfatuglihorriblgrossterriblselfishmiserpathetdisgustworthlessaw

ashamfuck

acceptanceokaioksamealrightsafelazirelaxpeaccontentnormalsecurcompletnumbfulfil

comfortdefeat

Must-link within a class Cannot-link between classes


Emotion Topics How do we express emotions?

JoyAnticipation AngerTopic 114omglovehahathankreallyTopic 107lovethankfollowwow

Topic 159gooddayhopemorningthankTopic 158lovethankmisshug

Topic 125hopebetterfeelthanksoonTopic 26goodthankhopemiss

Topic 146comewaitweekdayjuneTopic 146gooddaytimework

Topic 131lmaofuckassbitchshitTopic 4assyolmaonigga

Topic 19lmaoshitdamnfuckohTopic 13shitniggasmhyea

FearTopic 48omgohlmaoshitscareTopic 78happenheartattackhospital

Topic 27don’tcomenightsleepoutsideTopic 140timegotworkday

SurpriseTopic 172yeagknowthinktruefunnyTopic 89knowdon’tthinklook

Topic 15thinkdon’tknowmakereallyTopic 94hahadontthinkreally

29 70 21 14 5

Sadness DisgustTopic 6ohsorryhahaknowdidntTopic 59hurtgotgoodbad

Topic 106tweetreplydidn’treadsorryTopic 155ohreallymakefeel

Topic 116ohfuckdon’tyeewTopic 116lookhahaohknow

Topic 22don’tohthinkyeahlmaoTopic 174don’tthinksaypeople

AcceptanceTopic 43okohthankcoolokayTopic 102knowtryletok

Topic 199xxthankgoodokayfollowTopic 8nightlovegoodsleep

17 7 18 NeutralTopic 180comwwwhttpcheckyoutubeTopic 156twitterfacebookpeopleaccount

Topic 184accountgoogleappworkemailTopic 67foodchickencookrt

19

17


Emotion Topics How do we express emotions?

JoyAnticipationTopic 114omglovehahathankreallyTopic 107lovethankfollowwow

Topic 125hopebetterfeelthanksoonTopic 26goodthankhopemiss

SadnessTopic 6ohsorryhahaknowdidntTopic 59hurtgotgoodbad

NeutralTopic 180comwwwhttpcheckyoutubeTopic 156twitterfacebookpeopleaccount

GreetingCaring Sympathy IT/Tech

18


Emotion-tagged conversations

19

A (Love): @amithpr @dhempe @OperaIndia - Would you have any update on @mrunmaiy's health - hope she is recovering well?B (neut): @labnol @dhempe she is recovering but slow. The injury is on the spine therefore worrisome. Still in icu.A (Sadness): @amithpr thanks for the update.. extremely said to hear that news..B (neut): @labnol #prayformrun She is a fighter and will come out of this

B (neut): @AyeItsMeiMei just tell ur followers to report her for spam. then she'll be kicked off twitterA (Anger): @Jakeosaurous dude I didn't even do shit to her I'm just here tweeting & she calls me a ugly bitch? I was like oh wow thanks?B (neut): @AyeItsMeiMei yeah clearly shes so ugly she cant even use her real pic:P so dont feel badA (Love): @Jakeosaurous haha. I don't care. She's getting spammed with hate. Hahaha. (": thanks though.B (neut): @AyeItsMeiMei np


Emotion Transitions Plutchik’s Wheel of Emotions

Joy39.7%

0.51

Acceptance10.4%

0.23

Fear2.6%

0.11

Surprise7.4%

0.17

Anticipation15.1%

0.26

Disgust2.9%

0.11

Sadness9.1%

0.19

0.31Anger12.8%

0.37

0.33

0.32

0.31

0.33

0.21

0.34

0.15

0.140.13

0.15

20


Defining “Influence”

User A

User B

Having a tough day today. RIP Harrison. I’ll

miss you a ton :/

Just pray about it. God will help you.

Not really religious, but thanks man. :)

If you need talk you know I’m here.

Time

(Sadness) (Acceptance)

(Anticipation)

21


Defining “Influence”

emotion influencing tweet

User A

User B

Having a tough day today. RIP Harrison. I’ll

miss you a ton :/

Just pray about it. God will help you.

Not really religious, but thanks man. :)

If you need talk you know I’m here.

Time

(Sadness) (Acceptance)

(Anticipation)

21


Topic 117tweetpeopledon’treadpostTopic 59hurtgotbadpainfeel

Emotion Influences What can you say to make your partner feel better?

Joy → SadnessSadness → Joy

Topic 18wearlookthinkloveblackTopic 24lovethankgreatnewlook

Anticipation → Surprise

Topic 96musiclistenplaysonggoodTopic 178followtweetpeopletwitterthank

Acceptance → Anger

Topic 31i’mgotlmaxshitdaTopic 13lmaoshitniggasmhyea

Disgust → Joy

Topic 61watchnewlivetvtonightTopic 63watchgoodthinkknowlook

Suggesting GreetingSympathy

Swear words Complaining

22


Self-disclosure and relationship strength in online conversations

JinYeong Bak, Suin Kim, and Alice OhACL 2012

23


2012-07-11

Methodology} Twitter Data} 131K users } 2M conversations

} Relationship Strength} Chain frequency (CF)} Chain length (CL)

} Self-Disclosure} Personal information} Open communication} Profanity

} Analysis with Topic Models} Latent Dirichlet allocation (LDA, [Blei, JMLR 2003])} Aspect and sentiment unification model (ASUM, [Jo, WSDM 2011])

24


2012-07-11

Relationship Strength} Social psychology literature states relationship strength can be

measured by communication frequency and length [Granovetter, 1973;

Levin and Cross, 2004]} CF: chain frequency} The number of conversational chains between the dyad

averaged per month} CL: chain length} The length of conversational chains between the dyad

averaged per month} Relationship strength} A high CF or CL for a dyad means the relationship is strong} A low CF or CL for a dyad means the relationship is weak

25


2012-07-11

Self-Disclosure} Open communication - Openness} Negative openness} Nonverbal openness} Emotional openness} Receptive openness – difficult to find in tweets} General-style openness – not clearly defined in the literature

} Personal Information} Personally Identifiable Information (PII)} Personally Embarrassing Information (PEI)

} Profanity} nigga, ass, wtf, lmao

26


2012-07-11

Negative openness

} Method} We use ASUM with emoticons as seed words

[ “Aspect and sentiment unification model for online review analysis”, Jo, WSDM’11]} ASUM is LDA-based joint model of topic and sentiment} ASUM takes unannotated data and classifies each sentence (tweet) as

positive/negative/neutral

Self-Disclosure - Openness

27


2012-07-11

Self-Disclosure - OpennessNonverbal openness

} Method} We look for emoticons, ‘lol’, ‘xxx’} Emoticons are like facial expressions -- :) :( :P} ‘lol’ (laughing out loud) and ‘xxx’ (kisses) are very frequently used in a

similar manner to nonverbal openness

28


2012-07-11

Self-Disclosure - OpennessEmotional openness

} Method} Look for tweets that contain common expressions of feeling words

[We feel fine (Harris, J, 2009)]

29


2012-07-11

Self-Disclosure – Personal InformationPersonally Identifiable Information (PII)

Personally Embarrassing Information (PEI)

30

Ex) name, location, email address, job,social security number

Ex) clinical history,sexual life,job loss, family problem


2012-07-11

Self-Disclosure – Personal Information}

31


2012-07-11

Self-Disclosure – Personal InformationExample of PII, PEI and Profanity topics } Shown by high probability words in each topic

PII 1 PII 2 PEI 1 PEI 2 PEI 3 Profanity

san tonight pants teeth family nigga

live time wear doctor brother lmao

state tomorrow boobs dr sister shit

texas good naked dentist uncle ass

south ill wearing tooth cousin bitch

32


2012-07-11

Results


2012-07-1134

weak ßà strong weak ßà strong

weak ßà strong weak ßà strong

sentiment nonverbal emotional profanity PII & PEI


2012-07-1135

weak ßà strong

weak ßà strong

emotional PII & PEI

weak ßà strong

weak ßà strong


2012-07-11

Results: Interpretation} Emotional openness} When they are not very close, they express frequent encouragements,

or polite reactions to baby or pets

36


2012-07-11

Results: Interpretation} PII} When they meet new acquaintances, they use PII to introduce

themselves

37


2012-07-11

ResultsAnalyzing outliers: a dyad linked weakly but shows high self-disclosure

38


Computational Analysis of Agenda Setting Theory

Yeooul Kim and Alice [email protected]




Agenda Setting Theory How does media affect the thoughts of the audience?


Agenda Setting Theory (McCombs & Shaw, 1972)

• Media affects audiences by having an influence on

• What to think about

• How to think about it

• Examples of traditional media studies

• Media affects the outcome of presidential elections (Perloff and Krauss, 1985)

• Media coverage influences the control of infectious diseases (Cui et al., 2008)

• Tone of news articles affects the number of visitors to museums (Zyglidopoulos et al., 2012)


1.Use of traditional off-line newspapers and TV as target media

• Analysis is limited to a small volume over a short duration

• Issues are arbitrarily chosen

2.Use of off-line MIP (Most Important Problems) surveys

• Self-reports are not reliable

• Only a small subset of the population can be surveyed

3.Use of manual coding for content analysis

• You need experts

• It is difficult to replicate and generalize to other domains

Limitation of Traditional Media Studies


Computational Analysis of Agenda Setting Theory

1.Use of traditional off-line newspapers and TV as target media

• Crawl online news to get several years’ data

• Use machine learning to automatically discover the important issues

2.Use of off-line MIP (Most Important Problems) surveys

• Look at counts of social media shares

• Look at counts of user comments

3.Use of manual coding for content analysis

• Use unsupervised machine learning to analyze content for tone (polarity) of articles and comments

• Try it for different issues to see whether ML approach can generalize over many domains


44

Gay marriageCOMMENT

SHARE

AUDIENCE’S BEHAVIOR


45

Section #Articles #Comments #Commenters #Shares

Politics 1,863 174,680 14,106 2,080,889

Business 2,043 130,921 17,791 3,657,544

Opinion 4,820 149,618 30,556 6,620,489

Sports 814 17,282 5,484 712,507

Technology 456 13,571 4,993 570,732

Science 945 50,113 11,114 4,709,041

World 3,673 134,572 14,882 3,534,637

Health 3,060 92,964 18,185 6,001,082

Total 17,674 763,721 117,111 27,886,921

From http://www.npr.org/

2011.01 – 2013.04

DATA STATISTICS


http://www.npr.org/

http://www.npr.org/

http://www.npr.org/

http://www.npr.org/

46

Section Issue (Labeled by using Mturk) #Articles

Politics presidential electioninfringement of human rightsrace for Washingtongovernment economics presidential campaigns and money candidate-marriage & immigration political viewpoints

575195167274163261157

Business economic decline under Obamaemployment and paid slavery agriculturebanks and loan stock market and business housing markettax and businessenergy and finance new business and running

514218131198166170180222138

Health health care reform laws vaccinationHIV and treatment medication healthcare and costs food and obesitysleep study and children food and safety health tech and new treatment mental health in families

349189496197224245210223125117

Issue Detection using HDP

Detected Issue list and the number of articles of each issue for three sections out of eight sections.


47

▶ Effects from media exposure CORRELATION IN ISSUE


Contentious Issues

48


Contentious Issues

49


INFLUENTIAL FACTOR Tone (Polarity) of article

GOALIdentify the effects of article tone, positive and negative, on the commenting and sharing behaviors of the audience

50

Content Polarity & Audience Behavior


51

ARTICLE POLARITY


52

DETECTED POS./NEG. WORDS

The sets of positive and negative words obtained from model analysis for news articles. Words depending on sections differentiate positive and negative traits of each section.

BUSINESS HEALTH OPINION POLITICS Positive joined viral smoothly better balance respect forward empower fair moderate

Negative cutthroat axed lawsuit beating lose opposite battle unjust fuming sequester

Positive care respect admit clarify essential healthy repair benign hope repaired

Negative tough severe emergency affected risk dying war spitting tricks abnormal

Positive spectacular useful created prize confirm love sublime win confident mellow

Negative weird fog distressing slam doubted fail wrong fears slippery peril

Positive expert forward proud consent carol rights great worth integrity truth

Negative ironic heinous arguing dick undo grinding outlaw meaningless theft lost

SCIENCE SPORTS TECHNOLOGY WORLD Positive fortunate cleanup essential credit safety comforting milestone learn gang dim

Negative spill crude busted upset concern problems dark smash prize creating

Positive victory won grace fun champion passion ace belief luck balance

Negative chase shock busted beating defeat thwart lost alleged assault cockeyed

Positive best fancy easy help intelligence strong improve fit trust fame

Negative blocks shabby shy wicked rash shaky mortal grave pity unfinished

Positive free respected support moderate consistent prompt afford gratitude joined affluent

Negative tension protest heavy raging slam war crime oppress poverty poor


53

Positive and Negative Articles


For more information

David Blei’s homepage:h2p://www.cs.princeton.edu/~blei/

David Mimno’s bibliography:h2p://www.cs.princeton.edu/~mimno/topics.html

videolectures.net – David Blei, Yee-‐Whye Teh, Michael JordanConferences: NIPS, ICML, UAI, ECML, KDD, EMNLP

Tools: Mallet, GenSym, various LDA libraries

Email me: [email protected]


http://www.cs.princeton.edu/~blei/

http://www.cs.princeton.edu/~blei/

boston dataswap topic modeling by alice oh

Sports

slowdown recession cars

costs fuel

graphical viewobserved

competitiontopic distributions

social aspects of emotions

individuals future emotions

emotion cycles

range of emotions present