real time twitter trend mining system – rt2 m

Real Time Twitter Trend Mining System – RT²MNigar Gasimli

IndexAbout System overviewCase StudyConclusions and Future works

AboutSocial media data is being

generated in real-timeusing static data to detect

continuously changing social trends such as users’ interests and public issues could be ineffective

About

Real time twitter mining system allows:◦Crawl and store every textual data

(tweets)◦Keep track of social issues by

temporal Topic Modeling◦Visualize mention based user

networks

System overview of RT²M

System overview of RT²M Main page of the system

Term Co-occurrence retrievalCan display the result with an

option of 100, 500, 1,000, and 2,000 terms

Co-occurred terms are dynamically updated and displayed as more Twitter stream data is received

Visualization of Twitter Users

System visualizes the social network graph of Twitter users mentioned together with the query term

User Network AnalysisOpen source visualization tool –

JUNG (Java Universal Network Graph) Framework

Voltage clustering algorithm to detect user community

Similarity Calculation between two usersSimilarity between two users

comparing terms they use in their tweets, weighted by their tf-idf index

TF-IDF = Term Frequency * Inverse document frequency

Topic Modeling

Text mining techniquesMultinomial Topic ModelingCommunity Detection for Social

Network Analysis

Multinomial Topic Modeling

LDA- Latent Dirichey Allocation, represents documents as mixture of topics that spit out words with certain probabilities

An extensin of LDA, DMR – Dirichlet multinomial Regression used in this work

Conclusionmine dynamic social trends and

content-based networks generated in Twitter

Twitter would be a useful medium for keeping track of topical trends

The mention-based user network provides a basis for identifying any nodes with high betweenness

Future Work

Sentiment analysis to Twitter data to observe changes in public

opinion and the formation process of a certain issue, and

ultimately design the prediction model of social issues on social

media.

BibliographyA survey of Topic Modeling in Text

MiningBlei, D., Ng, A., and Jordan, M.

2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3: 993-1022.

Thank you!Any Questions?