real time twitter trend mining system – rt2 m
TRANSCRIPT
Real Time Twitter Trend Mining System – RT²MNigar Gasimli
IndexAbout System overviewCase StudyConclusions and Future works
AboutSocial media data is being
generated in real-timeusing static data to detect
continuously changing social trends such as users’ interests and public issues could be ineffective
About
Real time twitter mining system allows:◦Crawl and store every textual data
(tweets)◦Keep track of social issues by
temporal Topic Modeling◦Visualize mention based user
networks
System overview of RT²M
System overview of RT²M Main page of the system
Term Co-occurrence retrievalCan display the result with an
option of 100, 500, 1,000, and 2,000 terms
Co-occurred terms are dynamically updated and displayed as more Twitter stream data is received
Visualization of Twitter Users
System visualizes the social network graph of Twitter users mentioned together with the query term
User Network AnalysisOpen source visualization tool –
JUNG (Java Universal Network Graph) Framework
Voltage clustering algorithm to detect user community
Similarity Calculation between two usersSimilarity between two users
comparing terms they use in their tweets, weighted by their tf-idf index
TF-IDF = Term Frequency * Inverse document frequency
Topic Modeling
Text mining techniquesMultinomial Topic ModelingCommunity Detection for Social
Network Analysis
Multinomial Topic Modeling
LDA- Latent Dirichey Allocation, represents documents as mixture of topics that spit out words with certain probabilities
An extensin of LDA, DMR – Dirichlet multinomial Regression used in this work
Conclusionmine dynamic social trends and
content-based networks generated in Twitter
Twitter would be a useful medium for keeping track of topical trends
The mention-based user network provides a basis for identifying any nodes with high betweenness
Future Work
Sentiment analysis to Twitter data to observe changes in public
opinion and the formation process of a certain issue, and
ultimately design the prediction model of social issues on social
media.
BibliographyA survey of Topic Modeling in Text
MiningBlei, D., Ng, A., and Jordan, M.
2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3: 993-1022.
Thank you!Any Questions?