Analyzing User Modeling on Twitter for Personalized News RecommendationsBest Paper of Conference on User Modeling, Adaption and Personalization (UMAP’11)Authors: Fabian Abel, Qi Gao, Geert-Jan Houben and Ke Tao

Unit for Social Software

Presenter: Guangyuan Piao

Reading Group, 29/09/2015

Contents• Background & Related Work

• Design Options for User Modeling in Online Social Networks for Recommendations

• Research Questions

• Dataset for the study

• Study for Research Questions 1,2

• Experiment for Research Question 3

2/21

3/21

User Modeling in Online Social Networks for Recommendations

4/21

Related Work

Representation of a User Model• bag-of-words, Chen et al. for recommending and ranking URLs

posted in Twitter messages

Study of hashtags in Twitter• investigate the specificity, stability over time, Laniado & Mika• temporal dynamics of hashtags, Laniado & Mika / Huang et al.

Enrichment of tweets• Exploit metadata of research papers to enrich the semantics of

tweets for mapping tweets to conference talks, Rowe et al.

5/21

Design Options for User Modeling

Semantic Enrichment

Profile Type

Time Constraint

1. tweet-based2. further

enrichment1. hashtag-based2. entity-based3. topic-based

1. time period2. temporal patterns

User Profile

Google

… iPhone

0.09 … 0.08P(u) =

link news

monitorednews pool

an entity-based profile

6/21

Research Questions1. how does the semantic enrichment impact the

characteristics and quality of Twitter-based profiles?

2. how do (different types of) profiles evolve over time? Are there any characteristic patterns?

3. how do the different user modeling strategies impact personalization (personalized news recommendation) and does the consideration of temporal patterns improve the accuracy of the recommendations?

7/21

Dataset for the study Collected from Twitter with more than

• 20,000 Twitter users

• 2 months

• 10,000,000 tweets

• 75,000 news articles

Sample dataset for study

• 1,619 users with at least 20 tweets & 1 tweet/month (2,316,204 tweets)

8/21

Research Question 1

how does the semantic enrichment impact

the characteristics and quality of Twitter-based profiles?

9/21

• tweet-only-based user modeling, fails to create profiles for 100 users

• by enrichment with entities and topics obtained from linked news articles ➡ a higher # of distinct concepts and variety for per profile

10/21

The impact of news-based enrichment

• entity-based and topic-based strategies have higher coverage

Comparison of different types of profiles

11/21

Research Question 2

how do (different types of) profiles evolve over time?

are there any characteristic patterns?

12/21

• profile difference is measured by d1-distance

• user profiles change over time: the older a profile the more it differs from the current profile of the user

Profile difference over time

13/21

d1 = 2

iPad iPhone

Px(u) 1 0Py(u) 0 1

• the difference of weekday and weekend profiles is higher than that of other temporal patterns

Profile difference of weekday and weekend

14/21

• hashtag-based & entity-based profiles change most while the types of entities people refer to (person, product etc.) do not differ strongly

Profile difference of weekday and weekend

15/21

how do the different user modeling strategies impact personalization (personalized news

recommendation)?

and does the consideration of temporal patterns improve the accuracy of the recommendations?

Research Question 3

16/21

Evaluation Setup for News RecommendationsMain goal: analyze and compare the applicability of the different user modeling strategies in the context of news recommendations.

Recommendation algorithm: cosine similarity between user profile and tweets.

Evaluation Metrics:• MRR (Mean Reciprocal Rank): at which rank the first item

relevant to the user occurs on average.• S@k (Success at rank k): the mean probability that a relevant

item occurs within the top k of the ranking, k=10.

observation time period for Twitter

last week

used for user modelingget candidate news items

17/21

Results for News Recommendations – impact of news-based enrichment

• entity-based strategy is better than others

• exploiting both tweets and linked news articles for creating user profiles improves the performance significantly (p < 0.05)

18/21

Results for News Recommendations – impact of considering temporal patterns

• fresh user profiles for topic-based user modeling are more applicable for recommending news articles, while complete user profiles for entity-based user modeling yields better recommendations.

• similar patterns can be found for weekend news recommendations.

(fresh profile: two weeks before recommendation time)

19/21

Conclusions

Further enrichment with semantics extracted from news articles• enhanced the variety of the constructed profiles • improved the accuracy of news article recommendations

Temporal dynamics of user profiles• user profiles change over time• user vary from weekdays and weekends• consideration of temporal dynamics is beneficial to news

recommendations for topic-based user modeling strategy

20/21