personalized recommender systems: mining user preferencerecommender types of recommender systems - 6...
TRANSCRIPT
Personalized Recommender Systems:Mining User Preference
Joonseok LeeGeorgia Institute of Technology
2013/06/10
1
Joonseok Lee
Agenda Introduction Types of Recommender Systems Collaborative Filtering Toolkit PREA and Comparative Study Evaluation of Recommender Systems
- 2 -
Joonseok Lee
Why recommendation?
- 3 -
Joonseok Lee
Product recommendation
Friend recommendation
Rating prediction
Personalized web search
Examples
- 4 -
Joonseok Lee
Agenda Introduction Types of Recommender Systems Collaborative Filtering Toolkit PREA and Comparative Study Evaluation of Recommender Systems
- 5 -
Joonseok Lee
Content-basedRecommender
Types of Recommender Systems
- 6 -
RecommenderSystems
CollaborativeFiltering
Memory-based Model-based
HybridRecommender
Recommendinggood items
Predictingunseen ratings
Maximizing autility function
By approach By goal
Joonseok Lee
Contents-based Filtering
- 7 -
Contents information Demographic data about users
Age, gender, geographic location
Product attributes Movie: genre, director, release year Book: author, language, published year, genre
Depends on domain Domain-specific modeling is necessary. Algorithm is also different domain by domain. With abundant domain knowledge or data, can be very
powerful.
Joonseok Lee
Collaborative Filtering
- 8 -
Collaborative Filtering definition People collaborate to help one another perform filtering, by
recording their reactions to products they consumed. Use other users’ feedback to fit my preference!
Make use of rating data from users only. Direct feedback: rating, , Indirect feedback: click through, page view
Independent of domain Many models and algorithms can work regardless of the
domain.
Joonseok Lee
Agenda Introduction Types of Recommender Systems Collaborative Filtering Toolkit PREA and Comparative Study Evaluation of Recommender Systems
- 9 -
Joonseok Lee
User-based Collaborative Filtering
- 10 -
Find preferred items by similar users to me. Su: a set of similar users to user u. Trust other user’s opinion proportional to the similarity
between (s)he and I.
Joonseok Lee
User-based Collaborative Filtering
- 11 -
1 2 3
1 4
2 5 4
3 1 2
4 5
5 4
4 5
3
2
3 2
Users
Items
80%
50%
0.8 * 4 + 0.5 * 2 0.8 + 0.5 = 3.23
Joonseok Lee
Item-based Collaborative Filtering
- 12 -
Same way to user-based CF, but in column-wise.
More powerful than user-based CF. Item neighbors tend to be more stable than user neighbors.
Joonseok Lee
Memory-based CF Summary
- 13 -
Pros Simple, easy to implement. Can explain reason of recommendation.
Cons Huge memory consumption. Not scalable.
Joonseok Lee
Matrix Factorization
- 14 -
1 2 3
1 4
2 5
3 1 2
4 2 5
5 4
4 5
3
2
3 2
Users
ItemsV
U
4
5
1 2
2 5
4
3
2
3 2
≒
Joonseok Lee
Matrix Factorization
- 15 -
1 2 3
1 4
2 5
3 1 2
4 2 5
5 4
4 5
3
2
3 2
Users
Items
4
5
1 2
2 5
4
3
2
3 2
≒
3 1.4 3 2 1.8
1.3
1.6
0.6
1.4
1.3
Joonseok Lee
Matrix Factorization
- 16 -
1 2 3
1 4
2 5
3 1 2
4 2 5
5 4
4 5
3
2
3 2
Users
Items
3.9
4.8
1.8 1.8
2.0 4.2
3.9
3.2
1.1
2.6 2.3
≒
3 1.4 3 2 1.8
1.3
1.6
0.6
1.4
1.3
Joonseok Lee
Matrix Factorization
- 17 -
1 2 3
1 4
2 5
3 1 2
4 2 5
5 4
4 5
3
2
3 2
Users
Items
3.9 1.8 3.9
4.8 2.2 4.8
1.8 0.8 1.8
4.2 2.0 4.2
3.9 1.8 3.9
2.6 2.3
3.2 2.9
1.2 1.1
2.8 2.5
2.6 2.3
≒
3 1.4 3 2 1.8
1.3
1.6
0.6
1.4
1.3
Joonseok Lee
Matrix Factorization
- 18 -
1 2 3
1 4
2 5
3 1 2
4 2 5
5 4
4 5
3
2
3 2
Users
Items
3.9 1.8 3.9
4.8 2.2 4.8
1.8 0.8 1.8
4.2 2.0 4.2
3.9 1.8 3.9
2.6 2.3
3.2 2.9
1.2 1.1
2.8 2.5
2.6 2.3
≒
3 1.4 3 2 1.8
1.3
1.6
0.6
1.4
1.3
Joonseok Lee
Matrix Factorization Summary
- 19 -
Pros Prediction is (most) accurate.
Cons Computationally expensive. Difficult to explain why we recommend.
Joonseok Lee
Agenda Introduction Types of Recommender Systems Collaborative Filtering Toolkit PREA and Comparative Study Evaluation of Recommender Systems
- 20 -
Joonseok Lee
Why PREA? Since Netflix Prize (2006), a lot of state-of-the-art
algorithms were suggested. They are implemented only by the authors. Different language, different dataset, different evaluation
measures.
Standardized implementation is needed for fair comparison of CF algorithms. PREA implements those algorithms on the same interface.
- 21 -
Joonseok Lee
Features
- 22 -
Joonseok Lee
Comparative Study Motivation: each algorithm may perform well in
different situation. Observe how well each algorithm performs depending on dataset size and density.
Three variables: User count, Item count, Density Dataset: Netflix data Evaluation measures
MAE, RMSE Asymmetric measure
- 23 -
Joonseok Lee- 24 -
Accuracy tends to get better with larger and denser data.
Some algorithms highly depend on dataset size, only in sparse cases. (PMF, Slope1, etc.)
Shape of contour lines are different. (Constant is horizontal, while ItemAvg vertical.)
Joonseok Lee
Best-performing algorithm
- 25 -
The identity of the best-performing algorithm is non-linearly dependent on user/item count, density.
NMF is dominant in low density cases, while BPMFworks well for high density cases and larger dataset.
Regularized SVD and PMF perform well for density levels 2%-4%.
Joonseok Lee
Overall Comparison
- 26 -
Joonseok Lee
Conclusions
- 27 -
Performance of all algorithms depend on dataset size and density, but the nature of dependency varies a lot.
Matrix-factorization methods generally have the highest accuracy. NMF works well for sparse data.
There is general trade-off between accuracy and other factors such as low variance on dataset, computational efficiency, memory consumption, and small number of adjustable parameters.
Joonseok Lee
Agenda Introduction Types of Recommender Systems Collaborative Filtering Toolkit PREA and Comparative Study Evaluation of Recommender Systems
- 28 -
Joonseok Lee
Online Evaluation Test with real users, on a real situation!
Set up several recommender engines on a target system. Redirect each group of subjects to different recommenders. Observe how much the user behaviors are influenced by the
recommender system.
Limitation Very costly. Need to open imperfect version to real users.
May give negative experience, making them to avoid using the system in the future.
- 29 -
Joonseok Lee
Offline Experiments Filtering promising ones before online evaluation!
Train/Test data split Learn a model from train data, then evaluate it with test data.
How to split: Simulating online behaviors Using timestamps, allow ratings only before it rated. Hide ratings after some specific timestamps. For each test user, hide some portion of recent ratings. Regardless of timestamp, randomly hide some portion of
ratings.
- 30 -
Joonseok Lee
Predicting Ratings Goal: Evaluate the accuracy of predictions. Popular metrics:
Root of the Mean Square Error (RMSE)
Mean Average Error (MAE)
Normalized Mean Average Error (NMAE)
- 31 -
Joonseok Lee
Recommending Items Goal: Suggesting good items (not discouraging bad
items) Popular metrics:
- 32 -
THE END
Thank you for your attention!
33