introduction to collaborative filtering

WE KNOW YOU WILL LIKE THISIntroduction to Recommendation Engines

Monday, January 14, 13

Supervised

MLX X + Y

X X + Y

T + YT

Unsupervised

Regression Classification

Y=Turnout

301225

(numeric) Y=ClassSpamNot SpamSpam

(Categorical)

Clustering

Hierarchical Clustering


Recommendation

Content/Model-Based

(predicting the rating)

MarabooKarnaf Ima AdamaLivIdan 5 ? 3 ?Shahar 4 3 ? 2Gadi ? 1 ? 5

(Agnostic, Behavioural)


Rating Problem (Movies)

Preference Problem (Ads)


Related problem: Ranking


Maraboo Karnaf Ima Adama LivIdan 1 ? 1 ?Shahar 1 1 ? 1Gadi ? 1 ? 1



User-based Collaborative Filtering


Pearson’s Correlation 1-Distance

Cosine Similiarity

Jaccard Distance “We share 5 preferences out of 7!”

“Our preferences go in the same direction!”

(but only 2 such preferences do...)

Euclidean Distance

Log-LikelihoodRatio

Measure of “Surprise” at correlation


Item-Based Collaborative Filtering

Usually bounded


Case study: Amazon100,000,000 users

2,000,000 items

Each user expresses preference for 10 items

Each item has 500 reviewsUser-Based CF:

100,000,000 x 100,000,000 similarity matrix

2,000,000 x 500 sum terms

Item-Based CF:

2,000,000 x 2,000,000 similarity matrix

2,000,000 x 10 sum terms


Interpretability

“People who go to La Colombe

Torrefaction & FourSquare HQ tend

to go here”

“Coffee Shop connoisseurs tend

to come here”


EvaluationRating Problem: Predictive accuracy (regression) metrics

RMSE, MAE, etc.

Preference (Binary) Problem: Classification accuracy (IR) metrics

Accuracy, Precision, Recall, F-1, ROC, etc.

Benchmark vs. ‘random’ and ‘popular’

Ranking accuracy metrics: Similarity of permutations

Pearson’s correlation, Spearman’s rho, Kendall’s tau


Challenges

Cold-start problems (new item, new user)

“Black” and “Grey” sheep

Exploration-exploitation and reinforcement learning

Scale


Advanced Topics

Dimensionality Reduction

Map-Reducible calculations

Content-based (feature-based)

Multiple models


MapReduce Similarity Calculation


GadiMaraboo ?Karnaf 1Ima Adama ?Liv 1

GadiIdan 0Shahar 2Gadi 2

* =

A ui

GadiIdan 0Shahar 2Gadi 2

Idan Shahar GadiMaraboo 1 1 ?Karnaf ? 1 1Ima Adama 1 ? ?Liv ? 1 1

*

AT Aui

=Gadi

Maraboo 2Karnaf 4Ima Adama 0Liv 4

“User-based”

AT(Aui)User similarity vector




GadiMaraboo ?Karnaf 1Ima Adama ?Liv 1

* =

A

ui

Idan Shahar GadiMaraboo 1 1 ?Karnaf ? 1 1Ima Adama 1 ? ?Liv ? 1 1

*

AT

=Gadi

Maraboo 2Karnaf 4Ima Adama 0Liv 4

“Item-Based”

(ATA)ui

Maraboo Karnaf Ima Adama LivMaraboo 2 1 1 1Karnaf 1 2 0 2Ima Adama 1 0 1 0Liv 1 2 0 2


ATAItem similarity matrix

Similarity of item x to item y is <i ,i >x y



Recall row outer-product matrix multiplication:





=

+ +

uIdanuIdan uShaharuShahar uGadiuGadiT TT

Only one user’s list of items is used every time!



All of the classic similarity functions aremade up of 3 stages:

Preprocess (uses only one ELEMENT)

Norm (Can be done in reduce on one VECTOR)

Similarity utilizes the A A matrix joinedwith norm entries

T


BibliographyGoogle News Personalization: Scalable Online Collaborative Filtering - Das, Datar, Garg, Rajaram, WWW2007

Logistic Regression and Collaborative Filtering for Sponsored Search Term Recommendation - Bartz, Murthi, Sebastian, EC2006

Evaluating Collaborative Filtering Recommender Systems - Herlocker, Konstan, Tenveen, Riedl, ACM TIS2004

A Survey of Collaborative Filtering Techniques - Su, Khoshgoftaar, AAI2009

An Introduction to Information Retrieval - Manning, Raghavan, Schutze, Cambridge Press

Mahout in Action - Friedman, Dunning, Anil, Owen, Manning Publications

Lessons from the Netflix Prize Challenge - Bell, Koren, KDD2009

Factorization meets the Neighbourhood: a Multifaceted Collaborative Filtering Model - Koren, KDD2008

Accurate Methods for the Statistics of Surprise and Coincidence - Dunning, ACL1993

Item-Based Collaborative Filtering Recommendation Algorithms - Sarwar, Konstan, Karypis, Riedl, WWW2001

Matrix Factorization Techniques for Recommender Systems - Koren, Bell, Volinsky, IEEE2009

recommenderlab: A Framework for Developing and Testing Recommendation Algorithms - Hahsler, 2001

Scalable Similarity-Based Neighbourhood Methods with MapReduce - Schelter, Boden, Markl, RecSys2012


Thanks!

Nimrod Priell n[email protected]@nimrodpriellhttp://www.educated-guess.com


mailto:[email protected]

mailto:[email protected]

http://www.educated-guess.com

http://www.educated-guess.com