1

Collaborative Filtering

Rong JinDepartment of Computer Science and EngineeringMichigan State University

2

Outline Brief introduction information filtering Collaborative filtering

Major issues in collaborative filtering Main methods for collaborative filtering Flexible mixture model for collaborative filtering Decoupling model for collaborative filtering

3

Short vs. Long Term Info. Need Short-term information need (Ad hoc retrieval)

“Temporary need”, e.g., info about used cars Information source is relatively static User “pulls” information Application example: library search, Web search

Long-term information need (Filtering) “Stable need”, e.g., new data mining algorithms Information source is dynamic System “pushes” information to user Applications: news filter

4

Examples of Information Filtering News filtering Email filtering Movie/book/product recommenders Literature recommenders And many others …

5

Information Filtering Basic filtering question: Will user U like item X? Two different ways of answering it

Look at what U likes characterize X content-based filtering

Look at who likes X characterize U collaborative filtering

Combine content-based filtering and collaborative filtering

6

Other Names for Information Filtering Content-based filtering is also called

“Adaptive Information Filtering” in TREC “Selective Dissemination of Information” (SDI)

in Library & Information Science Collaborative filtering is also called

Recommender systems

7

Example: Content-based Filtering

Description:A homicide detective and a fire marshall must stop a pair of murderers who commit videotaped crimes to become media darlings

Rating:

Description: Benjamin Martin is drawn into the American revolutionary war against his will when a brutal British commander kills his son.

Rating:

Description: A biography of sports legend, Muhammad Ali, from his early days to his days in the ring

Rating:

History What to Recommend?Description: A high-school boy is given the chance to write a story about an up-and-coming rock band as he accompanies it on their concert tour.

Recommend: ?

Description: A young adventurer named Milo Thatch joins an intrepid group of explorers to find the mysterious lost continent of Atlantis.

Recommend: ?

No

Yes

8

Example: Collaborative Filtering

User 1 1 5 3 4 3

User 2 4 1 5 2 5User 3 2 ? 3 5 4

User 3 is more similar to user 1 than user 2

5 for movie “15 minutes” for user 3

5

9

Collaborative Filtering (CF) vs. Content-based Filtering (CBF) CF do not need content of items while CBF

relies the content of items CF is useful when content of items

are not available or difficult to acquire are brief and insufficient

Example: movie recommendation A movie is preferred may because

its actor its director its popularity

10

Application of Collaborative Filtering

11?

Collaborative Filtering Goal: Making filtering decisions for an individual user based

on the judgments of other users

u1

u2

…

um

Users: U

Objects: O

o1 o2 … oj oj+1… on

3 1 …. … 4 2 ?

2 5 ? 4 3

? 3 ? 1 2

utest 3 4…… 1

12

Collaborative Filtering Goal: Making filtering decisions for an individual user based

on the judgments of other users

General idea Given a user u, find similar users {u1, …, um}

Predict u’s rating based on the ratings of u1, …, um

13

Example: Collaborative Filtering

User 1 1 5 3 4 3

User 2 4 1 5 2 5User 3 2 ? 3 5 4

User 3 is more similar to user 2 than user 1

5 for movie “15 minutes” for user 3

5

14

Memory-based Approaches for CF The key is to find users that are similar to the test user Traditional approach

Measure the similarity in rating patterns between different users Example: Pearson Correlation Coefficient

0

0 00

,

,

( ( ) )ˆ ( )

y y y yy Y

y yy y

y Y

w R x RR x R

w

0 0

0

( )^ ( ), 2 2

( )^ ( ) ( )^ ( )

( ( ) )( ( ) )

( ( ) ) ( ( ) )o

o o

y y y yx X y X y

y yy y y y

x X y X y x X y X y

R x R R x R

wR x R R x R

15

Pearson Correlation Coefficient for CF Similarity between a training user y and a test user y0: 0,y yw

( ) : the rating of object given by user

: the average rate given by user

( ) : the set set of objects rated by user

y

y

R x x y

R y

X y y

0 0

0

( )^ ( ), 2 2

( )^ ( ) ( )^ ( )

( ( ) )( ( ) )

( ( ) ) ( ( ) )o

o o

y y y yx X y X y

y yy y y y

x X y X y x X y X y

R x R R x R

wR x R R x R

Remove the rating bias from each training user

Normalized Rate: ( )y yR x R

16

Pearson Correlation Coefficient for CF Estimate ratings for the test user

0

0 00

,

,

( ( ) )ˆ ( )

| |

y y y yy Y

y yy y

y Y

w R x RR x R

w

Weighted vote of normalized rates

17

Example

User 1 1 5 3 4 3

Normalized Rate

User 2 4 1 5 2 5

Normalized Rate

User 3 2 ? 3 5 4

Normalize Rate

0,y yw

18

Example

User 1 1 5 3 4 3

Normalized Rate -2.2 1.8 -0.2 0.8 -0.2

User 2 4 1 5 2 5

Normalized Rate 0.6 -2.4 1.6 -1.4 1.6

User 3 2 ? 3 5 4

Normalize Rate -1.5 -0.5 1.5 0.5

0,y yw

19

Example

User 1 1 5 3 4 3

Normalized Rate -2.2 1.8 -0.2 0.8 -0.2 0.85

User 2 4 1 5 2 5

Normalized Rate 0.6 -2.4 1.6 -1.4 1.6 -0.49

User 3 2 ? 3 5 4

Normalize Rate -1.5 -0.5 1.5 0.5

0,y yw

0.85 1.8 0.49 (-2.4) 3.5 0.85 0.49

5.5

r

20

Problems with Memory-based Approaches

User 1 ? 5 3 4 2

User 2 4 1 5 ? 5

User 3 5 ? 4 2 5

User 4 1 5 3 5 ?

Most users only rate a few items Two similar users can may not rate the same set of items Clustering users and items

21

Flexible Mixture Model (FMM)Cluster both users and items simultaneously

User 1 ? 5 3 4 2

User 2 4 1 5 ? 5

User 3 5 ? 4 2 5

User 4 1 5 3 5 ?

User clustering and item clustering are correlated !

22

Flexible Mixture Model (FMM)Cluster both users and items simultaneously

User Class I 1 p(4)=1/4p(5)=3/4

3

User Class II p(4)=1/4p(5)=3/4

p(1)=1/2p(2)=1/2

p(4)=1/2p(5)=1/2

Movie Type I

Movie Type II

Movie Type III

Unknown ratings are gone!

23

Flexible Mixture Model (FMM)

Zo Zu

O U R

uo ZZ

uoluloluolll ZZrPZuPZoPZPZPruoP,

)()()()()()( ),|()|()|()()(),,(

P(o|Zo) P(u|Zu) P(Zo) P(Zu)

P(r|Zo,Zu)

Zu: user class

Zo: item class

U: user

O: item

R: rating

Hidden variable

Observed variable

24

Annealed Expectation Maximization (AEM) algorithm E-step: calculate posterior probability for hidden

variables zu and Zo

b: temperature for Annealed EM algorithm M-step: updated parameters

Flexible Mixture Model: Estimation

uo ZZ

buoluloluo

buoluloluo

llluo ZZrPZuPZoPZPZPZZrPZuPZoPZPZP

ruozzP

,)()()(

)()()()()()( )),|()|()|()()((

)),|()|()|()()((),,|,(

),|();|();|();();( )()()( uoluloluo ZZrPZuPZoPZPZP

25

Flexible Mixture Model: Predication

Fold-in process Repeat the EM algorithm including ratings from

the test user Fix all the parameters except for P(ut|zu)

uo ZZ

uout

ouolt ZZrPZuPZoPZPZPruoP

,)( ),|()|()|()()(),,(

Key issue:What user class does the test user belong to ?

26

Another Prob. with Memory-based Approaches

User 1 2 5 3 4 2

User 2 4 1 4 1 3

User 3 5 2 5 2 5

User 4 1 4 2 3 1

Users with similar interests can have different rating patterns Decoupling preference patterns from rating patterns

27

Decoupling Model (DM)

Zo Zu

O U

Hidden variableObserved variable

Zu: user classZo: item classU: userO: itemR: rating

28

Decoupling Model (DM)

Zu: user classZo: item classU: userO: itemR: ratingZpref : whether users like items

Zo Zu

O U

Zpref

Hidden variableObserved variable

29

Decoupling Model (DM)

Zu: user classZo: item classU: userO: itemR: ratingZpref : whether users like items ZR: rating class

Zo Zu

O U R

Zpref

ZR

Separating preference and rating patterns User class + Rating class rating R

Zu Zpref and ZR +Zpref r

Hidden variableObserved variable

30

Experiment Datasets: EachMovie and MovieRating

Evaluation: Mean Absolute Error (MAE): average absolute deviation of the

predicted ratings to the actual ratings on items.

The smaller MAE, the better the performance

MovieRating EachMovieNumber of Users 500 2000Number of Items 1000 1682Avg. # of rated items/User 87.7 129.6Number of ratings 5 6

|)(|1 ^

)()( )( lyl

lTest

xRrL

MAEl

31

Experiment Protocol Test the sensitivity of the proposed model to the

amount of training data Vary the number of training users MovieRating dataset: 100 and 200 training users EachMovie dataset: 200 and 400 training users

Test the sensitivity of the proposed model to the information needed for the test user Vary the number of rated items provided by the test user

5, 10, and 20 items are given with ratings

32

Experimental Results:FMM and other baseline algorithms

0.750.770.790.810.830.850.870.89

0.750.770.790.810.830.850.870.89

Movie Rating, 100 Training Users

Movie Rating, 200 Training Users

0.90.951

1.051.11.151.21.251.3

0.90.951

1.051.11.151.21.251.3

Each Movie, 400 Training Users

Each Movie, 200 Training Users

Given: 5 10 20 Given: 5 10 20

Given: 5 10 20Given: 5 10 20

MAE

MAE

MAE

MAE

A smaller MAE indicates better

performance

33

FMM vs. DM

Results on Movie Rating

Results on Each Movie

Training Users Size Algorithms 5 Items

Given 10 Items

Given20 Items Given

100FMM 0.829 0.822 0.807

DM 0.791 0774 0.751

200FMM 0.800 0.787 0.768

DM 0.770 0.753 0.730

Training Users Size Algorithms 5 Items

Given10 Items

Given20 Items Given

200FMM 1.07 1.04 1.02

DM 1.06 1.02 1.00

400FMM 1.05 1.03 1.01

DM 1.04 1.01 0.99

Smaller value indicates better performance