Detection of Profile Injection Attacks

in

Recommender System

Presented By:Ashish Pannu

2

Agenda• Recommender System

- Introduction- Types of Recommender System - Why using Recommender System

• Profile Injection Attacks- Why Attacker Attacks on Recommender System (Example)- Attack Profile Structure- Types of Attacks

o Push Attackso Nuke Attacks

- Attack Detection Attributeso Generic Attributeso Model Specific Attributes

• Proposed Approach• Conclusion• References

3

Recommender System

Introduction: – Recommender System (RS) is based on information filter to predict

the rating or preference that a user would give to an unseen item [1]. – It is backbone of ecommernce websites (amazon, flipkart, myntra

etc.), social networking websites (facebook, linkedin, twitter, google + etc.), matrimonial websites (shaadi.com, bharatmatrimonial.com etc.) and many more.

4

Types of RS-

1. Collaborative Filtering (CF) based RS: based on correlation between different users. It states that if two user had similar tastes in past will also have same taste in future also.

– Most popular and widely used.

2. Content based RS: based on the information on the features(keywords) of items rather than on the opinion of other users. E.g. in movie recommendation keywords may be: movie name, actor, actress, genre etc.

3. Knowledge based RS: customer defines his requirement explicitly. E.g. “I want black color BMW car”.

Recommender System

5

Recommender System – why?

1. Value for the customer– Narrow down the set of choice.– Discover new things.– Just exploring new items.– Make shopping easier.

2. Value for the provider– Increase user satisfaction.– Increase the sell.– Unique and personalized service to each customer.– Obtain more knowledge about the customer.– Opportunities for promotions.

6

Profile Injection Attack

Profile injection attack: Example• Assume that user-user based CF is used. • Pearson correlation as similarity measure.• Neighborhood size of 1.

Item1 Item2 Item3 Item4 … Target Pearson

Ram 5 3 4 1 … ?

User1 3 1 2 5 … 5 -0.54

User2 4 3 3 3 … 2 0.68

User3 3 3 1 5 … 4 -0.72

User4 1 5 5 2 … 1 -0.02

7

Profile Injection Attack

Profile injection attack: Example• Assume that user-user based CF is used. • Pearson correlation as similarity measure.• Neighborhood size of 1.

Item1 Item2 Item3 Item4 … Target Pearson

Ram 5 3 4 1 … ?

User1 3 1 2 5 … 5 -0.54

User2 4 3 3 3 … 2 0.68

User3 3 3 1 5 … 4 -0.72

User4 1 5 5 2 … 1 -0.02

User2 most similar to Ram

8

Profile Injection Attack

Profile injection attack: Example• Assume that user-user based CF is used. • Pearson correlation as similarity measure.• Neighborhood size of 1.

User2 most similar to Ram

Item1 Item2 Item3 Item4 … Target Pearson

Ram 5 3 4 1 … ?

User1 3 1 2 5 … 5 -0.54

User2 4 3 3 3 … 2 0.68

User3 3 3 1 5 … 4 -0.72

User4 1 5 5 2 … 1 -0.02

Attack 5 3 4 3 … 5 0.87

Attack

9

Profile Injection Attack

Profile injection attack: Example• Assume that user-user based CF is used. • Pearson correlation as similarity measure.• Neighborhood size of 1.

Item1 Item2 Item3 Item4 … Target Pearson

Ram 5 3 4 1 … ?

User1 3 1 2 5 … 5 -0.54

User2 4 3 3 3 … 2 0.68

User3 3 3 1 5 … 4 -0.72

User4 1 5 5 2 … 1 -0.02

Attack 5 3 4 3 … 5 0.87

Attack

Attack most similar to Ram

10

Attack Profile Structure

In order to look similar to genuine user, attacker gives ratings in a specific manner so that it become hard to identify the attack profile [2].

Selected items: these items are chosen because of their association with the target item.

Filler items: these items are randomly chosen and rating is given based upon properties of attack.

Unrated items: No ratings are given to these items.Target item: singleton item to which attacker promote or demote.

Item1 … ItemK … ItemN … ItemR Target

r_1 … r_k … r_l … r_n X

Selected items Filler items Unrated items Target item

11

Types of AttacksPush Attacks- to promote a specific item. Maximum rating will be given to the

target item [3].

1. Random Attack: – No rating to selected items.– Average rating of system to the filler items. – Least cost attack.– Limited effect as compared to more advance attacks.

2. Average Attack:– No rating to selected items.– Average rating of item to the item of filler items.– Additional cost of finding the average rating is involved.– More effective as compared to random attack.

cont …

12

Types of Attacks

3. Bandwagon Attack: – Maximum rating is given to popular items (selected items).– Average rating of system to the filler items.– It is a low cost attack.

4. Segment Attack:– Maximum rating is given to the items of the same segment of target

item (selected items).– Minimum rating is given to filler items.– Best attack (impact wise) as compared to other attacks of same

category.

cont …

13

Types of Attacks

Nuke Attack- to demotion of an item. Minimum rating is given to the target item.

1. Love/Hate Attack:– No rating to selected items.– Maximum rating to set of filler items.

2. Reverse Bandwagon Attack:– Minimum rating is given to least popular items (selected item).– Average rating of system is given to the set of filler items.– More effective as compared to the love/hate attack.

We can not stop attacks, we can just increase the cost of attacks.

14

Attack Detection AttributesGeneric Attributes- based on general abnormal behavior of user [4]. It is

common for all attack types.

1. Rating Deviation from Mean Agreement (RDMA): finds profile’s average rating deviation per item.

is the number of ratings given by user u. is rating given by user u

to item i. is average rating of item i. is number of ratings provided for item i.

2. Degree Similarity with Top Neighbors (DegSim): find the average similarity of profile with top k neighbors.

Nu r i,u

ri ti

15

Attack Detection Attributes3. Length Variance: finds the variance in the length (# ratings given) of a given

profile from the average length of database.

Model Specific Attributes- focuses on the signature of attacks type.

4. Mean Variance (MeanVar): used for average and random attack.

is the set of items rated by user u.

2. Filler Mean Target Difference (FMTD): used for bandwagon, reverse bandwagon and segment attack.

is the set of filler items rated by user u.

pu

p f,u

16

Evaluation Metrics

1. Precision: count total number of profiles that are labeled as attack [4].

2. Recall: count total number of actual attack in the system.

Reality

PredictionActually Attack Actually Good

Rated Attack True Positive (Tp) False Positive (Fp)

Rated Good False Negative (Fn) True Negative (Tn)

17

Proposed Approach

Steps-1. Calculate the attack detection attributes (generic and model specific

attributes).2. Apply statistical models using k-fold cross validation3. Compare the accuracy of statistical models.4. Pick the top three performing models.5. Ensemble the models (resulting models from step 4) using voting

approach.

Data Set Used: MovieLens-100K– 943 users– 1682 movies– 100000 ratings

18

Performance Analysis

Table 1: Performance analysis of 10% average attack.

Table 2: Performance analysis for bandwagon attack at 5% filler size.

Filler Size 1% 10% 20% 30% 40% 50%

Models P R P R P R P R P R P R

Decision Tree .90 .892 .921 .93 .928 .919 .939 .912 .94 .94 .961 .968

Random Forest .929 .930 .939 .92 .948 .948 .952 .956 .962 .961 .973 .979

Ada Boost .9 .908 .914 .918 .93 .924 .935 .938 .948 .934 .950 .943

SVM .938 .927 .949 .94 .959 .959 .971 .975 .979 .981 .988 .989

Linear Regression .89 .862 .89 .907 .918 .91 .925 .918 .929 .902 .93 .931

Neural Network .949 .938 .95 .943 .951 .954 .959 .958 .968 .967 .979 .98

Ensemble .939 .932 .946 .934 .953 .953 .961 .963 .968 .969 .98 .982

Attack Size 1% 3% 6% 9% 12% 15%

Models P R P R P R P R P R P R

Decision Tree .822 .811 .844 .830 .859 .848 .868 .855 .889 .872 .901 .928

Random Forest .880 .872 .894 .897 .905 .912 .909 .902 .929 .917 .957 .922

Ada Boost .819 .802 .829 .810 .834 .828 .858 .840 .883 .868 .902 .915

SVM .901 .902 .912 .928 .928 .908 .937 .911 .972 .959 .99 .984

Linear Regression .862 .842 .872 .882 .882 .908 .908 .919 .918 .935 .939 .941

Neural Network .892 .901 .919 .911 .922 .929 .959 .958 .961 .969 .97 .964

Ensemble .891 .892 .908 .912 .918 .916 .935 .924 .954 .948 .972 .957

19

Conclusion

1. I tried to explain why RS is so important now a days for ecommerce.

2. I also tried to explain, how a attacker can manipulate the results of RS.

3. I focused on several keys areas of attacks against recommender system i.e. different types of attacks, attack detection attributes and model evaluation metrics.

4. I found that random forest, SVM and neural networks perform better than other models in attack detection.

5. I present ensemble approach for the attack detection and I found that performance is not best in any case but it also does not give worst performance in any case.

20

21

References

[1]. Davoodi, Fatemeh Ghiyafeh, and Omid Fatemi. "Tag based recommender system for social bookmarking sites." In Advances in Social Networks Analysis and Mining (ASONAM), 2012 IEEE/ACM International Conference on, pp. 934-940. IEEE, 2012.

[2]. Lam, Shyong K., and John Riedl. "Shilling recommender systems for fun and profit." In Proceedings of the 13th international conference on World Wide Web, pp. 393-402. ACM, 2004.

[3]. Mobasher, Bamshad, Robin Burke, Runa Bhaumik, and Chad Williams. "Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness." ACM Transactions on Internet Technology (TOIT) 7, no. 4 (2007): 23.

[4]. O'Mahony, Michael, Neil Hurley, Nicholas Kushmerick, and Guénolé Silvestre. "Collaborative recommendation: A robustness analysis." ACM Transactions on Internet Technology (TOIT) 4, no. 4 (2004): 344-377.