![Page 1: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/1.jpg)
Hybrid Web Recommender Systems
Robin Burke
Presentation by Jae-wook Ahn10/04/05
![Page 2: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/2.jpg)
10/5/05 Hybrid Web Recommender Systems 2
References
• Entrée system & dataset• Burke, R. (2002). Semantic ratings and heuristic similarity for colla
borative filtering. AAAI Workshop on Knowledge-based Electronic Markets 2000.
• Feature augmentation, mixed hybrid example• Torres, R., McNee, S., Abel, M., Konstan J., & Riedl J. (2004). En
hancing Digital Libraries with TechLens+. Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries.
• Hybrid recommender system UI issue• Schafer, J. (2005). DynamicLens: A Dynamic User-Interface for a
Meta-Recommendation System. Workshop: Beyond Personalization 2005, IUI’05.
• Collaborative filtering algorithm• Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based
collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web.
![Page 3: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/3.jpg)
Concepts and Techniques
![Page 4: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/4.jpg)
10/5/05 Hybrid Web Recommender Systems 4
Hybrid Recommender Systems
• Mix of recommender systems • Recommender system classification – knowledge source
• Collaborative (CF)• User’s ratings “only”
• Content-based (CN)• Product features, user’s ratings• Classifications of user’s likes/dislikes
• Demographic• User’s ratings, user’s demographics
• Knowledge-based (KB)• Domain knowledge, product features, user’s need/query• Inferences about a use’s needs and preferences
![Page 5: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/5.jpg)
10/5/05 Hybrid Web Recommender Systems 5
CF vs. CN
• User-based CF• Searches for similar users
in user-item “rating” matrix
• Item-based CF• Searches for similar items
in user-item “rating” matrix
• CN• Searches for similar items
in item-feature matrix• Example – TF*IDF term
weight vector for news recommendation
Items
Users Ratings
![Page 6: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/6.jpg)
10/5/05 Hybrid Web Recommender Systems 6
Recommender System Problems
• Cold-start problem • Learning based techniques• Collaborative, content-based, demographic Hybrid techniques
• Stability vs. plasticity problem• Difficulty to change established user’s profile Temporal discount – older rating with less influence
KB – fewer cold start problem (no need of historical data)
CF/Demographic – cross-genre niches, jump outside of the familiar (novelty, serendipity)
![Page 7: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/7.jpg)
10/5/05 Hybrid Web Recommender Systems 7
Strategies for Hybrid Recommendation
• Combination of multiple recommendation techniques together for producing output
• Different techniques of different types• Most common implementations
• Most promise to resolve cold-start problem
• Different techniques of the same type• Ex) NewsDude – naïve Bayes + kNN
![Page 8: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/8.jpg)
10/5/05 Hybrid Web Recommender Systems 8
Seven Types of Recommender Systems
• Taxonomy by Burke (2002)
1. Weighted 2. Switching3. Mixed4. Feature combination5. Feature augmentation6. Cascade7. Meta-level
![Page 9: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/9.jpg)
10/5/05 Hybrid Web Recommender Systems 9
Weighted Hybrid
• Concept
• Each component of the hybrid scores a given item and the scores are combined using a linear formula
• When recommenders have consistent relative accuracy across the product space
• Uniform performance among recommenders (otherwise other hybrids)
![Page 10: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/10.jpg)
10/5/05 Hybrid Web Recommender Systems 10
Weighted Hybrid Procedure
1. Training
2. Joint rating• Intersection –
candidates shared between the candidates
• Union – case with no possible rating neutral score (neither liked nor disliked)
3. Linear combination
![Page 11: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/11.jpg)
10/5/05 Hybrid Web Recommender Systems 11
Mixed Hybrid
• Concepts• Presentation of different components side-by-side in a
combined list
• If lists are to be combined, how are rankings to be integrated?• Merging based on predicted rating or on recommender
confidence
• Not fit with retrospective data• Cannot use actual ratings to test if right items ranked highly
• Example• CF_rank(3) + CN_rank(2) Mixed_rank(5)
![Page 12: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/12.jpg)
10/5/05 Hybrid Web Recommender Systems 12
Mixed Hybrid Procedure
1. Candidate generation
2. Multiple ranked lists
3. Combined display
![Page 13: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/13.jpg)
10/5/05 Hybrid Web Recommender Systems 13
Switching Hybrid
• Concepts
• Selects a single recommender among components based on recommendation situation
• Different profile different recommendation
• Components with different performance for some types of users
• Existence of criterion for switching decision• Ex) confidence value, external criteria
![Page 14: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/14.jpg)
10/5/05 Hybrid Web Recommender Systems 14
Switching Hybrid Procedure
1. Switching decision
2. Candidate generation
3. Scoring
• No role for unchosen recommender
![Page 15: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/15.jpg)
10/5/05 Hybrid Web Recommender Systems 15
Feature Combination Hybrid
• Concepts
• Inject features of one source into a different source for processing different data
• Features of “contributing recommender” are used as a part of the “actual recommender”
• Adding new features into the mix
• Not combining components, just combining knowledge source
![Page 16: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/16.jpg)
10/5/05 Hybrid Web Recommender Systems 16
Feature Combination Hybrid Procedure
1. Feature combination In training stage
2. Candidate generation
3. Scoring
![Page 17: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/17.jpg)
10/5/05 Hybrid Web Recommender Systems 17
Feature Augmentation Hybrid
• Concepts• Similar to Feature Combination
• Generates new features for each item by contributing domain
• Augmentation/combination – done offline
• Comparison with Feature Combination• Not raw features (FC), but the result of computation
from contribution (FA)
• More flexible to apply
• Adds smaller dimension
![Page 18: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/18.jpg)
10/5/05 Hybrid Web Recommender Systems 18
Feature Augmentation Hybrid Procedure
![Page 19: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/19.jpg)
10/5/05 Hybrid Web Recommender Systems 19
Cascade Hybrid
• Concepts• Tie breaker
• Secondary recommender• Just tie breaker• Do refinements
• Primary recommender• Integer-valued scores – higher probability for ties• Real-valued scores – low probability for ties• Precision reduction
• Score: 0.8348694 0.83
![Page 20: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/20.jpg)
10/5/05 Hybrid Web Recommender Systems 20
Cascade Hybrid Procedure
• Procedure1. Primary recommender
2. Ranks
3. Break ties by secondary recommender
![Page 21: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/21.jpg)
10/5/05 Hybrid Web Recommender Systems 21
Meta-level Hybrid
• Concepts
• A model learned by contributing recommender
input for actual recommender
• Contributing recommender completely replaces the original knowledge source with a learned model
• Not all recommenders can produce the intermediary model
![Page 22: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/22.jpg)
10/5/05 Hybrid Web Recommender Systems 22
Meta-level Hybrid Procedure
• Procedure
1. Contributing recommender
Learned model
2. Knowledge Source Replacement
3. Actual Recommender
![Page 23: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/23.jpg)
Experiments
![Page 24: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/24.jpg)
10/5/05 Hybrid Web Recommender Systems 24
Testbed – Entrée Restaurant Recommender
• Entrée System• Case-based reasoning• Interactive critiquing dialog
• Ex) Entry Candidates “Cheaper” Candidates “Nicer” Candidates Exit
• Not “narrowing” the search by adding constrains, but changing the focus in the feature space
![Page 25: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/25.jpg)
10/5/05 Hybrid Web Recommender Systems 25
Testbed – Entrée Restaurant Recommender (cont’d)
• Entrée Dataset• Rating
• Entry, ending point – “positive” rating
• Critiques – “negative” rating
• Mostly negative ratings
• Validity test for positive ending point assumption – strong correlation between original vs. modified (entry points with positive ratings)
• Small in size
![Page 26: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/26.jpg)
10/5/05 Hybrid Web Recommender Systems 26
Evaluation Methodology
• Measures • ARC (Average Rank of the Correct recommendations)
• Accuracy of retrieval • At different size retrieval set• Fraction of the candidate set (0 ~ 1.0)
• Training & Test set• 5 fold cross validation – random partition of training/test set
• “Leave one out” methodology – randomly remove one item and check whether the system can recommend it
• Sessions Sizes• Single visit profiles – 5S, 10S, 15S
• Multiple visit profiles – 10M, 20M, 30M
![Page 27: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/27.jpg)
10/5/05 Hybrid Web Recommender Systems 27
Baseline Algorithms• Collaborative Pearson (CFP)
• Pearson’s correlation coefficient for similarity• Collaborative Heuristic (CFH)
• Heuristics for calculating distances between critiques• “nicer” and “cheaper” dissimilar• “nicer” & “quieter” similar
• Content-based (CN)• Naïve Bayes algorithm – compute probability that a item is “liked” / “d
isliked”• Too few “liked” items modified candidate generation
• Retrieve items with common features with the “liked” vector of the naïve Bayes profile
• Knowledge-based (KB)• Knowledge-based comparison metrics of Entrée• Nationality, price, atmosphere, etc.
![Page 28: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/28.jpg)
10/5/05 Hybrid Web Recommender Systems 28
Baseline Evaluations
• Techniques vary in performance on the Entrée data• Content-based (CN) –
weak• Knowledge-based (KB) –
better on single-session than multi-session
• Heuristic collaborative (CFH) – better than correlation-based (CFP) for short profiles
• Room for improvement• Multi-session profiles
![Page 29: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/29.jpg)
10/5/05 Hybrid Web Recommender Systems 29
Baseline Evaluations
![Page 30: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/30.jpg)
10/5/05 Hybrid Web Recommender Systems 30
Hybrid Comparative Study
• Missing components
• Mixed hybrid• Not possible with retrospective data
• Demographic recommender• No demographic data
![Page 31: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/31.jpg)
10/5/05 Hybrid Web Recommender Systems 31
Results – Weighted
• Hybrid performance better in only 10 of 30
• CN/CFP – consistent synergy (5 of 6)
• Lacks uniform performance • KB, CFH
• Linear weighting scheme assumption – fault
![Page 32: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/32.jpg)
10/5/05 Hybrid Web Recommender Systems 32
Results – Switching
• KB hybrids – best switching hybrids
![Page 33: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/33.jpg)
10/5/05 Hybrid Web Recommender Systems 33
Results – Feature Combination
• CN/CFH, CN/CFP• Contributing CN
• Identical to CFH, CFP
• CFH maintains accuracy with reduced dataset
• CF/CN Winnow – modest improvement
![Page 34: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/34.jpg)
10/5/05 Hybrid Web Recommender Systems 34
Results – Feature Augmentation
• Best performance so far• Particularly CN*/CF*
• Good for multi-session profiles
![Page 35: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/35.jpg)
10/5/05 Hybrid Web Recommender Systems 35
Results – Cascade
• CFP/KB, CFP/CN• Great improvement
• Also good for multi-profile sessions
![Page 36: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/36.jpg)
10/5/05 Hybrid Web Recommender Systems 36
Results – Meta-level Hybrids
• CN/CF, CN/KB, CF/KB, CF/CN
• Not effective• No synergy
• Weakness of KB/CN in Entrée dataset
• Both components should be strong
![Page 37: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/37.jpg)
10/5/05 Hybrid Web Recommender Systems 37
Discussion
• Dominance of the hybrids over basic recommenders
• Synergy was found under• Smaller profile size
• Sparse recommendation density
hybridization conquers cold start problem
![Page 38: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/38.jpg)
10/5/05 Hybrid Web Recommender Systems 38
Discussion (cont’d)
• Best hybrids• Feature augmentation, cascade
• FA allows a contributing recommender to make a positive impact • without interfering with the performance of the better algorithm
![Page 39: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/39.jpg)
10/5/05 Hybrid Web Recommender Systems 39
Conclusions
• Knowledge-based recommendation is not limited• Numerously combined to build hybrids
• Good for secondary or contributing components
• Cascade hybrids are effective• Though rare in literatures
• Effective for combining recommender with different strengths
• Different performance characteristics• Six hybridization techniques
• Relative accuracy & consistency of hybrid components
![Page 40: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/40.jpg)
System Example & Related Issues
![Page 41: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/41.jpg)
10/5/05 Hybrid Web Recommender Systems 41
System Example – TechLens+• Hybrid recommender system
• Recommenders – CF, CN• Hybrid algorithms – CF/CN FA, CN/CF FA, Fusion (Mixed)
• Corpus• CiteSeer• Title, abstract (CN), citations (CF)
• Methodology• Offline experiment, Online user study with questionnaire (by asking satisfa
ction on the recommendation)• Results
• Fusion was the best• Some FA were not good due the their sequential natures• Different algorithms should be used for recommending different papers• Users with different levels of experiences perceive recommendations differ
ently
![Page 42: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/42.jpg)
10/5/05 Hybrid Web Recommender Systems 42
Meta-recommender – DynamicLens
• Can user provided information improve hybrid recommender system output?
• Meta-recommender• Provide users with personal
ized control over the generation of a recommendation list from hybrid recommender system
• MetaLens• IF (Information Filtering),
CF
![Page 43: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/43.jpg)
10/5/05 Hybrid Web Recommender Systems 43
Meta-recommender – DynamicLens (cont’d)
• Dynamic query• Merges preference & recommendation interfaces
• Immediate feedback
• Discover why a given set of ranking recommendations were made
![Page 44: Hybrid Web Recommender Systems Robin Burke Presentation by Jae-wook Ahn 10/04/05](https://reader035.vdocument.in/reader035/viewer/2022062321/56649e5f5503460f94b59afd/html5/thumbnails/44.jpg)
Questions & Comments