towards complex user feedback and presentation context in recommender systems

Towards Complex User Feedback and Presentation Context in

Recommender Systems

Peter Vojtas and Ladislav Peška

Department of Software Engineering, Charles University in Prague,

Czech Republic

Peska, Vojtas: Towards Complex User Feedback and Presentation Context in

Recommender Systems

2

Recommender Systems Propose relevant items to the right persons at the right time Machine learning application Expose otherwise hard to find, uknown items Complementary to the catalogues, search engines etc. „Win-win strategy“

PPI 2017, Stuttgart, Germany

User Feedbackrating, clickstream,time on page, buys…

User, Object ProfilesObject attributes

(Context)Time, location,Possible choices…

RECOMMENDERSYSTEM

Top-K Recommended objects

peska

Větší část introduction by se určitě dala vynechat - to nechám na tobě. ale myslím že by bylo dobré říct něco o- RecSys obecně- RecSys v malých E-commerce- implicitním feedbacku


Recommender Systems

3

Recommender Systems User feedback

Explicit feedback (user rating) Implicit feedback (user behavior)

User visited/bought object Usually binary feature

Recommending algorithms Collaborative filtering

(Users A and B were similar so far, the should like similar things in the future too)

Cold start problem Content-based filtering

(User A should like similar items to the ones he liked so far) Overspecialization, lack of diversity, obvious recommendations…



Recommender Systems

4

Challenge Recommending for small e-commerce websites

Tens of similar vendors, user can choose whichever she likes (Almost) no explicit feedback

(No incentives for users) Few visited pages

(Often usage of external search engines & landing on object details) Low user loyalty

(New vs. Returning visitors ratio 80:20)Þ Not enough data for collaborative filtering,

continuous cold-start problem

Focus on Implicit Feedback & Content-based recommendations Obtain precise implicit user feedback in enough quantity, as early as possible Gather external content to improve CB recommendations (other papers)


User FeedbackExplicit feedback Provided via website GUI

Rating an object via Likert Scale

Missing in small E-Commerces

Implicit feedback Often binary in the literature

User visited object User bought object

PPI 2017, Stuttgart, Germany Peska, Vojtas: Towards Complex User Feedback and Presentation Context in

Recommender Systems

5


Rating an object via Likert Scale Comparing objects explicitly is

not so common Missing in small E-Commerces

Implicit feedback Often binary in the literature

User visited object User bought object

Virtually any event triggered by user could be feedback

Get better picture about user engagement / preference


Recommender Systems

6


Rating an object via Likert Scale Comparing objects explicitly is

not so common Missing in small E-Commerces

Implicit feedback Virtually any event could be

used as feedback Tracked via JavaScript

Dwell time Number of page views, Scrolling,

mouse events, copy text, printing Purchase process etc.

Purchases represents fully positive feedback


Recommender Systems

7

User Feedback

Software: Peska, IPIget: The Component for Collecting Implicit User Preference Indicators


Recommender Systems

8

User Feedback – Past Resarch Combine multiple implicit feedback features to estimate user rating

Standard CB / CF recommender systems can be used afterwards

Improvements over the usage of simple implicit feedback

Peska, Vojtas: How to Interpret Implicit User Feedback?Peska, Eckhardt, Vojtas: Preferential Interpretation of Fuzzy Sets in E-shop Recommendation with Real Data Experiments


Recommender Systems

9

Dwell time: 10sScrolling: 100pxMouse movement:250px


Rating: 0.2

Rating: 0.8

User Feedback – Past Resarch Combine multiple implicit feedback features to estimate user rating

Standard CB / CF recommender systems can be used afterwards

Improvements over the usage of simple implicit feedback

Is that all we can do?PPI 2017, Stuttgart, Germany Peska, Vojtas: Towards Complex User Feedback and Presentation Context in

Recommender Systems

10



Rating: 0.2

Rating: 0.8

Context of User Feedback Combine multiple implicit feedback features to estimate user rating

Is that all we can do?

Pages may substantially vary in length, amount of content etc. This could affect perceived implicit feedback features Leveraging context could be important


Recommender Systems

11



Rating: 0.2

Rating: 0.8


Recommender Systems

12

Context of User Feedback


A B

Context of the user Location, Mood, Seasonality... Can affect user preference Out of scope of this paper

Context of device and page Page and browser dimensions Page complexity (amount of text, links, images,...) Device type Datetime Can affect percieved values of the user feedback

peska

2 motivace k používání kontextu: délka textu může ovlivnit čas strávený na stránce a poměr mezi velikostí obrazovky a stránky třeba scrolování

Outline of Our ApproachTraditional recommender User rates a sample of objects

Preference learning computes expected ratings of all objects

Top-k best rated objects are recommended

Our approach


Recommender Systems

13

𝑟𝑢 ,𝑜 :𝑜∈𝑺⊂𝑶 ;𝑟𝑢 ,𝑜∈[0,1]

�̂�𝑢={𝑜1 ,…,𝑜𝑘 }

¿

¿𝐹 𝑢 ,𝑜 ,𝐶𝑢 ,𝑜→𝑟𝑢 ,𝑜 :𝑜∈𝑺

Several imlicit feedback and contextual features are collected:

Learn estimated rating for visited objects based on feedback and context

„The more the better” heuristics (STD, CDF) Machine learning approach (J48)

Incorporate context As further feedback features (FB+C) As baseline predictors (AVGBP, CBP)

Learn rating on all objects as in traditional recommenders

peska

My do tradičního modelu doporučování vpodstatě přidáváme fázi, kdy z různých druhů implicitní zpětné vazby a kontextu na stránce se chceme naučit rating tohoto zobrazeného objektuk tomu využijeme buď machine learning (j48), nebo heuristiku typu "více je lépe"


Recommender Systems

14

Collecting User Behavior IPIget component for collecting user behavior

IPIget component download: http://ksi.mff.cuni.cz/~peska/ipiget.zip


Contextual featuresNumber of links

Number of images

Text size

Page dimensions

Visible area ratio

Hand-held device

Implicit Feedback FeaturesView Count

Dwell Time

Mouse Distance and Time

Scrolled Distance and Time

Clicks count

Hit bottom of the page

Purchase


Recommender Systems

15

Estimated Rating from Implicit Feedback


„The more the better” heuristics Various feedback features are not comparable in general

Dwell time (sec) vs. Distance travelled by mouse (pixels) Transform feedback features on comparable scale and average

Use standardization (STD) of feedback features (≈ N(0,1))

Use cummulative distribution (CDF) of each feedback feature

=>

peska

STD: odečtení průměrné hodnoty a vydělení (max-min), takže víceméně každá featura odpovídá normálnímu rozložení se středem v 0 a směr. odchylkou 1

peska

CDF: distribuční funkce -> výsledek je z intervalu [0,1], hodnoty jsou neklesající


Recommender Systems

16

Estimated Rating from Implicit Feedback


Machine learning approach J48 decision tree Purchases are golden standard

The only feedback which is a true indicator of positive preference Predict purchases based on other feedback features Use probability of purchase as estimated rating

peska

Zde se J48 učí přes všechny uživatele (jinak bychom neměli dost dat, většina uživatelů má 0 nebo 1 nákup


Recommender Systems

17

Employ Context in Rating Estimation


Use context in the same way as feedback (FB+C) Leave the decision about usage of context on the underlined model Plausible strategy for e.g. decision trees or rule mining learning

approaches Use context as a baseline predictor of feedback

Calculate estimated value of feedback feature for particular context value

Substract the estimation from the actual value Use feedback with baseline estimators instead of the original one

Either employ average baseline predictor over all context features (AVGBP)

Or use carthesian product of feedback features and baseline predictors based on each context feature (CBP)

peska

FB+C je opravdu pro rozhodovací stromy docela dobrý, protože umožňuje psát pravidla typu "pokud je kontext malý a feedback docela velký, pak je to dobře." "pokud je kontext velký, feedback musí být hodně velký, aby to bylo dobře"


Recommender Systems

18

Preference Learning and Recommendations


Collaborative filtering not applicable Continuous cold-start problem

Use combination of content-based and non-personalized VSM content-based recommendation

Vector of object features (TF-IDF) User is represented as weighted sum of visited object’s features Resulting score is a cosine similarity of user and object vectors

Most popular non-personalized algorithm Based on estimated ratings

Final score is a multiplication of VSM score and most popular score


Recommender Systems

19

Evaluation


Czech travel agency dataset 3 variants of rating estimation (STD, CDF, J48) 3 variants of context incorporation (FB+C, AVGBP, CBP) 2 baselines (use raw feedback, use binary visits)

Leave-one-out on purchased objects Ranking prediction nDCG, recall@top-10


Recommender Systems

20

Results

Results of nDCG, (*) = signifficant improvement of the best method J48 decision tree with both feedback and context on its input performs the

best Using „the more the better“ heuristics (CDF) with properly processed feedback

(AVGBP, CBP) also performs quite well


Processing method

Feedback and Context composition

Binary FB FB+C AVGBP CBP

STD + popVSM 0.255* 0.174* 0.197* 0.161* 0.158*CDF + popVSM 0.255* 0.257* 0.253* 0.258* 0.257J48 + popVSM 0.255* 0.256* 0.274 0.240* 0.247*J48 + objects popularity 0.180** 0.205* 0.211* 0.168* 0.186*

J48 + VSM 0.222* 0.224* 0.233 0.225* 0.224*

peska

Tady ty výsledky bych interpretoval tak, že pokud máme nějaký jasně preferovaný, ale řídký atribut jako jsou purchase, je ideální na to pustit machine learning - treba J48 a dát tomu všechny data. Pokud ale takový atribut nemáme - což se může stát na jiných doménách, heuristika typu "vice je lépe" může také celkem fungovat, jen je řeba jí dát vhodně předzpracovaná data

Conclusions, Future WorkKey outcomes Implicit feedback could be more than just a binary variable Observed feedback should be considered with respect to the context of

page and device Doing so could improve the quality of the recommended objects

Future work, Open Problems Better models of context employment and purchase prediction methods Further evaluation scenarios

Recommending on the beginning of a new session More refined feedback?

E.g. feedback on object’s attributes? On-line deployment and evaluation


Recommender Systems

21


Recommender Systems

22

Thank you!

Questions, comments?

Supplementary materials: http://bit.ly/2g79VVOSlides:

http://bit.ly/2g79VVO

http://bit.ly/2g79VVO

towards complex user feedback and presentation context in recommender systems

Science