Booking.com

Recommend Now, Not in the Past

Lucas Bernardi, Melanie Mueller Data scientists @ Booking.com

Leveraging Contextual User Profilesfor Destination Recommendations

Booking.com

Introduction: Recommending travel destinations

Part I: Ranking destinations

Part II: Contextual recommendations

Conclusions

Outline

Booking.com

Travel agents

Booking.com

Online travel agents

Booking.com

Destination Finder

Booking.com

Destination Finder

Booking.com

Destination Finder

Booking.com

Which destinations to recommend?

Destination recommendations

1) Match activities

2) Recommend best matching destinations

Booking.com

• 5 million reviews

Endorsement data

• 256 activities mined from reviews (LDA)

• Ask users to ‘endorse’ a destination after their stay,e.g. ‘Beaches’, ‘Temples’

Endorsement #given #destinations Shopping 876,726 11,708 Food 525,111 20,538 Beach 505,192 11,422 … … … Mythology 1,065 406 Heliskiing 354 165

Booking.com

Endorsement data

Booking.com

Endorsement data• Standard recommender: user gives rating for item

• Here: multi-criteria, negative opinions are hidden

Booking.com

Ranking for ‘beach’

• Naive Bayes

P(Miami | beach) = (# beach endorsements for Miami)

(# beach endorsements)

• Keep it simple!

Booking.com

Destination Finder

Booking.com

Evaluation

• Naive Bayes

• Random

• Popularity

How to test?

Booking.com

A/B testing• Version A • Version B • Version C

Booking.com

A/B testing

Ranker #users Random 10079 Popularity 9838 Naive Bayes 9895

• Which metric?

User engagement → clicks

Booking.com

A/B testing

Ranker #users #clickers conversion g-test Random 10079 2465 24.46% Popularity 9838 2509 25.50% 90.8% Naive Bayes 9895 2645 26.73% 99.9%

• Which metric?

User engagement → clicks

Booking.com

Introduction: Recommending travel destinations

Part I: Ranking destinations

Part II: Contextual recommendations

Conclusions

Outline

Booking.com

History is history

Traditional recommending systems use past user ratings to predict unknown ratings

User History is short: User Cold start problem. Users have different personas: History becomes less

relevant User Interests are volatile: History becomes less relevant

Continuous User Cold start

Booking.com

Context

DefinitionSet of features that inform about the current situation of the user.

ExampleLocation, device, weather conditions, season, day of the week,

hour of the day.

HypothesisUsers in similar contexts have similar interests.

Booking.com

Context

Traditional Collaborative Filtering RecommendationsU x I → R

Destination FinderU x I x C → <r0, r1, r2, …, rn>

Booking.com

Framework

Booking.com

Framework

Booking.com

Discovering contextual profiles

Data points: Reviews Features:

Endorsements Contextual features All features one-hot-encoded

Example:<Ubuntu, Firefox, Tuesday, beach, temples><0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0...,0,1,0...0>

Booking.com

Discovering contextual profiles

k-means Clustering Clean up clusters Final output

Q binary n-dimensional centroids Q Contextual Profiles

Booking.com

Discovering contextual profiles

Contextual Profiles

i - 1 i i + 1 i + 1

… iPhone.OS.7.Chrome Windows.Phone …

iPhone.OS.5.Chrome Windows.Vista

iPhone.OS.6.Chrome Friday

Android.2.2 Sunday

Android.2.2.Tablet

Android.3.1.Tablet

Android.4.0.Tablet

Android.4.4.Tablet

Android.2.1.Tablet

Android.3.0.Tablet

Android.4.1

Android.4.3.Tablet

Booking.com

Applying contextual profiles

Booking.com

Applying contextual profiles

Contextual Profiles are simply a binary vector Find the most similar Contextual Profile for each review Train a ranker for each Contextual Profile

Booking.com

Computing recommendations

Booking.com

Computing recommendations

For a given user, compute a feature vector using contextual features

Contextualize: Find closest Contextual Profile Compute recommendations using the ranker trained on the

selected Contextual Profile

Booking.com

Framework

Booking.com

Results

Ranker Users Conversion CTR

Baseline 13,306 21.7 ± 0.7% 18.5 ± 0.4%

Contextual 13,562 21.3 ± 0.7% 22.2 ± 0.4%

Improved CTR by 22.5%

Booking.com

Conclusions• Multi-criteria destinations Recommender System• Simple rankers increase user engagement• Context matters

•Improves user engagement•Attacks Continuous cold start problem

• Reusable Contextual Profiles• On-line evaluation

Booking.com

Julia Kiseleva, PhD student at Eindhoven University,Research intern at Booking.com Nov 204 - Mar 2015

Booking.com: Chad Davis, Ivan KovacekMats Stafseng Einarsen

Academia: Djoerd Hiemstra, Jaap KampsMykola Pechenizkiy, Alexander Tuzhilin

Thanks

Booking.com

We're hiring