yucheng lowmultiple domain user personalization deepak agarwal yahoo! research yucheng low carnegie...

Yucheng Low Multiple Domain User Personalization

Multiple Domain User

Personalization

Deepak AgarwalYahoo! Research

Yucheng Low Carnegie Mellon University

Alexander J. SmolaYahoo! Research

Yucheng Low Multiple Domain User Personalization

Information Flood

Yucheng Low Multiple Domain User Personalization

Personalization

3

Golf Reader Tech. Reader

Can we provide personalization to new

users?

Yucheng Low Multiple Domain User Personalization

MoviesUser 1

User 2

Impossible when you have only one domain.Best you can do is to have a good baseline.

One Domain Cold-Start

Yucheng Low Multiple Domain User Personalization

Movies NewsMusic

Possible when you have many domains.

Multiple Domains Cold Start

Yucheng Low Multiple Domain User Personalization

Personalization across all domain

Combine tokens from all spaces ignoring the

source domain User

Reads Golf NewsWatches MTV

Golf, Tiger,Music, Song

Expand token space to include source domain

Golf:1, Tiger:1,Music:2, Song:2

Your FavoritePersonalization

Algorithm

Yucheng Low Multiple Domain User Personalization

Personalization across all domain

Combine tokens from all spaces ignoring the

source domain User

Reads Golf NewsWatches MTV

Golf, Tiger,Music, Song

Expand token space to include source domain

Golf:1, Tiger:1,Music:2, Song:2

Your FavoritePersonalization

Algorithm

Domains with more observations will swamp out all other domains

What is a good personalization algorithm

that will work for all domains?

Yucheng Low Multiple Domain User Personalization

Solution Meta-Profile

User MetaProfile

User MusicProfile

User NewsProfile

Isolates each domain: Prevents larger domains from swamping out smaller domains.

PersonalizedNews

PersonalizedMusic

Yucheng Low Multiple Domain User Personalization

Solution Meta-Profile

User MetaProfile

User MusicProfile

User NewsProfile

User MovieProfile

Extensible: domains can be added/removed easily

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

Basketball

NBA, hoopTrain

3-point

Topic 1Golf,

Tiger, Woods,

Club, Green, Hole-in-

one

Topic 2Machine,

Learning, Neural,

Network,Train

Topic 3

DocumentTopic 1Topic 2Topic 3

Michael I. Jordan trains a

Neural Network to play golf

2Golf

3Network

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

N

Document

1. Each document has a mixture over topics

2. For each word in each document

a)Draw a topicb)Draw a word from the topic

A document is a bag of words.A topic is a mixture of words.

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

N

Document

1.Each document has a mixture over topics

2. For each word in each document

a)Draw a topicb)Draw a word from the topic

A document is a bag of words.A topic is a mixture of words.

Document

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

N

Document

1. Each document has a mixture over topics

2.For each word in each documenta)Draw a topicb)Draw a word from the topic

A document is a bag of words.A topic is a mixture of words.

Document

Sample From:

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

N

Document

1. Each document has a mixture over topics

2.For each word in each documenta)Draw a topicb)Draw a word from the topic

A document is a bag of words.A topic is a mixture of words.

Topic 1: Basketball, Michael, Jordan

Topic 2: Golf, Tiger, Woods, Club, GreenTopic 3: Machine, Learning, Neural

Yucheng Low Multiple Domain User Personalization

Latent Dirichlet Allocation

N

Document

1. Each document has a mixture over topics

2. For each word in each documenta)Draw a topicb)Draw a word from the topic

A document is a bag of words.A topic is a mixture of words.

Topics which make upeach document

Words which make up

each topic

Yucheng Low Multiple Domain User Personalization

Single Domain Personalization

N

1. Each user has a mixture over topics 2. For each word in each

documenta)Draw a topicb)Draw a word from the topic

A user’s interaction with a domain is a bag of words.A topic is a mixture of words.

User

Words which make up

each topic

Topics each user is interested in

Yucheng Low Multiple Domain User Personalization

Multiple Domain Personalization

N

User u’s interaction with domain dUser

A user’s interaction with a domain is a bag of words.A topic is a mixture of words.

Each user has a meta-profile:

Each domain has a latent matrix:

User’s prior interest in a domain is

Yucheng Low Multiple Domain User Personalization

Solution Meta-Profile

User MetaProfile

User MusicProfile

User NewsProfile

User MovieProfile

Yucheng Low Multiple Domain User Personalization

Users

Music

News

Movies

Topic->word table

Topic->word table

Topic->word table

Yucheng Low Multiple Domain User Personalization

Gibbs Sampling

N

User u’s interaction with domain p

LDA

Yucheng Low Multiple Domain User Personalization

Gibbs Sampling

N

User u’s interaction with domain p

Hold Constant

Sample using LDA Sampler

1: Sample

Hold Constant

Yucheng Low Multiple Domain User Personalization

Gibbs Sampling

N

User u’s interaction with domain p

Hold Constant

Hold Constant

1: Sample2: Sample

Sample Langevin Diffusion

Yucheng Low Multiple Domain User Personalization

Gibbs Sampling

N

User u’s interaction with domain p

Optimize

Hold Constant

1: Sample2: Sample 3: Optimize

Hold Constant

LBFGS

Yucheng Low Multiple Domain User Personalization

Experiments

Yucheng Low Multiple Domain User Personalization

Experiments @ Yahoo! 2 domain dataset.

Frontpage and News clicks of 5.6 million users. Frontpage/News: Article text for each click.

3 domain dataset: Frontpage, News and MyYahoo clicks of 5.6 million users. MyYahoo: Only has article IDs for each click with no text. Not semantically meaningful.

All user information was anonymized.

Yucheng Low Multiple Domain User Personalization

Test Protocol

Holdout proportion of users who see more than one domain. Hide one of those domain and try to predict the words.

Prediction metric is cosine similarityBaseline is “mean prediction”.

Yucheng Low Multiple Domain User Personalization

ImplementationDistributed implementation in C++ using Memcached for communication.

Alex Smola, Shravan Narayanamurthy “An Architecture for Parallel Topic Models” VLDB 2010

Distributed LBFGS line search: Implement standard MPI-like in Memcached.

BroadcastReduceBarrier

Takes 2-3 days for 500 iterations on 30 machines

Yucheng Low Multiple Domain User Personalization

2 Property Sanity Check

Yucheng Low Multiple Domain User Personalization

2 Property

Yucheng Low Multiple Domain User Personalization

3 Property

Yucheng Low Multiple Domain User Personalization

3 Property

Yucheng Low Multiple Domain User Personalization

sandra, oscar, oscars, red, carpet, bullock, golden, gown, bullocks, nominee, bestactress, sparkles, stunning,

vienna, bachelor, jake, pavelka, giraldi, finale, show, stars, dancing, love, season, time, abc,

bacteria, fight, super, struggling, developed, doctors, resistant, lethal, virtually, drugs, antibiotic, competitors, chad,

film, movie, movies, films, director, story, avatar, james, time, hollywood, big, make, hes, star,

Frontpage -> NewsCelebrity

Entertainment

Science

Science Fiction

Yucheng Low Multiple Domain User Personalization

iphone, apple, app, apps, ipod, google, store, apples, android, mac, mobile, touch, ipad, device, phone,

college, year, earn, years, 000, bestpaid, average, 129, colleges, graduates, ten, alums, schools, actor, likes,

health, care, bill, obama, president, rep, house, republican, senate, news, sen, democrats, fox, congress, reform

drafts, player, nfl, scouts, team, riskiest, peril, bryant, dez, pick, talented, nba, james, news,

News -> Frontpage

home, bank, facing, ceo, gomez, eviction, rosalina, bought, cleaning, foreclosed, client, janitor, offices, surprising, video,,

captured, inside, mountain, terrorist, observers, impresses, alqaidas, complexity, base, features, hideout, size, special, secret, struck,,

Politics Devices

College

Yucheng Low Multiple Domain User Personalization

Extension

User MetaProfile

User MusicProfile

User NewsProfile

Latent Dirichlet Allocation

Latent Dirichlet Allocation

User MovieProfile

Latent Dirichlet Allocation

Yucheng Low Multiple Domain User Personalization

Extension

User MetaProfile

User MusicProfile

User NewsProfile

Flexible: Allows different algorithm for each domain

Linear ModelMatrix

Factorization

User MovieProfilefLDA

Yucheng Low Multiple Domain User Personalization

It Is How You Use It

User MetaProfile

User MusicProfile

Personalized withAlgorithm X

Use the Meta Profile for Initialization.

Yucheng Low Multiple Domain User Personalization

It Is How You Use It

User MetaProfile

User MusicProfile

Personalized withAlgorithm X

Periodically Update the Meta Profile and Domain Latent Matrix

Yucheng Low Multiple Domain User Personalization

ConclusionAn generic, extensible model for combining domain personalization schemes. Scalable inference procedure that extends to millions of users.Demonstrate strong predictive performance on a large real world data

Yucheng Low Multiple Domain User Personalization

Questions?