learned embeddings for search and discovery at instacart
TRANSCRIPT
![Page 1: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/1.jpg)
Learned Embeddings for Search and Discovery at Instacart
Sharath RaoData Scientist / ManagerSearch and Discovery
Collaborators: Angadh Singh
![Page 2: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/2.jpg)
v
Talk Outline
• Quick overview of Search and Discovery at Instacart
• Word2vec for product recommendations
• Extending Word2vec embeddings to improve search ranking
![Page 3: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/3.jpg)
v
About me
• Worked on several ML products in catalog, search and personalization at Instacart since 2015
• Currently leading full stack search and discovery team
@sharathrao
![Page 4: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/4.jpg)
v
Search and Discovery@ Instacart
Help customers find what they are looking for and discover what they might like
![Page 5: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/5.jpg)
v
Grocery Shopping in “Low Dimensional Space”
Search
Restock
Discover
![Page 6: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/6.jpg)
v“Here is what I want”
Query has easily accessible information content
“Here I am, what next?”
Context is the query, and recommendations vary
as contexts do
Search Discovery
![Page 7: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/7.jpg)
v
Entities we try to model
items
products
aisles
departments
retailers
queries
customers
brands
Most of our data products are about modeling relationships
between them
(part of which is learning embeddings)
![Page 8: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/8.jpg)
v
Common paradigm for search/discovery problems
1st phase: Candidate Generation
2st phase: Reranking
![Page 9: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/9.jpg)
v
• Select top 100s from among potentially millions
• Must be fast and simple• often not even a learned model
• Recall oriented
Candidate Generation
![Page 10: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/10.jpg)
v
• Ranks fewer products, but users richer models/features
• Tuned towards high precision
• Often happens in real-time
Reranking
![Page 11: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/11.jpg)
vWord2vec for Product Recommendations
![Page 12: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/12.jpg)
v
“Frequently bought with” recommendations
Not necessarily consumed together
Help customers shop for the next item
Probablyconsumed together
![Page 13: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/13.jpg)
vQuick Tour of Word2vec
(simplified, with a view to develop some intuition)
![Page 14: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/14.jpg)
v• We need to represent words as features in a model/task
• Naive representation => one hot encoding
• Problem: Every word is dissimilar to every other word => not intuitive
One-hot encodings are good, but not good enough
![Page 15: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/15.jpg)
v
• stadium, field and restaurant are in some sense similar
• stadium and field are more similar than stadium and restaurant
• game has another meaning/sense that is similar to stadium/field/restaurant
What one-hot encodings fail to capture
Observation 1: “game at the stadium”
Observation 2: “game at the field”
Observation 3: “ate at the restaurant”
Observation 4: “met at the game”
Data What models must learn
None of these are easy to learn with one-hot encoded representations without supplementing with hand-engineered features
![Page 16: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/16.jpg)
vYou shall know a word by the company it keeps - Firth, J.R., 1957
Core motivation behind semantic word representations
![Page 17: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/17.jpg)
v
• Learns a feature vector per word
• for 1M vocabulary and 100 dimensional vector, we learn 1M vectors => 100M numbers
• Vectors must be such that words appearing in similar contexts are closer than any random pair of words
• Must work with unlabeled training data
• Must scale to large (unlabeled) datasets
• Embedding space itself must be general enough that word representations are broadly applicable
Word2vec is a scalable way to learn semantic word representations
![Page 18: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/18.jpg)
vWord2vec beyond text and words
![Page 19: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/19.jpg)
v
On graphs where random walks are the sequences
![Page 20: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/20.jpg)
v
On songs where plays are the sequences
![Page 21: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/21.jpg)
v
Even the emojis weren’t spared!
![Page 22: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/22.jpg)
vYou shall know a
word product
by the company it keeps by what its purchased with
![Page 23: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/23.jpg)
v
We applied word2vec to purchase sequences
Typical purchasing session contains tens of cart adds
Sequences of products added to cart are the ‘sentences’
![Page 24: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/24.jpg)
v
“Frequently Bought With” Recommendations
Extract Training
Sequences
Learn word2vec
representations
Eliminate substitute products
Event Data
Approximate Nearest Neighbors
Cache recommendations
![Page 25: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/25.jpg)
v
Word2vec for product recommendations
Surfaces complementary
products
![Page 26: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/26.jpg)
v
Next step is to make recommendations contextual
Not ideal if already shopped for sugar recently
inappropriate if allergic to walnuts
Not if favorite brand of butter isn’t
otherwise popular
![Page 27: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/27.jpg)
vWe see word2vec recommendations as a candidate generation step
![Page 28: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/28.jpg)
vWord2vec for Search Ranking
![Page 29: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/29.jpg)
v
The Search Ranking ProblemIn response to a query, rank products in the order that
maximizes the probability of purchase
![Page 30: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/30.jpg)
v
The Search Ranking Problem
Learning to Rank Model
Candidate Generation
Boolean matching
BM25 ranking
Synonym/Semantic Expansion
(Online) Reranking
Matching features
Historical aggregates
Personalization features
![Page 31: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/31.jpg)
v
Search Ranking - Training
Generate training features
Label Generation with Implicit Feedback
Event Data
Learning to Rank Training
Model Repository
![Page 32: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/32.jpg)
v
Search Ranking - Scoring
Online reranker
query
Model Repository
Reranked products
Top N products
Top N
productsprocessed query
Final ranking
![Page 33: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/33.jpg)
v
Features for Learning to Rank
Historical aggregates => normalized purchase counts
High coverage Low precision
Low coverage More precision
[query, product]
[query, product, user]
[query, brand]
[query, aisle][product]
But historical aggregates alone are not enough. Because sparsity and cold start.
[user, brand]
![Page 34: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/34.jpg)
v
Learning word2vec embeddings from search logs
• Representations learnt from wikipedia or petabyte text aren’t ideal for Instacart
• No constraint that word2vec models must be learnt on temporal or spatial sequences
• We constructed training data for word2vec from search logs
![Page 35: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/35.jpg)
v
Training Contexts from Search Logs
• Create contexts that are observed and desirable
<query1> <purchased description of converted product1> <query2> <purchased description of converted product2> .. ..
• Learn embeddings for each word in the unified vocabulary of query tokens and catalog product descriptions
![Page 36: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/36.jpg)
v
Examples of nearest neighbors in embeddings space
OK, so now we have word representations
Matching features require product features in the same space
bread
orowheat bonaparte zissel ryebread
raisins
sunmade monukka raisinettes fruitsource
mushrooms
cremini portabello shitake
![Page 37: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/37.jpg)
v
Deriving other features from word embeddings
• One option is to create contexts with product identifiers in training sequences
• Promising but we will have a product cold start problem
Use learnt word embeddings to derive features for other entities such as products, brand names, queries and users etc.
![Page 38: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/38.jpg)
v
Simple averaging of word representations works well
Product
Average embeddings of words in product description
Brands:
Average embeddings of products sold by the brand
User
Average embeddings of products purchased by the user
Aisles/Departments
Average embeddings of products in the aisle/department
Word Representation
Learnt from converted search logs
Query
Average embeddings of words in query
![Page 39: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/39.jpg)
v
Wait, so we have 2 different representations for products?
Embeddings learnt from purchase sequences
Embeddings derived from words from search logs
Products are similar if they bought together
Products are similar if their descriptions are semantically similar
![Page 40: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/40.jpg)
v
Product representations in different embedding spaces
Embeddings learnt from purchase sequences
Products are similar if they bought together
![Page 41: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/41.jpg)
v
Product representations in different embedding spaces
Embeddings derived from words from search logs
Products are similar if their descriptions are semantically similar
![Page 42: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/42.jpg)
v
Examples of nearest neighbors for products
Cinnamon Toast Crunch CerealGolden Grahams Cereal
Not much common between the product names
![Page 43: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/43.jpg)
v
Examples of nearest neighbors for brands
![Page 44: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/44.jpg)
v
Using word2vec features in search ranking
[query, product]
[query, brand]
[query, aisle]
[query, department]
[user, product]
[user, brand]
[user, aisle]
[user, department]
• Construct similarity scores between different entities as features
Matching Features Personalization Features
![Page 45: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/45.jpg)
v
We saw significant improvement with word2vec features
98.0%
99.0%
100.0%
101.0%
102.0%
103.0%
104.0%
AUC Recall@10
Relative(Improvement(with(word2vec(features
Baseline With:word2vec: features
![Page 46: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/46.jpg)
v
word2vec features rank high among top features
[query,(aisle](word2vec
[query,(product](historical(aggregate
[query,(department](word2vec
[query,(product](historical(aggregate
[user,(product](word2vec
lda(model(2
lda(model(1(
query(length
[user,(brand](word2vec
[query,(product](word2vec
bm25
position
XGBoost logistic loss
with early stopping
![Page 47: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/47.jpg)
vOther contextual recommendation problems
![Page 48: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/48.jpg)
v
Broad based discovery oriented recommendations
Including from stores customers may have never shopped from
![Page 49: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/49.jpg)
v
Run out of X?
Rank products by repurchase probability
![Page 50: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/50.jpg)
v
Introduce customers to products new in the catalog
Also mitigates product cold start problems
![Page 51: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/51.jpg)
v
Replacement Product Recommendations
Mitigate adverse impact of last-minute out of stocks
![Page 52: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/52.jpg)
v
Data Products in Search and Discovery
• query autocorrection• query spell correction• query expansion• deep matching/document expansion• search ranking• search advertising
• Substitute/replacements products• Frequently bought with products• Next basket recommendations• Guided discovery• Interpretable recommendations
Search Discovery
![Page 53: Learned Embeddings for Search and Discovery at Instacart](https://reader033.vdocument.in/reader033/viewer/2022052205/58ac097b1a28abb6718b70c3/html5/thumbnails/53.jpg)
v
Thank you!
We are hiring!
Senior Machine Learning Engineer http://bit.ly/2kzHpcg