![Page 1: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/1.jpg)
When Recommendation
Systems Go Bad
Evan Estola3/31/17
![Page 2: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/2.jpg)
About Me
● Evan Estola
● Staff Machine Learning Engineer, Data Team Lead @ Meetup
● @estola
![Page 3: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/3.jpg)
Meetup
● Do more
● 270,000 Meetup Groups
● 30 Million Members
● 180 Countries
![Page 4: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/4.jpg)
Why Recs at Meetup are Hard
● Cold Start
● Sparsity
● Lies
![Page 5: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/5.jpg)
Recommendation Systems: Collaborative Filtering
![Page 7: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/7.jpg)
Recommendation Systems: Rating Prediction
● Netflix prize
● How many stars would user X give movie Y
● Ineffective!
![Page 8: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/8.jpg)
Recommendation Systems: Learning To Rank
● Treat Recommendations as a supervised ranking problem
● Easy mode:
○ Positive samples - joined a Meetup
○ Negative samples - didn’t join a Meetup
○ Logistic Regression, use output/confidence for ranking
![Page 9: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/9.jpg)
![Page 10: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/10.jpg)
You just wanted a kitchen scale, now Amazon thinks you’re a drug dealer
![Page 11: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/11.jpg)
● “Black-sounding” names 25% more
likely to be served ad suggesting
criminal record
![Page 12: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/12.jpg)
●
● Fake profiles, track ads
● Career coaching for “200k+”
Executive jobs Ad
● Male group: 1852 impressions
● Female group: 318
![Page 13: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/13.jpg)
● Twitter bot● “Garbage in,
garbage out”● Responsibility?
“In the span of 15 hours Tay referred to feminism as a
"cult" and a "cancer," as well as noting "gender equality
= feminism" and "i love feminism now." Tweeting
"Bruce Jenner" at the bot got similar mixed response,
ranging from "caitlyn jenner is a hero & is a stunning,
beautiful woman!" to the transphobic "caitlyn jenner
isn't a real woman yet she won woman of the year?"”
Tay.ai
![Page 15: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/15.jpg)
Know your data
● Outliers can matter
● The real world is messy
● Some people will mess with you
● Not everyone looks like you
○ Airbags
● More important than ever with
more impactful applications
○ Example: Medical data
![Page 16: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/16.jpg)
Keep it simple
● Interpretable models
● Feature interactions
○ Using features against
someone in unintended ways
○ Work experience is good up
until a point?
○ Consequences of location?
○ Combining gender and
interests?
● When you must get fancy, combine
grokable models
![Page 17: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/17.jpg)
Ensemble Model, Data Segregation
Data:*InterestsSearchesFriendsLocation
Data:*GenderFriendsLocation
Data:Model1 PredictionModel2 Prediction
Model1 Prediction
Model2 Prediction
Final Prediction
![Page 18: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/18.jpg)
Diversity Controlled Testing
● CMU - AdFisher
○ Crawls ads with simulated user profiles
● Same technique can work to find bias in your own models!
○ Generate Test Data
■ Randomize sensitive feature in real data set
○ Run Model
■ Evaluate for unacceptable biased treatment
● Florian Tramèr
○ FairTest
![Page 19: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/19.jpg)
https://research.google.com/bigpicture/attacking-discrimination-in-ml/
![Page 20: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/20.jpg)
Human Problems
● Auto-ethics
○ Defining un-ethical features
○ Who decides to look for fairness in the first place?
![Page 21: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/21.jpg)
![Page 22: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/22.jpg)
By restricting or removing certain features aren’t you sacrificing performance? Isn’t it actually adding bias if you decide which features to put in or not?If the data shows that there is a relationship between X and Y, isn’t that your ground truth?
Isn’t that sub-optimal?
![Page 23: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/23.jpg)
It’s always a human problem
● “All Models are wrong, but some are useful”
● Your model is already biased
![Page 24: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/24.jpg)
Bad Features
● Not all features are ok!
○ ‘Time travelling’
■ Rating a movie => watched the movie
■ Cancer Surgery
![Page 25: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/25.jpg)
Misguided Models
● “It’s difficult to make predictions, especially about the future”
○ Offline performance != Online performance
○ Predicting past behavior != Influencing behavior
○ Example: Clicks vs. buy behavior in ads
![Page 26: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/26.jpg)
Asking the right questions
● Need a human
○ Choosing features
○ Choosing the right target variable
■ Value-added ML
![Page 27: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/27.jpg)
“Computers are useless,
they can only give you
answers”
![Page 28: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/28.jpg)
Bad Questions
● Questionable real-world applications
○ Screen job applications
○ Screen college applications
○ Predict salary
○ Predict recidivism
● Features?
○ Race
○ Gender
○ Age
![Page 29: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/29.jpg)
Correlating features
● Name -> Gender
● Name -> Age
● Grad Year -> Age
● Zip -> Socioeconomic Class
● Zip -> Race
● Likes -> Age, Gender, Race, Sexual Orientation...
● Credit score, SAT score, College prestigiousness...
![Page 30: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/30.jpg)
At your job...
Not everyone will have the same ethical values, but you don’t have to take
‘optimality’ as an argument against doing the right thing.
![Page 31: When recommendation systems go bad - machine eatable](https://reader031.vdocument.in/reader031/viewer/2022030306/58e4bcfc1a28ab1c1f8b69cb/html5/thumbnails/31.jpg)
You know racist computers are a bad idea
Don’t let your company invent racist computers
@estola