ai in adtech by ruslan shevchenko tech hangout #6
TRANSCRIPT
Overview• AI
• Models [ black-box, … ]
• Predict, Recognize, Recommend, Optimize
• Control, Interact; be part of ecosystem
• BIAS;
• Machine [BIAS-Variance]
• Human [in => out]
• World [Complexity]
Naive approach
Just insert AI as black box
Naive approach
Just insert AI as black box
- Confidence - Speed - Effects
S0
- Confidence [check against basemodel] - Speed
- History size - Effects
- Feature set [changing over time ?] - Interaction (other AI)
S1
- Q1. How to balance discover/usage (?) - Many-armed bandits - constant split
- Q2. Bayes / Variance problem - validation process
Simple Complex.
Feature engineering• What is important ?
• cross-validation, correlated subsets, etc …
• Second-order features
• distribution of requests, banner-lifetime, etc
• Automatic feature engineering.
• Learning to learn by gradient descent by gradient descent
S2
- Q3. Behind obvious - Expect some internal structure - [hipotese + PCA]
- Q4. Set of models - combine to see the best
- (boost, vote, random forest, etc .. ) - classify: what model is valid for some specific case
S2
- It is not pure machine learning now. - It is research
- of real decision making process - (fundamental model)
- using ML methods
S3
Feedback Loop• We change the world
• We receive information from changed world
• AdSearch example (by LÉON BOTTOU):
• if we have word from search request in ad-description, than probability of click is higher
• but: engagement ring ~~ cheap diamonds
• What to do: Increase variance, violate search-state limits
• Hidden Technical Debt in Machine Learning Systems. (google)
• Dependencies
• Model boundaries. (CACE : Changing Anything Changes Everything)
• Data
• Feedback loops.
• ML-Antipatterns
• Glue Code, Pipeline jungles, Abstraction debt.
• Measure, prioritize, refactor:
• As software engineering but harder.
Conclusion• Avoid ‘blackbox with magic’ approach
• Machine Learning ~~ building approximators
• Try to explore fundamental models
• Track feedback loops;
• Keep space for new ideas
Thanks for attention
• Q?
