rebuilding atlas advertising at - qconlondon.com...history of atlas •1997 –atlas started life as...
TRANSCRIPT
![Page 1: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/1.jpg)
![Page 2: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/2.jpg)
Rebuilding Atlas – Advertising at Scale at Facebook
Jason McHugh, Software Engineer | March, 2015
![Page 3: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/3.jpg)
Outline
• Ad serving technology
• History of Atlas
• The acquisition
• Challenges
• Architecture
• Lessons learned
![Page 4: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/4.jpg)
Advertising
![Page 5: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/5.jpg)
Advertising
• Advertising a huge industry with massive budgets
• Digital advertising is the fastest growing advertising medium
• More time spent with digital media than TV
• Time spent on mobile exceeded TV
Source: eMarketer, 2014; Flurry, 2014
![Page 6: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/6.jpg)
Third Party Ad Serving
![Page 7: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/7.jpg)
How Third Party Ad Serving Works
Advertiser
![Page 8: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/8.jpg)
How Third Party Ad Serving Works
Advertiser
Campaigns
Click-Through URLs
Atlas
Creative Concepts
![Page 9: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/9.jpg)
![Page 10: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/10.jpg)
Serving An Ad
Atlas
You, me, or
other random
human
![Page 11: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/11.jpg)
Retargeting
Atlas
![Page 12: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/12.jpg)
Retargeted Ad
Atlas
![Page 13: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/13.jpg)
History of Atlas
• 1997 – Atlas started life as Avenue A
• 2003 – Avenue A becomes aQuantive
• 2007 – aQuantive was purchased by Microsoft for $6 billion dollars
• 2012 – Microsoft took a $6.2 billion dollar writedown
• 2013 – Facebook acquired Atlas in April
• 2014 – Atlas New publically launched at Ad Week in September
![Page 14: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/14.jpg)
Challenges
![Page 15: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/15.jpg)
• Ad Tech Stack
• Architecture
• Data Model and databases
• Data flows
• Deployment
Challenge: Understand the System
One Single DB Instance
![Page 16: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/16.jpg)
Challenge: Huge Product
• Third party ad server
• Advertiser and publisher negotiations – RFP
• Search management
• Search optimization
• Email tracking
• Custom analysis and reporting
• Rich media including video
• Franchise Management
![Page 17: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/17.jpg)
Challenge: No Lift and Shift
• Lift and Shift
• Common approach after acquisition
• Take as much as exists at the time of the purchase
• Move to your data centers and then evolve
• Not possible here
• Non-open compute hardware
• Usage of Microsoft close-source technologies
![Page 18: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/18.jpg)
Architecture
• Holistic view of the logical architecture
• Detail a piece of the physical architecture
![Page 19: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/19.jpg)
Logical Architecture
![Page 20: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/20.jpg)
Logical Architecture
![Page 21: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/21.jpg)
Logical Architecture
![Page 22: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/22.jpg)
Logical Architecture - Focused
![Page 23: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/23.jpg)
Physical Architecture – Ad Delivery
![Page 24: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/24.jpg)
Physical Architecture – Data Processing Pipeline
![Page 25: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/25.jpg)
Scribe
• Large-scale, high throughput message queue
• Not lossless but guarantees are excellent and perfect for us
• Decouples producers from consumers
• Persistent for n days
• Sharded consumption
• Checkpoint streams
![Page 26: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/26.jpg)
Physical Architecture – Data Processing Pipeline
![Page 27: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/27.jpg)
Detailed Look – Stream Processing
• Mini workflows
• Scalability is hard
• Message queues can be costly
• Repeatable re-execution
![Page 28: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/28.jpg)
Physical Architecture
![Page 29: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/29.jpg)
Physical Architecture
![Page 30: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/30.jpg)
Physical Architecture
![Page 31: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/31.jpg)
Physical Architecture
![Page 32: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/32.jpg)
Physical Architecture
![Page 33: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/33.jpg)
Physical Architecture
![Page 34: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/34.jpg)
Lessons Learned
![Page 35: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/35.jpg)
Mistake – Minimize the www/hack tier
![Page 36: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/36.jpg)
Mistake – Minimize the www/hack tier
• Minimize the code in www tier to ensure a higher level of availability
• Hack and the www tier have come a long way in 2 years
• Huge improvements in availability
• API changes hit two separate systems
• Couldn’t leverage all the improvements and investments in hack
![Page 37: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/37.jpg)
Mistake – Looking ahead
• Lesson
• Look ahead to where the industry is going
• Or look to where an organization (or group) will invest efforts
• Plan to meet them there
![Page 38: Rebuilding Atlas Advertising at - qconlondon.com...History of Atlas •1997 –Atlas started life as Avenue A •2003 –Avenue A becomes aQuantive •2007 –aQuantive was purchased](https://reader034.vdocument.in/reader034/viewer/2022050100/5f3f49a11a255638480e14ec/html5/thumbnails/38.jpg)
Questions?