mini-training: measure anything optimize everything
DESCRIPTION
Discover in a few slides the basics of a metrics-driven approach where you get knowledge and monitor your applications, based on real-life metricsTRANSCRIPT
![Page 1: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/1.jpg)
Measure Anything
Optimize Everything
Mini-Training
![Page 2: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/2.jpg)
Make decisions based on a scientific
approach
Build a scientific approach based on what
is known of « real life » aka « Production »
“You can’t manage what you don’t measure.”
W. Edwards Deming, Peter Drucker
Measure anything, Optimize Everything
![Page 3: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/3.jpg)
• Different levels of knowledge:
• Level 0 : no monitoring, metrics or logs. Features are defined based on feeling, intuitions. This is a blind approach.
• Level 1 : metrics and logs are reviewed post dev. Success is defined based on real events and failures.
• Level 2 : decisions and actions are deducted from metrics and
statistics. We measure to validate the results of our actions.
Where are we ?
Get Knowledge
![Page 4: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/4.jpg)
• How do we know something is wrong ? – Customer Service: calls from customers
– Error Logs
• How do we know everything is ok ? – …
Production Knowledge
![Page 5: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/5.jpg)
• How do we know if the new killer feature is used ?
• How do we know how efficient is our code ?
Feature Knowledge
![Page 6: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/6.jpg)
• Define a metrics-driven process to :
– Know applications in real life
– Monitor
– Evolve in a measured and verified way
Objectives
![Page 7: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/7.jpg)
1. Ask a question
2. Set an idea
3. Establish a prediction along this idea
4. Experiment to test this prediction
5. Analyse results to validate or reject the idea
6. Share results
7. Repeat
Scientific approach
![Page 8: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/8.jpg)
• This approach needs a deep instrumentation of the
whole platform :
– Aggregate data from many sources
– Measure
– Monitor along time
– Analyse it in real-time or at posteriori
– Decide and act accordingly
Measure anything
![Page 9: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/9.jpg)
Measure anything
Network
Machine
Application
![Page 10: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/10.jpg)
• Tools to monitor network and machines :
– Nagios/Centreon
– SCOM
• Metrics :
– Server status
– Front / Service status
– Network Bandwidth
– …
Technical Monitoring
![Page 11: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/11.jpg)
• Hard to manage :
– Related to business
– Change as business changes (often)
– Can’t be easily automated, not easily catchable
• Examples :
– Login successful/failed
– Bets placed Sport / Turf
– Casino transactions
– Challenge subscriptions
– Banners displayed
What about application ?
![Page 12: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/12.jpg)
Application monitoring = Business monitoring
• Each application knows its own business
– Define and report their own metrics
• Examples :
– On Turf, how many races are on sale.
– On Sport, how many time it takes to place a bet.
– On Casino, how many games are currently available.
– …
Application monitoring
![Page 13: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/13.jpg)
13
All is about Metrics
High Level
Real Time
Fast issue detection
Confidence
Metrics should be part of every feature.
Simple Accessible Required Pertinent
“Measure what is measurable, and make measurable what is not so.” Galileo Galilei
![Page 14: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/14.jpg)
• Dashboards : – Dev oriented
– Display metrics
– Inform if all is OK
– Emit alerts
– Real time production status
Application monitoring
![Page 15: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/15.jpg)
Dashboards Level 2 (at Etsy)
![Page 16: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/16.jpg)
• If we can measure, we can optimize
– Reduce Errors logs
– Reduce response time
– Optimize call numbers
– Remove dead code/feature
– Increase offer/bet placed
– …
– Overall Quality
16
Finally : Optimize Everything
![Page 17: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/17.jpg)
17
Virtuous Cycle
Business
Feature
Metrics Report
Optimize
![Page 18: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/18.jpg)
• Etsy technical blog (code, deployment, devops,…):
http://codeascraft.com
• http://fr.slideshare.net/frankhoffsummer/measure-anything-measure-everything-8178303
• http://fr.slideshare.net/mikebrittain/metricsdriven-engineering
Source
![Page 20: Mini-Training: Measure Anything Optimize Everything](https://reader033.vdocument.in/reader033/viewer/2022052618/554f88b5b4c905435d8b4d18/html5/thumbnails/20.jpg)
About Betclic • Betclic Everest Group, one of the world leaders in online gaming, has a unique
portfolio comprising various complementary international brands: Betclic, Everest Gaming, bet-at-home.com, Expekt…
• Active in 100 countries with more than 12 million customers worldwide, the Group is committed to promoting secure and responsible gaming and is a member of several international professional associations including the EGBA (European Gaming and Betting Association) and the ESSA (European Sports Security Association).
• Through our brands, Betclic Everest Group places expertise, technological know-how and security at the heart of our strategy to deliver an on-line gaming offer attuned to the passion of our players.