Download - Building a Data Driven Business
![Page 1: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/1.jpg)
Building a data driven business
Konstantin Savenkov, CEO, Intento
![Page 2: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/2.jpg)
Konstantin Savenkov, PhDChief Research Officer, Zvooq COO, Bookmate and Dream Industries now Founder & CEO
![Page 3: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/3.jpg)
Turned a number of companies data-driven
• B(2B)2C content services
• edutainment
• ad tech• commercial space management
• consulting: online2offline, UGC, biotech
Bookmate, Zvooq
Theory&Practice, Exchanges
Unisound
DI Telegraph
@ksavenkov
![Page 4: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/4.jpg)
Without data, people take
decisions based on biases and
beliefs@ksavenkov
![Page 5: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/5.jpg)
…data replaces beliefs and provides a competitive
advantage@ksavenkov
![Page 6: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/6.jpg)
If you’re competitor-focused, you have to
wait until there’s a competitor doing something.
Being customer-focused allows you to be more
pioneering.Jeff Bezos Amazon
@ksavenkov
![Page 7: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/7.jpg)
…data helps to be customer-centric
@ksavenkov
![Page 8: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/8.jpg)
Information is the oil of the 21st century, and analytics is the combustion engine.
Peter Sondegaard Gartner
@ksavenkov
![Page 9: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/9.jpg)
…check the gas quality and do the maintenance
@ksavenkov
![Page 10: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/10.jpg)
Half of the money spent on advertising
is wasted; the trouble is I don’t know which half.
John Wanamaker or William Lever@ksavenkov
![Page 11: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/11.jpg)
…channel attribution and ROI will tell you
@ksavenkov
![Page 12: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/12.jpg)
The average conversion rate in the United States whether you are selling
elephants or iPods, will be 2%
(с) Авинаш Кошик@ksavenkov
![Page 13: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/13.jpg)
…data get you above the average
@ksavenkov
![Page 14: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/14.jpg)
MasteryBusiness Metamorphosis
Data AwareData Monetisation
Data-drivenBusiness Optimisation
AnalyticsBusiness Insights
Data-Driven Maturity Index
Collect DataBusiness
Monitoring
@ksavenkov
![Page 15: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/15.jpg)
@ksavenkov
![Page 16: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/16.jpg)
To be data driven, it’s not enough to
collect and mine data. You need the culture.
@ksavenkov
![Page 17: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/17.jpg)
Don’t optimise for data! Optimise for learning!
@ksavenkov
![Page 18: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/18.jpg)
All initiatives must improve KPI
Formulate hypotheses
Run experiments@ksavenkov
![Page 19: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/19.jpg)
Data validate hypotheses
behind your own decisions.
Data does not make decisions for you!
@ksavenkov
![Page 20: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/20.jpg)
DATA
CulturePeople
ProcessTechnology
Analytics Business model
PlanningIterations
KPI
Hypotheses
Experiments
Changes
@ksavenkov
![Page 21: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/21.jpg)
ISetting Goals
and KPI
@ksavenkov
![Page 22: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/22.jpg)
Looks simple
MRR MarginMAU
but there are issues
@ksavenkov
![Page 23: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/23.jpg)
IT’S UNCLEARwho’s responsible?
how to imrove?
how does that relate with resources spent?
(besides CEO)
(besides “work better”)
(when most of the business processes are automatic)
@ksavenkov
![Page 24: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/24.jpg)
KPI Trees
MAU
New Loyal Returned
Traffic Conversion Retention Reactivation
@ksavenkov
![Page 25: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/25.jpg)
KPI TreesMRR
MRR stable net new MRR
list MRR
MRR upgrades
MRR downgrades
New clients
MRR expansion
new MRR
Reactivation
direct sales
@ksavenkov
![Page 26: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/26.jpg)
KPI Trees
Margin
LTVCAC
COGSARPU
Lifetime
Commissions
@ksavenkov
![Page 27: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/27.jpg)
ADVANCED MODE
Ratio Analysis
Cohort Analysis
@ksavenkov
![Page 28: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/28.jpg)
HELPS TOIdentify bottlenecks
Define roles and responsibilities
Generate ideas
Measure impact of initiatives
@ksavenkov
![Page 29: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/29.jpg)
IIMeasure Impact
of Initiatives
@ksavenkov
![Page 30: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/30.jpg)
For planning, groups tasks in Epics that matter
Track results of launched Epics
A culture of data success learning
@ksavenkov
![Page 31: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/31.jpg)
Case studyMarketing project were tracked until launched
Study of a half-year long cross-promo campaign discovered a significant loss via COGS.
We’e identified key mechanics that led to the loss.
We’e adjusted the mechanics, re-negotiated with partners on the ongoing campaigns.
PROFIT@ksavenkov
![Page 32: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/32.jpg)
IIIHypotheses and
experiments
@ksavenkov
![Page 33: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/33.jpg)
It doesn’t work this way:
marketing campaign
@ksavenkov
impact!
![Page 34: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/34.jpg)
product feature launched
billing failure
pupils back from holidays
AppStore featuring
people back from vacations
@ksavenkov
It doesn’t work this way:
marketing campaign
impact!
![Page 35: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/35.jpg)
product feature launched
billing failure
pupils back from holidays
AppStore featuring
people back from vacations
@ksavenkov
It doesn’t work this way:
marketing campaign
impact!
DIFFERENT CHANNELS CONVERT WITH A
DIFFERENCE IN ORDER OF MAGNITUDE
![Page 36: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/36.jpg)
ALSO:A lot of Epics that shouldn’t have been launched at all
A culture of praising a random success and explaining failures by external causes
Impossibility to learn by mistakes
@ksavenkov
![Page 37: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/37.jpg)
formulate measurable hypotheses
carefully plan experiments
define a condition of success/failure prior to implementation
data collection and attribution, split-testing, controlled variables,
statistical significance
demonstrate explicit risks use models built on past data
prioritisation aid
and how the failure affects the roadmap
ability to prove multiple hypotheses simultaneously
@ksavenkov
![Page 38: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/38.jpg)
Case StudiesEstimating mechanics of marketing projects before they launched
Product improvement that results in 2.5x conversion, 2x lifetime
Split testing of targeting and creative materials for ad campaigns, resulting in conversion 2-3 times higher than organic
Immediate increase of “conversion” from all initiatives to successful ones
This approach is behind all conclusions in these slides
lots of failed experiments success despite of all external factors
NO OTHER WAY@ksavenkov
![Page 39: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/39.jpg)
Case Study: Recommender system for Conversion
• Hypotheses to prove:1. There’re enough users who will use RS output2. Their conversion will be above average
• A/B testing is the only way:– different channels convert with up to 20x difference– current traffic mix is unpredictable and hard to control
in case of app installs
• Do pilots:– Run with limited resources, then extrapolate and decide
if run full-scale
![Page 40: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/40.jpg)
Case Study: Recommender system for Conversion
![Page 41: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/41.jpg)
Let’s look at the economics
• In case of using a third-party RS on a CPO basis, in this case the CPO is limited by $0.14 (actually, much less)
• In case of a flat fee of $1000**/month, this is feasible starting from 7143 new subscribers/month, or $35K of marketing budget.
* CAC and marketing budget are model data** some arbitrary number
![Page 42: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/42.jpg)
IVLeading indicators
@ksavenkov
![Page 43: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/43.jpg)
Make decisions on monthly KPI or
finished projects is like going backwards
@ksavenkov
![Page 44: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/44.jpg)
Look ahead or at least watch your step, not
backwards
Make the data work for you
@ksavenkov
![Page 45: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/45.jpg)
Daily indicators
Incremental indicators
Leading indicators
Predictive models
spot problems and anomalies just in time
example: baremetrics.io
a perfect input for inbound marketing
accurate goals and perfect financial planning
@ksavenkov
![Page 46: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/46.jpg)
Case StudySome loyal subscribers churned away for 2-3 months, to come back later. It was historically attributed to holidays and other external factors.
The daily indicators have shown that all churned users subscribed on weekdays. What a riddle!
It turned out that on weekdays the code base is frequently deployed to the production server, flushing the message queue of subscription renewals.
The fix increased a lifetime of paying users by 20%
@ksavenkov
![Page 47: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/47.jpg)
Case StudyPredicting a lifetime for users that registered right now (10% accuracy)
Accurate unit economics for contracts with B2B2C partners,
content providers, pricing@ksavenkov
![Page 48: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/48.jpg)
Case Study
A probabilistic model for segmenting users
Input data for chained communication
(inbound marketing)
@ksavenkov
![Page 49: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/49.jpg)
Case StudyAccurate goals
loyal
new
churned
guaranteed growth?
inevitable stagnation!*
*unless KPI are increased@ksavenkov
![Page 50: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/50.jpg)
Case StudyAccurate financial planning
Operational model
Marketing plan-fact
Financialplan-fact
Marketing budget
CAC, conversion
organic forecast
deal terms
unit economics deals forecast revenue forecasts
@ksavenkov
![Page 51: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/51.jpg)
VBusiness process
automatisation
@ksavenkov
![Page 52: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/52.jpg)
Hire another employee
or train another model?
@ksavenkov
![Page 53: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/53.jpg)
Affects all business processes that scale linearly with a headcount:
customer supporteditorial office
content managementmarketing
@ksavenkov
![Page 54: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/54.jpg)
Case StudiesUser base grows from 1M to 2M, doubling the headcount in customer support?
Implemented auto-reply using our knowledge base and smart templates for support engineers
A number of markets increased, adding more editors?Created an algorithm to provide a short-lists
based on a user behavior
An amount of UGC explodes, more content-managers?
Improving reduplication and computer aid based on the collected data
@ksavenkov
![Page 55: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/55.jpg)
VIImproving the
Company Itself
@ksavenkov
![Page 56: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/56.jpg)
Operational analytics - improve the process of KPI improvement
(productivity)
@ksavenkov
![Page 57: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/57.jpg)
ITERATION
Case StudyImproving the Agile process
the expectations:
@ksavenkov
ROADMAPS DESIGN DEVELOPMENT QA SHIPPED
PLANNING
![Page 58: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/58.jpg)
ITERATION
Case StudyImproving the Agile process
the reality:
@ksavenkov
INFLATING EFFORT
ROADMAPS DESIGN DEVELOPMENT QA SHIPPED
BACKLOG
NEW STUFF
TECH. DEBT
PLANNING
BUGS
UX
SOFTWARE OPS
URGENT STUFF
UNCLEAR DESIGN
UNCLEAR TECH
ITERATING
![Page 59: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/59.jpg)
Data as an Asset
@ksavenkov
![Page 60: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/60.jpg)
Case Stuies• Compare B2B2C deals through unit economics
• Estimate traffic quality for partner ad networks
• Data partnerships
• Targeted user communication
• Personalisation and recommender systems (the next slide)
• Bonus track: Investigating a large number of purchase returns for an internet retailer
@ksavenkov
![Page 61: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/61.jpg)
Using Recommender Systemsand personalisation
CAC
LTV
ContentCosts
Marketing Expenses
New Customers
ARPU
Lifetime
Consumed Content Mix
Conversion
Retention
Reactivation
ExposedContent Mix
÷
×
* the recommendation fairy
*
![Page 62: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/62.jpg)
Innovative business experiments, as there’s no
recipes
To pioneer, you need to iterate quicker than competitors
@ksavenkov
![Page 63: Building a Data Driven Business](https://reader034.vdocument.in/reader034/viewer/2022042906/58a725ae1a28ab0d0d8b4f5d/html5/thumbnails/63.jpg)
Data, models built over the data and experimental results is the
main asset created and exploited
by the innovative business
@ksavenkov
“THE UNFAIR ADVANTAGE”