growth experiments: lessons from a failed attempt
TRANSCRIPT
![Page 1: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/1.jpg)
growthGrowth Experiments: Lessons from a failed attempt
![Page 2: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/2.jpg)
about meVassilena Valchanova
• 10 years of marketing experience
• Check out valchanova.me
• Digital Strategist @ Enhancv
![Page 3: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/3.jpg)
growthAbout Enhancv
• B2C SaaS startup
• Smart resume building platform
• 3 years old
• 300,000 users
![Page 4: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/4.jpg)
our storyOur Growth Team Story
WHY?
Speed up and learn more
HOW?
Company-wide growth
experiment process
![Page 5: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/5.jpg)
our storyThe Death of Experiments
• 10 experiment meetings
• 6 experiments carried out
• Participation from
different team members
![Page 6: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/6.jpg)
our storyTime to Reset
RESET
![Page 7: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/7.jpg)
The Growth ProcessPROCESS
GROWTH
Analyze
1.
Set
Objective
2.
Ideate
3.
Prioritize 4.
Test
5.
![Page 8: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/8.jpg)
growthBuild YOUR TEAM
![Page 9: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/9.jpg)
setupBuild Your TeamGROWTH LEAD
• Sets the process
• Leads the meetings
• Clears obstacles
MARKETER
• Acquisition
• Copy
• Driving traffic
PRODUCT/DEV
• Technical
implementation
• Automation
DESIGNER
• Mockups
• Wireframes
• Marketing visuals
DATA ANALYST
• Initial data
• Experiment setup
• Analysis
CUSTOMER SUCCESS
• Customer feedback
• Critical questions
• Early results
![Page 10: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/10.jpg)
our storyAlways Use The Same Team
• Ownership
• Helps them prepare
• Teams require time to
form
![Page 11: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/11.jpg)
growthEstablish CLEAR RULES
![Page 12: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/12.jpg)
setupClear Rules
• Is the experiment clear and
specific?
• Is the experiment related to the
objective?
• Can it be executed and analyzed
in 2 weeks?
![Page 13: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/13.jpg)
setupDictatorship is good in meetings
![Page 14: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/14.jpg)
growthStart with a SPECIFIC PROBLEM
![Page 15: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/15.jpg)
analyzeSpecific Problem
• Work with data
• Growth lead and management
decide
• Concrete objective
![Page 16: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/16.jpg)
analyzeWhat’s Concrete?
Our current blog new visitors is 4,000 per month.
We need to increase it by 15% without losing the
quality of the audience.
Let’s focus on acquiring new users
![Page 17: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/17.jpg)
growthRespect everyone’s TIME
![Page 18: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/18.jpg)
ideateICE scores are cool1-10 average of three scores:
IMPACT
If successful, how big is the difference?
CONFIDENCE
How sure are we it will work?
EASE
How easy is it to implement?
(time/people/money)
![Page 19: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/19.jpg)
ideateRespect everyone’s time
• Team members write down
ideas before the meeting.
• Each scores their idea.
• Only doubtful scores are
discussed.
• Scores are not the last word.
![Page 20: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/20.jpg)
growthEnsure there is TEST CADENCE
![Page 21: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/21.jpg)
testTest Cadence
• Two week experiment
cycles
• 5 experiments per cycle
• 30% success rate
• 32 successful experiments
per year
![Page 22: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/22.jpg)
growthSetup experiments THE RIGHT WAY
![Page 23: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/23.jpg)
testExperiment TypesA/B Tests Testing 2 versions simultaneously, only 1 difference per version.
Multivariate Tests Testing many versions at the same time, lots of differen-ces per version.
Sequential Tests Testing many versions one after the other.
Smoke Tests Landing page collecting leads to gauge interest.
Trigger Tests Measuring click rate of ads to see interest in a value proposition.
Wizard of Oz Deliver value by hand, if a feature requires lots of dev.
High Hurdle People pay with time (surveys, long forms) to see how engaged they are with a proposition.
Design your Own There are other options…
![Page 24: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/24.jpg)
testGet to Know Statistics
• VWO’s Significance Calculator
• Optimizely’s Sample Size Calculator
• Google Optimize
![Page 25: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/25.jpg)
growthSettle on a working TOOLSET
![Page 26: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/26.jpg)
testTools to Try
![Page 27: Growth Experiments: Lessons from a failed attempt](https://reader033.vdocument.in/reader033/viewer/2022050614/5a6d35127f8b9ade418b4fd9/html5/thumbnails/27.jpg)
growthThank you! Any Questions? The slides: valchanova.me/slides