[wmd 2015] qualaroo >> sean ellis, "the advantages & challenges of high tempo...
TRANSCRIPT
Sean Ellis @seanellis
High Tempo TestingMoving Beyond Hustle to Building
a Predictable Growth Machine
Sean Ellis @seanellis
3,2,1... Hustle!
Sean Ellis @seanellis
Early Growth: Twitter Focused
Feedback: “I find growth articleson Twitter”
Followers: Easy to find people that tweet growth articles
Content: Long tail of content ideal for Twitter
Sean Ellis @seanellis
Goal: Keep Conversation Going
Sean Ellis @seanellis
Some Vacations...
Sean Ellis @seanellis
Viability Achieved by Month 3
Sean Ellis @seanellis
Goals Not Met After First Year
3 Flat Months!
Sean Ellis @seanellis
“Inputs are what youactually control.”
Andrew Chen
Sean Ellis @seanellis
Problem: Limited Testing
Had specific growth goals, but just said “test more”.
Team was too busy for much testing...
Building product
Executing Twitter Playbook
Engaging with community
Sean Ellis @seanellis
Solution: High Tempo Testing
Launch 3 Tests Per Week(Clear “input” goal)
Sean Ellis @seanellis
Result...
Started high tempo testing
Sean Ellis @seanellis
Hustle › Growth Process
Sean Ellis @seanellis
High Tempo Testing Process
BuildKnowledgeBase
UnbridledIdeation
PrioritizeBacklog
HighTempoTesting
Sean Ellis @seanellis
Challenges of High Tempo Testing
1. Enough ideas: Ideas are fuel for tempo
2. Prioritizing ideas: Finding best ideas to test
3. Implementing tests: Getting the resources
4. Accurate testing: Instrumenting & analyzing
5. Avoiding repeats: Not running same tests
Sean Ellis @seanellis
Challenge 1: Enough Ideas
Unbridled ideation from full team
Ideas across all vectors
Individual & Team Accountability
Acqusition ActivationRetention
RevenueReferral
Sean Ellis @seanellis
Challenge 2: Prioritizing Ideas
Formalize ideas in “Experiment Doc”
Include research, hypothesis, target lever...
Score on impact, confidence & ease
Sean Ellis @seanellis
Challenge 3: Implementing Ideas
Weekly growth meeting to plan/assign current week’s tests. Accountability for previous week.
Recommend Autonomous growth team with dedicated development, design, analysis...
Empower less technical team members w tools
Sean Ellis @seanellis
Challenge 4: Accurate Testing
ReadPeep Laja(Next presenter)
Sean Ellis @seanellis
Challenge 5: Avoiding Repeats
Capture all tests in Knowledge Base
Ensure knowledge base shared, clear & searchable
Worked Failed Inconclusive
Sean Ellis @seanellis
High Tempo Test Requirements
Process People System
BuildKnowledgeBase
UnbridledIdeation
PrioritizeBacklog
HighTempoTesting
Sean Ellis @seanellis
High Tempo Testing Process
BuildKnowledgeBase
UnbridledIdeation
PrioritizeBacklog
HighTempoTesting
Sean Ellis @seanellis
Winning Test Example via Process
Moved email collector from bottom to top.
Result: 700% increasein collected emails
Impact prediction was4/10
Sean Ellis @seanellis
High Tempo Testing Advantages
1. Predictable Growth
2. Scaleable Growth
3. Easier Hiring
4. Address Bottlenecks
Sean Ellis @seanellis
Thank you!@seanellis