metrics and analytics, guest lecture, ucla
DESCRIPTION
Provided hour lecture on metrics and analytics for Jaime Levy's UX Design course at UCLA Extension.TRANSCRIPT
Analytics & MetricsUCLA Extension (Guest Lecture)
Darren Levy- Senior Director of Revenue, Retention at Mylife.com- Founder of Gatherspace.com
True or False:
Technology is the hardest part of building
an online business?
True or False:
Knowing the metrics is someone else’s job.
Breakdown of today’s discussion
- Part 1 – Definitions, why they are so critical- Part 2 – What metrics to focus on?- Part 3 – Optimizing metrics- Part 4 – Case Studies
I. Why do we need metrics?
What are metrics, why do we need them?
- Small units of measurement, typically ratio’s.- How are we doing? - Is business healthy?- Is the customer happy, is the product working?
99.1% Pure
…and what about analytics?
- Combining metrics, finding patterns of behavior.- What should we be working on? - Should we pivot?
Google Analytics – be cautious
- Storing all of your proprietary data with Google- Limitations on customizations- When shit hits the fan, getting customer support- You NEED to slice and dice and query your data
Why are metrics so importantfor YOU?!
- Be the Expert, be the Hero, the “go to” person- Don’t rely so much on others- Because we’re all Liars
Ice Cream Glove
Tightly ingrained withinLean Startup Process
Hypothesis
Experiment
Metrics
Validation
II. What metrics to focus on?
Vanity Metrics
- Not actionable- Not understandable- Examples:
• traffic • time on site• cumulative registrations• # of visits• Overall conversion rate
Single Key MetricOMTM
- Analysis Paralysis (don’t overdo it!)- For startups, business rally around a metric- Changes over time as business progresses
Customer Lifecycle – 5 Steps to SuccessAARRR!
Dave McClure’s Pirate MetricsAARRR!
AARRR Lifecycle
Acquisition Visit Site(or landing page, or widget)
* CPC* Keyword Rank* Ad CTR
CTR ~100%Value=.01
Category Lifecycle Experience Metrics Est Value
Activation Jump through initial hoop, signup (“a ha” moment)
* CTR CTR ~5%Value=$3
Retention Repeat Visitor, Open up emails
* Avg # of Visits* Engagement Score
CTR ~2%Value=$5
Revenue Customer generates revenue* RPV* AOV* ARPU
CTR ~1%Value=$25
Lean AnalyticsGrowth Stages (“Gates”)
Single Key MetricsFor Startups
How much do we spend to get a customer?Cost per Acquisition ($) CPA
How much do our customers pay us?Average Revenue per User ($) ARPU
At what rate are customers cancelling?Churn (%) Churn
How much is a customer worth over time?Lifetime Value per Customer ($)
LTV
Subscription v. Transactional Models
Subscription Business Transactional Business
VPR (value per reg)Churn / LTV / ARPUActivation & Engagement
Shopping Cart AbandonmentAverage Order Value (AOV)Upsell & Product Page CTR
III. How do we do this?
How do we get Metrics?
- Google Analytics- DIY – recording events, activities in a SQL db- AB testing- Other tools – Heatmaps
Types of AB Testing
- Simple AB testing- Multivariate testing- Price testing
Simple AB Testing
- Changing 1 element at a time?- Effecting CTR at any point in a funnel
Multivariate Testing
- More simultaneous variables- Takes longer to test- Better results than “simple”
Price Testing
- Testing different prices or terms- Not measuring CTR, measuring RPV- Pay attention to “Accumulation of Revenue”- Pricing Strategies (freemium, subsc., bundling)
Ugly Friend Concept
Famous Price Test (Ugly Friend)
- Decoy (ugly friend) pushes price selection- In this example, decoy pushed sales by 30%
Apple iPhone Pricing
~67% ~20% ~13%
10 Things you MUST Test
1. Calls to Action (CTA)2. Propositions3. Copy & Content4. Visual Media5. Funnel testing6. Forms 7. Emails 8. Pricing9. Shipping 10. Personalization
AB Test Example
Cohort AnalysisMeasuring engagement and value over time
Optimization via Heatmaps
Quantitative v. Qualitative Metrics
- Example of qualitative (fire alarm in middle of night, 99.97% up time, yet pissed off customers)- Why you need both. (user testing, surveys, call your customers)
IV. Case Studies
Price Test @ Dollarshaveclub.com
Control Test
4 7
7% higher CLTV4.4% higher 24 hr Net
Case Study: Netflix & House of Cards
- Goal to be “HBO” of online TV- Cost of $100m on production
- Analytics told them * Most customers watched David Fincher’s social network all the way* British version of “House of Cards” was highly watched* Significant overlap of British House of Cards watch Kevin Spacy films and Fincher films.
- Data so far * Q1 2013, brought in 3m new custmers* in Q1 alone, paid for cost of production* Increasing CLTV
Q&A