startup analytics

Download Startup analytics

Post on 01-Nov-2014

518 views

Category:

Documents

2 download

Embed Size (px)

DESCRIPTION

Presentation given to the Gener8tor portfolio companies on 1/31/2013.

TRANSCRIPT

  • 1. STARTUP ANALYTICSGetting Started Down the Path to Understanding Your Business and Your Users Dale Beermann Chief Technology and Analytics Ofcer dale@studyblue.com
  • 2. THE GOAL OF ANALYTICS: IMPROVING YOUR BUSINESS BYANSWERING AND ACTING ON QUESTIONSWith every question answered, ask yourself if it is the desired result. If not, determine what needs to be done to improve it.
  • 3. BUSINESS METRICS VERSUS USAGE METRICSYou should always be reporting on your business metrics.Analytics is the way to understand what is driving them. Effectively, business metrics are the aggregate result of your usage metrics.
  • 4. BUSINESS METRICSThe ultimate goal of business metrics is to evaluate the healthof your business. Examples: How fast is your business growing? What is your churn rate? What is your cost per acquisition for each channel? What is your Average Revenue per Active User?
  • 5. USAGE METRICSThe ultimate goal of usage metrics is to evaluate the health ofyour product. Examples: What percentage of your users are realizing your value propositions? Is your new feature reaching the expected audience? What percentage of users make it through the onboarding process? What percentage of users are using social channels?
  • 6. INFLUENCING BUSINESS METRICSKnow the answers to your high-level business metrics beforedigging into your usage.Use your usage metrics to determine how you can inuenceyour business metrics.
  • 7. BEST PRACTICES
  • 8. THE RIGHT TIME TO STARTHave you found your product/market t? There may be some high level business metrics that help you get there, but dont start your analysis on a product that is going through a massive amount of change.
  • 9. WHERE TO STARTHave you lled out a Business Model Canvas? What are yourbusiness most important metrics? How well are each of your customer segments doing when it comes to realizing your value propositions? Take your value propositions and work backwards through the paths that your users take to get there.
  • 10. DEVELOP GOOD HABITSMake Analytics a core part of your development workow.Ensure you are creating both good behavioral habits as well asgood programming habits.
  • 11. GOOD BEHAVIORAL HABITSReview your metrics on a regular basisContinually log changes that are going into your product There will inevitably be a point in the future where you ask yourself what happened six months ago to inuence a particular metric.
  • 12. GOOD PROGRAMMING HABITSCreate guidelines and tools that require you to implementmetrics as you build out your software E.g. Use abstract click handlers that can be easily refactored: display.addClassesHandler(new SBClickHandler(SBAnalytic.HOME_FIND_CLICK) { @Override public void doOnClick(ClickEvent event) { ... } });
  • 13. AVOID VANITY METRICSPage views dont matter (impressions may).Time On Site can be interesting, but doesnt necessarilyconvey usage.Its very difcult to inuence metrics like Page Views or Timeon Site. Attempting to do so will be a waste of your time.
  • 14. FOCUS ON ACTIONABLE METRICSThese are going to be different for every business.Again, you want to nd the metrics that mean the most toyour company and determine how you can inuence them.
  • 15. MAXIMIZING A METRIC CAN HAVE SIDE EFFECTSProviding multiple options splits your usage between them.Similarly, forcing users down one particular path means theycant take another. This can arise in subtle ways.In some cases, such as with a payment page, you may be ableto nd the optimal solution without many side affects.Ask yourself: What user segments are affected bythis change? Will any side effects be worth it?
  • 16. IMPLEMENTATION
  • 17. CAVEAT: I DO NOT SUBSCRIBE TO THE IDEATHAT YOU SHOULD LIMIT WHAT YOU TRACK. If you are smart about how youre doing your analysis, you will not fall into the trap of analysis paralysis.
  • 18. START WITH GOOGLE ANALYTICSIts free and you can throw everything at it without worryingabout usage tiers.We dont use the high level (vanity) metrics for much. Rather,by sending our events through Google Analytics, we have theability to answer a lot of questions.
  • 19. GETTING THE MOST OUT OF GOOGLE ANALYTICSTrack all of your events (views, clicks, actions). This isnt limited to your click stream. Track nal events for workows (e.g. completed_onboarding). This allows you to create Advanced Segments for those events.Set up proles for each platform (web, iOS, Android, etc.). Youre going to have very different usage patterns for each platform, and they should be analyzed separately.
  • 20. GETTING THE MOST OUT OF GOOGLE ANALYTICSMake use of custom variables. At the very least, you should be setting your (non personally-identiable) user ID as one of the variables. This will let you nd some per-user data that is otherwise difcult with Google Analytics. If you have organizational data, or if your users are segmented in pre-dened ways, this can help look at those segments more closely.
  • 21. IM TRACKING MY EVENTS. NOW WHAT?Funnel Analysis The goal of a funnel analysis is to determine where your users are falling off. Take one of your core metrics and walk through the steps it takes to get there.
  • 22. FUNNEL ANALYSIS EXAMPLEStudyBlue and IndexableContent We want to maximize the amount of content created that is paired with a class. How does that happen?
  • 23. HOW DO YOU IMPROVE YOUR FUNNELS?Think about how can you change an experience to improvethe end result. Sometimes this is as simple as changing a buttons color or using a modal popup (while thinking about the side effects).A/B Testing A/B Testing can be a reliable way to evaluate multiple paths. Caveat: Do your homework and understand statistical signicance. Learn what a chi-squared test is.
  • 24. TOOLS FOR FUNNEL ANALYSISGoogle Analytics does make it possible to do some of this. Their goal conversions are annoying if you dont use page views the way they expect. Create advanced segments for users with particular events.Other good for-pay tools are KissMetrics and Mixpanel.Roll your own. In all honesty, doing this stuff yourself isnt that hard.
  • 25. TANGENT: YOUR OWN IMPLEMENTATIONYoull want to use partitioned tables under the hood (if yourdata store supports it). In postgresql, we use triggers to write data to the correct table. Queries then only hit the necessary tables for the time span youve dened.We got away with a table per week for about 5 years. Ourtable schema: user_id, session_id, platform, activity_id, activity_timestamp, activity_detail
  • 26. ADVANCED ANALYTICS
  • 27. COHORT ANALYSISA cohort is a set of users grouped in a particular fashion.Typical cohorts are time-based (week of registration). Cohortscan also be based on acquisition campaigns (e.g. Adwords vs.Direct vs. SEO).The purpose of a cohort analysis is to understand userretention and if your changes are making an impact betweencohorts.
  • 28. WHY COHORT ANALYSISMost educated investors are going to ask for cohort analyses.Cohort analyses, and their corresponding retention rates helpdetermine: Engagement levels. Are you a one-and-done sort of site? Churn rates. If users arent coming back to your site, or if churn is higher than acquisition, your site will not grow. Quantifying the value of your existing userbase.
  • 29. A COHORT ANALYSIS