5 key measurements of product success
TRANSCRIPT
5 Key Measurements of Product SuccessAssessing the health of your customers and your products
2
Introductions5 KEY MEASUREMENTS FOR PRODUCT SUCCESS
What is Pendo? Who am I?
A venture-backed start-up based in Raleigh, NC. Pendo extends your product to capture all user
behavior, gather feedback, and guide users
Head of Product Marketing at PendoPreviously Product Marketing and Demand Gen at IBM, Product Manager at Quadbase Systems
3
How We Used To Measure Things5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
● Feedback
● Transactional Data
● (Some) User Observation
4
Our Development Path5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
● Idea / Intuition
● Sell
● Build
● Sell
● Feedback
● Revenue?
5
Transactional Data Says...5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
v1.0 Licenses sold: 1
6
The World is Different Now5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
Big Data SaaS Agile
7
Powering a New Product Experience Cycle5 KEY MEASUREMENTS OF PRODUCT SUCCESS
GoalsMetrics & Results
1. User OnboardingNew user becomes proficient with basic application functions, and account is setup (if needed). Onboarding experience is tailored based on external customer data
2. Product IntelligenceUser interactions with the application are measured, and feedback is captured.
3. Adaptive InterfaceCustomized guidance based on product intelligence helps the user get more value from the application. The UX itself adapts to the user’s context
4. Continuous EducationThe product is updated, and new learning content is delivered allowing the user to become proficient with the new version of the product.
8
We Can Measure Everything...5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● Unique visitors
● Pageviews
● Logged-in users
● Login frequency
● Clicks, focus, or hover events
● Load times
● Time in the application
● Enter / exit points
● Task completion
● Device, OS, and browser
● Session path
● Satisfaction ratings
...But Which Ones Matter?
9
It’s Ultimately About Revenue5 KEY MEASUREMENTS OF PRODUCT SUCCESS
10
And Revenue is Driven by Churn5 KEY MEASUREMENTS OF PRODUCT SUCCESS
11
5 Key Measurements of Product Success5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Breadth
How many active users do I have?
Frequency
How often do user login?
Depth
How much of my product do they use
Efficiency
How hard is it for users to complete tasks?
Satisfaction
How do employees rate the community experience?
12
Breadth of Use5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How many total users does my product have?
● How many of them are active users?
● Which customers / accounts are they from?
13
Measuring Visitors & Accounts5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Your SaaS Product
Account A Account B Account C
Visitors Visitors Visitors
14
Breadth Measurements5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Healthy> 5 Active Users
Improvement2 - 5 Active Users
Risk< 2 Active Users
15
Frequency of Use5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How often do my users login?
● How much time do they spend in the application when they do?
● Which customers / accounts are they from?
16
Frequency Measurements5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Healthy> 10 Logins per
Month
Improvement2 - 10 Logins per Month
Risk< 2 Logins per
Month
17
Segment Analysis5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Tracking breadth and frequency across groups of users
Account Age Account Size User Role
✓ Quarterly groups of new customers
✓ Enterprise vs. SMB customers
✓ Access across departments
18
Depth of Use5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● Which features get the most use, and how often are they used?
● Which are the most valuable to specific user segments?
● What percentage of the product gets regular use?
19
Understanding Key Features5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Core functionality common to the most productive users
20
Key Features for Pendo5 KEY MEASUREMENTS OF PRODUCT SUCCESS
TAG FEATURES & PAGES PUBLISH A GUIDE INTEGRATE WITH SFDC
LAUNCH A POLL CREATE A SEGMENT
21
Depth Measurements5 KEY MEASUREMENTS OF PRODUCT SUCCESS
HealthyUsed All Key
Features
ImprovementUsed 2 / 5 Key Features
RiskUsed < 2 Key
Features
22
Usability is Critical5 KEY MEASUREMENTS OF PRODUCT SUCCESS
23
Efficiency of User Actions5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How useable is the overall user experience?
● What features and functions do users struggle to understand?
● Which step(s) cause users to abandon a process?
Step 1
Step 2: 70%
Step 3: 50%
Step 4: 25%
24
How to Measure Efficiency5 KEY MEASUREMENTS OF PRODUCT SUCCESS
25
Efficiency Measurements5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Very Usable60%+ Completion
Rate
Usable30 - 60% Completion Rate
Users Struggling< 30% Completion
Rate
26
Satisfaction and Feedback5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● What is overall user satisfaction?
● Which is the motivation behind observed user behavior?
● How useful are specific features? How can they be improved?
27
What is NPS?5 KEY MEASUREMENTS OF PRODUCT SUCCESS
"Net Promoter System is based on the fundamental perspective that every company's customers can be divided into three categories. Customers can be categorized based on their answer to [our] ultimate question...
“The best way to gauge the efficiency of a company's growth engine is to take the percentage of customers who are promoters and subtract the percentage who are detractors. ”
~ Bain & CompanyNPS was introduced in the BHR in 2003
28 5 KEY MEASUREMENTS OF PRODUCT SUCCESS
“How likely are you to recommend us?”
29 5 KEY MEASUREMENTS OF PRODUCT SUCCESS
30
Additional Feedback - Feature Input5 KEY MEASUREMENTS OF PRODUCT SUCCESS
31
Additional Feedback - Demographic Information5 KEY MEASUREMENTS OF PRODUCT SUCCESS
32
Sources of Product Data5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Internal objects
Key application stats such as users, licences, etc, that is stored as part of application data
Web analytics
Page-level and user data captured by instrumenting application pages with Google Analytics or other web analytics software
Support cases
Current or archived support requests from help desk (kana, zendesk) or other repository software
User testing & surveys
Qualitative user feedback from observed user experience (UX) testing sessions and user surveys
33
Begin Optimizing Your Product Experience5 KEY MEASUREMENTS OF PRODUCT SUCCESS
GoalsMetrics & Results
