increase the roi of your betas - amazon web services · 2017-05-17 · increase the roi of your...
TRANSCRIPT
I N C R E A S E T H E
ROI OF YOUR BETAS
Luke Freiler: CEO and Founder of Centercode
‹#›
AGENDA
1. BETA TESTING BASICS
2. EXAMPLE BETA ROI
3. THE TYPICAL ROI OF BETA
4. INCREASING THE ROI OF BETA
5. BEST PRACTICES FOR A HIGHER ROI
6. NEXT STEPS + Q&A
Founder & CEO. We offer software, managed services, pre-qualified testers, andeducational resources to help companies improve beta testing ROI.
Automotive
Web
Robotics
Hardware
Accelerator Smart Home Wearables
EnterpriseSoftware
Startup
Gaming B2B
Telecom
Mobile
OU
R C
LIE
NT
S
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
‹#›
A test of an unreleased product byreal customers in real environments.
LEVEL SET
WHAT EXACTLY IS
BETA TESTING?
‹#›
MANY NAMES
Customer Validation (CV)
Customer Acceptance Testing (CAT)
User Acceptance Testing (UAT)
Friendly User Testing (FUT)
Field Trials
Pre-Release
Early Release
Enhanced Testing
User Testing
Customer Testing
Target Market Testing
In the Wild Testing
Beta Testing (we like this one)
WHY IT’S
CRITICAL
A beta test validates the total performance ofthe product in the hands of real customers.
HOW IT WORKS
1. Recruit a pool of candidates from your target market
2. Select qualified candidates as participants
3. Prep and NDA participants
4. Distribute product
5. Solicit feedback
6. Evaluate, distribute, and implement the results
HOW IT’S
CHALLENGING
1. Dedicating qualified resources
2. Recruiting appropriate beta testers
3. Maintaining user participation
4. Collecting relevant feedback
5. Evaluating and distributing feedback
WHY IT’S
SO PAINFUL
1. It’s probably not your full-time job
2. It typically lacks a well-defined process
3. It frequently has no clear owner and/or contributors
4. You likely lack appropriate tools to help
5. The timing is horrible
6. It involves real customers using a buggy product
7. You’re likely worried about controlling information
8. If you succeed, you’ve got a ton of data to deal with
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
IMPACT OF BETA TESTING
Sales and Costs
Estimated Sales 200k units
Unit Sale Price $100
Unit Cost $50
Beta Unit Cost $200
Incentive Costs $50/user
Product and Market
Release Type New
Product Type Hardware
Target Market Consumer
Competition High
Complexity High
Quality Confidence High
Beta Management / Resources
Tools Dedicated
Experience Part-Time
Wage $40/h
Desired Testers 30
Participation Rate 30%
Duration 4 Weeks
EXAMPLE ROI
ELEMENTS &
ASSUMPTIONS
Key Facts No Beta Basic Beta
Beta Management Hours 0 73
Total Beta Testers 0 100
Participation Rate (%) 0% 30%
Defects Found 0 41
Defects Fixed 0 25
Total Units Sold 190,000 200,000
Customers Requiring Support (%) 38% 36%
Product Return Rate 19% 19%
EXAMPLE BETA RESULTS
Financial Impact No Beta ($) Basic Beta ($)
Total Beta Management Cost - (5,491)
Total Logistics & Production Cost - (25,000)
Total Defect Cost (585,000) (562,250)
Total Support Cost (806,614) (787,510)
Total Return Cost (1,087,242) (1,092,025)
Total Warranty Cost (665,000) (633,333)
Gross Sales Profit 9,500,000 10,000,000
Net Gain - 538,246
TOTAL ROI 946%
FINANCIAL IMPACT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
BUG
REPORTS
‹#›
FEATURE
REQUESTS
‹#›
LATE STAGE
MARKET
RESEARCH
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
BETA
MIRRORS
YOU
BETA TOUCHES
THE ENTIRE
PRODUCT
BETA
REPRESENTS
THE CUSTOMER
A
B
‹#›
BETA IS AN
ARBITRATOR
BETA
IDENTIFIES
PRIORITY
BETA IS FOR
PREPPING
SUPPORT
BETA COVERS THE
ENTIRE CUSTOMER
EXPERIENCE
‹#›
BETA IS A
TOOLBOX
BETA IS FOR BUILDING
RELATIONSHIPS
BETA IS
KNOWLEDGE
BETA IS A
SAFETY NET
BETA ASSESSES
LAUNCH READINESS
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
1 FOCUS ON GOALS
IDENTIFY YOUR CONSTRAINTS
DETERMINE YOUR OBJECTIVES
MAP YOUR GOALS TO A SCHEDULE
DEFINE YOUR TARGET MARKET
2 PICK GREAT TESTERS
QUALIFY YOUR TESTERS
AVOID ARM’S REACH
SEEK OUT ENTHUSIASM
LIMIT YOUR POOL
3 BE BETA READY
READY YOUR SYSTEMS
READY YOUR TEAM
READY YOUR PRODUCT
READY YOUR TESTERS
4 REDUCE FRICTION
MESSAGE CLEARLY
OFFER SIMPLE FEEDBACK CHANNELS
PROVIDE SUPPORT
BE RESPONSIVE
t
5 EMBRACE THE RESULTS
FREQUENCY * SEVERITY = IMPACT
SHARE, BUT AVOID A FIREHOSE
INTEGRATE IF POSSIBLE
CONSIDER FEEDBACK IN CONTEXT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
CALCULATING
YOUR OWN
BETA ROI
FR
EE WWW.CENTERCODE.COM/ROI
WWW.CENTERCODE.COM/MYROIDIY
GUIDED
Thank you for attending.
Please join us for our next webinar on pricing, April 25th.
Download this webinar and sign up for more at
pragmaticmarketing.com/pragmaticlive