optimizely workshop: mobile walkthrough
Post on 16-Apr-2017
750 Views
Preview:
TRANSCRIPT
Mobile Walkthrough: From 0 to Experiment
Pamela Ongchin Mobile Solutions Architect
An Optimizely Online Workshop
A few tips:This webinar will be recorded.
We strongly encourage you to take notes!
Feel free to ask questions at any time.
There will be a live Q&A at the end of this session.
1. What are the biggest challenges in mobile app development? How can I address these with experiments?
2. How do I set up an Optimizely mobile experiment?
3. How does A/B testing integrate with my development process?
4. What are best practices when designing experiments?
After today’s workshop, you should be able to answer the following questions
But first, tell us a little more about you!
What department do you work in?
MarketingEngineering
Product
poll
What is the most challenging issue that you face in your mobile app?
question
On average, apps lose 80% of their new usersSource: Andrew Chen’s blog, http://andrewchen.co/
Some of the top apps in 2014
The top apps follow the same patternSource: Andrew Chen’s blog, http://andrewchen.co/
It’s difficult to keep users engaged
80% of new users drop off validate new features
It’s difficult to be agile …
high technical demands slow release process
one-star reviews
Common pain points:
• Measure the effectiveness of new features.
• Make data-driven decisions on what to build next based on test results.
• Develop experiences that retain users and extend lifetime value.
A/B testing can help you become more agile
The stylist and customer were 43% more likely to connect and ship a trunk of clothing, a key step in Trunk Club’s customer conversion funnel
Due to higher quality of leads from the app, stylist productivity improved 90%
No credit card required Credit card required
Okay, so A/B testing can help me be more agile.
What should I test?
What is the one key business metric that matters most for your app?
RevenueSignups
Engagement
poll
Key Metrics for Mobile Apps
Key Metrics for Mobile Apps
Opportunities to A/B test
1. Onboarding
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
1. Onboarding (show value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
Secret Escapes Optimizes Acquisition
• Goal: Looking to optimize LTV, sign up rates, and justify mobile ad spend
• Experiment: Should the app require users to sign up in order to view travel deals?
Experiment Setup
Version 1 Version 2
Sign up gate with mandatory sign in on first user experience
Sign up not required, added a “skip”
button to close the sign up form
Results
• Defying all expectations, a mandatory signup gate was by far the optimal experience
• Sign up rates increased 200%, LTV average increased, and LTV:CAC ratio was positive
1. Onboarding (show value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
Add to cart (or bag)
1. Onboarding (show value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
In-app message: Upsell
1. Onboarding (show value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
Facebook layout test
1. Onboarding (show value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
Checkout Flow
1. Onboarding (first value, engagement)
2. Product detail screens
3. Promotions and upsell
4. Homescreen
5. Registration flow and checkout flow
Top 5 areas Optimizely mobile customers are testing
1. What are the biggest challenges in mobile app development? How can I address these with experiments?
2. How do I set up an Optimizely mobile experiment?
3. How does A/B testing integrate with my development process?
4. What are best practices when designing experiments?
After today’s workshop, you should be able to answer the following questions
Now, let’s create a mobile experiment.
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
4
5 Key Steps
5
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
5
5 Key Steps
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
4
5 Key Steps
5
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
4
5 Key Steps
5
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
4
5 Key Steps
5
Make the Variations What to test?
Set Audiences Who sees the experiment?
Set Traffic Allocation How many visitors are included? How are they split?
1
2
3
Set Goals Why are you testing? How will you measure success?
4
Preview & Test Does it work the way you want to?
4
5 Key Steps
5
1. What are the biggest challenges in mobile app development? How can I address these with experiments?
2. How do I set up an Optimizely mobile experiment?
3. How does A/B testing integrate with my development process?
4. What are best practices when designing experiments?
After today’s workshop, you should be able to answer the following questions
A/B Testing Timeline
Brainstorm
A/B Testing Timeline
Brainstorm Prioritize & Design
A/B Testing Timeline
Brainstorm Prioritize & Design
A/B Testing Timeline
Implement
Brainstorm Prioritize & Design QA & Approve
A/B Testing Timeline
Implement
Brainstorm Prioritize & Design QA & Approve Run
A/B Testing Timeline
Implement
Brainstorm Prioritize & Design QA & Approve Run Analyze &
Review
A/B Testing Timeline
Implement
A/B Testing Timeline
1. What are the biggest challenges in mobile app development? How can I address these with experiments?
2. How do I set up an Optimizely mobile experiment?
3. How does A/B testing integrate with my development process?
4. What are best practices when designing experiments?
After today’s workshop, you should be able to answer the following questions
• Know what tests you want to run at the beginning of your sprint.
• Have a developer involved in the process.
• Use phased rollouts for new features.
• Use live variables and code blocks.
• Integrate Optimizely with your analytics platforms.
Best Practices
Pop Quiz!
If the Optimizely SDK is installed, you can make visual changes to your app in between
releases without additional code.
TrueFalse
poll
If the Optimizely SDK is installed, you can make visual changes to your app in between
releases without additional code.
True
poll
If I want to test an entirely new feature in my app, I need to get my developer involved.
TrueFalse
poll
If I want to test an entirely new feature in my app, I need to get my developer involved.
True
poll
Use A/B to validate new features.
A/B testing is a huge opportunity to improve metrics in your app.
Once your SDK is installed (by your developer), you’ll be able to make visual changes without additional code. Testing new features requires developer involvement.
Final Takeaways
Q&A
Have a question? Type it into the chat box!
top related