me - responsive design tested
DESCRIPTION
What a recent experiment reveals about the potential ROI of mobile designTRANSCRIPT
Responsive Design Tested What a recent experiment reveals about the potential ROI of mobile design
We’re sharing on Twitter!#WebClinic
Today’s speakers
Austin McCraw Senior Editorial Analyst
MECLABS
Jon Powell Senior Manager, Research
and Strategy MECLABS
What is responsive design?
ResponsiveUnresponsive
What is responsive design?
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
Friction: Psychological resistance to a given element in the conversion process, typically in the form of:
• Unnecessary length
• Unnecessary difficulty
What impact does responsive design have on friction experienced by the customer?
?
Background: A large news media organization trying to determine whether it should invest in responsive mobile design.
Goal: To significantly increase the number of free trial sign-ups.
Research Question: Which design will generate the highest rate of free trial sign-ups: responsive or unresponsive?
Test Design: A/B multifactor split test
Experiment ID: TP1933Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Experiment: Background
Experiment: Control (unresponsive)Desktop Tablet
*Android devices also included in this test
Experiment: Control (unresponsive)Mobile
*Android devices also included in this test
Desktop
Experiment: Treatment (responsive)
*Android devices also included in this test
TabletDesktop
Experiment: Treatment (responsive)
*Android devices also included in this test
MobileDesktop
Experiment: Side-by-side
ResponsiveUnresponsive
*Android devices also included in this test
Experiment: Results
Design Conversion Rate
Control – Unresponsive 3.71%
Treatment – Responsive 5.80%
% Relative Change: 56.41%
Aggregate increase in conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control.
*Android devices are also part of these results
Why did the responsive design outperform the unresponsive by so much?
?
Experiment: Results
Design Conversion Rate
Control – Unresponsive 3.71%
Treatment – Responsive 5.80%
% Relative Change: 56.41%
Aggregate increase in conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control.
*Android devices are also part of these results
Experiment: Results
Aggregate Increase in Conversion 56%The responsive design treatment showed an aggregate statistically significant difference in free trial subscriptions when compared to the control
Treatment – Responsive 5.80%
Design Conversion Rate
Control – Unresponsive 3.71%
+ Desktop %
+ Tablet %
+ Mobile %
+ Desktop %
+ Tablet %
+ Mobile %
Which device makes the greatest contribution to the total difference??
Today, we are going to review test data for each device category to determine its total impact on perceived friction.
!
Desktop PerformanceResponsive design tested
Experiment: Desktop control
Unresponsive
• For most screen sizes, all primary content is accessible without action
• The primary content is framed within a faux pop-up style box over a faded background
Experiment: Desktop treatment
Responsive
• Like the control, all primary content is accessible without action
• The faux pop-up frame, however, is completely eliminated
Experiment: Desktop side-by-side
Unresponsive Responsive
Experiment: Desktop results
Design Conversion Rate
Desktop Control – Unresponsive 2.65%
Desktop Treatment – Responsive 5.43%
% Relative Change: 104.57%
Increase in conversion 105%The responsive design desktop treatment showed a statistically significant difference in free trial subscriptions when compared to the control.
What was actually responsible for the difference in performance? ?
Tablet PerformanceResponsive design tested
Experiment: Tablet controlUnresponsive
iOS: • The majority of the content is
accessible without action (supporting image requires scrolling)
• A portion of the bottom of the page shows the pop-up
Android: • All of the content is accessible
without action; may require zoom for certain devices
• Supporting imagery and aesthetic quality is lost in the page load
Experiment: Tablet treatment
iOS: • The supporting image is
immediately visible and the pop-up background is completely eliminated
Android: • Significant improvement in
aesthetic and page load, though the supporting image is still lost
• Only vertical scrolling required for ancillary content
Responsive
Experiment: Tablet side-by-side
Unresponsive Responsive
Experiment: Tablet results
Design Conversion Rate
Tablet Control – Unresponsive 7.24%
Tablet Treatment – Responsive 8.81%
% Relative Change: 21.6%
No significant difference in conversionThe responsive design tablet treatment showed no statistically significant difference in free trial subscriptions when compared to the control.X
Why did the same treatments perform differently on different devices??
Mobile PerformanceResponsive design tested
Experiment: Mobile control
iOS: • Requires scrolling to read complete
sentences
• Also requires scrolling to see all required fields and the primary call-to-action
Android: • Requires one to two pinches to zoom
movements and read the content
• When zoomed in, requires horizontal scrolling to read complete sentences
Unresponsive
Experiment: Mobile treatment
• Requires vertical scrolling to get to all fields as well as the call-to-action
• Complete sentences are available without action
• Generally a consistent experience across major device platforms
Responsive
Experiment: Mobile side-by-sideUnresponsive Responsive
Experiment: Mobile results
Design Conversion Rate
Mobile Control – Unresponsive 5.14%
Mobile Treatment – Responsive 4.34%
% Relative Change: -15.6%
No significant difference in conversionThe responsive design mobile treatment showed no statistically significant difference in free trial subscriptions when compared to the control.X
Experiment: All results
% Relative Change: 56.41%
Treatment – Responsive 5.80% +56.4%
Design Conversion Rate % Rel. Change
Control – Unresponsive 3.71% -
+ Desktop 2.65% -
+ Tablet 7.24% -
+ Mobile 5.14% -
+ Desktop 5.43% +104.6%
+ Tablet 8.81% +21.6%
+ Mobile 4.34% -15.6%
What is the customer wisdom? Why did the treatment win in aggregate?
?
What you need to understand
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
The mobile and tablet treatments made significant changes in layout; they did not, however, make significant changes in frictionexperienced by the customer
Friction
What you need to understand
C = 4m + 3v + 2(i - f) - 2a©
The MECLABS Conversion Index
The desktop treatment did something slightly different
The move to responsive design coincidentally reduced both friction and anxiety by eliminating the pop-up appearance of the control
Friction
Anxiety
Responsive design tested
FKey Principles
1. Responsive design, like any new technology or technique, does not necessarily increase conversion. This is because we are not optimizing design, we are optimizing thought sequences.
2. Responsive design is only as effective as the degree of change it affects in the mind of the customer.
3. This requires you to consider not only how responsive design will impact the look of the page, but the customers perception on two levels:• Friction: How much will responsive design decrease perceived effort?
• Anxiety: How much will responsive design decrease perceived concern?
EXCLUSIVE SAVINGS
Save an EXTRA $200 on Email Summit Tickets
Use promo code:
515-ST-1010*Offer expires January 24
Background: A large technology company selling software to small businesses.
Goal: To significantly increase the number of software purchases.
Research Question: Which design will generate the highest rate of software purchases?
Test Design: A/B multifactor split test
Experiment ID: TP1933Record Location: MECLABS Research LibraryResearch Partner: [Protected]
Experiment: Background
Experiment: Side-by-side
COLUMN #1 COLUMN #2COLUMN #1
Control Treatment
Which treatment won?
Live: Jan. 22 at 4:00 p.m. EST
• What is the optimal number of columns for a page?
• How should columns be weighted on a page?• Which is better: a right sidebar or left
sidebar?
Watch live or get the full 35-minute video replay
To see the results …
To watch live or view the full video replay when it is available, register at the link below:
MarketingExperiments.com/Columns
See how you can conduct research with usMECLABS conducts rigorous experiments in the new science of optimization. We apply our discoveries to help leaders optimize the financial performance of their sales and marketing programs.
Learn more about how you may be a fit for a MECLABS Research Partnership:
• Select Research Partnership Opportunities on the post-webinar survey
• Contact us [email protected]
x