what do customers want? - meclabs...test background 10 10 background: kiehl’s wanted to present...

3 experts share lessons learned on the road to customer discovery

What Do Customers Want?

Lauren Pitchford Sr. Optimization Manager MECLABS

Ryan Hutchings Director of Marketing VacationRoost

Zee Aganovic Founder and CEO HiConversion

Play for a chance to win!

Winner will be randomly selected

What is your name?

Number: 22-333

Keyword: MyName

Please enter your first and last name to play

MarketingSherpa.com/LivePoll

OR OR

What is your name?

MyName Zee Aganovic

Zee Aganovic Founder and CEO

HiConversion

Keyword: MyName

Number: 22-333

Or go to MarketingSherpa.com/LivePoll. But you have to commit to one!

What is your name?

MyName Ryan Hutchings

Ryan Hutchings Director of Marketing

VacationRoost

Keyword: MyName

Number: 22-333


What is your name?

Keyword: MyName

MyName Lauren Pitchford

Lauren Pitchford Sr. Optimization Manager

MECLABS

Number: 22-333


Testing-optimization cycle boosts conversion

Source: MarketingSherpa.com/ecommerce

Challenge

How do we make our homepage slider more engaging?

1

3

5

1

2

3

4

5

Banner5 Banner3

Vote now! Text the keyword to 22-333 or go to MarketingSherpa.com/LivePoll

Test background

10

10

Background: Kiehl’s wanted to present the best-performing banners on its homepage slider with the optimal order and number included. Goal: To discover the most engaging slider content. Research Question: What is the optimum number of slider banners and the best order of play? Test Design/Optimization Engine: Multivariate/Adaptive

Campaign: Homepage Slider Design Client: Kiehl’s

Research Notes:

Experiment: Version A (control)

1

2

3

4

5

5-Banner Slider Animation

Experiment: Version B

1

3

5


Experiment: Side by side Version B Version A


1

3

5

1

2

3

4

5


Experiment: Results

! What You Need to Understand: Sliders are becoming common elements of e-commerce pages and provide an opportunity to effectively engage customers in the sales process.

Design RPV Lift

Version A (control) – 5-Banner Slider Animation -

Version B – 3-Banner Slider Animation 11.44%

Relative Difference +11.44%

11.44% increase in revenue per visit

Challenge

I sell a product that customers tend to try and buy in a brick-and-mortar store, how do I overcome the barrier to purchase?

WithoutCopy WithCopy


Test background

17

17

Background: A company that sells latex mattresses exclusively online; it has no brick-and-mortar stores or showroom abilities. Goal: To increase the amount of visitors purchasing a mattress. Research Question: Which page will result in the most conversions? Test Design: A/B split test

Record Location: MECLABS Research Library Research Partner: [Protected]

Research Notes:

Control: Without copy

Treatment: Supporting copy

Experiment: Side by side

Control

Treatment

Experiment: Results

! What You Need to Understand: By overcorrecting for anxiety in the direct eye-path, we were able to significantly increase conversion rate.

Design Conversion Rate

Control – Without copy .71%

Treatment – With supporting copy 1.23%

Relative Difference 74.26%

74% increase in conversions The treatment page increased the rate of conversion by 74%.

Challenge

How can I gain a higher clickthrough rate from a PPC landing page?

Above1 Below1


Test background

24

24

Background: A vacation booking site specifically for Park City, Utah. Goal: To determine if offering more options on a landing page would yield a higher CTR. Research Question: Which page would generate a higher CTR? Test Design: A/B split test

Record Location: VacationRoost Research Site: Park City Reservations

Research Notes:

Control: Above the fold

Treatment: Below the fold

More copy

CTA below the fold

Several CTAs

PPC Landing Pages: Side by side

Winner: Below the fold

58% increase in CTR

Challenge

How do I improve the buying experience while preserving brand image?

Clean Brand Look and Feel

CleanLook

Vote now! Text the keyword to 22-333 OR go to MarketingSherpa.com/LivePoll

Visible CTA with Product Description

Description

Test background

31

31

Background: With a high-fashion client like Elie Tahari, the brand image dominates the e-commerce experience. Goal: To explore product detail options that do not impact the brand image. Research Question: Will a more visible call-to-action and lengthier product description increase revenue? Test Design/Optimization Engine: Multivariate/Adaptive

Campaign: Product Detail Page Template Client: Elie Tahari

Research Notes:

Experiment: Version A

Clean Brand Look and Feel


Visible CTA with Product Description


Clean Brand Look and Feel Visible CTA with Product Description

Experiment: Results

! What You Need to Understand: The broad number of elements in the product detail template have a profound impact on e-commerce revenue results.

Design RPV Lift

Version A (control) – Clean Brand Look and Feel -

Version B – Visible CTA with Product Description 18.71%



Challenge

I have multiple levels of the same product. How do I push people toward the most lucrative one?

High2Low

Low2High


Popularity

Test background

38

38

Background: Home delivery subscription has multiple delivery options such as daily, weekend, weekday, Sunday only. Goal: To determine if reordering the delivery options would have an effect on the subscription mix and revenue. Research Question: Which order of subscription options will yield the most revenue? Test Design: A/B/C split test

Experiment ID: TP1772 Record Location: MECLABS Research Library Research Partner: [Protected]

Research Notes:

Control: Price high to low

Treatment 1: Price low to high

Treatment 2: Popularity

Results

! What You Need to Understand: By simply reordering the options displayed, we were able to significantly affect the product mix.

Design Revenue per Visit

Price high to low .4

Price low to high .5

Popularity .4

Relative Difference 20%

20% increase in revenue per visit (93% LOC)

Treatment 1 increased revenue per visit by 20%.

Results

Control Treatment 1

Daily

30%

Sunday only

34% Weekday

9%

Weekender

15%

Sat/Sun

12%

Daily

23%

Sunday only

37%

Weekday

8%

Weekender

17%

Sat/Sun

15%

Treatment 2

Daily

26%

Sunday only

37%

Weekday

12%

Weekender

18%

Sat/Sun

7%

Challenge

How can I optimize my category pages while still showing the many product options that the company provides?

Properties


Features1

Test background

46

46

Background: A vacation booking site specifically for Park City, Utah. Goal: To determine if redefining categories by feature rather than specific property would yield a higher CTR. Research Question: Which page would generate a higher CTR? Test Design: A/B split test

Record Location: VacationRoost Research Site: Park City Reservations

Research Notes:

Control: Properties

Treatment: Features

Top banner Security seal

Images Products at the bottom;

removing specific products

and prices

Category Landing Pages: Side by side

Winner: Features

32% lift in CTR

Challenge

Visitors are getting “stuck” at category/sub-category pages.

NoMessage WithMessage

Vote now! Text the keyword to 22-333 or go to marketingsherpa.com/LivePoll

Test background

53

53

Background: Rockport’s secondary navigation options are very extensive . Goal: To increase conversion by crafting a better secondary navigation experience and making it easier for shoppers to narrow down their options to find exactly what they are looking for. Research Question: Will messaging increase the effectiveness of the secondary navigation menu? Test Design/Optimization Engine: Multivariate/Adaptive

Campaign: Secondary Navigation Client: Rockport

Research Notes:

Experiment: Version A

No Messaging


With Messaging


No Messaging With Messaging

Experiment: Results

! What You Need to Understand: Secondary navigation is a critical aspect of the visitor consideration path.

Design RPV Lift

Version A (control) – No Messaging -

Version B – With Messaging 38.21%



Challenge

We have a high-quality video on our webpage, but nobody’s watching it. How can we increase clicks on the video?

Product1


Person1

Test background

60

60

Background: A single-product company that sells a liquid formula that repairs a part on cars. Goal: To increase video plays on the landing page. Research Question: Which video thumbnail will result in the highest video play rate? Test Design: A/B split test

Experiment ID: TP 1549 Record Location: MECLABS Research Library Research Partner: [Protected]

Research Notes:

Control: Person

Treatment: Product


Person Product

Experiment: Results

! What You Need to Understand: By more clearly communicating what value was available in the video (via text and imagery), we increased the number of people watching the video.


Control: Person is featured 6.74%

Treatment: Product is featured 8.06%


20% increase in video play rate The treatment page increased the video play rate by 20%.

Application

But YouTube only gives me a few options for thumbnails

Challenge

Does decreasing bounce rate on a homepage really translate to ROI?

ShortPage LongPage


Test background

68

68

Background: A VacactionRoost company that specializes in mountain/skiing getaways. Goal: To determine if simplifying the calls-to-action would decrease bounce rate. Research Question: Which page would generate a higher clickthrough rate? Test Design: A/B split test

Record Location: VacationRoost Research Site: Mountain Reservations

Research Notes:

Control: Long form

Emphasis on main CTA (searching)

Emphasis on value props (Destination

Expert advice)

Treatment: Short form Emphasis on main CTA

(searching)

Emphasis on value props (Destination

Expert advice)

Simplified to reduce anxiety

Homepage Redesign: Side by side

Winner: Short page

30% decrease in bounce rate $17,000 in additional sales annually

Challenge

How do I remove checkout funnel anxiety?

Common Navigation Area Content Minimized Navigation Area

Nav1 MiniNav


Test background

75

75

Background: Hugo Boss found that getting visitors to select a product is much easier than getting them to complete the full payment process. Goal: To reduce CAR by reducing the friction in the checkout funnel. Research Question: What is the best presentation of the main navigation area in the checkout process? Test Design/Optimization Engine: Multivariate/Adaptive

Campaign: Checkout Funnel Client: Hugo Boss

Research Notes:


Common Navigation Area Content


Minimized Main Navigation Area


Common Navigation Area Content Minimized Navigation Area

Experiment: Results

! What You Need to Understand: The checkout funnel requires a lot of user interaction – reducing anxiety is one of the easiest ways of reducing CAR.

Design RPV Lift

Version A (control) – Common Navigation Area Content -

Version B – Minimized Navigation Area 7.3%



Challenge

I have multiple customer types in my email list, but I don’t have the data to segment them. How do I speak to both in a single email?

1CTA 2CTA


Test background

82

82

Background: A financial company providing mortgage solutions to two different types of customers: 1.) Those refinancing 2.) Those purchasing a new home. Goal: Increase clickthrough from the email to the landing page. Research Question: Which presentation of CTAs will yield the highest clickthrough rate? Test Design: A/B split test

Experiment ID: TP 2084 Record Location: MECLABS Research Library Research Partner: [Protected]

Research Notes:

Control: Single CTA

Treatment: 2 CTAs

Experiment: Side by side Control Treatment

Experiment: Results

What You Need to Understand: Providing two equally weighted CTAs that a visitor could clearly understand which one was relevant to them was more effective than a single general CTA.


Control: Single CTA 1.12%

Treatment: 2 CTAs 1.27%


14% increase in email clickthrough The treatment page increased the email clickthrough rate by 14%.

!

Application

When are multiple CTAs competing?

Challenge

How much copy is too much copy? Is less always more on a landing page?

LightCopy


MoreCopy

Test background

90

90

Background: A vacation booking site for Mountain getaways. Goal: To determine if restructuring the destination landing page would yield a higher CTR. Research Question: Which page would generate a higher CTR? Test Design: A/B split test


Research Notes:

Control: Light copy

Treatment: Heavy copy

More content

Value prop in content

More CTAs, links

Destination Landing Pages: Side by side

26% lift in CTR

Winner: Heavy copy

Challenge

How can I create a more effective main site navigation?

Traditional


Mega1

Test background

97

97

Background: em-Cosmetics, a fast-growing startup with innovative products and social marketing initiatives aimed at serving a younger demographic. Goal: To enhance the e-commerce shopping experience. Research Question: What is the revenue impact of different dropdown menu styles? Test Design/Optimization Engine: Multivariate/Adaptive

Campaign: Navigation – Regular vs. Mega Dropdown Client: L’Oreal/em-Cosmetics

Research Notes:


Traditional Drop Down Menu


Mega Dropdown Menu


Version B Version A

Traditional Drop Down Menu Mega Dropdown Menu

Experiment: Results

! What You Need to Understand: Main navigation is the “low-hanging fruit” of every e-commerce optimization initiative.

Design RPV Lift

Version A (Control) – Traditional Dropdown Menu -

Version B – Mega Dropdown Menu 9.29%



Challenge

I want to test a complex process of my site, but I don’t have enough (development, time, financial) resources to do it right. What should I do?

Test background

103

103

Background: Large e-commerce company selling electronics that has a very in-depth checkout process with a complicated backend. Goal: Improve the conversion rate of the checkout process. Research Question: Which checkout process will yield the highest conversion rate? Test Design: User testing and A/B split testing

Record Location: MECLABS Research Library Research Partner: [Protected]

Research Notes:

Step 1: User testing

Treatment 1 Treatment 2 Treatment 3 Treatment 4



We chose the best-performing treatment from user testing


This treatment was the only one that was put into development



This treatment was the only one that was put into development

Treatment 2

9.9% increase in overall conversions

Challenge

How can I get higher qualified leads to Sales while optimizing the customer experience?

Information1 Request1


Test background

110

110

Background: A vacation booking site specializing in Hawaiian vacations. Goal: To determine if revealing more information on the front end would qualify more leads. Research Question: Which page would generate higher qualified leads for Sales? Test Design: A/B split test


Research Notes:

Control: Request more info

Treatment: Giving more information up front

Booking Page: Side by side

Control Treatment

Winner: Information

Control Treatment

30% - 40% decrease in leads 11% increase in booking conversion (bookings per visit to the website)

Challenge

I have too many leads coming in and not enough call reps, how do I solve this problem?

1Page 2Pages


Test background

117

117

Background: A global information company seeking quote requests and Sales-ready leads from a lead generation form. Goal: To find the best balance between quantity (request for quotes) and quality (Sales-ready) of leads. Research Question: Which form flow will increase the number of Sales-ready leads? Test Design: A/B split test

Experiment ID: TP1238 Record Location: MECLABS Research Library Research Partner: IHS

Research Notes:

Control: Single page

Treatment: Two pages

Compared

Experiment: Results

Design Lead Gen Rate

Control 60.84%

Treatment 44.36%

Relative Difference -27.1%

27% decrease in total leads (RQFs) The treatment decreased overall quantity of leads by 27%.

Experiment: Results

Design Lead Gen Rate

Control 6.38%

Treatment 15.09%


137% increase in Sales-ready leads The treatment increased the amount of high-quality leads by 137%.

Challenge

Other companies are reselling the same product I am, how do I differentiate myself?

Challenge

I’m running out of a certain product, but I don’t want to turn customers away. What should I do?

what do customers want? - meclabs...test background 10 10 background: kiehl’s wanted to present...

Documents