always be testing getting started with google website ...€¦ · 28.06.2009 · title: always be...
TRANSCRIPT
![Page 1: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/1.jpg)
Always Be Testing Getting Started with Google Website Optimizer
John Quarto-vonTivadar
Chief Thinking Officer
FutureNow, Inc.
New York City
![Page 2: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/2.jpg)
Latest Book:“Always Be Testing”
By Me.
available in bookstores everywhere
…and it’s an Amazon Best-Seller!
Shameless
Plug
![Page 3: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/3.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 4: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/4.jpg)
Why Test Online?
Is There Even a Problem?
You BET!
Many sorts of web site problems can be fixed with experience…
…and those that cannot, can be fixed with testing.
![Page 5: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/5.jpg)
No Scent = Lost Persuasive Momentum
Studying drop-off data indicates that would-be customers visiting a site lose the relevant “scent” of what put them on the trail to your site; without that scent they are unmotivated to go on.
Drop off after:1 page view 9.52%2 page views 54.60%3 page views 16.56%
Based on a sample of 2 million visitors(source: OnStat.com, March 2004)
1 Page
9.52%
2 Pages
54.60%
3 Pages
16.56%
4 or more
19.32%
Page views Drop-off Cumulativ
e
1 page view 9.52% 9.52%
2 page views 54.60% 64.12%
3 page views 16.56% 80.68%
Data source: OnStat.com, March
2004
![Page 6: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/6.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 7: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/7.jpg)
A Friendly
Purchase…
Common Late-Funnel Problem:
Flow From Page to Page
![Page 8: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/8.jpg)
…With an
Unfriendly
Shopping
Cart
![Page 9: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/9.jpg)
Common Late-Funnel Problem:
Perception of Time Needed
![Page 10: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/10.jpg)
Not That Long – It’s All About Perception
![Page 11: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/11.jpg)
Common Late-Funnel Problem:
Setting Expectations
![Page 12: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/12.jpg)
Common Late-Funnel Problem:
CRM instead of GTC
Was ist diese verdamnte “GTC” ??
GTC = “Get The Cash”
![Page 13: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/13.jpg)
Common Late-Funnel Problem:
The Never-ending Checkout
![Page 14: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/14.jpg)
Thank you, sir! May I have another?
…because, of course, we only sell to
honest people!
![Page 15: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/15.jpg)
Yes, 3 FULL Screens of Forms!
![Page 16: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/16.jpg)
So Is the Shopping Cart ReallyWhere the Problem Is?
![Page 17: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/17.jpg)
Or Was The Problem Already In Place Long Before Checkout?
![Page 18: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/18.jpg)
Common “Early Funnel” Problem Categories
• Relevance/Planning
• Structural
• Momentum
• Communications
• Value
![Page 19: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/19.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 20: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/20.jpg)
What to test?
![Page 21: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/21.jpg)
What To Work On...…if You Have Buy In
•Your Top 5 High Bounce Rate Pages
•Your Top 5 High Exit Rate Pages
•Your Top 5 Lowest Time Spent Pages
•Your Top 5 key pages (i.e., checkout, cart, registration, top product)
![Page 22: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/22.jpg)
What to Work On...If You Don’t Have Buy-In
•Pick a few PPC terms and landing pages
•Work on an email campaign
•Work where the BPUs aren’t
Qu’est-ce que c’est, le “BPU” ?
BPU = “Business Prevention Unit”
![Page 23: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/23.jpg)
Setting Up Your Test1. Create a Descriptive Name
1. “Call to Action Button Test: Improving Product Pages”
2. Identify if this is site wide test or for a campaign
2. Define Your Goal
1. “I want to increase conversion...”
2. How will you measure this success or failure? What are the Key Performance Indicators (KPI)?
3. How Will You Achieve your Goal?
1. What are the variables?
2. What are the variations?
4. Define the Control
1. What is your prediction/hypothesis?
2. What are you basing that on?
5. Let the Test Run
6. Measure & Analyze!
1. Did we improve or not?
2. What did we learn?
3. What do we do next?
![Page 24: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/24.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 25: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/25.jpg)
Online Tools for Testing
• Free or low-cost: Google Website Optimizer(google.com/websiteoptimizer), HiConversion.com
• Commercial: SiteSpect, Omniture Test & Target, etc.
![Page 26: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/26.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 27: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/27.jpg)
3 scripts key to GWO
• Control script – has visitor been here before? If so, show the same combo again
• Tracking script – if visitor hasn’t been here before, serve up a combination and increment combination count
• Conversion script – did the visitor get to the conversion page? If so, increment the successful conversion count, and mark test as complete for this visitor.
![Page 28: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/28.jpg)
3
• Classic A/B
•Classic Multivariate
•Split Path
•Multi-Page Multivariate
•Linger
•Do Anything
6 Tests You Can Try With GWO
![Page 29: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/29.jpg)
7
Classic A|B• Find winning single page among various similar pages
• Test conversion on a complete single-page basis
• Optimizer experiences not just pages
1. Add A/B control script to top of test page
2. Add tracking script to bottom page
3. Add conversion script to bottom of goal page
4. Specify variation pages
5. Click launch
![Page 30: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/30.jpg)
7
The Multivariate Section Test• Find winning Combination of varying sections
• Test conversion success of various combinations of variations
1. Add control script to top of test page
2. Add section script to each part of a page to be tested
3. Create variation of each section
4. Add tracking script to bottom page
5. Add conversion script to bottom of goal page
6. Click launch
![Page 31: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/31.jpg)
7
The Split Path• Find winning series of pages
• Test conversion stories that go beyond page sections
• Optimizer experiences not just pages
1. Add A/B control script to top of test page
2. Add section script to preliminary call to action pointing to original path
3. Create variation of preliminary call to action pointing to alternate path
4. Add tracking script to bottom page
5. Add conversion script to bottom of goal page
6. Click launch
![Page 32: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/32.jpg)
9
The Multi-Page Multivariate• Optimize a single path
• Discover cross-page interactions
1. Add control script to top of all pages
2. Add section script to each page’s sections
3. Add tracking script to bottom of first page
4. Add conversion script to bottom of goal page
5. Create a “fake” test page containing all sections to bypass validation
6. Click launch
![Page 33: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/33.jpg)
11
The Linger• Find content that people will actually read
• Only requires one page
1. Add control script to top of page
2. Add section scripts
3. Add tracking script to bottom of page
4. Add timer conversion script to bottom
of page
5. Validate a standalone goal page
6. Click launch
![Page 34: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/34.jpg)
12
The Do Anything
• Find content that engages people
• Isn’t limited to a “goal”
1. Add control script to top of page
2. Add section scripts
3. Add tracking script to bottom of page
4. Add conversion script to all other
pages or actions on the test page
5. Validate one of your goal pages
6. Click launch
![Page 35: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/35.jpg)
Let’s focus on the Multivariate Section Test (most useful)
• Example 1: single variable, 3 variations
![Page 36: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/36.jpg)
Testing page sample
Control
script
Tracking
script
Original
simple
page html
code
Section
name
identified
in page
![Page 37: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/37.jpg)
Conversion page sample
Conversion
script
Original
simple
page html
code
See this test live http://www.johnquarto.com/GWOtests
![Page 38: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/38.jpg)
Sample Results From GWO
![Page 39: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/39.jpg)
Useful free tool for online testing
• Firefox WebDeveloper toolbar (easy to kill off testing cookie, and see combinations
• http://chrispederick.com/work/web-developer
![Page 40: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/40.jpg)
How Long to Test?
Sample Sizemargin of Error
(%)
10 31.6%
20 22.4%
50 14.1%
100 10%
200 7.1%
500 4.5%
1,000 3.2%
2,000 2.2%
5,000 1.4%
10,000 1.0%
Unfortunately, no stone-cold
answer. But start by asking how
accurate you want to be.
It can take a lot to be precise but
it doesn’t take much to be
actionable.
The trade-off between accuracy
and actionable improvement is
your real question to answer.
![Page 41: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/41.jpg)
Trade off Example:• A Landing page micro-conversion rate is currently 42%. You send 20
random customers to a new Landing Page and it scores 51%. What do you do?
• Answer: a sample size of 20 means the margin of error is about 22%, so the “real” number for the test is somewhere between 29%-73% (51+/-22). Hmm, that’s not really enough to know for sure, compared to the original of 42%.
• Let the sample size grow to 200, and MoE is now 7%. Assume you still see the same conversion rate on the test (51%) so the real number is somewhere between 44% and 58% (51+/- 7). Compared to the original of 42% you can make a darn good argument for switching.
• don’t be too agressive. Switch but keep the test running. “PERFECT IS THE ENEMY OF GOOD”
![Page 42: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/42.jpg)
Outline
• Why Test Online – What’s wrong?
• Types of Online Problems to be tested
• Deciding What to test
• Online Tools for Testing
• Types of GWO Online tests
• Is It Worth It?
![Page 43: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/43.jpg)
Doubt Testing is Worth It?
Overstock.com
before
![Page 44: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/44.jpg)
$25M/year ($68K Daily) Gain
33% reduction in abandonment
Overstock.com after
“I asked FutureNow to do a study of our web site...found something we had to fix immediately…and saw an immediate 5% lift in our sales.”
- Patrick Byrne, CEO of Overstock.com
![Page 45: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/45.jpg)
Thank You! I Wish I Were in Paris with You!!
John Quarto-vonTivadarChief Thinking Officer, FutureNow
site: www.FutureNowInc.comblog: www.GrokDotCom.com
Latest Book:“Always Be Testing”
available in bookstores everywhere
…and it’s an Amazon Best-Seller!
![Page 46: Always Be Testing Getting Started with Google Website ...€¦ · 28.06.2009 · Title: Always Be Testing Getting Started with Google Website Optimizer Author: John Quarto Created](https://reader036.vdocument.in/reader036/viewer/2022090605/605ac59a453551421d72db4a/html5/thumbnails/46.jpg)
END