customer analytics for online marketing

1

1All Rights Reserved DSA/DMA 2008 1

Customer Analytics for Online Marketing

Presented by David Shepard Associates

Debbie MegeeSenior Consultant, David Shepard Associates• Over 25 years of experience• Developed a 7 terabyte database system which contains well over 100 million customers, supporting name selection processes for direct marketing insurance and membership products

Rus Rempala Senior Consultant, David Shepard Associates• Over 20 years experience• Significant management positions in customer

l d l d b

2

analysis & modeling, customer database management and circulation management

All Rights Reserved DSA/DMA 2008

2

Agenda

• Challenges facing online marketers

• How to select appropriate metrics• How to select appropriate metrics

• Analytical Support For Website Design

• Website / Web Page design testing

3

• Multi‐Channel Issues

All Rights Reserved DSA/DMA 2008 3

Challenges Facing Online Marketers• Highly fragmented channels

• Great opportunities and great confusion

• Technology vs. marketing goals and strategies

• Our jobs are to solve business problems

• Lead with strategy, define dimensions, and measure the right things

• How does the economy effect online marketing?

4

• How does the economy effect online marketing?

– ROI

– Customers needs


3

Finding a Path Through the Chaos• Questions abound:

– How do you get attention without being disruptive?disruptive?

– How do you engage customers?

– How do you take advantage of online tools and tactics?

– How do you measure the right things?

5

– What new measurements are needed for Web 2.0?


Choosing the Right Metrics

6All Rights Reserved DSA/DMA 2008

4

Decisions About Metrics

• Key Performance Indicators

Page views Visits Clicks Uniques– Page views, Visits, Clicks, Uniques, Repeats, Sources, Abandons, Bounce, Shopping Cart, etc.

– Don’t measure just because it’s there (just for measurement’s sake)

7

• Dashboards

– Benchmark performance


Key Performance Indicators

• Clear goals and relevant measures

• Encourage strategic thinking

• Align measurement with goals

• Put metrics in context

• Continuously refine metrics

• Make analysis a habit

8

• Make analysis a habit


5

Developing Dashboards• Reduce metric overload

• Provide appropriate data designed to the audienceaudience

• Provide consistency

• Measure effectiveness

9

• Measure effectiveness

• Benchmark performance

• Measure success of strategies

• Drive decisionsAll Rights Reserved DSA/DMA 2008

Web 2.0 Challenges• Sophistication and long term relationships

• Customer control

• Need for more analytical approach

• Interactions over time

• New technologies such as blogs, wikis, RSS

10

• Understand which customers use these channels, how they perform for you

• New customer segmentsAll Rights Reserved DSA/DMA 2008

6

Web 2.0 Analytics• Web 2.0 means interacting with customers over

time

• Tracking customers’ behaviors g

• Complete history of interactions at the unique customer level

• Attributing the conversion to the right activity

• Measuring the impact of influencing the customer i h l i l k i i i i

11

with multiple marketing activities

• Making better decisions about marketing investments

• Tracking the right metricsAll Rights Reserved DSA/DMA 2008

Web 2.0 Evolving Analytics

• Technology will continue to evolve

• New analytics must enable better tracking at y gan individual level

• Online and digital personalization

• Behavioral analytics solution

12

• Simple statistics about click throughs and page hits are not sufficient

• Customer centric data is essentialAll Rights Reserved DSA/DMA 2008

7

Online Metrics Examples• Blogs: Conversion, Frequency,

Retention

• Comparison Shopping: Source , Conversion

• Forums: Conversion New

•

• Forums: Conversion, New customers, Segmentation, Purchases

• Podcasts: Activity, Conversion, Retention

• RSS: Activity, New subscribers, Test content, Conversion

• Social Networks: New• Social Networks: New customers, Conversion

• User Reviews: Purchase patterns, New customers, Repeat behavior, Retention

• Wikis: New customers, Segmentation, Conversion


Choosing the Right Metrics


8

Capturing Data for Online Marketing

• Channel specific tools and vendors to capture data

• Increasingly more difficult to manage marketing strategy and analyze results

• A centralized database is needed

• This database must capture and integrate data

15

• This database must capture and integrate data at the customer level

• Data to track activity and responses must be managed across all online channels


Building the Solution

• Must account for multiple marketing initiatives

• No easy solution

• Must bring data together, standardize terms, resolve inconsistencies

16

• Bring together Technology, Analytics, Strategic Planning


9

Modern Testing MethodsApplied To Web Page Design


Establish A Culture of Testing

• The more you do– The better your results

Th b i– The better you get at testing

– The more you’ll accomplish each time

• And you’ll NEVER be done

18 18All Rights Reserved DSA/DMA 2008

10

Landing Page LinksSearch Phrase: “Winter Cruises”


Registration Forms


11

Features & Prices

21 21

Web Testing Gets Complicated

• Often have many features to test

• Test one at a time with A/B tests

OffersLink IdentificationPage LayoutProduct ImagesColorsTest one at a time with A/ tests

• Based on:– Actual customer actions– Statistical sampling methods

ColorsPriceFeature BundlesImagesRegistrationHeadline TextPayment StepsValue PropositionBanner Ads

22 22

Landing PagesSubmit ButtonsCross Sell OffersText SizeProduct DescriptionFAQ’sNavigation


12

Issues With Typical A/B Tests

• Choosing initial feature to test– Management reviews– Political, personality pressure– What’s most expedient

• Perceived problems with testing– Time consuming– Very complicated

23

– Expensive


Insurance Website Example

• Current Offer (Control)– Price: High– Commitment: Short Period– Bonus Feature: One

• 4% Signup Rate

• Good percentage renew


13

What’s To Improve?

• Even though renewal rate is good, marketing to pursue renewals is costly

• Test for ways to increase commitment period– Reduce marketing costs– Raise retention rates

• Concern that longer commitment will reduce purchases


Feature Combination Map


14

Typical A/B Testing Approach

• Keep it simple… test only one thing at a time

• Time honored testing technique• Time‐honored testing technique

• Chose to test commitment period– Lower risk– Likely to make a real difference


Typical A/B Testing Approach

• Control (“A”, “Champion”)– High price– Short commitment– 1 bonus feature

• Test (“B”, or “Challenger”)– High price– Longer commitment– 1 bonus feature


15

A/B Test Sample Size

29 29

Sample Size = 32777 for both samples!


A/B Tests

Control

• Changing one thing at a time – Commitment Period

LongerCommitment

30

• Control Group obviously better


16

A/B Tests

LowerPrice

• Changing one thing at a time– Price Point

31

• Combining both tests: Lower price & Shorter Commitment is better?


A/B Tests

• Changing one thing at a time– The untested combination

32

• Interaction: Longer commitment + Lower price

‐‐ Yields ‐‐

Increased response rate + Lower remarketing cost


17

Multi‐Variable Test Designs

• Change all features at once, but in a systematic manner

• Test more combinations, find more interactions

• Difficult to define interactions testing 1 feature at a time

• Testing efficiency grows with the number of

33

g y gtested features (instead of declining as with A/B Tests)


Types of Multi‐Variable Designs

– Full Factorial• Too many combinations, Example: testing 7 factors, with just 2 levels,

you would need 128 combinations!

– Pre‐Planned designsPre Planned designs• Taguchi, Plackett‐Burman, …• Reduce the number of combinations you need• But, interactions are a problem (assume negligible)• And, constraints are a problem (can’t include/exclude)

– Fractional Factorial• Carefully chosen, interactions are not a problem• Not as many test cells as full factorial• But usually more test cells than you can handle• And constraints are still a problem

34

– Optimal Design• Can choose to sacrifice orthogonality for greater reality, and feasibility• Need specialized software• Need specialized knowledge• When done correctly, ALWAYS BEST• Great for follow‐up


18

Multi‐Variable TestingMore Efficient Than A/B Testing

• A/B testing requirements

• 3 price points 3 policy terms• 3 price points, 3 policy terms– 9 combinations to test

– 9 A/B testing cycles @ 66,000 visitors

– 594,000 visitors to read all combinations

• Time required for 9 testing cycles



1. Determine A/B test sample size (33,000) for each feature

36

2. Divide sample size by # feature options (33,000 / 3 price points)

• Each column and row adds up to 33,000

3. Yields 11,000 required per test cell


19


• Each feature level occurs equally (4 times)

37


• Scenario #8 represents the control offer


Slide: Multi‐Variable TestingMore Efficient Than A/B Testing


38

• Scenario #8 represents the control offer


20



Optimal Testing:Identify Component Parts for Testing

Images Used

Layout; including

Right/Left

Copy and Order

Of S ll P i t

Form

Headline

Copy

Form Layout

Main headline

40

Of Sell Points

Endorsements

Submit Buttons

Include Phone

Number?


21

Potentially Millions of Possible Permutations


Individual Results

35% 29%

29%

30%

30%

34%

38%30%

29%

42

NOTE: for illustrative purposes only - not real data!!!!

31%34%


22

Optimal Testing:Recombine Component Parts for Predicting

Individual Effects

PredictedWinner

4% 15%

+9%

+5%

-4%

-4% +15%

-3%

43

NOTE: for illustrative purposes only - not real data!!!!43

+29%-3%

3%

PredictiveModel(s)


Questions?

Anyone?

Ferris Bueller… Questions?

Anyone?

Anyone?

44

Anyone?


23

Contact Information• Rus Rempala

[email protected](303) 554‐7675

• Debbie [email protected](972) 867‐6534

4545

• Visit the DSA Website:www.dsadirect.com


46All Rights Reserved DSA 2008 46

Best Practices in Customer AnalyticsPresented by David Shepard Associates

customer analytics for online marketing

Documents