webanalytics congress the netherlands, amsterdam june 2nd 2010, judah phillips

The Practice and Evolution of Web Analytics: A Manager’s

Perspective *

By Judah Phillips Senior Director, Global Site Analytics

Monster Worldwide

*opinions are mine, not Monster’s

The Little Engine that Could…

  An American children’s story that teaches the value of optimism and hard work.

  After many larger engines reject helping move cargo over a mountain, a smaller engine succeeds by thinking “I know I can”

Proprietary and Confidential. Do not use without Permission from Judah Phillips.

Web Analytics is the Engine that Can…

  Amidst a sea of other, maybe larger, engines, that can’t including: > Ad Servers (internal and external)

> Customer Relationship Management Systems > Business Intelligence Systems > Financial Systems > Enterprise Data Warehouses > Silo’ed datamarts

> Spreadmarts > Third-party data providers

  YOU are the engine that can – and you have to believe it!


The Engine that Can Do Many Things PAST PRESENT FUTURE

INFORMATION What Happened?

(Data Mining and Reporting)

What is Happening Now?

(Alerts)

What will Happen?

(Trending, extrapolation)

INSIGHT How and Why did it Happen?

(Data Modeling and Experimental Design)

What’s the Next Best Action?

(Recommendation)

What’s the Best and Worst that can

happen?

(Prediction, Simulation)

ACTION How do we Leverage what we

Already Now?

(Dynamic Interaction/Profiling)

How do we Dynamically Modify

the Site in Real-time?

(Detection)

How can we Apply the Data to the

Future?

(Ongoing Optimization)

First two rows from Tom Davenport. “Analytics at Work.” Proprietary and Confidential. Do not use without Permission from Judah Phillips.

The Engine that Serves Many Functions Marketing Product Sales Executive IT

Landing Page Optimization

Behavioral Analysis

Customer Value Dashboarding Performance Monitoring

Life Time Value / RFM Models /

Customer Segmentation

Search Engine Optimization

Sales Readiness Scorecarding Site Usage for Capacity Planning

Search Engine Marketing

Demo/Geo/Firma Analysis

Sales Collateral Custom Research Disaster Recovery

Ad and Media Plan Optimization

Funnel and Flow Optimization

RFP’s and RFI’s Financial Performance

Infrastructure Enhancements

Social Media Optimization

Application and Product

Performance

Customer Usage Information

Competitive Intelligence

Tagging and QA


The Engine That Has Many Challenges…   Challenges:

> Never enough time! > No precedent. > Decision makers are

experienced. > Some things can’t be

measured.

> Errors and Mistakes. > Measuring too late or takes

too long. > Mistrust and dissatisfaction.

  Solutions: > Ask the right questions! > Verify assumptions. > Guide with the facts.

> Understand the data. > Suggest other ideas. > Prove your data is “right.” > Generously listen. > Set data expectations.

> Don’t over-commit


What Does it Take to Make the Engine Work?   Data that is easily available, self-service, integrated,

appropriately granular, standardized/defined, and of high-quality.

  Company that believes in data, not the just numbers, and is framed around want to use data for decisions.

  Managers and executives who can form leadership and governance around the data.

  Goals, goals, and more goals. They provide context for performance measurement.

  People, people, people

  Technology is the least important. All the tools do mostly the same thing – and all “suck” in different ways.


The Engine needs a Conductor   Leaders, and I’m not talking about the C*.   People-oriented and skilled.

  Leads by example.   Focuses on delivery and results and prove it.   Hires smart people, trust them, and give them credit.   Advocates for data and analysis.   Teaches people what they are talking about.

  Forms business relationships for leverage.   Doggedly persistent and tenacious.   Works cross-functionally.   Builds out an analytics “ecosystem.”   Knows when to say “NO.”


The Engine needs Stewards, Ticketmasters, Mechanics, and Railworkers   These are the analysts, engineers, developers, and QA

people: > Numeracy (i.e. quantitatively focused)

> Sufficiently Technical. > Business Focused. > Visually-oriented and pattern recognizers.

> Consultative. > Thoughtful. > Inquisitive.


How could you structure the Engine?

 Centralized  Decentralized  Consultative  Stovepiped by function  Hub and Spoke (i.e. Center of Excellence)  Federated

When is it the Right Time to Use the Engine?  When you have data!  Need information in the data.  Need consistency and control.  “Many cooks” who have input into the decisions.  When the company is very cross-functional.  When it is possible to improve the situation with

data.  Revenue is at risk.


When is it the Wrong Time to Use the Engine?  When you have no data or the data is bad.  Have not defined why your site exists.  Have no agreement on business goals.  Have ill-defined “business questions.”  Unanswerable questions that aren’t data focused.  Takes too long to get the data.  No support from teams on which you are dependent.  No ability to take action on the data.


What Can You Achieve?

 Improve revenue.  Reduce costs.  Maximize efficiency.  Enhance the customer experience.  Optimize the web site.


Web Analytics Evolving 1990’s (wild west) | | 2005-present (sheriff in town) 2000-5 (rough country)

Log file parsing on premise

Basic metrics: •  Visitors •  Visits •  Page Views •  Referrers •  Exit/Entry pages •  Hits

Managed by an IT department.

Business could use the data to understand popular pages and traffic patterns.

Reporting

JavaScript page tagging on-demand or on-premise

Detailed metrics: •  Time-derivatives •  Campaigns •  Keywords •  Conversion •  Pathing

Managed by a distributed business function with heavy IT support.

Business could use the data to ANALYZE site performance.

Hybrid forms of data collection: page tags, log files, db logging, 3rd party and internal data feeds.

Detailed metrics highly-specific to the overall business. Everything in earlier generations, plus custom metrics and event models (AJAX and Flash). Improved segmentation.

Managed by a centralized business function with little IT support beyond tagging.

Business could use the data to understand performance across various systems and channels whether online, offline, in-store, third party, or partner AND to optimize the site.

Data can be fed out of analytics systems and integrated with other tools, such as BI, Ad Servers, Surveys, Bid Management (SEM), SEO, Site Optimization, Predictive, Social, Mobile, Video, CRM, ERP.

Reporting > Analysis > Data


What does a Web Analytics Organization Look Like?  People are key. Followed by the technology.  At Monster we split the Site Analytics function:

1.  Architecture Team 2.  Reporting Team

3.  Analysis Team

 The end result is the creation, distribution, and operationalization of analysis, reporting, and data that guides the business on both tactical and strategic decisions.


Audience Measurement (AM) and Web Analytics (WA): The Yin and Yang

  Too much complaining about how these two systems don’t match or conflict with each other – waste of time – expecting them to match is unrealistic. Get over it!

  Determine how these systems can complement each other according to your business questions. Pick one tool as the “gold standard” for different purposes.

  Sometimes you use different data from both systems for the same audiences.


I wonder what the future holds…   New Data Collection Models

>  Universal tagging

>  Application logging

  Automated Tagging >  Flash events directly into Analytics without manual tagging

  Deeper Capabilities for Integration >  Roundtrips from EDW, Ad Servers, CRM, Targeting technologies, Finance

  Enhanced Targeting and Closed-loop, Self-optimizing, Customer Behavioral Feedback Systems >  Support more targeting attributes and direct feeds into technologies

>  Event and rules based detection modifying the user experience

  Improved attribution modeling >  First, last, indirect, direct, appropriate – can be confusing and insufficient.


Still wondering…   Global consensus-based, Practiced (not preached) industry standards

adhered to by vendors and enforced via demand by customers >  Global, not regional, country, association/organization, or company specific

for key measures/metrics. Let’s go beyond the Big 4!

>  Verticalized KPI’s for specific industries.

  Increased importance of auditing >  Requires deeper standards. Global companies don’t have the time,

resources, or often need to participate in all standards bodies.

  It’s about Multichannel >  It’s not just web – it’s mobile, social, and nonline sources (call

center, kiosks)

  Site Optimization based on Performance >  Not just conversion, but the macro and micro events on the site that drive

revenue.


 Further Cultural Change and Quicker Shifts in Organizational Dynamics: > To support data-driven decision making

• Not the HIPPO.

• Not based in Feelings. • Not based on Intuition. • Based on improved data analysis.

 To the point where companies are truly competing on (Web) Analytics!

All this will continue lead to…


So What? What does this mean to you? A Business   You need to define what you are solving for.

> Why are you doing analytics? > What are your business goals and drivers? > What are your site goals in the context of business goals? > Don’t get bogged down in the details. Think of the larger themes.

  You need to staff appropriately to compete on Web Analytics: > One part timer is not enough. You need staff.

  You need to invest in technology > Technology without people is useless. You need staff.

  You need to create cross-functional teams and processes > How do you define goals? Tag? QA? Distribute? Analyze? Optimize?


So What? What Does This Mean to a Consultant or Vendor?   Help get the data in order!   Figure out the tagging and QA problems.   Define key targets and segments.

  Build dashboards and scorecards.   Recommend how to take action on the data.   Build predictive models.

  Create organizational processes.   Define what optimization means and do it!


The Engine That Could Said…   Answer why your site exists.   Figure out the business goals.   Structure your team correctly for your goals.

  Be relentless and ruthless about data quality definitions.   Determine your key performance indicators and the drivers

for those indicators.   Report the data against goals and historic benchmarks to

give context.   Analyze the data in words and use visualizations.

  Define what it means to optimize the site.   Remember “You can do it!”


BEDANKT!

 Questions? [email protected]

[email protected] @judahp

Works Cited: Tom Davenport “Analytics at Work,” Eric T Peterson “Web Analytics Demystified.” Proprietary and Confidential. Do not use without Permission from Judah Phillips.

webanalytics congress the netherlands, amsterdam june 2nd 2010, judah phillips

Technology

data modeling

data mining

modifythe data

set data expectations

little engine

judah phillips senior

engine needs stewards

business goals