adobe digital analytics - sitecatalyst, test & target workshop

52
0 SiteCatalyst Introduction and Reporting Overview

Upload: digital-vidya

Post on 15-Jul-2015

1.597 views

Category:

Education


6 download

TRANSCRIPT

0

SiteCatalyst

Introduction and

Reporting Overview

1

SiteCatalyst User Training Topics

• How SiteCatalyst Works

• Major SiteCatalyst Metrics

• SiteCatalyst Variables (props, eVars, events)

• SiteCatalyst UI Reporting Overview

• Creating Segments

• Other Reporting Methods

Topics to be Covered

2

How SiteCatalyst Works

JavaScript code is included on the page that:

1. Calls the SiteCatalyst “control files” (s_code.js and wa_config.js)

2. Sets page-level variables

3. Executes the code, sending an image request to the Adobe

servers

• The URL of the image

request contains all of the

SiteCatalyst variables and

data either set on the

page or within the s_code

file

• The variables are set in

name/value pairs and are

contained in the query

string of the URL

3

How SiteCatalyst Works

There are a number of tools available to

allow users to see what variables and

values are tracked by SiteCatalyst on a

page or link click. The easiest to use is

the DigitalPulse Debugger. Details on

setting up the Debugger are at the link

below, or go to the Help section on

SiteCatalyst and search for “debugger”.

https://microsite.omniture.com/t2/hel

p/en_US/home/index.html#kb-using-

digitalpulse-debugger

4

How SiteCatalyst Works

What does SiteCatalyst measure?

• SiteCatalyst can measure and report on practically anything that

happens in a visitor’s browser. Some of this is done via the “base

code” and some via custom coding.

• The base code tracks aspects of the visitor such as IP address

(which can be translated into geographic information), technology

(operating system, browser), the domain of the visitor’s ISP, and

whether the visitor has previously been to the site (via cookies).

• The base code also tracks the load of the current page, capturing

information such as the date/time, and the last page from which the

browser came.

• Through custom coding, almost any aspect of a visitor’s activities

and attributes can be tracked. Actions on the page such as link

clicks can be tracked.

5

How SiteCatalyst Works

What does SiteCatalyst not measure?

• SiteCatalyst, and any web analytics platform, is unable to track

certain actions taken in the browser window. These include use of

the browser’s back and forward buttons and right-clicking an

element on the page.

• By contractual obligation and business regulation, SiteCatalyst

cannot track PII.

• Because SiteCatalyst tracking is done in a visitor’s browser via

JavaScript and cookies, less than 100% of the actual data is sent

to the Adobe servers. Adobe states that SiteCatalyst data is

generally “accurate” to within 3%-5% of actual traffic.

6

How SiteCatalyst Works

What should/shouldn’t SiteCatalyst measure?

• It is best practice to only track those items that are necessary to

generate the actionable reporting needed to help optimize your

site.

• SiteCatalyst is meant to track and report on data in the aggregate,

not at the individual visitor or user level. While SiteCatalyst is

technically capable of producing reporting of actions at the

individual level, it is not best practice to do so – such data is best

produced by application-level backend systems.

• It is not good practice to use SiteCatalyst reporting as “system of

record” reporting for any metrics other than site traffic. Metrics such

as the number of logins, self-service transactions (e.g., transfers),

the dollar amounts of transactions, etc., can be tracked by and

reported on by SiteCatalyst, but should be used for trending and

general sizing only. These metrics should not be used for

corporate or regulatory reporting.

7

Key Metrics

Metrics & Custom Variables

Metrics and Variables

Major Metrics

8

Key Metrics

• Page Views - A page view occurs when a visitor views a page on your web site.

• Visits - Occurs when a visitor views at least one page. The visit begins when a person first

views a page and ends after 30 minutes of inactivity.

• Unique Visitors – The number of unduplicated users that visit the website over the course

of a specified time period. The Visitors metric is defined primarily by cookies, and is best

understood as the number of “devices” (rather than “people”) that visit the site.

Metrics & Custom Variables

Metrics and Variables

Major Metrics

9

Add or Change Metrics

Reporting Overview

9

SiteCatalyst Reporting UI

Page Report Metrics

Key Metrics Definition

Unique Visitors Visitor who visits the site at least once for the selected date range.

Visits

A visit occurs when a visitor views at least one page. A visit may

consist of multiple page views. The visit persists until 30 minutes of

inactivity or 12 hours of continuous activity.

Page Views A page view occurs when a visitor views a page on your web site.

Average Page Depth The number of pages into the visit that the page is seen on average.

Average Time Spent on

Page The amount of time (in minutes) that a visitor spends on a page.

Entries The number of times a page is captured as the first page of a visit.

Exits The number of times a page is captured as the last page of a visit.

Reloads Counted when the same page is loaded twice; i.e., page is refreshed.

Bounces

The number of times a page or value is captured as the only image

request during a visit; i.e. page A is the only page visited, and then

the visitor left your site.

Bounce Rate Bounces/Entries

10

Key Metrics

Metrics & Custom Variables

Metrics and Variables

Variables

11

Key Metrics

Metrics & Custom Variables

Metrics and Variables

Major Variables

• props - used to track page-by-page site traffic activity

• eVars - used to report on subsequent conversion events, in addition to traffic activity

• events – track conversions, actions that we want to our visitors to take on our site

• Products – track the products being viewed, sold, added to cart, or removed from cart

12

• Props (Custom Traffic variables) are used to track page-by-page site traffic activity (page

views, visits, and visitors).

• Props do not persist between pages.

• Key applications of traffic variables:

• Anytime that you wish to capture a value which can be associated with Page Views,

Visits and Visitors

• Find ‘most popular’ of a specific value

• Measure traffic for a specific segment

• Examples of props on websites: Line of Business, New vs. Repeat Visitor, Page

Errors/Messages (example report below)

Metrics & Custom Variables

12

Metrics and Variables

Props (Custom Traffic)

13

• eVars (Custom Conversion Variables) are used to report on subsequent conversion events, in addition to traffic activity.

• eVars are persistent throughout the visit (or for other periods, depending on the variable’s expiration settings).

• eVars can be used to report on event variables that are set subsequent to the eVar being set; traffic metrics (visits, page views) can also be reported for eVars

• Examples of eVars on Ally sites: Search Term, Internal Promotions (example report below)

Metrics & Custom Variables

13

Metrics and Variables

eVars (Custom Conversion)

14

• events (Custom Conversion Events) track conversions - actions that we want to our

visitors to take on our site.

• events can be reported on individually (via a trend report) or added to eVar reporting

• events can be “serialized” so that they are reported only once per visit, even if the user

reloads the page the variable was set on

• Examples of events: Site Search, Application Starts, Application Submitted (example

trended report below)

Metrics & Custom Variables

14

Metrics and Variables

events (Custom Conversion Events)

15

Key Metrics

Metrics & Custom Variables

SiteCatalyst Reporting UI

Pages Report

16

Reporting Interface Overview

5. Use the Text Filter to search for your pages

2. Search for any Report or Metric, or use the Report Menu to locate a report

7. Set chosen metrics as default

for the report

1. Choose a Report Suite

4. Choose how you would like to view the data - ranked table or top 5 values

trended over time

3. Choose a Date Range by clicking on the

calendar

6. Choose metrics to display by

selecting “Add Metrics”

Reporting Overview

16

SiteCatalyst Reporting UI

17

SiteCatalyst Reporting UI

Report Suites

• SiteCatalyst data is housed in separate “buckets,” called Report Suites.

• Each site has its own Report Suite, e.g., Corporate, Bank, Auto, Dealer

• Some portions of some sites are also segmented into separate Report Suites, e.g., ATM

Locator

• The Global Report Suite contains data from several sites, listed below. This Report Suite

can be used to report on visitor movement between our sites.

• Global Report Suite sites:

• Corporate

• Bank

• Auto

• Mobile Web

18

SiteCatalyst Reporting UI

Report Locations Site Metrics Site Content Mobile Devices

• Site Overview (opening

screen)

• Key metrics (Visits,

Page Views, Visitors)

• Visits

• Visitors (Daily, Weekly,

etc.)

• Time Spent per Visit

• Events

• Pages

• Site Sections (channel)

• Links (Downloads, Exit

Links, Custom Links)

• Various reports about

technology and other

aspects of mobile

visitors

19

SiteCatalyst Reporting UI

Report Locations

Path Reports Traffic Sources Visitor Profile

• Next Page/Flow

• Previous Page/Flow

• Fallout

• Full Paths

• Entries/Exits

• Search Keywords (all,

paid, natural)

• Search Engines (all,

paid, natural)

• Referring Domains

• Referrers

• Referrer Types

• Search Keywords (all,

paid, natural)

• Search Engines (all,

paid, natural)

• Referring Domains

• Referrers

• Referrer Types

20

Add or Change Metrics AllyGlobal (allyglobal) 1. Choose metrics by clicking the

“Selected Metrics” link at the top of the report window, or select “Add Metrics” above the report results

2. Choose metrics by double clicking, or drag & drop them to the canvas

3. Click Ok

4. Participation and Calculated metrics can be added from the “Metric Type” drop-down

1

1

2

4

3

Reporting Overview

20

SiteCatalyst Reporting UI

Selecting Metrics

21

Creating Calculated Metrics

1. Choose Calculated from the “Metric

Type” drop-down, then click

2. Click “Define New Metric”

3. Name the metric

4. Select the Type (e.g. Percent)

5. Select the base metric. Use Standard

Metrics, not “Total” Metrics

6. Add the metric to the formula (double-

click or select “Add to Formula”

7. When the formula is complete, save

the new metric

Note: Users can create and edit their own

calculated metrics, but they are not shared

across all users

1

2

7

6

5

4 3

Reporting Overview

21

SiteCatalyst Reporting UI

Calculated Metrics

22

Select Date Range

1. Choose a date range by clicking

on the calendar

2. Select a custom date range by

dragging the mouse over a range

of dates, or type in the dates in

the cells.

3. Select a preset date using the

drop-down list

4. Compare data for two different

date ranges by selecting

“compare dates” (e.g. January vs

February)

1

2

4

3

AllyGlobal (allyglobal)

Reporting Overview

22

SiteCatalyst Reporting UI

Date Range

23

• Reports can be toggled between

Ranked and Trended views

• “Ranked” report type shows a bar graph

of top values, and a table view of all

values ranked in descending order

• To sort by a different metric, simply click

the metric name you want to sort by

• “Trended” report type shows a trend

graph of the top values (up to 5), and a

table view of top values trended over

the selected time period

• You can change the 5 values that are

trended by clicking “Selected Items”

• You can toggle the granularity of the trend

by clicking “View By”

• You can only trend by one selected metric.

The metric can be changed by clicking

“Selected Metrics”.

RANKED

TRENDED

Reporting Overview

23

SiteCatalyst Reporting UI

Ranked/Trended Reports

24

• eVars and props can be “broken down” by other metrics, including other eVars,

products, Campaigns, traffic sources, etc.

• props can be broken down by other props on a limited basis – correlations must

be set by an admin, and are limited in number

Reporting Overview

24

SiteCatalyst Reporting UI

Breakdowns – Relations between Variables

25

Downloading & Sending Reports • Reports can be downloaded to your

computer in a variety of formats

• Reports can also be scheduled to be

sent on a recurring basis with detailed

customization options

1. Click the “Send” Button in the report

toolbar

2. Enter the email addresses where the

report should be sent. Publishing Lists

can also be created.

3. Schedule the report to be sent

immediately, or weekly, monthly, etc.

4. Click “Advanced Delivery Options” to

set more advanced report scheduling

options and recipients

5. A “Rolling” date will automatically

advance the date of your report, which

is useful for recurring

daily/weekly/monthly reports

6. Once all settings are chosen, hit

“Schedule”

1

2

4

5

3

6

Saving & Sending Reports

25

SiteCatalyst Reporting UI

Downloading/Sending Reports

26

Bookmarks

Bookmarks a way to save custom reports so you can refer back to the them

later. They differ from Custom Reports in that they can be scheduled and

shared with other users.

To create & manage bookmarks:

1. Click “Bookmark” in the report toolbar

2. Enter a user friendly name for the bookmark and add to a new or existing folder.

3. Make the bookmark available for users by clicking on “Make Public”. “Display

report upon login” will show the report when you log in.

4. Click Save

5. Saved bookmarks can be deleted, shared, scheduled, etc. through the Bookmark

Manager

4

3

2

1

5

Saving & Sending Reports

26

SiteCatalyst Reporting UI

Bookmarking Reports

27

Dashboard Reportlets

• Dashboards are comprised of multiple reportlets and can be customized based on

your reporting needs

• Your custom dashboards can be set as your default landing page, and can be can

be saved and shared

4

3

2

1

Saving & Sending Reports

27

SiteCatalyst Reporting UI

Dashboards

To create reportlets:

1. Click “Dashboard” in the report

toolbar to create a new reportlet

2. Enter a user friendly name for the

reportlet and add to a new or existing

dashboard.

3. Choose the date range of the

reportlet

4. Click “Create New”

28

Dashboards • To add a reportlet to a dashboard:

1. Click “Add Dashboard” under SiteCatalyst

> Favorites > Dashboards

2. Type a name for the dashboard

3. Click 3 x 2 or 2 x 2 to specify how many

reportlets you want on the dashboard

page

4. Configure the dashboard page layout: o Adds a blank page to the dashboard

o Specify a paper size (e.g. landscape, portrait,

A4)

o Search for content in the Add Content and

Dashboard Contents menus

5. Add saved reportlets to the dashboard by

dragging items to the reportlet canvas

6. Click Save

7. Dashboards can be edited at any time by

clicking “Layout” in the dashboard view

8. Dashboards can also be managed

(deleted, shared, scheduled, etc.) through

the Dashboard Manager

9. To set the dashboard as your landing

page, click More Actions > Set as Landing

Page.

1

8

1

4

2 6

3

5

7 9

Saving & Sending Reports

28

SiteCatalyst Reporting UI

Dashboards

29 29

• Path flow and list reports allow you see

what pages visitors viewed before

(previous) or after (next) a selected page.

The report will highlight when visitors enter

or exit your site.

• Results can be shown graphically as flow

reports or in a list view. The report metric

is path views.

• You can use this report to:

• Understand what steps are taken most

frequently after viewing a selected page

• Identify where visitors are going instead

of your desired goal pages

22.91% exited the site after viewing the

Corp Homepage

12.68% went to the Customer login

username page, then the Customer login

password page

• To access Flow reporting, go to Paths > Pages > Next (or Previous) Page Flow in the Reporting Menu

• To access a List view, go to Paths > Pages > Next (or Previous) Page in the Reporting Menu

Advanced Reporting

SiteCatalyst Reporting UI

Pathing Reports

30

Pathing Reports - Fallout • Fallout reporting shows the passthrough between

up to 7 checkpoints you define.

• The fallout report will count all visits that passed

through the checkpoints in the order they are

defined, regardless of whether there were other

pages visited in between checkpoints.

• The report shows visits, % of starting visits,

conversion % (continued) and fallout % (lost)

• You can use this report to examine:

• Conversion rates through specific processes (such as account application or enrollment process)

• General, wider scope traffic flows: Of the people who saw the homepage, how many went on to a

product page and then how many of them eventually went on to start an application?

• To access Fallout reporting, go to Paths > Pages > Fallout in the Reporting Menu

Advanced Reporting

SiteCatalyst Reporting UI

Fallout Report

31

Key Metrics

Metrics & Custom Variables

SiteCatalyst Reporting UI

Create a Segment

32

Segment Definition Builder aids in building segments within Data Warehouse, Discover, and

segmentation within SiteCatalyst15.

1. Title – This is where the segment is

named

2. Containers – Serves as the

framework for how data will be

populated for the segment.

3. Events – Contains a list of all

success metrics that have been

defined for the site. Also includes

“instances of eVar<X>”.

4. Segment Canvas – contains Include

& Exclude tabs. This is where the

segment is built.

The 4 Components of the Segment Definition Builder

1

2

3

4

Segmentation

SiteCatalyst Reporting UI

Segmentation

33

Segment Containers

• The Page View container will only retrieve data where the

condition is true on that page – i.e., the variables/values set on

the page load.

• Example: page views where prop31 (Errors/Messages) contains

certain text.

Segmentation

SiteCatalyst Reporting UI

Segmentation

• The Visit container will only retrieve data where the condition is

true for that session.

• Example: visits where eVar36 (User Type) is contains both

“Anonymous” and “Bank Customer”

• Will be most often used segment type.

• The Visitor container will only retrieve data where the condition

is true for each visitor within the selected timeframe.

• Example: visitors where the number of logins (event1) is greater

than 1

34

Creating a Segment

1 1. Click “Add Segment” under the

Segment toolbar

2. Give the segment a name

3. Select your container – page

view, visit, or visitor. You can

also select an event, which will

act as a page view container.

4. Drag the container to the

segment canvas

AllyGlobal (allyglobal)

2

3 4

Segmentation

SiteCatalyst Reporting UI

Segmentation

35

Applying Segment Filters • To set the rules or conditions by which the segment will operate, click the underlined

name of the segment.

• The “Define Visit” window will open, and

then select the following:

1. Choose whether the visit will match all rules

or at any of the rules

2. Select the item from which the criteria will be

set

3. Choose whether the item should be “equals”,

“greater than”, etc. to the value

4. Select the value of the criteria – either enter

the value or select from a list

5. Don’t forget to click the “Add” button add the

new rule to the visit filter

6. Continue this process until all rules have

been created, and then click “OK” when

finished

7. Click “Save” in the Segment Definition Builder

window

• Check reports to verify that the segment is

generating the expected data

1

2 3 4 5

6

Segmentation

SiteCatalyst Reporting UI

Segmentation

7

36

Segment Exclude Canvas

• The Exclude canvas provides the ability to build a

segment based on criteria that should be

excluded from the data.

• In this example, the client wanted to view activity

for all pages other than the apply:step pages.

• This segment was built using the Exclude canvas

to exclude all page views that contained

apply:step, returning data for all other page views

that occurred on the site.

Segmentation

SiteCatalyst Reporting UI

Segmentation

37

Events in Segmentation • Success events can be added to the canvas to help define the segment, as well as

instances of a particular eVar.

• The example below shows a segment for all visits where an Application page was

viewed and an application was submitted.

• Note that the success event App Submitted can be clicked, and additional rules can

be applied

Segmentation

SiteCatalyst Reporting UI

Segmentation

38

And/OR Container Logic • The segment canvas can also be used to created AND/OR Logic.

AND Logic (nested containers) OR Logic (adjacent containers)

• This segment will return all data for visits where an apply:step page was viewed at some point during the visit AND an application was submitted within the same visit.

• This segment will return all data for

visits where an apply:step page was

viewed. It will also return all data for

visits where an application was

submitted.

Segmentation

SiteCatalyst Reporting UI

Segmentation

39

And/OR Container Logic • DataWarehouse

Requested reports that display advanced data relationships from raw data based on your

unique questions. Unlike most reports, which are displayed in the SiteCatalyst interface,

DataWarehouse reports are emailed or sent via FTP, and may take up to 72 hours to

process. Access to DataWarehouse is limited.

• Pro: get data for any combination of metrics and/or variables

• Con: need to wait for report; report is raw data; each change in dimensions, date range,

etc., requires another report request

• ReportBuilder

Pull data directly into Excel using the ReportBuilder add-in. ReportBuilder enables

customized data requests from SiteCatalyst, which can be inserted into Excel worksheets

and dynamically reference specific cells. Using ReportBuilder's data filters, extract and

combine data points into highly customized reports. Once an advanced dashboard has been

created, it can be manually refreshed or automatically published with the most up-to-date

information.

• Pro: data ported directly into Excel – can manipulate and combine with other data

• Con: limited in types of reporting (e.g., limited in breaking down variables by others); must

create segments in UI; changing metrics, date ranges, etc. not as simple as in UI; limited

number of licenses to use the add-in

Segmentation

SiteCatalyst Reporting

Other Types of Reporting

40

And/OR Container Logic

Segmentation

Discover: Ad Hoc

Analysis

41

Discover: Raw Power of Java

42

Interactive Reports

43

And/OR Container Logic

Segmentation

Test & Target: Targeting the

Right Way

44

Key Metrics

• Audience - An audience is a group of people who meet a rule condition, like "users on

Firefox" or "new visitors." You can display an activity to a particular audience. In your

reports, you can show how an activity performed for a particular audience. Several

audiences are pre-configured for you, and you can create others that meet your specific

requirements. Activities can be created with a single audience.

• A/B Testing- It is randomized experiment with two variants, A and B, which are the control

and treatment in the controlled experiment. Two versions of the same element (A and B)

are compared, like an image. Version A might be the currently used version (control), while

Version B is modified in some respect (treatment)

• Multivariate Testing – It is a process by which more than one component of a website

may be tested in a live environment. It can be thought of in simple terms as numerous A/B

tests performed on one page at the same time. A/B tests are usually performed to

determine the better of two content variations; multivariate testing can theoretically test the

effectiveness of limitless combinations.

Metrics & Custom Variables

Target

Terminology

45

Key Metrics

• A/B testing involves comparing a control environment with a the test environment and

coming up with the insights about the performance of your marketing strategies

• Earlier web marketers used to rely on the process of re-directing traffic, 50% page A, 50%

page B, to verify the effect of experience on the visitor segments

• It required a lot of development effort to redirect visitors on the basis of different pre-

conditions like source, url parameter, useragent, IP address, etc

• All the experience were hosted on the same web server, increasing load time of the

webpage

• There used to be no default view if there was any failure in loading of the desired

experience

• Due to developmental efforts, there used to be many tests but too small of changes to truly

gauge the effect of any change

Metrics & Custom Variables

Target

Conventional Wisdom

46

And/OR Container Logic • Target Standard

It provides visual interface to create and manage tests and rule based targeting on your

website. Just a single line of code manages Target functionality on your webpage

• Target Classic

It is a form based workflow to create and manage A/B tests and multivariate Tests on your

website. Classic supports email testing, which is not available with Standard yet.

• Target Premium

Target Premium is an advanced Target license that adds automated personalization to

Target Standard. Automated Personalization provides advanced machine learning algorithms

to drive personalized experiences and improved conversion rates for digital experiences.

Segmentation

Target

Types of Implementation

47

Key Metrics

• Mbox - Target Classic required you to use Mbox to be wrapped around the element that

you want to test, change. Target Standard requires you to use Target.js instead, load it on

every page to use the Target functionalities.

• Visual Interface – Target.js lets you manage the content on your page with the help of

Visual Experience Composer, whereas Classic needs you to change through the forms that

it provides you.

You can use Target to edit text and HTML on your page, change link descriptions, swap

contents of an element, hide an element of a page etc.

Metrics & Custom Variables

Target

Target Standard Vs Classic

48

Key Metrics

It provides advanced machine learning algorithms to drive personalized experience and

improved conversion rates. It is a part of Target Premium package.

Modeling system automatically determines which piece of content to deliver to each individual

based on all behavioral data the system has about the visitor. This provides a personalized

experience for each visitor. The marketer does not need to run a test, then analyze the results,

then deliver a winner before realizing the lift found from optimization.

You need to specify the percentage of audience that would see the control result, this will help

you gauge the lift that is provided by personalization of experience.

Two kinds of algorithms are provided in automated personalization:

• Random Forest

• Residual Variance Model

Metrics & Custom Variables

Target

Automated Personalization

49

Key Metrics

You can configure a campaign in Target Classic or an Activity in Target Standard to use Adobe

Analytics

as the reporting source.

Before you set up a campaign that uses Analytics as the reporting source, establish the goal for

the campaign, such as improving revenue per visitor (RPV) or increasing clicks on your shopping cart.

Choose a final success metric for the campaign

We can create A/B Tests with Analytics, only when Adobe Analytics, SiteCatalyst is linked with

Adobe Target.

Create A/B with Analytics activity rather than just A/B activity in the Adobe Standard to get

reports within Adobe Analytics.

Note: After a visitor completes your goal, that visitor is no longer included in the campaign. If

the visitor re-enters your campaign after completing an activity, he or she is counted as a new visitor.

Metrics & Custom Variables

Target

A/B Test with Analytics

50

Key Metrics

• Start Small- Do not try to test a lot of elements on a single page. That will only result in

convoluted data, and insights

• Shorten Implementation Time – Implementation time can get extended by tagging

multiple elements on the page. Such effort might again influence the conversion rate and

hide true picture of ROI

• Do not Tinker with the Default – Remember that if the test experience is taking too long

to load/failure to load, then TnT will showcase the default experience to the visitor. So,

make sure that you are not changing the default functinalities

TnT is not just for testing the ROI of different experiences, but it can also be used as a tool to

target various segments of people, for example, different pictures or color scheme for people

coming from Google, different images based on the IP address, “Welcome Back” in place of

“Welcome” to returning visitors.

Metrics & Custom Variables

Target

Points to Remember

51

And/OR Container Logic

Segmentation

Questions/Discussion