007 www.esigmagroup com advanced site optimization

8/6/2019 007 Www.esigmaGroup Com Advanced Site Optimization

1/16

ADVANCED SITE OPTIMIZATIONSecond-Generation Digital Analytical Tools in Action

Eric T. Peterson, Web Analytics Demystified, Inc.

Sponsored by
http://www.webanalyticsdemystified.com/http://www.webanalyticsdemystified.com/http://www.tealeaf.com/


2/16

AdvancedSiteOptimization:SecondGenerationDigitalAnalyticalToolsinAction

1999 2010TealeafTechnology,Inc.AllRightsReserved.

TABLEOF CONTENTS

ExecutiveSummary ........................................................................................................2

TheChallengewithDigitalAnalysisToday............................................................. 3

DataVolumesCreateTheirOwnProblems..................................................... 3

WebAnalyticsApplicationsLackAnalyticalCapabilities ........................4

AnalysisStaffArentTrainedforAnalysis ......................................................5

NetResult:InsightsAreSlowtoMaterialize................................................... 5

AdvancedSiteOptimization ........................................................................................6

PuttingDatatoWorkforYou(NottheOtherWayAround)........................6

FirstStepsTowardstheComingRevolution................................................... 6

SecondGenerationDigitalAnalysisTools ...............................................................7

MultivariateandA/BTesting............................................................................. 8

AnalyticsIntelligence ..........................................................................................9Tealeaf8............................................................................................................... 10

Conclusions.................................................................................................................... 14

AbouttheAuthor ..........................................................................................................15

AboutWebAnalyticsDemystified ...........................................................................15

AboutTealeaf ................................................................................................................15


3/16


1999 2010TealeafTechnology,Inc.AllRightsReserved. 2

EXECUTIVE SUMMARY

Given the increasing importance of connected channels to nearly all businesses and government

functions,itisnowonderthatanalysisofthesechannelsisgettingmoreattention.Digitalanalysisisthe

practice of making decisions using digitally collected qualitative and quantitative databy leveragingclickstream,feedback,andcustomerexperiencemanagementsolutions.Poweredbybusinessprocessand

performedbydigitalanalysts,digitalanalysisispurportedtohavetheabilitytodramaticallytransform

businessoutcomes.

Exceptthatitdoesnt.

Nearlyadecadeintothedigitalanalysissectorsgrowth,cracksinthefoundationarebeginningtoshow.

Business leaderswhohavecommittedsignificantresourcesstillcomplainaboutgetting toomuchdata,

toofewinsights,andnotnearlythereturnoninvestmentpromised.

At Web Analytics Demystified it is our belief that this situation arises from a perfect storm of

challengesrelating topeople,process,and technology.Widelyavailabledigitalmeasurementplatforms

are decreasingly able to keep up with the volume of data generatedby mobile, social, and traditional

digitalmarketingandcommunicationchannelsofeverybusiness.Whatsmore,whilethevolumeofdata

needing analysis continues to grow, the few existing dedicated resources are dragged downby aging

businessandknowledgemanagementprocesses.

Fortunately,anewbreedoftechnologies isemergingsecondgenerationdigitalanalysisapplications

within the web analytics, customer experience management, and testing and optimization sectors.

Theseapplicationsandplatformsbridgethegapbetweenbasicreportingsolutionsandthetypeofbig

ironbusinessandcustomerintelligencesolutionscommonlyfoundintheofflineenterprise.Bymoving

beyonddatafordatassakeintotherealmofstatisticalmodelsanddataforanalysissake,thesevendors

customers are pushing the limits of what is possible in digital channels and creating a competitiveadvantageintheprocess.


4/16



THE CHALLENGE WITHDIGITAL ANALYSIS TODAY

The promise of digital analysis, using tools like web analytics, has alwaysbeen about surfacing high

value insights drivenby the tremendous volume of data produced through digital channels. These

insightswould thenbepresented tomanagement in the formofrecommendationsand,whenenacted,thevisitorexperiencewouldbecontinually improved.Thisprocesswouldberepeatedand,over time,

keymeasuresofbusinesssuccesslikerevenue,profit,andsatisfactionwouldshowsubstantialgains.

Unfortunately,thisrarelyhappens.

Mostbusiness leaders todayare frustrated with the lack of informationandrecommendations flowing

fromtheiranalyticsefforts,despitesubstantialinvestmentsintechnologyandconsultants.Accordingtoa

2009studyfromAberdeenResearch1,28%ofcompaniesdescribedasbestinclassadmittedthatthey

stillstruggletointerpretdataprovidedbytheirwebanalyticssolution.Reasonsforthisstruggleinclude

lackof insight intohow toconvertmetrics into tangibleactions, lackofresources,difficultyassociated

with data extraction, and difficulty with interpretation. The net result, according to Aberdeen, is that

nearlyonethirdofcompaniespayingforwebanalyticsdonotbelievetheyaregettingthefullvaluefrom

theirinvestment.

ThisdataiscorroboratedbyfindingsfromtheWebAnalyticsAssociationsOutlook2010:SurveyReport2where nearly 70% of respondents indicated that a key initiative in 2010 wouldbe acting on data to

improve site performance, followedby makingbusiness decisions drivenby analytics (63.5%). The

same report cited that 39% of respondents were challengedby taking action on available data, 37.6%

struggled to make business decisions driven by analytics, and 37.2% struggled to gain executive

managementawarenessandsupportforwebanalyticsefforts.

AtWebAnalyticsDemystifiedwebelievethisunfortunatesituationarisesdirectlyfromaperfectstorm

ofchallengestoomuchdatabeingfedintoinadequatesystemsandrunbyundertrainedresources.

DATAVOLUMESCREATETHEIROWNPROBLEMS

TheInternetislikenoothersourceofdatathathasbeenanalyzedbybusinessowners.Whereasoffline

effortstounderstandcustomersaretypicallylimitedtoknownindividualsorbroadsamplesbasedon

demographics or psychographics, digital channels create an almost unmanageable source of data, one

that grow with every mouse movement, click, and interaction.Every single thing each one of us does

onlineistrackedsomewherebysomeone,usuallywiththegoalofoptimizinginteractionsanddriving

conversion.

Problem is, for many sites having access to this much data creates issues rather than solving them.

Especially when dealing with multiple disparate systems such as email marketing systems, socialnetworks,adnetworks,andwebdeliveryplatforms,thehoopsthatdatamanagersneedtojumpthrough

toconnecteachindividualsinteractionarearduousatbest.Theresultisusuallyindependentreporting

silos,eachofwhichhasatendencytoproducetheirownversionofthetruth.

1http://www.aberdeen.com/AberdeenLibrary/6048/RPwebanalyticsmarketing.aspx2http://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waaoutlooksurveyreport20.pdf
http://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspxhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa-outlook-survey-report-20.pdfhttp://www.aberdeen.com/Aberdeen-Library/6048/RP-web-analytics-marketing.aspx


5/16



Highlymotivatedorganizationshavehadsomesuccessdevelopingintegrateddatamodels,oftenbuilton

topofcostlydatawarehouses.Mostbusinessessimplytryandmakedowiththeirdatasilos,hopingthat

analysisresourceswillreconcilethedifferencesandprovideaconsistentandbelievableassessmentofthe

online opportunity using the technology they have at their disposal. Unfortunately, collecting and

keeping data for datas sake nearly always fails to produce the type and level of insights most

companiesexpect.

WEBANALYTICSAPPLICATIONSLACKANALYTICAL CAPABILITIES

Despiteassurances to thecontrary, themajorityof technologiesdeployedforwebanalytics todaydo

very little analysis of data. While certainly improving, most are little more than glorified reporting

interfacesallowingforasmallamountofcustomizationandsegmentation.Themostpowerfulsolutions

advanceddatavisualizationandsegmentationsuiteslikeOmnitureInsights,CoremetricsExplore,and

Unicas NetInsights do an amazingjob of summarizing and packaging the databut stop short of

puttingthatdatatousealgorithmicallyandmathematically.

When pressed to explain why their analytics applications dont do any real analysis eitherby

leveraging statistical models, supporting predictive analytics, or simply providing confidence intervals

andqualityoffitscoresforsegmenteddatatheanswerinevitablycomestowedidntthinkouruserswouldunderstandthat typeofoutput.While thismayhavebeentrue in thepast,and tosomeextent

remainstruetoday,theresultisaprofoundgapbetweenwhatthesystemscoulddoforthebusinessand

whattheyactuallydo.

Thisgapmanifestsinavarietyofways:

Insteadofhighlightingchangesinratesandmeasuresbasedonstatisticalanomaliesdesignedto

call attention to unexpected changes, most systems simply present line graphs and require

analyststoexportthedataintoExcel;

Insteadofprovidingongoingrealtimeanalysispotentiallyacrossmultipledimensionsofdata

thatallowsanalystsandbusinessleaderstorespondinrealtimetoemergingchallenges,most

systems expect that the business cant respond quickly and assume that data latency is

acceptable;

Instead of extrapolating from existing data to make even basic predictions about potential

changes thereby enabling business owners to respond proactively, revise forecasts, and

otherwisebepreparedmostsystemsfocusonproviding informationaboutevents thathave

alreadyoccurred.

Expedia.com has longbeen known for their aggressive approach towards site testing and conversion

optimization. The company regularly conducts Opportunity Cost Analysis essentially an analytic

that uses statistical modeling to differentiate the likelihood to convert visitors experiencing problemsversusthosewhodonot.UsingTealeaf,Expediafocusesonspecificstepsinabusinessprocessflowand

developsdifferentsegmentsbasedoncustomerbehaviors.Forexampleconversionrateforcustomers

thattriggeredformvalidationmessageswhenenteringpersonalinformationversusthosewhodidnot.

Expedia then quantifies the opportunity cost for customers not convertingby applying an average

transactionvalueforsimilartypesoftransactions.

WhilethiseffortcurrentlyrequiresthatExpediaexportdatafromtheircustomerexperiencemanagement

environments forofflineanalysis, thevalue to thebusiness isclear:onecorrectedproblemreturnedan


6/16



estimated $2 million to $3 million this year and, given that the problem had existed for some time, is

likelytoleadtotensofmillionsofdollarsinincrementalrevenueovertime.ByusingthisOpportunity

CostAnalysisanalytic,Expediaisabletoprioritizeprojectsbythebenefittotheirbusiness.Thisanalytic

has also enabled them to manage staffing levels focused on site optimization versus other competing

internalprojects,therebyensuringthatthebusinessisworkingonprojectswiththegreatestreturn.

ANALYSISSTAFFARENTTRAINEDFORANALYSIS

Given the limitationsof themajorityofwebanalyticsapplicationssold today, theresponsefrom the

business hasbeen to apply human resources to fill the gaps. Problem is, these operators web

analystsbydesignationandtradearerarelytrainedasanalystsintheclassicalsense,mostentering

the fieldby accident rather than intent. Even worse,because vendors have done such a goodjob of

convincingtheircustomersthatwebanalyticsiseasyandthetechnologypracticallyrunsitself,too

feworganizationshaveevensearchedforandhiredwebanalystresources,regardlessoftheirexperience

andeducation.

According to the Web Analytics Associations Survey 2010 report, 47% of respondents indicated threeyearsor lessexperienceinthefieldand76%hadsixyearsor lessexperiencein thefield.Comparedtomore traditional analyst roles in business intelligence and financial analysis, where training and

traditional education are firm requirements for employment, a somewhat disturbing pattern emerges:

business leaders are forced to assign increasingly highvalue analysis projects to immature technology

managedbypotentiallyinexperiencedhumanresources.

Wedontblametheanalystsorthebusinessforthissituationeveryoneissimplydoingwhattheycan

ina rapidly changingenvironment.Remember, five yearsago therewas no Facebook; three years ago

therewasnoTwitter;twoyearsagotherewasnoiPhone;lastyeartabletcomputingwassomethingthat

hadbeentriedandhadfailed.Theproblemisnotasmuchthepeople,data,ortechnology;itishowthe

technologyleveragesthedatasothatthepeoplecandothejobtheyvebeenaskedtodo.

NETRESULT:INSIGHTSARESLOWTO MATERIALIZE

Thenetresultof thecurrentstate is thatmostbusinessesstruggle to translate thevolumesofdigitally

collecteddataintomeaningfulinsights,muchlesssolidbusinessrecommendations.Whatsworseisthat

we at Web Analytics Demystified see the potential for a perfect storm in digital analytics: senior

leadershipisincreasinglyaskingformoreadvanced,morerobust,andmorevaluableanalysis,butdata

volumesareincreasingexponentially,inmanycaseswithoutacommensurateincreaseinthecapabilities

ofmostavailabledataanalysisapplications.

Fortunately,thereishopeonthehorizoncomingintheformofsystemsandmethodologiesdesignedto

accelerate time to insightanddramatically improvethevalueexistingdataprovides.Thesesystemsdo

morewith

the

available

data,

ranging

from

rapid,

statistically

powered

design

iteration

support

to

full

blown,modelbasedanalysistomoreeffectivelysurfaceinsights.


7/16



ADVANCEDSITEOPTIMIZATION

AsdiscussedinlengthinourSASsponsoredwhitepaperTheComingRevolutioninWebAnalytics3,WebAnalyticsDemystifiedbelievesweareon the cuspofa paradigm shift in howcompanies analyze and

value digitally collected data. Increasingly we see movement towards what we described as second

generation digital analysis tools applications and platforms that allow forboth greater depth ofexplorationofexistingdataandbegintoautomatethesurfacingofinsightsinwaysthatfirstgeneration

toolssimplydonot.

PUTTINGDATATOWORKFOR YOU(NOTTHEOTHERWAYAROUND)

Fundamentally,advancedsiteoptimizationisaboutgettingyourdatatoworkforyou,notrequiringyou

tocontinue toworkforyourdata.Again, thestatusquo inwebanalytics todayisatworstwhatmy

friendDennisMortensenofYahoo!hascoinedreportsurfing4andatbest,hypothesisdrivenresearch

that requires tremendous attention to detail to account for myriad nuances in data collection, data

processing, and systems management. In either situation the analyst tasked with converting data into

information, information into insights, and insights into recommendations is required to spend hours

siftingthroughdatatoensurethatA)achangeismanifestandB)thechangemanifestissignificant.

Thisistremendouslywasteful,especiallyinthecontextofthelimitednumberoftrulyexperienceddigital

analystsworking in themarket todayand thefact that theofflineworldhasdoneaprettygoodjobof

leveraging data, statistical models, and predictive systems to focus analysis. In fact, when it comes to

timetoinsightsdespitethecommoncomplaintaboutbusinessandcustomerintelligenceinitiatives

onlineanalystshavealottolearnfromtheirofflinecounterparts.

FIRSTSTEPSTOWARDSTHECOMINGREVOLUTION

Theobservationthatmostwebanalyticseffortstodayarenotparticularlyefficientjuxtaposedagainstthe

demonstrablevalueofmodelbasedsystemshaslongprovidedWebAnalyticsDemystifiedwithonlyone

possibleoutcome:webanalyticspractitionersanddigitalmarketersneedbebegintoadoptmorerobust

andcapabletools,technologies,andprocesses.AswedescribedindetailundertheheaderofHow[web

analytics] Technologies Fail Us Today in The Coming Revolution in WebAnalytics,business owners,marketers,andanalystsalikeneedmorepowerfultoolstoaugmenttheirreactivereportingeffortstoday.

Referencing Tom Davenport andJeanne Harris in Competing onAnalytics, we agree that a sustainablecompetitiveadvantagebuiltaroundanalyticswillmanifestonlywhenthebusinesshasaddressedbothits

needforongoingreportingandadvancedanalytics.InFigure1,thetechnologiesDavenportandHarris

proposetocreatethistypeofadvantageareoutlined:

3http://www.webanalyticsdemystified.com/content/whitepapers.asp4http://visualrevenue.com/blog/2007/09/webanalyticsreportsurfingandhowto.html
http://www.webanalyticsdemystified.com/content/white-papers.asphttp://www.webanalyticsdemystified.com/content/white-papers.asphttp://www.webanalyticsdemystified.com/content/white-papers.asphttp://www.webanalyticsdemystified.com/content/white-papers.asphttp://www.webanalyticsdemystified.com/content/white-papers.asphttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://visualrevenue.com/blog/2007/09/web-analytics-report-surfing-and-how-to.htmlhttp://www.webanalyticsdemystified.com/content/white-papers.asp


8/16



Figure1:ReportingandAnalyticsToolsandCapabilities,takenfromCompetingonAnalytics(DavenportandHarris,2007.)

The red line is our addition, indicating the break between firstgeneration and second and third

generationdigitalanalytics tools.Everythingbelowthelineisessentiallywebanalyticsasweknow it

todayandeverythingabovethelineisdigitalanalyticsaswewilldoittomorrow.

When we published this white paper, many in the community responded negatively, essentially

concerned that the technology and processes above the line were difficult to use, expensive, and

fundamentallyoutsidetheirareaofexpertise.Whilethismaybetrue,itisdifficulttoreconcilecomplaints

thatareprimarilypersonalagainsttheneedsofthebroaderbusiness.Thelastconcern,cost,isreasonable,

althoughinourresearchwediscoveredthatmostenterpriseclasscompaniesactuallyhavemanyofthe

necessary technologies already in place. All that is required is to bridge the gap between online

analystsandbusinessanalystswithinthebusiness.

Thatsaid,weatWebAnalyticsDemystifiedprideourselvesongivingpractical,realworldadviceand

guidance and, given this response, we sought examples in the market today of the type of second

generation toolsand technologies likely tocatalyze theparadigmshiftwehavedescribed.Fortunately,

we foundagoodpartner for thiswork inTealeaf, thispaperssponsor,whohaverecentlyreleasedan

excellentexampleoftechnologythatscapableofputtingthedatatoworkforthebusinessbyleveraging

welldesigned statistical models applied to digitally collected data. We also found examples in Google

Analytics Analytics Intelligence feature and the increasingly adopted Multivariate and A/B testing

platformsprovidedbyvendorslikeGoogle,OmnitureandAutonomy.

SECONDGENERATION DIGITAL ANALYSIS TOOLS

Businessownerswishing togetaheadstarton transforming theirdigitalanalyticaleffortsand leading

the revolution are in luck. The past year has seen two new secondgeneration tools to complement

existing testing platforms,both taking good advantage of the available data to surface otherwise non

obviousinsightsandevidenceofconsumerstruggleonline.


9/16



MULTIVARIATEANDA/BTESTING

As discussed in our paper on The Coming Revolution in Web Analytics, we believe that A/B andMultivariate testing tools provide an excellent example of analytical systems leveraging digitally

collected data to focusbusiness decisions. Without going into detail, these systems essentially power

controlled experiments, exposing a sample of site visitors to alternative messages in an attempt to

determinewhichmessageismostlikelytodrivethedesiredoutcome.

The important thing in the context of secondgeneration tools is that these systems rely entirely on

statistical modeling to determine the winner in an experiment. The model usually depends onboth

participationandoutcomedataforexample,peoplebeingexposedtotheexperimentandthengoing

on tocomplete whatevermeasurablesuccessevent isbeing tracked. In this context, the resultsarenot

presentedasthismanypeopledidXbutratherwitha95%confidencethemodelfindsthatXis10%

betterthanY.

Figure2:Exampleresults fromamultivariate test inGoogleWebsiteOptimizershowing thesystemspredictionandconfidence

thatCombination2 OwnaBusinesswilloutperformthecontrolgroup(47.8%improvementwith96.5%confidence)

Web Analytics Demystifieds point of view is that A/B and Multivariate testing systems are incredible

secondgenerationdigitalmeasurementsystemsforthreemainreasons:

1. They shift expectations away from whole numbers and raw counts towards samples and

confidenceintervals;

2. They are able to dramatically simplify reporting, focusing on winners and losers without

needinghugespreadsheets;

3. They explicitly point the business toward logical actions for example, widely deploy

combinationnumbertwoandincreaseyourconversionrate.

While testing continues to gain significant traction within the enterprise, these systems are oftentimes

difficult to deploy and require a specific set of expertise in order to reap their fullbenefits. The same

cannotbesaidforthenextnewestsecondgenerationdigitalanalyticstoolprovidedbyGoogle.


10/16



ANALYTICSINTELLIGENCE

GooglesGoogleAnalyticsofferinginitiallygainedtractioninthemarketplaceasanexcellententrylevel

tool,capableofmeasuringareasonablenumberof common digitalactivitiesandavailableat thebest

possiblepricefree.However,intheyearsfollowingtheapplicationslaunch,theteamatGooglehas

builtouttheproducttocreateoneofthemostuserfriendlyandusefulclickstreammeasurementtoolsin

thewebanalyticsindustry.Theycurrentlysetthebarforavarietyofanalyticalneeds,includingcustom

datacollection,segmentation,andcustomreporting.

InOctober2009,GoogletookGoogleAnalyticstoanentirelynewlevelbyintroducingsecondgeneration

reportingcapabilities,anindustryfirst,intotheapplicationwiththeirAnalyticsIntelligencefeature.Still

inbeta,accordingtoGoogle:

[AnalyticsIntelligenceis]theinitialphaseofanalgorithmicdrivenIntelligenceenginetoGoogleAnalytics.AnalyticsIntelligencewillprovideautomaticalertsofsignificantchanges inthedatapatternsofyoursitemetricsanddimensionsoverdaily,weeklyandmonthlyperiods.

Figure3:Exampleof Analytics Intelligence in Google Analytics. Thebars at thebottom indicate the numberof alerts generated

basedonthealertsensitivity(aproxyfornumberofstandarddeviations)

Whilehelpingtoidentifythedaysthatstatisticallysignificantchangesareoccurringisonething,going

onestepdeeperandhighlightingthebasisforthosechangesissomethingelsecompletely.


11/16



Figure4:Exampleofdrilldown inAnalyticsIntelligence inGoogleAnalyticshighlightingthedimensionsofdataprompting the

alerttobegenerated,theexpectedrangeandobservedvalues,andtherelativesignificance(graybars).

WhileGooglesofferingisanexcellentexampleofsecondgenerationfunctionality,criticsaptlypointout

thatGooglesuseofstatisticsissomewhatrudimentary.Moreimportantly,Googlesdataisoftendelayedbyseveralhoursormore, thus limiting theapplicationsrealutilityasa insightdiscoveryengine.This

problemhasbeensolvedbythispaperssponsor,Tealeaf,intheirmostrecentofferingTealeaf8.

TEALEAF8

Tealeafhas longbeenknown for their enterpriseclass capabilities formonitoring visitorandcustomer

behaviorinthewebandmobilechannels.TheircustomersalistthatincludesExpedia.com,BestBuy,

Walmart.com, Comcast, Wells Fargo, and hundreds of other name brands across all sectors and

geographic segments have long used the companys applications to help identify the points where

visitors struggle and fail. Through integrations with popular clickstream, feedback, and customer

relationshipmanagementapplications,Tealeafhasbecomean integralcomponent in theenterprisesite

optimizationstack.

Because Tealeaf is physically installed in the network architecture for a site, the application is able to

provide true realtime feedback on visitor behavior and has tremendous oversight into online

interactions. This capability imparts a variety of advantages to Tealeaf customers, including uptothe

momentkeyperformanceindicators,richsegmentationcapabilitiesandbrowserreplaythefeaturefor

whichthecompanyhashistoricallybeenknown.

However,inthecontextofadvancedsiteoptimization, thecompanysmostrecentapplicationupgrade

(Tealeaf8)takesautomateddiscoverytoanentirelynewlevel.SimilartoGooglesAnalyticsIntelligence,

Tealeafleveragesstatisticalmodelstoevaluatewhetherchangesfoundinthedataaresignificant.Unlike

GoogleAnalytics,

Tealeaf

8tracks

customer

behaviors

within

segments,

in

real

time,

and

gives

system

usersalmostunlimiteddrilldowncapabilitiestoquantifytheactualrevenueimpactandexploretheroot

causeofobservedchanges.


12/16



Figure5:ExampleofTopMoversandDriversdashboardprovidedinTealeaf8illustratinghowTealeafusesstatisticalmodelingto

automaticallyuncoverstruggleandothercustomerbehaviorissueswithinspecificcustomersegments,inrealtime.

While there hasbeen a longstanding debate over the value of realtime analytics within the digital

measurementcommunity,WebAnalyticsDemystifiedsopinionhasalwaysbeen that there isacertain

class ofbusinesses that require immediate reporting. Highvolume retail, financial services, travel and

transportation, and organizations that depend as much on customer support as sales (e.g., Comcast,

DirectTV, Sky) all have sound financial and satisfactionrelated drivers to uncover challenges and

opportunitiesasquicklyaspossible,especiallyinthesocialera.

Case in point: Matt Raines,Vice President of Technology at online retailer BlueFly, notes thatbecauseBlueFly focuses on ontrend and inseason clothing, any service outage or inability to transact is

unacceptable.Recently,oneofthecompanyscreditcardprocessingpartnersmadeaseeminglyharmless

change that stopped BlueFlys ability to transact using a handful of credit cards. Because this change

occurredatmidnight,andbecausecustomersweregettingonlygenericerrors,theproblemwasslowto

surface.

WithTealeaf8inplace,BlueFlyhastheabilitytosurfaceobscure,highimpactissueslikethisonebefore

theybecomesignificantpointsofstrugglefortheircustomers.AccordingtoMr.Raines,Tealeafprovides

my team the ability to identify, diagnose, and resolve problems like thisbefore theybecome an issue

across the entire company. The realtime, automated insights providedby Tealeaf are the difference

betweenaresponse

measured

in

hours

and

one

measured

in

days

for

me,

and

that

is

huge.


13/16



Figure6:ExampleofrevenueimpactanalysisanddrilldowntovisualsessionreplayusingTealeaf8.

AnotherTealeaf

customer,

ING

DIRECT

USA,

the

nations

largest

direct

bank,

has

been

using

Tealeafs

analytical capabilities to quickly identify problems on the site. Once identified, analysts work to

understandthenatureandseverityofthechallengetobettercommunicatewithintheorganization.This

effort,poweredbyTealeaf,reducestheamountofuncertaintywhenproblemsemerge,allowingING

DIRECTUSAtomakethebestdecisionspossiblebasedondataandintuition,notoneortheother.

WerelyheavilyonTealeaffortrackingcriticalprocessesonoursites,saysEthanJohn,ManageratING

DIRECT USA. Because we are an agile development shop, realtime monitoring of our critical online

business and customer processes gives us the ability to identify, size, and proactively respond to

customerneeds.OutputfromtheanalyticalcapabilitiesinTealeafhasbecomepartofourdailyscorecard,

andasaresultTealeafdriveninsightshavewidespreadacceptancewithinthebusiness.


14/16



Figure7:ExampleofTealeaf8managementdashboardprovidingasnapshotofkeycustomerexperienceindicators.

AnotherTealeafcustomer,Expedia.com,identifiesissuesimpactingconversionbymakingextensiveuse

ofstatisticalregressiontoidentifyPopsandDropsofkeyindicatorsandcustomerbehaviorsgenerated

byTealeaf.GiventhatthecompanyhasthousandsofTealeafeventstheyareactivelytracking,thereisnt

anoptiontousebasicdashboardorspreadsheetreporting.Insteadtheyrelyonabasicstatisticalanalysis

oftheexporteddatatomineforeventsthathavechangedmorethanexpecteddayoverdayandweek

overweek.

Figure8:Exampleofdrilldownintorealtimerevenueopportunitylossanalysistriggeredbyasignificantdeviationinnumberof

customersstrugglingtocheckoutdetectedbyTealeaf8.


15/16



Usingpopsanddropsservesasavaluableaddition to theotheranalysiswedoacrossExpedia.com,

saysMichaelGulmann,SeniorDirectorofGlobalSiteConversion.Tohaveanylevelofefficiencyatall

weneedourdatatoworkforus,nottheotherwayaround.Byleveragingverybasicstatisticalregression

functionality thatshouldbe ineverywebsiteoptimizationpackage inmyopinionweareable to

quicklydiscoveropportunitiesforoptimization.Byleveragingthissimpledatareportingstrategy,Mr.

Gulmann estimates that nearly $10 million hasbeen either saved or added to Expedia.coms FY 10

revenuestream.

WhileTealeaf8isnotthecompanysfirstproductusingstatisticalmodelingtosurfaceinsights,inWeb

AnalyticsDemystifiedsopiniontherecentupgradeisasignificantimprovementinhowthatinformation

is communicated. Even more crucial, Tealeaf is creating a new expectation of how enterpriseclass

analyticsapplicationswillleveragethedatatheycollecttoexpeditetheanalysisprocess.

CONCLUSIONS

Giventhatthevolumeofinformationcollectedindigitalchannelsisunlikelytodecreaseanytimesoon,

wehopethatreadersofthispapercanappreciatethenecessityoflookingforinnovativewaystomanage

data and surface insights. More importantly, we hope that readers appreciate the value second

generationwebanalyticsarelikelytoprovide.Bystreamliningthepresentationofdatabasedoneventhe

most simple data mining and modeling, these applications will give analysts and organizations a

potentially dramatic competitive advantage over companies that must continue to manually mine for

insightsusingfirstgenerationtoolsandtechnology.

Whilewestillbelieve thatdigitalchannelswillultimatelybemeasuredandmanagedusingevenmore

powerful thirdgenerationtechnologies thatarealreadywidelydeployedacross theenterprise, it isour

opinionthattheadoptionanduseoftestingplatformsandrealtimemonitoringserviceslikeTealeaf8are

a critical step towards developing internal comfort and competencies with samples, statistical models,andalgorithmicallyderived insights.Thosecompanieswishing toexplore thepotential impactsecond

generation technology might have on their businesses are encouraged to reach out to the vendors

describedinthisdocumentortoWebAnalyticsDemystifieddirectlyforguidanceandsupport.

Attheendoftheday,businessesarerunonacombinationofdata,gutinstinct,andsoundjudgment.Our

hopeisthatthiswhitepaperhasclarifiedhowWebAnalyticsDemystifiedseestheuseofdataevolving

over time tobecome more timely, relevant, and valuable as part of decisionmaking processes. As

businesses increasingly depend onconnectedchannels,adeptanalysisof data from thesechannelshas

thepotentialtocreatecompetitiveadvantagesthathaveneverexisted.Yourbusinesshasanopportunity

toleveragethoseadvantagestogrowandthriveinyourmarketplace.

Wewelcomeyourfeedbackandcomments.


16/16



ABOUTTHE AUTHOR

Eric T. Peterson, CEO and Principal Consultant at Web Analytics Demystified, has worked in web

analytics since the late 1990s in a variety of roles including practitioner, consultant, and analyst for

several marketleading companies. He is the author of threebestsellingbooks on the subject, WebAnalyticsDemystified,WebSiteMeasurementHacks,andTheBigBookofKeyPerformanceIndicators,aswellasoneofthemostpopularwebanalyticsbloggersatwww.webanalyticsdemystified.com.Mr.Petersonhas

committedmuchofhislifetothebettermentofthewebanalyticscommunity,somuchsothatJimSterne,

President and cofounder of the Web Analytics Association says Erics leadership in the industry in

unparalleled, his devotion to the community is legendary, and his years of experience translate

immediatelyintostrategicandtacticalcompetitiveadvantageforeverybodywhoworkswithhim.

ABOUTWEBANALYTICS DEMYSTIFIED

Web Analytics Demystified, founded in 2007 by internationally known author and former Jupiter

ResearchanalystEricT.Peterson,providesobjectivestrategicguidancetocompaniesstrivingtorealize

the full potential of their investment in web analytics. Bybridging the gap between measurement

technology andbusiness strategy, Web Analytics Demystified has provided guidance to hundreds of

companiesaroundtheworld,includingmanyofthebestknownretailers,financialservicesinstitutions,

andmediapropertiesontheInternet.

FormoreinformationonEricT.PetersonandWebAnalyticsDemystified,pleasevisit

www.webanalyticsdemystified.com,[email protected],or

call(503)2822601.

ABOUTTEALEAF

This whitepaper is sponsored by Tealeaf, the leading provider of online customer experience

management solutions. Tealeafs CEM solutions includeboth a customerbehavioranalysis suite anda

customer service optimization suite. For organizations that are making customer experience a top

priority, these solutions provide unprecedented enterprisewide visibility into every visitors unique

online interactions forongoinganalysisandwebsiteoptimization.Onlineexecutivestakeholders from

ebusiness and IT to customer service and compliance are leveraging Tealeaf to build a customer

experiencemanagementcompetencyacrosstheorganization.Foundedin1999,Tealeafisheadquartered

inSanFrancisco,California,andisprivatelyheld.Formoreinformation,visitwww.tealeaf.com.
http://www.webanalyticsdemystified.com/http://www.tealeaf.com/http://www.tealeaf.com/http://www.webanalyticsdemystified.com/

007 www.esigmagroup com advanced site optimization

Documents