how to scale your analytics in a maturing organization

Alyson Murphy

Scaling your Analytics in a Maturing Organization

December 11, 2014

@seanvwork #KISSwebinar

Join the conversation on Twi!er

@AlysonMurphy #KISSwebinar

Join the conversation on Twi!er

Sean Work - KISSmetrics - @seanvwork Sean Work runs the blog at KISSmetrics.com. He’s been with the team since 2010. He loves working from Southern California where he can surf, snowboard and camp under the bright starts with no blankies.

Alyson Murphy - Moz - @AlysonMurphy Alyson Murphy is the in-house Senior Data Architect at Moz. She works with stakeholders to build a data solution that help Moz make data informed business decisions.

1 Section One – Some Context

Data Infrastructure

Data Integrity

Data Access and Visualization

Infrastructure Change Process

Data Utilization Process

People in your Organization

2 Section Two –Where to Focus

3 Section Three – Lessons Learned

Table of Contents

Ways to Get Buy-in

Ways to Ship Solutions

Ways to Leverage and Empower Others

WATCH WEBINAR RECORDING NOW

Some Context

1

Introduction

I work as an in-house data analyst and architect at Moz. We sell inbound marketing so"ware to help

people do be!er marketing. Moz is B2B so there will be many B2B examples but all of what I am

talking about can be applied in a B2C context.

Some Context

We have been growing a lot

We had just broken 100 employees when I started less than two years ago and now we are around

150. Scaling so quickly is exciting but can present challenges as well.

Some Context

And Changing a lot

We re-branded and changed our name to

align with the changes in the industry.

We released a new flagship product to

serve the shi"ing needs of the industry.

Some Context

Our data has been changing too

We consolidated key data that was routinely used for analysis onto one reporting server. We then funneled

key pieces of data into our web analytics solution so that there were fewer places to have to look for data

when it was time to do an analysis.

Some Context

Moz.com Production Data

Tool Usage & Subscription Info

Moz email

Moz Analytics Campaign Data

Moz Local data

Reporting Server

Web Analytics Tool

Our Process has Been Changing as Well

We used to use email to get and prioritize projects. We shi"ed to Trello which allows us to have

templates to ensure request quality and to be transparent about when certain projects will be worked

on.

Some Context

Email Trello

We aren’t there yet though

We’ve learned a lot of lessons se!ing us up to scale. Some were things we just needed to struggle with,

but there were some tips I knew two years ago. Hopefully some of them will help you and your team.

Some Context

LOG IN WITH GOOGLE

Start Your Free KISSmetrics Trial

Where to Focus

2

Imagine a ball

Where to Focus

Get close

Where to Focus

Where to Focus

That’s a lot of Green!

Where to Focus

This is how we start. Tactical, in the weeds.

Then we may branch out into orange

and other colors.

But that’s all it is, a series of colors.

It’s not until much later

that we start to see the entire

picture.

Where to Focus

In reality,

your ball probably

looks like this.

Goal 1: Build the

Minimum Viable Ball

Where to Focus

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

There are 6 main areas of focus for building a successful and scalable data system. All parts are required

with equal focus to make the system work.

Where to Focus

Data Infrastructure

Consolidating your data sources will make analysis easier and quicker which is important when you

start adding people to your team. You want your team to be as efficient as possible.

The data infrastructure is tied closely with Data Access and Visualization. The backend choices you

make will affect the front end options that you have.

Moz.com Production Data

Tool Usage & Subscription Info

Moz email

Moz Analytics Campaign Data

Moz Local data

Reporting Server

Web Analytics Tool

Where to Focus

Data Integrity

Data Infrastructure and Data Integrity are perhaps the most important places to start because decisions

in these areas waterfall into the other areas of your Data System.

Spot Check

atTime of Analysis

All Systems Comprehensively

Checked And Maintained

Where to Focus

Data Access and Visualization

Data Access and Visualization is key as your company starts to grow. The goal is to make the data as

easy to access as possible for people who have the skills to fish for their own data. You want to be able to

have business users be able to answer simple questions on their own so that your time can be be!er

spent on more complex business questions.

Types of Business Problems you Need to Consider

•  Simple business questions for researching upcoming projects

•  Complex business questions for researching upcoming projects

•  Health Dashboards for monitoring to ensure nothing is broken

•  Dashboards and Reporting for product/website releases and achievement of Key Results.

Where to Focus

Infrastructure Change Process

When you are in startup mode, everyone might have access to do what they need to do quickly to

implement the changes they need to make. In a small organization this works out because everyone

knows what everyone else is working on. When you start to grow though, this becomes less sustainable

because not every does not know about everything that is going on so they will not understand the

complete ramifications of their actions. Therefore a point person will need to be introduced to sign off on

changes.

Where to Focus

Data Utilization Process

When you are in startup mode, everyone uses whatever data they can get their hands on. Everyone is an

analyst. They might aggregate information from a dashboard and an analysis that was used to answer a

different business question in an effort to be as lean as possible.

However when you scale, it becomes more efficient to have analyst do the analytics work and pass on

the results to the business person. This is because the work will be done more quickly and completely. It

is also the best use of business person’s time to allow the analyst to do the work so they can focus more

on strategizing and managing their area of the business.

Where to Focus

People in your organization

As your business grows, there will be more of a need to level-up people in your organization so that

everyone can access the data and draw sound conclusions.

Level of Complexity

Strategic

Tactical

Analysts Business Users Developers

Research potential projects through

analysis.

Leverage complex analyses to make good

business decisions.

Leverage the work of the analysts in a Production

environment.

Builds and QA’s tools so others can access data.

Pull basic requests correctly.

Implement and maintain the technical-side of the Web Analytics Solution.

Wrangles tools to access data. Ask Analyst to pull data. Structure Databases for

Production Use.

Lessons Learned

3

Goal 1: Build the

Minimum Viable Beach Ball

To optimize the system, you may have to

sub-optimize the subsystem.

Generating change is a PEOPLE PROBLEM

(even if the change is technical)

Over COMMUNICATE what sections you are working on

(helps with buy-in)

Ways to get buy-in

Lessons Learned

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

Lessons Learned

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

Lessons Learned

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

Buying Decisions

Pre-Research Buying Decisions

Lessons Learned

Buying Decisions

Below is the typical process one might go through to pitch why a new tool should be bought.

1 Define problem 2 Lay out possible solutions

3 Research solutions 4 Make decision on solution

HAVE MEETING HERE!

Lessons Learned

Buying Decisions

For the group, a 22 slide deck and 30 minutes was all it took to make the decision to buy a new web

analytics tool.

This is because a lot of research went into those slides. We made a well informed decision quickly.

Lessons Learned

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

Process

Changes

Co-create Process and Content Changes

Lessons Learned

Process and Content Changes

Below is the typical process one might go through to pitch why a new process should be changed or why

we should look at KPI’s differently going forward.

1 Define problem 2 Lay out possible solutions

3 Research solutions 4 Make decision on solution

HAVE MEETING HERE!

HAVE MEETING HERE!

Lessons Learned

Process and Content Changes

In order to re-evaluate the KPI’s we look at, we had a collaborative meeting where each of the groups

came up with a dashboard. We then looked for areas where we needed to create alignment. A"er that, we

started building.

Lessons Learned

Data Infrastructure

Data Integrity

Data Access & Visualization

Infrastructure Change Process

Data Utilization

Process

People in your Org

Components of a Data System

Process

Changes

Empower Leveling-Up

Lessons Learned

Leveling Up

1 Be intentional about the resources you give people.

2 Provide in-person training for people who should use data everyday.

3 Provide optional training for all.

Ways to Ship

Lessons Learned

Minimum Viable Product (MVP) vs. All-in-one

Do you want to ship as li!le as possible as soon as possible and learn and add versus shipping a totally

finished product all at once.

A hybrid exists where you have only key stakeholders involved in the MVP-style iteration then do an All-

in-one rollout to the rest of the interested parties.

Phase and Benefit Minimum Viable Product All-in-one

Planning Phase Must plan theoretical phases of

implementation, thought that may all change as you go.

Must do more detailed planning around what you are shipping since

you are shipping more stuff.

Implementation Phase

You may ship a dashboard in 5 phases with a step to re-evaluate the rest of the plan between each

phase.

You ship it all at once. There will still likely be optimization but it will be

small tweaks as opposed to huge feature changes.

Benefit Allows you to ship sooner so people have something to work with.

Allows for a smoother rollout to a larger group because they will not

be lost in where in the implementation phase the tool is.

Create dashboards people live in.

Lessons Learned

Ship your Minimum Viable Product and Learn

Plan, then ship an MVP in an environment that allows for quick creation, update manually for a few

updates where you learn what needs to change, change it, then automate in a more permanent solution.

Plan 1. Ship MVP and Learn 2. Automate 3.

Ways to Leverage and Empower others

Lessons Learned

Create a Transparent Process

Put all requests in a public place.

Have teams self-prioritize the list to save you time so all you have to do is work on the projects on the

top of the list.

List I used to Prioritize Lists Teams Prioritize

Lessons Learned

Enable Teams to Use a Framework

You may not have time run all of the processes that an analyst would in a larger organization. Create a

framework that the teams use to self-govern. This may not be a long-term solution but it will help free

up time to allow you to optimize other areas of your Data Analytics System.

1 Section One – Some Context

Data Infrastructure

Data Integrity

Data Access and Visualization

Infrastructure Change Process

Data Utilization Process

People in your Organization

2 Section Two –Where to Focus

3 Section Three – Lessons Learned

Table of Contents

Ways to Get Buy-in

Ways to Ship Solutions

Ways to Leverage and Empower Others

Goal 2: Expand the Beach Ball

1 Build the Minimum Viable Beach Ball

2 Pre-Research Buying Decisions

3 Co-Create Processes and Content Changes

Key Take-Away’s

4

5

Empower Leveling-Up

Expand the Beach Ball

THANK YOU

Alyson Murphy @AlysonMurphy