building a data driven organization

@AbeStanway

BUILDING A DATA DRIVEN ORGANIZATION

1. why 2. how

1. why 2. how

“DATA IS THE NEW GOLD”

Predict the future!

Retain Customers!

Grow the business!

Recommend content!

Drive Engagement!

unclear paths to $$$

IN IT, It’s clear.

Data are Dollars

. IT Working = +$$$ IT Not Working = -$$$ . .

How do you know if your IT is working

right now?

How do you know if you are earning money

right now?

KPIs. What are they?

Etsy: Literally a Money per

second Graph

Planet Labs: Literally an Images

per day graph

What are the Kpis for kips?

$ per second

items bought per

second page requests

per second

database queries per

second memcache hits per second

fread() per second

If you do not have the data about your infrastructure,

it is already broken.

LEsson:

Test driven development ->

data driven development

Without data, you are flying blind

How do you know you’re hitting your goals?

How do you know if You’re making the right ones in the

first place?

How do you know if you’re still in

business?

How do you even know what planet

you live on?

Assumptions are death

You need data, yo.

1. why 2. how

1. collect 2. analyze 3. ??? 4. Profit!

1. collect 2. analyze 3. ACT 4. Profit!

data that cannot be acted upon should

not be analyzed.

You are running a business, not an

art museum

You are Trying to Win the market, not

a fields medal

This can be disappointing

Data SCientist?

Data Scientist? Realist.

Find a way to Align your employees intellectual curiosity

With your Real business needs.

LEssoN:

Train your organization

you need a data culture.

“It’s not shipped until it’s monitored”

“If you are not looking at Dashboards, you are

not doing your job”

Building instrumentation and watching dashboards

are hard And Time consuming

App code -> statsD -> Graphite -> Dashboards -> Insightsby hand by hand by hand by hand by hand

Developers just want to code

Let’s automate

Which is easier to automate?

Insights or data collection?

Insights are sexy and fun

Collection is hard And unsexy

Collection is hard And Boring

Collection is hard And unsexy

We’re on track to have excellent

automated insights

anomaly detection

App code -> statsD -> Graphite -> Dashboards -> Insightsby hand by hand by hand AUTOMATIC!AUTOMATIC!

(…if only we had the data)

How do we automate data collection?

currently have ganglia, New relic,

collectD, etc

NOT WHAT WE NEED

they provide data about your raw machines, not your

CUSTOM DEVELOPED TECHNOLOGY And Application level logic

Healthy servers don’t make you money. Healthy services do.

enter LARIMAR

Full disclosure: this is my new PROJECT

!

we’re going to talk about it because i’m pretty excited and the

beta is opening up soon.

LARIMAR uses raw machine metrics to infer App level architecture and inform developers about problems

A service:cpu resources

disk io PCAP data

ports Used syscalls

A service:cpu resources

disk io PCAP data

ports Used syscallsservice fingerprint

MACHINE LEARNING

A service:cpu resources

disk io PCAP data

ports Used syscallsABNORMAL BEHAVIOR

MACHINE LEARNING

a system:service service service service service

MACHINE LEARNING

graphical system fingerprint

a system:service service service service service

MACHINE LEARNING

Abnormal, holistic system behavior

Larimar automates both analysis And

Relevant data collection

so your developers can focus on coding

and acting on insights

No configuration!

App code -> statsD -> Graphite -> Dashboards -> Insightsby hand AUTOMATIC!AUTOMATIC!AUTOMATIC! AUTOMATIC!

1. collect 2. analyze 3. ACT 4. Profit!

organizational shifts are still

needed to inspire ACTION on Data

but ACTION is easier to inspire when there is lots of data and lots

of insight everywhere

Create a culture where your developers create

these kinds of tools

When a data driven mindset is the default,

tools will build themselves.

Thanks! @abestanway

!

!

larimar.io @larimarhq