josh wills, head of data engineering, slack at mlconf sf 2016

23
Several People Are Tuning: Data and ML at Slack

Upload: mlconf

Post on 09-Jan-2017

319 views

Category:

Technology


5 download

TRANSCRIPT

Page 1: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Several People Are Tuning:Data and ML at Slack

Page 2: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

About Me●Slack’s Head of Data Engineering

●Used to work at Cloudera, Google, other places

●Wrote popular tweet that I’m sort of tired of talking about

●Only owns one hat

Page 3: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Our Mission:To make people’s workinglives simpler, more pleasant,and more productive.

Page 4: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Investing in Data(when you’re not an ad/fintech company)

Page 5: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Data Is A Portfolio Business

Page 6: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

The Vision Thing

Page 7: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Org Stuff: Analytics, Data, SLI

Page 8: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

In The Beginning...

Page 9: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Enter Data Engineering

ProdMySQL

(sharded), Solr, Redis

Third Party

Data WarehouseS3, Hive, Spark, Presto, MySQL

Webapp(Apache, PHP)

Sqooper

Logs (via

Kafka)

Page 10: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

The Ghost City

Page 11: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Staffing Up

Page 12: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Sprucing Up The Place

Page 13: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

The Nature of the Work

Page 14: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

The Product Problem

Page 15: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

“We Don’t Sell Saddles”

Page 16: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Building A Knowledge Base

Page 17: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Weird, Wonderful ML

Page 18: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Minimum Viable Data Products

Page 19: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Running Experiments

Page 20: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Channel Recommendations

Page 21: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Data Models for Serving

Page 22: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Giant Shoulders

Page 23: Josh Wills, Head of Data Engineering, Slack at MLconf SF 2016

Thanks! We’re hiring!