josh wills, head of data engineering, slack at mlconf sf 2016
Post on 09-Jan-2017
319 Views
Preview:
TRANSCRIPT
Several People Are Tuning:Data and ML at Slack
About Me●Slack’s Head of Data Engineering
●Used to work at Cloudera, Google, other places
●Wrote popular tweet that I’m sort of tired of talking about
●Only owns one hat
Our Mission:To make people’s workinglives simpler, more pleasant,and more productive.
Investing in Data(when you’re not an ad/fintech company)
Data Is A Portfolio Business
The Vision Thing
Org Stuff: Analytics, Data, SLI
In The Beginning...
Enter Data Engineering
ProdMySQL
(sharded), Solr, Redis
Third Party
Data WarehouseS3, Hive, Spark, Presto, MySQL
Webapp(Apache, PHP)
Sqooper
Logs (via
Kafka)
The Ghost City
Staffing Up
Sprucing Up The Place
The Nature of the Work
The Product Problem
“We Don’t Sell Saddles”
Building A Knowledge Base
Weird, Wonderful ML
Minimum Viable Data Products
Running Experiments
Channel Recommendations
Data Models for Serving
Giant Shoulders
Thanks! We’re hiring!
top related