how “stranger things” can happen with visual analytics · how “stranger things” can happen...

How “Stranger Things” can happen with Visual Analytics

Jason FlittnerSenior Analytics Engineer / ManagerNetflix - Content Data Engineering and Analytics

#NetflixData

● About Netflix

● Tableau + Big Data

○ Lessons Learned

○ Where we are today

● Analytics and Iterating Quickly

What is Netflix?

● 93+ million members

● 190 countries

● 1,000+ devices

● 10B hours/qtr

We plan on spending ~$6B in 2017 on content for our members

Metrics

● ~60 PB DW on S3

● ~1400 Tableau users

● Live & extract connections

● Analytics on billions of rows

(Hadoop clusters)

Storage Compute Data Interface Data Access, Analytics and Visualization

AWS S3

● About Netflix

● Tableau + Big Data

○ Lessons Learned

○ Where we are today

● Analytics and Iterating Quickly

Choosing a source

● Hive

● Spark

● Presto

● Redshift

● Published Data Source

● etc...

● Powerful and scalable backend

● “Slower” 1,000,000,000/hr

● Hive + Tableau

○ Thrift Servers

○ Custom SQL vs Tables

○ Metadata

○ ODBC Optimization

● Scalable

● Faster than Hive in many cases

● Spark + Tableau

○ Thrift Servers

○ Long running job on Cluster

○ Query reliability

● Fast query engine

● Great for experimenting and

“smaller” data sets

● Connecting to Tableau

○ Web data connector

○ ODBC

● About Netflix

● Tableau + Big Data

○ Lessons Learned

○ Where we are today

● Analytics and Iterating Quickly

Tableau Data Extract Publish to Server

Tableau Extract API

Create Tableau Data ExtractProvision Container ResourceIssues Command Create Extract

Publish to Server

Distributed Tableau Extract API

● Very fast loads from S3

● Native Tableau connector

● Quick Tableau Iteration

● Live or Extract

● Concurrency

Amazon Redshift

BIG Data● Too big to extract?

● Optimized live connections

○ SQL

● Custom data viz with Druid

● Tableau + Hyper!?

● About Netflix

● Tableau + Big Data

○ Lessons Learned

○ Where we are today

● Analytics and Iterating Quickly

Business users

Analytics Engineer

Analytics:

● Binge Analysis

● Viewing Patterns

● Hours Viewed

● Customer Joy

● Content Quality

Bringing it all together

● Content analytics

● Iterate quickly

● Move between backend sources

● Strong user adoption

Merci

Thank you

Jason Flittner -