Machine Learning System Stack

Pooyan Jamshidi

UofSC

[The slides are mainly based on UW Systems for ML Course]

Machine Learning Systems Juggle

Typical Machine Learning System Stack

Typical Machine Learning System Stack

Example: Logistic Regression

Logistic Regression in Numpy

Logistic Regression in Numpy

Logistic Regression in TinyFlow (TensorFlow like API)

Logistic Regression in TinyFlow

Logistic Regression in TinyFlow

Logistic Regression in TinyFlow

Logistic Regression in TinyFlow

Typical Deep Learning System Stack

The Declarative Language: Computation Graph

Computational Graph Construction by Step

Computational Graph Construction by Step

Computational Graph Construction by Step

Computational Graph Construction by Step

Execution only Touches the Needed Subgraph

Discussion: Computational Graph

Discussion: Numpy vs TF Program

Computational graphs in other frameworks: PyTorch

Typical Deep Learning System Stack

Computation Graph Optimization

Parallel Scheduling

Graph Simplifications

Layout Optimization

Layout Optimization

Layout Optimization

Layout Optimization

Computation Graph Optimization

Typical Deep Learning System Stack

GPU Acceleration

Typical Deep Learning System Stack

Supporting More Hardware backends

Each Hardware backend requires a software stack

New Trend: Compiler based Approach