api, whizzml and apps

BigML, Inc 1

Automation

Poul Petersen @pejpgrep CIO, BigML, Inc @bigmlcom

API, WhizzML and Predictive Applications

BigML, Inc 2ML Crash Course - API/WhizzML/Predictive Apps

BigML ArchitectureTools

REST API

Distributed Machine Learning Backend

Source Server

Dataset Server

Model Server

Prediction Server

Sample Server

WhizzML Server

Evaluation Server

Web-based Frontend

Visualizations

Smart Infrastructure (auto-deployable, auto-scalable)


The Need for a ML API

• Workflow Automation - reduce drudgery

• Abstraction - reuse code

• Composability - powerful combinations of APIs

• Integration - Dashboard or UI component

• Automate deployment

• Repeatable results


Predictive Applications

Collect & Format

Data

Define ML

Problem

ETL

Model & Evaluate

no

yes

Explore

Collect & Format

DataModel

Automate

Consume & Monitor

Predict Score Label

Drift & Anomaly

featureengineer

NotPossible

tunealgorithm

Goal Met?


BigML API Endpoint

https://bigml.io/ / /{id}?{auth}

sourcedatasetmodel

ensembleprediction

batchpredictionevaluation

…

andromedadev

dev/andromeda

• Path elements: • /andromeda specifies the API version (optional) • /dev specifies development mode • if not specified, then latest API in production mode

• {id} is required for PUT and DELETE • {auth} contains url parameters username and api_key • api_key can be an alternative key


BigML API Endpoint

https://bigml.io/...{JSON} {JSON}

Operation HTTP Method Semantics

CREATE POST Creates a new resource. Returns a JSON document including a unique identifier.

RETRIEVE GET Retrieves either a specific resource or a list of resources.

UPDATE PUT Updates a resource. Only certain fields are putable.

DELETE DELETE Deletes a resource


BigML Bindingshttps://github.com/bigmlcom/io

https://github.com/bigmlcom/io


Python Binding OverviewOperation HTTP Method Binding Method

CREATE POST api.create_<resource>(from, {opts})

RETRIEVE GET api.get_<resource>(id, {opts}) api.list_<resource>({opts})

UPDATE PUT api.update_<resource>(id, {opts})

DELETE DELETE api.delete_<resource>(id)

• Where <resource> is one of: source, dataset, model, ensemble, evaluation, etc • id is a resource identifier or resource dict • from is a resource identifier, dict, or string depending on context


Diabetes Anomalies

DIABETES SOURCE

DIABETES DATASET

TRAIN SET

TEST SET

ALL MODEL

CLEAN DATASET

FILTER

ALL MODEL

ALL EVALUATION

CLEAN EVALUATION

COMPARE EVALUATIONS

ANAOMALY DETECTOR

BigML, Inc 10


WhizzML

• Complete programming language

• Machine Learning operations are first-class citizens

• Server-side execution abstracts infrastructure

• API First! - Everything is composable

• Shareable

A Domain-Specific Language (DSL) for automating Machine Learning workflows.


WhizzML vs APIWhizzML API / Bindings

Executes server-‐side

Zero latency

Paralleliza?on built-‐in

Sharing built-‐in

Code agnos?c workflows

Workflows can be UI integrated

Requires local execu?on

Every API call has latency

Manual paralleliza?on

Manual sharing

Code specific workflows

Workflows external to UI


WhizzML vs FlatlineWhizzML Flatline

Concerned with resources

Turing complete

Op?mized for paralleliza?on

Concerned with datasets

More specific to features

Op?mized for speed


Simple Workflow

SOURCE DATASET MODEL


Redfin Workflow

Model Predicts

Sale PriceSold

HomesCompare

List to Prediction


Redfin Workflow

MODEL

FILTERSOLD HOMES

BATCH PREDICTION

NEW FEATURES

DATASET DEALS DATASET

FILTERFORSALE HOMES NEW FEATURES


WhizzML Resources

LIBRARY

CITY 1 SOLD HOMES

CITY 1 DEALS DATASET

EXECUTION

CITY 1 FORSALE HOMES

SCRIPT


WhizzML Resources

LIBRARY

CITY 2 SOLD HOMES

CITY 2 DEALS DATASET

EXECUTION

CITY 2 FORSALE HOMES

SCRIPT


Scriptify

• "Reifies" a resource into a WhizzML script.

• Rapid prototyping meets automation.


WhizzML FE

Worth More

Worth Less


WhizzML FE

LATITUDE LONGITUDE REFERENCE LATITUDE

REFERENCELONGITUDE

44.583 -123.296775 44.5638 -123.2794

44.604414 -123.296129 44.5638 -123.2794

44.600108 -123.29707 44.5638 -123.2794

44.603077 -123.295004 44.5638 -123.2794

44.589587 -123.301154 44.5638 -123.2794

Distance (m)

700

30.4

19.38

37.8

23.39

Flatline!


WhizzML FE

https://en.wikipedia.org/wiki/Haversine_formula


WhizML FE

LIBRARY

SCRIPT

Haversine


WhizzML FEFix Missing Values in a “Meaningful” Way

Filter Zeros

Model insulin

Predict insulin

Select insulin

FixedDataset

AmendedDataset

OriginalDataset

CleanDataset


WhizzML Workflow Types

Op?miza?onModel or Ensemble

Best-‐First Features

SMACdown

AlgorithmsStacked Generaliza?on

Gradient boos?ng

Cross Valida?on

Transforma?onsFlatline Wrappers

Remove Anomalies

Domain SpecificApplica?on Workflow

Repe??ve Tasks


Best-First Features{F1}

CHOOSE BEST S = {Fa}

{F2} {F3} {F4} Fn

S+{F1} S+{F2} S+{F3} S+{F4} S+{Fn-1}

CHOOSE BEST S = {Fa, Fb}

S+{F1} S+{F2} S+{F3} S+{F4} S+{Fn-1}

CHOOSE BEST S = {Fa, Fb, Fc}


Model Selection

ENSEMBLE LOGISTIC REGRESSION

EVALUATION

SOURCE DATASET

TRAINING

TEST

MODEL

EVALUATIONEVALUATION

CHOOSE


Model Tuning

ENSEMBLE N=20

EVALUATION

SOURCE DATASET

TRAINING

TEST

EVALUATIONEVALUATION

ENSEMBLE N=10

ENSEMBLE N=1000

CHOOSE


SMACdown

• How many models? • How many nodes? • Missing splits or not? • Number of random candidates? • Balance the objective?

SMACdown can tell you!


Path to Automatic ML

time

Auto

mat

ion

REST API

Programmable Infrastructure

A

Sauron • Automatic deployment and auto-‐scaling

Data Generation and Filtering

C

Flatline • DSL for transformation and new field generation

B

Wintermute • Distributed Machine Learning Framework

2011 Spring 2016

Automatic Model Selection

E

SMACdown • Automatic parameter optimization

Workflow Automation

D

WhizzML • DSL for programmable workflows


Higher Level Algorithms

• Stacked Generalization

• Boosting

• Adaboost

• Logitboost

• Martingale Boosting

• Gradient Boosting


Stacked Generalization

ENSEMBLE LOGISTIC REGRESSION

SOURCE DATASET

MODEL

BATCH PREDICTION

BATCH PREDICTION

BATCH PREDICTION

EXTENDED DATASET

EXTENDED DATASET

EXTENDED DATASET

LOGISTIC REGRESSION


Why WhizzML• Automation is critical to fulfilling the promise of ML • WhizzML can create workflows that:

• Automate repetitive tasks. • Automate model tuning and feature selection.

• Combine ML models into more powerful algorithms.

• Create shareable and re-usable executions.

api, whizzml and apps

Data & Analytics