sciops, because data scientists need devops too! · language agnostic - just connect to rabbitmq...

SciOps, because Data Scientists need DevOps too!

What happens when an idea is born?

What actually happens

Data scientist should have full control over the solutions they implement

A move to useful solutions from brilliant ideas

Frustration!

Engineer Data Scientist

Boring Long iterations

Repetitive No control

No time for Innovation Lots of waiting

● Single library import will turn any function into a high performance server

● Works best with Docker and Kubernetes● Facilitate DevOps culture

AssafSimple Model Serving Platform

Centralized services

Request queues

Response queues

Models

Assaf infrastructure on Kubernetes

Connectors

Major Design Decisions

Request queues

Response queues

Message Queues

● Request queue: messages are picked up by the connector on the designated process

● Response queue: messages are picked up and served to clients

● Both: one-way message queues

Request queues

Response queues

Benefits

● Decoupling of producers and consumers ● Resilient communications - even if services are down

requests are persisted by the queues ● Language agnostic - just connect to rabbitmq

● Connectors handle parallelism ● Prefer using native message passing

○ Go -> Channels○ Scala -> Actors

● ØMQ is used when messages are not supported natively

Message Passing For Concurrency

Connectors

Anatomy an assaf connector

● Embarrassingly Parallel Applications● Write procedural, run in parallel

○ Requests must be atomic ● High Availability out of the box● Infinite horizontal scaling

Benefits

Centralized Services

● High performance collector

● Tracing

● Monitoring Centralized

services

Benefits

● Freedom to choose best tool for

the job

● Change once, enjoy everywhereCentralized

services

Demo

What improvements did we deliver??

Centralized application

Cohesive process for shipping to production

Low effort deployment

Incremental improvements

Propagation of improvements

Responsibility stays with the implementer

CI/CD

Easy to use library

https://github.com/assaf-platform

@raam86

@dalidepicciottoart: