sciops, because data scientists need devops too! · language agnostic - just connect to rabbitmq...
TRANSCRIPT
![Page 1: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/1.jpg)
SciOps, because Data Scientists need DevOps too!
![Page 2: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/2.jpg)
What happens when an idea is born?
![Page 3: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/3.jpg)
![Page 4: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/4.jpg)
What actually happens
![Page 5: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/5.jpg)
![Page 6: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/6.jpg)
![Page 7: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/7.jpg)
Data scientist should have full control over the solutions they implement
![Page 8: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/8.jpg)
A move to useful solutions from brilliant ideas
![Page 9: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/9.jpg)
Frustration!
![Page 10: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/10.jpg)
Engineer Data Scientist
Boring Long iterations
Repetitive No control
No time for Innovation Lots of waiting
![Page 11: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/11.jpg)
● Single library import will turn any function into a high performance server
● Works best with Docker and Kubernetes● Facilitate DevOps culture
AssafSimple Model Serving Platform
![Page 12: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/12.jpg)
Centralized services
Request queues
Response queues
Models
Assaf infrastructure on Kubernetes
Connectors
![Page 13: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/13.jpg)
Major Design Decisions
![Page 14: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/14.jpg)
Request queues
Response queues
Message Queues
● Request queue: messages are picked up by the connector on the designated process
● Response queue: messages are picked up and served to clients
● Both: one-way message queues
![Page 15: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/15.jpg)
Request queues
Response queues
Benefits
● Decoupling of producers and consumers ● Resilient communications - even if services are down
requests are persisted by the queues ● Language agnostic - just connect to rabbitmq
![Page 16: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/16.jpg)
● Connectors handle parallelism ● Prefer using native message passing
○ Go -> Channels○ Scala -> Actors
● ØMQ is used when messages are not supported natively
Message Passing For Concurrency
Connectors
![Page 17: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/17.jpg)
Anatomy an assaf connector
![Page 18: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/18.jpg)
● Embarrassingly Parallel Applications● Write procedural, run in parallel
○ Requests must be atomic ● High Availability out of the box● Infinite horizontal scaling
Benefits
![Page 19: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/19.jpg)
Centralized Services
● High performance collector
● Tracing
● Monitoring Centralized
services
![Page 20: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/20.jpg)
Benefits
● Freedom to choose best tool for
the job
● Change once, enjoy everywhereCentralized
services
![Page 21: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/21.jpg)
Demo
![Page 22: SciOps, because Data Scientists need DevOps too! · Language agnostic - just connect to rabbitmq Connectors handle parallelism Prefer using native message passing Go -> Channels Scala](https://reader036.vdocument.in/reader036/viewer/2022071009/5fc71f2b9b14333f8b4a3caa/html5/thumbnails/22.jpg)
What improvements did we deliver??
Centralized application
Cohesive process for shipping to production
Low effort deployment
Incremental improvements
Propagation of improvements
Responsibility stays with the implementer
CI/CD
Easy to use library