mlops automation with git based ci/cd for ml
TRANSCRIPT
![Page 1: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/1.jpg)
MLOps Automation with Git Based CI/CD for ML
Yaron HavivCTO, Iguazio@yaronhaviv
![Page 2: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/2.jpg)
80% of AI Projects Never Make it to Production
Research Environment Production Pipeline
Build from Scratch
with a Large Team
Manual extraction
In-mem analysis
Small scale training
Manual evaluation
Real-time ingestion
Preparation at scale
Train with many params & large data
Real-time events & data features
ETL Streaming APIs
Sync
![Page 3: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/3.jpg)
Did you Try Running Notebooks in Production?
3
Refactor and operationalize
Micro-services
![Page 4: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/4.jpg)
Model and Code Development are Just the First Step
Develop and Test Locally Package
─• Dependencies• Parameters• Run scripts• Build
Scale-out─• Load-balance• Data partitions• Model distribution• AutoML
Tune─• Parallelism• GPU support• Query tuning• Caching
Instrument─• Monitoring• Logging• Versioning• Security
Automate─• CI/CD• Workflows• Rolling upgrades• A/B testing
Weekswith one data
scientist or developer
Monthswith a large team of developers,
scientists, data engineers and DevOps
Production
![Page 5: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/5.jpg)
DevOps MLOps
“Combine Dev and Ops to shorten the systems-development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.”
And data-science
![Page 6: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/6.jpg)
Training
BatchReference
data Serving
Tracking stream
Real-Time Model Monitoring
TSDB
Drift Analysis
RequestsTrigger Fix
IngestionKafka Processing(Aggregation & Analysis)
Auto Feature Detection & Split
Validation
Model
Training set
Raw data Base features
reports
Prediction
Real-time
Example: Predictive Maintenance Pipelinehttps://github.com/mlrun/demo-network-operations
Ops DB
ETL
Model Creation
Online Operations
![Page 7: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/7.jpg)
You can use Separate Tools & Services, Or you can Use Kubernetes as the Baseline
7
![Page 8: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/8.jpg)
8
Collect Any Data
Prepare at Scale
Accelerate Training
Deploy Anywhere
Real-Time Data
Bring your AI Applications to Life withData Science Automation and MLOps
![Page 9: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/9.jpg)
What is an Automated ML Pipeline ?
9
ETL, Streaming, Logs, Scrapers, ..
Ingest Prepare TrainWith hyper-params,multiple algorithms
Validate Deploy ++Join, Aggregate, Split, ..
Test, deploy, monitor model & API servers
End to end pipeline orchestration and tracking
Serverless:ML & Analytics Functions
Features/Data:Fast, Secure, Versioned base features train + test datasets model report report metricsRT features
feedback
Selected model with test data
Real-time
![Page 10: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/10.jpg)
Under The Hood:Open, Scalable, Production Ready
Auto ML Experiment Tracking
Feature Store
Workflows(Kubeflow)
Pipeline Orchestration
Managed Functions and Services
Serverless Automation
Shared GPU/CPU Resources
External Data Sources(file, object, DBs)Real-Time Data LayerData layer
![Page 11: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/11.jpg)
Serverless Simplicity, Maximum Performance
• Automated code to production
• Elastic resource scaling (zero to N)
• Effortless logging, monitoring, and versioning
• High-performance runtimes + fast data access
• Glue-less pipeline and tracking integration
• Reusable internal/public function marketplace
11
18 MB/s
500 MB/s
Simple Python
”Moving from Hadoop/Java to Serverless reduce 90% of our code footprint and got us much better performance”
![Page 12: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/12.jpg)
Serverless: Resource Elasticity, Automated Deployment and Operations
Serverless Today Data Prep and Training
Task lifespan Millisecs to mins Secs to hours
Scaling Load-balancer Partition, shuffle, reduce, Hyper-params
State Stateless Stateful
Input Event Params, Datasets
6
So why not use Serverless for training and data prep?
It’s time we extend Serverless to data-science!
![Page 13: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/13.jpg)
Dynamic Scaling for Intensive Workloads
• Scale + Performance for intensive ML & Data processing tasks
• Seamless transition from user code to elastic, auto tracked jobs + data
• AutoML & Hyper-params are built-in
• Make frameworks “Serverless”• Spark, Dask• MPI/Horovod• SQL (via Presto)• Nuclio
13
Fast inter cluster messaging (MPI, Dask, Spark, ..)
Low-latency data layer (shared code, files, dataframes)
Dynamically Scaled Containers + Distributed Tracking
https://github.com/mlrun/mlrun
![Page 14: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/14.jpg)
KubeFlow: Automated ML Pipelines & Tracking
Manage experiments, runs, and artifacts
Build workflows using code or reusable components (across many cloud/3rd party ML and data framework)
With MLRun & Nuclio:
Automated code to serverless function
Glueless data access and parallelism
Distributed training and GPU Acceleration
Code + execution + data tracking and versioning14
Integrating and Extending KubeFlow Pipelines
![Page 15: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/15.jpg)
Simple, Production-Ready Development Process
15
1. Write and test functions locally
2. Add requirements, run on the cluster
Specify image, packages, cpu/gpu/mem, data, .. Requirements via annotation or function spec
push code and configuration
3. Build/run ML pipeline (interactive or via triggers)
Trigger
Auto-tracking
Run jobBuild
(if needed)
Run job on the cluster (distributed)
Auto-tracking
Summary
Pull/merge request
clone Run steps on the cluster
Auto-tracking
Notifications
Dynamically provisioned serverless jobs/functions
Data-Scientist / Developer Git
ML Engineer
Import functions
MLOps Automation
![Page 16: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/16.jpg)
Building CI/CD Process for ML(Ops)
16
Dev(latest)
Create new branch
Local dev & test
(Notebook)
Convert to micro-svc/function
Test on a cluster
Create Pull Request
Code/spec review
Test entire pipelineMerge PR
Monitor service/drift
Promote to 100%
Deploy canary
Tag version
Master(stable)
Regression tests
release
Develop
Continuous Integration
Continuous Deployment
![Page 17: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/17.jpg)
Traditional Fraud-Detection Architecture (Hadoop)
17
SQL ServerOperational database
ETL to the DWH every 30min
Data warehouse Mirror table
Offline processing (SQL)
Feature vector Batch predictionUsing R Server
40 minutes to identify suspicious money laundering account
40 Precious Minutes (detect fraud after the fact)
Long and complex process to production
![Page 18: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/18.jpg)
Real-Time Fraud Prediction & Prevention
18
SQL ServerOperational database
CDC(Real-time)
Real-timeAnalysis Online + Offline
Feature Store
Model Training
Model Inferencing
Block account !Queue
BatchAnalysis
12 Seconds (prevent fraud)
12 Seconds to detect and prevent fraud ! Automated dev to production using a serverless approach
![Page 19: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/19.jpg)
19
DEMO
![Page 20: MLOps Automation with Git Based CI/CD for ML](https://reader033.vdocument.in/reader033/viewer/2022050612/627407cf7d8ee5080f647498/html5/thumbnails/20.jpg)
Start creating real-world business impact with AI, today
www.iguazio.com