MayaData deliversdata agility

Rancher Kubernetes Master Class

AgendaIntroduction to MayaData - MDAP technology:

● Where are we today? ○ OpenEBS Community / Adoption

● Technology Introduction○ Control plane & Data plane(s) (OpenEBS, Jiva, cStor)○ Node Disk Manager (NDM) - Upstream in Kubernetes○ What is MDAP?

Demo

MayaData and OpenEBSData Agility. Delivered● Proven global team with depth in Storage, DevOps, SaaS and managed services.

Funding by Fidelity / Eight Roads and other long term VCs.● 73 employees. Sponsor of leading container attached storage project OpenEBS and leading

chaos engineering project Litmus.● Committed to our Data Agility mission and our PLOW values.

More at: https://mayadata.io/aboutus

● OpenEBS:○○ 2.5M+ Docker pulls○ 5500+ GitHub stars○ 8k+ users○ 1.3k+ Slack Community users

See OpenEBS vs all CNCF projects: devstats.openEBS.io

Nodes run on physical machines

Pod group containers, share an IP address, and each include a Kubelet agent

K8s Master services include: etcd, APIs, the scheduler, the control manager & others

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

K8s MasteretcdAPIs

CntrlSchld

Architecture: Kubernetes

Data Containers run in Pods on physical machines

Data Containers group physical disks local or remote, manage replicas, provide QoS control and more

OpenEBS runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

K8s MasteretcdAPIs

CntrlSchld

OpenEBS SchldAPIs

Converged : Kubernetes + OpenEBS

Data Containers run in Pods on physical machines

Data Containers group physical disks local or remote, manage replicas, provide QoS control and more

OpenEBS runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others

Maya delivered as SaaS solution to fully automate storage operations

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Node

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

Container

Kubelet

POD

Container

Container

K8s MasteretcdAPIs

CntrlSchld

OpenEBS SchldAPIs

MayaVisualization

Cross DC controlRBAC

ML coaching

Cloud Converged: K8S + OpenEBS + Maya

Storage with CAS detail

K8s NodeDisks

Storage Pool

PVs

Pool or Storage Pod

Pool Types{

Local ext4 ( hostDir) OpenEBS JivaLinux LVMOpenZFSPortworxGlusterPoolCePHPool

}

Local disks

iSCSI targets

GPD

EBS

FC targets

Controller (IO) Pod

Controller (IO) Pod

Critical IO Path

Long Running

Resilient to Disk Errors

Replace/Attach Disks

“CAS” - A Storage Implementation Pattern

Control plane <-> Data plane

Provisioning

API server

Volume exporter

NDM architecture

Node

Control PlaneMonitoring/Alert

Manager (Prometheus)

Node Disk Manager

eBPF, NPD

Kube API Server

Storage Control Plane (Local PV/ OpenEBS/GlusterFS)

Storage Pods(OpenEBS/GlusterFS

External Storage Provisioners

node-disk-manager(ndm) runs as a DaemonSet detects the disks attached to the node and creates corresponding Disk objects.

ndm monitors and detects the faults on the disks and can be configured to push events to Storage Control Plane or Storage Pods running on the local node. It can also consume disk related events from Node Problem Detector (NPD), registering to disk events via eBPF.

ndm will expose metrics that can be used by Prometheus

ndm can be configured to provision additional disks by interacting with external storage provisioners.

Node-disk-manager (NDM)

Jiva storage engine is developed with Rancher's LongHorn and gotgt as the base. Written in GO language and runs in the user space. LongHorn controller synchronously replicates the incoming IO to the LongHorn replicas. The replica considers a Linux sparse file as the foundation. It supports thin provisioning, snapshotting, cloning of storage volumes.

Pluggable OpenEBS Storage EnginescStor is a high performing storage engine built with proven building blocks of storage components. Access protocol iSCSI stack is a linux ported BSD based Multi-threaded iSCSI protocol stack. Field tested at thousands of installations for many years". The storage block layer is the DMU layer of user space ZFS inherited from the proven OpenSolaris stack. Highly reliable for storing and protecting enterprise data.

MayaData Confidential

+ OpenEBS: Leading OSS containerized storage

+ Litmus:Leading OSS chaos engineering for stateful workloads

+ MayaOnline:Control and visibility

+ Services such as DMaaS: Data migration as a service

+ Proactive support

MDAP

16

MDAP commercial solution

○ Leading OSS containerized storage

○ Trivial to set up and run

○ Accelerates & empowers DevOps thanks to snaps and automation and much more

○ Cost savings and independence vs. proprietary cloud or hardware storage

○ Leading OSS chaos engineering for stateful workloads on Kubernetes

○ Increases confidence and resilience

○ User stories directly translated into automated scenarios

○ Integrated into MayaOnline

○ Unequaled visibility into stateful workloads on Kubernetes

○ Reveals hot spots, inconsistencies and more

○ Free version for initial usage

○ Customers receive RBAC enabled version with other extended capabilities

Question & Answer