ec2 storage for docker 150526b

EC2 Storage for DockerPersistent Containers the Right Way with Container Data Volumes

EMC {code} - @clintonskitson - emccode.github.io 1

What the heck is persistence?$$$


Is anyone using containers for persistence today?


Where's the beef? How's it done today?


ComposabilityThe Container Eco-System.. A Container Data Volume


A Heterogeneous Stack and Granular Services

4 Multiple storage platforms, container runtimes, container OSs, container management platforms

4 Fast Bytes

4 Data Services

4 Data Mobility

4 Availability


Want to use EBS volumes with container granularity?


How are Docker Images, Graph, and Volume drivers related?--storage-driver= and --volume-driver=


First, Big Thanks to the Devs!4 David Calavera @calavera

4 Brian Goff @cpuguy83

4 Arnaud Porterie @icecrime

4 Luke Marsden @lukemarsden

4 Michael Crosby @crosbymichael

4 Alexander Morozov @LK4D4mathEMC {code} - @clintonskitson - emccode.github.io 13

4 Jessie Frazelle @frazelledazzell

4 Tibor Vass @tiborvass

4 Jeff Lindsay @progrium


EXPERIMENTAL (Docker 1.7)4 Volume Enhancements #13161

4 Refactor Volume Interface

4 Prepare for Volumes as 1st class citizens

4 Create a plugin client in Docker to request storage from a Volume Manager API


Volume Manager4 Simple

4 Plumbing of Storage control plane

4 Advanced

4 Abstraction for storage, os, container runtimes

4 Guest storage introspection

4 Inter/intra-storage platform operationsEMC {code} - @clintonskitson - emccode.github.io 17

REX-Ray

github.com/emccode/rexray

Flocker

github.com/clusterhq/flocker


Volume Mananger in the stackBlock is more difficult than NAS (NFS,CIFS) - NAS is globally advertised to a host or set of hosts - Block is targeted towards a host - Block can be surfaced as virtual hardware (below), and networked devices (aside) - Block and NAS can be managed for containers in similar ways with the right abstractions


How does REX-Ray work?


1) OS Driver for Linux


Linux serves as an OS that receives storage volumes as devices and must present these appropriately as mounts.

OS capabilities- Get mounts- Mount/Unmount- Format


2) Storage Driver for EC2 (EBS)


EBS volumes serve as the storage platform underneath the VMs, creating volumes and attaching SCSI devices on demand.

Storage capabilities- Create/Remove Volumes- Create Volumes from Scratch/Snapshot/Other Volume- Attach/Detach Volumes- Create/Remove Snapshot- Copy Snapshot between regions


Introspection- rexray get-instance = Who am I?- rexray attach-volume = Attach to me.

Storage Options- --iops = supported IOPS- --volumetype = General Purpose SSD (gp2), Provisioned IOPS (io1), Magentic (standard)- --size = size in GB- --availabilityzone = target AZ


Basic Storage Workflow- rexray new-volume --size=100 --volumetype=io1 --iops=1000 (storage adapter)- rexray attach-volume --volumeid=vol-ssss (storage adapter - introspection)- rexray format-device --devicename=/dev/xvd_ --fstype=ext4 (os adapter)- rexray mount-device --devicename=/dev/xvd_ --mountpoint=/mnt/test (os adapter)


3) Volume Manager Driver for Docker


The Volume Manager combines both storage and os drivers to simplify management for container runtimes and others. Managing block storage management is simplified and aligns to NAS.

Volume Manager Capabilities- Create/Remove- Mount/Unmount- Path


What Changes4 Old way

4 Advertise block/NFS storage to instance

4 Create FS (block)

4 Mount device to mount point /mnt/vol1

4 docker run -ti -v /mnt/vol1:/vol1 busybox

4 ..then undo it allEMC {code} - @clintonskitson - emccode.github.io 30

Hybrid of Storage/OS/Volume Workflow- rexray new-volume --volumename=testing10 --size=1- rexray mount-volume --volumename=testing10


Advanced Storage Workflow - Copy Volume between AZ- rexray new-volume --volumename=testing20 --size=1 (host1)- rexray mount-volume --volumeid=vol-sss (host1)- touch /var/lib/docker/volumes/testing20/test- rexray get-volumemap (host2)- rexray get-volume --volumeid=vol-sss (host2)- rexray new-volume --volumename=testing20b --volumeid=vol-sss --availabilityzone=blah


4) Docker Volume Driver


How does it work?- docker create - Create Volume- docker start - Mount Volume- docker stop - Unmount Volume- docker rm -v - Remove Container and Volume- docker run - Create, Mount, Unmount Volume- docker run --rm - Create, Mount, Remove Volume


4 Commands get passed via VolumeDriverProxy

4 /usr/share/docker/plugins/rexray.spec (Unix Socket or TCP)

4 unix:///usr/share/docker/plugins/rexray.sock

4 tcp://127.0.0.1:8080

4 Currently only passes name of volume

4 Different instances can represent different profiles in the short term..


4 echo -e "GET /Plugin.Activate HTTP/1.1\r\n" | socat unix-connect:/usr/share/docker/plugins/rexray.sock STDIO

4 /usr/src/go/bin/curl-unix-socket -v -X POST -d '{"Name":"test22"}\r\n' unix:///usr/share/docker/plugins/rexray.sock:/VolumeDriver.Create

4 /usr/src/go/bin/curl-unix-socket -v -X POST -d '{"Name":"test22"}\r\n' unix:///usr/share/docker/plugins/rexray.sock:/VolumeDriver.Mount


4 New Docker way with Volume Drivers

4 docker run -ti --volume-driver=rexray -v volumeName:/externalVol busybox

4 docker run -ti --volume-driver=rexray -v volumeName2:/externalVol2 busybox


Host Mobility- (host1) docker run -ti --volume-driver=rexray -v testing100:/testing100 busyboxtouch /testing100/testexit- (host2) docker run -ti --volume-driver=rexray -v testing100:/testing100 busyboxls /testing100/test


4 What does this enable?

4 Run important things with data in containers

4 Composability of Volume features

4 Storage platform services are directly available to containers

4 Volumes can be easily moved between container hosts


4 Container images remain non-persistent, while the container data volumes are persistent

4 Container OSs remain immutable


4 How do you customize volumes in this experimental mode? != ideal

4 Integrate against storage platform separately

4 Create them ahead of time

4 Dedicate Unix sockets to storage profiles (size, io, type, az)


Future- SSL and Authentication tokens- Storage profiles and settings- Volumes as 1st class citizens- Storage services- Volume updates- Asynchronous calls- Distributed volume manager- Direct to storage platforms (Docker -> EC2?)


Thank YouEMC {code} - @clintonskitson - emccode.github.io 43

4 go get github.com/Soulou/curl-unix-socket

4 docker run --rm -it -v $GOPATH:/go -w /go/src/github.com/emccode/rexraycli golang:1.4.2-cross make release

4 docker run --rm -ti -e DOCKER_EXPERIMENTAL=true --privileged -v ~/go/src/github.com/docker/docker:/go/src/github.com/docker/docker dockercore/docker hack/make.sh binary


ec2 storage for docker 150526b

Technology

linux emc

storage volumes

regions emc

progrium emc

lk4d4math emc

format emc

storage driver

volume manager api emc