docker, the future of devops

Anders [email protected]

@andersjanmyr

mailto:[email protected]

Outline

• Introduction

• What is Docker and why care?

• Docker Overview

• Docker in depth

• Orchestration and Clustering

Anders Janmyr

• Developer 20 years

• Works @jayway

• Full stack developer

• Consultant Company• Malmö, København, Halmstad, Helsingborg,

Stockholm, San Fransisco

• A lot of competence activities• Competence weekends, competence days, Øredev

• We are hiring!

Docker What?

Docker What?

A Lightweight Virtual Machine

Docker What?

chroot on steroids

Lightweight VM

VM vs. Docker

VM vs. Docker

Size

Startup

Integration

Contract

• Standardized Environments

• Few changes

• Isolated dependencies

Ops• Always new

Environments

• Many changes

• Isolated dependencies

Dev

DevOps

Docker Why?

Better Utilisation

1900's 2000 now

Dependencies

The Matrix of Hell

The Docker Solution

100x Faster Enables

• Throwaway dev environments• Databases, Filesystems, Tools

• Throwaway test environments• Filesystems, Databases

• Throwaway prod environments• Upgrade the server by plopping in a new container

8.4.22

9.4.1

Development Environment

2.6.7

2.2.7

3.0.0

DEMODEMO

DEMO0.0.2

Docker Overview

Docker Client-ServerDocker Client

Docker Client

Docker Client

Docker Client

Docker Daemon

DockerContainer

DockerContainer

DockerContainer

DockerHost

Docker Concepts

Registry

Images

Host

Images

ContainerContainerContainer

Volumes

Docker Interactions

Registry

Images

Host

Images


Volumes

push

pull

build

run, createcommit

start, stop, restart

tag

Images

Images are just a hierarchy of files*

* And some metadata

Images

Images

$ docker images # shows all images.$ docker import # creates an image from a tarball.$ docker build # creates image from Dockerfile.$ docker commit # creates image from a container.$ docker rmi # removes an image.$ docker history # list changes of an image.

Images# Deprecated but works$ docker images -viz | dot -Tpng -o tree.png

Example Image Sizes

Name Size Files

scratch 0 0

busybox 2.4 MB ~10500

debian:jessie 122 MB ~18000

ubuntu:14.04 188 MB ~23000

Creating Images

$ docker commit <container-id>

$ docker import <url-to-tar>

$ docker build .

docker commit <container-id>$ docker run -i -t debian:jessie bashroot@e6c7d21960:/# apt-get updateroot@e6c7d21960:/# apt-get install postgresqlroot@e6c7d21960:/# apt-get install noderoot@e6c7d21960:/# node --versionroot@e6c7d21960:/# curl https://iojs.org/dist/v1.2.0/iojs-v1.2.0-linux-x64.tar.gz -o iojs.tgzroot@e6c7d21960:/# tar xzf iojs.tgzroot@e6c7d21960:/# lsroot@e6c7d21960:/# cd iojs-v1.2.0-linux-x64/root@e6c7d21960:/# lsroot@e6c7d21960:/# cp -r * /usr/local/root@e6c7d21960:/# iojs --version1.2.0root@e6c7d21960:/# exit$ docker ps -l -qe6c7d21960$ docker commit e6c7d21960 postgres-iojsdaeb0b76283eac2e0c7f7504bdde2d49c721a1b03a50f750ea9982464cfccb1e

DockerfilesFROM debian:jessie

RUN apt-get updateRUN apt-get install postgresqlRUN curl https://iojs.org/dist/iojs-v1.2.0.tgz -o iojs.tgzRUN tar xzf iojs.tgzRUN cp -r iojs-v1.2.0-linux-x64/* /usr/local

$ docker build -tag postgres-iojs .

Dockerfiles

$ docker build -tag postgres-iojs .

FROM debian:jessie

RUN apt-get update && \ apt-get install postgresql && \ curl https://iojs.org/dist/iojs-v1.2.0.tgz -o iojs.tgz && \ tar xzf iojs.tgz && \ cp -r iojs-v1.2.0-linux-x64/* /usr/local

1 FROM debian:wheezy 2 3 MAINTAINER NGINX Docker Maintainers "[email protected]" 4 5 RUN apt-key adv --keyserver pgp.mit.edu --recv-keys \ 6 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 7 RUN echo "deb http://nginx.org/packages/mainline/debian/ wheezy nginx" >> 8 /etc/apt/sources.list 9 10 ENV NGINX_VERSION 1.7.10-1~wheezy 11 12 RUN apt-get update && \ 13 apt-get install -y ca-certificates nginx=${NGINX_VERSION} && \ 14 rm -rf /var/lib/apt/lists/* 15 16 # forward request and error logs to docker log collector 17 RUN ln -sf /dev/stdout /var/log/nginx/access.log 18 RUN ln -sf /dev/stderr /var/log/nginx/error.log 19 20 VOLUME ["/var/cache/nginx"] 21 22 EXPOSE 80 443 23 24 CMD ["nginx", "-g", "daemon off;"]

Dockerfile

BUILD Both RUNFROM WORKDIR CMDMAINTAINER USER ENVCOPY EXPOSEADD VOLUMERUN ENTRYPOINTONBUILD.dockerignore

Containers

Containers are running* instances of images

* Or stopped, think process

Containers

Containers

$ docker create # creates a container but does not start it.$ docker run # creates and starts a container.$ docker stop # stops it.$ docker start # will start it again.$ docker restart # restarts a container.$ docker rm # deletes a container.$ docker kill # sends a SIGKILL to a container.$ docker attach # will connect to a running container.$ docker wait # blocks until container stops.$ docker exec # executes a command in a running container.

docker run

$ docker run -it --rm ubuntu

--interactive (-i)

--tty (-t)

--rm

ubuntu

docker run

$ docker run -d ubuntu

-d (detached mode)

docker run --env

$ docker run \ --name mydb \ --env MYSQL_USER=db-user \ -e MYSQL_PASSWORD=secret \ --env-file ./mysql.env \ mysql

docker run publish

$ docker run -p 8080:80 nginx

1 FROM debian:wheezy 2 3 MAINTAINER NGINX "[email protected]" 21 22 EXPOSE 80 443 23

docker run publish

$ docker run \ -p 127.0.0.1:8080:80 nginx


docker run publish-all

$ docker run -P nginx


docker run link

$ docker run --name db postgres

$ docker run \ --link postgres:db myapp

docker run limits

$ docker run -m 256m yourapp

$ docker run \ --cpu-shares 512 \ # of 1024 myapp

$ docker run -u=www nginx

docker exec

$ docker exec -it 6f2c42c0 sh

Volumes

Volumes provide persistent storage

outside the container

docker run -v

docker run -v

$ docker run \ -v /var/log \ nginx

host filesystem

/var/lib/docker/volumes/ec3c543bc..535

docker run -v

$ docker run \ -v /tmp:/var/log \ nginx

host filesystem

/tmp

--volumes-from

-v /var/logs/nginx -v /var/www

--volumes-from <nginx>--volumes-from <stats>

statsnginx

worker

Docker Hub

Docker Hub

RepositoryPublic (free), Private (fee)

Automatic BuildsIntegration with Github and Bitbucket

Official reposCertified by vendors

docker pull/push

$ docker pull postgres

$ docker push myname/postgres

Alternativesdocker/docker-registry

Google Container Registry

Core OS

Docker Interactions

Registry

Images

Host

Images


Volumes

push

pull

build

runcommit

start, stop, restart

tag

Inspecting

Inspecting

$ docker ps # shows running containers.$ docker inspect # info on a container (incl. IP address).$ docker logs # gets logs from container.$ docker events # gets events from container.$ docker port # shows public facing port of container.$ docker top # shows running processes in container.$ docker diff # shows changed files in container's FS.$ docker stats # shows metrics, memory, cpu, filsystem

docker ps

$ docker ps --allCONTAINER ID IMAGE COMMAND NAMES9923ad197b65 busybox:latest "sh" romantic_fermatfe7f682cf546 debian:jessie "bash" silly_bartik09c707e2ec07 scratch:latest "ls" suspicious_perlmanb15c5c553202 mongo:2.6.7 "/entrypo some-mongofbe1f24d7df8 busybox:latest "true" db_data

docker inspect$ docker inspect silly_bartik 1 [{ 2 "Args": [ 3 "-c", 4 "/usr/local/bin/confd-watch.sh" 5 ], 6 "Config": { 10 "Hostname": "3c012df7bab9", 11 "Image": "andersjanmyr/nginx-confd:development", 12 }, 13 "Id": "3c012df7bab977a194199f1", 14 "Image": "d3bd1f07cae1bd624e2e", 15 "NetworkSettings": { 16 "IPAddress": "", 18 "Ports": null 19 }, 20 "Volumes": {}, 22 }]

Tips and Tricks

id of last container

$ docker ps -l -q

c8044ab1a3d0

ip of a container

$ docker inspect -f \ '{{ .NetworkSettings.IPAddress }}' \ 6f2c42c05500

172.17.0.11

env-vars of container

$ docker exec -it \ 6f2c42c05500 \ env

PATH=/usr/local/sbin:/usr... HOSTNAME=6f2c42c05500 REDIS_1_PORT=tcp://172.17.0.9:6379 REDIS_1_PORT_6379_TCP=tcp://172.17.0.9:6379

tcp://172.17.0.9:6379

Faster Dev Cycle

Use volumes to avoid having to rebuild the image

Faster Dev Cycle 1 FROM dockerfile/nodejs:latest 2 3 MAINTAINER Anders Janmyr "[email protected]" 4 RUN apt-get update && \ 5 apt-get install zlib1g-dev && \ 6 npm install -g pm2 && \ 7 mkdir -p /srv/app 8 9 WORKDIR /srv/app 10 COPY . /srv/app 11 12 CMD pm2 start app.js -x -i 1 && pm2 logs 13

$ docker build -t myapp . $ docker run -it --rm myapp

Faster Dev Cycle

$ docker run -it --rm \ -v $(PWD):/srv/app \ myapp

Security

Security Problems

• Image signatures are not properly verified

• If you have root in a container, you can, potentially, get root to the whole box

Security Remedies

• Use trusted images

• Don't run containers as root, if possible

• Treat root in a container as root outside a container

Container "Options"

DrawbridgeLXD

Orchestration

What is the problem?

Deploy a Group of Tasks to a Single Host

nginx

Postgres Redis

NodeNode

Deploy Services Dynamically to a Cluster

nginx

Java Node

Node

Kafka

Go

Ruby

GoClojure

Docker Compose (Fig)

docker-compose.yml

1 web: 2 build: . 3 command: python app.py 4 ports: 5 - "5000:5000" 6 volumes: 7 - .:/code 8 links: 9 - redis 10 redis: 11 image: redis

docker-compose up

1 $ docker-compose up 2 Pulling image orchardup/redis... 3 Building web... 4 Starting figtest_redis_1... 5 Starting figtest_web_1... 6 redis_1 | [8] 02 Jan 18:43:35.576 # Server 7 started, Redis version 2.8.3 8 web_1 | * Running on http://0.0.0.0:5000/

http://0.0.0.0:5000/

docker-compose up -d

1 $ docker-compose up -d 2 Starting figtest_redis_1... 3 Starting figtest_web_1... 4 $ docker-compose ps 5 Name Command State Ports 6 ------------------------------------------------------------ 7 figtest_redis_1 /usr/local/bin/run Up 8 figtest_web_1 /bin/sh -c python app.py Up 5000->5000

Other stuff

1 # Get env variables for web container 2 $ docker-compose run web env

3 # Scale to multiple containers 4 $ docker-compose scale web=3 redis=2

5 # Get logs for all containers 6 $ docker-compose logs

Docker Hosting

Machine, Swarm, Compose

Flynn

dokku

Core OS

Core OS Linux

• Minimal 114MB RAM on boot

• No package manager, Docker

• Dual-partition scheme• Read-only boot partitions

• Managed Linux

systemd

• Performance, boots extremely fast

• Logging journal has great features• JSON export

• Forward secure sealing

• Indexing for fast querying

• Socket Activation

etcd

• A distributed, consistent key value store for shared configuration and service discovery

Etcd 1 # Set a value that expires after 60s 2 # Also creates dir /foo is it does not exist 3 $ etcdctl set /foo/bar "Hello world" --ttl 60 4 Hello world 5 6 # Get a value 7 $ etcdctl get /foo/bar 8 Hello world 9 25 # List a directory recursively 26 $ etcdctl ls --recursive / 27 /coreos.com 28 ... 33 # Watch a key for changes 34 $ etcdctl watch /foo/bar 35 Hello world

fleet

• Cluster manager

• Extension of systemd

• Uses etcd for configuration

• On every Core OS machine

fleet

DEMO

DEMO

• Start a single container

• Start a container on every host

• Start a dependent container

Summary

• Ready for production

• But it is not seamless

Beanstalk

• Works right now

• Still as slow as before

Amazon ECS

• Elastic Container Service

• Preview 3

• Not ready for production

Amazon ECS

1 $ aws ecs create-cluster # Creates a cluster 2 $ aws ec2 run instances --image-id i123456 # Starts instances 3 $ aws ecs register-task-definition # Registers a new task 4 $ aws ecs run-task # Runs a task 5 $ aws ecs list-tasks # Lists tasks 6 $ aws ecs list-container-instances # Runs instances

Summary

• Docker fixes dependency hell

• Containers are fast!

• Cluster solutions exists• But don't expect it to be seamless

Anders [email protected]

@andersjanmyr

Questions?

mailto:[email protected]

docker, the future of devops

Software