DOCKER AND CONFIGURATION

MANAGEMENTPUNE DOCKER MEETUP, 25 APRIL 2015

MUKTA APHALE

AGENDA

• Use Case: Considering Docker in Production

• Configuration Management and Docker

• Docker and Ansible

• Docker and Chef

@muktaa

GOAL

@muktaa

•git push

•Triggers build

CodeGenerate

DockerImage

Build Process

Save ImageDockerImage

Unique TagDockerRegistry

•docker pull

•dockerstop

•docker run

Deploy

EXAMPLE

• git push to https://github.com/muktaa/HelloScala

• Triggers a build on your CI server

• sbt docker

• docker push muktaa/hello-scala

• Deploy

• Build tools offer docker integration

• Eg: Maven has docker-maven-plugin

• https://github.com/spotify/docker-maven-plugin

• mvn clean package docker:build -DpushImage

~/github/HelloScala > sbt docker

[info] Loading project definition from /Users/muktaaphale/github/HelloScala/project

[info] Set current project to hello-scala (in build file:/Users/muktaaphale/github/HelloScala/)

[info] Creating docker image with name: 'muktaa/hello-scala'

:

[info] Sending build context to Docker daemon

[info] Step 0 : FROM dockerfile/java

[info] ---> 1126c85d8a06

[info] Step 1 : ADD /app/hello-scala_2.11-1.4-one-jar.jar /app/hello-scala_2.11-1.4-one-jar.jar

[info] ---> Using cache

[info] ---> 61871958f108

[info] Step 2 : ENTRYPOINT java -jar /app/hello-scala_2.11-1.4-one-jar.jar

[info] ---> Using cache

[info] ---> a8005b32ddc4

[info] Successfully built a8005b32ddc4

[info] Successfully built Docker image: muktaa/hello-scala

[success] Total time: 1 s, completed Mar 3, 2015 2:10:04 PM

~/github/HelloScala > docker images | grep hello-scala

muktaa/hello-scala latest a8005b32ddc4 12 hours ago 715 MB

~/github/HelloScala > docker run muktaa/hello-scala

Hello, world! #1

Hello, world! #2

Hello, world! #3

DOCKER REGISTRYDocker Hub

Link: https://registry.hub.docker.com/u/muktaa/hello-scala

Automated Build in Docker: https://registry.hub.docker.com/u/muktaa/helloscala-automated-build/

CHALLENGES

• Docker based CI server (Travis-CI)

• Tight coupling with build tool

• Base Image

• Monitoring

• Debugging

• Handling configuration for various environments

• Secure Credential Management

@muktaa

LESSONS LEARNT

• Running apps in containers is easy

• Debugging apps in containers is difficult

• You can very well run multiple services inside a dockercontainer

• Ah the woes of Docker networking!

• Sequential Progression

@muktaa

CONFIGURATION MANAGEMENT VS DOCKER

• Control the environment Vs System Image / Runtime image

• Tradeoff between flexibility and convenience

• CM is the vein of DevOps

• Shell scripts -> Chef

• Immutable Infrastructure

@muktaa

DOCKER AND ANSIBLE

@muktaa

ANSIBLE

• Ansible Tower

• Playbooks

• Ansible Tower API

@muktaa

•git push

•Triggers build

CodeGenerate

DockerImage

Build Process

Save ImageDockerImage

Unique TagDockerRegistry

•docker pull

•docker stop

•docker runDeploy

ANSIBLE TOWER API

• http://www.ansible.com/tower

• Dashboard

• Job handling and scheduling ease

• Rest API

• Tower CLI

@muktaa

ANSIBLE TOWER CLI

• https://github.com/ansible/tower-cli

• Configure

• $ tower-cli config host tower.example.com

• $ tower-cli config username mukta

• $ tower-cli config password password

• Launch a job.

• $ tower-cli job launch --job-template=144

@muktaa

ANSIBLE PLAYBOOK

- name: pull latest repo

command: docker pull muktaa/hello-scala

- name: Stop existing container

shell: docker ps | grep 8585 | awk -F" " '{print $1}'

register: result

ignore_errors: true

@muktaa

ANSIBLE PLAYBOOK

- name: Run new docker image

command: docker run -d -i -t --privileged -p 8585:8585 -h={{ ansible_hostname }} muktaa/hello-scala -DCLUSTER_IP={{ ansible_hostname }} -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -DSEED_HOST={{ ansible_seed }}

register: resultignore_errors: true

@muktaa

ANSIBLE DOCKER MODULE

- name: akka seed container

docker:

name: seed

image: muktaa/hello-scala

state: started

expose:

- 8585

volumes_from:

- mydata

env:

SECRET_KEY: xyz

@muktaa

DOCKER AND CHEF

@muktaa

DOCKER COOKBOOK

• Available in Supermarket: https://supermarket.chef.io/cookbooks/docker

• Install docker

• Build docker image

• Pull image and run container

• Push docker image to registry

• LWRPs

• Docker_container

• Docker_image

• Docker_registry

• https://github.com/bflad/chef-docker/blob/master/README.md

CREDENTIAL MANAGEMENT

secret = Chef::EncryptedDataBagItem.load_secret

@docker_cred = Chef::EncryptedDataBagItem.load(

node['docker']['creds']['databag'],

node['docker']['user'],

secret

)

docker_registry‘https://registry.hub.docker.com/u/muktaa/hello-scala/’ do

email docker_cred['email']

username docker_cred['username']

password docker_cred['password']

end

DOCKER_IMAGE

# Build a docker image using docker_imageresource

docker_image node['docker']['image'] do

tag node['docker']['image']['tag']

source '/var/docker'

action :build

end

# Push the image to docker registery

docker_image node['docker']['image'] do

action :push

end

# Delete the image from the machine

docker_image node['docker']['image'] do

action :remove

end

DOCKER_CONTAINER

# Run Container

docker_container ‘muktaa/hello-scala’

detach true

port ‘8081:8081’, ‘8085:8085’

env ‘ENVIRONMENT=pre-prod’

volume ‘/mnt/docker/docker-storage’

action :run

end

GENERATE DOCKERFILE

# Generate a docker file using template.

template "#{node['docker']['directory']}/Dockerfile" do

source 'dockerfile.erb'

variables image: node['docker']['base']['image']['name'],

maintainer: @docker_cred['maintainer'],

email: docker_cred['email'],

build_cmd: node['docker']['build']['commands'],

entry_point: node['docker']['build']['entry_point']

action :create

end

WORKFLOW

Build Application

• Save the Artifact to a Repository Manager

Build DockerImage

• Docker cookbook would build and save the docker image

Deploy• Docker cookbook runs the

container on the nodes

CHEF CONTAINERS

• Package

• Provides Configuration Management for containers

CHEF CONTAINER COMPONENTS

chef-client

runit

chef-init

WHY CHEF CONTAINERS?

• Bootstrap chef-client without SSH connection• Manage multiple services inside your

container• Manage running state of your container• Consistency across Architectures• Mixed Architecture Applications

BEST SUITED FOR

• Transitioning traditional architecture to containers

• Handling last mile configuration when container boots

• Getting the best of two worlds without complexity

KNIFE CONTAINER DOCKER INIT

• Gem install knife-container

• knife container docker init NAMESPACE/IMAGE_NAME [options]• -f base docker image (default is ubuntu 12.04) - chef container

should be already installed on it

• -r runlist

• -z chef client local mode

• -b use berkshelf

EXAMPLE

$ sudo knife container docker init muktaa/hello-scala-cc Compiling Cookbooks...Recipe: knife_container::docker_init* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc] action create* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Dockerfile] action

create- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/Dockerfile from none to 943017- * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/.dockerignore]

action create- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/.dockerignore- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/.dockerignore from none to e3b0c4* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef] action create

- create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/client.rb] action

create- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/client.rb- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/client.rb from none to 7de61f* file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first-boot.json] action

create- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first-

boot.json- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/first-boot.json from none to 5269ef* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/.node_name]

action create- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/.node_name- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/.node_name from none to 4764d2* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Berksfile] action

create (skipped due to only_if)* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/secure] action

create- create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/secure* file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/secure/validation.pem] action create- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/secure/validation.pem- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-

cc/chef/secure/validation.pem from none to ec1f3e- change mode from '' to '0600'Downloading base image: chef/ubuntu-12.04:latest. This process may take awhile...Tagging base image chef/ubuntu-12.04 as muktaa/hello-scala-cc

Context Created: /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc

KNIFE CONTAINER DOCKER BUILD

• run command docker images

• knife container docker build

• resolve docker dependencies

• build docker image

• cleanup chef artifacts

EXAMPLE

$ sudo knife container docker build muktaa/hello-scala-ccSending build context to Docker daemon 9.728 kBSending build context to Docker daemon Step 0 : FROM muktaa/hello-scala-cc---> 50d3c5c9e133

Step 1 : ADD chef/ /etc/chef/---> 4933cc9e13e0

Removing intermediate container da0a08413a91Step 2 : RUN chef-init --bootstrap---> Running in add27db609cc

[2015-03-31T21:44:44+00:00] INFO: Starting Supervisor...[2015-03-31T21:44:44+00:00] INFO: Supervisor pid: 9[2015-03-31T21:44:49+00:00] INFO: Starting chef-client run...[2015-03-31T21:44:50+00:00] INFO: Forking chef instance to converge...[2015-03-31T21:44:50+00:00] INFO: *** Chef 11.16.2 ***[2015-03-31T21:44:50+00:00] INFO: Chef-client pid: 16[2015-03-31T21:44:53+00:00] INFO: Client key /etc/chef/secure/client.pem is not present - registering[2015-03-31T21:44:53+00:00] INFO: HTTP Request Returned 404 Object Not Found: error[2015-03-31T21:44:54+00:00] INFO: Setting the run_list to [] from CLI options[2015-03-31T21:44:54+00:00] INFO: Run List is [][2015-03-31T21:44:54+00:00] INFO: Run List expands to [][2015-03-31T21:44:54+00:00] INFO: Starting Chef Run for muktaa-hello-scala-cc-build[2015-03-31T21:44:54+00:00] INFO: Running start handlers[2015-03-31T21:44:54+00:00] INFO: Start handlers complete.[2015-03-31T21:44:55+00:00] INFO: Loading cookbooks []

[2015-03-31T21:44:55+00:00] WARN: Node muktaa-hello-scala-cc-build has an empty run list.[2015-03-31T21:44:55+00:00] INFO: Chef Run complete in 1.121705004 seconds[2015-03-31T21:44:55+00:00] INFO: Running report handlers[2015-03-31T21:44:55+00:00] INFO: Report handlers complete[2015-03-31T21:44:55+00:00] INFO: Sending resource update report (run-id: 6f637baf-18cc-4620-b3e2-9afc90e8cd6b)---> 2c2ec6fab1efRemoving intermediate container add27db609ccStep 3 : RUN rm -rf /etc/chef/secure/*---> Running in 30a3611b083f---> cab28d6eed90

Removing intermediate container 30a3611b083fStep 4 : ENTRYPOINT ["chef-init"]---> Running in 0a9f4e96bbf7---> a8577b66b103

Removing intermediate container 0a9f4e96bbf7Step 5 : CMD ["--onboot"]---> Running in f9a444817229---> 21b3800bc9b3

Removing intermediate container f9a444817229Successfully built 21b3800bc9b3

DOCKER IMAGES

$ sudo docker images

REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE

muktaa/hello-scala-cc latest 21b3800bc9b3 2 hours ago 311.9 MB

<none> <none> b343c8301cc8 2 hours ago 311.9 MB

chef/ubuntu-12.04 latest 50d3c5c9e133 6 months ago 311.9 MB

$ sudo docker push muktaa/hello-scala-cc

$ sudo docker –d run muktaa/hello-scala-cc

THANK YOU!

@muktaa