Download - Deploying Dockerized Applications with Salt
![Page 1: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/1.jpg)
Hello.Nice to meet you!Deploying Dockerized apps with Salt
![Page 2: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/2.jpg)
1
Who am I?
![Page 3: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/3.jpg)
Senior Developer @ SOON_
I'm @krak3n on GitHub, Twitter etc
Hi, I’m Chris
![Page 4: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/4.jpg)
2
What is SOON_?
![Page 5: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/5.jpg)
We make brands successful by combining insightful strategy, seamless technology, persuasive design and relevant content
We can help you be more digital** we don’t like the word “digital”, but it’s the word people use. Ask us and we’ll tell you why.
![Page 6: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/6.jpg)
3
What is Do-It?
![Page 7: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/7.jpg)
Do-It is the UK’s biggest volunteering network
Listing volunteering opportunities from thousands of charities and social action groups throughout the UK
Many of the opportunities on Do-it come from physical Volunteer Centres around the UK
![Page 8: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/8.jpg)
![Page 9: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/9.jpg)
![Page 10: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/10.jpg)
4
The approach
![Page 11: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/11.jpg)
Complete re-platform of the Java application. Start from scratch
Use modern technologies and frameworks
Separate Backend and Frontend builds for an API driven application
![Page 12: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/12.jpg)
5
Docker 101
![Page 13: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/13.jpg)
Wrapper around LXC* - method of running multiple isolated Linux systems
Run single processes
Ship your application with your OS
docker run --rm -it ubuntu:14.04 /bin/bashroot@7623d871ac08:/#
Easy to build and run applications anywhere (that supports docker)
* Linux Containers - Released 2006
![Page 14: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/14.jpg)
Version control for LXC
$ docker run ubuntu:14.04 apt-get update$ docker ps -lCONTAINER ID IMAGE 9a0fa929a3c7 ubuntu:14.04 $ docker commit 9a0fa929a3c7 ubuntu:14.04
$ docker run apt-get install -y curl$ docker ps -lCONTAINER ID IMAGE97957263ea5c ubuntu:14.04$ docker commit 97957263ea5c ubuntu:14.04
$ docker run ubuntu:14.04 curl http://google.com
![Page 15: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/15.jpg)
Build Images with Dockerfile$ cat ./DockerfileFROM ubuntu:14.04RUN apt-get update && apt-get install -y curl
$ docker run --rm ubuntu_with_curl curl http://google.com
$ docker build -t ubuntu_with_curl .
Push images to a central repository
$ docker login -e [email protected] -u you -p 123$ docker push ubuntu_with_curl
$ docker login -e [email protected] -u you -p 123 http://your.index.com$ docker push your.index.com/ubuntu_with_curl
![Page 16: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/16.jpg)
Pull Images from a central repository.
$ docker pull ubuntu_with_curl$ docker run --rm ubuntu_with_curl curl http://google.com
Pass environment variables to containers$ docker run --rm -it -e FOO=foo ubuntu:14.04 bashroot@2281fb4f13a4:/# echo $FOOfoo
Run containers in detached mode
$ docker run -d foo ubuntu:14.04 foo
![Page 17: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/17.jpg)
You can name containers
$ docker run -d --name foo ubuntu:14.04 some_process
Then you can stop, start and restart your contains like processes
$ docker stop foo$ docker start foo$ docker restart foo
View the logs of your containers
$ docker logs --follow foo
![Page 18: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/18.jpg)
Or attach* to running containers$ docker attach foo$ docker start -a foo* does not allow you to execute commands inside the container
Execute commands in running containers - useful for debugging$ docker run -d --name foo ubuntu:14.04 foo$ docker exec -it foo /bin/bashroot@4cc94bc02b5f:/# ps aux
Remove containers
$ docker rm foo$ docker rm -f foo
![Page 19: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/19.jpg)
Also remove images$ docker rmi ubuntu:14.04$ docker rmi -f ubuntu:14.04
Containers can be linked with each other$ docker run -d -e RABBITMQ_USER=chris -e RABBITMQ_PASS=chris --name rabbitmq tutum/rabbitmq
$ docker run --rm -it --link rabbitmq:rabbitmq ubuntu:14.04 /bin/bash
$ root@507079e8b54a:/# echo $RABBITMQ_PORTtcp://172.17.0.19:5672
![Page 20: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/20.jpg)
Sharing data with volumes
$ docker run --rm -it \ -v /host/path:/container/path/ foo ubuntu:14.04 \ /bin/bash
$ docker run --rm -it \ -v /host/path/file.x:/container/path/file.x foo \ ubuntu:14.04 /bin/bash
e.g your PostgreSQL data directory
$ docker run -v /data/psql:/var/lib/postgresql \ --name db orchardup/docker-postgresql
![Page 21: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/21.jpg)
$ docker run --privileged --rm -it \ foo ubuntu:14.04 \ /bin/bash
![Page 22: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/22.jpg)
6
A simple application(we will get to Salt SOON_)
![Page 23: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/23.jpg)
Let's look at a simple Flask application we'll deploy later with Salt and Docker
/demo├── Dockerfile├── setup.py└── foo ├── __init__.py └── app.py
#!/usr/bin/env python# encoding: utf-8
from flask import Flaskapp = Flask(__name__)
@app.route('/')def hello_world(): return 'Hello World!'
if __name__ == '__main__': app.run(host='0.0.0.0')
Tree app.py
![Page 24: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/24.jpg)
And the DockerfileFROM ubuntu:14.04RUN apt-get update && apt-get install -y python python-dev \ && apt-get clean \ && apt-get autoclean \ && apt-get autoremove -y \ && rm -rf /var/lib/{apt,dpkg,cache,log}/ADD . /fooWORKDIR /fooRUN python setup.py installEXPOSE 5000CMD ["python", "/foo/foo/app.py"]
Build it, Push it, Run It!
$ docker build -t soon/foo .$ docker push soon/foo$ docker run --rm -p 5000:5000 foo
![Page 25: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/25.jpg)
![Page 26: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/26.jpg)
7
Salt ♥ Docker
![Page 27: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/27.jpg)
Before Salt can use docker we need to install docker and docker-Py
docker-ppa: pkgrepo.managed: - name: deb https://get.docker.io/ubuntu docker main - keyserver: hkp://keyserver.ubuntu.com:80 - keyid: 36A1D7869245C8950F966E92D8576A8BA88D21E9 - require: - pkg: software-properties-common - pkg: apt-transport-https
lxc-docker: pkg: - installed service.running: - name: docker - sig: /usr/bin/docker - require: - pkg: lxc-docker
docker-py: pip.installed: - reload_modules: True - require: - pkg: python-pip
![Page 28: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/28.jpg)
We have already pushed soon/foo to Docker Hub*, using Salt, we can pull down that image
soon/foo: docker.pulled: - tag: latest - force: True - require: - pip: docker-py - service: lxc-docker
* this is a public repo - anyone can download it: docker pull soon/foo
docker-py must be installed
force: True - Always pull the latest image
![Page 29: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/29.jpg)
Now we need to create the containerfoo-container: docker.installed: - name: foo - image: soon/foo - require: - docker: soon/foo
Then we can run the new containerfoo: docker.running: - container: foo - port_bindings: "5000/tcp": HostIp: "" HostPort: "5000" - require: - docker: foo-container
![Page 30: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/30.jpg)
Simples! But thats not enough...
We also need to be able to stop and remove containers when the image has changed.
We can watch for changes to the image
foo-absent: cmd.wait: - name: docker rm -f foo - watch: - docker: soon/foo
We can’t use docker.absent
![Page 31: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/31.jpg)
Like the private IP address of the container
$ salt-call docker.get_containers$ salt-call docker.get_containers inspect=True$ salt-call docker.inspect_container foo
Getting information about our running containers
"NetworkSettings": { "IPAddress": "10.1.0.13", "Gateway": "10.1.42.1", "Ports": { "5000/tcp": null }}
![Page 32: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/32.jpg)
Now we can set up an Nginx* config to proxy directly to the container* assumes you have Nginx installed via Salt
foo-nginx-config: file.managed: - name: /etc/nginx/conf.d/foo.conf - source: salt://foo.nginx.conf - template: jinja - watch_in: - service: nginx - require: - docker: foo
![Page 33: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/33.jpg)
In our foo nginx config, we can get the IP of our foo container and proxy directly to it without the need for port binding
{% set foo = salt['docker.inspect_container']('foo')['out'] %}{% set ip = foo['NetworkSettings']['IPAddress'] %}
server { listen 80; server_name foo.com; location / { proxy_pass http://{{ ip }}:5000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-for $remote_addr; }}
![Page 34: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/34.jpg)
We also need to clean up after ourselves
$ docker rmi $(docker images -q -f dangling=true)
This only needs to be run if the image changed and after the container was removed
cleanup: cmd.wait: - name: docker rmi $(docker images -q -f dangling=true) - watch: - cmd: foo-absent
![Page 35: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/35.jpg)
We can also run multiple containers of the same image{% for x in range(1, 5) %} # 4 containers
foo-{{ x }}-absent: cmd.wait: - name: docker rm -f foo-{{ x }} - watch: - docker: soon/foo - watch_in: - cmd: cleanup
foo-{{ x }}-container: docker.installed: - name: foo-{{ x }} - image: soon/foo - require: - docker: soon/foo - watch: - cmd: foo-{{ x }}-absent
![Page 36: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/36.jpg)
foo-{{ x }}: docker.running: - container: foo-{{ x }} - require: - docker: foo-{{ x }}-container - require_in: - file: foo-nginx-config
{% endfor %}
Tell nginx how many containers we run
foo-nginx-config: file.managed: - name: /etc/nginx/conf.d/foo.conf - source: salt://foo3.nginx.conf - template: jinja - context: no_containers: 4 - watch_in: - service: nginx
![Page 37: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/37.jpg)
Add an upstream backend to nginx to proxy to our 4 containersupstream backend {{% for x in range(1, no_containers + 1) -%}{%- set foo = salt['docker.inspect_container']('foo-' + x|string)['out'] -%} server {{ foo['NetworkSettings']['IPAddress'] }}:5000;{% endfor -%}}
server { listen 80; server_name foo.com; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-for $remote_addr; }}
![Page 38: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/38.jpg)
Be careful with docker logs
They are not rotated and everything from STDOUT and STDERR ends up in them
They are only cleaned out when you remove a container
https://github.com/docker/docker/issues/7333
Set your app to log to a centralised log store, like LogStash
![Page 39: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/39.jpg)
Private docker repositories for your secret source
docker-registries: https://index.docker.io/v1/: email: [email protected] password: 124 username: foo
Just add this to your pillars
![Page 40: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/40.jpg)
8
Bonus Round
![Page 41: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/41.jpg)
Continuous delivery with CircleCImachine: services: - docker environment: REPO: soon/foo TAG: $(sed 's/master/latest/;s/\//\-/' <<<$CIRCLE_BRANCH)
dependencies: pre: - sed "s/<EMAIL>/$DOCKER_EMAIL/;s/<AUTH>/$DOCKER_AUTH/" < .dockercfg.template > ~/.dockercfg override: - docker build -t $REPO:$TAG .
test: override: - docker run -it --name test --net=host $REPO:$TAG python setup.py test
deployment: release: branch: master commands: - docker push $REPO:$T AG
![Page 42: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/42.jpg)
Salt REST API to trigger state runsrest_cherrypy: port: 8000 host: 127.0.0.1 disable_ssl: True webhook_disable_auth: True
$ service salt-api start
Add reactor events to react to webhook callsreactor: - 'salt/netapi/hook/circleci/success': - /srv/salt/reactor/deploy.sls
Start the Salt API Service
![Page 43: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/43.jpg)
Create a reactor sls to handle the event
def run(): ret = {} ret['deploy'] = { 'cmd.state.highstate': [ {'tgt': '*'}, ] } return ret
Send the event by hitting the webhook urlcurl -sS -k http://domain.com/hook/circleci/success
![Page 44: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/44.jpg)
How about Slack notifications?import jsonimport urllibimport urllib2
def slack(message, color=None): attachment = { "text": message, } if color: attachment['color'] = color payload = { "channel": "#your-channel", "attachments": [attachment] } payload = urllib.pathname2url(json.dumps(payload)) payload = 'payload=' + payload request = urllib2.Request('https://slack.hook', payload) urllib2.urlopen(request)
... slack('Deploying...')... return ret
![Page 45: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/45.jpg)
And update the CircleCI config
deployment: release: branch: master commands: - docker push $REPO:$TAG - curl -sS -k https://domain.com/hook/circleci/success
![Page 46: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/46.jpg)
9
Questions?
![Page 47: Deploying Dockerized Applications with Salt](https://reader033.vdocument.in/reader033/viewer/2022052912/55a20aab1a28ab8b368b46b2/html5/thumbnails/47.jpg)
Thank you! :)