introduction to docker at the azure meet-up in new york
DESCRIPTION
This is the presentation given at the Azure New York Meet-Up group, September 3rd. It includes a quick overview of the Open Source Docker Engine and its associated services delivered through the Docker Hub. It also covers the new features of Docker 1.0, and briefly explains how to get started with Docker on Azure.TRANSCRIPT
Docker
September 2014—Docker 1.2
@jpetazzo
● Wrote dotCloud PAAS deployment tools
– EC2, LXC, Puppet, Python, Shell, ØMQ...● Docker contributor
– Security, Networking...● Runs all kinds of crazy things in Docker
– Docker-in-Docker, VPN-in-Docker,KVM-in-Docker, Xorg-in-Docker...
Let's start with Questions
Raise your hand if you have ...
● Tried Docker (online tutorial)
Raise your hand if you have ...
● Tried Docker (online tutorial)● Tried the real Docker (e.g. deployed remote VM)
Raise your hand if you have ...
● Tried Docker (online tutorial)● Tried the real Docker (e.g. deployed remote VM)● Installed Docker locally (e.g. with boot2docker)
Raise your hand if you have ...
● Tried Docker (online tutorial)● Tried the real Docker (e.g. deployed remote VM)● Installed Docker locally (e.g. with boot2docker)● Written a Dockerfile (and built it!)
Raise your hand if you have ...
● Tried Docker (online tutorial)● Tried the real Docker (e.g. deployed remote VM)● Installed Docker locally (e.g. with boot2docker)● Written a Dockerfile (and built it!)● An image on Docker Hub (pushed or autobuilt)
Raise your hand if you have ...
● Tried Docker (online tutorial)● Tried the real Docker (e.g. deployed remote VM)● Installed Docker locally (e.g. with boot2docker)● Written a Dockerfile (and built it!)● An image on Docker Hub (pushed or autobuilt)● Deployed Docker images for dev/QA/test/prod...
Agenda
● What is Docker and Why it matters● What are containers● The Docker ecosystem (Engine, Hub, etc.)● Deployment options and first steps● What's new in the latest Docker?
Whatis Docker
Whyit matters
Deploy everything
● Webapps● Backends● SQL, NoSQL● Big data● Message queues● … and more
Deploy almost everywhere
● Linux servers● VMs or bare metal● Any distro● Kernel 3.8+ (or RHEL 2.6.32)
Currently: focus on x86_64.
(But people reported success on arm.)
Deploy reliably & consistently
Deploy reliably & consistently
● If it works locally, it will work on the server● With exactly the same behavior● Regardless of versions● Regardless of distros● Regardless of dependencies
Deploy efficiently
● Containers are lightweight– Typical laptop runs 10-100 containers easily
– Typical server can run 100-1000 containers
● Containers can run at native speeds– Lies, damn lies, and other benchmarks:
http://qiita.com/syoyo/items/bea48de8d7c6d8c73435http://www.slideshare.net/BodenRussell/kvm-and-docker-lxc-benchmarking-with-openstack
Infiniband throughput and latency:no difference at all
Booting 15 OpenStack VMs:KVM vs Docker
Memory speed:Bare Metal vs Docker vs KVM
OK, but what is
Docker?
Docker Engine+ Docker Hub
= Docker Platform
The Docker Engine runs containers.
OK, but what is a
container?
High level approach:it's a lightweight VM
● Own process space● Own network interface● Can run stuff as root● Can have its own /sbin/init
(different from the host)
« Machine Container »
Low level approach:it's chroot on steroids
● Can also not have its own /sbin/init● Container = isolated process(es)● Share kernel with host● No device emulation (neither HVM nor PV)
« Application Container »
Stop.Demo time.
Alright, I get this.Containers = nimble Vms.
Let's just tell the CFO,and get back to work!
What happens when something becomes
10-100x cheaper?
Random example:testing
● Project X has 100 unit tests● Each test needs a pristine SQL database
Random example:testing
● Project X has 100 unit tests● Each test needs a pristine SQL database
● Plan A: spin up 1 database, clean after each use– If we don't clean correctly, random tests will fail
– Cleaning correctly can be expensive (e.g. reload DB)
Random example:testing
● Project X has 100 unit tests● Each test needs a pristine SQL database
● Plan B: spin up 100 databases– … in parallel: needs too much resources
– … one after the other: takes too long
Random example:testing
● Project X has 100 unit tests● Each test needs a pristine SQL database
● Plan C: spin up 100 databases in containers– fast, efficient (no overhead, copy-on-write)
– easy to implement without virtualization black belt
Containers make testing(and many other things)
way easier
The container metaphor
Problem: shipping goods
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
Solution:the intermodal shipping container
Solved!
Problem: shipping code
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
Solution:the Linux container
Solved!
Separation of concerns:Dave the Developer
● Inside my container:– my code
– my libraries
– my package manager
– my app
– my data
Separation of concerns:Oscar the Ops guy
● Outside the container:– logging
– remote access
– network configuration
– monitoring
Docker's Entourage
Docker: the cast
● Docker Engine● Docker Hub● Docker, the community● Docker Inc, the company
Docker Engine
● Open Source engine to commoditize LXC● Uses copy-on-write for quick provisioning● Written in Go, runs as a daemon, comes with a CLI● Everything exposed through a REST API● Allows to build images in standard, reproducible way● Allows to share images through registries● Defines standard format for containers
(stack of layers; 1 layer = tarball+metadata)
… Open Source?
● Nothing up the sleeve, everything on the table– Public GitHub repository: https://github.com/dotcloud/docker
– Bug reports: GitHub issue tracker
– Mailing lists: docker-user, docker-dev (Google groups)
– IRC channels: #docker, #docker-dev (Freenode)
– New features: GitHub pull requests (see CONTRIBUTING.md)
– Docker Governance Advisory Board (elected by contributors)
Docker Hub
Collection of services to make Docker more useful.● Public registry
(push/pull your images for free)● Private registry
(push/pull secret images for $)● Automated builds
(link github/bitbucket repo; trigger build on commit)● More to come!
Docker, the community
● >500 contributors● ~20 core maintainers● >8,000 Dockerized projects on GitHub● >30,000 repositories on Docker Hub● >250 meetups in >90 cities in >30 countries● >750,000 downloads of boot2docker
Docker Inc, the company
● Headcount: ~50● Led by Open Source veteran Ben Golub
(GlusterFS)● Revenue:
– t-shirts and stickers featuring the cool blue whale
– SAAS delivered through Docker Hub
– Support & Training
Using Docker
One-time setup
● On your dev env (Linux, OS X, Windows)– boot2docker (25 MB VM image)
– Natively (if you run Linux)
● On your servers (Linux)– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)
– Single binary install (Golang FTW!)
– Easy provisioning on Azure, Rackspace, Digital Ocean...
– Special distros: CoreOS, Project Atomic
Azure deployment
VMNAME=jpetazzoIMAGE=b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04-LTS-amd64-server-20140724-en-us-30GBUSER=jpetazzoPASSWORD=1234abcdABCD@LOCATION="West US"
azure vm docker create $VMNAME \ $IMAGE $USER $PASSWORD -l "$LOCATION" export DOCKER_HOST=tcp://$VMNAME.cloudapp.net:4243docker --tls versionazure vm endpoint create $VMNAME 80
Authoring imageswith a Dockerfile
FROM ubuntu:14.04
RUN apt-get updateRUN apt-get install -y nginxRUN echo 'Hi, I am in your container!' \ >/usr/share/nginx/html/index.html
CMD nginx -g "daemon off;"
EXPOSE 80
docker build -t jpetazzo/staticweb .docker run -P jpetazzo/staticweb
FROM ubuntu:12.04
RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz \ | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" \ > /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .
FROM debian:jessie
RUN apt-get -y updateRUN apt-get install -y python-pip
RUN mkdir /srcWORKDIR /src
ADD requirements.txt /srcRUN pip install -r requirements.txt
ADD . /srcRUN python setup.py install
Do you evenChef?
Puppet?Ansible?
Salt?
Summary
With Docker, I can:● put my software in containers● run those containers anywhere● write recipes to automatically build containers
Advanced concepts
● naming– give a unique name to your containers
● links– connect containers together
● volumes– separate code and data
– share data between containers
Recent features: 0.10
● TLS support for API access● Configurable DNS search● BTRFS is no longer experimental● Integration with systemd cgroups● Use proxy environment variables (for registry)
Recent features: 0.11
● SELinux integration(works better with CentOS)
● DNS integration for links(access linked containers by hostname)
● « docker run --net »– use host networking for high speed
– share network of another container
Recent features: 0.12
● docker pause/unpause● more importantly: 1.0 release candidate :-)
Docker 1.1
● .dockerignore(don't upload your .git anymore!)
● docker logs --tail– further logging improvements on the way
(truncate)
Docker 1.2
● New cool options for docker run
--restart=always/no/on-failure
--cap-add=NETADMIN
--cap-drop=CHOWN
--device=/dev/kvm:/dev/kvm
Coming soon(maybe)
● logging improvements● device mapper tuning● image squashing● ARM support● use secrets in builds
● volume management● hairpin nat● IPV6 support● seccomp + native● user namespaces
Thank you! Questions?
http://docker.com/
@docker
@jpetazzo