docker hk meetup - 201707

Docker Hong Kong Meetup (Jul 2017)

Introduction to Docker

Clarence Ho

Independent Software EngineerDocker HK Meetup Co-organizer@HoClarenceho.clarence@gmail.com

3

Topics

• Introduction to Docker• Latest Features of Docker• Docker Adoption• Docker Editions• Demo• Open Discussion

What is Docker?

Introduction to Docker

5

A brief explanation of Containers

An image is a lightweight, stand-alone, executable package that includes everything needed to run a piece of software• Contains the application executable and their dependencies• Built with instructions from a Dockerfile

A container is a runtime instance of an image – what the image becomes in memory when actually executed• Run apps natively on the host machine’s kernel• Running in a discrete process (isolated environment)• Containers on the same machine share a single kernel

6

Containers vs Virtual MachineVirtual Machine Diagram Container Diagram

7

Container vs VM - Performance Benchmark(Just for reference)

On a modest Intel server (16GB Ram)• 536 Linux Containers• 37 KVM Virtual Machines

Reference: https://insights.ubuntu.com/2015/06/11/how-many-containers-can-you-run-on-your-machine/

8

Virtualization

9

Containerization

10

Benefits of Containers

• More efficient in resource utilization− The same computing resources can run more containers than VMs− Containers organically consume the resources they need (bound by the

maximum value assigned). For VM, it will take up all the resources assigned when startup

• Better for cloud deployment (Microservices and Devops)− It’s a general practice to have separate images for difference components

for the same application (e.g. DB, App Server, Web Server)− More easy to deploy/upgrade/scale an individual component, without

impacting others

Latest Features of Docker

(Content based on Dockercon 2017)

12

Latest Features of Docker

• Versioning and Release Schedule• Builder• Runtime• Swarm Mode• Compose

Version and Release Schedule

Latest Features of Docker

14

New Versioning

15

New Release Schedule

Builder

Latest Features of Docker

17

Multi-Stage Builds

Traditional Dockerfile that includes build tools:

➜ Target is to reduce the size of Docker image

FROM alpine

RUN apk add make g++

ADD . /src

RUN cd /src && make

EXPOSE 80

ENTRYPOINT /usr/local/bin/app

18

Multi-Stage BuildsA Dockerfile that use multi-stage build:

➮ Final image will not include the build tools and libraries

FROM alpine AS build-env

RUN apk add make g++

ADD . /src

RUN cd /src && make

FROM busybox

COPY --from=build-env /src/build/app /usr/local/bin/app

EXPOSE 80

ENTRYPOINT /usr/local/bin/app

Runtime

Latest Features of Docker

20

Data Management Commands

• docker system df

➜ docker system sub-command added

$ docker system df

TYPE TOTAL ACTIVE SIZE RECLAIMABLE

Images 5 1 2.777 GB 2.647 GB (95%)

Containers 1 1 0 B 0B

Local Volumes 4 1 3.207 GB 2.261 GB (70%)

• docker system prune

• docker container/image/network/volume prune

Demo

22

Docker Playground

• Play with Docker− http://labs.play-with-docker.com

• Github− https://github.com/play-with-docker/play-with-docker

Swarm Mode

Introduction to Service Orchestration

24

Introduction to Service Orchestration

• Management− Need a manager to maintain the cluster state, and serve requests for

container management (schedule/stop/scale up/scale down)• Security

− All nodes within the cluster should be able to communicate securely• Service Discovery

− Need to be able to identify and locate a container service by using DNS• Load Balancing

− Need to be able to scale up/down containers with auto load balancing• Networking

− Able to segregate the network for different scenarios• Update/Rollback

− Support update and rollback of container services across the cluster

⌘ Container Services need Orchestration

25

Docker’s answer to Service OrchestrationDocker Swarm mode

26

Docker Swarm ModeSecurity - All managers and nodes communicates via TLS

27

Docker Swarm ModeLoad Balancing - Ingress Routing Mesh

28

Docker’s answer to Service OrchestrationLoad Balancing - External Load Balancer

29

Docker’s answer to Service OrchestrationLoad Balancing - Service to Service Communication

30

Introduction to Service Orchestration

• A DNS server was embedded in a Swarm cluster• Swarm mode has an internal DNS component that

automatically assigns each service in the swarm a DNS entry

• The swarm manager uses internal load balancing to distribute requests among services within the cluster based upon the DNS name of the service

Service Discovery with DNS

Swarm Mode

Latest Features of Docker

32

Service Rollback on Failure

“rollback” action added to --update-failure-action(in addition to “pause” and “continue”)

with all the associated flags

--rollback-delay--rollback-failure-action--rollback-max-failure-ratio--rollback-monitor--rollback-parallelism

swarm mode improvement

33

Topology Aware Scheduling

docker service create --replicas=6 postgresdocker service create --replicas=2 webapp

swarm mode improvement

34

Topology Aware Scheduling

docker service create --replicas=6 --placement-pref-add=rack postgresdocker service create --replicas=2 --placement-pref-add=rack webapp

swarm mode improvement

docker node update --label-add rack SFO-1 docker node update --label-add rack SFO-2

35

Service Logsswarm mode improvement

$ docker service create --replicas 2 --name redis redis$ docker service logs redisredis.2.najk8sq1klac@node2 | _.-``__ ''-._redis.2.najk8sq1klac@node2 | _.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 64 bitredis.1.lfkijq3fx3q8@node1 | _.-``__ ''-._redis.2.najk8sq1klac@node2 | .-`` .-```. ```\/ _.,_ ''-._redis.1.lfkijq3fx3q8@node1 | _.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 64 bitredis.2.najk8sq1klac@node2 | ( ' , .-` | `, ) Running in standalone moderedis.1.lfkijq3fx3q8@node1 | .-`` .-```. ```\/ _.,_ ''-._redis.2.najk8sq1klac@node2 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379redis.1.lfkijq3fx3q8@node1 | ( ' , .-` | `, ) Running in standalone moderedis.2.najk8sq1klac@node2 | | `-._ `._ / _.-' | PID: 1redis.1.lfkijq3fx3q8@node1 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379redis.2.najk8sq1klac@node2 | `-._ `-._ `-./ _.-' _.-'redis.1.lfkijq3fx3q8@node1 | | `-._ `._ / _.-' | PID: 1...

Swarm Mode - Secrets ManagementLatest Features of Docker

37

Securely Distributing Passwords

● Service often require sensitive information (like passwords, keys, etc.)

● Need a way to securely distribute such information across the cluster

38

Securely Distributing PasswordsThe Old Way

Pass as environment:$ docker service create -e password=TOTALLYSECURE dockercon

Password is stored on host and mount by container as volume:$ docker service create -v some/host/dir:/password dockercon

39

Securely Distributing PasswordsThe Old Way > Pass as environment > Problem

A developer need to debug the service, and the environment is dump into a debug log file.

40

Securely Distributing PasswordsThe Old Way > Save Secret in Volume > Problem

Volume must exist on every node that service needs to run on.

When service is rescheduled, secret stay on the host!

41

Docker SecretsSecrets are stored in the Raft Store

The Raft log is encrypted and secure

42

Docker SecretsSecrets are stored in the Raft Store

The encryption key of the Raft log can be further encrypted for added security

$ docker swarm update --autolock=true

43

Docker SecretsCreate a new secret

$ docker secret create my-password password.file

44

Docker SecretsUpon creation, secret shared across managers via the Raft Store

45

Docker SecretsUpdate service to use the secret

$ docker service update --secret-add=my-password Dockercon

46

Docker SecretsSecret only sent to nodes running the serviceStored in tmpfs mounted into the container

47

Docker SecretsNode failureService instance need to be rescheduled

48

Docker SecretsSecret moves with the serviceDead worker node does not have secret

49

Docker SecretsSecrets are new first-class objectsThe right way is also the easy way

Docker Compose

Latest Features of Docker

51

Compose to Swarm

It is now possible to deploy services using compose files directly from docker

➜ docker stack sub-command added

● docker stack deploy --compose-file docker-compose.yml <my_stack>

● docker stack list

● docker stack rm <my_stack>

52

Compose Format Version 3

Main differences from v2 are:

docker-compose.yml improvements

● Removed the non-portable options○ build○ volume-from○ …

● Added Swarm specific options

○ replicas

○ mode

○ ...

53

Long Syntax for Portsdocker-compose.yml improvement

ports:- 3000- 3000-3005- 49100:22- 9090-9091:8080-8081- 127.0.0.1:8001:8001- 127.0.0.1:5005-5010:5005-5010- 6060:7060/udp

Old Format (for port publishing):

54

Long Syntax for Portsdocker-compose.yml improvement

ports:- target: 6060

published: 7060protocol: udp

New Format (for port publishing):

55

Long Syntax for Volumesdocker-compose.yml improvement

volumes: - /var/lib/mysql - /opt/data:/var/lib/mysql - ./cache:/tmp/cache - datavolume:/var/lib/mysql - ~/configs:/etc/configs/:ro

Old Format (for volume mounting):

56

Long Syntax for Volumesdocker-compose.yml improvement

volumes: - type: bind source: ~/configs target: /etc/configs read_only: true

New Format (for volume mounting):

Docker Adoption

(Content based on Dockercon 2017)

58

What a Difference 3 Years Makes

Docker in Enterprise

Docker Adoption

60

Docker in in the Enterprise

Docker on Windows

Docker Adoption

62

Docker on Windows Server 2016● Now 98% of enterprise workloads supported by Docker● Proven benefits of Docker on Linux available to Windows Server

developers and IT Pros● One Docker platform and one adoption journey for all enterprise

applications and infrastructure● Docker CS Engine with Windows Server 2016 at no additional cost

63

Docker on Windows Server 2016

Docker EE is free and support by Microsoft directly

64

Windows and Hyper V Containers

65

Windows vs Linux Containers (Docker Store)

Oracle in Docker Store

Docker Adoption

67

Oracle on Docker Store

68

Oracle Database Enterprise Edition

Available as Docker imageFree for development and testing

Modernizing Traditional ApplicationsDocker Adoption

70

Legacy to Containerized AppThe proper way

71

I Want to Escape from VM ASAP, what to do?A faster way ⇨ Image2Docker

72

Sample Use Case2 applications (1 Linux, 1 Windows) running on VM

73

Sample Use Case2 applications (1 Linux, 1 Windows) running on VM

74

Sample Use Case2 applications (1 Linux, 1 Windows) running on VM

75

Sample Use Case

Steps:

76

Image2Docker - Linux

make preparemake buildmake builtin-prep

sudo bin/v2c-darwin64 build -n img.vmdk

https://github.com/docker/communitytools-image2docker-linux

77

Image2Docker - Windows

Install-Module Image2DockerImport-Module Image2Docker

ConvertTo-Dockerfile ` -ImagePath c:\iis.vhd ` -OutputPath c:\i2d2\iis ` -Artifact IIS

https://github.com/docker/communitytools-image2docker-win

78

Create a Hybrid Swarm

79

Deployment

Docker Editions

(Content based on Dockercon 2017)

Community and Enterprise EditionsDocker Editions

82

Enterprise and Community Editions

83

Docker Enterprise Edition (EE)CaaS enabled platform for the modern software supply chain

84

Docker Enterprise Edition (EE)Docker EE Components

85

Docker Enterprise Edition (EE)Docker EE Architecture

86

Docker Enterprise Edition (EE)

Docker EE Plans● Basic● Standard● Advanced

87

Docker Enterprise Edition (EE)Image - Promotion Branching

88

Docker Enterprise Edition (EE)Image - Scanning

89

Docker Enterprise Edition (EE)Image - Scanning Result (UCP)

90

Docker Enterprise Edition (EE)Mixed Windows/Linux Cluster

Docker for Various Platforms

Docker Editions

92

Docker CE and EESupported Platforms

93

Docker for various PlatformsExample : Docker for AWS

94

Docker for various PlatformsExample : Docker for Google Cloud (GCP)

Docker Cloud

Docker Editions

96

Docker Cloud• Manage Build and Images

− Provides a hosted registry service− Link to your source code repository

• Swarm Mode (Beta)− Provision swarms or register existing swarms to popular cloud providers− Support multiple providers in a single user interface− Use your Docker ID to authenticate and securely access personal or team

swarms• Standard Mode

− Link to your hosts, upgrade the Docker Cloud agent, and manage container distribution

− Deploy and manage nodes, services, and applications in Docker Cloud• Pricing

− Contact Docker

97

Docker CloudDocker Cloud provisions Docker CE Editions

98

Docker CloudProvision Swarms for multiple cloud providers

99

Docker CloudSwarm management

100

Docker Cloud vs Enterprise EditionFeature Docker EE Docker Cloud

Docker Engine Version Docker EE Docker CE, Docker EE (Basic)

Private Image Registry Your own registry Host by Docker

User Interface Docker UCP(Universal Control Plane)

Docker Cloud UI

Image Security Scan Support Support

User Security Create your own user/group,Role based access control

Docker ID

Docker Datacenter Included (Standard, Advance) Not included

Automated Development Pipelines Included Not included

Private Cloud Full Support Partially Support (Bring your own Swarm)

Pricing Visit Docker site Contact Docker

✦ Contact Docker for latest information

Service Orchestration (Alternatives)Docker Editions

102

Container Service Orchestration PlatformAlternatives

• Public Cloud Providers− Amazon EC2 Container Service− Google Container Engine (based on Kubernetes)

• Redhat Openshift− Redhat Enterprise Linux, Docker, Kubernetes

• CoreOS− Container Linux, Quay Container Registry, Tectonic Kubernetes

• Apache Mesos− DC/OS (Datacenter Operating System)

• IBM, HPE, Oracle, etc.

Demo

104

Docker Playground

• Play with Docker− http://labs.play-with-docker.com

• Github− https://github.com/play-with-docker/play-with-docker

105

Sample Application

• Github− https://github.com/clarenceh/docker-contact

Final Words

107

Let’s Keep the Meetup Running

• Let’s work together to keep the meetup active• Speakers WANTED• Share with each other about your Docker journey• Reach out for venues for deep dive

− Workshops− The best way to learn is to do some real stuff

• Containerize your application• Setup a Docker Swarm cluster• Use Docker Compose to deploy your stack

Hey, I need HELP!!!

Open Discussions

THANK YOU