docker concepts for oracle/mysql dbas and devops

Docker Concepts for Oracle/MySQL DBAs and DevOps

Zohar Elkayam CTO, Brillix

Zohar@Brillix.co.ilwww.realdbamagic.com

Twitter: @realmgic

• Zohar Elkayam, CTO at Brillix

• Programmer, DBA, team leader, database trainer, public speaker, and a senior consultant for over 19 years

• Oracle ACE Since 2014

• ilOUG Board Member – Israel Oracle User Group

• Blogger – www.realdbamagic.com and www.ilDBA.co.il

Who am I?

About Brillix

• We offer complete, integrated end-to-end solutions based on best-of-breed innovations in database, security and big data technologies

• We provide complete end-to-end 24x7 expert remote database services

• We offer professional customized on-site trainings, delivered by our top-notch world recognized instructors

Some of Our Customers

• Founded in 1999, over 7000 members at its peak

• Divided into Technologies (Databases) and Applications groups

• Free Bi-monthly meetups for DBAs and Developers at Oracle Israel offices

• Yearly conference with guests speakers from all over the world:• January 22-23, 2018• Over 30 Speakers from all over the world: Ask TOM team, Oracle product mangers

and Oracle ACEs from 9 different countries!

• You can join our community: meetup group, facebook group or follow us on twitter (@ilOUG_org)!

• Our website: www.ilOUG.org

ilOUG – Israel Oracle User Group

Agenda

• Docker overview – why do we even need containers?

• Installing Docker and getting started

• Images and Containers

• Docker Networks

• Docker Storage and Volumes

• Oracle and Docker

• Docker tools, GUI and Swarm

Our Goal Today

• Learn what Docker is and how to use it

• See how Docker containers might be helpful with Oracle

• We will not expert everything

• Did you hear about Oracle 18c?

• This is a starting point – don’t be afraid to try!

.בסיום יום הסמינר יחולק טופס משוב ונשמח לקבל את חוות דעתכם•

!היא זו המאפשרת לנו לעשות את הסמינרים טובים יותר:דעתכם חשובה•

Our REAL Agenda

הפסקה10:30-10:45

י הכנס בגן המלוןארוחת צהריים לכל משתתפ12:30-13:30

הפסקה מתוקה במתחם קבלת הפנים15:00-15:15

הולכים הביתה16:30

Docker

What is Docker?

• Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating system–level virtualization on Linux

What’s the Buzz Around Containers?

• The software industry has changed dramatically

• 5-10 years ago:• Monolithic applications• Long development cycles• Slowly scaling up

• Now:• Decoupled services, Microservices• Fast, iterative improvements• Quickly scaling out

• Google claims all of its services now run in containers

It’s All About CI/CD

• CI/CD: Continuous Integration and Continuous Deployment

• We want to always integrate new things into our applications and have a fast deployment

• The challenge: development and deployment became complex: multiple stacks, multiple targets. Keeping everything in sync and deploying a reliable environment in short spans to production is hard

• Docker Goal: Provide a uniformed wrapper around a software package: «Build, Ship and Run Any App, Anywhere»

• Using Docker to achieve CI/CD is the most common usage for Docker today

The Analogy: Cargo Transport

The Analogy: Cargo Transport Solution

Docker Solution: Containers

Docker Containers

• The concept: One Application per Container

• Self contained - everything you need is in an isolated environment

• Isolation for security - containers not aware of each other

• Isolation for performance – hard limits on CPU, RAM and IO per container

VMs vs. Containers

• Less overhead but It’s not a lightweight VM – it’s a Software Delivery Platform

• It's about isolation - Containers run on Linux kernel of host and are visible on host

Why Use Docker Containers

• It’s highly portable

• It Runs everywhere: bare metal, VM, and even the cloud (AWS, GCP, Oracle Cloud, Microsoft Azure)

• Small, lightweight and highly scalable

• Better performance than VMs, smaller foot print

• Very cost effective

Why Does IT Cares About Docker?

Developers care because:

• Quickly create ready-to-run packaged applications

• A clean, safe, hygienic, portable runtime environment

• No missing/conflicting dependencies or packages

• Each app runs in an isolated container

• Automate testing, integration, packaging

• Reduce/eliminate platform compatibility issues

• Cheap/zero cost deployment, with instant replay and reset

Why Does IT Cares About Docker? (2)

Administrators care because:

• Configure once, run many times

• Makes app lifecycle efficient, consistent and repeatable

• Eliminate environment inconsistencies between development, test, production

• Supports segregation of duties

• Improve speed and reliability of continuous integration and deployment

• Lightweight containers address performance, costs, deployment and portability issues

Common Use Cases

• Development and Test Environment

• Micro-services

• Application Deployment without “dependency hell” of multiple applications

• It Solves the “It Worked For Me!” issue

• Continuous Integration

• Platform-as-a-Service (PaaS)

Docker Inc.

• Founded in 2009

• Primary sponsor of the Docker Project• Hires maintainers and contributors• Provides infrastructure for the project• Runs the Docker Hub and Store

• How does Docker Inc. make money?• Docker Hub has free and paid services• DTR (Docker Trusted Registry)• Enterprise support for Engine and other products• Training and professional services

Docker Adoption Graph

Docker Adoption

Installing Docker

Let’s Begin: Installing Docker

• Docker is easy to install• Runs on practically every Linux distribution (Kernel must be 2.6.32+ or

3.10+)• Runs on OS X via a virtual machine• Microsoft Windows (10) via virtual machine

• Community Edition (CE) and Enterprise Edition are available• EE has support, and some certifications to hardware/software• CE has the community support and is way more common

• Latest version is 17.09 (released on September, 2017), I still use 17.06.2…

Docker on Windows 10

• Make sure your virtualization is on:

Docker on Windows

• Download the Docker Toolbox from https://www.docker.com/products/docker-toolbox

• That will install:• Docker Client for Windows

• Docker Toolbox management tool and ISO

• Oracle VM VirtualBox

• Git MSYS-git UNIX tools

• Click the “QuickStart Terminal” on your desktop and enable the User Account Control permission

Installing Docker: Oracle Linux 7

• Unlike other distros, Oracle Linux doesn’t enable the yum repository holding Docker by default, so we need to first enable it:

• Use yum to install Docker engine:

[root@lnx7-oracle-1 ~]# vi /etc/yum.repos.d/public-yum-ol7.repo

[ol7_addons]

name=Oracle Linux $releasever Add ons ($basearch)

baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle

gpgcheck=1

enabled=1

[root@lnx7-oracle-1 ~]# yum install docker-engine

• Create a btrfs mount point for the docker file system

• This filesystem will provide better performance and copy-on-write support

• More on copy-on-write and btrfs - later

mkfs.btrfs -L btrfs1 /dev/sdb

echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab

mkdir -p /var/lib/docker

mount /var/lib/docker

• Start the Docker daemon and enable it to start automatically:[root@lnx7-oracle-1 ~]# systemctl start docker

[root@lnx7-oracle-1 ~]# systemctl enable docker

Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service

to /usr/lib/systemd/system/docker.service.

[root@lnx7-oracle-1 ~]# systemctl status docker

● docker.service - Docker Application Container Engine

Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor

preset: disabled)

Drop-In: /etc/systemd/system/docker.service.d

└─docker-sysconfig.conf

Active: active (running) since Thu 2017-11-16 21:30:31 IST; 13min ago

Docs: https://docs.docker.com

Main PID: 881 (dockerd)

Memory: 18.6M

CGroup: /system.slice/docker.service

├─881 /usr/bin/dockerd --selinux-enabled

Check Docker Version

• After install, verify the docker version and docker info[root@lnx7-oracle-1 ~]# docker version

Client:

Version: 17.06.2-ol

API version: 1.30

Go version: go1.8.3

Git commit: d02b7ab

Built: Fri Oct 6 00:02:23 2017

OS/Arch: linux/amd64

Server:

Version: 17.06.2-ol

API version: 1.30 (minimum version 1.12)

Go version: go1.8.3

Git commit: d02b7ab

Built: Fri Oct 6 00:03:48 2017

OS/Arch: linux/amd64

Experimental: false

The docker Group

• Add a docker group

• Add a user to the docker group

• Restart the docker deamon

• Relogin to the use the user, without using sudo to run things from root

$ sudo groupadd docker

$ sudo gpasswd -a $USER docker

$ sudo systemctl restart docker.service

Docker and Security

• The docker user is root equivalent

• It provides root-level access to the host

• You should restrict access to it like you would protect root

• If you give somebody the ability to access the Docker API, you are giving them full access on the machine

• Therefore, the Docker control socket is (by default) owned by the docker group, to avoid unauthorized access on multi-user machines

Getting Started:Working With Docker

Docker Components

Docker Architecture

• The Docker daemon (or "Engine"): receives and processes incoming Docker API requests

• The Docker client talks to the Docker daemon via the Docker API• We'll use mostly the CLI embedded within the docker binary

• GUI tools available, we will talk about them later

• Docker Hub Registry (and Docker Store): Collection of public images• The Docker daemon talks to it via the registry API

Hello World: Busybox

• Let’s run a container that echo “hello world”

• This started a very small (>1 MB) container called Busybox, and ran the single command we provided

• Busybox is usually used for embedded system (routers, switches, etc.)

[zohar@lnx7-oracle-1 ~]$ docker run --rm busybox echo hello world

Unable to find image 'busybox:latest' locally

latest: Pulling from library/busybox

0ffadd58f2a6: Pull complete

Digest: sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0

Status: Downloaded newer image for busybox:latest

hello world

Using Shell to Busybox

• Let’s connect to that container and have a look around

• We used -it option to indicate “interactive” and “tty” (terminal)

[zohar@lnx7-oracle-1 ~]$ docker run --rm -it busybox sh

/ # ps -ef

PID USER TIME COMMAND

1 root 0:00 sh

5 root 0:00 ps -ef

/ # uname -a

Linux a451342a146a 4.1.12-103.9.2.el7uek.x86_64 #2 SMP Tue Oct 31 16:43:46 PDT

2017 x86_64 GNU/Linux

Which Containers are Up?

• We can check which containers are up using docker ps

• We can check which container was the last to start by using docker ps -l

Detaching an Interactive Container

• We can detach the interactive container by using CTRL+P,Q

• When checking on the host what containers are up, we can see it’s running in the background

• We can re-attach back to it by using docker attach command

• We can use the container name or ID in order to re-attach

[zohar@lnx7-oracle-1 ~]$ docker attach vibrant_borg

Running Ubuntu in Container

• Let’s try a more useful container: Ubuntu Linux

• Here we can actually install things on it (it’s connected to the internet!)

[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu

root@502f2202f0bc:/#

root@502f2202f0bc:/# apt-get update

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]

Get:3 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [52.0

Get:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]

Working in the Container

• Let’s have some fun: let’s install figlet

• This only install the program on our current container, it does not change our image or our host system

root@502f2202f0bc:/# apt-get install figlet

Reading package lists... Done

Building dependency tree

Reading state information... Done

The following NEW packages will be installed:

figlet

root@502f2202f0bc:/# figlet hello

| |__ ___| | | ___

| '_ \ / _ \ | |/ _ \

| | | | __/ | | (_) |

|_| |_|\___|_|_|\___/

Running Containers in the Background

• We can start containers and run them in the background using the run -d flag – but what happens to their console output?

• Console output is being logged – we can attach to the full output by using the docker logs command

[zohar@lnx7-oracle-1 ~]$ docker run -d jpetazzo/clock

70e4be1a0c7ece5831632d169f21974ef9945234c6f48223a24ac92f8e73517f

[zohar@lnx7-oracle-1 ~]$ docker ps -l

CONTAINER ID IMAGE COMMAND CREATED

STATUS PORTS NAMES

70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 12 seconds ago

Up 10 seconds tender_murdock

[zohar@lnx7-oracle-1 ~]$ docker logs 70e

Fri Nov 17 11:23:50 UTC 2017

Fri Nov 17 11:23:51 UTC 2017

Fri Nov 17 11:23:52 UTC 2017

Fri Nov 17 11:23:53 UTC 2017

View Tail of the Log

• To view only the tail of the log, we use docker logs --tailcommand (with or without –f)

[zohar@lnx7-oracle-1 ~]$ docker logs 70e --tail 3

Fri Nov 17 11:27:30 UTC 2017

Fri Nov 17 11:27:31 UTC 2017

Fri Nov 17 11:27:32 UTC 2017

Stopping the Container

• Stopping our container gracefully:

• We can also use the kill command which will not wait for the container to stop gracefully for 10 seconds

• When using interactive mode, CTRL+C will send SIGINIT and close the container

[zohar@lnx7-oracle-1 ~]$ docker ps

STATUS PORTS NAMES

502f2202f0bc ubuntu "/bin/bash" 5 minutes ago

Up 5 minutes tender_keller

[zohar@lnx7-oracle-1 ~]$ docker stop 502

[zohar@lnx7-oracle-1 ~]$

One-Off Containers

• Using the --rm flag when running the container, will cause the container to be removed completely when it is taken down

• This is useful for “one-off” containers that will do an action and then disappear (like the “hello world” we did before)

• We can also use the docker rm command to delete unused containers

Showing Stopped Containers

• Stopped containers still exist – they are not automatically removed

• We can list all containers (running and stopped) by using docker ps –a

[zohar@lnx7-oracle-1 ~]$ docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS

70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 16 minutes ago Exited (137) 1 second ago

502f2202f0bc ubuntu "/bin/bash" 2 hours ago Exited (0) 16 minutes ago

262ebcab83c8 ubuntu "/bin/bash" 2 hours ago Exited (0) 2 hours ago

Restarting and Attaching to Our Container

• If we restart our container, the figlet command will be available (but we still can’t duplicate our container)

• If we start another container (using run), the figlet will not be available because it’s not in our base image

[zohar@lnx7-oracle-1 ~]$ docker start 502

[zohar@lnx7-oracle-1 ~]$ docker attach 502

root@502f2202f0bc:/# figlet hello again

_ _ _ _

| |__ ___| | | ___ __ _ __ _ __ _(_)_ __

| '_ \ / _ \ | |/ _ \ / _` |/ _` |/ _` | | '_ \

| | | | __/ | | (_) | | (_| | (_| | (_| | | | | |

|_| |_|\___|_|_|\___/ \__,_|\__, |\__,_|_|_| |_|

docker exec Command

• In some cases, attaching to a container will not allow us to interact with it (we’ll talk about CMD and ENTRYPOINT later)

• In those cases, we will want to execute a command in the container to interact with it:

• We can run whatever command we want, but bash is one of the more common…

[zohar@lnx7-oracle-1 ~]$ docker exec -ti plugpdb bash

Summary for the Basic Commands

• docker ps [-a] [-l] [-q]

• docker run [-it|-d] [--rm]

• docker attach

• docker start

• docker stop

• docker kill

• docker rm (docker rmi)

Under the Hood: Images and Layers

Images and Layers

• An image is a collection of files + some meta data (technically: those files form the root filesystem of a container)

• Images are made of layers, conceptually stacked on top of each other

• Each layer can add, change, and remove files

• Images can share layers to optimize diskusage, transfer times, and memory use

• Layer default location: /var/lib/docker

[Source: https://docs.docker.com/terms/layer/]

Differences Between Containers and Images

• An image is a read-only filesystem - we can’t change them after they are created

• A container is an encapsulated set of processes running in a read-write copy of that filesystem

• Images are like templates/stencils we create Containers from

• To optimize container boot time, copy-on-write is used instead of regular copy

• The command docker run starts a container from a given image

Copy-On-Write

• Copy-on-write, sometimes referred to as implicit sharing or shadowing is a resource-management technique to efficiently implement a "duplicate" or "copy" operation on modifiable resources

• If a resource is duplicated but not modified, it is not necessary to create a new resource

• Modifications must still create a copy, but the copy operation is deferred to the first write

• By sharing resources in this way, it is possible to significantly reduce the resource consumption of unmodified copies, while adding a small overhead to resource-modifying operations

UnionFS and BTRFS

• AUFS is a union filesystem. The aufs storage driver was previously the default storage driver used for managing images and layers on Docker

• Btrfs is a next generation copy-on-write filesystem that supports many advanced storage technologies that make it a good fit for Docker

• Btrfs is included in the mainline Linux kernel

• Docker’s btrfs storage driver leverages many btrfs features for image and container management: block-level operations, thin provisioning, copy-on-write snapshots, and ease of administration

Where Do We Get a Base Image?

• Build our own from “Scratch”

• Docker Hub and Docker Hub Store• Some images might require approve in the store before pulling it

• Get recipes off GitHub or BitBucket

• Remember: Images are read-only, we can’t change them after they are created but we can evolve them

Wait! Images Are Read-Only?!

• Question: If images are read-only, how do we change them?

• Answer: We don’t!• We create a new container from that image

• We make changes to that container

• When we are satisfied with those changes, we evolve them into a new layer

• A new image is created by stacking the new layer on top of the old image

The Chicken and Egg Problem

Creating the First Image

• There is a special empty image called scratch which means the first image is built from “scratch”

• The docker import command loads a tarball into Docker• The imported tarball becomes a standalone image

• That new image has a single layer

• We probably don’t need this – we will use more “common” base images

The Docker Hub Repository

• Docker Hub is a public repository of pre-built images

• We can upload (push) images, or download (pull) other people/organizations images to base our containers/layers on

• If we don’t want to use the public Hub, we can create our own local repositories (and namespaces)

Downloading an Image

• Explicitly: we can download an image without initiating a container using the docker pull command

• Download the exact image only

• Image kept on the Docker host ready to use

• Implicitly: when executing docker run command, it will download all the images needed that are not locally stored

• Download the entire dependency layers

• Only being done once, on the first run of that image

Pulling Image Example

• Using the run command, we tried to start a container from the Ubuntu image. Some layers did not exist so they were automatically pulled and downloaded:

[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu bash

Unable to find image 'ubuntu:latest' locally

latest: Pulling from library/ubuntu

ae79f2514705: Downloading [===================> ]

18.19MB/47.62MB

c59d01a7e4ca: Download complete

41ba73a9054d: Download complete

f1bbfd495cc1: Download complete

0c346f7223e2: Download complete

Listing The Local Docker Images

• List all the local images by using the docker images command[zohar@lnx7-oracle-1 ~]$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

figlet latest 564ca4364896 5 days ago 162MB

mysql latest 5709795eeffa 12 days ago 408MB

ubuntu latest dd6f76d9cc90 13 days ago 122MB

busybox latest 6ad733544a63 13 days ago 1.13MB

jpetazzo/namer master f1557ed35146 2 years ago 723MB

jpetazzo/clock latest 12068b93616f 2 years ago 2.43MB

But How Do We Find Images?

• Finding an image in the repository is easy :• Find it online using hub.docker.com or store.docker.com

• Use the shell interface:$ docker search mysql

Finding an Image: Shell

• Name is the name and namespace of the image we are searching for• Root-like (i.e. mysql or mariadb)• User/organizaion (i.e. zabbix or realmgic)• Self hosted (e.g. registry.example.com:5000/my-private-image)

• Description is an optional description for your images

Namespaces

• Root: official images. They are put there by Docker Inc., but they are generally authored and maintained by third parties

• Small, "swiss-army-knife" images like busybox• Distro images to be used as bases for your builds, like ubuntu, fedora• Ready-to-use components and services, like redis, postgresql, mysql

• User: The user namespace holds images for Docker Hub users and organizations

• Self-Hosted: Images which are not hosted on Docker Hub, but on third party registries. They contain the hostname (or IP address), and optionally the port, of the registry server.

Finding an Image (cont.)

• Stars indicate how poplar an image is

• Official indicate it is part of the root namespace

• Automated images are built automatically by the Docker Hub using submitted Dockerfiles in GitHub - this means that their build recipe is always available

A Word of Warning!

• Images on the Hub are not being verified and might be harmful

• Use trusted images from the store to make sure legitimate source

• You can also use images with a known Dockerfile – for example, Automated Docker Images where you can verify their build process

Image Tags

• Images can have tags to define image versions or variants

• docker pull ubuntu will refer to ubuntu:latest

• The :latest tag is generally updated often

• We only specify tags when:• We want to ensure that the same version will be used everywhere

• Recording a procedure into a script

• Going to production

• Ensure repeatability later

Saving a New Image

• docker commit• Saves all the changes made to a container into a new layer

• Creates a new image (effectively a copy of the container)

• Be careful, it saves everything you did to the container!

• docker build• Performs a repeatable build sequence in a Dockerfile (recipe)

• This is the preferred method!

Pushing an Image to the Hub

• Once the image is ready, pushing it to the hub is simple - using the docker push command

• Make sure your image is tagged before uploading[zohar@lnx7-oracle-1 ~]$ export DOCKER_ID_USER=“realmgic“

[zohar@lnx7-oracle-1 ~]$ docker login

Login with your Docker ID to push and pull images from Docker Hub. If you don't

have a Docker ID, head over to https://hub.docker.com to create one.

Username (realmgic):

Password:

Login Succeeded

[zohar@lnx7-oracle-1 ~]$ docker push $DOCKER_ID_USER/my_image

Building a New Image Example

• Let’s use the example from earlier and build an image that has the figlet tool in it

• Our base image will be the Ubuntu Image

• The base package might not be updated, so let’s update its Linux packages and install figlet

root@502f2202f0bc:/#

root@502f2202f0bc:/# apt-get update && apt-get install figlet

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]

Check Container Differences

• We can check what changed between this container and the base image using the docker diff command

• Since Docker uses copy-on-write, these are the only files kept for this container on disk. Multiple containers share the base image files (due to UnionFS/btrfs for example)

[zohar@lnx7-oracle-1 ~]$ docker diff 502

C /root

A /root/.bash_history

C /etc

C /etc/alternatives

A /etc/alternatives/figlet

A /etc/alternatives/figlet.6.gz

Commit The Changes

• Create a new image from the newly built container using the docker commit command

• The new image will look like this since we didn’t tag it with a name:

• Tag the new image:

[zohar@lnx7-oracle-1 ~]$ docker commit 502

sha256:8cc5680434bb94b272e59624cd44667eaab191a55a08c38a90c472f9f9b45b3f

[zohar@lnx7-oracle-1 ~]$ docker images

<none> <none> 8cc5680434bb About a minute ago 162MB

[zohar@lnx7-oracle-1 ~]$ docker tag 8cc5680434bb figlet

[zohar@lnx7-oracle-1 ~]$ docker images

figlet latest 8cc5680434bb 5 minutes ago 162MB

Manual Bad, Automatic Good

Automated Docker Images: Dockerfile

• Dockerfile is a build recipe to customize the base (r/o) image and make it into a new image

Base Docker Image

Instruction

• Run commands• Add files/directories• Define ENV variables• Define processes to run• Set Entrypoint

New Docker Image

Automated Docker Images on Hub

• Docker Hub can automatically build images based on Dockerfilesplaced on Github

• These Images can be built automatically by the Hub or we can download the recipe and build it ourselves

• Advantages:• Images built this way are built exactly as specified

• The Dockerfile is available to anyone with access to your Docker Hub repository

• Your repository is kept up-to-date with code changes automatically

Creating a Dockerfile

• Our new Dockerfile must be in an empty directory

• In the Dockerfile, we specify the base Image we start from using the FROM command

• The RUN commands are never interactive, so we’ll need to sort it out when we build the script

[zohar@lnx7-oracle-1 ~]$ mkdir figlet

[zohar@lnx7-oracle-1 ~]$ cd figlet/

[zohar@lnx7-oracle-1 figlet]$ vi Dockerfile

FROM ubuntu

RUN apt-get update -y

RUN apt-get install –y figlet

Build The Image

• Create a new image from the Dockerfile we createdusing the docker build command

[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .

Sending build context to Docker daemon 2.048kB

Step 1/3 : FROM ubuntu

---> dd6f76d9cc90

Step 2/3 : RUN apt-get update -y

---> Running in 174cec778baa

---> 035ad14c7312

Removing intermediate container 174cec778baa

Step 3/3 : RUN apt-get install figlet -y

---> Running in e9d2c2870c5d

---> e6e0b459c15c

Removing intermediate container e9d2c2870c5d

Successfully built e6e0b459c15c

Successfully tagged figlet_auto:latest

Reading the Output: Step 1

• Our base image is Ubuntu – so we start a new container for it (dd6f76d9cc90) and make it our base image

• The -t indicates the tag for the name of our new image

[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .

Sending build context to Docker daemon 2.048kB

Step 1/3 : FROM ubuntu

---> dd6f76d9cc90

• We start a new container from our base image (174cec778baa) and modify it using our RUN command

• At the end of this step, we’ll have a new base image: 035ad14c7312

• The intermediate base image will be removed

Step 2/3 : RUN apt-get update -y

---> Running in 174cec778baa

---> 035ad14c7312

Removing intermediate container 174cec778baa

• We repeat the process for step 3

• The final container becomes our base image (e6e0b459c15c) and it is tagged to the name we chose (figlet_auto:latest)

Step 3/3 : RUN apt-get install figlet -y

---> Running in e9d2c2870c5d

---> e6e0b459c15c

Removing intermediate container e9d2c2870c5d

Successfully built e6e0b459c15c

Successfully tagged figlet_auto:latest

Image History

• The history command lists all the layers composing an image

• For each layer, it shows its creation time, size, and creation command

• When using Dockerfile, each line indicates a single command

[zohar@lnx7-oracle-1 figlet]$ docker history figlet_auto

IMAGE CREATED CREATED BY SIZE

COMMENT

4cbdf48e2076 About a minute ago /bin/sh -c apt-get install figlet -y 1.02MB

c12b1ebe20b8 2 minutes ago /bin/sh -c apt-get update -y 39.2MB

dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B

<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B

<missing> 13 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB

<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B

<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B

<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB

Image History (cont.)

• When NOT using Dockerfile, some lines will not explain what really happened:

[zohar@lnx7-oracle-1 figlet]$ docker history figlet

IMAGE CREATED CREATED BY SIZE

COMMENT

8cc5680434bb 43 minutes ago /bin/bash 40.3MB

dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B

<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B

<missing> 13 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB

<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B

<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B

<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB

The Caching System

• Building the same image again will be instantaneous

• After each step docker takes a snapshot of the resulting image

• Before executing the step, Docker checks if it has already built that sequence (based on the text in the Dockerfile)

• Example: • Changing the apt-get install figlet and add a space, it will run again

• The apt-get update will not run again even though the apt mirrors updated (same command)

• We can force a rebuild with docker build --no-cache

CMD or ENTRYPOINT?

• Both ENTRYPOINT and CMD give you a way to identify which command should be run when a container is started from your image

• For example, most Linux container will use /bin/bash or /bin/shas their default CMD to allow shell access

• Running a container without a CMD or ENTRYPOINT will result with an error (either built in or via docker run command)

• Each image can have only one CMD and one ENTRYPOINT. Specifying more than one in the Dockerfile will override the previous command

Using CMD

• Example: let’s say we want to change or figlet image to display “hello” message.

• We need to modify the Dockerfile and build it:

• When running a new image:

FROM ubuntu

RUN apt-get install figlet -y

CMD figlet hello

[zohar@lnx7-oracle

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto

| |__ ___| | | ___

| '_ \ / _ \ | |/ _ \

| | | | __/ | | (_) |

|_| |_|\___|_|_|\___/

Override CMD

• If we want to override the CMD, we just send in a new command in the run command

• The bash canceled our figlet command – but what happens if we don’t want it to override? What if we want to send in a parameter?

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash

root@388ac4ed1d28:/#

Using ENTRYPOINT

• We want to be able to specify a different message on the command line (parameter)

• When running a new image:

FROM ubuntu

ENTRYPOINT [“figlet”, “-f”, “banner”]

[zohar@lnx7-oracle

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto banner

##### ## # # # # ###### #####

# # # # ## # ## # # # #

##### # # # # # # # # ##### # #

# # ###### # # # # # # # #####

# # # # # ## # ## # # #

##### # # # # # # ###### # #

Override ENTRYPOINT

• If we want to override the ENTRYPOINT, we can’t just send in a new command in the run command because it will be printed…

• We need to override it with the --entrypoint flag:

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash

##### ## #### # #

# # # # # # #

##### # # #### ######

# # ###### # # #

# # # # # # # #

##### # # #### # #

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto

root@5dd5da9a0825:/#

Using CMD and ENTRYPOINT Together

• In our example, figlet needs and extra parameter to work (the message). How can we specify a default value for it?

• We can combine using CMD and ENTRYPOINT• ENTRYPOINT will specify the command to run

• CMD will specify the default parameter

FROM ubuntu

ENTRYPOINT ["figlet", "-f", "banner"]

CMD ["CMD"]

Using ENTRYPOINT and CMD Example

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto

##### # # ######

# # ## ## # #

# # # # # # #

# # # # # #

# # # # #

# # # # # #

##### # # ######

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto test

##### ###### #### #####

# # # #

# ##### #### #

# # # #

# # # # #

# ###### #### #

Copying Files From Host to Image

• We can copy files from our host to the image when we build it

• We can see the file on our image’s container

• Changing the file will invalidate the cached layer – so rebuilding the image with a new file will generate a new image

FROM ubuntu

COPY zohar.txt /

ENTRYPOINT ["figlet", "-f", "banner"]

CMD ["CMD"]

[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto

root@b65dd0df7597:/# cat /zohar.txt

Testing a copy file...

The ADD Command

• Since the COPY command does not support copying file from a URL, we can use the ADD command for that

• ADD also allows us to copy tar files and automatically open them

• It is best practice to use COPY unless you must use ADD

The WORKDIR Instruction

• The WORKDIR instruction sets the working directory for subsequent instructions

• It also affects CMD and ENTRYPOINT, since it sets the working directory used when starting the container

• You can specify WORKDIR again to change the working directory for further operations

WORKDIR /src

The ENV Instruction

• The ENV instruction specifies environment variables that should be set in any container launched from the image

• This is the same as exporting in the OS level

• ENV variables can be overridden by the -e flag in the docker runcommand

ENV ORACLE_SID orclcdb

docker run -p 1521:1521 -e ORACLE_SID=orcl store/oracle/database-enterprise:12.2.0.1

The USER Instruction

• The USER instruction sets the user name or UID to use when running the image

• It can be used multiple times to change back to root or to another user

Docker Networks

Docker Network Basics

• Containers cannot have public IPv4 addresses

• They do have private addresses

• Services have to be exposed port by port

• Ports have to be mapped to different ports on the host to avoid conflicts

• Containers can see the network by default – it uses our default bridge network

• Docker --link only worked on a single host – does not work in swarm/cluster and considered deprecated

Example: NGINX Server

• Starting a NGINX (web) server

• The -P flag (publish all) indicates every exposed port to be mapped automatically to a port in the host server.

• In this case, port 80 will be mapped to port 9000

• We can also ask the container for his mapped ports:

[zohar@lnx7-oracle-1 ~]$ docker run -d -P nginx

[zohar@lnx7-oracle-1 ~]$ docker ps -l

PORTS NAMES

47733cd2d419 nginx "nginx -g 'daemon ..." About a minute ago Up About a minute

0.0.0.0:9000->80/tcp gallant_poincare

[zohar@lnx7-oracle-1 ~]$ docker port 477

80/tcp -> 0.0.0.0:9000

Example: NGINX Server (cont.)

• When we try to reach the container port, we use the localhost port for it:

[zohar@lnx7-oracle-1 ~]$ curl -l localhost:9000

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

Manual Binding of Ports

• Setting the ports manually can be done during the run command

• The convention is host-port:container-port

• Trying to start another container with the same port will result with an error

[zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx

6abc743b8dbaa46b0af1d6f4d72f6313bd41baa9c74524649ac7fe489e387709

docker: Error response from daemon: driver failed programming external connectivity

on endpoint friendly_bohr

(dde052b96b19300988e0c4ea389fe5d0a91eab3e91bafd0dfd9d1d454869dc27): Bind for

0.0.0.0:80 failed: port is already allocated.

Exposing Ports in the Image using Dockerfile

• The EXPOSE instruction tells Docker what ports are to be published in this image

• The Dockerfile doesn’t actually map the port to a port on the host –we still need to make it public in our run command

EXPOSE 1521

Finding Container Private IP

• We can use docker inspect command to find the private IP of a container

• The docker inspect is an advanced command, that can retrieve tons of information about our containers

• We can ping our container

[zohar@lnx7-oracle-1 ~]$ docker inspect --format '{{ .NetworkSettings.IPAddress }}'

keen_neumann

172.17.0.3

[zohar@lnx7-oracle-1 ~]$ ping 172.17.0.3

PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data.

64 bytes from 172.17.0.3: icmp_seq=1 ttl=64 time=0.109 ms

64 bytes from 172.17.0.3: icmp_seq=2 ttl=64 time=0.148 ms

Docker Networks

• Conceptually, a Docker network is a virtual switch

• It can be local (to a single Engine) or global (across multiple hosts)

• Each network has a dedicated subnet

• Containers with explicit names are discoverable via DNS

• A network is managed by drivers – some of them are built in and some can be added by plugins

[zohar@lnx7-oracle-1 ~]$ docker network ls

NETWORK ID NAME DRIVER SCOPE

03d9803daff0 bridge bridge local

3eddac377534 host host local

e733f2a48ed2 none null local

The Default Bridge Driver

• By default, the container gets a virtual eth0 interface

• It is connected to the Docker bridge network

• Addresses are allocated on a private, internal subnet

• Outbound traffic goes through an iptables MASQUERADE rule

• Inbound traffic goes through an iptables DNAT rule

• The container can have its own routes, iptables rules, etc.

The Host Driver

• The host network is internal network between the host and the container

• A container with a host driver sees (and can access) the network interfaces of the host

• Network traffic doesn't have to go through NAT, bridge, or veth

• Performance = native!

The null Driver

• It only gets the lo loopback interface. No eth0

• It can't send or receive network traffic

• Useful for isolated/untrusted workloads

Creating a Network

• Create a network

• Use network for a container

[zohar@lnx7-oracle-1 ~]$ docker network create OracleWeek

72635c667b4c3e54e350925339cef73cf480528cef6368d6f59954d2358c210d

[zohar@lnx7-oracle-1 ~]$ docker network ls

72635c667b4c OracleWeek bridge local

03d9803daff0 bridge bridge local

3eddac377534 host host local

e733f2a48ed2 none null local

[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont1 --net OracleWeek busybox

/ # ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue

link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff

inet 172.18.0.2/16 scope global eth0

Containers DNS

• All containers with specific name in the same network share the same dynamic DNS (placed at 127.0.0.11):

[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont2 --net OracleWeek busybox

/ # ping cont1

PING cont1 (172.18.0.2): 56 data bytes

64 bytes from 172.18.0.2: seq=0 ttl=64 time=0.176 ms

/ # nslookup cont1

Server: 127.0.0.11

Address 1: 127.0.0.11

Name: cont1

Address 1: 172.18.0.2 cont1.OracleWeek

/ # nslookup cont2

Server: 127.0.0.11

Address 1: 127.0.0.11

Name: cont2

Address 1: 172.18.0.3 05efb3eb1863

Connecting Multiple Networks

• Networks are independent from each other but a container can be connected to multiple networks

• We can attach a network to a running container

• Example:• Create a new network and a new container in that network

[zohar@lnx7-oracle-1 ~]$ docker network create ilOUG

41244791088e6970dca741b54fa54a8e0f0a08eeb1b184a78220a9cf03d2528a

[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont3 --net ilOUG busybox

/ # ping cont1

ping: bad address 'cont1‘

Connecting Multiple Networks (cont.)[zohar@lnx7-oracle-1 docker network connect ilOUG cont1

[zohar@lnx7-oracle-1 ~]$ docker attach cont1

/ # ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

/ # ping cont2

/ # ping cont3

Multi-host Networking: Overlay Driver

• When using cluster, overlay network will interact between nodes

Overlay Driver

• The overlay network driver can support a clustered (swarm mode) environment

• When you create a service that uses the overlay network, the manager node automatically extends the overlay network to nodes that run service tasks

• Only swarm services can connect to overlay networks, not standalone containers

• The overlay network without swarm requires a valid key-value store service (Consul, Etcd, and ZooKeeper (Distributed store)) and might be deprecated in the future

Docker Storage

Volumes, Bind Mounts and tmpfs

Docker “External” Storage

• Our containers are independent – they can’t see the host filesystem , but sometimes we want to share files/directories with the host or between containers

• The solution: map a volume to the container so it will be available to both the host and all the using containers

Named Volume

• Created and managed by Docker

• We can create a volume explicitly using the docker volumecreate command, or Docker can create a volume during container or service creation

• When creating a volume, it is stored within a directory on the Docker host (empty)

• When you mount the volume into a container, this directory is what is mounted into the container

• A given volume can be mounted into multiple containers simultaneously

Named Volumes Example

• We can create volumes, list them and inspect for actual location:[zohar@lnx7-oracle-1 ~]$ docker volume create test

[zohar@lnx7-oracle-1 ~]$ docker volume ls

DRIVER VOLUME NAME

local fc7d0d5897f2090f51a3ee7589ba360841344dc34c605626337debff4dfa9860

local test

[zohar@lnx7-oracle-1 ~]$ docker inspect test

"Driver": "local",

"Labels": {},

"Mountpoint": "/var/lib/docker/volumes/test/_data",

"Name": "test",

"Options": {},

"Scope": "local"

Volumes Functionality

• Volumes act as passthroughs to the host filesystem

• Volumes are easier to back up or migrate than bind mounts

• Volumes can be more safely shared among multiple containers

• When you docker commit, the content of volumes is not brought into the resulting image

• Volume drivers allow you to store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality

Bind Mounts

• Bind mounts have limited functionality compared to volumes

• When we use a bind mount, a file or directory on the host machine is mounted into a container

• The file or directory is referenced by its full path on the host machine

• The file or directory does not need to exist on the Docker host already, if it does not exist – it will be automatically created (when using –v)

• We use bind mounts when we want to mount a non-empty directory from the host to the container

Binding the Mount Volumes

• Volumes can be mounted in two ways:• on the docker run command

• Within a Dockerfile, with a VOLUME instruction

docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox

VOLUME /zohar

Mounting a Volume Manually

• We can map volumes on the docker run command

• The –v flag mounts a directory from your host into your Docker container. The flag structure is:

• If [host-path] or [container-path] doesn't exist it is created

• You can control the write status of the volume with the ro and rw options

• If you don't specify rw or ro, it will be rw by default

docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox

[host-path]:[container-path]:[rw|ro|z|Z]

Volumes and SElinux

• The permission on the volume will be the same as on the host

• That might be a problem if the files are owned by a user the container doesn’t know

• We can “help” Docker and SElinux play nice together by specifying the “z” (r/o) or “Z” (r/w) flag:

[zohar@lnx7-oracle-1 ~]$ docker run -it -v $(pwd):/zohar --rm --name cont1 --net

ilOUG busybox

/ # ls -l /zohar/

ls: can't open '/zohar/': Permission denied

/ # [zohar@lnx7-oracle-1 docker run -it -v $(pwd):/zohar:Z --rm --name cont1 --net

ilOUG busybox

/ # ls /zohar/

Desktop Documents Downloads Music Pictures Public Templates Videos

figlet myimage namer oradata

tmpfs Mounts

• A tmpfs mount is not persisted on disk, either on the Docker host or within a container

• It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information

[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --tmpfs /zohar/tmpfs busybox

/ # df -h /zohar/tmpfs/

Filesystem Size Used Available Use% Mounted on

tmpfs 1.8G 0 1.8G 0% /zohar/tmpfs

Orphaned Volumes

• What happens when you remove containers with volumes?

• They stay on disk – even if no one will ever use them again

• Use the –v flag to remove the container with the volumes attached

• We can prune the unused volumes using the docker volumeprune command

• Ultimately, you are the one responsible for logging, monitoring, and backup of your volumes!

Oracle RDBMS, MySQLand Docker

Oracle RDBMS Official Image

• Oracle provides their own Oracle RDBMS official image with database (12.2.0.1 EE) included (free):

• https://store.docker.com/images/oracle-database-enterprise-edition

• In order to pull this image, we need a Docker repository account

• We also need to accept the conditions on the store itself

• Official Image from Oracle Store:• https://container-registry.oracle.com/

docker login

docker pull store/oracle/database-enterprise:12.2.0.1

Oracle Official Image Limitations

• Supports a single instance database,

• No multiple homes

• No Grid Infrastructure/ASM

• No multiple instances on the same container

• No AMM

• Dataguard is not supported

• Database options and patching are not supported

Oracle Official Slim Image

• Since the official image is quite big (~4GB) there is also a slim version:

• This image does not support the following features - Analytics, Oracle R, Oracle Label Security, Oracle Text, Oracle Application Express and Oracle DataVault.

• The slim version is around 1.4GB

• Oracle support does not require you to use the provided Docker files – we can build our own images!

docker pull store/oracle/database-enterprise:12.2.0.1-slim

Building Our Own Images

• Oracle official Docker team github account with different Dockerfilesand samples: https://github.com/oracle/docker-images

• On the repository:• Oracle 11.2.0.2 (XE) Dockerfile

• Oracle 12.1.0.2 Dockerfile and scripts (SE2, EE)

• Oracle 12.2.0.1 Dockerfile and scripts (SE2, EE)

• Lots of other Oracle stuff: Oracle Java, Weblogic, GoldenGate and NoSQL…

• The build recipes build our own images, so we can customize it

• Recipes require ZIP files of the software for deployment on the image

How It’s Going to Work

• The official image based on Oracle Linux 7

• The image contains an instance running inside the container

• The ORACLE_HOME and datafiles are kept on a volume (host machine)

• The image also runs the listener on port 1521 and exposes this port

• The instance runs inside the container is a root container (CDB) in our multitenant database. It also has a single plugged database

• Persistence: • There is a volume plug we can use to map our local pluggable database files

to the database and then we can plug our database to the instance

Creating the Container

• Creating the container (might take some time):[zohar@lnx7-oracle-1 ~]$ docker volume create OracleDBData

[zohar@lnx7-oracle-1 ~]$ docker run --name plugpdb -p 1521:1521 -p 5500:5500 -e

ORACLE_SID=MYCDB -v OracleDBData:/ORCL /home/zohar/oradata:/opt/oracle/oradata

store/oracle/database-enterprise:12.2.0.1

Setup Oracle Database

Oracle Database 12.2.0.1 Setup

Sat Nov 18 16:09:47 UTC 2017

Check parameters ......

log file is : /home/oracle/setup/log/paramChk.log

paramChk.sh is done at 0 sec

untar DB bits ......

log file is : /home/oracle/setup/log/untarDB.log

untarDB.sh is done at 166 sec

config DB ......

log file is : /home/oracle/setup/log/configDB.log

Inside the Container

• The entire database home and database file are stored in the same location: /ORCL, divided into u01-u04 for different usages

• We can mount these areas into our own disks or volume

• In our example, the entire /ORCL is mounted into one volume: OracleDBData

• We can mount other mount points or volumes to keep our pluggable databases

Connecting to the Database

• Connecting to the database, via the container:

• The initial password for SYS is “Oradoc_db1”

• We can also connect using bash command and work like any other “server”

docker exec -it plugdb bash -c "source /home/oracle/.bashrc; sqlplus /nolog"

Building Our Own Image

• Download the appropriate directory from and unzip it on the server• https://github.com/oracle/docker-

images/tree/master/OracleDatabase/dockerfiles

• Download the Oracle Software from the Oracle site and place them in the same directory

• Modify whatever you need – scripts, Dockerfile, keep some of the options manually removed, etc.

• Build the image

docker build --force-rm=true --no-cache=true -t

oracle/database:12.2.0.1-ee -f Dockerfile.ee .

Start the Container

• Running the container is just like before – we can pass some envvariables but most of them already has default values

• This database doesn’t have to use containers (like the official image): ORACLE_CONTAINERDB=false

docker run --name orcl1 \

-p 1522:1521 -p 5502:5500 \

-e ORACLE_SID=orclcdb \

-e ORACLE_PWD=manager1 \

-e ORACLE_CHARACTERSET=WE8MSWIN1252 \

-e ORACLE_CONTAINERDB=false \

oracle/database:12.2.0.1-ee

• MySQL database is ideal for Docker Containers – its containers exist since forever

• There are actually two MySQL main images:• mysql (also available on the store)

• mysql/mysql-server (automated, connected to mysql GitHub)

• Both images are tagged with multiple version: 5.5, 5.6, 5.7 (latest) and 8.0 (preview). They also run sub version (e.g. 5.6.38, 5.7.20, etc.)

Starting the MySQL Container

• Starting the container

• Connecting to the database using root:

docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d

mysql/mysql-server:tag

docker exec -it my-container-name mysql -uroot -p

Where to Store Data

• There are several ways to store data used by applications that run in Docker containers:

• Let Docker manage the storage of your database data by writing the database files to disk on the host system using its own internal volume management. This is the default and is easy and fairly transparent to the user

• Create a data directory on the host system (outside the container) and mount this to a directory visible from inside the container. This places the database files in a known location on the host system, and makes it easy for tools and applications on the host system to access the files

Connecting to Existing Database

• If you start your mysql container instance with a data directory that already contains a database (specifically, a mysql subdirectory), it will use that as its database

• Note: the $MYSQL_ROOT_PASSWORD variable should be omitted from the run command line; it will in any case be ignored, and the pre-existing database will not be changed in any way

• If there is no database initialized when the container starts, then a default database will be created.

Creating Database Dumps

• Most of the normal tools will work, although their usage might be a little convoluted in some cases to ensure they have access to the mysqld server. A simple way to ensure this is to use docker exec and run the tool from the same container, similar to the following

$ docker exec some-mysql sh -c 'exec mysqldump --all-databases -uroot \

-p"$MYSQL_ROOT_PASSWORD"' > /some/path/on/your/host/all-databases.sql

Overriding Options and MySQL.cnf

• We can override options by using the --option flag

• To provide a different mysql.cnf, use the –v flag:

• Remember SELinux? We need to allow the file for the container:

docker run --name my-container-name -d mysql/mysql-server --option1=value --

option2=value

docker run --name my-container-name -d mysql/mysql-server --character-set-

server=utf8 --collation-server=utf8_general_ci

docker exec -it my-container-name cat /etc/my.cnf > /my/custom/config-file

docker run --name my-new-container-name -v /my/custom/config-file:/etc/my.cnf -e

MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:tag

chcon -Rt svirt_sandbox_file_t /my/custom/config-file

Docker Compose

Docker Compose Tool

• Dockerfiles are great to build a single container

• But when you want to start a complex stack made of multiple containers, you need a different tool. This tool is Docker Compose

• Docker Compose is optional (needs to be installed separately)

• The general idea of Compose is to enable a very simple, powerful onboarding workflow:

• Clone your code• Run docker-compose up

• Your app is up and running!

Compose overview

• You describe a set (or stack) of containers in a YAML file called docker-compose.yml

• You run docker-compose up

• Compose automatically pulls images, builds containers, and starts them

• Compose can set up networks, volumes, and other Docker options

• Compose can run the containers in the background, or in the foreground

• When containers are running in the foreground, their aggregated output is shown

Installing Compose

• We can download and install Docker Compose right off its github project: https://github.com/docker/compose/releases

• Make sure to use the latest version (current version: 1.17.1):curl -L https://github.com/docker/compose/releases/download/1.17.1/docker-compose-

`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

docker-compose.yml

• Example for a Docker compose yaml file:

• This file will start two containers: one based on the local Dockerfile(www) and one based on the redis image. It will also mount a volume and map a port.

version: '2'

services:

build: .

ports:

- "5000:5000"

volumes:

- .:/code

redis:

image: redis

docker-compose.yml for Oracle DBMS

• Another example for a Docker compose yaml file:

• In this example, we will start a database server using the 12.2.0.1-eeimage, map some ports. Our Oracle SID is also predefined.

version: '2'

services:

database:

image: oracle/database:12.2.0.1-ee

volumes:

- /home/oracle/oradata:/opt/oracle/oradata # persistent oracle database data.

ports:

- 1521:1521

- 8080:8080

- 5500:5500

environment:

-ORACLE_SID=DEVOPSUATCDB Example by Gerald Venzl

Docker Compose Usages

• Starting a Docker Compose environment:

• A network and two containers were created

• Check what is up (in our work directory only):

[zohar@lnx7-oracle-1 wordpress]$ docker-compose up -d

Creating network "wordpress_default" with the default driver

Creating wordpress_db_1 ...

Creating wordpress_db_1 ... done

Creating wordpress_wordpress_1 ...

Creating wordpress_wordpress_1 ... done

[zohar@lnx7-oracle-1 wordpress]$ docker-compose ps

Name Command State Ports

-------------------------------------------------------------------------------------

wordpress_db_1 docker-entrypoint.sh mysqld Up 3306/tcp

wordpress_wordpress_1 docker-entrypoint.sh apach ... Up 0.0.0.0:8000->80/tcp

Docker and Docker Compose

• We can still use our original docker commands[zohar@lnx7-oracle-1 wordpress]$ docker ps

STATUS PORTS NAMES

ce8847b28d67 wordpress:latest "docker-entrypoint..." 2 minutes ago

Up 2 minutes 0.0.0.0:8000->80/tcp wordpress_wordpress_1

daff10f9d8b9 mysql:5.7 "docker-entrypoint..." 2 minutes ago

Up 2 minutes 3306/tcp wordpress_db_1

[zohar@lnx7-oracle-1 wordpress]$ docker network ls

4f395cc553ad wordpress_default bridge local

Stopping the Service

• We can stop the entire service by using docker-compose stop

• We can remove the service by using the docker-compose down command. This will delete all containers as well (but not the volumes)

[zohar@lnx7-oracle-1 wordpress]$ docker-compose stop

Stopping wordpress_wordpress_1 ... done

Stopping wordpress_db_1 ... done

[zohar@lnx7-oracle-1 wordpress]$ docker-compose down

Stopping wordpress_wordpress_1 ... done

Stopping wordpress_db_1 ... done

Removing wordpress_wordpress_1 ... done

Removing wordpress_db_1 ... done

Removing network wordpress_default

Docker Containers After Restart

• When the host shut down, all the containers will automatically stop

• When the host comes back up, all containers will be DOWN

• In order to automatically start the container after boot, we can use the --restart option on the run command

$ docker run -dit --restart unless-stopped redis

Flag Description

no Do not automatically restart the container. (the default)

on-failure Restart the container if it exits due to an error, which manifests as a non-zero exit code.

unless-stoppedRestart the container unless it is explicitly stopped or Docker itself is stopped or restarted.

always Always restart the container if it stops.

Limiting Container Performance

• We can limit the performance on a container:• CPU: number of cpu (which cpus to use), number of cycles, and different

quotas

• Memory: amount of memory, amount of virtual memory

docker run --rm --cpuset-cpus=0 busybox echo hello world

docker run --rm --cpuset-cpus=0 --cpu-period=50000 busybox echo hello world

HEALTHCHECK

• HEALTHCHECK instruction in DockerFile

• Tells Docker how to test a container to check that it is still working properly (not just “up”) – i.e. the database/listener are up

• New status added to container lists

• Adds “(healthy)” to Status column in a “docker ps response”

[zohar@lnx7-oracle-1 ~]$ docker ps

d2926bc4b490 portainer/portainer "/portainer" 39 minutes ago Up 39 minutes

6a204203d9e7 store/oracle/database-ente... "/bin/sh -c '/bin/..." 3 hours ago Up 3 hours (healthy)

Docker Secrets

• Stored encrypted in swarm mode only

• Exposed only to nodes that run services that need them

• Presented in container via RAM only tmpfs files• never persisted to disk in encrypted format

• when container stops, secret is no longer present

• All communications between swam nodes via TLS, so secret never in the clear on the wire either

• Different secret values per environment using tags

Docker Security

• Use trusted images / with a known Dockerfile

• Think (twice) about pulling images from public repos / Docker hub

• Docker images can be signed

• Docker can use TLS (client to daemon)

• Do NOT disable SELinux or set enforce off (avoid setenforce 0)

• Do not keep secrets in images

• Remember: The host can always access container

GUI, Orchestration and Clusters

Swarm, GUI Interfaces

Docker GUI

• Kitematic – the default (now obsolete) interface from Docker themselves

• Portainer - very simple, very robust GUI interface for Docker

• Shipyard – Built on Docker Swarm, Shipyard gives you the ability to manage Docker resources including containers, images, private registries and more

• Rancher – high end solution that interfaces nicely with cloud solutions

• There are others – but I suggest going with Portainer (for beginners) or Rancher or Shipyard (more advanced)

Portainer

• Portainer is a simple but robust GUI interface for Docker

• It runs from within a Docker container

• It’s easier to see what is going on:• Containers• Volumes• Networks• Logs• Etc.

docker run -d --privileged -p 9000:9000 -v

/var/run/docker.sock:/var/run/docker.sock -v /opt/portainer:/data

portainer/portainer

Portainer Deshboard

Portainer Containers

Using Console From Web

Monitoring Performance

Kubernetes: Container Orchestration

• Kubernetes is an open source container orchestration tool developed by Google to allow a wider usage of Docker with more control and better scaling and availability

• It is one of the most commonly used container orchestration tool and has a very large community

• Kubernetes allows you to completely control your container deployment across the cluster with custom storage and network solutions and well defined deployment rules

• Kubernetes allows for easy integration with public (AWS, GCE, Azure) and private (VMWare) cloud solutions

Kubernetes Concepts

• While it provides a useful, scalable, and powerful tool, Kubernetes also gets a lot of clout because it’s an intricate and sometimes overly-complicated system

• Installing Kubernetes is a slightly more complex endeavor since you will need additional tools to provide network (i.e. flannel) and store the cluster data (i.e. etcd).

• Basic Concepts:• Pod: The basic deployment unit, One or more containers (usually one) that share a

common function.• ReplicaSet: Controls the replication and availability of a pod.• Deployment: Controls the pod’s deployment by creating and managing ReplicaSets.

Allows for rolling updates.• Service: Advertises a set of pods to other pods or to the outside world. Allows easy

discovery with the DNS add-on.• StatefulSet: Manages the deployment and scaling of a set of Pods, and provides

guarantees about the ordering and uniqueness of these Pods.

Kubernetes

Source: wikipedia

Docker Swarm

• Native Docker cluster same API as a single engine

• Fast provisioning, about 500 msec

• Scheduling Algo: spread, binpack, rand

• Features are optional, you can continue useKubernetes etc.

• There is NO insecure mode

Docker Swarm Overview

Docker Swarm

• Since Docker 1.12

• Swarm is merged with Docker engine:• Load balancer included

• Service discovery

• Cluster scheduler

• Swarm has many features like Google's Kubernetes• Easier to get started

• Better performance

Swarm Services

• To deploy an application image when Docker Engine is in swarm mode, you create a service

• Frequently a service will be the image for a microservice within the context of some larger application

• Examples of services might include an HTTP server, a database, or any other type of executable program that you wish to run in a distributed environment

Setup Swarm and Machine

• Create Swarm ID

• Create Machine with Swarm master

• Create Machine with Swarm agent01 / 02

• Set Docker env for Swarm master

• Deploy service to the swarm

• If needed, scale the service to run more container on more nodes

Learn More

More information:

• https://docs.docker.com/• Reference, information, how-to

• https://training.docker.com/• Training, self paced and trainer

• Blog: https://blog.docker.com/

• Videos: https://www.youtube.com/user/dockerrun

Oracle Support

• Support for Docker Running on Oracle Linux (Doc ID 1921163.1)• https://support.oracle.com/epmos/faces/DocumentDisplay?id=1921163.1

• Oracle Support Document 2216342.1 (Oracle Support for Database Running on Docker):

• https://support.oracle.com/epmos/faces/DocumentDisplay?id=2216342.1

• Gerald Venzl (will be here in ilOUG Tech Days 2018 (January 2018), speaking about Docker!)

• https://geraldonit.com/2017/08/21/creating-an-oracle-database-docker-image/

Did You Hear AboutOracle 18c?

• Oracle is changing the way they number products• Products will be release more often• Version names will be based on last two digits of the year and a

subversion• There is no more Release 1-Release 2 for major versions, we can’t tell

when we jumped to the next major version and when for the next minor• Examples:

• 12.2.0.1 will be followed by 18.1 (and not 12.2.0.2)• SQL Developer 4.2.1 was followed by 17.3

• Read more at the updated MOS Note 742060.1 – Release Schedule of Current Database Releases

Wait! 18c?! What happened to 13c?

Summary

• We talked about Docker and Docker containers use cases

• We learned how to use Docker (slides will be available soon)

• We saw how Docker works with Oracle and MySQL

• We overviewed swarm mode (not for databases)

• We talked about Docker in the cloud environment

Q&AAny Questions? Now will be the time!

Zohar Elkayamtwitter: @realmgicZohar@Brillix.co.il

www.ilDBA.co.ilwww.realdbamagic.com

docker concepts for oracle/mysql dbas and devops

Technology

linux internals for mysql dbas - percona · pdf filelinux...

mongodb for dbas

mysql on windows: a survival guide for linux-based dbas ·...

postgresql for oracle mysql dbas · in mysql, data...

for oracle and mysql dbas introduction to postgresql · for...

collaborate 2012 - administering mysql for oracle dbas

dbas series connectors

driving innovation with mysql and dockermysql server 8.0:...

albus-insec arthur caranta.pdf · docker • registry...

kafka for dbas

openworld 2017 mysql for oracle dbas - · pdf file ......

using docker in biohpc cloud optional exercises docker...

reputation system for collaborative community · 2018. 6....

mysql backup and restore using zmanda recovery manager...

mysql e oracle para dbas

guide to mysql database service in oracle cloud€¦ · •...

a visual guide for migrating to postgres plus from mysql ·...

mysql for oracle dbas

what’s new in mysql 8€¦ · mysql 8.0 builds on this...

mysql and docker strategies - percona · mysql and docker...