docker demystified for mainframersvmworkshop.org/pres2017/dokrtech.pdf · docker host: the...

34
1 IBM Systems Technical Events | ibm.com/training/events © Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM. © Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM. Docker demystified for Mainframers Wilhelm Mild IBM Executive IT Architect Integration Architectures for Mobile, Linux & z Systems

Upload: others

Post on 19-May-2020

31 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

1

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker demystified for Mainframers

Wilhelm Mild

IBM Executive IT Architect

Integration Architectures

for Mobile, Linux & z Systems

Page 2: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

3

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Virtualization vs. application isolation

Virtualization and application isolation

Virtualization:

Infrastructure oriented

Virtual server resource

management

Several applications per server

Isolation in virtual servers

Persistence via Virtual server

Security in virtual server

Application isolation:

Service oriented

Application management inside

a virtual server

Solution decomposed into

several units

Isolation on application level

depends on implementation

Security based on access

security in OS and Virtualization

layer

Page 3: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

4

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Virtualization vs. application isolation

z/OS and Linux applications

In z/OS:

LPAR or z/VM Hypervisor virtualizes z/OS Systems as guest

applications are running in address spaces

address space characteristics:

isolate applications

enable application control, horizontal scalability

applications share data

address space priorities allow application prioritization

In Linux:

LPAR or z/VM & KVM Hypervisors virtualization is for Linux

guest systems not for pure application isolation

applications are running as processes

no hard isolation between processes

multiple start of a single application on the same kernel not

easy

Linux Containers – concept to improve application isolation

their scalability and flexibility

System z Host

IBM z System Infrastructure

LPAR

Hypervisor

z/OS Linux guests

z/OS

MQ

2 A

ddr

Sp

ace

MQ

1 A

ddr

Sp

ace

Ba

tch

Addr

Sp

ace

DB

2

Addr

Sp

ace

MQ

2 A

dd

rS

pa

ce

MQ

1 A

ddr

Sp

ace

Ba

tch

Addr

Sp

ace

DB

2

Addr

Sp

ace

z/OS

App1

App4

App1’

App3

App2

App2’

App3

App4’App2’

LNX1 LNX2 LNX3

Page 4: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

5

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Containers in Linux – for application isolation

• linuxcontainers.org is the umbrella project behind Linux Containers (LXC), Linux Container

management (LXD), Linux Container FileSystem (LXCFS) and Linux cgroup manager

daemon (CGManager).

• The goal is to offer a Linux distro and vendor neutral environment for the development of

Linux container technologies.

• The main focus is system containers. This is achieved through a combination of kernel

security features such as namespaces, mandatory access control and control groups

(cgroups).

• Container goals and characteristics:

Isolated application environments within a Linux OS instance

Each container has its own, different address space but same kernel

Serve a single task

Self contained set of files for applications

Startup time and efficiency compare to native execution

Page 5: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

7

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Containers application isolation

To simplify:

“cgroups” will allocate resources to your container

CPU

Memory

Disk I/O throughput

“namespace” will isolate

process IDs

Hostnames

User IDs

network access

interprocess communication

filesystems

Kernel

Linux Guest

cgroups

Kernel

Namespaces

Container 1

App

cgroups

Kernel

Namespaces

Container 2

App

App App App

Page 6: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

8

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker

In 2014, Docker teamed with Canonical, Google, Red Hat, and Parallels to create a standardized

open-source program libcontainer that allows containers to work within Linux namespaces and

control groups (cgroups) without needing administrator access. Docker initially used lxc as

underlying technology to communicate with the kernel, today, it uses it’s own library, libcontainer.

Docker is one implementation of Linux containers and their management

Open, portable, light-weight run-time and packaging tool

Container in standard operating environment and delivery vehicle for

applications with wildly different requirements

Much faster to spin-up and efficient to run than a VM

Isolated from each other

Easily build and ship complex application, without worrying about infrastructure

differences or interference from other software stacks

Quickly and reliably deploy and run applications on many infrastructures

Essential for horizontally scaling apps on the cloud

*

Page 7: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

9

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Linux virtual server vs. Docker containers overhead

Kernel

Container 1

SUSE - Image

ADD VIM - Image

ADD APACHE -

Image

Container 2

Debian - Image

ADD VIM - Image

ADD MySQL -

Image

The virtualization approach usually provides a high level of isolation and security as all communication

between the guest and host is through the hypervisor.

It is also usually slower and incurs some overhead due to the infrastructure emulation.

To reduce this overhead, another level of virtualization called "container virtualization" was

introduced which allows to run multiple isolated user space instances on the same kernel.

Containers layered approach share common files and use copy-on-write filesystems

Page 8: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

10

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Approaches for Application Deployment Virtualization vs. Containers

Virtualization - Infrastructure oriented

Customers have virtualized their servers to gain efficiencies

Focus is on virtual server resource management

Few applications per Guest VM / Operating System instance

Provides application isolation - An application failure does not adversely affect other applications residing in other Guest VMs

Provides persistence across server restarts

Docker containers - Service oriented

Application-centric - infrastructure resources are assumed to be already in place

Focus is on application isolation / management

All containers for a given application reside in a single Guest VM / single Operating System instance

Has specific DevOps implications

Provides a very dynamic application deployment model

Hypervisor

OpenStack

(running in

a Guest VM)

App n

(running in

Guest VM n)

App 1

(running in

Guest VM 1)

App 2

(running in

Guest VM 2)

OS Kernel OS KernelOS KernelOS Kernel

. . .

Virtual

ComputeVirtual

Storage

InfrastructureVirtual

Network

Hypervisor / LPAR

Container Manager

DOCKER

(running in a Guest VM)

App 1

(running in

Container 1)

App 2

(running in

Container 2)

App n

(running in

Container n)

. . .

OS Kernel

Virtual

ComputeVirtual

Storage

InfrastructureVirtual

Network

Page 9: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

11

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Virtualization vs. Container application isolation in z Systems

z/OS and Linux virtualization

IBM z System Infrastructure

LPAR

Hypervisor

z/OS

z/OS

MQ

2 A

ddr

Sp

ace

MQ

1 A

ddr

Sp

ace

Ba

tch

Addr

Sp

ace

DB

2

Addr

Sp

ace

MQ

2 A

ddr

Sp

ace

MQ

1 A

ddr

Sp

ace

Ba

tch

Addr

Sp

ace

DB

2

Addr

Sp

ace

App1

App1’

App3

App2

App2’

App

4

Linux1 Linux2z/OS

Docker Container deployment in Linux

Virtualization: Infrastructure oriented

Virtual server resource management

Several applications per server

Isolation per virtual server

IBM z System Infrastructure

Linux Guest 1

Docker Engine

Bins/Libs

App1

Bins/Libs

App2

Bins/Libs

App3

Linux guest system 2

LPAR

Hypervisor

Docker Engine

Bins/Libs

App1

Container

Container Container

Container

Containers: Service oriented

Application management via container

Solution decomposed into several units

Dynamic, isolation in container

Page 10: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

12

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker in Linux on z Systems

Docker Terminology:

Docker Image: set of self contained files. Read-only. The image from which containers

are launched. An image is the read-only layer that never changes (golden image).

Images can be created based on committed containers.

z/OS – a shared library and SVA – Shared Virtual Area

Docker Container: a running instance, generated from a Docker image. Self-contained

environment built from one or more docker images. Information available at the

Container level include image info from which it is generated, memory used, ip address

assigned to it, port opened for that container etc.

z/OS – an Address Space running an application (MQ, DB2, COBOL appl.)

Dockerfile: A text document that contains all the commands to build a Docker image.

z/OS – JCL to start an Address Space with the application set started in it (CICS)

Docker Compose: A .yml file used to create multi-container applications from Docker

images ( z/OS: Link process with multiple compiled modules /binaries for one phase)

Docker Swarm: A cluster management of Docker container (placement, scheduling,.. )

It turns a pool of Docker hosts into a single, virtual Docker host. Any tool communicating

with a Docker daemon can use Swarm to transparently scale to multiple hosts.

*

Page 11: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

13

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker in Linux on z Systems

Docker Terminology – for deployments:

Docker host: the (virtual) server where a Linux guest and Docker is running

Docker Hub: share container images as building blocks and automate workflows

z/OS – Repository for binaries for z/OS applications

Container Group: A group of containers, which all share the same image.

z/OS – applications in multiple address spaces sharing code in SVA

Docker Registry: A service responsible for hosting and distributing images.

The default registry is the Docker Hub. A Registry can be local, remote, private or public.

z/OS – Libraries with binaries that can run in an Address Space

Repository: A collection of related images, usually providing different versions of the same application or

service.(i.e different python images in its repository on Docker Hub).

z/OS – Repository with Versioning control of modules or application components like JCL, binaries

Essential for horizontally scaling apps on-prem and in the cloud

Docker in Linux on z Systems distributions:

SLES 12 SP1

UBUNTU 16.04

Page 12: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

14

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Container Lifecycle with a Docker Engine

Client-server architecture with daemon deployed on physical or virtual host

Uses Linux kernel cgroups and namespaces for resource management and isolation

Uses copy-on-write filesystem for git-like image change management

A portable, lightweight application runtime and packaging tool

built on top of kernel container primitives

Build Ship Run

Docker Engine

Page 13: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

15

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

What are the Basic Functions of Docker?

DockerImage

Repository

Dockerfile for Application

Image N

Docker Engine (Build)

Server

Host OS

Docker Engine

Co

ntain

er A

Co

ntain

er B

Co

ntain

er N

Push new Image to

Repository

Describes steps to build container automatically

from source

Get N

…Run N

Build MSource Code Repository

Build Ship Run

Developer

Creates App, Builds

Container And pushes

to Registry

Operator

Deploys

Containers

Page 14: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

17

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Dockerfile demystified

The Dockerfile contains instructions

how to build a Container image.

Based on that image a container can

be instantiated or transferred.

# Dockerfile to build a container with IBM IM

FROM sles12.1:wlp

MAINTAINER Wilhelm Mild "[email protected]"

# Set the Install Reopsitories – replace defaults

RUN rm /etc/zypp/repos.d/*

COPY server.repo /etc/zypp/repos.d

# install the unzip package

RUN zypper ref && zypper --non-interactive install unzip

# copy package from local into docker image

COPY IM.installer.linux.s390x_1.8.zip tmp/IM.zip

# expose Ports for Liberty and DB2

EXPOSE 9080

EXPOSE 50000

# unzip IM, install it and delete all temp files

RUN cd tmp && unzip IM.zip && ./installc -log

/opt/IM/im_install.log -acceptLicense && rm -rf *

Build

Kernel

Container 1

SLES12 - Image

ADD UNZIP - Image

ADD IM - Image

Page 15: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

18

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker Hub

Enable sharing and collab of Docker Images

Private and public repositories of images

Certified base images by ISVs

Ship

Docker Repository

(Docker Hub)

$ docker push $ docker pull

Page 16: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

19

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Run a Container

It’ll run in an isolated user space within a running Linux OS

Kernel is shared across containers

All packages and data in an isolated run-time, saved as a filesystem.

Resource management implemented with cgroups

Resource isolation through namespaces

Customize and build applications on top of a running container

Run

Ap

p A

Host OSServer

Ap

p A

Bins/Libs

Ap

p B

Ap

p B

Ap

p B

Ap

p B

’ContainerA

pp

A’

Bins/Libs

Ap

p A

Bins/Libs

User access to a Container (running)

> docker exec -it containerIdOrName bash

> docker run container-name

Display all Container images in the System

> docker images

Display all running Containers in a Linux System

> docker ps –a

Run a container

Bins/Libs

Page 17: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

20

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

What is a Docker container in Linux finally

A Docker Container is a form of isolation for individual applications or workloads

z/OS: Can be compared with a z/OS Address Space (new: zospt)

It is a flexible way of deployment of isolated applications

z/OS: starting an application in a certain Address Space

The Docker container is build via a Dockerfile, which contains instructions to include

necessary components in the container image that is built

z/OS: Use of JCL and Conditional Procedures to run applications

Based on a Container image a container can be instantiated or transferred.

z/OS: Binary member transfer – is not containing application dependencies like IP or runtime lib.

A Container (runtime) is not persistent

z/OS: A workload in an Address Space can be HA in a Parallel Sysplex but is not persistant to it

A Container can be used as base to build another container image

Page 18: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

21

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Application isolation in z/OS and Docker

z Systems Host

z CPU, Memory and IO

Support Element

DPM or PR/SM™

IBM z System Infrastructure

LPAR

Hypervisor

z/OS

z/OS

MQ

2 A

ddr

Sp

ace

MQ

1 A

ddr

Sp

ace

Ba

tch

Ad

dr

Sp

ace

DB

2

Addr

Sp

ace

MQ

2 A

ddr

Sp

ace

MQ

1 A

ddr

Sp

ace

Ba

tch

Addr

Sp

ace

DB

2

Addr

Sp

ace

z/OSLinux guest 1

Docker EngineLinux guest 2Docker Engine

Bins/Libs

App1

Bins/Libs

App2

Bins/Libs

App3

Bins/Libs

App1

Container

Container Container

Container

Docker Containers

Second Level Guest

First Level Guest via LPAR

Page 19: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

23

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Microservices Architectures

From monolithic architectures …

aligned and optimized to server

boundaries

Installed on these servers

Layered applications with coarse

granularity

Static, long living

Often virtualized, but no progress

in solution structure

Use components which fit and

expand the solution

… to microservices

Solution broken into small units

Delivered in containers

Independent services, loosely

coupled

Short life time, fast changes

Resilient services

Scaling components is simple

Using best tool for each

subtask

Linux containers are designed for flexible Microservices

in today’s two speed IT

Page 20: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

24

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Scale up for maximum efficiency

Isolation, QoS and scaling for tiers

and tenants

Grouping microservices end-to-end

allows for simple scaling and optimized

local communication

Microservices Deployment Types

...

Page 21: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

27

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker Ecosystem: Cluster Orchestration

• Docker swarm and compose

• Simple cluster framework fitted to run Docker containers

• Composite applications with compose

• Docker acquired makers of Mesos Aurora scheduling framework,

for integration of Aurora parts into swarm

• Apache Mesos

• Large scale cluster project

• Marathon framework schedules containers

• Mesos intends to run containers natively (without additional framework)

• IBM intends to add value with Platform Computing scheduler

• Google Kubernetes

• Large scale cluster manager/scheduler by Google

• Base for CNCF (Cloud Native Compute Foundation) orchestration

• Grouping and co-location of containers as pods, forming a service

• Hashicorp Nomad

• Cluster manager and scheduler for VMs, Containers, language runtimes

• Simple, efficient, scalable, limited scope

(just cluster management and scheduling)

Page 22: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

28

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Enabling Docker for Linux on z Systems

• Docker on Linux on z support on SLES 12 & UBUNTU 16.04

• Binaries available for RHEL 7

• http://www.ibm.com/developerworks/linux/linux390/docker.html•Instructions on setting up and building base images:

• http://containerz.blogspot.ca/2015/03/first-steps-with-docker.html

• http://containerz.blogspot.ca/2015/03/creating-base-images.html•Create a Docker image that runs a private repository on Linux on z

• Same instruction as for other platforms:

• https://github.com/docker/docker-registry/blob/master/ADVANCED.md• IBM-managed images uploaded to https://hub.docker.com/r/ibmcom/ in Docker Hub

•Linux on z images are named with “_s390x” suffix, until multi-arch support is available

• Docker is Docker is Docker… on Linux on z too!

•Follow the blog:

http://containerz.blogspot.com/

Page 23: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

29

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker Enterprise Edition (EE) announcement for Linux on z Systems

29

Docker CEO Ben Golub highlights availability for Docker EE for Linux on z and LinuxONE at DockerCon in April.

• Takes advantage of z and

LinuxONE capabilities to build

apps with unmatched reliability,

performance and security

• Brings consistent container

platform for app development

across software supply chain

Provides new enterprise version of container platform for

building, running, testing and operating applications

• Docker EE on Linux on z

• https://www.docker.com/enterprise-edition

• http://www-03.ibm.com/press/us/en/pressrelease/52096.wss

Page 24: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

30

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker And Cross-Platform Portability

Docker user experience (CLI, REST API) is identical across platforms

Containers in binary form are not portable, the source code or a s390x

binary must be available

Multi-arch support available using external tools

http://containerz.blogspot.com.br/2016/07/multi-arch-registry.html

Micro-service architecture often have clean structure and simple individual

components

Containers are often created with Dockerfiles (build descriptions) containing:

Specification of the base image

If the same distribution is available on s390x, usually simple

If the base image is not available, some creativity is required

Additional steps to modify the image are often platform independent

Add packages (needs to match the base image)

Download files

Perform build

Page 25: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

32

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker on Linux on z Systems: Content

Images on Docker Hub, search for “s390x”

https://hub.docker.com/search/?q=s390x

Page 26: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

35

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

“Scalable Financial Trading Analysis & Insights”

Input Data

Sentiment Analysis

Trade Transactions

Geospatial Analysis

Open Source Content

News Feed

Historical S&P 500 Index

Visualization Dashboard

https://www.youtube.com/watch?v=VWBNoIwGEjo

Page 27: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

36

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Scalable Financial Demo Architecture

https://www.dropbox.com/s/3szus6vk77rg1nh/linuxcondemowithgraphs.mp4?dl=0http://yktgsa.ibm.com/gsa/yktgsa/home/e/n/engd/linuxcondemo.mp4

Demo w/o sound ~ 9min

Apache

CuratorService discovery and

registration

ChefSystem

Orchestration

NginxLoad

Balancer

Spark +

Node.jsAnalytics (Spark as a Service)

MongoDBnoSQL

MariaDBSQL

PostgreSQLSQL

Apache KafkaMessage Queue

(ingestion)

S&P

500

TwitterNews/Blogs

Systems

of

Engageme

nt

Operating

SystemSLES 12

Page 28: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

37

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Docker Container Scaling on Linux on z & LinuxONE due

to Volume of Financial Trades, News Feeds & Tweets

Page 29: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

38

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

• Containers: simple way to build and deploy SW with Docker

currently leading framework

• Single LinuxONE Emperor ran more than 1 Million light

Docker containers

•Workload: busybox httpd server (no NAT)

• LinuxONE Emperor runs 4K active Docker containers on ave

2.0x better than comparable Haswell-based system!

• Workload: 50% WAS Liberty 8.5.5.2, IBM JDK 8.0, Apache

Solr 4.10.0, and 50% busybox httpd server

• With GOLANG now avail on z!

• LinuxONE Emperor can host over 10K Docker containers with

mixed (heavy & light) workloads

• Workload: 4K WAS Liberty 8.5.5.2, IBM JDK 8.0, Apache

Solr 4.10.0 plus 6K busybox httpd server (no NAT)

Lab measurements with Docker® on Linux on z

0

1000

2000

3000

LinuxONE Haswell

thro

ugh

pu

t (#

tr

ansa

ctio

ns/

sec)

The throughput and response-time for a single

Linux host running 4Kcontainers

Higher is better

Lower is better

0

1000

2000

LinuxONE Haswell

late

ncy

(m

sec)

Docker is a registered trademarks of Docker, Inc. in the United

States and/or other countries

Disclaimer:

This claim is based on results from internal lab measurements. Performance results may vary depending on the workload and other factors.Benchmark:

o Apache Solr search queries driven by Apache Jmeter*System Stack:

o LinuxONE Emperor (IBM z13): Native LPAR on 36 CPU cores with 755GB memory

o Haswell-based alternative system (Lenovo System x3650 M5 w/ E5-2699 v3 processors): Native Linux on 36 CPU cores with 755GB memory

o Heavy Docker Container: Apache Solr v4.10.0, WebSphere Liberty v8.5.5.2, IBM Java 1.8.0 SR1

o Lightweight Docker Container: BusyBox

o System SW: Docker 1.10.0-dev** w/ aufs storage backend, RHEL 7.1***

Note:

• Each active container is driven by a client thread in Apache Jmeter, which keeps sending the same Solr query repeatedly to the container to search documents that contain given key words in a pre-loaded & pre-indexed 46GB Wikipedia snapshot. ** The docker runtime was modified to increase a thread count limit, to avoid connection time-out, and to separate a dockerinit binary from a docker binary.*** A modified Linux 4.3.0 kernel to support more than 1024 network bridge ports was installed on RHEL 7.1.

Page 30: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

39

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

Everything should be made as simple as possible,

but no simpler

– Albert Einstein

Page 31: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

40

IBM Systems Technical Events | ibm.com/training/events© Copyright IBM Corporation 2017. Technical University/Symposia materials may not be reproduced in whole or in

part without the prior written permission of IBM.© Copyright IBM Corporation 2017. Technical University/Symposia materials may

not be reproduced in whole or in part without the prior written permission of IBM.

OCI Membership

https://www.opencontainers.org/

Page 32: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

www.share.org/sanan

tonio-evalhttp://creativecommons.org/licenses/by-nc-nd/3.0/

Questions?

IBM Deutschland Research

& Development GmbH

Schönaicher Strasse 220

71032 Böblingen, Germany

Office: +49 (0)7031-16-3796

[email protected]

Wilhelm Mild

IBM Executive IT Architect

42

Page 33: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

www.share.org/sanan

tonio-evalhttp://creativecommons.org/licenses/by-nc-nd/3.0/

The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM

Trademarks, see www.ibm.com/legal/copytrade.shtml:

IBM, the IBM logo, BladeCenter, Calibrated Vectored Cooling, ClusterProven, Cool Blue, POWER, PowerExecutive, Predictive Failure Analysis, ServerProven,

System p, System Storage, System x , z Systems, WebSphere, DB2 and Tivoli are trademarks of IBM Corporation in the United States and/or other countries. For a

list of additional IBM trademarks, please see http://ibm.com/legal/copytrade.shtml.

The following are trademarks or registered trademarks of other companies: Java and all Java based trademarks and logos are trademarks of Sun Microsystems,

Inc., in the United States and other countries or both Microsoft, Windows,Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in

the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep,

Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries or both. Linux is a trademark of Linus Torvalds in the United States,

other countries, or both. Cell Broadband Engine is a trademark of Sony Computer Entertainment Inc. InfiniBand is a trademark of the InfiniBand Trade Association.

Other company, product, or service names may be trademarks or service marks of others.

NOTES: Linux penguin image courtesy of Larry Ewing ([email protected]) and The GIMP

Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many

factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on

development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Users of this document should verify the

applicable data for their specific environment. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our

warranty terms apply.

Information is provided “AS IS” without warranty of any kind. All customer examples cited or described in this presentation are presented as illustrations of the

manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics

will vary depending on individual customer configurations and conditions.

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the

information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products

and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be

addressed to the suppliers of those products.

Prices are suggested US list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features.

Contact your IBM representative or Business Partner for the most current pricing in your geography. Any proposed use of claims in this presentation outside of the

United States must be reviewed by local IBM country counsel prior to such use. The information could include technical inaccuracies or typographical errors.

Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements

and/or changes in the product(s) and/or the program(s) described in this publication at any

Trademarks & Disclaimer

Page 34: Docker demystified for Mainframersvmworkshop.org/pres2017/dokrtech.pdf · Docker host: the (virtual) server where a Linux guest and Docker is running Docker Hub: share container images

www.share.org/sanan

tonio-evalhttp://creativecommons.org/licenses/by-nc-nd/3.0/

Any information contained in this document regarding Specialty Engines (“SEs”) and SE eligible workloads provides only general

descriptions of the types and portions of workloads that are eligible for execution on Specialty Engines (e.g., zIIPs, zAAPs, and IFLs).

IBM authorizes customers to use IBM SE only to execute the processing of Eligible Workloads of specific Programs expressly

authorized by IBM as specified in the “Authorized Use Table for IBM Machines” provided at

www.ibm.com/systems/support/machine_warranties/machine_code/aut.html (“AUT”).

No other workload processing is authorized for execution on an SE.

IBM offers SEs at a lower price than General Processors/Central Processors because customers are authorized to use SEs only to

process certain types and/or amounts of workloads as specified by IBM in the AUT.

Notice Regarding Specialty Engines