scaling docker containers using kubernetes and azure container service

Scaling Docker Containers using Kubernetes

and Azure Container Service

@Ben_HallBen@BenHall.me.ukKatacoda.com

Scaling Docker Containers using Kubernetes

and Azure Container Service

@Ben_HallBen@BenHall.me.ukKatacoda.com

WH

O A

M I?

Learn via Interactive Browser-Based LabsKatacoda.com

Agenda

• Why containers?

• Building Containerized ASP.NET Core app

• Deploying Containers on Kubernetes and Azure

• Advantages of Kubernetes

What if we could remove configuration complexity, dependency conflicts and

uncertainty?

> docker run –p 6379:6379 redis:3.0.3_.-``__ ''-._

_.-`` `. `_. ''-._ Redis 3.0.3 (00000000/0) 64 bit

.-`` .-```. ```\/ _.,_ ''-._

( ' , .-` | `, ) Running in standalone mode

|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379

| `-._ `._ / _.-' | PID: 1

`-._ `-._ `-./ _.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' | http://redis.io

`-._ `-._`-.__.-'_.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' |

`-._ `-._`-.__.-'_.-' _.-'

`-._ `-.__.-' _.-'

`-._ _.-'

`-.__.-'

1:M 05 Nov 10:42:24.402 # Server started, Redis version 3.0.3

1:M 05 Nov 10:42:24.402 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition.

To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl

vm.overcommit_memory=1' for this to take effect.

1:M 05 Nov 10:42:24.402 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will

> docker run -e 'ACCEPT_EULA=Y’ \

-e 'SA_PASSWORD=yourStrong133tPassword' \

-p 1433:1433 microsoft/mssql-server-linux

This is an evaluation version. There are [154] days left in the evaluation period.

2017-06-09 22:16:14.27 Server Setup step is copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf'.

2017-06-09 22:16:14.36 Server Setup step is copying system data file 'C:\templatedata\mastlog.ldf' to '/var/opt/mssql/data/mastlog.ldf'.

2017-06-09 22:16:14.37 Server Setup step is copying system data file 'C:\templatedata\model.mdf' to '/var/opt/mssql/data/model.mdf'.

2017-06-09 22:16:14.40 Server Setup step is copying system data file 'C:\templatedata\modellog.ldf' to

'/var/opt/mssql/data/modellog.ldf'.

2017-06-09 22:16:14.42 Server Setup step is copying system data file 'C:\templatedata\msdbdata.mdf' to

'/var/opt/mssql/data/msdbdata.mdf'.

2017-06-09 22:16:14.45 Server Setup step is copying system data file 'C:\templatedata\msdblog.ldf' to '/var/opt/mssql/data/msdblog.ldf'.

2017-06-09 22:16:14.57 Server Microsoft SQL Server 2017 (CTP2.1) - 14.0.600.250 (X64)

May 10 2017 12:21:23

Copyright (C) 2017 Microsoft Corporation. All rights reserved.

Developer Edition (64-bit) on Linux (Ubuntu 16.04.2 LTS)

2017-06-09 22:16:14.58 Server UTC adjustment: 0:00

2017-06-09 22:16:14.58 Server (c) Microsoft Corporation.

2017-06-09 22:16:14.58 Server All rights reserved.

2017-06-09 22:16:14.58 Server Server process ID is 4116.

Own Process Space

Own Network Interface

Own Root Directories

Sandboxed

It’s not a VM!

Container

Native CPU

Native Memory

Native IO

No Pre-Allocation

No Performance Overheard

Container

Milliseconds to launch

Docker – “Kernel Virtualisation”

Three Key Concepts

• Docker Containers – Running Processes

• Docker Images – “Layered Zip Files”

• Docker Registry – Where Images are stored

Docker isn’t the only container runtime!

Open Container Initiative• Driven by Linux Foundation

• Docker Containers === Runtime Specification

• Docker Images === Image Specification

• CRI-O === Kubernetes Container Runtime

> docker run -e 'ACCEPT_EULA=Y’ \

-e 'SA_PASSWORD=yourStrong133tPassword' \

-p 1433:1433 microsoft/mssql-server-linux

This is an evaluation version. There are [154] days left in the evaluation period.

2017-06-09 22:16:14.27 Server Setup step is copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf'.

2017-06-09 22:16:14.36 Server Setup step is copying system data file 'C:\templatedata\mastlog.ldf' to '/var/opt/mssql/data/mastlog.ldf'.

2017-06-09 22:16:14.37 Server Setup step is copying system data file 'C:\templatedata\model.mdf' to '/var/opt/mssql/data/model.mdf'.

2017-06-09 22:16:14.40 Server Setup step is copying system data file 'C:\templatedata\modellog.ldf' to

'/var/opt/mssql/data/modellog.ldf'.

2017-06-09 22:16:14.42 Server Setup step is copying system data file 'C:\templatedata\msdbdata.mdf' to

'/var/opt/mssql/data/msdbdata.mdf'.

2017-06-09 22:16:14.45 Server Setup step is copying system data file 'C:\templatedata\msdblog.ldf' to '/var/opt/mssql/data/msdblog.ldf'.

2017-06-09 22:16:14.57 Server Microsoft SQL Server 2017 (CTP2.1) - 14.0.600.250 (X64)

May 10 2017 12:21:23

Copyright (C) 2017 Microsoft Corporation. All rights reserved.

Developer Edition (64-bit) on Linux (Ubuntu 16.04.2 LTS)

2017-06-09 22:16:14.58 Server UTC adjustment: 0:00

2017-06-09 22:16:14.58 Server (c) Microsoft Corporation.

2017-06-09 22:16:14.58 Server All rights reserved.

2017-06-09 22:16:14.58 Server Server process ID is 4116.

> docker run -it microsoft/azure-cli bashd5f51519a9b1:/# azure

info: _ _____ _ ___ ___

info: /_\ |_ / | | | _ \ __|

info: _ ___/ _ \__/ /| |_| | / _|___ _ _

info: (___ /_/ \_\/___|\___/|_|_\___| _____)

info: (_______ _ _) _ ______ _)_ _

info: (______________ _ ) (___ _ _)

info:

info: Microsoft Azure: Microsoft's Cloud Platform

info:

info: Tool version 0.10.11

Building ContainerisedASP.NET Core app

> cat Program.cs

namespace dotnetapp {

public class Program {

public static void Main(string[] args) {

var host = new WebHostBuilder()

.UseKestrel()

.UseContentRoot(Directory.GetCurrentDirectory())

.UseUrls("http://0.0.0.0:5000")

.UseIISIntegration()

.UseStartup<Startup>()

.Build();

host.Run();

}

}

}

> cat Startup.cs

namespace dotnetapp {

public class Startup {

public void Configure(IApplicationBuilderapp, IHostingEnvironment env, ILoggerFactoryloggerFactory) {

loggerFactory.AddConsole();

app.Run(async (context) => {

await context.Response.WriteAsync("Request processed by " + System.Environment.MachineName);

});

}

}

}

$ ls

.git Controllers Program.cs dotnetapp.csproj

Dockerfile Makefile Startup.cs wwwroot

$ cat Dockerfile

FROM microsoft/dotnet:1.1.1-sdk

$ cat Dockerfile

FROM microsoft/dotnet:1.1.1-sdk

RUN mkdir /app

WORKDIR /app

COPY dotnetapp.csproj /app/

RUN dotnet restore

$ cat Dockerfile

FROM microsoft/dotnet:1.1.1-sdk

RUN mkdir /app

WORKDIR /app

COPY dotnetapp.csproj /app/

RUN dotnet restore

COPY . /app

RUN dotnet publish -c Release -o out

$ cat Dockerfile

FROM microsoft/dotnet:1.1.1-sdk

RUN mkdir /app

WORKDIR /app

COPY dotnetapp.csproj /app/

RUN dotnet restore

COPY . /app

RUN dotnet publish -c Release -o out

EXPOSE 5000/tcp

CMD ["dotnet", "out/dotnetapp.dll"]

$ docker build -t katacoda/dotnet-example:v1 .Sending build context to Docker daemon 137.7 kB

Step 1/11 : FROM microsoft/dotnet:1.1.1-sdk

Step 2/11 : RUN mkdir /build

Step 3/11 : WORKDIR /build

Step 4/11 : COPY dotnetapp.csproj .

Step 5/11 : RUN dotnet restore

Step 6/11 : COPY . .

Step 7/11 : RUN dotnet publish -c Release -o out

---> Running in fcde25425eee

Microsoft (R) Build Engine version 15.1.548.43366

Copyright (C) Microsoft Corporation. All rights reserved.

dotnetapp -> /build/bin/Release/netcoreapp1.1/dotnetapp.dll

Step 8/11 : FROM microsoft/dotnet:1.1.1-runtime

Step 9/11 : WORKDIR /app

Step 10/11 : CMD dotnet dotnetapp.dll

Step 11/11 : COPY --from=0 /build/out /app/

Successfully built 1dbec44d4150

Successfully tagged katacoda/dotnet-example:v1.1

$ docker run -d \

-t -p 5000:5000 \

--name app \

katacoda/dotnet-example:v1

$ curl dockerhost:5000

Request processed by zb8gh24wva

CI/CD Pipeline

Git PushGitlab

Starts Builddockerbuild

docker run

> cat Dockerfile

FROM node:6

RUN mkdir -p /usr/src/app

WORKDIR /usr/src/app

COPY package.json /usr/src/app

RUN npm install

COPY . /usr/src/app

CMD [ "npm", "start" ]

> docker build –t nodeapp .

> docker run –d –p 3000 nodeapp

Example – ASP.NETFROM microsoft/dotnet:1.1.1-sdkWORKDIR /app

# copy csproj and restore as distinct layersCOPY dotnetapp.csproj /app/RUN dotnet restore

COPY . /app/RUN dotnet publish -c Release -o out

CMD ["dotnet", "out/dotnetapp.dll"]

Deploying Build Tools

REPOSITORY TAG IMAGE ID CREATED SIZE

katacoda/dotnet-example v1 b8f8b523d3ca

6 minutes ago 894.7 MB

Example – ASP.NETFROM microsoft/dotnet:1.1.1-runtimeWORKDIR /appCOPY out /appENTRYPOINT ["dotnet", "dotnetapp.dll"]

# First StageFROM microsoft/dotnet:1.1.1-sdkWORKDIR /appCOPY dotnetapp.csproj /app/RUN dotnet restoreCOPY . /app/RUN dotnet publish -c Release -o out

# Second StageFROM microsoft/dotnet:1.1.1-runtimeWORKDIR /appCMD ["dotnet", "dotnetapp.dll”]COPY --from=build out /app/

Optimised Image

REPOSITORY TAG IMAGE ID CREATED SIZE

katacoda/dotnet-example v1 d69cf725c406

5 seconds ago 266.3 MB

katacoda/dotnet-example v1 b8f8b523d3ca

6 minutes ago 894.7 MB

Production?

Docker Push / Pull> docker push katacoda/dotnet-example:v1

The push refers to a repository [docker.io/katacoda/dotnet-example]

3ed827b6362a: Pushing [=====> ] 1.637 MB/14.81 MB

79d391888f28: Pushed

2bd3dca2fc5a: Mounted from microsoft/dotnet

ecb9a3d4923d: Mounted from microsoft/dotnet

6c1558b80cc8: Mounted from microsoft/dotnet

8d4d1ab5ff74: Mounted from microsoft/dotnet

> docker pull katacoda/dotnet-example:v1

> docker run katacoda/dotnet-example:v1

Docker Push / Pull> docker push katacoda/dotnet-example:v1.1

The push refers to a repository [docker.io/katacoda/dotnet-example]

5ed6d70495ef: Pushing [========> ] 2.479 MB/14.81 MB

79d391888f28: Layer already exists

2bd3dca2fc5a: Layer already exists

ecb9a3d4923d: Layer already exists

6c1558b80cc8: Layer already exists

8d4d1ab5ff74: Layer already exists

> docker pull katacoda/dotnet-example:v1.1

> docker run katacoda/dotnet-example:v1.1

Public Docker Registry

CI/CD Pipeline

Git PushGitlab

Starts Builddockerbuild

Gitlab Start Release

dockerpush

docker run

docker pull

> docker login katacodademoreg1.azurecr.io

Username: katacodademoreg1

Password:

Login Succeeded

> docker build –t katacodademoreg1.azurecr.io/katacoda/dotnet-

example:v1.1 .

> docker push katacodademoreg1.azurecr.io/katacoda/dotnet-

example:v1.1

“docker run” in production?

Container Orchestration

Kubernetes is an open-

source system for

automating deployment,

scaling, and management

of containerized

applications.

http://queue.acm.org/detail.cfm?id=2898444

Kubernetes

Automatic binpacking

Self-healing

Horizontal scaling

Service discovery and load balancing

Automated rollouts

Secret and configuration management

Batch execution

Role based access control

Auditing

API and Extension Hooks

> az login

> az account set --subscription "8640e4e6-"

> az group create -n "demo-k8s" -l "westeurope"

> az ad sp create-for-rbac --role="Contributor" --

scopes="/subscriptions/8640e4e6/resourceGroups/demo-k8s"

Kubernetes in 1 command…

OK, it’s three…

> DNS_PREFIX=some-unique-value

> CLUSTER_NAME=any-acs-cluster-name

> az acs create --orchestrator-type=kubernetes \

--resource-group $RESOURCE_GROUP \

--name=$CLUSTER_NAME --dns-prefix=$DNS_PREFIX \

--generate-ssh-keys

> az acs kubernetes install-cli

> az acs kubernetes get-credentials \

--resource-group=$RESOURCE_GROUP \

--name=$CLUSTER_NAME

> ssh katacoda@52.174.195.10

or install client tools (kubectl) locally…

$ kubectl get nodesNAME STATUS AGE VERSION

k8s-agent-4c8a65bf-0 Ready 9m v1.6.2

k8s-master-4c8a65bf-0 Ready,SchedulingDisabled 9m v1.6.2

$ kubectl run dotnetapp \

--image katacoda/dotnet-example:v1

deployment "dotnetapp" created

$ kubectl run dotnetapp --image katacoda/dotnet-example:v1

deployment "dotnetapp" created

$ kubectl get deploymentsNAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

dotnetapp 1 1 1 1 5m

$ kubectl get podsNAME READY STATUS RESTARTS AGE

dotnetapp-2582115574-b59rg 1/1 Running 0 20s

Schedule Workload

Find Available Node

Allocate Workload

Start Container

$ kubectl expose deployments dotnetapp \

--port=80 --target-port=5000 \

--type=LoadBalancerservice "dotnetapp" exposed

$ kubectl get svcNAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

dotnetapp 10.0.47.252 <pending> 80:31078/TCP 8s

kubernetes 10.0.0.1 <none> 443/TCP 15m

$ kubectl expose deployments dotnetapp --port=80 --target-port=5000 --type=LoadBalancer

service "dotnetapp" exposed

$ kubectl get svc

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

dotnetapp 10.0.47.252 <pending> 80:31078/TCP 8s

kubernetes 10.0.0.1 <none> 443/TCP 15m

wait for Kubernetes to configure Azure LB

$ kubectl get svc

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

dotnetapp 10.0.47.252 40.118.100.56 80:31078/TCP 5m

curl 40.118.100.56

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

$ kubectl scale --replicas=3 deployment/dotnetappdeployment "dotnetapp" scaled

$ kubectl get deployments

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

dotnetapp 3 3 3 3 6m

$ kubectl get pods

NAME READY STATUS RESTARTS AGE

dotnetapp-2582115574-0l028 1/1 Running 0 1m

dotnetapp-2582115574-b59rg 1/1 Running 0 7m

dotnetapp-2582115574-hdbc6 1/1 Running 0 1m

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-0l028

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-hdbc6

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

What did we do?$ kubectl run dotnetapp --image katacoda/dotnet-example:v1

$ kubectl expose deployments dotnetapp \

--port=80 --target-port=5000 \

--type=LoadBalancer

$ kubectl get svc

$ kubectl scale --replicas=3 deployment/dotnetapp

What happens if an agent goes down?

$ kubectl get nodes

NAME STATUS AGE VERSION

k8s-agent-4c8a65bf-0 Ready 34m v1.6.2

k8s-agent-4c8a65bf-1 NotReady 15s v1.6.2

k8s-master-4c8a65bf-0 Ready,SchedulingDisabled 34m v1.6.2

wait for node to be configured…

$ kubectl get nodes

NAME STATUS AGE VERSION

k8s-agent-4c8a65bf-0 Ready 34m v1.6.2

k8s-agent-4c8a65bf-1 Ready 2m v1.6.2

k8s-master-4c8a65bf-0 Ready,SchedulingDisabled 34m v1.6.2

$ kubectl scale --replicas=6 deployment/dotnetapp

$ kubectl get deployment

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

dotnetapp 6 6 6 3 23m

wait for containers to be downloaded…

$ kubectl get deployment

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE

dotnetapp 6 6 6 3 24m

Deploy new versions?

> kubectl set image \

deployment/dotnetapp=katacoda/dotnet-example:v1.1> curl 40.118.100.56

[v1.1] Request processed by dotnetapp-2582115574-0l028

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-b59rg

> curl 40.118.100.56

Request processed by dotnetapp-2582115574-hdbc6

> curl 40.118.100.56

[v1.1] Request processed by dotnetapp-2582115574-b59rg

What about the Azure Registry Service?

> kubectl create secret docker-registry myregistrykey \

--docker-server=katacodademoreg1.azurecr.io \

--docker-username=katacodademoreg1 \

--docker-password=<REMOVE> \

--docker-email ben@benhall.me.uk

secret "myregistrykey" created.

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

labels:

run: dotnetapp

spec:

replicas: 6

strategy:

rollingUpdate:

maxSurge: 1

maxUnavailable: 1

type: RollingUpdate

template:

spec:

containers:

- image: katacodademoreg1.azurecr.io/katacoda/dotnet-example:v1.1

imagePullSecrets:

- name: myregistrykey

metadata:

labels:

run: dotnetapp

Access to API and Event Loop

Abstraction

Storage

> cat sqlserver.yaml

apiVersion: v1

kind: Secret

metadata:

name: azure-secret

type: Opaque

data:

azurestorageaccountname: <name>

azurestorageaccountkey: <key>

---

apiVersion: v1

kind: Pod

metadata:

labels:

name: mssql

role: master

name: mssql

spec:

containers:

- env:

- name: ACCEPT_EULA

value: "Y"

- name: SA_PASSWORD

value: yourStrong133tPassword

image: microsoft/mssql-server-linux

volumeMounts:

- mountPath: /var/opt/mssql/data

name: azure

volumes:

- azureFile:

readOnly: false

secretName: azure-secret

shareName: k8stest

name: azure

Is it still up?

Monitoring with Prometheus and Kubernetes

apiVersion: extensions/v1beta1

kind: DaemonSet

metadata:

name: node-exporter

spec:

template:

metadata:

labels:

app: node-exporter

name: node-exporter

spec:

containers:

- image: prom/node-exporter

name: node-exporter

ports:

- containerPort: 9100

hostPort: 9100

name: scrape

DaemonSet? When new nodes are deployed, automatically schedule

workload

What should we monitor?

apiVersion: v1

kind: Service

metadata:

annotations:

prometheus.io/scrape: 'true'

labels:

app: node-exporter

name: node-exporter

name: node-exporter

spec:

clusterIP: None

ports:

- name: scrape

port: 9100

protocol: TCP

selector:

app: node-exporter

type: ClusterIP

Windows Containers

> type Dockerfile

FROM microsoft/iis:windowsservercore-10.0.14393.693

SHELL ["powershell", "-command"]

RUN Install-WindowsFeature NET-Framework-45-ASPNET; Install-WindowsFeature Web-Asp-Net45

RUN Remove-Website -Name 'Default Web Site'; \

mkdir c:\NerdDinner; \

New-Website -Name 'nerd-dinner' \

-Port 80 -PhysicalPath 'c:\NerdDinner' \

-ApplicationPool '.NET v4.5‘

EXPOSE 80

COPY NerdDinner c:\NerdDinner

PS C:\> docker build –t nerddinner .

PS C:\> docker run -d -p 80:80 \

nerddinner

Microsoft & Red Hat

https://github.com/kubernetes/features/issues/116

The Future?

SQL Server as a Container

Visual Studio as a Container?

Everything as a Container

Deploy Anywhere

www.katacoda.com

Online Kubernetes

Training for Companies

at Katacoda.com

@Ben_Hall

Ben@BenHall.me.uk

Blog.BenHall.me.uk

www.Katacoda.com