1 6 m ar 2 0 1 9 , f oss as ia, si ng apore f rom m onol ... · f rom m onol it h to m ic ro-s er v...

122
From Monolith to Micro-services with Kubernetes 16 Mar 2019, FOSS Asia, Singapore Michael Bright, @mjbright Slides & source code at https://mjbright.github.io/Talks 1 / 74

Upload: others

Post on 17-Jan-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

From Monolith to Micro-services with Kubernetes16 Mar 2019, FOSS Asia, Singapore

Michael Bright, @mjbrightSlides & source code at https://mjbright.github.io/Talks 1 / 74

Page 2: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Michael Bright, @mjbright

Freelance Consultant & Trainer on CloudNative Solutions

Past researcher, dev, team lead, dev advocate

British, living in France for 27-years

Docker Community Lead, Python User Group

linkedin.com/in/mjbright github.com/mjbright

2 / 74

Page 3: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Outline[Why?] Monoliths to Micro-services

Orchestration: Kubernetes

Deployment Strategies

Architecture Design patterns

Summary

@mjbright 3 / 74

Page 4: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Outline[Why?] Monoliths to Micro-services

Orchestration: Kubernetes

Deployment Strategies

Architecture Design patterns

Summary

@mjbright 4 / 74

Page 5: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

First ... a bit of history

@mjbright 5 / 74

Page 6: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

First ... a bit of history

Note: The future will be hybrid ... (technologies, providers, on-prem/cloud ...)

@mjbright 6 / 74

Page 7: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

[Why?] Monoliths to Micro-servicesTraditionally software has been delivered as large packages which can only bedeployed, scaled, upgraded, reimplemented as a whole.

web . . . .

Problem: A paradigm ill-adapted to enterprise or web-scale

@mjbright 7 / 74

Page 8: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

[Why?] Monoliths to Micro-servicesTraditionally software has been delivered as large packages which can only bedeployed, scaled, upgraded, reimplemented as a whole.

web . . . .

Problem: A paradigm ill-adapted to enterprise or web-scale

Tightly-coupled components exist as a unit, are difficult to reuseWaterfall release cycles make software difficult to patchDifficult to innovate due to slow release cycles

@mjbright 7 / 74

Page 9: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Monoliths to Micro-servicesMicro-services use small loosely-coupled software components

Individual components can be deployed, scaled, upgraded, replaced ...

web

Micro-service architecture components are lightly-coupled

interconnected by networkcan be scaled independentlycan be deployed/upgraded independently

@mjbright 8 / 74

Page 10: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

@mjbright 9 / 74

Page 11: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

Smaller focussed Projects/teams

@mjbright 9 / 74

Page 12: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

Smaller focussed Projects/teams

Ease Scaling, Deployment, Testing, Evolution

@mjbright 9 / 74

Page 13: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

Smaller focussed Projects/teams

Ease Scaling, Deployment, Testing, Evolution

Allow for composition of new services

@mjbright 9 / 74

Page 14: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

Smaller focussed Projects/teams

Ease Scaling, Deployment, Testing, Evolution

Allow for composition of new services

Can be re-implemented with "Best in class" tech

@mjbright 9 / 74

Page 15: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Advantages of Micro-servicesSeparation of Concerns: "do one thing well"

Smaller focussed Projects/teams

Ease Scaling, Deployment, Testing, Evolution

Allow for composition of new services

Can be re-implemented with "Best in class" tech

So are they a panacea?

@mjbright 9 / 74

Page 16: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

DisadvantagesGreater complexity

Require orchestration, and rigorous component version managementNeed to evolve to greater organizational complexityMonitoring, debugging, end-2-end test are more difficult

@mjbright 10 / 74

Page 17: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

DisadvantagesGreater complexity

Require orchestration, and rigorous component version managementNeed to evolve to greater organizational complexityMonitoring, debugging, end-2-end test are more difficult

Network communication is criticalNeed good error handling, Performance, Circuit-breakers

@mjbright 10 / 74

Page 18: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

DisadvantagesGreater complexity

Require orchestration, and rigorous component version managementNeed to evolve to greater organizational complexityMonitoring, debugging, end-2-end test are more difficult

Network communication is criticalNeed good error handling, Performance, Circuit-breakers

Useless without adopting best practicesBehaviour and Test-Driven Development, CI/CDRequire rigorous documentation of interfaces/APIsStable APIs and backward-compatibility support

@mjbright 10 / 74

Page 19: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Outline[Why?] Monoliths to Micro-services

Orchestration: Kubernetes

Deployment Strategies

Architecture Design patterns

Summary

@mjbright 11 / 74

Page 20: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration: KubernetesProblem: As our systems scale it becomes impossible to manage 1000's ofdiverse containers running across a data center of 100's of nodes.

on which nodes should you schedule?to ensure availabilityto satisfy affinity, non-affinity constraintsto take advantage of specialized h/w

@mjbright 12 / 74

Page 21: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration: KubernetesProblem: As our systems scale it becomes impossible to manage 1000's ofdiverse containers running across a data center of 100's of nodes.

on which nodes should you schedule?to ensure availabilityto satisfy affinity, non-affinity constraintsto take advantage of specialized h/w

which containers are malfunctioning?

@mjbright 12 / 74

Page 22: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration: KubernetesProblem: As our systems scale it becomes impossible to manage 1000's ofdiverse containers running across a data center of 100's of nodes.

on which nodes should you schedule?to ensure availabilityto satisfy affinity, non-affinity constraintsto take advantage of specialized h/w

which containers are malfunctioning?which are started and ready to go?

@mjbright 12 / 74

Page 23: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration: KubernetesProblem: As our systems scale it becomes impossible to manage 1000's ofdiverse containers running across a data center of 100's of nodes.

on which nodes should you schedule?to ensure availabilityto satisfy affinity, non-affinity constraintsto take advantage of specialized h/w

which containers are malfunctioning?which are started and ready to go?how to easily upgrade applications?

@mjbright 12 / 74

Page 24: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration: KubernetesProblem: As our systems scale it becomes impossible to manage 1000's ofdiverse containers running across a data center of 100's of nodes.

on which nodes should you schedule?to ensure availabilityto satisfy affinity, non-affinity constraintsto take advantage of specialized h/w

which containers are malfunctioning?which are started and ready to go?how to easily upgrade applications?how to auto-scale applications?

@mjbright 12 / 74

Page 25: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

We need Orchestration

@mjbright13 / 74

Page 26: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Orchestration Feature Wish-listHealth checks - to Verify when a task is ready to accept trafficDynamic port-mapping - Ports are assigned dynamically when a newcontainer is spun upZero-downtime deployments - Deployments do not disrupt end usersService discovery - Automatic detection of new containers and servicesAuto scaling - Automatically scale resources up or down based on theload

Provisioning - New containers should select hosts based on resources andconfiguration

Other - Load balancing, logging, monitoring, authentication andauthorization, security... predictability, scalability, and high availability...

@mjbright 14 / 74

Page 27: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Architecture

WorkerNode 1

Master Node

WorkerNode 2

WorkerNode 3

@mjbright 15 / 74

Page 28: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Master Nodes

etcd Master

MasterMaster

API

GUI(dashboard)

CLI(kubectl)

APIServer

Scheduler Controller

@mjbright 16 / 74

Page 29: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Worker Nodes

Pod

Kubelet ContainerEngine

kube-proxy

flatnetwork

Pod

Pod

PodPodPod

dashboardkube-dnsAdd-ons

@mjbright 17 / 74

Page 30: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Pods

same ip, e.g. 192.168.1.20

Containers share some namespaces: - PID, IPC, network , time sharing

A pod houses one or more containers

SidecarMain container

@mjbright 18 / 74

Page 31: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes Demo

Master Node"Worker"

Docker Desktop tainted single-node

Flask1 Flask2 Flask3

Redis

@mjbright 19 / 74

Page 32: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Redis

kubectl create -f redis-deployment.yaml deployment

ReplicaSet

Pod12e76: redis

@mjbright 20 / 74

Page 33: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Redis

@mjbright 21 / 74

Page 34: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Redis (yaml)

@mjbright 22 / 74

Page 35: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Flask

kubectl create -f flask-deployment.yaml deployment

ReplicaSet

Pod21f3d: flask:v1

Pod12e76: flask:v1

@mjbright 23 / 74

Page 36: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Flask# kubectl run flask-app --image=$IMAGE --port=5000

$ kubectl apply -f flask-deployment.yamldeployment.extensions "flask-app" created

$ kubectl get podsNAME READY STATUS RESTARTS AGEflask-app-8577b44db-96cht 0/1 Pending 0 1sredis-68595c4d95-rr4pr 0/1 ContainerCreating 0 1s

@mjbright 24 / 74

Page 37: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Deploying Flask (yaml)apiVersion: extensions/v1beta1kind: Deploymentmetadata: labels: run: flask-app name: flask-appspec: replicas: 1 selector: matchLabels: run: flask-app template: metadata: labels: run: flask-app spec: containers: - image: mjbright/flask-web:v1 name: flask-app ports: - containerPort: 5000

@mjbright 25 / 74

Page 38: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Scaling# kubectl scale deploy flask-app --replicas=4

$ kubectl edit -f flask-deploy.yaml

...spec: replicas: 4

@mjbright 26 / 74

Page 39: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Scaling Flask (yaml)apiVersion: extensions/v1beta1kind: Deploymentmetadata: labels: run: flask-app name: flask-appspec: replicas: 4 selector: matchLabels: run: flask-app template: metadata: labels: run: flask-app spec: containers: - image: mjbright/flask-web:v1 name: flask-app ports: - containerPort: 5000

@mjbright 27 / 74

Page 40: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Scaling Flask$ kubectl apply -f flask-deployment-r4-v1.yamldeployment.extensions "flask-app" created

$ kubectl get podsNAME READY STATUS RESTARTS AGEflask-app-8577b44db-96cht 1/4 Pending 0 1hredis-68595c4d95-rr4pr 1/1 Running 0 1h

@mjbright 28 / 74

Page 41: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Outline[Why?] Monoliths to Micro-services

Orchestration: Kubernetes

Deployment Strategies

Architecture Design patterns

Summary

@mjbright 29 / 74

Page 42: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Deployment StrategiesProblem: How can we simply/automatically upgrade micro-services ?

across a data center

in the cloud

@mjbright 30 / 74

Page 43: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Deployment StrategiesProblem: How can we simply/automatically upgrade micro-services ?

across a data center

in the cloud

Solution: Several deployment strategies exist

Some strategies can be implemented by Kubernetes alone

Some strategies must be handled by external routing

@mjbright 30 / 74

Page 44: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Micro-service Deployment StrategiesService Upgrade Strategies

Health Checks

Strangler Pattern - migration pattern

@mjbright 31 / 74

Page 45: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesSeveral strategies existRef: Kubernetes deployment strategies, Container Solutions, github

recreate   - terminate old version before releasing new one

@mjbright 32 / 74

Page 46: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesSeveral strategies existRef: Kubernetes deployment strategies, Container Solutions, github

recreate   - terminate old version before releasing new one

ramped   - gradually release a new version on a rolling update fashion

@mjbright 32 / 74

Page 47: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesSeveral strategies existRef: Kubernetes deployment strategies, Container Solutions, github

recreate   - terminate old version before releasing new one

ramped   - gradually release a new version on a rolling update fashion

blue/green   - release new version alongside old version then switch

@mjbright 32 / 74

Page 48: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesSeveral strategies existRef: Kubernetes deployment strategies, Container Solutions, github

recreate   - terminate old version before releasing new one

ramped   - gradually release a new version on a rolling update fashion

blue/green   - release new version alongside old version then switch

canary   - release new version to subset of users, proceed to full rollout

@mjbright 32 / 74

Page 49: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesSeveral strategies existRef: Kubernetes deployment strategies, Container Solutions, github

recreate   - terminate old version before releasing new one

ramped   - gradually release a new version on a rolling update fashion

blue/green   - release new version alongside old version then switch

canary   - release new version to subset of users, proceed to full rollout

a/b testing   - release new version to subset of users in a precise way(HTTP headers, cookie, weight, etc.).

@mjbright 32 / 74

Page 50: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Service Upgrade StrategiesRamped# kubectl set image deploy flask-app flask-app=mjbright/flask-web:v2

$ kubectl edit -f flask-deploy.yaml$ kubectl rollout status deployment/flask-app

... spec: containers: - image: mjbright/flask-web:v2

@mjbright 33 / 74

Page 51: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Demo

Master Node"Worker"

Docker Desktop tainted single-node

Flask1 Flask2 Flask3

Redis

@mjbright 34 / 74

Page 52: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Containers - Are you healthy, ready ?Problem: But how can the system determine if a Service is healthy andavailable

We'd like the system to not route traffic to unhealthy service instances.

@mjbright 35 / 74

Page 53: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Containers - Are you healthy, ready ?Problem: But how can the system determine if a Service is healthy andavailable

We'd like the system to not route traffic to unhealthy service instances.

Kubernetes Healthchecks (Liveness and Readiness probes) provide asolution.

Ref: Kubernetes Liveness, Readiness Probes Documentation

Liveness probe can be used to force re-creation of blocked image

Readiness probe can be used to await startup

@mjbright 35 / 74

Page 54: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Healthchecks

Liveness probesThis probe is used to establish if the container is healthy

(or blocked, unable to progress).

The probe can specify

A command to executeAn http request to tryA TCP request to try

@mjbright 36 / 74

Page 55: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Healthchecks

Liveness probesThis probe is used to establish if the container is healthy

(or blocked, unable to progress).

The probe can specify

A command to executeAn http request to tryA TCP request to try

Readiness probesOnce started the container still needs time before being able to accepttraffic

This probe tests the readiness to receive and process requests

Probe types are as for Liveness probes

@mjbright 36 / 74

Page 56: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Liveness probesapiVersion: v1kind: Podmetadata: labels: test: liveness name: liveness-execspec: containers: - name: liveness image: k8s.gcr.io/busybox args: - /bin/sh - -c - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 livenessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5

@mjbright 37 / 74

Page 57: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Operations - Readiness probesIt is sufficient to replace 'livenessProbe:' by 'readinessProbe:' in the yaml

readinessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5

@mjbright 38 / 74

Page 58: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

How to Migrate to Micro-services ?Problem: We may not have the luxury of a Greenfield deployment !!

So how can we migrate an existing Monolith to Micro-services ?

@mjbright 39 / 74

Page 59: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

How to Migrate to Micro-services ?Problem: We may not have the luxury of a Greenfield deployment !!

So how can we migrate an existing Monolith to Micro-services ?

It's a monolith after all !

@mjbright 39 / 74

Page 60: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

How to Migrate to Micro-services ?Problem: We may not have the luxury of a Greenfield deployment !!

So how can we migrate an existing Monolith to Micro-services ?

It's a monolith after all !

Do we wait 6 months before having a new implementation

(*with no extra features!*) ?

@mjbright 39 / 74

Page 61: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

How to Migrate to Micro-services ?Problem: We may not have the luxury of a Greenfield deployment !!

So how can we migrate an existing Monolith to Micro-services ?

It's a monolith after all !

Do we wait 6 months before having a new implementation

(*with no extra features!*) ?

The Strangler Pattern provides a possible solution.

@mjbright 39 / 74

Page 62: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Migration - Strangler PatternThe Strangler is a pattern used in the initial migration from a Monolithicarchitecture to a Micro-services architecture

Ref: Azure Docs - "Strangler pattern"

@mjbright 40 / 74

Page 63: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Micro-service - Architecture Design PatternsHere, we are not concerned with:

Standard Component Design Patterns

Micro-services themselves (!) - Fine-grained SOA

Sidecar

@mjbright 41 / 74

Page 64: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Micro-service - Architecture Design PatternsWe are concerned with:

Exposing Services

Ingressproviding access to the Kubernetes cluster ...

@mjbright 42 / 74

Page 65: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Micro-service - Architecture Design PatternsWe are concerned with:

Exposing Services

Ingressproviding access to the Kubernetes cluster ...

and ways of providing offload-functionality

API Gateway

Service Mesh

Hybrid Apps - "API Gateway Pattern"

@mjbright 42 / 74

Page 66: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Micro-service - Architecture Design PatternsWe are concerned with:

Exposing Services

Ingressproviding access to the Kubernetes cluster ...

and ways of providing offload-functionality

API Gateway

Service Mesh

Hybrid Apps - "API Gateway Pattern"Note: This is the new war-zone as API Gateways battle it out, Service Meshesbattle it out and both battle it out!

@mjbright 42 / 74

Page 67: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

@mjbright 43 / 74

Page 68: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

@mjbright 43 / 74

Page 69: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

@mjbright 43 / 74

Page 70: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

@mjbright 43 / 74

Page 71: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

What happens if a Pod dies ... it just might happen ;-)

@mjbright 43 / 74

Page 72: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

What happens if a Pod dies ... it just might happen ;-)

(it's a joke: it will happen)

@mjbright 43 / 74

Page 73: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

What happens if a Pod dies ... it just might happen ;-)

(it's a joke: it will happen)

Also - we don't want to expose our infrastructure details !!

@mjbright 43 / 74

Page 74: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

What happens if a Pod dies ... it just might happen ;-)

(it's a joke: it will happen)

Also - we don't want to expose our infrastructure details !!

Also - they should be on isolated networks

@mjbright 43 / 74

Page 75: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Accessing our ServicesProblem: We've deployed, scaled & upgraded Services across our Cluster

But how do we access those services ?

We can access the Pods/containers directly at their IP and port addresses

Don't !! - they are ephemereal

What happens if a Pod dies ... it just might happen ;-)

(it's a joke: it will happen)

Also - we don't want to expose our infrastructure details !!

Also - they should be on isolated networks

So we provide well-known endpoints to reliably/safely expose services

@mjbright 43 / 74

Page 76: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Kubernetes - Exposing ServicesThe general pattern is to provide a cluster-wide, well-known endpoint whichremains available as Pods come and go

Master User

WorkerWorker

podpod

podpod

ServiceIP:port

@mjbright 44 / 74

Page 77: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - ServicesServices can be exposed via

NodePort

HostPort

ClusterIP

LoadBalancer

@mjbright 45 / 74

Page 78: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (NodePort)

Master User

WorkerWorker

podpod

podpod

IP:port

User connects

to IP/port of one

of the NodesIP:port

Service

@mjbright 46 / 74

Page 79: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (LoadBalancer)

Master User

Worker

podpod

IP:port ExternalLoad Balancer

Worker

podpod

Service

@mjbright 47 / 74

Page 80: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (IngressController)

Master User

Worker

podpod

Ingressroute

Worker

podpod

Service

@mjbright 48 / 74

Page 81: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Redis Service (LoadBalancer)# kubectl expose deployment redis --type=LoadBalancer

$ kubectl apply -f redis-service.yamlservice "redis" created

$ kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5hredis LoadBalancer 10.101.158.201 <pending> 6379:31218/TCP 1s

@mjbright 49 / 74

Page 82: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Redis Service (LoadBalancer)apiVersion: v1kind: Servicemetadata: labels: run: redis name: redisspec: ports: - port: 6379 protocol: TCP targetPort: 6379 selector: run: redis type: LoadBalancer

@mjbright 50 / 74

Page 83: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Flask Service (LoadBalancer)# kubectl expose deployment flask-app --type=LoadBalancer

$ kubectl apply -f flask-service.yamlservice "flask-app" created

$ kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEflask-app LoadBalancer 10.103.154.19 <pending> 5000:32201/TCP 1skubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5hredis LoadBalancer 10.101.158.201 <pending> 6379:31218/TCP 2s

@mjbright 51 / 74

Page 84: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Flask Service (LoadBalancer)apiVersion: v1kind: Servicemetadata: labels: run: flask-app name: flask-appspec: ports: - port: 5000 protocol: TCP targetPort: 5000 selector: run: flask-app type: LoadBalancer

@mjbright 52 / 74

Page 85: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - IngressIngress is the general term for controlling incoming traffic

(and *Egress* is the term for *outgoing* traffic)

@mjbright 53 / 74

Page 86: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - IngressIngress is the general term for controlling incoming traffic

(and *Egress* is the term for *outgoing* traffic)

In the context of Kubernetes it refers to the ability (limited feature set) tocontrol incoming traffic. See Kubernetes Docs - Ingress

@mjbright 53 / 74

Page 87: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - IngressIngress is the general term for controlling incoming traffic

(and *Egress* is the term for *outgoing* traffic)

In the context of Kubernetes it refers to the ability (limited feature set) tocontrol incoming traffic. See Kubernetes Docs - Ingress

A set of Ingress Rules is specified to be implemented by a KubernetesController which typically implements Load Balancer, Gateway features.

There are many projects providing such controller functionality such as Nginx,HAproxy, Ambassador, Gloo, Traefik

@mjbright 53 / 74

Page 88: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (Ingress)$ minikube addons enable ingressingress was successfully enabled

$ kubectl apply -f misc/ingress-definition.yaml ingress.extensions "ingress-definitions" created

$ sudo vi /etc/hosts...192.168.99.100 minikube.test flaskapp.test

@mjbright 54 / 74

Page 89: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (Ingress)apiVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-definitions annotations: nginx.ingress.kubernetes.io/rewrite-target: /spec: backend: serviceName: default-http-backend servicePort: 80 rules: - host: minikube.test http: paths: - path: / backend: serviceName: k8sdemo servicePort: 8080 - host: flaskapp.test http: paths: - path: /flask backend: serviceName: flask-app servicePort: 5000

@mjbright 55 / 74

Page 90: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (Ingress)$ minikube service list|-------------|----------------------|-----------------------------|| NAMESPACE | NAME | URL ||-------------|----------------------|-----------------------------|| default | flask-app | http://192.168.99.100:32201 || default | k8sdemo | http://192.168.99.100:31280 || default | redis | http://192.168.99.100:31218 || kube-system | kubernetes-dashboard | http://192.168.99.100:30000 ||-------------|----------------------|-----------------------------|

$ curl http://192.168.99.100:31280

$ curl http://minikube.test/k8sdemo

@mjbright 56 / 74

Page 91: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Exposing Services (Ingress)$ minikube service list|-------------|----------------------|-----------------------------|| NAMESPACE | NAME | URL ||-------------|----------------------|-----------------------------|| default | flask-app | http://192.168.99.100:32201 || default | k8sdemo | http://192.168.99.100:31280 || default | redis | http://192.168.99.100:31218 || kube-system | kubernetes-dashboard | http://192.168.99.100:30000 ||-------------|----------------------|-----------------------------|

$ curl http://192.168.99.100:32201[flask-app-8577b44db-kbwpn] Redis counter value=214

$ curl http://flaskapp.test/flask[flask-app-8577b44db-kbwpn] Redis counter value=215

@mjbright 57 / 74

Page 92: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - API GatewayRef: "What is an API Gateway?"

Classic API Gateways date back to Web Service (SOAP APIs) which offloadedIngress functions into a single system.

API Gateways are API proxies between the client (API consumer) and server(API Provider).

API Security

API Control and governance

API Monitoring

API Administration

API Transformation: See "API Gateway Pattern"

@mjbright 58 / 74

Page 93: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - API Gateway

Service 1(REST/http)

API Gateway

Service2(JSON-RPC)

Service3(WAMP)

https://api.com/....

External entrypoint exposesAPIs

Offloads common Ingress functions => reduces µ-service complexity

rate limiting, security, authorisation, DDOS protectionProtocol version translation, e.g. REST to SOAP, *-RPC ...TLS decryption/encryption

Hides internal infrastructure detail => controls access

service routing, load-balancingAllows to refactor/scale/mock internal implementation

@mjbright 59 / 74

Page 94: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - API Gateway

Service 1(REST/http)

API Gateway

Service2(JSON-RPC)

Service3(WAMP)

https://api.com/....

External entrypoint exposesAPIs

Offloads common Ingress functions => reduces µ-service complexity

rate limiting, security, authorisation, DDOS protectionProtocol version translation, e.g. REST to SOAP, *-RPC ...TLS decryption/encryption

Hides internal infrastructure detail => controls access

service routing, load-balancingAllows to refactor/scale/mock internal implementation

Needs to scale, be H.A.

@mjbright 59 / 74

Page 95: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - API GatewayThere are many API Gateways including

NGInx, HA-Proxy,

Newer generation: Envoy-based such as Ambassador, Gloo

@mjbright 60 / 74

Page 96: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - API GatewayThere are many API Gateways including

NGInx, HA-Proxy,

Newer generation: Envoy-based such as Ambassador, Gloo

But can API Gateways resist the pressure coming from the next contender ...

@mjbright 60 / 74

Page 97: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshProblem: Micro-services are fine, but we see the need for common functions

Logging and tracingReliable network communicationEncryption betweem components

@mjbright 61 / 74

Page 98: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshProblem: Micro-services are fine, but we see the need for common functions

Logging and tracingReliable network communicationEncryption betweem components

BUT if every micro-service reimplements the same functionalities we will getmicro-monoliths !!

@mjbright 61 / 74

Page 99: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshProblem: Micro-services are fine, but we see the need for common functions

Logging and tracingReliable network communicationEncryption betweem components

BUT if every micro-service reimplements the same functionalities we will getmicro-monoliths !!

The problem is compounded by the polyglot nature of micro-services,requiring good library support for functions

@mjbright 61 / 74

Page 100: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshProblem: Micro-services are fine, but we see the need for common functions

Logging and tracingReliable network communicationEncryption betweem components

BUT if every micro-service reimplements the same functionalities we will getmicro-monoliths !!

The problem is compounded by the polyglot nature of micro-services,requiring good library support for functions

Service Mesh helps to address this issue by offloading such functionality

This keeps our micro-services small and simple.

Offload-functionality is provided through Sidecar containers - not libraries.

@mjbright 61 / 74

Page 101: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshAbstraction above TCP/IP, secure reliable inter-service connectivity.

Platforms such as Linkerd (v2) and Istio (v1) provide offload for µ--services

@mjbright 62 / 74

Page 102: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshAbstraction above TCP/IP, secure reliable inter-service connectivity.

Platforms such as Linkerd (v2) and Istio (v1) provide offload for µ--services

Offloads functionality from services in a distributed way.

@mjbright 62 / 74

Page 103: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshAbstraction above TCP/IP, secure reliable inter-service connectivity.

Platforms such as Linkerd (v2) and Istio (v1) provide offload for µ--services

Offloads functionality from services in a distributed way.

@mjbright 62 / 74

Page 104: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshAbstraction above TCP/IP, secure reliable inter-service connectivity.

Platforms such as Linkerd (v2) and Istio (v1) provide offload for µ--services

Offloads functionality from services in a distributed way.

@mjbright 62 / 74

Page 105: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Design Pattern - Service MeshAbstraction above TCP/IP, secure reliable inter-service connectivity.

Platforms such as Linkerd (v2) and Istio (v1) provide offload for µ--services

Offloads functionality from services in a distributed way.

@mjbright 62 / 74

Page 106: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Hybrid Apps - API Gateway PatternProblem: But wouldn't it be better if we could mix legacy and new paradigms

The Strangler pattern is an option but requires being able to rebuild theoriginal monolith to extract functionality.

It would be useful to be able to add new functionality in a less invasive way.

@mjbright 63 / 74

Page 107: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Hybrid Apps - API Gateway PatternThere is a "API Gateway" pattern whereby the gateway has the ability tounderstand the API protocols.

It may also understand the underlying Infrastructure and Platform APIs.

This allows to perform API translation and routing and really take advantageof the orchestration platforms.

"Microservices Patterns Book

@mjbright 64 / 74

Page 108: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Hybrid Apps - API Gateway PatternGloo allows to route between legacy apps, micro-services and serverlessincrementally adding new functionality.

https://medium.com/solo-io/building-hybrid-apps-with-gloo-1eb96579b070

@mjbright 65 / 74

Page 109: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Hybrid Apps - API Gateway PatternGloo understands the infrastructure on which it is running and the APIs beingused.

Gloo is one of several open source projects from Solo.io to facilitate theadoption of modern paradigms such as Micro-services

Gloo: API GatewaySqoop: Tool for modelling API interactionsSquash: Micro-service debugging tool

@mjbright 66 / 74

Page 110: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Hybrid Apps - API Gateway PatternGloo understands the infrastructure on which it is running and the APIs beingused.

Gloo is one of several open source projects from Solo.io to facilitate theadoption of modern paradigms such as Micro-services

Gloo: API GatewaySqoop: Tool for modelling API interactionsSquash: Micro-service debugging tool

@mjbright 66 / 74

Page 111: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

So API Gateways or Service Mesh ?Service Mesh and API Gateways provide similar functionality

Service Mesh control mainly E-W traffic between micro-servicesAPI Gateway control N-S (Ingress) traffic

@mjbright 67 / 74

Page 112: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

So API Gateways or Service Mesh ?Service Mesh and API Gateways provide similar functionality

Service Mesh control mainly E-W traffic between micro-servicesAPI Gateway control N-S (Ingress) traffic

Service Mesh technology is quickly advancing

May be overkill for some use cases

Istio now includes basic Gateway (N-S) functionality

Service Mesh Vendors say we still need API Gateways for the moment.

Linkerd just received new VC funding

@mjbright 67 / 74

Page 113: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

So API Gateways or Service Mesh ?Service Mesh and API Gateways provide similar functionality

Service Mesh control mainly E-W traffic between micro-servicesAPI Gateway control N-S (Ingress) traffic

Service Mesh technology is quickly advancing

May be overkill for some use cases

Istio now includes basic Gateway (N-S) functionality

Service Mesh Vendors say we still need API Gateways for the moment.

Linkerd just received new VC funding

But, API Gateways will continue to offer advanced functionality for Ingresscontrol.

@mjbright 67 / 74

Page 114: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

So API Gateways or Service Mesh ?Service Mesh and API Gateways provide similar functionality

Service Mesh control mainly E-W traffic between micro-servicesAPI Gateway control N-S (Ingress) traffic

Service Mesh technology is quickly advancing

May be overkill for some use cases

Istio now includes basic Gateway (N-S) functionality

Service Mesh Vendors say we still need API Gateways for the moment.

Linkerd just received new VC funding

But, API Gateways will continue to offer advanced functionality for Ingresscontrol.

Going forward we can expect to see Service Mesh incorporating more andmore Gateway functionality

@mjbright 67 / 74

Page 115: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Outline[Why?] Monoliths to Micro-services

Orchestration: Kubernetes

Deployment Strategies

Architecture Design patterns

Summary

@mjbright 68 / 74

Page 116: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

SummaryMicro-services offer new deployment

possibilities

- with ease of deployment, scaling, upgrading

- facilitate "Best in Class" technology choices/replacements

@mjbright 69 / 74

Page 117: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

SummaryMicro-services offer new deployment

possibilities

- with ease of deployment, scaling, upgrading

- facilitate "Best in Class" technology choices/replacements

BUT moving to µ-services requires

- organizational changes and best practices !

- incremental rollout - small steps / Strangler

- hybrid approaches - old/new, cloud/on-premise, VM/container/µ-service

- o�oad via API Gateway and/or Service Mesh@mjbright 69 / 74

Page 118: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Thank you !From Monologue to Discussions ... ?

Questions ?

Michael Bright, @mjbright

Cloud Native Training (Docker, Kubernetes, Serverless)

linkedin.com/in/mjbright github.com/mjbright

Slides & source code at https://mjbright.github.io/Talks 70 / 74

Page 119: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

SummaryGetting started with KubernetesStart by learning Docker principles

Experiment by Dockerizing some applications

Learn about Container Orchestration

Hands-on with Kubernetes online orMinikube(*)

Kubernetes Visualization with KubeView

https://github.com/mjbright/kubeview

@mjbright 71 / 74

Page 120: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Resources

Download https://github.com/kubernetes/minikube/releasesDocumentation https://kubernetes.io/docs/getting-started-guides/minikube/

Hello Minikubehttps://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/

@mjbrightSlides & source code at https://mjbright.github.io/Talks 72 / 74

Page 121: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Resources - ArticlesMartin Fowler https://martinfowler.com/articles/microservices.htmlMuleSoft, "The top 6Microservices Patterns"

https://www.mulesoft.com/lp/whitepaper/api/top-microservices-patterns

FullStack Python https://www.fullstackpython.com/microservices.html

Idit Levinehttps://medium.com/solo-io/building-hybrid-apps-with-gloo-1eb96579b070

SSolahttps://medium.com/@ssola/building-microservices-with-python-part-i-5240a8dcc2fb

Deploymenthttp://container-solutions.com/kubernetes-deployment-strategies/

@mjbrightSlides & source code at https://mjbright.github.io/Talks 73 / 74

Page 122: 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore F rom M onol ... · F rom M onol it h to M ic ro-s er v ices w it h K u ber netes 1 6 M ar 2 0 1 9 , F OSS As ia, Si ng apore Michael Bright,

Resources - BooksPublisher Title, Author

O'Reilly"Building Microservices", Sam Newman,July 2015

PacktPub"Python Microservices Development",Tarek Ziade, July 2017

kNative - O'Reilly

Istio - Manning

Istio - O'Reilly

Testdriven.io

@mjbrightSlides & source code at https://mjbright.github.io/Talks 74 / 74