sf bay area kubernetes meetup dec8 2016 - deployment models
TRANSCRIPT
Copyright © 2015 Mirantis, Inc. All rights reserved
www.mirantis.com
Kubernetes deployment models(modelling complex applications in K8S)
Dec8 2016Piotr Siwczak(https://www.linkedin.com/in/psiwczak)
Copyright © 2015 Mirantis, Inc. All rights reserved
Agenda
● Challenges in managing complex microservice architectures
● What’s missing in K8S to manage complex microservice architectures efficiently
● K8S AppController as an enhancement to handle complex architectures
● Demo of AppController
● Q&A about AppController
Copyright © 2015 Mirantis, Inc. All rights reserved
Challenges in managing microservice architectures
Copyright © 2015 Mirantis, Inc. All rights reserved
Linux as a microservice apps platform
Collection of small, independent programs acting together to form larger systems.
Programs communicate over standardized protocols/API-s
Abstracts computing resources (kernel)
Copyright © 2015 Mirantis, Inc. All rights reserved
Unix/Linux simplified architecture
Kernel
Libraries
Init system Interactive shell
Userspace apps
Resource access
Orchestration
User functionality
Copyright © 2015 Mirantis, Inc. All rights reserved
The role of init system
SysVInit … Upstart … Systemd
Init makes sure that apps start in proper order and deps for them are handled
e.g.
Network subsystem -> Iptables -> ssh
Copyright © 2015 Mirantis, Inc. All rights reserved
Apps - combined K8S resources
Kubernetes clients (kubectl...)
Kubernetes resources (pod, service…)
Unix/Linux vs K8S
Kernel
Libraries
Init system Interactive shell
Userspace apps
?
Copyright © 2015 Mirantis, Inc. All rights reserved
Do we have init equivalent in k8s?
Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”
“Containers start in parallel after volumes are
mounted, leaving no opportunity for
coordination between containers...”
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md
Copyright © 2015 Mirantis, Inc. All rights reserved
...to make a good dish one needs to follow steps
Original photo by https://www.flickr.com/people/waferboard/ License: CC BY v2.0, Title: “wholesome stirfry”
● heat oil● add garlic● add veggies and sauce● add meat
Copyright © 2015 Mirantis, Inc. All rights reserved
K8S challenge for complex apps
(T3) wordpress depl/service
(T2) mysql depl/service
(T1) mysql password
password
db dns name & password
kubectl create -f mysql-pass.yaml
kubectl create -f mysql-deployment.yaml
kubectl create -f wordpress-deplyment.yaml
Copyright © 2015 Mirantis, Inc. All rights reserved
Is lack of deps really a problem for K8S?
We all know microservices are supposed to orchestrate themselves and tolerate failures
...but…
Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://blog.xebialabs.com/2015/04/13/before-you-go-over-the-container-cliff-with-docker-mesos-etc-points-to-consider/
“"A common definition for a microservice we often hear mentioned is an “independently-deployable unit”, and indeed it is good practice to design your
microservices so they can start up successfully without requiring all kinds of other components to be available. But in the vast majority of cases, “no microservice is an
island”...
Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
Docker-compose and Mesos application groups are here and being used:
https://docs.docker.com/compose/gettingstarted/
https://mesosphere.github.io/marathon/docs/application-groups.html
Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://github.com/vishnubob/wait-for-it
Copyright © 2015 Mirantis, Inc. All rights reserved
...deps seem to be needed
https://github.com/mesosphere/kubernetes-mesos/issues/119
http://stackoverflow.com/questions/27701994/specify-order-dockers-run-on-kubernetes-pod
https://github.com/kubernetes/kubernetes/issues/29804
Copyright © 2015 Mirantis, Inc. All rights reserved
https://github.com/Mirantis/k8s-AppController
AppController
Copyright © 2015 Mirantis, Inc. All rights reserved
● way to express dependencies between K8S objects,
● thus allowing to deploy complex, multi-tier applications in fully automated fashion
● k8s object dependency graph:● definitions (nodes)● dependencies (edges)
AppController - really short summary...
Copyright © 2015 Mirantis, Inc. All rights reserved
Before…
● kubectl create -f t1.yaml● check status…● kubectl create -f t2.yaml● check status…● kubectl create -f t3.yaml● ….
After…
● kubectl create -f graph.yaml
● k8s-appcontroller ac-run
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController architecture
Kubernetes
k8s-appcontroller pod
kubeac binary k8s API extensions
3rd party resources:dependency
definition
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - definitions
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
kubectl create -f definitions.yaml
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - definitions
apiVersion: appcontroller.k8s/v1alpha1
kind: Definition
metadata:
name: secret-mysql-pass
secret:
apiVersion: v1
data:
password.txt: cXdxd3F3
kind: Secret
metadata:
creationTimestamp: 2016-12-06T16:56:02Z
name: mysql-pass
namespace: default
secret/mysql-pass
Standard K8S resource
(secret)
Objects are not created in k8s until triggered by AppController!
Copyright © 2015 Mirantis, Inc. All rights reserved
Definitions - summary
Definition:
● “node” in the graph● wrapper over regular k8s resource● defers the creation of the resource until triggered (in
contrary to “kubectl create -f” which creates the resource immediately
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - dependencies
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
start end
kubectl create -f deps.yaml
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - dependencies
apiVersion: appcontroller.k8s/v1alpha1
kind: Dependency
metadata:
name: mysql-pass--to--mysql-deployment
parent: secret/mysql-pass
child: deployment/mysql
secret/mysql-pass
Standard K8S resource
(secret)
deployment/mysql
Standard K8S resource
(deployment)
Copyright © 2015 Mirantis, Inc. All rights reserved
Dependencies - summary
Dependency:
● “edge” in the graph● links definitions together● provides the sense of dependency between definitions
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController - application rollout
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
Definition
Standard K8S resource
(deployment, pod, service…)
start end
kubectl exec k8s-appcontroller ac-run
kubectl exec k8s-appcontroller kubeac get-status
Copyright © 2015 Mirantis, Inc. All rights reserved
AppController workflow - summary
● “wrap” regular k8s resources into definitions > defs.yaml● load defs.yaml to k8s
● create dependencies between resources > deps.yaml● load deps.yaml into k8s
● trigger the app deployment from AppController application (kubectl exec k8s-appcontroller ac-run)
Copyright © 2015 Mirantis, Inc. All rights reserved
def_db_password
AppController - wordpress deployment
secret: db_password
def_db_deploymtdeployment:
mysql-deployment
def_db_service
service:db_service
def_wp_deploymt
deployment:wordpress-dep
loyment
def_wp_service
service:wordpress-ser
vice
DB_PASS
DB_HOSTNAME, DB_PASS
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q: What if I abort the deployment in the middle - how does AppController recover
from partially provisioned graph?
A:AppController will check the status of already provisioned resources. Will only
provision the ones which are absent
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:Can I run multiple AppControllers on a single K8S ?
A:You can run 1 AppController per namespace
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:What K8S resources can be currently wrapped into definitions?
A:- Daemonset
- Job- Petset- Pod
- Replicaset- Service
- ConfigMap- Secrets
- Deployments
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:How is provisioning of resources validated?
A:Status of the k8s resource is checked.
AppController implements also some checks of its own (e.g. for replicasets readiness probe is based on “success factor” or all resources ready. Success
factor is a part of appcontroller and for services we are checking service selector and see if the backends are ready - e.g. replica sets)
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:What’s on the roadmap?
A:Graph notifications, reactions, error handling
More resources supported
Usability improvements
Better documentation (incl. real-life complex examples)
Copyright © 2015 Mirantis, Inc. All rights reserved
Q&A
Q:How AppController is different from Init Containers?
A:Supports more complex deployments (complex graphs)
Handles deps not only between containers
Checks resource states (no need to implement custom probes in the container)
Keeps debug logs in one place
Can react to changes in the graph
Copyright © 2015 Mirantis, Inc. All rights reserved
Recording
https://www.youtube.com/watch?v=7GSwSTtBAYo&utm_content=38600000