mongodb evenings dc: get mean and lean with docker and kubernetes

Getting MEAN and Lean with MongoDB and Kubernetes

MongoDB Evenings DC 20164/12/2016

Golden Age of Software Development

{ }Developer

: )User

DeploymentTesting

www.

Native

About Me

Inovalon - R&D, Enterprise Architecture, Software Dev, Healthcare

AOL Advertising - High volume transactions, Big Data

3 Startups – Healthcare and Financial Services

FIRST Robotics Mentor

https://www.linkedin.com/in/vadimpolyakov

Impedance mismatch metaphor

“Lossless” transmission line

Impedance mismatch

Database impedance mismatch

Existing concept: Database or Object-relational impedance mismatch

ORM Tools/code (Hibernate, Entity Framework)

Database technology

• Object Databases

• Document Databases

Application and database schema are unified

Process impedance mismatch

This talk - tie the stack together

How do we maximally leverage tools like MongoDB to deliver the right thing quickly?

Tie together a “thin line architecture” of tools that support agility and leanness.

11

The stack

Stack Impedance

Client

HTML, CSS, JS

Server

.Net, Java

Database

SQL

Browser(HTML, JS, CSS)

ServerREST API

Data

MEAN - Single Page App

RoutableComponents[Service, contoller, direcrive]

RoutableComponents[Middleware]

JSON

Flexible/ComposableFlexible schemaCollections

JavaScript/TypeScript

JSON and JS Pipeline

URL

View – HTML template

Controller – Manage the view

Service – Manage the model

URL

Business Logic

Injectable Abstract Data Store

MongoDB

Schema – Service granularity

Schema modeling ⇔ Service Design granularity

Overnormalized ⇔ Chatty services or joins

Giant documents ⇔ Network overhead, client resources (mobile)

Gettting the most of MEAN

Adopt “Architecturally evident” coding style

George Fairbanks, AgileRoots 2010

Architectural styles

Components

Layers

Service orientation

Architecturally Evident Project

Beyond development

Dev QA UAT Prod

Docker“Docker containers wrap up a piece of software ... This guarantees that it will always run the same, regardless of the environment it is running in.” (https://www.docker.com/what-docker)

Opinionated configuration of cgroups and namespaces

Management API

Git-like semantics (push, pull, diff…)

Repository of existing images (public, private)

Starts in milliseconds (it’s a process)

Dev QA UAT Prod

A “right” choice

Managed as code – DockerfileSimple and fast to build imagesSimple deploymentRetain prior version images for rollback and quick troubleshooting

Application – multiple containers and nodes

nginxnginx

Static content volume

(HTML, JS, CSS)nginx

Nginx

nginx

APINode

/data/dbvolumenginx

DataMongoDB

nginxnginx

Static content volume

(HTML, JS, CSS)nginx

Nginx

nginx

APINode

/data/dbvolumenginx

DataMongoDB

MongoDB backup

Git Syncnginxnginx

Static content volume

(HTML, JS, CSS)nginx

Nginx

nginx

APINode

/data/dbvolumenginx

DataMongoDB

MongoDB backup

Git Syncnginxnginx

Static content volume

(HTML, JS, CSS)nginx

Nginx

nginx

APINode

/data/dbvolumenginx

DataMongoDB

MongoDB backup

Custer Config Kubernetes

Git Sync

Kubernetes Components - Node

Node - physical resources

Registers with the master

Resource capacity

Labels

- tags used to filter the environmente.g. has kernel config for MongoDB

Kubernetes Components - Pod

Pod

- One or more co-located components with shared context (cgroups, namespaces)

Kubernetes Components - Replication Controller

Replication Controller

- Keep certain number of pod replicas running

Kubernetes Components - Services

Services

- Set of pods

- Typically selected using labels

- Virtual IP

External Services

Headless Service

App deployment Deployment as code

svc-www.jsonrc-www.json

svc-app.jsonrc-app.json

svc-arb.jsonsvc-mongo-db1.jsonsvc-mongo-db2.jsonpod-mongo-db1.jsonpod-mongo-db2.jsonrc-arb.json

www-rc

nginx nginx

www-svc

app-svc

app-rc

app app app

mongo1-svc mongo2-svc arbiter-svc

Arbiter-rc

arbitiermongo-db2mongo-db1

Application upkubectl create –f ./configdir - runs all the applications in the config

Before the demo:• Start cluster,

• Assign Elastic IP to the cluster

• Update kubernetes security group to allow port 80

• Label the nodes

Demo:• Deploy application

• Configure replica set

• Reload nginx config

• Access the web app

After the demo:

• Teardown the cluster, run kube-down

https://youtu.be/rOkKTUdQFGQ

MongoDB on Kubernetes

“Datacenter as a Computer”

Most nodes in the cluster are “cattle” not “pets”

Some nodes are special

Databases and DB servers are special

• Working set, kernel config

• Data is an asset, state of the enterprise

MongoDB on Kubernetes

Replication Controller?

• Restart automatically on failure

• Schedule on multiple nodes

Naked Pods

Services

• Discoverability

• DNS node=mongo-db2arbiter=true

node=mongo-db2arbiter=true

mongo-db1(naked pod)

mongo-db2(naked pod)

mongo-db1-svc(service)

mongo-db2-svc(service)

aribiter-rc

arbiter(pod)

arbiter-svc(service)

MongoDB on Kubernetes: Demo

MongoDB Instance

MongoDB Instance

MongoDB Arbiter

MongoDB Instance

MongoDB Instance

MongoDB Instance

MongoDB Arbiter

Replication Controller

MongoDB Instance

MongoDB Instance

MongoDB Arbiter

Replication Controller

MongoDB Instance

Replication Controller

MongoDB Instance

MongoDB Arbiter

Replication Controller

MongoDB Instance

Service

MongoDB Arbiter

Replication Controller

MongoDB Instance

MongoDB Instance

ServiceService

Service

MongoDB Arbiter

Manual Intervention

https://youtu.be/bFnhl7QbjWg

Golden Age of Software Development

{ }Developer

: )User

DeploymentTesting

www.

Native

Future workEnable experimentation/Blue-Green deployment

http://martinfowler.com/bliki/BlueGreenDeployment.html

Architecting application for canary deployment

Data layer for blue green deployment

Questions

https://www.linkedin.com/in/vadimpolyakov

https://github.com/polyakov