load balancing in the cloud using nginx & kubernetes

Load-Balancing inthe Cloud using

Lee Calcote http://calcotestudios.com/talks

Nginx Kubernetes&

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

blog.gingergeek.com

[email protected]

clouds, containers, infrastructure,applications and their management

Private

Public

Application DeliveryControllers

Load-Balancershave evolved.

@lcalcote

Our Bloat-a-lithA 1GB jar!

Our Case Study

is an IDE for game developers created by game development studiobased in Austin creates a real-time, collaborativegame development engine

“Google Docs for game development”

@lcalcote

Vert.x

Our Bloat-a-lith

EventBus Handler

Persistence Verticle

SockJS

Analytics

Analytics UI

Hazelcast

GridConnection

Telemetry

Telemetry Analysis

Projects, Streams,

Users, Sessions,

Tenants

Kafka Producer

Platform Client

Producer

Spark, R

GCE Network

Load Balancer

Kafka

Rendering Engines

Platform Publishing

Tool

Game Analysis

Single instance

@lcalcote

beloved

Shaping Up

Faster delivery, rolling updates

Horizontal scale out on-demand;

on an individual service basis

Modular architecture

Easy integration and deployments

Service isolation, resilience and fail-safe recovery

Benefits of Microservices

Democratization of language and technology choice

@lcalcote

a quick review

it's an excellent time to be a developer

The promise of...

Characteristics of Microservices

how small is small?

Who has a system that istoo big and that you’d

like to break down?Can a small team manage it?

More moving partsincreases complexity.

Can you make a change to a serviceand deploy it by itself without

changing anything else?

@lcalcote

small, autonomous services that work together.

independent, autonomous serviceself-contained functional unit

App is Reactive

Leverages sockets

Limited resources

Culture (DevOps / Cloud maturity)

Support containers AND VMs

Propagation of huge data sets

The ChallengeOn-the-fly intelligent rendering

Distribution of content

Extremely low latency for Reactive services

Blocking vs non-blocking

On-premises telemetry collection and

analytics

@lcalcote

Our Microbloat v2

Analytics UI

Authentication

Telemetry

Telemetry Analysis

Spark, R

Kafka

Rendering Engines

Platform Publishing

Tool

Game Analysis

Masteretcd

Locking

Projects

Authorization

kube-proxy kube-proxy

SockJS

Authorization

kube-proxy

DaemonSet

Node

Node

kube-api, etc.

@lcalcote

Node

Pod

Pod

Deployment

Authorization

DaemonSet

Service

ComparingServices

(not shoes)

SSL Termination

@lcalcote

Kubernetes 1.5 No

Swarm 1.13 No

Mesos+Marathon Yes

ELB Classic Yes

ELB L7 Yes

Beanstalk Yes (EC2)

IOT Yes

ECS Yes (EC2)

Load-Balancer No

App Gateway Yes

Container Service ?

Cloud LB (HTTP) Yes

Cloud LB (Network) Yes

GKE No

Container

Orchestrators

Clouds

AWS

Azure

GCP

Link

LinkLink

Link

Link

Link

Link

Link

Link

SSL Proxy

SSL Proxy

Websocket Support

Kubernetes 1.5 No

Swarm 1.13 No

Mesos+Marathon Yes

Container

Orchestrators

Clouds

AWS

Azure

GCP

Link

Link

Link

Link

Link

Link

Link

Link

Link

@lcalcote

ELB Classic Yes

ELB L7 Yes

Beanstalk Yes

IOT Yes

ECS Yes

Load-Balancer No

App Gateway Yes

Container Service ?

Cloud LB (HTTP) No

Cloud LB (Network) Yes

GKE No

Kubernetes & Nginxto the Rescue

@lcalcote

There are soooo many ways to skin this cat.

Microbloat v3Going deeper with Nginx & Kubernetes

As an ingress controller in Kubernetes

SSL termination

Path-based rules

Web socket support

@lcalcote

Service Discovery with Nginx Plus

Need for locating service instances

instantly without reconfiguring

On-the-fly Reconfiguration API

Work with etcd

Kubernetes

- group of co-scheduled containers and volumes Replication Controller - reconciliation loop to keep currentstate congruent with desired state

- a set of pods that comprise a common function

- manages updates for Pods and Replica Sets - store and retrieve sensitive data

ConfigMap

Pod

Service

DeploymentSecrets

\ˈnō-mən-ˌklā-chəra brief Kubernetes construct review

@lcalcote

Exposing Kubernetes Services

- service to be reachable only from inside ofthe cluster.

- It serves as a way to return an alias toan external service residing outside the cluster.

- exposes service on a port on each node ofthe cluster.

- cluster-internal IP and exposing serviceon a NodePort, also ask the cloud provider for a loadbalancer which forwards requests to the Serviceexposed as a <NodeIP>:NodePort for each Node.

ClusterIP

ExternalName

NodePort

LoadBalancer

@lcalcote

runs on each node in the cluster a network proxy that represents Services on each node integral to how services are exposed in the cluster limited to layer 4 (tcp/udp) load-balancing

kube-proxy

kube-proxyno·men·cla·ture

@lcalcote

kube-proxy

iptables

Container AA

Container A


Node A Node B

Client

Pod A

Service A

iptables

Inbound

Outbound

NodePort

@lcalcote

Traffic flow withNodePort/LoadBalancer

Container BB

Container B

Pod B

Service B

An is a collection of rulesthat allow inbound connections to

reach the cluster services. - howyou expose and route to the

.

Ingress

service

Ingressno·men·cla·ture

@lcalcote

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: projects spec: tls: - hosts: - api.maxplay.io secretName: api-secret rules: - host: api.maxplay.io http: paths: - path: /projects backend: serviceName: tenant-svc servicePort: 80 - path: /tenants backend: serviceName: user-svc servicePort: 80 - path: /users backend: serviceName: user-svc servicePort: 80

Secrets

$ kubectl create secret generic api-secret --from-file nginx.conf

@lcalcote

An is acontrol loop that manages rules

enabling inbound traffic toapplications.

Ingress Controller

Ingress Controllerspecification

@lcalcote

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

name: nginx-ingress-dp

labels:

app: nginx-ingress

spec:

replicas: 1

selector:

app: nginx-ingress

template:

metadata:

labels:

app: nginx-ingress

spec:

containers:

- image: maxplay/nginx-ingress:latest

imagePullPolicy: Always

name: nginx-ingress

ports:

- containerPort: 80

hostPort: 80

- containerPort: 443

hostPort: 443

args:

- /nginx-ingress-controller

- --default-backend-service= \

$(POD_NAMESPACE)/nginx-default-backend

Make sure you review controller specific docs so you understand the caveats of eachone.

iptables

Container AA

Container AIngress

Controller


Node A Node B

Client

Pod AIngress B

Service A

iptables

Inbound

Outbound

@lcalcote

Traffic flow withIngress Controller

Annotation in the Ingressresource definition specifies

which services are web socketservices -

"socks-svc"

Web Socket SupportIngress Resource Annotation

@lcalcote

apiVersion: extensions/v1beta1 kind: Ingress metadata: name: api-ingress annotations: nginx.org/websocket-services: "sockjs-svc" spec: tls: - hosts: - api.maxplay.io secretName: api-secret rules: - host: api.maxplay.io http: paths: - path: /sockjs backend: serviceName: sockjs-svc servicePort: 8181

Why Nginx

1. Hybrid needsOn-premises and Google Cloud Platform

2. Consistent administration and capabilitiescentral load balancing and proxy platform

3. Support for VM and container-based technologieswith minimal configuration change

4. Deeper feature set available as services/team maturesuse as an Application Delivery Controller

Platform independence

@lcalcote

Common Administration w/Nginx Plus

App

App App

@lcalcote

A/B Testing Using NginxIngress Controller and Deployments

Requests

App v0.1

App v0.2

90% of requests go to v0.1

10% of requests go to v0.2

@lcalcote

GSLB & Content Cachingw/Nginx Plus

App

App

AppGeoDNSUS

Regions

Session Persistence and Sticky Routing help in performance of requestrouting and localized contentContent Caching provides faster retrieval of dataPerformance, reliability and availability

GlobalRegions

@lcalcote

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

blog.gingergeek.com

[email protected] you.Questions?

clouds, containers, infrastructure,applications and their management

http://calcotestudios.com/ talks

load balancing in the cloud using nginx & kubernetes

Technology