deploying dc/os on premises with juju

Deploying DC/OS On Prem

(With a little help from your friends)

Who’s the guy at the front?Tom Barber

NASA JPL Devops and Data Developer

Spicule & Meteorite BI

Data guy and open source hacker… erm developer.

A bit of history…...

CaveatWe want to run our services on hardware supplied by

We also want to retain control of the underlying hardware to control costs, networking and scalability.

Our Goal

We also want to be able to run our services in different data centres for

resilience and data locality purposes.

So how can we achieve our goals?

Container Management SystemsDocker Swarm

Kubernetes

Apache Mesos

CoreOS

DC/OS

Deployed to fixed hosts

I’m a sucker for a nice looking dashboard

The next question we asked was:“We have a bunch of Bigstep bare metal servers, what's the most flexible way to manage them?”

Introducing Metal As A Service!Deploy workloads onto bare metal servers

Integration with Juju to manage the services

DNS & IP management

Internal fabric networking

Operating system agnostic

We already manage a lot of our services using Juju… so could we make DC/OS work with Juju?

Application modeling with Juju

Juju Key ConceptsBootstrap

Deploy

Relate

Scale

Repeat!

BootstrapPick a cloud

AWS

Azure

CloudSigma

Google

Joyent

Rackspace

LXD Local

Manual

MAAS!

Deployjuju deploy dcos-master

juju deploy dcos-agent

Relatejuju add-relation dcos-master dcos-agent

Scalejuju add-unit -n 10 dcos-agent

Repeat! (Anywhere)Export my model to a YAML file

Import into a new cloud or MAAS service

Deploys in the same manner

Crowdsourcing OpsMajority of users get Charms from the Juju Charm Store. Similar to

PuppetForge et al

Charms written by a user or company and then added to and extended by others

Charms that have been validated and have a test suite are often promulgated to the top level recommended charms to try and ensure users have a good experience

Development flow to enhance charms

Monitoring DC/OSjuju deploy nagios

juju deploy nrpe

juju add-relation nagios:monitors nrpe:monitors

juju add-relation dcos-master:local-monitors nrpe:local-monitors

Logging DC/OSjuju add-relation logstash dcos-master

(Finally) Deploying Containers to DC/OSjuju deploy dcos-nginx

juju add-relation dcos-nginx dcos-master

Enriching your deployment with MetadataYou can make use of non dockerized or Mesos-ized services from other charms

Passing around metadata allows you to configure applications dynamically even within and across DC/OS core systems

Current State of PlayDCOS Master Nodes (Scalable)

DCOS Client Nodes (Scalable)

Monitoring connectivity

Ability to deploy docker containers

Actions to do various admin tasks

RoadmapImplement disk mounts for NFS and Loopback devices.

Add more actions.

Implement Monitoring.

Improve Security.

Expose running app ports automatically.

Simplify the upgrade process.

ELK to Beats?

Custom Cluster Name

Private Docker Reg

Marathon Load Balancer

Support and Testinginfo@spicule.co.uk <- me

juju@lists.ubuntu.com <- the community

tom@analytical-labs.com <- also me

https://jujucharms.com/u/spicule/basic-dcos-bundle <- a bundle to get you going

https://github.com/buggtb/dcos-master-charm <- the master

https://github.com/buggtb/dcos-agent-charm <- the agents

But it doesn’t end there…..

DC/OS is hard to configure once installedYou can’t easily change from public to private units

You can’t enable or disable OAuth logins

You can’t enable or disable telemetry

Layered MesosMesos master and slaves

Marathon support for Docker deployment

Utilise the Zookeeper charm

SSL Communication

HA support

SDN support