docker in prod
TRANSCRIPT
A Little bit about our Production
● High Traffic: 2.5 Billion requests per day
● Maintaining Lots of data
● Data integrity
● Fast growing user base
● Need to deploy new features fast
● Bottom line: We need tools that can help us grow fast!
What this lecture is about and what not
Yes:
✔ Sharing our experience with docker in production ✔ Production deployments and automation with dockers✔ Echo system tools that we integrate with docker✔ Tools we developed locally to ease containers usage✔ The reasons to use or not use docker for specific Application / Infrastructure✔ Where we failed and how to avoid that
No:
✗ Docker/containers basics✗ Testing environments✗ Tutorial✗ Kernel implementations behind the technology✗ Analogs between harbor and docker principles
External tools we use with Docker● Docker registry (internal server)
● Consul – Docker deployment configuration
● Mesos for docker clustering
● Marathon as Mesos API
● Chef for “parent host configuration”
● Custom aws “docker AMI”
Additional tools we use for the Deployment flow:
➔ Jenkins – builds / pull resources from Bitbucket
➔ Rabbitmq – message passing between central deployment and agents
➔ LDAP – Deployment Authentication
➔ Graphite – Containers metrics
Stuff we built internally
● Santa – Docker deployment tool
● Docker log tailer
● Docker metric collection
● Chef – docker protection
● Configuration bootstrap
● Marathon – Consul integration
● What we dockerize and what not, examples and reasons
● When we use Santa and when Mesos
● Best practices in the deployment flow (Auto image creation & store, version management, reverts, configuration set in Consul, Using Docker/Consul/Jenkins API, reviving from failures, feedback loop, etc)
● Viewing containers logs
● Monitoring the containers
Best Practices
Failures/Mistakes & Fixes
● Chef and dockers: Clash of the titans
● Forcing an elephant into a box
● Connection tracking table, mmm...
● Deployment failures: retries & revive
● Volumes hell
Provisioning & Deploy Flow with Mesos – Stage 1Provisioning & Deploy Flow with Mesos – Stage 1Provisioning & Deploy Flow with Mesos – Stage 1
I need x docker instances in AWS I can do that
Now we have Mesos cluster of X instances, each with Chef, Docker Daemon, Mesos slave and Consul agent which serves as discovery source
Provisioning & Deploy Flow with Mesos – Stage 2Please deploy an application to the cluster from the registry
That's fine by me
Provisioning & Deploy Flow with Mesos – Stage 2Please deploy an application to the cluster from the registry
That's fine by me Deploy, deploy, deploy
Also, I will need you to provide me the servers and ports that serve this application, I would like to update my buddy consul
Sure
Here's what Marathon knows, go gossip about it