stackengine problem space demo
TRANSCRIPT
Managing Docker Applications in Production
Boyd E. Hemphill - @behemphi Mike Raab - @stackengine
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged), Evangelist
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
History - BoydOracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
Working with Docker on the first Docker Days Austin
History - Boyd
Oracle Developer, MySQL DBA, PHP Developer, System Architect, DevOps Director (snickering encouraged)
Started Austin DevOps In 2012
Co-Organizer of DevOps Days Austin
Became involved in Docker Austin in August.
Working with Docker for the first Docker Days Austin
Keep an eye on @stackengine to see if we can make it happen!
History -StackEngine
CopperEgg
History -StackEngine
CopperEgg
Started in May 2014 to address challenges facing Docker adopters
History -StackEngine
CopperEgg
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
History -StackEngine
CopperEgg
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
Currently in private beta
History -StackEngine
CopperEgg
Started in May 2014 to address challenges facing Docker adopters
Funding announcement October
Currently in private beta
Common Ground
What is The Goal of Your
Company?
What is The Goal of Your
Company?Make Money!
So … What is DevOps?
Is DevOps a Process?
Is it an intersection of overlapping concerns?
Is DevOps a Culture?
DevOps:
DevOps is the way in which a technology organization embeds itself in a business to the benefit of that business.
The Ideal Docker Application
Netflix (unicorn)Service Oriented Architecture
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
BenefitsFeature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
Density - Reduce Costs
Benefits
Feature velocity - Increase developer productivity using containers to model application topology
Innovation velocity - Decreased build and test times make more time for trying that next great idea!
Density - Reduce Costs
Separation of Concerns - “Code to an interface” moves to the infrastructure level
Benefits
Taken together a company that can get to the end state has a significant and lasting competitive advantage
Today’s Docker Reality
Concerns
This is still very early days for Docker
Concerns
This is still very early days for Docker
Tools to support Dockerized applications are immature or do not exist
Concerns
This is still very early days for Docker
Tools to support Dockerized applications immature or do not exist
Service oriented architecture is hard
Concerns
This is still very early days for Docker
Tools to support Dockerized applications immature or do not exist
Service oriented architecture is hard
Increasing developer throughput is only a local optima, thus a false economy
Concerns
Docker adoption is the first major DevOps initiative not driven by Ops.
Dev Driven DevOps
DevOps is hallmarked by Ops adopting the best practices of developers, servers as code, check in all the things, test driven infrastructure
Dev Driven DevOps
DevOps is hallmarked by Ops adopting the best practices of developers, servers as code, check in all the things, test driven infrastructure
The Docker Revolution is the first mass DevOps initiative that harkens from Developers applying ops thinking
Dev Driven DevOps
There is a huge social engineering effort along side the tech
Dev Driven DevOps
There is a huge social engineering effort along side the tech
DEFN: The application of the findings of social science to the solution of actual social problems.
SDLC
Developers are moving forward
SDLC
Developers are moving forward
QA sees production modeling, large test grids, more effective integration testing
SDLC
Developers are moving forward
QA sees production modeling, large test grids, more effective integration testing
Build engineers see parallelized build grids for faster feedback and better build quality
xkcd … er … SDLCOperations panics - no tools or instrumentation
The Final Mile
The Final Mile
Developer uses containers
The Final Mile
Developer uses containers
System is built and images are produced
The Final Mile
Developer uses containers
System is built and images are produced
Images stored in a private registry
The Final Mile
Developer uses containers
System is built and images are produced
Images stored in a private registry
Now we need to deploy to a QA environment
The Final Mile
So we build a manual solution depending on human behavior
The Final Mile
So we build a manual solution depending on human behavior
Someone builds a tool
Docker DeploymentCreate a component to represent your registry
Docker DeploymentCreate a component to represent the image you would like to run
Docker DeploymentDistribute that image and run it on the hosts in your Mesh.
Docker DeploymentMagic happens.
Docker DeploymentLife is good
Deployment
StackEngine is developing the tooling to make this policy based
Docker DeploymentFluffy bunnies and rainbows ensue
Container Sprawl
Container Sprawl
First there was VMware and suddenly a large corps departments could easily spin up environments
Container Sprawl
First there was VMware and suddenly a large corps departments could easily spin up environments
Then there was the cloud and developers could complete circumvent Ops
Container SprawlFirst there was VMware and suddenly a large corps departments could easily spin up environments
Then there was the cloud and developers could complete circumvent Ops
Now there is Docker and Developers are creating numerous abstractions that are not managed like traditional VMs
Container Sprawl
Containers on VMs on phyical hardware located in availability zones in data centers in geographic regions.
My Head HurtsContainer Sprawl
Container Sprawl
StackEngine is working to make this policy based
Inventory Management
Where’s My StuffHosts
Where’s My StuffImages
Where’s My StuffContainers
Where’s My StuffGeography, Data Center, Zone
Telemetry
Telemetry
What does monitoring a docker app look like?
Telemetry
What does monitoring a docker app look like?
http://goo.gl/VkAQo4
What does log aggregation look like?
Telemetry
What does monitoring a docker app look like?
http://goo.gl/VkAQo4
What does log aggregation look like?
http://goo.gl/uNPe9P
What’s my stuff (not) doing
I have this much stuff.
What’s my stuff (not) doing
How are the containers on a host doing
What’s my stuff (not) doing
What is the container state?
What’s my stuff (not) doingMore Location, Host and Container metrics coming!
Components
Components
Components are infrastructure abstractions (cloud provider, data center, docker registry, docker image)
Components
Components are infrastructure abstractions (cloud provider, data center, docker registry, docker image)
Components can be arranged in logic ways to represent physical infrastructure.
Composing an AppWhere are your components coming from
Composing and AppWhat makes the application
Composing an AppStart the db
Composing and AppStart the web server
Composing and AppRun the application server
Composing and AppRun the Cache
Service Discovery
Service Discovery - Consistency Models
Zookeeper
Etcd / Consul
Chef Server / Heira
Service Discovery
Allows multiple deployments of the same service on the same host.
Service Discovery
Allows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
Service Discovery
Allows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
We favor something like consul with a search interface like the chef server.
Service DiscoveryAllows multiple deployments of the same service on the same host.
Containers register themselves upon startup and are found by tags rather than URI’s, fixed ports, etc.
We favor something like consul with a search interface like the chef server.
A step towards cattle containers instead of pet containers
Roadmap
Roadmap
Service Discovery
Roadmap
Service Discovery
User Management
Roadmap
Service Discovery
User Management
Addition of new Components
Roadmap
Service Discovery
User Management
Addition of new Components
Better telemetry
Roadmap
Service Discovery
User Management
Addition of new Components
Better telemetry
Scheduling
Roadmap
Service Discovery
User Management
Addition of new Components
Better telemetry
Scheduling
A/B Deployment support
Live Product DemoAl Gore and his gremlins willing
Boyd E. Hemphill - @behemphi Mike Raab - @stackengine
Thank You for Your Time and Comments.