humans by the hundred (devops days ohio)
TRANSCRIPT
Deployment: the early days
Get a few people together in slack/irc/etc.
Merge up the code
Run the tests
Poke at it in stage
Cross your fingers
Things get slower...
Tests take longer to run
More hosts = longer downloads, bounces
More developers = more eyeballs
More features = more code
The Problem, With Math
Assume:Every change has a chance of success: 98%
That means no test failures, no reverts, etc.
Every deploy has a number of changes: n
Any failure in the pipeline invalidates the deploy
Let’s figure out the probability of a
successful deployment: p
The Problem, With Math
Only youp = 98%
You and a friendp = .98 * .98 = 96%
You and nine co-workersp = .98 * .98 * .98 * … * .98 = 82%
This doesn’t scale!
More developers = more changes
More changes = longer deploys
Longer deploys = less time to develop
Less time to develop = slower to iterate
Making it harder to screw up
Write more tests
Write better tests
Get better code reviews
Get better infrastructure
Switch programming languages
Use better tools
The Real World
Testing builds confidence in our changes
Testing does not protect you from failure
Better tools, tests, and infrastructure can
raise our success rates
Service-Oriented Architecture
Large monolith → smaller services
Services communicate over networkUsually HTTP, but you can do RPC, SOAP, etc.
Service = independent code base
Independent deployments
Service-Oriented Architecture
BenefitsSmaller code bases = upper bound to n
Failure domains become isolated
Technology independence
Federated responsibility
Service-Oriented Architecture
Drawbackseverything becomes decoupled
function calls start looking like HTTP requests
versioning can be a nightmare
tracking dependencies is hard
data consistency becomes challenging
end-to-end testing becomes hard(er), if not
impossible
Conquering SOA
Treat everything as distributed
Pick a size that works for youmicro
macro
somewhere in between
Size doesn’t have to be uniform!
Ops Deputies
Developers ‘deputized’ to do operations
Elevated privileges
Tackle infrastructure needs for their team
Contribute improvements to shared infra
Become first-hop for operations questions
Glue
Define interfaces between componentsMakes it easy to swap out later
Expect change
Minimize code, keep it simple
Think about EOL when you startYour company is changing, and so will your needs