software development 2020 - swimming upstream in the container revolution
TRANSCRIPT
![Page 1: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/1.jpg)
Swimming upstream in the container revolution
Practical continuous delivery with open source and cloud services
Bert Jan Schrijver@[email protected]
![Page 2: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/2.jpg)
Bert Jan SchrijverL e t ‘ s m e e t
@bjschrijver
![Page 3: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/3.jpg)
OutlineS o w h a t ‘ s n e x t ?
Approach
Principles
Introduction
DefinitionsChallenges and lessons learned
Business benefits
Looking ahead
![Page 4: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/4.jpg)
Every change goes through the build/test pipeline and automatically gets put into production.
Continuous Deployment
Development and operations engineers participate together in the entire product lifecycle.
DevOpsBuilding and testing software in
such a way that the software can be released to production at any time.
"Ship early, ship often, sacrificing
features, never quality" -‐ Kyle Neath
Continuous Delivery
Team members integrate their work frequently. Commits are verified by
automated builds and tests.
Continuous Integration
DefinitionsWho’s who in CD
![Page 5: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/5.jpg)
Malmberg is an educational publisher in the Netherlands. Malmberg is building modern, rich and scalable e-‐learning
applications using Java 8, Vert.x, AngularJS and MongoDB,
running on Amazon cloud services.
About Malmberg
![Page 6: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/6.jpg)
![Page 7: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/7.jpg)
Differences lead to issuesCommunication between development and operations was slow, problem analysis in production was difficult and releases took a lot of time.
Traditional operationsProduction environments managed
by an external operations partner. Differences in infrastructure between development and
production.
Modern development cultureModern tools and lots of
automation. Test environments are managed by developers.
HistorySome months ago
Things needed to changeIssues and differences between development and operations were slowing us down. We needed to shift strategies to keep progressing.
![Page 8: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/8.jpg)
…said no product manager ever. J. Paul Reed
Let’s spend the next few months..
..working on automated testing and build/release infrastructure, and redesigning how our application is written. We can postpone our feature development.
“
”
![Page 9: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/9.jpg)
Build a complete new setup to allow development teams to
transform to the new situation at their own pace.
Keep the shop openBuild a dedicated team of
Devs, Ops and Cloud experts.
Expert teamDefine key points that identify
your approach and help you set goals.
Define principles
ApproachHow we initiated change.
![Page 10: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/10.jpg)
Principles
![Page 11: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/11.jpg)
Master branch is always releasablePrinciple 1
Every change is developed and tested in a feature branch.
Don’t merge it until it’s done.
![Page 12: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/12.jpg)
Unit/integration (Java & JavaScript), mutation, end-to-end (FitNesse/BrowserStack), Sonar for quality and coverage reporting.
Rely on multiple layers of tests.
Each commit is tested extensivelyPrinciple 2
![Page 13: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/13.jpg)
Manage builds, tests, QA and deployments from a single place.
Jenkins as the heart of the delivery process.
Every delivery step is a Jenkins jobPrinciple 3
![Page 14: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/14.jpg)
After deploying 6 new features, when one has an issue, why roll back 5 good features? Don’t. Just roll out a fix quickly.
Keep moving ahead.
Deployments are roll-forward onlyPrinciple 4
![Page 15: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/15.jpg)
No logging in to servers. Need a change or upgrade? Just update the recipe.
Hands off.
Infrastructure as code - for everythingPrinciple 5
![Page 16: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/16.jpg)
The flexibility and resilience is well worth it. What about using containers? The EC2 instance is our container.
Even when you don’t need to scale.
Put everything in auto scaling groupsPrinciple 6
![Page 17: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/17.jpg)
You can’t explain maintenance windows to modern end users anymore.
Our end users are the Facebook generation.
No downtime in productionPrinciple 7
![Page 18: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/18.jpg)
Make sure you find the problem before it finds you.
Work proactive, not reactive.
Eyes and ears in productionPrinciple 8
![Page 19: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/19.jpg)
Repeating tasks such as viewing logs and doing deployments must be common jobs.
Specialisms are OK, but only for incidental tasks.
Repeating tasks are executable for all team membersPrinciple 9
![Page 20: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/20.jpg)
Differences between teams are OK. A team that’s dependent on external help is not.
Give teams the freedom to work in a way that works for them.
DevOps teams work on a self service basisPrinciple 10
![Page 21: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/21.jpg)
Don't assume that cultural change won't be an issue. It will.
Resistance
It turns out that something in recent AWS Linux kernels triggers slave disconnects. We found out the hard way.
Jenkins slaves can be a pain
Not all devs are comfortable with managing infrastructure and
middleware.
Devs need to step up their game
This kills team progress.
Don’t depend on availability of Ops
Challenges
02
0304
05
0601
When you automate everything and keep growing, chances are you’re going to hit limits.
Amazon has limits
All environments are provisioned automatically. Challenge: how to
prevent testing directly in production.
How to test Puppet changes
and lessons learned
![Page 22: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/22.jpg)
Auto scaling and pro-active monitoring boost availability. A lot.
Availability
Business benefits
High level of automation results in shorter release cycles and faster time to market.
Agility
Automated provisioning makes sure that every environment can be re-built from scratch in minutes.
Continuity
Lower operations costs due to scheduling and scaling. Lower maintenance costs due to high degree of automation.
Cost reduction
Faster problem analysis and solution.Better reaction speed
How to sell this to your boss.
![Page 23: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/23.jpg)
Get the teams the information they need, readily available on a dashboard visible from their desks.
Better monitoring and dashboards
Looking ahead
01
Nightly performance runs on test environments and continuous end-‐user performance monitoring in production.
Continuous performance testing03
The only way to be really prepared for failure is to make sure that things will fail by making it fail yourself.
Automated resilience testing02
This is a tough one. We’re still trying to work out what works for us.
Continuous security testing04
The near future
![Page 24: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/24.jpg)
Questions?
@bjschrijver
![Page 25: Software Development 2020 - Swimming upstream in the container revolution](https://reader030.vdocument.in/reader030/viewer/2022032503/55c10d96bb61eb5a348b47aa/html5/thumbnails/25.jpg)
Thanks for your time.Liked it? Tweet it! @bjschrijver