devops state of the union 2015

The Agile Admins● Ernest Mueller (@ernestmueller)● James Wickett (@wickett)● Karthik Gaekwad (@iteration1)

Who Are We?

Who Are You?

Things DevOpsers Care About

Source: Devopsdays Austin Survey 2015 (500 participants)

Topics covered today

● What is DevOps?● Why would I use it?● What’s hot in DevOps?● What does the future look like?

DevOps Defined

● DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.

● DevOps is also characterized by operations staff making use many of the same techniques as developers for their systems work.

The Layers of DevOps

DevOps Principles

DevOps Tools

DevOps Practices

DevOps Principles

● The Three Ways (Gene Kim)o Systems Thinkingo Amplify Feedback Loopso Culture of Continual Experimentation

● CAMS (John Willis)o Culture – People > Process > Toolso Automation – Infrastructure as Codeo Measurement – Measure Everythingo Sharing – Collaboration/Feedback

● Informed by the values in the Agile Manifesto and Lean Theory of Constraints

DevOps Practices

•Version Control For All•Automated Testing•Proactive Monitoring and Metrics•Kanban/Scrum•Visible Ops/Change Management•Configuration Management•Incident Command System•Continuous Integration/Deployment/Delivery•“Put Developers On Call”•Virtualization/Cloud/Containers•Toolchain Approach•Transparent Uptime/Incident Retrospectives

DevOps Tools

● Hardware (System provisioning/automation)● Software (Software delivery pipeline)● Wetware (Process/human element)● Monitoring/Metrics (of all the above)

DevOps Adoption Impact

● 2014 State of DevOps survey by Puppet Labs, Thoughtworks, and IT Revolution Press

● Top indicators of high organizational performance:o IT performance - Leaders here are twice as likely to

exceed their profitability, market share, and productivity goals. Deployment frequency, lead time for changes, mean time to recover from failure are positively impacted by DevOps practices.

o Organizational culture and climate for learningo Job satisfaction

What’s Hot Now

● Security and DevOps● Internet of Things● Containers and Docker● Microservices and 12-Factor Apps● ChatOps● Unicorns to Horses to Donkeys

Security is where ops was 5 years ago...

Epiphanies in Security and DevOps

● Smaller batches make you more secure● Departure from the perimeter mentality● Instrument your runtime environment and

correlate disparate metrics (e.g. transactions to csrf token generations)

● Emphasis on MTTD and MTTR

Epiphanies in Security and DevOps

● We can go fast and still do compliance● Major vulnerabilities like Heartbleed exposed

weaknesses in our software supply chain● Security testing can be done in development

Security Companies to Watch

● Signal Sciences● Threat Stack● Sonatype● Contrast Security

Internet of Things

DevOps and IoT

● Fully distributed compute on networks you can’t/don’t control

● Software and tracking components becomes really hard

● Automation even more necessary● Security even more necessary

Containers and Docker

Open source platform for developers and sysadmins to build, ship and run distributed applications anywhere.

What is Docker?

Docker Components

● Docker Engine o Runtime and packaging toolo Installed on hosts that run Docker

● Docker Hub o Cloud service for storing and sharing appso Save your docker images (public/private)

Why is it so popular?

● Portabilityo Write once, run anywhere

● Standardized Environmentso Dev/QA/Production can be modeled exactly the

same way● Rapid scale-up/scale-down

o Containers take seconds to deploy● Ability to build a continuous delivery (CD)

pipeline

Use cases for Docker

● Continuous Integration & Deliveryo Yelp, Spotify, Mailgun, Cambridge Healthcare

● PaaSo Yandex, Baidu

● Easier Development & Deploymento Ebay, Bleacher Report, New Relic

● Hosting Legacy Applications

https://www.docker.com/resources/usecases/

Players in the space

Microservices

“Independent processes communicating with each other with well defined API’s to form larger

more complex applications”

Microservices

Microservices characteristics

● Do one thing, and do it well (fine grained)● Independently built- common language to

communicate● Independently deployable● Fault tolerant and reliable

12 Factor App

● Use declarative formats for setup automation; minimize time and cost for new developers joining the project

● Clean contract with the underlying OS, offering maximum portability between execution environments;

● Suitable for deployment on modern cloud platforms● Minimize divergence between development and

production- enabling continuous deployment● Scale up without significant changes to tooling,

architecture, or development practices.

Relationship with containers

● Microservices and containers are a great fit.● Single service on a container.

o Isolates service and makes it easy to manage ● Emerging best practice for new architectures

with containers.

● Consider building your architecture in this manner.

ChatOps

ChatOpsChatOps

Unicorns to Horses to Donkeys

Gartner: “By 2016 25% of the G2000 will be employing DevOps.”

“By 2018, 90% of I&O organizations attempting to use DevOps without specifically addressing their cultural foundations will fail.”

The Future...

Future

● More Lean Extension● Security Fully DevOpsed● Cloudification/Compute Fabric● Testing and Monitoring● Devops - The New Normal● Containers Rule The World

Nice (Software Supply) Chain!

Extension of Lean

Delivery Pipeline extension to Lean

● Your BOM is part of your product● Ability to track where all code came from

which products are using it just like automotive and aeronautical industries

● Quality metrics extending to customer usage

Security into Devops

● Security rebranded in language of Rugged as features (ruggedsoftware.org)

● Growth around integrating security at all phases of the delivery pipeline

● Instrument runtime environment with security metrics

● Movement from security training to security testing

● Searching for aspirational moment like “10 deploys a day” did for devops

Cloud/Compute Fabric

● Rightscale 2015 State of the Cloud Survey reveals 93% of organizations are running apps on or experimenting with IaaS

● 82% of enterprises have a hybrid cloud strategy

● IoT and Big Data mean large heaps of data it’s hard to move around the network

● Containerization and microservices mean it’s easier to move compute to the data than vice versa

● Eventual: Complex, ultra-distributed compute

Testing and Monitoring

● Automated testing is a best practice● Light/fast testing feeds CD● App and system monitoring is a best

practice● Logging, analytics being absorbed into it● There is no difference between monitoring

and regression testing theoretically - only historically

● Eventual: Convergence of regression testing and monitoring

Containers rule the world

● Yesterday: Docker didn’t exist● Today: Engineers playing with Docker in

dev/test.● Tomorrow: Production Docker● Even lesser emphasis on hardware/cloud.

● Eventual: Microservices and 12 factor apps inside containers will be the defacto standard to deploy apps.

DevOps: The New Normal

● Devops is penetrating enterprises.● Yesterday: Unicorns● Today: Horses & Donkeys● Tomorrow: Everyone

● Backlash: Where’s the silver bullet? What is the correct implementation of DevOps?

● Eventual: Devops will be the new normal

Q&A

http://bit.ly/devops2015