confoo - devops & agile infrastructure

@cloudops_ www.cloudops.com

DevOps & Agile Infrastructure

Confoo 2015


Who Am I● Will Stevens - Lead Developer @ CloudOps

● CloudStack Committer & PMC Member

● CloudOps is a Montreal based company with customers around the world

● We build and operate clouds of all shapes and sizes

● We develop cloud infrastructure solutions and operational models

● We created and operate cloud.ca


Winds of Change

The way infrastructure is consumed and managed is changing. But is this relevant to developers?


What is DevOps

● Solve operational problems with software and automation

● Gives the application deployment process efficiency and repeatability

● Gives deployments the agility and flexibility to be implemented on a variety of infrastructures

● Gives quicker turn around on application changes and security fixes

● Improve application quality by adding automated testing into the release cycle


What is Agile Infrastructure

● Infrastructure orchestrated through APIs

● Cloud enabled resources are the backbone of agile infrastructure

● Self service and provision resources on demand

● Remove the need for IT departments to provision resources for other departments

● Manage the infrastructure as a whole rather than a collection of small distinct sub-systems

● Pooling resources to bring down the overall infrastructure cost


● Enabling developers to be involved and engaged through the entire application lifecycle

● Close the loop between development and operation to add consistency throughout

● Developers will play a major role in continuing the advancements in the DevOps space

Why do I care?


Lets change gears a bit

“Less theory please. I care about practical stuff!”


Tools of the trade (not exhaustive)● CloudStack

○ Manage infrastructure

● Jenkins

○ Build/test/deploy code

● KitchenCI

○ Test infrastructure as code

● Chef

○ Manage configurations

● Fabric

○ Deployment recipes


CloudStack

● Infrastructure as a Service platform

○ Multi-tenant environment

○ Can share resources in groups

○ Virtual Machines

○ Network automation

○ Storage and snapshots

○ Everything is controlled by APIs

● Open source and actively developed

● Build enterprise private clouds with CloudStack


Jenkins

● Integrates with code repositories (Git, SVN, etc) to access developers code

● Builds a variety of languages (Java, .Net, etc)

● Runs tests and reports (JUnit, TestNG, etc)

● Runs code analyzers (Checkstyle, FindBugs, etc)

● Deploy artifacts to systems (Tomcat, SSH, etc) on successful build completion

● Package deployments into OS specific packages (rpm, deb, etc) and publish to a repository

● Has a strong plugin ecosystem for extension


KitchenCI

● Integrates with all the major IaaS providers (Amazon EC2, CloudStack, OpenStack, etc)

● Supports many test frameworks (RSpec, Serverspec, etc)

● Tight integration with Chef so you can test your infrastructure with your configuration manager

● Automate the validation of deployments on different operating systems and hardware

● Plugin architecture enables you to write the tests you want to run on your infrastructure


Chef

● Implements infrastructure as code

● All environments are configured using the same process from a library of core components

● Never manually change env configuration, automate so your changes propagate correctly

● Centralized source of truth for all your configurations and components

● Enables collaboration between the people managing the application and the people who manage the infrastructure


Fabric

● It is a stretch to add this one (but I love it)

● Simple Python command line tool to manage application lifecycle

● Self contained so deployment procedures can be tracked with the application for portability

● Deploy code into multiple environments with the same procedure

● Can take care of copying deployed files as well as managing release revisions and backups

● A very simple tool, but the options are endless


Use cases

“I thought we were getting to practical stuff…”


cloud.ca application lifecycle

A software development use case...

● Develop and commit Java code into Git

● Jenkins builds and runs tests on the commit

● If the tests pass, then a WAR file is created

● Jenkins pushes the WAR file into a staging env

● The new WAR file are validated in staging

● Once all the validation is completed in staging, the WAR files are pushed into production

● There is still more we can automate


cloud.ca infrastructure config

Automate environment configuration…

● Create and configure a Chef role for each required application environment

● Chef (knife) will then deploy and configure the environments according to the needed roles

● Chef cookbook manages all the installed software and configuration

● Chef is used to manage performance and tuning configurations for the different environments


Automate from end to end

● Develop and commit code into Git

● Jenkins builds and unit tests the commit

● Jenkins kicks off KitchenCI for infra testing

● KitchenCI provisions real infra in multiple flavors (CentOS, Ubuntu, etc) using Chef

● Chef configures all aspects of the env including the new code we are developing

● KitchenCI runs tests against the envs to validate

● If all the functional tests pass we have validated both the application and the configuration


Build what fits your needs


Questions

Will Stevens : @swillops | CloudOps : @cloudops_

confoo - devops & agile infrastructure

Technology

cloud infrastructure

changethe way infrastructure

overall infrastructure

developers code

code chef

provision resources

exhaustive cloudstack

application changes