getting your toolbox together

Getting Your Toolbox Together

Daan van Renterghem

June 7th, 2013

Welcome!

Daan van Renterghem

Software Engineer @Ibuildings

Welcome!

Daan van Renterghem


https://joind.in/8453

Welcome!

Daan van Renterghem


Conference Social!

What is this talk about?

Story Time!

http://thedailygrowl.co.uk/2009/10/14/its-story-time/

Welcome Aboard!

Welcome Aboard!

http://www.quickmeme.com

Welcome Aboard!

All you have to do…

Welcome Aboard!

All you have to do…• Step 1: get the code from VCS

Welcome Aboard!

All you have to do…• Step 1: get the code from VCS• Step 2: download base image

Welcome Aboard!

All you have to do…• Step 1: get the code from VCS• Step 2: download base image• Step 3: configure shared folders

Welcome Aboard!

All you have to do…• Step 1: get the code from VCS• Step 2: download base image• Step 3: configure shared folders• Step 4: start image, say it is copied over, create a new shell user, transfer ownership of

this list of files to yourself, make sure these folders are writeable, mount your shared folder, check if your virtual hosts settings are correct, that your /etc/hosts file is pointing to the correct IP, which you need to get from the VM, oh, and if you’re not using linux, just google for the location of the hosts file, check if the browser doesn’t show the default it works page, resolve a bunch of errors to get the application up and running, ah cool, that’s at least not a fatal error! Ehhhr, yeah, I believe that is caused because of this, oh, and you need to run this list of database patches in that particular sequence, and then resolve some more errors, create you local configuration, and just see if you can get the thing to work, it took me only 5 days, so yeah… Good luck!

In other words:

Could it be done differently?


Yes, yes it could!


Yes, yes it could!

And it should!

Welcome Aboard (Tooling Style)


• Step 1: get the code from VCS


• Step 1: get the code from VCS• Step 2: copy configuration sample


• Step 1: get the code from VCS• Step 2: copy configuration sample• Step 3: run “vagrant up”


• Step 1: get the code from VCS• Step 2: copy configuration sample• Step 3: run “vagrant up”• Step 4: ???

Your Toolbox

Tools, what do you mean?


Having a toolbox means you are using tools to solve problems (rather than checklists and procedures)


Having a toolbox means you are using tools to solve problems (rather than checklists and procedures)

Make development, maintenance and deployment as easy and fast as possible, with the least amount of interaction.

Why are Tools important?


In devops you either are contribute to tool or you are tool.

- DevOps Borat

https://twitter.com/DEVOPS_BORAT/status/139898439621296129



- DevOps Borat

Tools help you accomplish (repetitive) tasks in an easy, consistent manner.




- DevOps Borat

Tools help you accomplish (repetitive) tasks in an easy, consistent manner.

Added bonus: no human error


When to use tools?

Chances are

You’re already using tools!

Chances are

You’re already using tools!• VCS

Chances are

You’re already using tools!• VCS• Editor / IDE

Chances are

You’re already using tools!• VCS• Editor / IDE• CI - server

Where can tools be used


• Code Management


• Code Management• Development Environment


• Code Management• Development Environment• Any other environment



• Deployment



• Deployment• Quality Control



• Deployment• Quality Control• Monitoring

Development Environment


Different options


Different options• Local Webserver


Different options• Local Webserver• Single Shared Server


Different options• Local Webserver• Single Shared Server• Development Images

Local Webserver

Pro: Con:

Local Webserver

Pro:•Portable

Con:

Local Webserver

Pro:•Portable

Con:•No consistent environment

Local Webserver

Pro:•Portable

Con:•No consistent environment•Multiple projects can cause conflicts

Local Webserver

Pro:•Portable

Con:•No consistent environment•Multiple projects can cause conflicts•Different OS, different build

Single Shared Server

Pro: Con:


Pro:•Possible per project

Con:



Con:•Single state for everyone



Con:•Single state for everyone•Single state for every project



Con:•Single state for everyone•Single state for every project•Costs (hardware, management)



Con:•Single state for everyone•Single state for every project•Costs (hardware, management)•Downtime, network

Development Image

Pro: Con:

Development Image

Pro:•Specific per project

Con:

Development Image

Pro:•Specific per project•Portable

Con:

Development Image

Pro:•Specific per project•Portable•Consistent stack between devs

Con:

Development Image

Pro:•Specific per project•Portable•Consistent stack between devs•Allows for local differences where needed

Con:

Development Image

Pro:•Specific per project•Portable•Consistent stack between devs•Allows for local differences where needed

Con:•Takes time to set up

Development Image

Development Image

Requirements

Development Image

Requirements• Reliable, maintainable

Development Image

Requirements• Reliable, maintainable• OS agnostic

Development Image

Requirements• Reliable, maintainable• OS agnostic• Allow for consistent configuration

Development Image

Requirements• Reliable, maintainable• OS agnostic• Allow for consistent configuration• Reusable, Idempotent

Development Image

Now we know what we want, but how to create one?

Development Image


Use a tool!

Development Image


Use Vagrant!

Vagrant

Vagrant

Create and configure lightweight, reproducible, and portable development environments.

Vagrant

Create and configure lightweight, reproducible, and portable development environments.•Provider agnostic

Vagrant

Create and configure lightweight, reproducible, and portable development environments.•Provider agnostic•Uses “boxes”: base images to quickly clone a virtual machine

Vagrant

Create and configure lightweight, reproducible, and portable development environments.•Provider agnostic•Uses “boxes”: base images to quickly clone a virtual machine•Provisioning: automated installation of software

Vagrant - Resources

Vagrant - Resources

Website:http://www.vagrantup.com

Vagrant - Resources


Installation:http://downloads.vagrantup.com

Vagrant - Resources


Installation:http://downloads.vagrantup.com

Documentation:http://docs.vagrantup.com

Vagrant - Usage

Vagrant - Boxes

Box: Portable files which can be used by others on any platform that runs Vagrant to bring up a working environment.

Vagrant - Boxes


$ vagrant box add <name> <url> <provider>

Vagrant - Boxes


$ vagrant box add <name> <url> <provider>

http://www.vagrantbox.es

Vagrant - Boxes

Define configuration, in Vagrantfile

Vagrant - Boxes

Define configuration, in Vagrantfile

Vagrant.configure("2") do |config|

config.vm.box = "<name>"

end

Vagrant - Configuration

Configure amount of memory, give it a name


config.vm.customize [

"modifyvm", :id,

"--name", "foobar",

"--memory", 2048

]

end


Shared folder configuration


config.vm.share_folder(

"htdocs",

"/var/www/my_project/"

"/path/to/local/folder"

)

end


Using local configurationrequire "yaml"

conf = YAML.load_file(

"./deploy/dev/conf.yaml"

)


Using local configurationrequire "yaml"

conf = YAML.load_file(

"./deploy/dev/conf.yaml"

)


# conf available here

end


Shared folder configuration


config.vm.share_folder(

"htdocs",

"/var/www/" + conf["project_name"],

conf["shared_folder"]

)

end


We can localize this too!


config.vm.customize [

"modifyvm", :id,

"--name", conf["project_name"],

"--memory", conf["mem_limit"]

]

end


Even the IP on which it can be reached.


config.vm.network :hostonly, conf["ip"]

end


Basic configuration


Basic configuration•Commit Vagrantfile to VCS


Basic configuration•Commit Vagrantfile to VCS•Commit sample dev config to VCS


Basic configuration•Commit Vagrantfile to VCS•Commit sample dev config to VCS•Ignore dev config

Vagrant - Commands

Vagrant - Commands

• $ vagrant up

• $ vagrant halt

• $ vagrant suspend

• $ vagrant resume

• $ vagrant destroy

• $ vagrant provision

Vagrant – Recap

Vagrant

Single command to get your VM running

Vagrant


$ vagrant up

Vagrant


$ vagrant up

And it’s running!

Vagrant


$ vagrant up

And it’s running!But that’s not usable as server right now…

Vagrant

We have a machine…

Vagrant

We have a machine…… but its not a server

Vagrant

We have a machine…… but its not a server… and we do not want everyone to install everything themselves

Vagrant

We have a machine…… but its not a server… and we do not want everyone to install everything themselves

Enter: Provisioning

Vagrant - Provisioning

Configuration Management


Provisioning


Provisioning

+


Configuration Management


Configuration Management•Infrastructure as code


Configuration Management•Infrastructure as code•Since it is code… VCS!


Various provisioning tools•Puppet•Chef•Ansible•Salt (requires gem)•Shell

No Talk About Chef…

No Talk About Chef…

http://www.telegraph.co.uk/culture/qi/8735232/QI-Quite-interesting-facts-about-hats.html

Chef

Infrastructure Automation Framework

Chef


•Chef Server

Chef


•Chef Server•Chef Solo

Chef

Core Concepts

Chef

Core Concepts•Cookbooks

Chef

Core Concepts•Cookbooks•Roles

Chef - Cookbook

Fundamental Unit of Configuration and Policy Distribution in Chef

Chef - Cookbook


Or: Definition of how to install what and where it should be installed

Chef - Cookbook


Or: Definition of how to install what and where it should be installed

http://community.opscode.com/cookbooks

Chef - Cookbook

StructureCookbook/

/attributes

/recipes

/templates

Chef - Cookbook

StructureCookbook/

/attributes

/recipes

/templates

Attributes: parameters

Chef - Cookbook

StructureCookbook/

/attributes

/recipes

/templates

Attributes: parametersRecipes: code that needs to be executed

Chef - Cookbook

StructureCookbook/

/attributes

/recipes

/templates

Attributes: parametersRecipes: code that needs to be executedTemplates

Chef - Role

A way to define certain patterns and processes that exist across nodes in a Chef organization as belonging to a single job function.

Chef - Role

A way to define certain patterns and processes that exist across nodes in a Chef organization as belonging to a single job function.

Or: A collection of cookbooks and attributes that should be used

Chef - Role

• Simple json file• Name• Default Attributes• Override Attributes• Run List

Chef - Role

Allows for per environment configuration of setup.

As example, a basic example of Apache, php and mysql

Chef - Role - Example

{

"name": "development",

"default_attributes": { },

"override_attributes": { },

"run_list": [

"recipe[apache2]",

"recipe[apache2::mod_php5]",

"recipe[php]",

"recipe[mysql]”,

]

}

Vagrant & Chef

Vagrant - Chef

Vagrant - Chef

config.vm.provision :chef_solo do |chef|

chef.cookbooks_path = "deploy/chef/cookbooks"

chef.roles_path = "deploy/chef/roles"

chef.add_role('development')

end

Vagrant - Chef

What have we got so far?

Vagrant - Chef


•Consistent virtual machine with known OS

Vagrant - Chef


•Consistent virtual machine with known OS•Standardized stack for all developers

Vagrant - Chef



Where to keep it?

Vagrant - Chef



Where to keep it?Inside your project!

Vagrant - Chef

Custom Cookbooks

Vagrant - Chef

Custom Cookbooks

•Database

Vagrant - Chef

Custom Cookbooks

•Database•Fixtures

Vagrant - Chef

Custom Cookbooks

•Database•Fixtures•File generation

Remember This?


• Step 1: get the code from VCS• Step 2: copy configuration sample• Step 3: run “vagrant up”• Step 4: ???


• Step 1: get the code from VCS• Step 2: copy configuration sample• Step 3: run “vagrant up”• Step 4: there is no step 4

Vagrant - Chef

So, how to get started then?

$ git clone [email protected]:adent/42.git

$ cp ./deploy/dev/conf.yaml.sample \

./deploy/dev/conf.yaml

$ vagrant up

And ready to go!

Other Environments

Automated Deployment•Phing•Capistrano• Webistrano• Capifony

•Fabric•Vlad the Deployer•…

Other Environments

Run Tasks•Create Backup•Build / Update Environment•Deploy

Build / Update the Environment with the same tools you use for your dev env!

Other Environments

Building or Updating Environment not possible through tools?

Other Environments


Configure your tools to reflect the production environment

Other Environments


Configure your tools to reflect the production environment•Version locking•Recreate the production environment

Shameless Plug

Deploying your apps… the right way! Robbert van den Bogerd 13:30, track 2

Recap

• Use Development Images

Recap

• Use Development Images• Takes time to set up, but pays off in the

long run

Recap


long run

• Automate Deployment

Recap


long run

• Automate Deployment• Use Provisioning for every Environment

Further Reading

Documentation•http://www.vagrantup.com•http://opscode.com/chef/

Resources•http://community.opscode.com/cookbooks•http://www.vagrantbox.es

Thank You!

Feedback? Please let me know:

@DRvanR https://joind.in/8453

See you at the Conference Social!

getting your toolbox together

Technology

vcs step

tools important

shared folders step

base image step

configuration sample

local webserverpro

local configuration

tool oryou