STANDARDISE DEVELOPMENT ENVIRONMENTS AND MACHINE IMAGES WITH PACKER

Marcelo Pinheiro http://salizzar.net - @salizzar

SUMMARY• Motivation

• Installation

• How it works

• Inside Templates

• Some examples

• Our experience

• FAQ

MOTIVATION• How to take control inside the

following environment issues?

• Common workstation problems (HD failure, dead computer)

• Different OS’s

• Extra machine configuration to enable devs to work (programming languages, databases, plugins, etc), taking one or two days to be ready-to-code

• “Works on my machine” syndrome

MOTIVATION

• Vagrant or Docker. Period.

• Embrace virtualisation

• Each offers a way to pre-setup your VM with necessary libraries, databases and so on

• No more development databases in your pre-staging DB server, additional dependencies

• A try to make development environment more similar to production

MOTIVATION• It sounds good, but… how to take control over each application VM?

• Sometimes your team needs to use some tools that are not available in official package repository (or are too old), forcing to manual configuration after up a VM

• Even running a VM, developer personal choices can contaminate the application (example: rspec add-ons, irb plugins)

• Some developers don’t have knowledge about Chef / Puppet recipes

• How to maintain Vagrant Custom Boxes / Docker Custom Images when you need to add / change tools, repositories or configs?

MOTIVATION

• For sysadms / sysops:

• How to export a new machine image to your virtualisation server (KVM, Xen, VMWare, etc) when a new OS release is launched without “dist-upgrade”?

• How to automate it?

MOTIVATION

• Packer for the rescue

• Written in Go

• Owner: Mitchell Hashimoto (Vagrant, Serf)

• http://www.packer.io

INSTALLATION• http://www.packer.io/downloads.html

• Download zipped binaries for your OS

• OSX

• Linux

• Windows

• FreeBSD

• OpenBSD

INSTALLATION

• Move binaries to your /usr/local/bin, ~/bin, whatever

• It’s done.

HOW IT WORKS• Packer recipes are JSON files

• Validate template:

• $ packer validate your_recipe.json

• Run template:

• $ packer run your_recipe.json

INSIDE TEMPLATES• Packer templates have the following structure:

• Variables

• Builders

• Provisioners

• Post-processors

INSIDE TEMPLATES: VARIABLES

• User-defined variables to be used along template

• Can be declared in a custom file

INSIDE TEMPLATES: VARIABLES

"variables": {! "box_ostype": "centos",! "box_osversion": "6.5",! "box_nick": "6.5",! "box_arch": "x64",! "box_type": "base",! "box_memory": "512",! "box_cpus": "1",!! "iso_arch": "x86_64",! "iso_type": "netinstall",! "iso_md5": "939fd1d87c11ffe7795324438b85adfb",!! "ssh_user": "vagrant",! "ssh_pass": "vagrant",! "hostname": "vagrant-centos-6.5",! "domain": "vagrantup.com"! }

INSIDE TEMPLATES: VARIABLES

# my_variables.json! {! "type": "vmware-iso",! "vm_name": "mybox-vmw“,! "guest_os_type": “centos",! "disk_size": “4096”,!! (… other definitions here…)! }!

INSIDE TEMPLATES: BUILDERS

• Create a machine image from scratch

• Download a ISO from official OS mirror, select a base image to start

• Set CPU cores, memory size, disk size

• See documentation for further details (a lot of options)

INSIDE TEMPLATES: BUILDERS

"builders": [! {! "type": "virtualbox-iso",! "iso_url": "http://an.repository.com/an-image.iso",! "iso_checksum": "an-checksum",! "iso_checksum_type": "md5",! "http_directory": "http",! "ssh_username": "root",! "ssh_password": "apassword",! "ssh_wait_timeout": "100000s",! "shutdown_command": "echo {{ user `ssh_user` }} | sudo halt -p",!! "boot_command": [! "<esc> ",! "install ",! "auto “,!! (… other definitions here …)!! "<enter><wait>"! ]! }! ]

INSIDE TEMPLATES: BUILDERS

• How to automate setup mundane tasks?

• Minimal set of packages

• Disk partition

• Network

• Timezone

INSIDE TEMPLATES: BUILDERS

• For CentOS: Kickstart

• For Debian: Preseed

• For Windows:

• Windows Automated Installation Kit (AIK)

• Microsoft Deployment Toolkit (MDT)

INSIDE TEMPLATES: BUILDERS

• Available builders:

• QEMU - KVM and Xen (experimental)

• VMWare

• Virtualbox

• Docker

!

• OpenStack

• Google Compute Engine

• Amazon EC2

• Digital Ocean

INSIDE TEMPLATES: BUILDERS / VMWARE & VIRTUALBOX• VMWare:

• vmware-iso: create from scratch

• vmware-vmx: create from a base VMX file

• Virtualbox:

• virtualbox-iso: create from scratch

• virtualbox-ovf: create from a base OVF file

INSIDE TEMPLATES: BUILDERS / QEMU

• Create KVM / Xen images from scratch

• Packer depends on qemu-system-x86_64, available only on Debian at this time as a binary

• CentOS have qemu-kvm, but you need to manually override all Packer default options

INSIDE TEMPLATES: BUILDERS / DOCKER

• Creates a Docker image by pulling a existent, starting a container, provision it and exports a .tar file

• Provision without Dockerfile

INSIDE TEMPLATES: BUILDERS / OTHERS

• For other builders, you simply need to inform:

• username / password, API key

• base image

• zone and other related information

• See Packer documentation

INSIDE TEMPLATES: PROVISIONERS

• After the setup of a machine image, it’s time to configure it

• Here is where magic happens:

• Add packages, useful scripts

• Standardise config files

• Apply existent recipes from a CM

INSIDE TEMPLATES: PROVISIONERS

• Available provisioners:

• Shell Scripts

• File Uploads

• Ansible

• Chef Solo

• Puppet

• Salt

INSIDE TEMPLATES: PROVISIONERS / SHELL SCRIPTS

• Most simple way to setup machine

• Run apt-get, yum and friends

INSIDE TEMPLATES: PROVISIONERS / SHELL SCRIPTS "provisioners": [! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/locale.sh",! "scripts/elrepo.sh"! ]! },! {! "type": "shell",! "pause_before": "30s",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/vagrant.sh",! "scripts/sudoers.sh"! ]! }! ]

INSIDE TEMPLATES: PROVISIONERS / FILE UPLOADS

• Need to set default configuration files or upload some custom packages (.tar, .deb / .rpm) to be installed later?

• Upload them and after process with a shell script or CM recipe

INSIDE TEMPLATES: PROVISIONERS / FILE UPLOADS "provisioners": [! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "scripts": [! "scripts/lamp/vagrant.sh",! "scripts/lamp/apache2.sh",! "scripts/lamp/php5.sh",! "scripts/lamp/mysql.sh"! ]! },! {! "type": "file",! "source": "files/lamp-vagrant/vhost",! "destination": "/etc/apache2/sites-available/lamp-php"! },! {! "type": "shell",! "execute_command": "echo 'root' | sh '{{ .Path }}'",! "script": "scripts/lamp/enable-vhost"! }! ]

INSIDE TEMPLATES: PROVISIONERS / OTHERS

• The following provisioners requires installation before run:

• Ansible

• Puppet

• Salt

• Chef Solo is installed by Packer if not present

• At this time, all provisioners are executed in client mode (no remote server)

INSIDE TEMPLATES: PROVISIONERS / OTHERS

"provisioners": [! {! "type": "ansible-local",! "playbook_file": "recipes/ansible/lamp.yml"! },! {! "type": "chef-solo",! "cookbook_paths": [ "recipes/chef/lamp" ]! },! {! "type": "puppet-masterless",! "manifest_file": "recipes/puppet/lamp"! },! {! "type": "salt-masterless",! "local_state_tree": "recipes/salt/lamp"! }! ]

INSIDE TEMPLATES: POST-PROCESSORS

• After create / setup a machine image, you can:

• Convert to a Vagrant Custom Box

• Locally add it as a Docker container

• Publish in a Docker registry

• Publish in a vSphere endpoint

INSIDE TEMPLATES: POST-PROCESSORS / VAGRANT

• Defines a box output name

• You can attach a Vagrantfile template and other template files (cookbooks)

• Change compression rate if you want

INSIDE TEMPLATES: POST-PROCESSORS / VAGRANT

"post-processors": [! {! "type": "vagrant",! “output": "lamp-vagrant.box"! }! ]

INSIDE TEMPLATES: POST-PROCESSORS / DOCKER• You can locally import a Docker image

• You can push a Docker image to a registry

• Needs manual login (automated soon)

• Important: Docker pushes a completely new image, not incremental

INSIDE TEMPLATES: POST-PROCESSORS / DOCKER

"post-processors": [! {! "type": "docker-import",! "repository": "salizzar/packer",! "tag": "0.1"! },! "docker-push"! ]

INSIDE TEMPLATES: POST-PROCESSORS / VSPHERE

• Upload to a vSphere endpoint

INSIDE TEMPLATES: POST-PROCESSORS / VSPHERE "post-processors": [! {! "type": "vsphere",! "host": "a-vsphere-host.com",! "username": "my_user",! "password": "my_password",! "cluster": "a-cluster",! “datacenter": "xyz",! "datastore": "zyx",! “resource_pool": "zyx",! "vm_folder": "images",! "vm_name": "lamp",! “vm_network": "staging"! },! ]

SOME EXAMPLES

• It’s time to see some code!

• All examples are available on:

• https://github.com/salizzar/packer-examples

OUR EXPERIENCEIvan IV Vasilyevich (the Terrible)

OUR EXPERIENCE

• Prepare to argue (sometimes fight :)

• It’s hard to change development tradition of premature optimisation, ultra-high performance, personal choices, “nightly build” syndrome

• Create a culture first

OUR EXPERIENCE• Sometimes the better choice must be autocracy-based

• Use OS package system ASAP (or backport / automate installation if package not exists)

• Introduce to developers a wisdom to use the same package of programming language / DB / whatever that runs in production (!)

• If is old, upgrade your app to use a newer version

• The same for tools that “vendorize" your app libraries (maven, bundler, etc)

OUR EXPERIENCE• Make all applications ready-to-setup-and-run with one command

• Track all dependencies with Dockerfile or Vagrant Shell Scripts

• Bash scripts are more easy to setup than 3rd party CM tools at first time

• Adopt a convention to make all applications more similar as possible about their structure

• Code generators

OUR EXPERIENCE• Divide to conquer

• Adopt a bottom-up strategy

• Minor systems that are easy to setup

• Minor teams

• Start to apply with more systems and greater teams

• Standardise ASAP

OUR EXPERIENCE• At this time, major systems in Locaweb PaaS are Vagrant-ready

• git clone, vagrant up, vagrant ssh

• Docker in development

• Internally created a gem to apply standardisation of Rails apps:

• Packaging (Debian)

• Vagrant

• Packer recipes to create Vagrant custom boxes, using our mirrors

FAQ

• Questions?

• New recipes available on:

• https://github.com/salizzar/packer-vmware

THANK YOU! :)