Killer R10K Workflow

Automating the Killer Robots, all 10K of Them

Who Am I@phil_zimmerman

Early Days

R10K Workflow Awesomeness

Demo Time

The Journey of A Thousand Miles Begins

With A Single, Monolithic Repo

A Single Repo? What the F*@K?!!

Why Put Everything in a Single Repo?

Simplify Development

Easy Jenkins Flow

Puppet Code and Hiera Data Together

Just Starting

Out

Single CI Job • rspec-puppet • syntax check • lint

Single Release Job • create/push tag

Single Deploy Job • Capistrano tasks

• poor man’s dynamic environments • kludgy git logic w/conditionals in Capfile

For All Modules

Forge Modules

Capistrano -> Puppet Module Tool

Worked Well… Until It Didn’t

Did I mention the

Capistrano tasks?

KLUDGY

task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end

task "create_puppet_env", :roles => :puppet_master do if exists?(:branchname) run "if [ -d #{environment_basedir}/#{branchname} ]; then cd #{environment_basedir}/#{branchname} && git pull origin #{branchname} ; else \ git clone #{module_repository} #{environment_basedir}/#{branchname} --branch #{branchname} ; fi" else puts "Please provide a valid git branch name as an argument" end end

Upgrade Forge Modules

Version Management == Face Palm

Upgrading Affects all Environments!

Let Me Count the Other Ways…

This Started to Fail

Simple Comes at a Price

You mean I have to wait for tests

to run for ALL modules before I know if my changes are good?”

“I only changed one module….

I have to deploy

everything in order to get my changes on the masters? Sigh”

“I only changed one module….

everything just to get my hiera data on the masters?”

“I need to change some hiera data…. I have to deploy

I’m Losing

My Patience!

Oh - and that little problem where

upgrading forge modules can break production…

Stop the Madness !!

Recap of Early Days:Monolithic RepoLong CI CyclesAll-Or-Nothing DeploysUpgrading Forge Modules

Tools Can Be

Awesome

(we just needed a couple more)

Make Them Work For You

R10K

R10K

R10K!

https://github.com/adrienthebo/r10k

Deploys Puppet Code

Handles Git/Svn Fu

Is Awesome

R10K and Puppetfile

Match Made in Heaven (or Portland)

Manage Module Versions

Inventory of Puppet Environment

Puppetfile

Format

Inventory of Modules and Their Versions

Puppetfile lives in its

own repository

Puppetfile lives in its own

Repository

Inventory of Modules and Their Versions

Each module in its own

repository AND r10k

R10KDeploy

r10k deploy environment test -p

deploys all modules in Puppetfile for the test branch

r10k deploy module tomcatdeploys a single module!

CI Job Per Module • rspec-puppet • syntax check • lint

Release Job Per Module • create/push tag • select module from dropdown list

Deploy Job For Each Module And Hiera • simpler Capistrano tasks • wrap r10k calls to each master/node

desc "for specified branch in puppetfile repo, use r10k to deploy all modules for the specified environment." task "update_environment", :roles => :puppet_master do if exists?(:branchname) run "r10k -v debug deploy environment #{branchname} -p" else puts "Please provide a valid git branch name as an argument" end end

Puppetfile Manipulation and Branch Creation

mod 'tomcat', :git => 'git@github.webapps.rr.com:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'

Tomcat Example

go to tomcat dir (production branch) !

‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’

Create the ‘dev_change_foo’ branch

in Puppetfile repo

Change the :ref for tomcat to

‘dev_change_foo’

Call r10k to deploy ‘dev_change_foo’

Notify Me When It’s Finished

Explain the Magics

Hint:

Create Branchr10k deploy environment

Modify Branchr10k deploy module

Delete Branchauto-delete Puppetfile branch

Testing Multiple Modules in the

Same Environment

go to profile dir (production branch) !

‘git checkout -b dev_change_foo’ ‘git push origin dev_change_foo’

mod 'profile', :git => 'git@github.webapps.rr.com:fylgia/profile.git', :ref => 'RELEASE_0.1.124' mod 'tomcat', :git => 'git@github.webapps.rr.com:fylgia/tomcat.git', :ref => 'RELEASE_1.0.13'

production

mod 'profile', :git => 'git@github.webapps.rr.com:fylgia/profile.git', :ref => ‘dev_change_foo’ mod 'tomcat', :git => 'git@github.webapps.rr.com:fylgia/tomcat.git', :ref => 'dev_change_foo'

dev_change_foo

Truly Dynamic Environments!

The Post-Receive Hook

reaktor

Default SetupGitHub or GitHub Enterprise

Hipchat

Just provide some config!

REAKTOR_PUPPET_MASTERS_FILE

REAKTOR_HIPCHAT_TOKENREAKTOR_HIPCHAT_ROOMREAKTOR_HIPCHAT_FROM

PUPPETFILE_GIT_URL

Other Chat ProvidersCampfire, Slack, etc

If it has an API, it’s pluggable!

Other Git ProvidersGitlab, Bitbucket, etc

Need to Determine Best Approach

Create Release

Deploy - Ship It!

Create ReleaseModulefile

RELEASE_1.0.13 -> RELEASE_1.0.14

versionfile (hieradata)

DeployParameterized Job

environment

version to deploy

Puppetfile Manipulation (again)

change :ref to selected version

r10k deploy selected environment

commit/push updated Puppetfile

One-Click Production

Deploy

Workflow Recap:R10K and PuppetfileEach Module in Own RepoPost-Receive Hook GoodnessProduction Deployments

Conclusion:Early DaysR10K Workflow AwesomenessDemos

Helpful Links and References

https://github.com/adrienthebo/r10k

http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/

http://garylarizza.com/blog/2014/03/07/puppet-workflow-part-3b/

http://philzim.com/2014/05/02/an-automated-r10k-workflow-that-works/

https://github.com/pzim/reaktor