testing for ops: going beyond the manifest - puppetconf 2013

TESTING FOR OPS:GOING BEYOND THE MANIFEST

Christopher Webber (@cwebber)Infrastructure Engineer, Demand Media

ABOUT ME

• Infrastructure Engineer at Demand Media

• Always been in Support or Operations Roles

• Been a Puppet user since v 0.24.7 (2008)

ROLE MODULES VS

LIBRARY MODULES

Post By Craig Dunn (http://www.craigdunn.org/2012/05/239/)

OPS IMPLICITLYGETS TESTING

THEN WE TRY TO DO IT• And it is dumb

• Might as well write a script to transform the code from one format to another

• Start with unit testing frameworks, looking for integration testing

TYPES OF TESTING• Syntax checking (puppet parser validate)

• Linting

• Unit Tests

• Integration Tests

SETTING UP OUR ENVIRONMENT

• Ruby• Use the version you use in prod

• Gems• puppet (use the version you use in prod)• rspec-puppet• puppet-lint• puppetlabs_spec_helper

THE TRIFECTA

1 # == Class: ssh 2 # 3 class ssh { 4 5 package { 6 'openssh-server': 7 ensure => installed 8 } 9 10 file { 11 '/etc/ssh/sshd_config': 12 ensure => present, 13 owner => 'root', 14 group => 'root', 15 mode => '0644', 16 content => template('ssh/sshd_config.erb'), 17 require => Package['openssh-server'] 18 } 19 20 service { 21 'sshd': 22 ensure => running, 23 enable => true, 24 hasstatus => true, 25 hasrestart => true, 26 subscribe => File['/etc/ssh/sshd_config'] 27 } 28 29 }

THE TESTS 1 require 'spec_helper' 2 3 describe 'ssh' do 4 5 it { should contain_package('openssh-server') } 6 7 it do 8 should contain_file('/etc/ssh/sshd_config') \ 9 .with_ensure('present') \ 10 .with_owner('root') \ 11 .with_group('root') \ 12 .with_mode('0644') \ 13 .with_require('Package[openssh-server]') 14 end 15 16 it do 17 should contain_service('sshd') \ 18 .with_ensure('running') \ 19 .with_enable(true) \ 20 .with_hasstatus(true) \ 21 .with_hasrestart(true) \ 22 .with_subscribe('File[/etc/ssh/sshd_config]') 23 end 24 end

IS THIS USEFUL?At first,

I said NO

BUT WHY?It really doesn’t capture the things we care

about.

ENTER INFOSEC

• SSH MUST HAVE THE FOLLOWING SETTINGS:

• PermitRootLogin no

• X11Forwarding no

ADD USEFUL TESTS 25 context "InfoSec Requirements" do 26 27 it do 28 should contain_file('/etc/ssh/sshd_config') \ 29 .with_content(%r{^PermitRootLogin no$}) 30 end 31 32 it do 33 should contain_file('/etc/ssh/sshd_config') \ 34 .with_content(%r{^X11Forwarding no$}) 35 end 36 37 end

AND FAILFailures:

1) ssh InfoSec Requirements Failure/Error: .with_content(%r{^PermitRootLogin no$}) expected that the catalogue would contain File[/etc/ssh/sshd_config] with content matching `/^PermitRootLogin no$/` but its value of `"<file contents>"` does not # ./spec/classes/ssh_spec.rb:29:in `block (3 levels) in <top (required)>'

2) ssh InfoSec Requirements Failure/Error: .with_content(%r{^X11Forwarding no$}) expected that the catalogue would contain File[/etc/ssh/sshd_config] with content matching `/^X11Forwarding no$/` but its value of `"<file contents>"` does not # ./spec/classes/ssh_spec.rb:34:in `block (3 levels) in <top (required)>'

Finished in 0.47736 seconds5 examples, 2 failures

Failed examples:

rspec ./spec/classes/ssh_spec.rb:27 # ssh InfoSec Requirementsrspec ./spec/classes/ssh_spec.rb:32 # ssh InfoSec Requirements

WHO CARES?

• Can now fix and validate

• Becomes one more check and balance

• Safety in changes

OPERATIONAL EXAMPLES

LIBRARY MODULE EXAMPLES

https://github.com/cwebberOps/puppetconf-ssh

CONTINUING WITH SSH• New Requirements

• We have a system that other systems ssh to and drop info

• We have determined that we need to increase the MaxStartups to 40

NEW TEST CODE 25 it do 26 should contain_file('/etc/ssh/sshd_config') \ 27 .with_content(%r{^MaxStartups 10$}) 28 end 29 30 context "maxstartups => 40" do 31 32 let (:params) {{ :maxstartups => 40 }} 33 34 it do 35 should contain_file('/etc/ssh/sshd_config') \ 36 .with_content(%r{^MaxStartups 40$}) 37 end 38 39 end https://github.com/cwebberOps/puppetconf-ssh

NEW PUPPET CODE 1 # == Class: ssh 2 # 3 class ssh ( 4 $maxstartups = 10 5 ){

https://github.com/cwebberOps/puppetconf-ssh

FEEDBACK LOOP

• Did you remember to update the template with your new variable?

• Much faster than vagrant destroy && vagrant up or even vagrant provision

MOVING ON TO THE ROLE

• In the end, it is the module that matters the most

• Should test that the config has the right config for the app

https://github.com/cwebberOps/puppetconf-infra

TEST CODE 1 require 'spec_helper' 2 3 describe 'infra::collector' do 4 5 it do 6 should contain_file('/etc/ssh/sshd_config') \ 7 .with_content(%r{^MaxStartups 40$}) 8 end 9 10 end


PUPPET CODE

1 # Class for setting up the infrastructure collector 2 class infra::collector { 3 4 class { 5 'ssh': 6 maxstartups => 40 7 } 8 9 }


BUT THE TESTS... THEY DON’T WORK

MORE SETUP• Rakefile

• puppet_rspec_helper

• .fixtures.yml

RAKEFILE & SPEC_HELPER

1 require 'rake' 2 3 require 'rspec/core/rake_task' 4 require 'puppetlabs_spec_helper/rake_tasks'

New

1 require 'rake' 2 3 require 'rspec/core/rake_task' 4 5 RSpec::Core::RakeTask.new(:spec) do |t| 6 t.pattern = 'spec/*/*_spec.rb' 7 end

Old

.FIXTURES.YML 1 fixtures: 2 repositories: 3 ssh: "https://github.com/cwebberOps/puppetconf-ssh.git" 4 symlinks: 5 infra: "#{ source_dir }"

FUN EXAMPLES

INFRA::JENKINS 1 require 'spec_helper' 2 3 describe 'infra::jenkins' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('java') } 11 it { should include_class('jenkins') } 12 13 end


INFRA::JENKINS 1 # Class for standing up a jenkins box 2 class infra::jenkins { 3 4 class { 5 'java': 6 } -> 7 8 class { 9 'jenkins': 10 } 11 12 }


WHAT’S BROKE?

http://projects.puppetlabs.com/issues/2053The Bug

The Tweet

INFRA::NODEJS & INFRA::NODEJS::DEV

1 require 'spec_helper' 2 3 describe 'infra::nodejs' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('nodejs') } 11 12 it { should_not contain_package('nodejs-dev') } 13 14 it { should contain_file('/app_specific_stuff') } 15 16 end



1 require 'spec_helper' 2 3 describe 'infra::nodejs::dev' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('nodejs') } 11 it { should contain_package('nodejs-dev') } 12 13 end



1 # Demo class that sets up nodejs 2 class infra::nodejs { 3 4 class { 5 '::nodejs': 6 dev_package => false 7 } 8 9 file { 10 '/app_specific_stuff': 11 ensure => directory 12 } 13 14 }



1 # Override class 2 class infra::nodejs::dev inherits infra::nodejs { 3 4 Class['::nodejs'] { 5 dev_package => true 6 } 7 8 }


WHAT’S BROKE?

http://projects.puppetlabs.com/issues/5517

The Bug

WHY DO THESE EXAMPLES MATTER?

WHAT’S NEXT

• Integration testing using rspec-system

• Continuous Integration

PREVIEW

RSPEC-SYSTEM 1 require 'spec_helper_system' 2 3 describe 'ssh' do 4 5 it 'class should converge' do 6 pp = <<-EOS 7 class { 'ssh': } 8 EOS 9 10 puppet_apply(pp) do |r| 11 r.exit_code.should_not == 1 12 r.refresh 13 r.exit_code.should be_zero 14 end 15 end 16 17 describe service('sshd') do 18 it { should be_enabled } 19 it { should be_running } 20 end 21 end

JENKINS

QUESTIONS?

testing for ops: going beyond the manifest - puppetconf 2013

Technology

class ssh

infosec ssh

end https

ssh new requirements

systems ssh

ssh infosec requirements

end old

content matching