jeff mc cune sf 2010
TRANSCRIPT
![Page 1: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/1.jpg)
Are we compliant?Auditing Change Management Policies
with Splunk and Puppet
http://bit.ly/puppetsplunkslides
Jeff [email protected]
1Monday, October 11, 2010
![Page 2: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/2.jpg)
Jeff McCune
• Joined Puppet Labs in May, 2010
• Former SA at Netsmart Technologies
• Solaris / RedHat Web App Infrastructure
• Human Health Information Systems
• HIPPA, SAS 70 Type II Compliance
2Monday, October 11, 2010
![Page 3: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/3.jpg)
What’s this all about?
• Audits are a fact of life
• Systems drift
• Puppet Master manifests change
• The logs provide no link
• Puppet and Git in synchrony with Splunk
3Monday, October 11, 2010
![Page 4: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/4.jpg)
Fun with Regulations
• Increased focus on compliance
• SAS 70
• HIPPA
• IPA
• PCI DSS
• etc, etc...
4Monday, October 11, 2010
![Page 5: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/5.jpg)
Compliance is Easy
Golden VM
Clones
5Monday, October 11, 2010
![Page 6: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/6.jpg)
Drifting in and out of Compliance
Follow procedures
Justify the change
Firefighting
Inevitable
Constant drift
6Monday, October 11, 2010
![Page 7: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/7.jpg)
The Trouble with Time
• Are we compliant?
• right now?
• last week?
• last year?
• Why weren’t we?
• Why is this difficult?
7Monday, October 11, 2010
![Page 8: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/8.jpg)
Advanced Management
• We have next-generation tools
• Puppet
• Git
• Subversion
• Splunk
• Redmine
8Monday, October 11, 2010
![Page 9: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/9.jpg)
Two major issues
• Propagation
• Time
9Monday, October 11, 2010
![Page 10: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/10.jpg)
Larry’s commit
Change PropagationMany
Nodes
a872b46
10Monday, October 11, 2010
![Page 11: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/11.jpg)
Time
“Why did that one thing happen that one time?”
11Monday, October 11, 2010
![Page 12: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/12.jpg)
Bridge the GapEvents Commits
12Monday, October 11, 2010
![Page 13: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/13.jpg)
The Missing Link
• puppetmasterd -‐-‐config-‐version \ /demo/get-‐config-‐version-‐script
• [root@puppet ~]# /demo/get-‐config-‐version ref="refs/heads/jeff" commit="b585f7fe"
• Jeff ’s processor, --reports=logversionShould ship with puppet “soon”
13Monday, October 11, 2010
![Page 14: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/14.jpg)
get-config-version#! /bin/bashset -‐uset -‐ecd /demo/puppet-‐demotoolsref="$(git symbolic-‐ref HEAD)"if [[ -‐f .git/"${ref}" ]]; then commit="$(cat .git/${ref})"else commit="UNKNOWN"fiecho "ref=\"${ref}\" commit=\"${commit}\""
14Monday, October 11, 2010
![Page 15: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/15.jpg)
logversion.rb
# Create logversion.rb by copying log.rbdef process self.logs.each do |log| saved_message = "#{log.message}" log.message << " " << log.version Puppet::Util::Log.newmessage(log) log.message = saved_message
endend
15Monday, October 11, 2010
![Page 16: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/16.jpg)
Untagged Events
16Monday, October 11, 2010
![Page 17: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/17.jpg)
Tagged Events
17Monday, October 11, 2010
![Page 18: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/18.jpg)
Who to blame?
Blame this guy The commit proves it
18Monday, October 11, 2010
![Page 19: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/19.jpg)
Putting it all together
• Demo time!
19Monday, October 11, 2010
![Page 20: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/20.jpg)
Steps to Reproduce
• Fork and clone puppet-‐demotools on github
• logversion.rb goes into/usr/lib/ruby/site_ruby/1.8/puppet/reports
• --config_version /path/to/your/script
• --reports=logversion,store
• Make sure syslog catches daemon.* and splunk is indexing syslog
• Note: syslog outputs are off with -‐-‐verbose
20Monday, October 11, 2010
![Page 21: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/21.jpg)
Future Work• Commit hooks into puppetmasterd
activation and the ticketing system
• Splunk URLs to redmine, trac, salesforce...
• Closed loop from business case to system modification by puppet.
21Monday, October 11, 2010
![Page 22: Jeff mc cune sf 2010](https://reader033.vdocument.in/reader033/viewer/2022052908/559512f11a28ab91598b45fb/html5/thumbnails/22.jpg)
Questions?
• Google Moderator
• http://bit.ly/arewecompliant?
• http://bit.ly/puppetsplunkslides
• Twitter: 0xEFF
• Email: [email protected]
22Monday, October 11, 2010