chef - configuration management for the cloud
DESCRIPTION
Presentation given at PDX Cloud meeting on ChefTRANSCRIPT
![Page 1: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/1.jpg)
Configuration Management in the Cloud with Chef
PDX Cloud
James [email protected]: jamesc_000
GitHub: jamescwww.opscode.com
Tuesday, April 30, 13
![Page 2: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/2.jpg)
• Instant infrastructure
• Unlimited capacity
• Autoscaling
• No commitment
• Immediate replacement
Clouds are great !
Tuesday, April 30, 13
![Page 3: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/3.jpg)
APIs are awesome!
• You can provision compute resources in seconds
• You can provision storage resources in seconds
• You can provision network resources in seconds
• That’s cool.
http://www.flickr.com/photos/jdhancock/3634246981/
Tuesday, April 30, 13
![Page 4: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/4.jpg)
The Dark Side of the Cloud
Tuesday, April 30, 13
![Page 5: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/5.jpg)
• Performance
• Security
• Price
• Reliability
Why not the Cloud?
Tuesday, April 30, 13
![Page 6: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/6.jpg)
See Node
Application Server
Tuesday, April 30, 13
![Page 7: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/7.jpg)
See Nodes
Application Server
Application Database
Tuesday, April 30, 13
![Page 8: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/8.jpg)
See Nodes Grow
Application Server
Application Databases
Tuesday, April 30, 13
![Page 9: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/9.jpg)
Application Servers
Application Databases
See Nodes Grow
Tuesday, April 30, 13
![Page 10: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/10.jpg)
Application Servers
Application Databases
Load Balancer
See Nodes Grow
Tuesday, April 30, 13
![Page 11: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/11.jpg)
See Nodes Grow
Application Servers
Application Databases
Load Balancers
Tuesday, April 30, 13
![Page 12: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/12.jpg)
See Nodes Grow
Application Servers
Application Database Cache
Load Balancers
Application Databases
Tuesday, April 30, 13
![Page 13: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/13.jpg)
Tied together with Config
Application Servers
Application Database Cache
Load Balancers
Application Databases
Tuesday, April 30, 13
![Page 14: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/14.jpg)
Infrastructure is a Snowflake
Application Servers
Application Database Cache
Load Balancers
Floating IP?
Application Databases
Tuesday, April 30, 13
![Page 15: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/15.jpg)
Evolving Complexity
Load Balancers
Application Servers
NoSQL
Database Slaves
ApplicationCache
Database Cache
Database
Tuesday, April 30, 13
![Page 16: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/16.jpg)
Complexity Grows Quickly
DC1
DC3
DC2
Tuesday, April 30, 13
![Page 17: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/17.jpg)
http://www.flickr.com/photos/16339684@N00/2681435235/
And it Continues to Evolve
Tuesday, April 30, 13
![Page 18: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/18.jpg)
http://www.flickr.com/photos/16339684@N00/2681435235/
And it Continues to Evolve
Ok, so I’ve got a problem.
What’s the solution ?
Tuesday, April 30, 13
![Page 19: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/19.jpg)
Golden Images are not the answer
• Gold is heavy
• Hard to transport
• Hard to mold
• Easy to lose configuration detail
http://www.flickr.com/photos/garysoup/2977173063/
Tuesday, April 30, 13
![Page 20: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/20.jpg)
Configuration Management and Automated Systems Integration
is the Answer
http://www.flickr.com/photos/philliecasablanca/3354734116/Tuesday, April 30, 13
![Page 21: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/21.jpg)
• Turn code and hardware into infrastructure
• From bare metal to services in production
• Scale applications as needed
• Conform to policy
• Align to business goals
Configuration Management ?
Tuesday, April 30, 13
![Page 22: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/22.jpg)
Chef - Infrastructure as Code
http://www.flickr.com/photos/louisb/4555295187/
• Programmatically provision and configure
• Treat like any other code base
• Reconstruct business from code repository, data backup, and bare metal resources.
Tuesday, April 30, 13
![Page 23: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/23.jpg)
• Chef-Client generates configurations directly on nodes from their run list
• Reduce management complexity through abstraction
• Store the configuration of your programs in version control
http://www.flickr.com/photos/ssoosay/5126146763/
Nodes
Tuesday, April 30, 13
![Page 24: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/24.jpg)
Collections of Resources
• Networking
• Files
• Directories
• Symlinks
• Mounts
• Routes
• Users
• Groups
• Tasks
• Packages
• Software
• Services
• Configurations
• Other Stuffhttp://www.flickr.com/photos/stevekeys/3123167585/
Tuesday, April 30, 13
![Page 25: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/25.jpg)
Declarative Interface to Resources
• Define policy
• Say what, not how
• Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/Tuesday, April 30, 13
![Page 26: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/26.jpg)
Ruby!
extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
Tuesday, April 30, 13
![Page 27: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/27.jpg)
Or thissearch(:users, '*:*') do |u| user u['id'] do uid u['uid'] shell u['shell'] home "/home/#{u['id']}" end directory "#{home_dir}/.ssh" do
owner u['id'] group u['gid'] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u['id'] group u['id'] mode "0600" variables :ssh_keys => u['ssh_keys']
endend
Tuesday, April 30, 13
![Page 28: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/28.jpg)
Recipes and Cookbooks
• Recipes are collections of Resources
• Cookbooks contain recipes, templates, files, custom resources, etc
• Code re-use and modularity
• Hundreds already on Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/Tuesday, April 30, 13
![Page 29: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/29.jpg)
http://www.flickr.com/photos/kathycsus/2686772625
• IP addresses
• Hostnames
• FQDNs
• Search for nodes with Roles
• Find configuration data
Search
Tuesday, April 30, 13
![Page 30: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/30.jpg)
pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end
Pass Results to Templates
Tuesday, April 30, 13
![Page 31: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/31.jpg)
# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>
Pass Results to Templates
Tuesday, April 30, 13
![Page 32: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/32.jpg)
Jboss App
Memcache
Postgres Slaves
Postgres Master
So when this
NagiosGraphite
Tuesday, April 30, 13
![Page 33: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/33.jpg)
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
Becomes this
Tuesday, April 30, 13
![Page 34: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/34.jpg)
Jboss App
Memcache
Postgres Slaves
Postgres Master
NagiosGraphite
Updates can be automatic
Tuesday, April 30, 13
![Page 35: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/35.jpg)
NagiosGraphite
Count the resources
Jboss App
Memcache
Postgres Slaves
• Load balancer config
• Nagios host ping
• Nagios host ssh
• Nagios host HTTP
• Nagios host app health
• Graphite CPU
• Graphite Memory
• Graphite Disk
• Graphite SNMP
• Memcache firewall
• Postgres firewall
• Postgres authZ config
• 12+ resource changes for 1 node addition
Tuesday, April 30, 13
![Page 36: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/36.jpg)
http://www.flickr.com/photos/evelynishere/2798236471/
CLONING CANNOT COPE WITH THIS
• Chef can.
Tuesday, April 30, 13
![Page 37: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/37.jpg)
Build anything
• Simple internal applications
• Complex external applications
• Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.flickr.com/photos/hyku/245010680/
Tuesday, April 30, 13
![Page 38: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/38.jpg)
And manage it simply
http://www.flickr.com/photos/helico/404640681/
• Automatically reconfigure everything
• Linux, Windows, Unixes, BSDs
• Load balancers
• Metrics collection systems
• Monitoring systems
• Cloud migrations become trivial
Tuesday, April 30, 13
![Page 39: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/39.jpg)
knife
Tuesday, April 30, 13
![Page 40: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/40.jpg)
Upload your infrastructureknife cookbook upload apt
knife cookbook upload chef-client
knife cookbook upload java
knife cookbook upload jpackage
knife cookbook upload ntp
knife cookbook upload sudo
knife cookbook upload tomcat
knife cookbook upload users
knife cookbook upload sample
knife role from file base-cloud.rb
knife role from file tc.rb
knife role from file sample.rb
knife data bag create users
knife data bag from file users jamesc.jsonTuesday, April 30, 13
![Page 41: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/41.jpg)
Build it somewhere
#EC2knife ec2 server create -S jamesc -i ~/.ssh/jamesc.pem -x ubuntu -G default -I ami-a7a97dce -f m1.small -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Rackspaceknife rackspace server create --image 110 --flavor 2 -i ~/.ssh/jamesc.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#CloudStackknife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i ~/.ssh/jamesc.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Ubuntu Linuxknife bootstrap test.lab -r 'role[webserver]' -i ~/.ssh/jamesc.pem -x ubuntu --sudo -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
Tuesday, April 30, 13
![Page 42: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/42.jpg)
knife ec2
$ knife ec2Available ec2 subcommands: (for details, knife SUB-COMMAND --help)
** EC2 COMMANDS **knife ec2 flavor list (options)knife ec2 instance data (options)knife ec2 server create (options)knife ec2 server delete SERVER [SERVER] (options)knife ec2 server list (options)
$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]'
Tuesday, April 30, 13
![Page 43: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/43.jpg)
knife openstack
$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)
** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)
$ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]'
Tuesday, April 30, 13
![Page 44: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/44.jpg)
Chef for Infrastructure Portability
• knife ec2
• knife rackspace
• knife hp
• knife google
• knife azure
• knife cloudstack
• knife openstack
• knife vsphere
• ... and many others
Tuesday, April 30, 13
![Page 45: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/45.jpg)
The Chef Community
• Apache License, Version 2.0
• 850+ Individual contributors
• 150+ Corporate contributors
• HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more
• 550+ cookbooks
• http://community.opscode.com
Tuesday, April 30, 13
![Page 46: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/46.jpg)
Summary
• Every infrastructure is a unique snowflake
• You need tools to let you do what you want
• You need the power to grow your infrastructure
• You need the ability to change your cloud provider
• Automated Configuration Management is the solution
Tuesday, April 30, 13
![Page 47: Chef - Configuration Management for the Cloud](https://reader036.vdocument.in/reader036/viewer/2022062617/54c688ab4a795962378b4574/html5/thumbnails/47.jpg)
Questions?
http://www.flickr.com/photos/mrchippy/443960682/
Questions?
Tuesday, April 30, 13