cloud meets fog & puppet a story of version controlled infrastructure
DESCRIPTION
Talk at rootconf - A conference at Bangalore for sysadmins. Gist of the talk:- Puppet is a great configuration management tool and git is great at version controlling.AWS lets you create instances in few clicks. But when it comes to large deployments only automation(where tools come together) can make you productive and happy. I will take you through following.. Fog - The Ruby cloud services library and how it helps you to create vendor neutral cloud deployments, Puppet- Multi region puppet masters, Ruby- How Ruby pulls the strings together in EC2/ELB/RDS creation, Security group creation, IP authorization, Route53 DNS etc, Git- how we use git to version control deployment configs/configurations.TRANSCRIPT
![Page 1: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/1.jpg)
Cloud meets Fog & PuppetA Story of Version Controlled Infrastructure
Habeeb Rahman | apigee | @habi_pk | [email protected]
![Page 2: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/2.jpg)
About:apigee
We love APIs Developers => API <= Enterprises
Usergrid
![Page 3: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/3.jpg)
Take 1Third-party web
tool and ...
![Page 4: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/4.jpg)
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApply Patches GenerateNagiosConfig
+ a Few Misc
![Page 5: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/5.jpg)
Take 2Cloud meets puppet and ...
(aws console, git, puppet)
![Page 6: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/6.jpg)
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApplyPatches GenerateNagiosConfig
+ a Few Misc
![Page 7: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/7.jpg)
Take 3Cloud meets Fog and …
(puppet cloud provisioner, fog, ruby)
![Page 8: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/8.jpg)
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApplyPatches GenerateNagiosConfig
+ a Few Misc
![Page 9: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/9.jpg)
Customer in a 'yaml' file...
![Page 10: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/10.jpg)
![Page 11: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/11.jpg)
Git version controls Puppet Configurations & Cloud Deployments
(Yes! Version Controlled Infrastructure)
![Page 12: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/12.jpg)
![Page 13: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/13.jpg)
Git hooks
Pre-update => puppet syntax checks(puppet parser, puppet lint)
Post-update => sync with regional puppet masters
![Page 14: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/14.jpg)
Puppet
Git branches => Puppet environments
Single puppet master => Test Dev Production
manifests and files/data => Separate git repo
Puppet masters per AWS regions
Hiera: a pluggable hierarchical data store
![Page 15: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/15.jpg)
Why Fog?
![Page 16: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/16.jpg)
Cross service compatibility(multitude of services=> Storage, Compute, DNS…)
Avoid vendor lock-in(Vendor independent=>EC2, Rackspace, Slicehost, Linode…)
Power of Ruby
![Page 17: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/17.jpg)
gem install fog
![Page 18: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/18.jpg)
require ‘fog’
![Page 19: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/19.jpg)
~/.fog
![Page 20: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/20.jpg)
Fog quick starter kit
Look at the test suites (thanks to TDD) https://github.com/fog/fog/blob/master/tests/aws/requests/compute/instance_tests.rb
![Page 21: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/21.jpg)
Fog command line
[root@zebra ~]$ less ~/.fog[root@zebra ~]$ fog testaws Welcome to fog interactive! :testaws provides AWS>>
![Page 22: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/22.jpg)
ec2 = Fog::Compute.new( provider: "AWS", aws_access_key_id: "KEY", aws_secret_access_key: "SECRET")
rsc = Fog::Compute.new( provider: "Rackspace", rackspace_api_key: "KEY", rackspace_username: "SECRET")
ec2.servers.create( flavor_id: "c1.medium", image_id: "ami-6936fb00", # Ubuntu 10.04 LTS key_name: "mykeyname", groups: ["default"],)
rsc.servers.create( flavor_id: 5, # 4GB Memory image_id: 49, # Ubuntu 10.04 LTS name: 'my_server_name')
![Page 23: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/23.jpg)
Open Source Fog alternatives
libcloud (Python)jclouds (Java)
deltacloud (Ruby)
![Page 24: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/24.jpg)
Why Ruby?
![Page 25: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/25.jpg)
“Ruby is designed for programmer productivity and fun”
Matsumoto(creator of the ruby language)
![Page 26: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/26.jpg)
'everything-ruby' by Ajey Gorehttp://confreaks.com/videos/921-rubyconfindia2012-everything-ruby
![Page 27: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/27.jpg)
irb is your friend (ruby interactive shell)
root@zebra:~/$ irb1.9.3-p125 :001 >
irb alternative = > pry (gem install pry)
![Page 28: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/28.jpg)
The Layers of Abstraction...
![Page 29: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/29.jpg)
Fog => Cloud abstraction
![Page 30: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/30.jpg)
Puppet => OS abstraction
![Page 31: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/31.jpg)
Notes to self >>
![Page 32: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/32.jpg)
Start small...
![Page 33: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/33.jpg)
...and iterate reiterate & re reiterate
![Page 34: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/34.jpg)
Select right tools(Plenty of tools => Puppet/Chef/Cfengine/Ansible
but what is good for you/team/culture?)
![Page 35: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/35.jpg)
Design your SaaS but make sure it's 'ops' ready
(from the day you start the design)
![Page 36: Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure](https://reader033.vdocument.in/reader033/viewer/2022052822/554fa1bfb4c9057b298b4950/html5/thumbnails/36.jpg)
http://puppetlabs.com/http://fog.iohttp://git-scm.com/http://ansible.github.com/