Cobbler, Func and Puppet:Cobbler, Func and Puppet:Tools for Large Scale EnvironmentsTools for Large Scale Environments
Francesco CrippaFrancesco Crippafcrippa @ fedoraproject,orgfcrippa @ fedoraproject,org
This presentation is licensed under a Creative CommonsAttribution-NonCommercial-ShareAlike (BY-NC-SA) 3.0 license.
what is a“Large Scale Environment”?
From this...
Google first WebFarm
...to this!
CC by Johnnie Walker: http://flickr.com/photos/johnniewalker/359440369
Large scale environments
● Huge number of machines● Huge number of environments
– Production
– Pre-production
– Test
– Develop
● Huge number of people● Need to scale!!!
All is complex
● An example: make a web server on line● In theory it should be easy:
– Buy hardware
– Install operating system (next, next, next...)
– Install a http server (yum install httpd)
– Configure http (vi /etc/httpd/conf/httpd.conf)
– That's all
● ...but...●
All is complex
● In a LSE you need:– 2 web server to maintain service in high availability
– 2 geographical sites to support disaster recovery
– A testing environment to test configurations
– A development environment to setup machines
– Backup
– Monitoring
– Clustering● Shared storage (SAN) required for each cluster
A lot of web servers
1... ...7 servers!
Where is the complexity?
● Web server configuration is simple● But you need to configure:
– 7 different web server (with exactly the same configuration!!!)
– Keep all configurations in sync
– 3 clusters (with different configurations)
– Shared storages (maybe with shared filesystems)
● The complexity is not on the “business”, but on the “infrastructure”
Typical work flow
● Installing OS to a new hardware● Configure machines as you need● Make (sometimes) unconventional tasks
InstallingInstallingO.S.O.S. ConfigurationsConfigurations UnconventionalUnconventional
TasksTasks
Typical work flow
InstallingInstallingO.S.O.S. ConfigurationsConfigurations UnconventionalUnconventional
TasksTasks
COBBLERCOBBLER PUPPETPUPPET FUNCFUNC
All under your control!
Copyright by WarnerBras (Matrix)
Installing OS...COBBLER
OS Installation
● Usually KickStart driven● Two main targets for a new installation:
– Real hardware
– Virtual Machine
● Usually expensive because require some manual human tasks– Put booting cd-rom into reader, configure network
and location of kickstart...
Cobbler
● Cobbler is a Linux provisioning server that allows for rapid setup of network installation environments.
● It keeps in sync:– DHCP
– TFTP
– RPM Repositories
– KickStarts
Importing repositories
● Check configuration files● Import repositories
[root@a~]# cobbler check[root@a~]# cobbler import
mirror=rsync://servergoeshere/path/to/distroname=fedora9
Add profiles and systems
● Distros● Profiles● Systems
[root@a~]# cobbler distro add[root@a~]# cobbler profile add[root@a~]# cobbler system add
Keep in sync
● Generate output in:– /tftpboot
– /var/www/cobbler
– /etc/dhcp.conf
[root@a~]# cobbler sync
KickStart templates
● Kickstart is a template● You can pass values for all your variables
through “cobbler system add”
[root@a~]# cobbler system add ksmeta=”KEY=us”
...keyboard $KEY...
Koan
● Client side● Provisioning for virtual machines ●
● koan --server=server.expl.org --profile=RHEL5-i386
●
● koan -–server=server.expl.org --profile=RHEL5-i386 --virt
Configure all your systemsPUPPET
Configuration Management
– Centralize all configurations into a single place
– Defining “templates” to generalize similar configurations (make one, apply hundreds)
– Manage exceptions
– Make possible for a lot of people to work all together
Working all together
Working all together
Working all together
Working all together
Configuration Items
– Files
– Packages
– Services
– Users
– Groups
– Mount points
– ...
– You can define your own items :-)
– Subscribe
– Require
– Notify
– ...
– Arrays
– Conditions
– ...
A formal definition for HTTPDClass http inherits security-hardening {
file {“/etc/httpd/httpd.conf”:
owner => apache,mode => 644,require => Package[“httpd”]
}
package {“httpd”:
ensure => “latest”}
service {“httpd”:
ensure => “running”,enabled => “true”,subscribe => [
File[“/etc/httpd/httpd.conf”],Package[“httpd”]
]}
}
A formal definition for HTTPD
node 'www.example.com' inherits default {import httpd
}
node 'default' {import baculaimport nagiosimport bacula-backup
}
Puppet architecture
– Client (the puppet) polls the server (the puppet-master) to know if there are new configuration files to download (every 30 minutes by default)
– Client is recognized through its hostname
– Server try to find a right configuration for that hostname, else it uses a “default” config.
– ...caching, md5, checksum and ssl are part of infrastructure
●
Python API for your networkFUNC
Goals
● Manage your hosts as python objects– Groups, ACLs, SSL certificates, ...
[root@a~]# func “*.com” call service restart “httpd”[root@a~]# func “*.lan” call command run “df h”
>>> from func.overlord import client>>> client1 = client.Client(“*.lan”)>>> client1.service.restart(“httpd”)>>> client1.command.run(“df h”)
Modules and new features
● Func based on modules architecture● A module support new stuff● 20 modules (libVirt, jboss, info, process,
command, iptables, nagios, etc)● Writing a new module is simple. ● When you write a module, it works on both CLI
and PyScripting, no modification on func are needed.
THANK YOU.
Credits: Byte-Code, Tango Icons, Crystals Icons