Chef OpsworkHorizontal Scaling, made simple!

- Hamza Waqas

High-level Overview

● A library for configuration management.● A configuration management system.● A system integration platform.● An API for your entire infrastructure.● Change Management made simple.

Principles

● Idempotent● Reliability● Reasonability● pre-configured defaults settings.● Hackability.● Re-usability.

Problems

● You are just an engineer.● DevOps were never meant to you.● ~N horizontal scaling nodes.● System in Production, Images aren’t synced.● Reflect a change to all nodes without doing

that manually.● Generic practice missing.

More problems?

● OS packages been updated frequently.● Inject / Reject any dependency anytime from

entire cluster.● Multiple clusters of differentiated stack i.e

(App Cluster, MySQL Cluster, glusterFS cluster).

● Tired of manual SSH connections to each machine.

Infrastructure Specs

● Manages configuration as Idempotent Resources.

● Merge them together in Recipe.● Version Control System on the bottom.● Keeps underlying generic configuration

separate from `client` environment.● Extendable and replicable.

Infrastructure as Code

A technical domain revolving around building and managing infrastructure programmatically. Enables the reconstruction of the business from nothing but a source code repository, an application data backup and bare metal resources. All you’ve to do is to take care of middleware building your stack instead of nodes, itself.

Chef Client

● CLI deployed over node.● Communicates with Chef Server to pull

configuration and updates.● Should exists on each node apart of the

belonging role cluster.● Carries unique IP and name providing

independent identity in server

Chef Server

Chef Server is the major man in the entire workflow. It communicates with `knife` and `chef-client` and at the mean time, it also collaborate to persist recipes and cookbooks of configuration with versioning so that, `chef-client` could ask for relevant configuration later.

Knife

`Knife` is a command-line tool, providing an ability to communicate with Chef Server and indirectly with chef-client. If you actually want to make changes over each ~N node configured in your Chef- `knife` would be responsible for passing an argument to `Chef Server` and that will take care of `chef-client` (nodes) itself.

Cookbook

A cookbook is the fundamental unit of configuration and policy distribution. A cookbook defines a scenario and contains everything that is required to support that scenario. Measuring Recipes, Attributes, File distribution and Templates.

Recipe

Its a fundamental unit of cookbook. It should reside inside `cookbook` respectively. However, `Recipe` could be written using `Ruby` language exposing the behavior to be done on the client. It should define everything that is required to configure part of a system. Recipe could be inherited or included inside another recipe and could be a dependency for other `recipe` as well.

Role

Role is a deliberate tag, assigned to each node individually making it possible for every node in a cluster to demonstrate and individually recognize themselves. Roles usually allow us to group individual nodes of the cluster to be recognized based upon the task we’re supposed to be taken from them.

Run List

● Should exists in node (`chef-client`).● run_list occupies `recipe` and `roles`

related to the node.● “run_list”: [

“role[api]”,“recipe[apt]”, “recipe[redis]”

]

Flow

Abbr● package: Used to manage packages on a node.● service: Used to manage service on a node.● user: Manage user on a node.● group: Manager user groups.● template: Manage files with ruby templates.● cookbook_file: Transfer file from cookbook to node.● file: manage content of a file on node.● directory: manage directories on a node.● execute: Executes a command on a node.● CRON: Edit an existing cron file on a node.

Installing Chef-

solo

$ curl -L https://www.opscode.com/chef/install.sh | bash

$ wget http://github.com/opscode/chef-repo/tarball/master

$ mv ./chef* ~/chef-repo

$ cd ~/chef-repo

$ mkdir -p .chef

$ echo "cookbook_path [ '/root/chef-repo/cookbooks' ]" >

.chef/knife.rb

$ knife cookbook create redis

$ nano cookbooks/redis/recipes/default.rb

bash "install_redis" do

user 'root'

cwd "/home/hamza"

code <<-EOH

wget http://download.redis.io/releases/redis-3.0.0.tar.gz

tar -xvf redis-3.0.0.tar.gz

cd redis-3.0.0/

make install

EOH

end

execute "Start Redis Server" do

command "redis-server &"

end

$ nano ~/chef-repo/solo.rb

file_cache_path "/root/chef-solo"

cookbook_path "/root/chef-repo/cookbooks"

$ nano ~/chef-repo/web.json

{

“run_list”: [

“recipe[redis]”

]

}

$ chef-solo -c solo.rb -j web.json

Starting Chef Client, version 11.4.0

… … … … … …

… … … … … …

… … … … … …

Chef Client finished, 1 resources updated

Installing Chef

Server

$ apt-get install curl wget build-essential git -y

$ rpm -ivh https://opscode-omnibus-

packages.s3.amazonaws.com/el/6/x86_64/chef-server-

11.0.8-1.el6.x86_64.rpm

$ chef-server-ctl reconfigure

// Installs Erchef, RabbitMQ, PostgreSQL and all the

related dependencies

$ chef-server-ctl test; //Stop apache first.

Installing `chef` on

workstation

$ curl -L https://www.opscode.com/chef/install.sh | bash

…. …. …. …. …. …. …. …. …. …. …. …. …. …. …. …. …. ….

$ chef-client -v

Chef: [VERSION]

$ mkdir -p ~/.chef

// Copy Certificate keys from Chef Server.

$ scp root@10pearls-chef:/etc/chef-server/admin.pem

~/.chef

$ scp root@10pearls-chef:/etc/chef-server/chef-

validator.pem ~/.chef

$ knife configure -i

// It will prompt for values for chef-server url, `user` name,

admin name, location of keys and etc$ cat ~/.chef/knife.rb

log_level :info

log_location STDOUT

node_name 'knife-user1'

client_key '/root/.chef/knife-user1.pem'

validation_client_name 'chef-validator'

validation_key '/root/.chef/admin.pem'

chef_server_url 'https://chef-server.example.com:443/'

syntax_check_cache_path '/root/.chef/syntax_check_cache'

$ knife client list

// chef-validator , chef-webui

$ knife user list

admin user1 (assuming user1 is what we created with `knife configure -i`)

Writing your first

Cookbook!

$ mkdir -p ~/chef-repo && cd $_;

// Assuming, you have `knife` installed

$ knife cookbook create nginx

$ cd cookbooks/nginx;

// ls: attributes, CHANGELOG.md, definitions, files, libraries, metadata.rb, providers, README.md,

recipes, resources, templates

$ nano recipes/default.rb;

package ‘nginx’ do

action :install

end

service ‘nginx’ do

action [ :enable, :start ]

end

cookbook_file “/usr/share/nginx/www/index.html” do

source “index.html”

mode “0644”

end

Upload to Chef-Server

Once, we’re done with all the basic configurations and writing our recipes inside cookbook - it’s now ready to be uploaded to Chef Server.

PS: `chef-client` (node) will pull the cookbook from Chef Server

$ cd files/default

$ nano index.html

<html>

<head>

<title>Chef’s alive!</title>

</head>

<body>

<h1>Portland’s Macho! </h1>

</body>

</html>

$ nano metadata.rb

name 'nginx'

maintainer '10Pearls'

maintainer_email 'hamza.waqas@10pearls.com'

license 'All rights reserved'

description 'Installs/Configures nginx'

long_description IO.read(File.join(File.dirname(__FILE__),

'README.md'))

version '0.1.0'

depends "build-essential"

$ knife cookbook upload -a

// OR

$ knife cookbook upload nginx

Deploying

cookbook!

$ chef-client// from the `node` instance.

The story is over now!

Follow me on Twitter:https://twitter.com/HamzaWaqas

Follow me on Github:http://github.com/ArkeologeN