Upgrading from CFEngine2 to

CFEngine 3

Agenda

• The Benefits of Upgrading

• What’s New in CFEngine 3

• Promise Theory and How It Drives CFEngine 3

• Planning your Upgrade

• Policy Conversion Methods

• System Upgrade Methods

• Q&A

Why Upgrade?

• Simplifies and extends CFEngine 2

• More consistent in syntax and behavior

• Does not require "under the hood" programming to extend the language – up to 10x less code

• Does not hard-code configuration details

• Enables greater agility; 5 minute update default

• Provides tools for debugging and testing

• Adds native support and integration

What’s new in CFEngine 3?

• Native Support and Integration• Standard Integration

• Package Management

• Enhanced Service Management

• Database

• Virtualization

• Enterprise Extensions• Windows support

• LDAP and Active Directory

• Design Center

• GUI Reporting

What’s New in CFEngine 3?

• Language Enhancements• Bodies and Bundles

body common control {

bundlesequence => { "test" };}

bundle agent test{reports:

cfengine_3::"Hello world!";

}

What’s New in CFEngine 3?

• Language Enhancements

• Standard Library

/var/cfengine/inputs/cfengine_stdlib.cf

• Arrays and Lists

• Pattern matching and Iteration

• Comments and Handles

CFEngine Enterprise - Mission Portal GUI

• Features

• Auditing and Compliance

• Monitoring

• Reporting

• REST API

• Design Center

• Inventory management

Promise Theory and CFEngine 3

• Promise Theory

Voluntary cooperation between individual, autonomous

actors or agents who publish their intentions to one

another in the form of promises

--

Mark Burgess

Promise Theory - Basic Concepts

• Promise Theory: Applied

• Promises are fundamental statements

Set perms on /etc/passwd

Use latest Apache Package

• A policy is a collection of promises

• Desired state is maintained through policies

• Updates are pulled autonomously

Notable Differences – CFEngine 3• Connections

• Trust relationships are established by design

• Bootstrapping – The process of binding a client to the hub or policy server

• Key exchange – managed by CF3

• Policy Organization• Policies and bundle references are located on all

bootstrapped systems

• Managed by the promises.cf

CFEngine 2 Upgrade Preparation

CFEngine 2 Upgrade Preparation

• Identify peer systems

• Consult documentation

• From Policy Server command line:cfshow -sIP + 192.168.1.101 192.168.1.101 [Tue Jan 23 16:13] not seen for (6.42) hrs

IP - 192.168.1.101 192.168.1.101 [Tue Jan 23 16:13] not seen for (6.42) hrs

cat <path>/cfrun.hosts

• When all else fails, scripting is your friend

CFEngine 2 Upgrade Preparation• Catalog Existing Policies

• Where are they?

• Source control?

• Local inputs?

• Local hosts?

CFEngine 2 Policy Conversion

• Methods

• Functional translation

• What problem does it solve?

• Direct translation

• Line for line

• Be flexible!

• Let the policy be your guide

CFEngine 2 Policy Conversion

• Functional Translation Method

• Holistic viewpoint – the Big Picture approach

• Opportunity for improvement

• Recommended conversion strategy

CFEngine 2 Policy Conversion

• Direct Translation Method

• Direct language translation

• Translation guide:

http://cfengine.com/manuals/cf3-upgrade.html

• Time consuming

• Missed opportunities

CFEngine 2 Policy Conversion: CF2 Processes Policy

processes: "inetd" signal=hup

"bootp" signal=kill exclude=rpc.bootparamd

"cfservd" restart "/usr/local/sbin/cfservd" useshell=false

# matches=>6 warn number of matches is greater than or equal to 6 # matches=1 warn if not exactly 1 matching process # matches=<2 warn if there are less than or equal to 2 matching processes

CFEngine 2 Direct Conversion: CF3 Processes Policy

processes: "inetd" signals => { "hup" }; "bootp" signals => { "kill" }, process_select => exclude_procs(".*rpc.bootparamd.*");

"cf-serverd" restart_class => "start_cfserverd"; # process_count => check_range(cfserv,6,inf); warn number of matches is >= equal to 6 # process_count => check_range(cfserv,1,1); warn if not exactly 1 matching process # process_count => check_range(cfserv,0,2); warn if there are =< to 2 matching processes

commands: start_cfserverd:: "/usr/local/sbin/cf-serverd";

reports: cfserv_out_of_range:: "cf-serverd is out of control!!";

CFEngine 2 Functional Conversion: CF3 Processes Policy

vars:

"daemons" slist => { "cf-monitord", "cf-serverd", "cf-execd" };

processes:

"named" restart_class => "restart_named"; "$(daemons)" restart_class => canonify("start_$(component)");

commands:

"/bin/echo /var/cfengine/bin/$(component)" ifvarclass => canonify("start_$(component)");

restart_named:: "/local/sbin/named -u dns" action => inform;

CFEngine 2 Functional Conversion: CF2 File Ops Policy

This CFEngine 2 Policy: cf2_file_op.cfcontrol: domain = ( mydomain.com ) serverip = ( 172.16.100.129 ) #server ip address master = ( /var/cfengine/inputs ) actionsequence = ( copy files links editfiles )

copy: /master/cfengine/inputs server=$(serverip) dest=$(master) recurse=inf trustkey=on

files: any:: /tmp/cfengine_is_good mode=0644 owner=root group=root action=touch

links: any:: /tmp/how_is_cfengine -> /tmp/cfengine_is_good

editfiles: cfengine_2:: { /etc/motd AppendIfNoSuchLine “Running CFEngine" }

CFEngine 2 Functional Conversion: CF3 File Ops Bundle

Converts to this CFEngine 3 Bundle:

bundle agent old_cfagent {

files:

"/tmp/cfengine_is_good" perms => mog("644","root","root");

"/tmp/how_is_cfengine" link_from => ln_s("/tmp/cfengine_is_good");

cfengine_3:: "/etc/motd" edit_line => append_if_no_lines(“Running CFEngine");}

CFEngine 2 Policy Conversion• Tips and Tricks

• Install CFEngine 3 in a test environment

• Safety first

• Start small

• How would you eat an elephant?

• Focus on the similarities

• The language may be different, but the core concepts remain

CFEngine 2 Policy Conversion• Tips and Tricks

• Convert CF2 policies to bundles; not standalone files

• CFEngine 3 is a different animal

• Client connection and control activities: Handled

• Part of the initial bootstrap process

• The promises.cf file controls automated activity

• Bundles referenced in the bundlesequence stanza• Input bundle files are referenced in the inputs stanza

CFEngine Conversion Tool• Learning tool or killer utility?

• Learning tool

• Requires cleanup; but helpful in learning the language

• Location: https://github.com/cfengine/cf22cf3

• Zip file containing code:

https://github.com/cfengine/cf22cf3/archive/master.zip

• May also clone via HTTPS, SSH, or Subversion. 

CFEngine Conversion Tool - Setup• Pre-requisite and Download Instructions

• This example uses the CentOS 5 distribution• Pre-requisite work:

yum groupinstall "Development tools"yum install db4-develyum install openssl-devel

• Download from GIT: https://github.com/cfengine/cf22cf3|

• Download cf22cf3-master.zip, or if you have a GIT/SVN repo set up locally, clone it

CFEngine Conversion Tool - Setup• Manual Compilation

• Create a compilation area on a local system

mkdir /sandbox

• Copy zip to compilation area and unpack

cp cf22cf3-master.zip   /sandboxcd   /sandboxunzip cf22cf3-master.zipcd cf22cf3-master chmod 755 configure

 

CFEngine Conversion Tool - Setup• Compilation instructions

• Compile./

configure

make

make install

• Binary Directory:  /usr/local/sbin

• Examples Directory:   /usr/local/share/cf23convertBinary:  /usr/local/sbin/cfconvert   

CFEngine Conversion Tool - Usage• Usage

Cfengine Conversion Utility1.0.0Free Software Foundation 1994-Donated by Mark Burgess, Oslo University College, Norway

Options:--file      (-f)             --variables      (-v)               --server       (-s)                 --bundle      (-b)                

Debug levels: 1=parsing, 2=running, 3=summary, 4=expression eval

Bug reports to bug-cfengine@cfengine.orgGeneral help to help-cfengine@cfengine.orgInfo & fixes at http://www.cfengine.org   

CFEngine Conversion Tool - Example• Convert CFE2 policy file to a CFE3 bundle :

• Create a CFEngine 2 policy file in /tmp( We’ll use the policy example in slide 21: cf2_file_op.cf )

• Convert to a bundle and pipe the bundle to stdout

cfconvert -f /tmp/cf2_file_op.cf -b

• Convert to a bundle and pipe to a file( Save the converted file as cf3_file_op.cf )

cfconvert -f /tmp/cf2_file_op.cf -b > /tmp/cf3_file_op.cf

CFEngine 2 Upgrade Plan

• In Place Upgrade Overview

• CF2 and CF3 designed to be interoperable

• Replace CF2 Policies at your pace

CFEngine Upgrade Plan

• Upgrade Notes:

• Replace cfexecd with CFEngine 3's cf-execd

• Access control remains untouched

• Runs cf-agent

• Sample inputs files contain integration promises

• Launched automatically

• Changes crontab

CFEngine Upgrade Plan

• In Place Upgrade Steps

• Backup CFEngine 2 policies and inputs repo

• Install the CFEngine 3 software on a local host

rpm -ivh cfengine-community-3.2.1-.el5.x86_64.rpm

• Copy newly installed /var/cfengine/inputs files to your CF2 master

update repository

• Remove any rules to reinstall CFEngine 2 or add cfexecd or cfagent

to crontabs

• Remove cfexecd from start up processes

chkconfig cfexecd off

chkconfig --del cfexecd

CFEngine Upgrade Plan

• In Place Upgrade Steps

• Change directory to the inputs directory

cd /var/cfengine/inputs

• Edit the update.cf file to point to your CF2 master update repository

• Set the email options for the executor in promises.cf.

• As root, run:

cf-agent --bootstrap

• If all went well, you are now running CFEngine 3. To bootstrap to a

policy server, run:

cf-agent --bootstrap <policy server IP>

CFEngine Upgrade Plan

• In Place Upgrade Steps

• Remove all rules or policies that are capable of activating

CFEngine 2 components

• Convert cfservd.conf into a server bundle

• Place a reference to this bundle in promises.cf

• Remove all rules to run cfservd

• Replace them with rules to run cf-serverd

• Add converted CFEngine 2 policies or create new

CFEngine 3 policies

CFEngine 2 Upgrade Plan

• Replacement Model

• CFEngine 3 installed on separate server

• Converted hosts bootstrap to new server

CFEngine Upgrade Plan

• Replacement Method

• Install CFEngine 3 as a new policy server

• Select a CFEngine 2 host

• Stop all CFEngine 2 processes or daemons on host

• Convert policies, move them to the new policy server

• Remove CFEngine 2 application from the host

• Remove or move CFEngine 2 file system on the hosts

• Install CFEngine 3 on the host

• Bootstrap host to the policy server

CFEngine Upgrade Plan

• Considerations: In Place vs. Replacement

• Complexity of environment

• Uptime Requirements or SLA

• Effort and resources

• Conversion effort: One time vs ongoing

CFEngine Policy Conversion

• Additional Resources

• Best practices guides Upgrading from CFEngine 2 to 3

Additional LinksCFEngine 3 Reference Manual

CFEngine 3 Quick Start Guide

CFEngine 3 Concept Guide

CFEngine 3 Beginning Examples

CFEngine Special Topics

CFEngine 2 Reference Manual

• Join the conversation on our community help forumhttps://groups.google.com/forum/?fromgroups&hl=en#!forum/help-cfengine

Next Steps

• Learn More check out our documentation

• Read Learning CFEngine 3 by Diego Zamboni