ncf - a powerful and structured cfengine framework
DESCRIPTION
After 4 years of "experience in the trenches" providing enterprise configuration management solutions based on CFEngine 3, it became clear that our customers wanted CFEngine's speed, small footprint, and features but were having a hard time with the language and tooling, and needed an easier way. The goal of ncf is to abstract the knowledge of "how" to achieve tasks in CFEngine, and expose these capabilities for non-expert users. Thus, you can express your intent in a very succinct and expressive manner, integrating simply with CFEngine and its power, while keeping everything "under the hood" tunable. ncf promotes DRY-ness and sharing knowledge. It is built from the ground-up to be easy to learn, flexible and extensible. The framework is written in pure CFEngine language, introducing a well-structured design built on multiple decoupled layers with clearly-defined interfaces and roles. In our presentation we will introduce ncf and explain how ncf can be used to solve real-world examples, explain our design choices, the architectural approach taken, and demo the new magic at our finger tips to build CFEngine solutions that are extensible and easier to work with.TRANSCRIPT
![Page 1: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/1.jpg)
Normation – CC-BY-SAnormation.com
ncf
A powerful and structuredCFEngine framework
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)
![Page 2: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/2.jpg)
Normation – CC-BY-SAnormation.com 2
ncfwww.ncf.io
Who am I?
● Jonathan Clarke
● Title: Co-founder & Product lead at Normation
● Origins: Sysadmin, infrastructure management
● Now: Automation + “running a company”-stuff
● Contributor to free software:
– Co-creator of Rudder
– Contributor to CFEngine, OpenLDAP
● Co-organizer of events:
![Page 3: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/3.jpg)
Normation – CC-BY-SAnormation.com 3
ncfwww.ncf.io
Intro
This presentationis about cakes
Photo CC BY 2.0 from http://www.flickr.com/photos/misscreativecakes/
![Page 4: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/4.jpg)
Normation – CC-BY-SAnormation.com 4
ncfwww.ncf.io
Intro
Reminder: how are cakes made?
1. Ingredients
+2. Recipes
Photo CC BY-NC 2.0 from http://www.flickr.com/photos/tnemily/
Photo CC BY 2.0 from http://www.flickr.com/photos/misscreativecakes/
![Page 5: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/5.jpg)
Normation – CC-BY-SAnormation.com 5
ncfwww.ncf.io
Background
A bunch ofCFEngine consultants
● > 4 years● Multiple companies: small, large & huge● Various uses: security, provisioning, DR...
We always got the same feedback
![Page 6: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/6.jpg)
Normation – CC-BY-SAnormation.com 6
ncfwww.ncf.io
Feedback #1: CFEngine rocks!
CFEngine rocksMulti-platformLinux, Android, BSD, AIX, HP-UX, Solaris, Windows...
Open SourceGPLv3
Small footprint, scalableA few MB of RAM,just seconds to run...
Continuous checkingAgent based approach,no push
Resilient to errorsNetwork outages, failures,unavailable resources...
![Page 7: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/7.jpg)
Normation – CC-BY-SAnormation.com 7
ncfwww.ncf.io
Feedback #2: CFEngine is hard!
CFEngine is hard
Too much “do it yourself”Building your own policy structure from blank slate
Steep learning curveSyntax is unusual, hardto learn and understand
Lack of feedbackOutput is hard to read,“what is going on?”
Workarounds for bugsNo way of systematically using a workaround
“Flour, eggs, milk and butter”As in: https://digitalelf.net/2013/04/a-case-study-in-cfengine-layout/
![Page 8: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/8.jpg)
Normation – CC-BY-SAnormation.com 8
ncfwww.ncf.io
Feedback #2: CFEngine is hard!
Learning to bake cakes on your own is frustrating.
Photo CC BY-NC-SA 2.0 from http://www.flickr.com/photos/penguincakes/
![Page 9: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/9.jpg)
Normation – CC-BY-SAnormation.com 9
ncfwww.ncf.io
Brief dilemma...
Can we fix this?
We've worked around this for customers, let's make it reusable!
CHALLENGE ACCEPTED.
![Page 10: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/10.jpg)
Normation – CC-BY-SAnormation.com 10
ncfwww.ncf.io
Approach
Too much do it yourselfBuilding your own policy structure from blank slate
1) Provide a structured layout to start from
2) Provide single-purpose, reusable “methods” to get the basics done
![Page 11: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/11.jpg)
Normation – CC-BY-SAnormation.com 11
ncfwww.ncf.io
Steep learning curveSyntax is unusual, hardto learn and understand
Approach
1) Hide the weirder syntaxinside these reusable “methods”
2) Only require a subset of syntaxto write everyday policies (method calls)
![Page 12: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/12.jpg)
Normation – CC-BY-SAnormation.com 12
ncfwww.ncf.io
Approach
1) Implement workarounds into those reusable “methods” (and use them)
2) Automated tests to make sure the bugs don't “come back”
Workarounds for bugsNo way of systematically using a workaround
![Page 13: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/13.jpg)
Normation – CC-BY-SAnormation.com 13
ncfwww.ncf.io
Lack of feedbackOutput is hard to read,“what is going on?”
Approach
1) Now everything goes through reusable “methods”, build automatic feedback in
2) Make the feedback format customisable and extensible
![Page 14: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/14.jpg)
Normation – CC-BY-SAnormation.com 14
ncfwww.ncf.io
Result
“Forget baking,I'm gonna get some
cakes from the shop”Photo CC BY-NC-SA 2.0 from http://www.flickr.com/photos/omarsc/
![Page 15: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/15.jpg)
Normation – CC-BY-SAnormation.com 15
ncfwww.ncf.io
Result
We created ncf
ncf is a frameworkthat runs in pure CFEngine language,to help structure CFEngine policy and
provide reusable, single purpose components distributed under the GPLv3 license.
![Page 16: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/16.jpg)
Normation – CC-BY-SAnormation.com 16
ncfwww.ncf.io
Result
Example === 1000 words
With ncf:
![Page 17: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/17.jpg)
Normation – CC-BY-SAnormation.com 17
ncfwww.ncf.io
Result
Example === 1000 wordsWithout ncf:
This is actuallyover-simplified:- No feedback- No exceptions for different OSes- No advanced options
![Page 18: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/18.jpg)
Normation – CC-BY-SAnormation.com 18
ncfwww.ncf.io
Result
Example === 1000 words
With ncf, automatic feedback:R: [DEBUG] Promise kept, not doing anything: Install package ntp in version latestR: [INFO] Promise repaired, made a change: Build file /etc/ntp.conf from template ntp.confR: [INFO] Promise repaired, made a change: Restart service ntpR: [DEBUG] Promise kept, not doing anything: Ensure that service ntp is running
![Page 19: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/19.jpg)
Normation – CC-BY-SAnormation.com 19
ncfwww.ncf.io
How does it work?
How does it work?CFEngine 3 can have self-contained “bundles” that you can call with parameters.
ntp
logger
package_install file_from_template service_restart
Outputs structured messages
![Page 20: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/20.jpg)
Normation – CC-BY-SAnormation.com 20
ncfwww.ncf.io
A layered approach
A layered approachServices
Techniques
IT Ops Knowledge
Generic methods
CFEngine basics
ncf internals
IT services: “Corporate web site”
Components of services: “Apache”
Shared information: “httpd” package name
Unit tasks: “Copy file”, “Install package”
CFEngine standard libraries
Framework config and magic :)
![Page 21: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/21.jpg)
Normation – CC-BY-SAnormation.com 21
ncfwww.ncf.io
A layered approach
A layered approachServices
Techniques
IT Ops Knowledge
Generic methods
CFEngine basics
ncf internals
IT services: “Corporate web site”
Components of services: “Apache”
Shared information: “httpd” package name
Unit tasks: “Copy file”, “Install package”
CFEngine standard libraries
Framework config and magic :)
![Page 22: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/22.jpg)
Normation – CC-BY-SAnormation.com 22
ncfwww.ncf.io
Each generic_method does one thing and one thing only
Philosophy
Core principles
Objective not subjective
KISSDRY
ExtensibleOpen source
![Page 23: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/23.jpg)
Normation – CC-BY-SAnormation.com 23
ncfwww.ncf.io
Available generic_methods
![Page 24: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/24.jpg)
Normation – CC-BY-SAnormation.com 24
ncfwww.ncf.io
Online documentation
http://www.ncf.io/pages/reference.html
![Page 25: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/25.jpg)
Normation – CC-BY-SAnormation.com 25
ncfwww.ncf.io
Current status
Project is young, but robustNeed more generic methods
Ohloh statistics:
Source: http://www.ohloh.net/p/ncf-project
Actually CFEngine
![Page 26: ncf - a powerful and structured CFEngine framework](https://reader034.vdocument.in/reader034/viewer/2022042714/5553eda4b4c90577468b45ee/html5/thumbnails/26.jpg)
Normation – CC-BY-SAnormation.com
Questions?
Check it out on:http://www.ncf.io/
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)