puppet and telefonica r&d

PuppetandTelefónica I+D

Puppet Camp Barcelona 2013

Xavi Carrillo

01Introduction

3

Introduction01

Who am I?

§ Puppet user for the last 2 years§ Working at Telefonica I+D for 4 months§ Telefonica Digital curently involved in a process

of modernization§ Helping on that by puppetizingTelefónica I+D

4

What do we do at I+D?01

5

Initiatives01

02The Problem Domain

7

Environment02

Development

Staging

Production

x7

8

The problem domain02

§ Machine Setups: manually (howtos,etc)§ Environments: docs, wikis.§ Releases: Hand over an RPM…

§ At least one SysAdmin per initiative

9

The needs

• Infrastructure for many sysadmins, initiatives and environments.• Each initiatives’ setups are isolated.• Some manifests are common.• User-friendly repo management.• Updates must be automatic, with some degree of control.

02

03The Solution

11

The Solution03

12

Git Frontends

Internal Github

§ Licencing§ Proprietary

(can’t add post-receive hooks)

03

13

Git Frontends

Git + Gitolite or Gitosis + Gitweb

§ Takes time (install, make it work, learn)

§ Not the most user friendly combo.§ Ugly

03

14

Gitlab03

§ Open Source§ Free§ Easy to install§ Easy to administrate§ LDAP integration§ Owner of repo can manage

team members§ Looks great!

04Implementation

16

Products + Environments

• Puppet Environments = Initiative + Environment

• Git repos = Initiative + ‘_ puppet’cdn_puppetm2m_puppet

• Git branches = development, staging and production

• Puppet Agent: --environment cdn_dev or m2m_sta

04

18

Hooks

Pre-receive§ Puppet Style Guide§ Puppet parser

04

Post-receive§ Updates the puppet master

19

Hooks04

pushorigindevelop

pre-receive

YES?

post-receive

Update theinitiative’smanifests

Commit goes through

sysadmins

20

The Big Picture04

CDN

CDN

M2M

M2M

cdn_dev

m2m_staDEVELOPMENT

STAGING

sysadmins

21

Yeah!

• Different sysadmins can configure their systems in several environments in a centralized way

• All machines in a controlled state• Inventory• Changes are tracked (commits)• No more how-tos: a git repository is all you

need• Deploying new environments is much

easier• Reduce duplication of work (Apache,

MySQL, etc)

04

22

Work in progress

Production is not puppetized yet

04

05Final notes

24

Evangelization05

• [email protected]• Separate data from code (Hiera)• Always have the future in mind

(parametrized classes, OS changes, multiple networks, etc.)

25

Vagrant

Vagrant::Config.run do |config|config.vm.box = “RH5.8_2”config.vm.box_url = http://xxx/rh58_base.boxconfig.vm.host_name = “m2mvagrant2”

config.vm.provision :puppet_server do |puppet|puppet.puppet_server = "puppet.aislada“puppet.options = ["--environment","m2m_dev"]

endend

05

26

Mcollective

$ mco puppet runall 10

2013-03-13 14:26:50: Running all nodes with a concurrency of 102013-03-13 14:26:50: Discovering enabled Puppet nodes to manage

$ mco package status kernel

Summary of Ensure:

2.6.32-220.el6 = 1472.6.32-279.19.1.el6 = 192.6.32-220.7.1.el6 = 3

05

27

Mcollective – Detect performance issues

$ mco puppet summary

Total resources: ▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂min: 66.0 max: 4.7kOut Of Sync resources: ▇▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁min: 0.0 max: 7.0

Failed resources: ▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁min: 0.0 max: 1.0Changed resources: ▇▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁min: 0.0 max: 6.0

Config Retrieval time (seconds): ▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁min: 0.7 max: 1.2Total run-time (seconds): ▇▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁min: 1.2 max: 10.9

Time since last run (seconds): ▄▃▂▂▃▅▄▄▆▃▃▇▁▂▄▂▂▁▂▅min: 31.0 max: 1.8k

$ mco find -S "resource().total_time > 10"node-demo-fe-01

05

06Future

29

Future06

§ ENC§ Hiera§ Semi-automatic VM provisioning§ PuppetDB and Puppet Inventory§ Automated monitoring with Nagios (exported resources)

07Thanks and Questions

31

Thanks

§ Puppet Labs§ Telefónica I+D§ Thank you!

§ We’re hiring!

06

[email protected]

[email protected]

32

Questions?… and hopefully, answers

06

puppet and telefonica r&d

Technology