1 puppet and dsc. introduction and usage in continuous delivery process. viktar vedmich pavel...

1

PUPPET AND DSC. INTRODUCTION AND USAGE IN CONTINUOUS DELIVERY PROCESS.VIKTAR VEDMICHPAVEL PESETSKIY

AUGUST 1, 2015

2

Need to applynew Java version to

all web servers??????

CONFIGURATION ISSUES

JimUpdate_java

Web01x64 Web01

x86Web01

x86Web01

x64Web01

x64Web01

x86Web01

x64Web01

x64

Update_javaUpdate_java

3

CONFIGURATION ISSUES

DevWeb01Web02

Forget add Web03

Jim

Need to get all web servers for dev

Topology

Web01 Web03Web02 Web01 Web03Web02

DevWeb01Web02

Topology

update_java.sh update_java.sh

Why we still have security hole?

Need to update java

Why we still reproduce old version’s bug?

Jim

Why we still security hole?

4

REQUIREMENTS TO CONFIGURATION FLOW

Configuration point

Central configuration

storageInventory

We can install Java on all

platform types

We know were to install java

update

Configuration edit audit

Know were profiler installed

Configuration

5

• In software engineering, software configuration management (SCM) is the task of tracking and controlling

changes in the software, part of the larger cross-disciplinary field of configuration management.

• Configuration identification

• Configuration control

• Configuration status accounting

• Configuration auditing

• Environment management

WHAT IS SCM?

SOFTWARE СONFIGURATION MANAGEMENT  (SCM)

GOALS OF SCM

6

2

1

3

PUPPET AND POWERSHELL DESIRED STATE CONFIGURATION

CROSSPLATFORM

IDEMPOTENCY

DECLARATIVE APPROACH

7

PUPPET

• Puppet's declarative language

• Manifests

• Facter

• Client–server paradigm

• SSL authentication

• Report

HOW WORKS

8

HOW PUPPET WORKS

Agent

Request catalog(send node name, facts)

Master

Classify(Who is and what do?)

Class Class Class

Compile

Catalog

Defined system state

Apply

Query status

Enforce defined

Report

9

• PowerShell Declarative and Imperative Code

• Local Configuration Management (LCM)

•WMF

•MOF (Management Object Format) files

• Push/Pull

• Pull Mode over SMB, HTTP/S, OData

DSC - DESIRED STATE CONFIGURATION

HOW WORKS

10

HOW DSC WORKS

Previous.MOF Pending.MOFCurrent.MOF

Hello, My name is Server. Give configuration.

Download MOF and Resources

Compare config with Current

state. Install

LCM AgentPull Server

11

PUPPET MANIFESTS

Copy msi file

Install msi package

Copy config file

Ensure service is running

1

2

3

4

12

POWERSHELL CONFIGURATION

Copy source

Install service

Ensure service is running

2

3

1

13

java

net40

Site1 httpdSite1

Site2Site2IISIIS

NODES AND MODULES MAGIC

Node web01

Node web02

Module web

Module framework

IIS

Site1 httpd

net40

java

14

•Manage files and directories

• Install apache

• Configure hosts for apache

• Add second agent (Ubuntu)

DEMO

PUPPET

•Manage files and directories

• Install Windows Features

• Install SQL

• Run Windows PowerShell scripts (fill database)

DSC

15

2

1

3

SEPARATING DATA FROM DSC CODE

HASH TABLE

CONFIGURATION DATA VARIABLE

CONFIGURATION DATA FILE

16

•Write a configuration to be applied to a target node

• Use the configuration to generate a MOF file

• Generate a checksum file to accompany the MOF

• Create the pull server itself

• Deploy the MOF and checksum files to the pull server

• Configure the target node to use the pull server

DSC – PULL SERVER

PRACTICE – DEPLOY PULL SERVER

17

1 2 3

PARTIAL DSC CONFIGURATIONS

OS Configuration Fragment

• Join Domain

SQL Configuration Fragment

• .NET Framework 4.5

• Install SQL Server

• Configure SQL Instance Settings

OS and Application Configuration

• Service Configuration

• Application Configuration

18

Service Config

App Config

.NET 4.5

Install SQL

Config SQL

Join Domain

PARTIAL DSC CONFIGURATIONS

Node SQL01

OS Configuration

Join Domain

SQL Configuration

.NET 4.5

Install SQL

Config SQL

Service Config

Application Configuration

App Config

19

AZURE POWERSHELL DSC EXTENSION

•Write a configuration to be applied to a target node

• Publish Configuration to Azure

• Connect Azure Extension

• Create New VM with Extension

20

• Create Simple Configuration

• Publish to Azure

• Create New VM

• Check Status of Deployment

DSC DEMO

AZURE EXTENSION

21

•Built-In PowerShell DSC Resources

•Build Custom PowerShell DSC Resources

• PowerShell DSC Resource Designer Tool

DSC RESOURCE

DSC RESOURCE BASE WMF 4

22

2

1

3

DEPLOY AND DISCOVER DSC RESOURCES

DEPLOY

$env: psmodulepath (folder)                                     |- DSCResources (folder)                                                             |- <DSCResourceName1.psd1> (file, required)                                                            |- <DSCResourceName1.psm1> (file, required)                                                            |- <DSCResourceName1.schema.mof> (file, required)

DISCOVER Import-DscResource [-Name <ResourceName(s)>] [-ModuleName <ModuleName(s)>]

DEPLOY TO PULL SERVER

Archive Resource • ModuleName_Version.zip

Generate new Checksum• ModuleName_Version.zip.checksum

23

ROLES AND PROFILES : BUSINESS NEEDS

Web Application Mail

Business View

24

Node www150prod

mysql networking

apache tomcat

php jdk

ROLES AND PROFILES : NON-PATTERN APPROACH

Node www1dev

mysql networking

apache tomcat

php jdk

Node www2prod

mysql networking

apache tomcat

php jdk

Node mail1

exim networking

Need to change jdk version

Need to change 150 nodes!

25

ROLES AND PROFILES : IMPLEMENTATION REALITY

MailWeb

Application

Technical Reality

26

ROLES AND PROFILES : APPLYING PATTERN

Node www1dev

mysql networking

apache tomcat

php jdk

Node www2prod

mysql networking

tomcat

jdk

profile::webserver

apache

php

profile::common

networking

profile::appserver

tomcat

jdk

profile::sqlserver

mysql

apache

php

role:webappdb_prod

role:webappdb_dev

profile::appserver

tomcat

27

ROLES AND PROFILES : APPLYING PATTERN

profile::webserver

apache

php

profile::common

networking

jdk

profile::sqlserver

mysql

profile::appserver

tomcat

Node www1dev

role:webappdb_dev

Node www2prod

role:webappdb_prod

role:webappdb_prod

profile::common

profile::webserver

profile::webserver

profile::webserver

28

jdk {version -> 1.75}

ROLES AND PROFILES : EDITS WITH PATTERN

Node www1dev

role:webappdb_dev

Node www2prod

role:webappdb_prod

Need to change jdk versionjdk

profile::appserver

tomcat Need to change one appserver

profile

29

HIERA: ISSUE WITHOUT HIERA

Sets the binding variable for different

environments

Creates config based on variable name

30

HIERA: ISSUE WITHOUT HIERA

31

HIERA: MAIN CONCEPT

HIERA SEPARATESDATA

FROMTHE CODE

32

HIERA: LOOKUP

www1

Environment: production

Facts

Easy to editable data

in one file

33

TIME FOR DEMO

DEMO

34

DEMO FOR REAL CUSTOMER

35

APPLICATION INFRASTRUCTURE

mysite myservice sqlserverMy Application

Puppet Master

Zabbix

Log audit, monitoring data

LogstashApplication, IIS logs

Puppet dashboard

Puppet reports

Build Server

36

ANY QUESTIONS