DevOps examples on NonStopTools OverviewCor Geboers, ATC Consultant

About me

Cor GeboersSenior Consultant in NonStop ATC, based in Belgium

35+ years in IT development and support25+ years NonStop experience

Working with middleware and new technologiesOSS believer since the beginning

What is DevOps?

DevOps Interaction (simplified)

4

Development

QA

Operations

Time

Examples in this presentation

– Using Ansible on NonStop

– Using Jenkins for Continuous Deployment

5

What is Ansible?

– Simple IT automation engine – cloud provisioning– configuration management– application deployment

– Designed for multi-tier deployments– Parallel execution on different nodes at the same time

– Uses no agents and no additional custom security infrastructure

– Easy to deploy

– Uses a very simple language (YAML)

Ansible Architecture

7

AnsibleMgmt Node

HostInventory Playbook

webserver

webserver

webserver

SOAP

SOAP

Switch

ssh

Ansible and NonStop

8

Why use a Tool like Ansible?

–Infrastructure as Code–Automation–Documented Progress–Deterministic Process–Idem-potency–Continuous Delivery

9

Ansible Playbook

–A playbook is the term that Ansible uses for a configuration management script–Is written in YAML–Contains list of tasks to execute sequentially

– Multiple nodes at once– Can change user using sudo

–Task contains name and module–Modules are built-in scripts

10

Ansible Module

–Set of scripts that come with Ansible–Extensible

– Development kit– Extensions local

–Version control– Preferred git

11

Playbook relationships

12

Playbook Play Host

Task Module

Ansible Inventory

–Inventory contains hosts– Text file– DNS names or IP addresses

–Hosts can be grouped–Configuration options for each host

– Authorization key– Default Username– TCP/IP address and port numbers

13

Inventory Example

14

nsblde9 ansible_port=22 ansible_user=cgeboersnsblde6 ansible_host=nsblde6.atc-hp.com ansible_port=22 ansible_user=corgnsx09 ansible_host=nsx09.atc-hp.com ansible_port=22 ansible_user=corg

[webservers]nsblde9 ansible_host=nsblde9.atc-hp.com ansible_port=22 ansible_user=cgeboersnsx09 ansible_host=nsx09.atc-hp.com ansible_port=22 ansible_user=corg

Installation

–Need console installation on Linux/Windows(Cygwin) system–Need valid NonStop SSL installation–Need private/public key

– Authorized_keys on managed node– Use SSHCOM to install public key for operator key

–Need “sudo” for scripts requiring SUPER access–Requires Python 2.5 or later

– Python 2.7.x provided as part of T1203 (coreutils scripts)– no other Python modules needed for NonStop

–No need for installing agents on the NonStop

15

Why Ansible on NonStop?

–Facilitates installation and deployment of tools–Think of development machines that might need additional subsystems–Configure using templates–Use Playbooks

–No real NonStop knowledge required once playbooks are written and tested

–Virtual NonStop Node deployments

16

Initial Testing

17

ansible all -m pingnsx09 | SUCCESS => {

"changed": false,"ping": "pong"

}nsblde6 | SUCCESS => {

"changed": false,"ping": "pong"

}nsblde9 | SUCCESS => {

"changed": false,"ping": "pong"

}

Example Playbook for NonStop

–Simple scenario–New (virtual) node configured–Need to deploy iTP Webserver using standard configuration–Need to deploy NSSOAP 4 inside created webserver–Operations to be done from management node

–Use variables for easy customization

18

Playbook install_webserver.yml

19

---- hosts: webservers

tasks:

- name: Check if distribution is already on systemstat: path="{{ dist_folder }}/{{ itp_version }}"register: sym

- name: Unpack iTP Webserver archivecommand: "/bin/pax -r -s:/usr/tandem/webserver:{{ dist_folder }}: -s:/usr/lib:{{ dist_folder }}/lib:

-s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax"when: sym.stat.islnk is not defined

- name: Setup answer filetemplate: src=templates/itpwebserver_setup.sh.j2

dest={{ dist_folder }}/itpwebserver_setup.shmode="u=rwx,g=rx,o=rx"

- name: Install iTP Webservercommand: "{{ dist_folder }}/itpwebserver_setup.sh"

Playbook customization

–Uses YAML files which have name of groups–Automatic activation from directory layout

20

---webserver_location: /home/hp/corg/tmp/webserverdist_folder: /home/hp/corg/distitp_version: T8996H03_01APR15_AFA_H329_01

Playbook output (1/2)

21

ansible-playbook install_webserver.yml

PLAY [webservers] **************************************************************

TASK [setup gather_subset=all] *************************************************ok: [nsblde9]

TASK [Check if distribution is already on system path={{ dist_folder }}/{{ itp_version }}] ***ok: [nsblde9]

TASK [Unpack iTP Webserver archive _raw_params=/bin/pax -rv -s :/usr/tandem/webserver:{{ dist_folder }}: -s :/usr/lib:{{ dist_folder }}/lib: -s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax] ***changed: [nsblde9]

TASK [Setup answer file dest={{ dist_folder }}/itpwebserver_setup.sh, src=templates/itpwebserver_setup.sh.j2, mode=u=rwx,g=rx,o=rx] ***ok: [nsblde9]

TASK [Install iTP Webserver _raw_params={{ dist_folder }}/itpwebserver_setup.sh] ***changed: [nsblde9]

PLAY RECAP *********************************************************************nsblde9 : ok=5 changed=2 unreachable=0 failed=0

Playbook repeated output

22

ansible-playbook install_webserver.yml

PLAY [webservers] **************************************************************

TASK [setup gather_subset=all] *************************************************ok: [nsblde9]

TASK [Check if distribution is already on system path={{ dist_folder }}/{{ itp_version }}] ***ok: [nsblde9]

TASK [Unpack iTP Webserver archive _raw_params=/bin/pax -rv -s :/usr/tandem/webserver:{{ dist_folder }}: -s :/usr/lib:{{ dist_folder }}/lib: -s:/usr/include:{{ dist_folder }}/include: -f /G/system/zossutl/t8996pax] ***

skipping: [nsblde9]

TASK [Setup answer file dest={{ dist_folder }}/itpwebserver_setup.sh, src=templates/itpwebserver_setup.sh.j2, mode=u=rwx,g=rx,o=rx] ***ok: [nsblde9]

TASK [Install iTP Webserver _raw_params={{ dist_folder }}/itpwebserver_setup.sh] ***changed: [nsblde9]

PLAY RECAP *********************************************************************nsblde9 : ok=4 changed=1 unreachable=0 failed=0

Ansible Templates

–Templates are processed by the Jinja2 templating language–Six additional variables can be used in templates

– sandboxed execution– powerful automatic HTML escaping system for XSS prevention– template inheritance– compiles down to the optimal python code just in time– optional ahead-of-time template compilation– easy to debug. Line numbers of exceptions directly point to the correct line in the template.– configurable syntax

23

Template Example

24

#!/bin/bash#cd {{ dist_folder }}/{{ nssoap4_version }}/bin./deploy.sh << EOF1{{ nssoap4_location }}nssoap4nssoap4{{ webserver_location }}

n44EOF

Template example 2

25

#!/bin/bash## setup for iTP Webserver requires /bin/cp !export PATH=/bin:$PATHcd {{dist_folder}}/{{itp_version}}./setup {{webserver_location}} <<EOFy n y 1 3 /G/ztc0 33000 /G/w33k /G/sas1/web33k y EOF

Ansible for NonStop usage?

– Other usages to think of:– Install database– Install Pathway environment– Install NSSOAP4 services

26

Jenkins – Continuous Integration

27

Java is a registered trademark of Oracle and/or its affiliates

What is Jenkins

–Open-source, continuous integration tool–Written in Java, forked from Hudson in 2011–http://jenkins-ci.org–Very active community, releases every few months–Very feature-rich–Runs on NonStop servers under OSS–Executes in web containers

28

Jenkins

Jenkins – Main Characteristics

–Easy to install on NonStop– Deploy in NSJSP– ..or run it standalone, as it embeds the Jetty web container

–Easy to configure– Web GUI is simple to use– Plug-in add additional functionality into existing configuration pages

–Lots of plug-ins!–Extensible

– Jenkins plugins are easy to create

–Distributed– Jenkins can distribute builds & tests to multiple machines. Can be running different OSes.

29

Global Features: Jenkins Dashboard– Dashboard showing status of all jobs, with links to go into

details about each job– Result of last run, as well as “weather report” shown for each

job– Create new jobs from Dashboard

– Create custom views– Can run jobs on demand from dashboard– Set up RSS feeds for results– Manage credentials

30

Java is a registered trademark of Oracle and/or its affiliates

Global Features: Credential Support

– Out of the box support for– Username/password– SSH– PKCS 12 Certificate

– Available plug-ins– LDAP– Kerberos– Active Directory– CAS– OpenID– SAML– Google OAuth Credentials– Many more, see

Jenkns Auth Plugins

31

Java is a registered trademark of Oracle and/or its affiliates

Global Features: Plug-Ins Categories– Source code management– Build triggers– Build tools– Build wrappers– Build notifiers– Slave launchers and

controllers– Build reports– Artifact uploaders– Other post-build actions

– External site/tool integrations– UI plugins– List View column plugins– Page decorators– Authentication and user

management– Cluster management and

distributed build– CLI extensions– Maven

– Parameters– iOS development– .NET development– Android development– Ruby development– Library plugins– Scala plugins– Miscellaneous– Uncategorized plugins

32

Java is a registered trademark of Oracle and/or its affiliates

Global Features: Job Triggers– On demand

– Scheduled

– Based on SCM update (polled)

– Based on build result

– Monitor folders (Files Found Trigger)

– Triggered from another Jenkins job, upon successful or failed completion (pipelines)

– Via URL, e.g. http://YOURHOST/jenkins/job/PROJECTNAME/build

– From a remote Jenkins master

– Others…– gerrit, – ivy-script, – groovy script, – pollurl, – generic script

33

Jenkins on NonStop - Installation hints- Example with Pathway Domains and 3GLs

34

Java is a registered trademark of Oracle and/or its affiliates

Jenkins on the NonStop - Installation Hints–Only run one copy listening on a particular port...MAXSERVERS 1 if using

NSJSP– multiple standalone “slave” copies can run on different ports CPUs/IPUs– they must be configured to use different work directories

–Expected startup exception: No suitable implementation found for Free Swap Space monitor – not an issue

–Give Jenkins plenty of heap space -Xms512m -Xmx512m–Some plugin installations require a restart of Jenkins, stop and restart the

process–Turn on the “auto refresh” for the job status page (“ENABLE AUTO REFRESH”

upper right hand corner of the job status page)

35

Example: Jenkins Continuous Deployment without downtime

–Develop serverclass in NSDEE–Use Source Control tool like git–Jenkins monitors git repository

– Can work from sources and check build– Can work with delivered binaries

–On successful modification deploy into a TS/MP domain

36

Works with your COBOL, C/C++, TAL,…or new technologies

NonStop Server

Jenkins Flow

37

Pathway Domain“%PM00”

$PM01

serverserverserver

$PM02

serverserverserver

Program file

1. Commit changes to repository2. Jenkins checks repository3. Create local copy4. Deploy to first Pathmon5. Deploy to second Pathmon

Thank youcor.geboers@hpe.com

38