InsertCustomSessionQR if Desired.

DevOps : IIB Administration for Continuous Delivery and DevOps

Rob Converyconvery@uk.ibm.com

Session 18863Thursday 3rd March 2016

Notices and Disclaimers• Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from

IBM.

• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

• Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

• Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

• Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

• References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

• Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

• It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

3

Notices and Disclaimers Con’t. • Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or

other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

• The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

• IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

4

Whatwillthissessioncover?

• Existing Administrations Tools for DevOps– Integration Developer– Administrator

• Developing Continuous Delivery– Tools of the trade– Common administrative tasks

InteractionWithToolsWMBV8andIIBV9

IntegrationToolkit Commandline

IntegrationNode

ThirdPartyTools

IntegrationAPI(CMP)

RESTAPI

IntegrationExplorer

WebAdmin

RESTAPI WASAdmin

InteractionWithToolsIIBV10

IntegrationToolkit

IntegrationNode

Commandline

IntegrationAPI(CMP)

RESTAPI

ThirdPartyTools

WebAdmin

(enhanced)

RESTAPI WASAdmin

IntegrationToolkitViewforDevelopers

IntegrationExplorer

• Management option designed for administrators• Plug-in to MQ Explorer• Extra features

− Create/Manage Configurable Services− Performance Views− Group integration nodes− Administration Log− Administration Queue− Manage statistics & tracing

− Replaced in v10 with Web UI

• A wide selection of tools for scripting actions• Requires a configured environment

– mqsicommandconsole (Windows)– mqsiprofile (Linux/UNIX) – JCL or ISPF (z/OS)

• Most commands work against local or remote integration nodes

• New “IIB” command in V10

Commandlinetools

BIP1121I: Creates an integration server.

Syntax: mqsicreateexecutiongroup integrationNodeSpec -e integrationServerName [-w timeoutSecs] [-v traceFileName]

Command options: 'integrationNodeSpec' is one of: (a) 'integrationNodeName' : Name of a locally defined integration node(b) '-n integrationNodeFileName' : File containing remote integration node connection parameters (*.broker) (c) '[integrationNodeName] -i ipAddress -p port' : hostname and port of a remote integration node. 'integrationNodeName' optional for validation. (d) '[integrationNodeName] -i uri' : uri for the remote integration node in the form tcp://[<user>[:<password>]@]<host>:<port>

If a user name and password are supplied, any URI Reserved characters in the password must be percent-encoded. '-e integrationServerName' name of the new integration server'-w timeoutSecs' maximum number of seconds to wait for the integration server to be created (default is 60) '-v traceFileName' send verbose internal trace to the specified file.

IntegrationAPI(CMP)

• Java interface that enables the administration tools• Use for custom administration requirements• Fully documented and samples available

• WMBv8andlaterallowsyoutocreateandeditmessageflowstoo– Buildyourentire systemprogrammatically!

Web Visualisation and Analytics

§ A comprehensive tool for web management– Manage all integration resources from zero-footprint client– Analyze integration performance in real-time– Supported on a variety of browsers: IE10, Firefox, Safari…– Complements MQ Explorer and WAS Admin consoles

§ Managing Integration Resources– View top-level integration node properties– Add/remove/change integration servers– Start/Stop integration data flows – Role based access to control usage– Advanced options include data replay, policy & monitoring– Exploits underlying public REST/JSON API

§ Integration Performance Analysis– Operational experience; no developer intervention required

• New and existing flows can exploit without change– Many metrics of integration flow available in real-time

• CPU & I/O time shown by default in integration analyzer• Other metrics include thread, data sizes, errors…

– Flexible display includes data tables and flow profile• Drill down to understand detailed behaviour

– Exploits underlying MQTT web sockets technology• Asynchronous notification at low CPU cost

IBM Integration Bus V9

V10WebUIAdministrationImprovements

• IIB Web UI becomes the primary means of runtime administration– Browser approach is lightweight and universal– Integration Bus Explorer no longer provided as part of IIBv10

• Programmatic intervention using public Java and REST APIs

• Integration Bus Explorer admin capabilities re-located– Policy Set configuration moved to the IIB Toolkit– Export Port Configuration for external HTTP listeners in Web UI– Integration Server Create, Rename and Delete added to Web UI– BAR file deployment added to Web UI

14

Understandingintegrationnodebehaviour

• The tools include a lot of information that is useful to the administrator– Administration log, Message flow and resource statistics– Warnings when features that affect performance are enabled– Activity Log – New in V10.0.0.4

• Use this information to understand recent configuration changes− How the integration server is performing, connected endpoints, etc.

SignificantToolchangesinV10

• Single install package and single installed component– Unit test integration node automatically started with IIB development tool– Can create additional integration nodes as required

• No longer any MQ pre-requisite– All existing MQ use-cases still work (and now support remote MQ connections)– All administration done through the web UI rather than the Integration Explorer– Remote administration commands need to point at the integration node’s administration port rather than the queue

manager

• New “IIB Console”– e.g. iib tools, iib help– All mqsi* commands continue to work

(noting remote administration caveat above)

DevelopmentLifecycleandEnvironments

• IntegrationDeveloper– Developsmessage flows,message

modelsetc.– UnitTestsonlocal machine– Createsarchive (BAR)files

containing required artefacts

• Administrator– CustomizesBARfortarget

environment (message flowproperties including queues,databasenamesetc.)

– DeploysBARtotargetenvironment– Management andoperational control

Development Test QA Production

ControllingIntegrationswithPolicy

• Integration Workload Management– Provide intelligent mechanisms to control processing speed– Most common scenario is to reduce back-end server load– Design allows more policy-based processing over time– Can be applied to new or existing integration data flows

• Policy defines threshold limits and relevant actions– Set thresholds for integration data flow throughput– Specify actions at threshold, for example:

• NOTIFY: Higher (or lower) than threshold generates publication• DELAY: Excessive workload will have latency added to shape throughput

• Web Console used to manage WLM policy– Sophisticated behaviour controllable by broker WLM policy– Workload can be managed across classes of message flows (e.g. batch vs. online)– Policies stored in local registry, and dynamically configurable– Developer can also specify limits as integration data flow properties

18

0

20

40

6080

100

120

140

160

180

200

0

20

40

60

80

100

120

140

160

180

200

020406080100120140160180200

Optimizingandtuning

• The tools allow you to modify the configuration operationally • These tweaks more efficient to make than modifying message flows • Encourage developers to create message flows that enable operational tweaks to be made

– User-defined properties, Configurable Services, User-defined configurable services, Policies

CommonAdministrativeTasks

• Planning and configuration– Bringing a new integration node online– Making an integration node highly available– Planning for disaster recovery– Securing an integration node

• Managing integration nodes– Deployment and redeployment– Understanding behaviour– Optimizing and tuning– Migration– Maintenance

Bringinganewintegrationnodeonline

• Typical steps required– Preconfiguration (e.g. OS, userids)– Installation (IIB, additional software)– Creating the integration node– Creating integration servers– Creating Configurable Services– Deploying BAR files– Additional configuration (e.g. security, userids)

• And don’t forget:– Scripting– Documentation– Virtualization

ScriptingandAutomation

• General approaches for IIB provisioning– Creating and configuring brokers manually– Use of commandline tools from within scripts (e.g. shell scripts, Jenkins,

Ant, Maven…)– Hypervisor images for deployment directly on public or private cloud– Use of technologies such as Chef and Puppet for all environments– Container technologies e.g.: Docker

• IIB Docker containers can now be used in production!

• Choose an approach that enables your environment to be reproduced easily <project>

<target name=“deploy”><cvs command=“checkout” … /><mqsipackagebar … /><mqsiapplybaroverride … /><mqsideploy … />

</target></project>

Continuous Delivery - Demo

• Following components– RHEL 7.1– IIB V10 Runtime with Node created and started– IIB V10 Toolkit with Git client installed– Git repository– Jenkins

– PerfHarness – http://ibm.biz/PerfHarness

23

Continuous Delivery - Demo

• Scenario– Background ongoing workload (simulated using PerfHarness)

• Simple Application running HTTPIn -> Transformation -> HTTPOut– Two Applications under development

• HTTPESQLEchoApp

• HTTPJavaEchoApp

24

Continuous Delivery - Demo

Time for a Demo

25

Further information available at:https://developer.ibm.com/integration/blog/2015/10/02/continuous-build-and-deploy-automation-with-ibm-integration-bus-v10-using-ant-git-and-jenkins/

Continuous Delivery

• Demo only covered the subset of what’s possible– Unit Test– Component Testing– System Testing

• Many different system test possibilities using the existing administration tools– Complete end to end testing including node creation– Provisioning to the cloud– Use the API to access Flow Exerciser data

• Possible to run against cut-down installation

26

Alternative Technologies

• Swiss National Train system – GitHub– https://github.com/SchweizerischeBundesbahnen/iib-maven-plugin

• Apache - IIB9 Maven Build Lifecycle– http://www.vadosity.com/maven/

• https://github.com/ot4i-cookbooks/ibm_integration_bus

27

MacOS

• IIB Toolkit is currently available on Windows & Linux

• New IIB on MacOS available via the V10 Open Beta program.– https://ibm.biz/iibopenbeta

• Limited runtime capability• Can connect and administer remote nodes.

28

This was Session 18863. The rest of the week ……Day Monday Tuesday Wednesday Thursday Friday

8:30 Nobody Uses Files Any More Do They? New Technologies for Old Technology, File Processing in MQ MFT and IIBRoom 225D

Common Problems and Problem Determination for MQ z/OS

MQ and CICS - Integration Options and CostsRoom 302B

10:00 Introduction to MQ - Can MQ Really Make My Life Easier?

Introduction to the New MQ Appliance

DevOps : Using z/OSMF to Provision MQ for z/OS

MQ for z/OS: The Insider Story MQ for z/OS, Using and Abusing New Hardware and the New v8 Features

11:15 Introduction to IBM Integration Bus on z/OS

MQ Security Bootcamp: Understanding SSL/TLS Principles -Taking You from Beginner to Expert, Part 2 of 3

MQ Labs - Room 303AOR Giving It the Beans: Using IBM MQ as the Messaging Provider for JEE Applications in IBM WebSphere Application Server

13:45 What's New in the Messaging Family - MQ v8 and More [z/OS & Distributed]

Thoughts on MQ Architecture & Design

DevOps : IIB Administration for Continuous Delivery and DevOpsRoom 304B

15:15 What's New in IBM Integration Bus and IIB on Cloud

IBM Integration Bus MQ flexibility [z/OS & Distributed]

MQ Security Bootcamp: Security Features Deep Dive -Securing your Enterprise,Part 3 of 3

IBM MQ: Are z/OS & Distributed Platforms Like Oil & Water?ORDevOps : Empowering the Delivery of Data Centre Operations Through Increased Automation and Cloud [Distributed] Room 304B

16:30 MQ Security Bootcamp: Securing MQ from End to End, Part 1 of 3

Digging into the MQ SMF Data Programming with PCF Messages

Monitoring and Auditing MQ