wtui10 - effective administration in ibm integration bus
DESCRIPTION
The latest release of IBM Integration Bus includes many features that make administering the product easier. Come along to this session to discover the right ways to effectively administer and operate the product, and learn tips and tricks that should be in every IBM Integration Bus administrator's toolbox.TRANSCRIPT
© 2014 IBM Corpora/on
Please Note IBM’s statements regarding its plans, direc/ons, and intent are subject to change or withdrawal without no/ce at IBM’s sole discre/on. Informa/on regarding poten/al future products is intended to outline our general product direc/on and it should not be relied on in making a purchasing decision.
The informa/on men/oned regarding poten/al future products is not a commitment, promise, or legal obliga/on to deliver any material, code or func/onality. Informa/on about poten/al future products may not be incorporated into any contract. The development, release, and /ming of any future features or func/onality described for our products remains at our sole discre/on.
Performance is based on measurements and projec/ons using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considera/ons such as the amount of mul/programming in the user’s job stream, the I/O configura/on, the storage configura/on, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
1
What will this session cover? • WMB and IIB have support for two main user roles
– Integra/on Developer – Administrator
• This session will aim to cover accepted best prac>ce for the most typical administrator tasks – Including some essen/al “top /ps”
• Topics to discuss – Tools of the trade – Common administra/ve tasks
• I’ll focus on WMB V8 and IIB V9 – but will point out some V10 Open Beta improvements too!
Interac>on With Tools
Integra>on Toolkit Command line
Integra>on Node
Third Party Tools
Integra>on API (CMP)
REST API
Integra>on Explorer
Web Admin
REST API WAS Admin
Integra>on Toolkit View for Developers
Integra>on Explorer
• Management op/on designed for administrators
• Plug-‐in to MQ Explorer
• Extra features − Create/Manage Configurable
Services
− Performance Views
− Group integra/on nodes
− Administra/on Log
− Administra/on Queue
− Manage sta/s/cs & tracing
BIP1121I: Creates an execution group. Syntax: mqsicreateexecutiongroup brokerSpec -e egName [-w timeoutSecs] [-v traceFileName] Command options: 'brokerSpec' is one of: (a) 'brokerName' : Name of a locally defined broker (b) '-n brokerFileName' : File containing remote broker connection parameters (*.broker) (c) '-i ipAddress -p port -q qMgr' : hostname, port and queue manager of a remote broker '-e egName' name of the new execution group '-w timeoutSecs' maximum number of seconds to wait for the execution group to be created '-v traceFileName' send verbose internal trace to the specified file.
Command line tools • A wide selec>on of tools for scrip>ng ac>ons • Requires a configured environment
– mqsiprofile or mqsicommandconsole (Windows) mqsiprofile (Linux/UNIX) JCL or ISPF (z/OS)
• Most commands work against local or remote integra>on nodes
Integra>on API (CMP) • Java interface that enables the administra>on tools • Use for custom administra>on requirements • Fully documented and samples available
• MB v8 and later allows you to create and edit message flows too – Build your en/re system programma/cally!
Web Visualisa>on and Analy>cs § A comprehensive tool for web management
– Manage all integra/on resources from zero-‐footprint client – Analyze integra/on performance in real-‐/me – Supported on a variety of browsers: IE10, Firefox, Safari… – Complements MQ Explorer and WAS Admin consoles
§ Managing Integra>on Resources – View top-‐level integra/on node proper/es – Add/remove/change integra/on servers – Start/Stop integra/on data flows – Role based access to control usage – Advanced op/ons include data replay, policy & monitoring – Exploits underlying public REST/JSON API
§ Integra>on Performance Analysis – Opera/onal experience; no developer interven/on required
• New and exis/ng flows can exploit without change
– Many metrics of integra/on flow available in real-‐/me • CPU & I/O /me shown by default in integra/on analyzer • Other metrics include thread, data sizes, errors…
– Flexible display includes data tables and flow profile • Drill down to understand detailed behaviour
– Exploits underlying MQTT web sockets technology • Asynchronous no/fica/on at low CPU cost
8
Significant Tool changes in V10 Open Beta
• Single install package and single installed component – Unit test integra/on node automa/cally started with IIB development tool – Can create addi/onal integra/on nodes as required
• No longer any MQ pre-‐requisite – All exis/ng MQ use-‐cases s/ll work (and now support remote MQ connec/ons) – All administra/on done through the web UI rather than the Integra/on Explorer – Remote administra/on commands need to point at the integra/on node’s administra/on
port rather than the queue manager
• Uses a single command ‘iib’ – e.g. iib studio, iib help, iib list – Equivalent of ‘mqsicommand’ is ‘iib command’ – Exis/ng commands con/nue to work
(no/ng remote administra/on caveat above)
Development Lifecycle and Environments
• Integra>on Developer – Develops message flows, message
models etc. – Unit Tests on local machine – Creates archive (BAR) files
containing required artefacts
• Administrator – Customizes BAR for target
environment (message flow proper/es including queues, database names etc.)
– Deploys BAR to target environment – Management and opera/onal control
Development Test QA Produc/on
Common Administra>ve Tasks
• Planning and configura>on – Bringing a new integra/on node online – Making an integra/on node highly available – Planning for disaster recovery – Securing an integra/on node
• Managing integra>on nodes – Deployment and redeployment – Understanding behaviour – Op/mizing and tuning – Migra/on – Maintenance
Bringing a new integra>on node online
• Typical steps required – Preconfigura/on (e.g. OS, userids) – Installa/on (MQ, IIB, addi/onal sojware) – Crea/ng the integra/on node – Crea/ng integra/on servers – Crea/ng Configurable Services – Deploying BAR files – Addi/onal configura/on (e.g. security, userids)
• And don’t forget:
– Documenta/on – Scrip/ng – Virtualiza/on
Scrip>ng and Automa>on
• Four general approaches for IIB provisioning – Crea/ng and configuring brokers manually – Use of commandline tools from within scripts (e.g. shell scripts, Jenkins, Ant, Maven…) – Hypervisor images for deployment directly on public or private cloud – Use of technologies such as Chef and Puppet for all environments (cloud, non-‐cloud)
• Choose an approach that enables your environment to be reproduced easily
<project> <target name=“deploy”> <cvs command=“checkout” … /> <mqsipackagebar … /> <mqsiapplybaroverride … /> <mqsideploy … /> </target> </project>
Making the integra>on node highly available
• How do I ensure that the integra>on node is con>nually processing messages? – Ac/ve/Ac/ve vs. Ac/ve/Passive – Agree SLAs with the business (% up/me)
• The integra>on node includes restart recovery of integra>on servers – But this is usually not sufficient on its own – Does not cover machine restart
• Two main op>ons – Third-‐party solu/ons (e.g. VCS/HACMP/MSCS) – Mul/-‐instance queue managers and integra/on nodes
• Distributed Cache – Built in cache support in 8.0.0.1 and later. – Enables HA scenarios
Planning for disaster recovery
• What would you do if your primary WMB loca>on goes down?
• Distribute WMB to mul>ple sites if possible – This introduces data replica/on and latency concerns
• Keep DR concerns separate from HA! – HA: Systems at a single site with a single configura/on
DR: Systems at mul/ple sites with replicated configura/ons – An HA failover can be a planned ac/vity – DR is unplanned.
Another approach to disaster recovery…
• Consider an architecture where no one service was taken for granted… – Trea/ng all elements of a system, regardless of business u/lity, as ‘disposable’ – Forces you to design in a way that enforces resilience and scalability – Requires up-‐front effort, but takes many concerns off the table (e.g. backup)
IIB IIB IIB
• IIB enables an architecture that removes single points of failure – Even easier with IIB V10 Open Beta due to remote queue manager support
…
Cloud Integra>on • The idea of disposability is facilitated by cloud…
– Trea/ng compu/ng resources like u/li/es such as gas, electricity – Pay for only what you use
• IIB has a range of IaaS cloud op>ons – Private Cloud
• IBM Integra/on Bus Hypervisor Edi/on • IBM Workload Deployer • Pure Applica/on Systems Papern • Chef scrip/ng
– Public Cloud • Support for public cloud providers, e.g. Sojlayer • Chef scrip/ng
– Future plans around IaaS (e.g. Sojlayer), PaaS (e.g. Bluemix) and SaaS connec/vity
• Consider virtualiza>on for new environments – Makes it easy to provision systems (and restore known state!) – Understand maintenance and performance aspects
SaaS
IaaS
PaaS
Securing WMB and IIB
• Simplified administra>ve security • 3 levels of authorisa>on for administra>ve ac>ons
– Reading (e.g. View integra/on node proper/es -‐ mqsilist) – Wri/ng (e.g. Change integra/on node proper/es -‐ mqsicreateexecu/ongroup)
– Execu/ng (i.e. star/ng and stopping) – See IIB Knowledge Center topic bp43540 for more details
• On two object types: – “Integra/on node” and “Integra/on servers”
• Administra>ve Security is not enabled by default – Enable through mqsichangebroker / mqsicreatebroker
• Access controlled using MQ queues on the integra>on node’s queue manager – Give authority to users or groups
+inq = Read
+put = Write
+set = Execute
Security Queues SYSTEM.BROKER.AUTH SYSTEM.BROKER.AUTH.<egname>
Managing what’s deployed
• It can be difficult to understand what individual deployed resources are used for • Applica>ons and libraries can allow you to understand why each file is there
– Applica/on: Encapsulates a single use case or scenario – Library: Promotes re-‐use of a shared set of files
– Libraries are sta/cally bound to the applica/on (V8, V9, V10 OB) or shared (V10 OB only)
• Concepts are shared between developers and administrators – The developer chooses to create an applica/on or library; the collec/on is then carried all the way through
to the run/me
Backing Up
• Consider using mqsibackupbroker to backup the integra>on node’s configura>on – Ideally, ajer all configura/on changes – Best run when stopped – But can be used whilst ac/ve as long as not undergoing configura/on changes
• Also consider any other required resources, for example: – Database tables – Source artefacts (message flows, BAR files)
• Ensure regular restore tes>ng – Prove your process works
• Paeern to recons>tute message flows from a running integra>on node – Available on MQSeries.net
Understanding integra>on node behaviour • The tools include a lot of informa>on that is useful to the administrator
– Administra/on queue and log, Message flow and resource sta/s/cs – Warnings when features that affect performance are enabled
• Use this informa>on to understand recent configura>on changes
− How the integra/on server is performing, connected endpoints, etc.
Understanding integra>on node behaviour • The Ac>vity Log shows you all recent ac>vity on a message flow or resource manager
– For example, “Show me all recent JMS ac/vity” • Visible in Integra>on Explorer and/or wrieen to a file • Customisable rota>on rules (based on age or size) and content
Op>mizing and tuning
• The tools allow you to modify the configura>on opera>onally • These tweaks more efficient to make than modifying message flows • Encourage developers to create message flows that enable opera>onal tweaks to be made
– User-‐defined proper/es, Configurable Services, User-‐defined configurable services, Policies
Managing Unresponsive Integra>on Flows § Target unresponsive flows through policy to improve overall system reliability
– Addi/onal WLM op/on aimed at unresponsive integra/on flows – An integra/on flow can become unresponsive for mul/ple reasons
• e.g. Wai/ng for external system, infinite loop, deadlock, malformed XML
§ Flexible configura>on, ac>ons and repor>ng op>ons – Specify threshold at which flows are considered unresponsive, e.g. 30 seconds for processing
• Configured via WLM policy, or directly on the flow in the BAR file
– Define ac/on to trigger when flow considered unresponsive • Administra/ve no/fica/on through a new “/meout exceeded” event message
– If flow eventually con/nues through to comple/on, a second event is published • Restart the integra/on server (execu/on group) on which the unresponsive flow is running
– New command op/on to forcibly stop integra/ons manually: mqsistopmsgflow –f
25
Waiting for response
Version-‐to-‐Version Migra>on § Migra>on to IIB V9 from WMB V6.1, V7 and V8
– All development assets (e.g. message flows, ESQL, DFDL, Java, Maps and XSLT) import directly • Right-‐click convert ac/on for pre-‐V8 maps; some manual tasks may be required
– Migrate brokers using a single command, or create new integra/on nodes for phased migra/on • No redeployment necessary when using built-‐in migrate command • All exis/ng BAR files can be deployed to IB V9 integra/on nodes without change
§ Migra>on commands for in-‐place migra>on – Includes migra/on of configura/on data including databases, queues and registry – Forwards and backwards migra/on of exis/ng components, in situ
• mqsimigratecomponents command (includes –t op/on for rollback to V7 and V8)
§ Flexible co-‐existence op>ons remove the need for addi>onal hardware when migra>ng – IB V9 co-‐exists on the same OS with all previous MB versions – MQ V7.5.0.1 required for all IB V9 integra/on nodes
• MQ V7.5.01 supported with V7 and V8 brokers for the purposes of V9 migra/on • For V6.1 migra/on, upgrade MQ and MB simultaneously
26
4 3 2 1 Install IB V9 Stop broker Run migrate command Start broker
4 3 2 1 Install IB V9 Create new broker Deploy exis/ng assets Stop old broker
or
Maintenance
• Schedule regular maintenance windows – IBM recommends that you are on the latest maintenance level – Plan exactly what will be applied and when – A highly available environment ensures that there is no down/me
Summary
• We have discussed a number of different ways that help ensure that administra>on is trouble-‐free
• My Top Tips
– Always ensure your environment is reproducible – Treat DR and HA separate – Ensure regular backups – Encourage developers to create message flows that enables opera/onal tweaks to be made
– Schedule regular maintenance windows