hhm 6894 messaging apis for cloud, enterprise and digital applications

InterConnect2017

HHM-6894Messaging APIs for Cloud, Enterprise & Digital Applications

Matthew WhiteheadIBM MQ Development

• IBM Messaging APIs and Servers

• Languages, Wire Formats, and APIs

• Recap of the Key Messaging Features

• Features supported in the different

Messaging APIs

Agenda

IBM Messaging APIs & Servers

3 3/19/2017

IBM Messaging

• IBM MQ

• Enterprise & hybrid messaging

• Ideal for inter-cloud communication, cloud ingress/egress

• Deployable anywhere that can host VMs or containers

• MessageHub

• Bluemix message streaming service based on Apache Kafka

• Ideal for micro-service architectures

• Streaming & real-time analytics

• MQ Light Development Server

• Ideal for rapid development of nodejs, Ruby, Java apps

• Simple messaging interface designed with language specific idioms

• Develop on Windows/Linux/Mac, deploy against MQ or MessageHub

• MessageSight and Watson IoT Platform

• Internet of Things platforms

• Connect huge numbers of devices

• Well suited to low-bandwidth networks

IBM Messaging

IBM

MessageHub

MQ Bridge

MQ Light

Microservice

Applications

Apache

Spark

Watson IoT

Platform / Message Sight

IBM Messaging

IBM

MessageHub

MQ Bridge

MQ Light

Microservice

Applications

Apache

Spark

• C MQI

• MQ JMS

• XMS C/C++/.Net

• MQ Light/AMQP 1.0

• MQTT 3.1

• …

Watson IoT


IBM Messaging

IBM

MessageHub

MQ Bridge

MQ Light

Microservice

Applications

Apache

Spark

• MQTT v3.1

Watson IoT


IBM Messaging

IBM

MessageHub

MQ Bridge

MQ Light

Microservice

Applications

Apache

Spark

• MQ Light / AMQP 1.0

Watson IoT


IBM Messaging

Watson IoT


IBM

MessageHub

MQ Bridge

• Kafka

MQ Light

Microservice

Applications

Apache

Spark

Languages, Wire Formats, and APIs

10 3/19/2017

Wire Formats

APIs

Languages/

Runtimes


Wire FormatsMQ Wire Format

(& Local Bindings)

APIs

Languages/

Runtimes



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQIMQ Object

Oriented



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

Oriented



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS MQTT



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS MQ HTTPMQTT



(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS HTTPMQTT MQ LightMQ HTTP MQ Light


APIs


(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS MQTT MQ HTTP MQ Light



(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMS WCF JMS MQTTMQ Object

OrientedMQ HTTP MQ Light



(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

MQI XMSMQ Object



APIs


(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

MQI XMSMQ Object



APIs


(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

MQI XMSMQ Object



HTTP MQ LightMQ HTTP MQ LightAPIs


(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

MQI XMSMQ Object





(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

MQI XMSMQ Object


C#




(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net Ruby

MQI XMSMQ Object


C#




(& Local Bindings)

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

nodejs

Ruby

MQI XMSMQ Object


C#




(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

C

MQI XMSMQ Object



Cobol

PL/1

VB

C++

Perl ActiveX

WCF

Java

.Net

nodejs

RubyC#


(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMSMQ Object

OrientedWCF JMS HTTPMQTT MQ Light

Native C++ is now

stabilised.

Recommendations

are XMS or the C MQI

MQ HTTP MQ Light


Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

nodejs

RubyC#


(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Base Java is now

stabilised. JMS is the

recommended MQ-

Java interface

MQI XMS

MQ Object

Oriented

(Base Java)WCF JMS HTTPMQTT MQ LightMQ HTTP MQ Light


Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

nodejs

RubyC#


(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

MQI XMS

MQ Object

Oriented

(Base Java)

WCF JMS HTTPMQTT MQ LightHTTP MQ LightMQ HTTP MQ Light


Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

nodejs

RubyC#


(& Local Bindings)

APIs

Languages/

Runtimes

MQTT HTTP AMQP 1.0

Cobol

PL/1

VB

C++

Perl

C

ActiveX

WCF

Java

.Net

nodejs

RubyC#

MQI XMSMQ Object

OrientedWCF JMS HTTPMQTT MQ LightHTTP MQ LightMQ HTTP MQ Light


Wire Formats

C

Cobol

Java

WCF

PL/1

VB .Net

C++

Perl ActiveX

MQ Wire Format

(& Local Bindings)AMQP 1.0MQTT

nodejs

Ruby

HTTP

APIs

Languages/

RuntimesC#

MQI XMSMQ Object

OrientedWCF JMS HTTPMQTT MQ LightHTTP MQ LightMQ HTTP MQ Light


Recap of Key Messaging Features

Features supporting in IBM Messaging APIs

47 3/19/2017

Point-to-Point messaging

Publish/Subscribe messaging

Shared subscriptions

Message persistence

Message expiry

Message grouping

Message segmentation

Message selection

Local transactions

Global transactions

HA failover

MQ Messaging Features

Message properties

Asynchronous consume

Message browsing

Qualities of service

• At most once msg delivery

• At least once msg delivery

• Exactly once msg delivery

MQ Wire Format MQTT HTTP AMQP 1.0

~

MQI MQ OO MQTT MQ HTTP MQ Light

~

Examples

C/MQIstrncpy(od.ObjectName, “Q1”, (size_t)MQ_Q_NAME_LENGTH);

MQOPEN(Hcon, &od, O_options,&Hobj, &OpenCode, &Reason);

JMSdestination = session.createQueue(“Q1”);

producer = session.createProducer(destination);

nodejs/

MQ Light

Notes:

• ~ can be used indirectly

at the subscriber

var options = { ttl: “300000” };

client.subscribe(“topic1”, “share1”, options, msgArrivedCallback);

Point-to-point messaging


~


Examples

C/MQI

od.ObjectString.VSPtr=“my/first/messaging/topic”;

od.ObjectString.VSLength=(MQLONG)strlen(argv[1]);

od.ObjectType = MQOT_TOPIC;

MQOPEN(Hcon, &od, O_options,&Hobj, &OpenCode, &Reason);

JMSdestination = session.createTopic(“my/first/topic”);

producer = session.createProducer(destination);

nodejs/

MQ Light

client.send(“my/first/messaging/topic”, body, options,

callback);

Notes:

• ~ can be used

indirectly

Publish/Subscribe Messaging


~


JMS 2

Examples

C/MQI

JMS 2

MessageConsumer messageConsumer =

session.createSharedConsumer(topic,"mySubscription");

MessageConsumer messageConsumer =

session.createSharedDurableConsumer(topic,"myDurableSub");

nodejs/

MQ Light

client.subscribe(pattern, “myShare”, options,

function(err, pattern) { // Handle errors });

N/A

Notes:

• ~ can be used indirectly

• Not supported by C MQI

• Primarily added to MQ for

JMS 2 support

• MQI has other ways of

achieving similar pattern

Shared Subscriptions


~


~ ~

Examples

C/MQIMQMD md = {MQMD_DEFAULT};

md.Persistence = MQPER_PERSISTENT;

…

MQPUT(Hcon, Hobj, &md, &pmo, messlen, buffer, &CC, &RC);

JMSMessageProducer producer = session.createProducer(dest);

producer.setDeliveryMode(DeliveryMode.PERSISTENT);

nodejs/

MQ Light

var options = { qos: mqlight.QOS_AT_LEAST_ONCE };

client.send(topic, body, options, callback);

Notes:

• ~ can be set indirectly

Message Persistence



Examples

C/MQIMQMD md = {MQMD_DEFAULT};

md.Expiry = 3000; // Expire in 5

minutes

…


JMSMessageProducer producer = session.createProducer(dest);

producer.setTimeToLive(300000); // Expire in 5 minutes

nodejs/

MQ Light

var options = { ttl: 300000 }; // Expire in 5 minutes


Notes:

• MQ expiry value is ‘tenths of a

second’

Message Expiry



Examples

C/MQI

JMSString selector = “NewsType = ’Sports’”;

MessageConsumer consumer =

session.createConsumer(queue, selector);

nodejs/

MQ LightN/A

Notes:

• Use on MQOPEN to filter

queued messages

• Applies to message properties

od.SelectionString.VSPtr=“Colour=‘red’”;

od.SelectionString.VSLength=MQVS_NULL_TERMINATED;

MQOPEN(Hcon, &od, O_options, &Hobj, &CC, &RC);

Message Selection (1)



Examples

C/MQI

JMSString selector = “NewsType = ’Sports’”;

MessageConsumer consumer =

session.createConsumer(queue, selector);

nodejs/

MQ LightN/A

Notes:

• Use on MQSUB to filter

published messages

• Applies to message properties

sd.SelectionString.VSPtr=“Colour=‘red’”;

sd.SelectionString.VSLength=MQVS_NULL_TERMINATED;

MQSUB(Hcon, &sd, &Hobj, &Hsub, &CompCode, &Reason);

Message Selection (2)

• Ability to put or get multiple messages to/from a destination, either all of them or none at all

• Messages are hidden from other applications until the transaction is committed.

• If a failure occurs mid-sequence, the transaction is rolled back and none of the messages are delivered. Other applications are never aware of the messages.

• If everything completes successfully, the messages are made available to other applications.

Local Transactions



Examples

C/MQI

pmo.Options = MQPMO_SYNCPOINT;


MQCMIT(Hcon, &CC, &RC);

JMS

session = connection.createSession(true, Session.AUTO_ACKNOWLEDGE);

// Get and put messages

session.commit();

nodejs/

MQ LightN/A

gmo.Options = MQGMO_SYNCPOINT;

MQGET(Hcon, Hobj, &md, &gmo, bufflen, buff, &msglen, &CC, &RC);


Notes:

• Only queue manager

resources involved

• JEE containers manage

JMS transactions for you

Local Transactions

Similar to local transactions, but with multiple resources to update e.g.

• Application consumes 5 related messages from a queue and writes them all to a database

• If everything happens successfully, the messages are guaranteed to be removed from the queue and written to the database

• If a failure occurs, the messages are rolled back to the queue and none are in the database

• Potential for a transaction to go in-doubt needing manual resolution

Global Transactions



Examples

C/MQIMQBEGIN(Hcon, &bo, &CC, &RC);

// Read from database




JMS

session = connection.createXASession();

// Read from database, put messages

xaResource = session.getXAResource();

xaResource.prepare(xaId, false);

xaResource.commit(xaId, false);

nodejs/

MQ LightN/A

Notes:

• Multiple resources coordinated

together (e.g. MQ and

Database)

• In this example the queue

manager is coordinating the

transaction

• Can be coordinated by DB/WAS

etc.

• Can result in in-doubt

transactions if a resource fails

mid-transaction

• In JMS/JEE the container does

the work

• MQ must be configured with the

DB XA libraries

Global Transactions

Describes the level of assurance given that a message will be delivered

• At most once delivery

The message might not arrive, but it is guaranteed that you won’t receive duplicates

• At least once delivery

The message will definitely arrive, but you may receive duplicates

• Exactly once delivery

The message is guaranteed to be delivered once and exactly once.

Often considered to be the ‘best’ option but brings with it various additional costs and complexities

Where are you measuring from?

Quality of Service



Examples

C/MQImd.Persistence = MQPER_NOT_PERSISTENT;

…


JMS

nodejs/

MQ Light

var options = { qos: mqlight.QOS_AT_MOST_ONCE };


producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);

session = connection.createSession(transacted,

Session.AUTO_ACKNOWLEDGE);

producer.send(destination, message)

At-Most-Once Delivery



Examples

C/MQI

md.Persistence = MQPER_PERSISTENT;




JMS

nodejs/

MQ Light

var options = { qos: mqlight.QOS_AT_LEAST_ONCE };


Notes:

• Can’t always just turn at-least-

once ON with a single

parameter

• Relates to several areas:

• Reliability of network protocol

• Persistence of messages

• Durability of subscriptions

• ACK features in client libraries

producer.setDeliveryMode(DeliveryMode.PERSISTENT);

session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);

message.acknowledge();

At-Least Once Delivery



Examples

C/MQI

MQBEGIN(Hcon, &bo, &CC, &RC);




JMS

session = connection.createXASession();

// Put messages

xaResource = session.getXAResource();

xaResource.prepare(xaId, false);

xaResource.commit(xaId, false);

nodejs/

MQ LightN/A

Notes:

• Requires XA/Global

transactions

• Useful if messages MUST

arrive and MUSTN’T be

duplicated, but:

• Network flow more

complicated

• Might require a human

to resolve in-doubt

transactions

Exactly-Once Delivery


N/A N/A N/A N/A


N/A N/A N/A N/A N/A

Client Built-in HA reconnect?

C MQI Client

JMS MQ Client

Cobol MQI Client

MQ Base Java Client

C++ MQ Client

XMS C/C++/.Net

VB

.Net WCF

MQ Light Node/Ruby/Java/Python clients

Notes:

• Really a feature of the client

library rather than the wire

format or API

HA Failover/Auto-Reconnect


Not base java

Message properties


Not base java

or .Net/C#

some langs

Asynchronous consume


Message browsing

Miscellaneous…


Some langs

Message segmentation


Message grouping

InterConnect2017

Questions?

69 3/19/2017

InterConnect2017

Thank you

IBM Messaging Developer Centre

developer.ibm.com/messaging

IBM Messaging Youtube

www.youtube.com/IBMmessagingMedia

LinkedIn

ibm.biz/ibmmessaging

Twitter

@IBMmessaging

IBM MQ Facebook

facebook.com/IBMMQ

InterConnect2017

IBM MQ Sessions at a Glance

71

Monday

1:00-1:45pm 6880 IBM MQ Appliance: Messaging in a Box

6894 Messaging APIs

2:00-2:45pm 6879 IBM MQ Advanced

3:15-4:00pm 6882 What’s New in the World of IBM MQ

4:15-5:00pm 3559 Banrisul Bank’s Migration to the MQ Appliance

6904 Help Shape the Future of IBM MQ

6:00-6:20pm 7441 IBM MQ in the Cloud

Tuesday

11:30-12:15pm 1487 IBM MQ Versus Open Source Messaging

1:30-2:15pm 1183 IBM MQ Appliance: Best Practices from the Field

2:30-3:15pm 6881 Unlock the data flowing through the infrastructure

3:45-4:30pm 4555 Allstate’s Experience with the MQ Appliance

4:15-4:35pm 7440 Managed Event Streams

4:45-5:30pm 5214 Managing Loyalty Programs with IBM MQ

6895 IBM MQ for z/OS: The Latest and Greatest

Wednesday

8:00-8:45am 2699 IBM MQ for z/OS: Performance Tuning

6885 Deploying IBM MQ into the Cloud

6906 IBM MQ Appliance: Hands-on Lab (8:00-9-45)

10:15-12:00pm 6907 IBM MQ Hybrid Cloud: Hands-on Lab

11:15-12:00pm 6884 Designing MQ for the Cloud Generation

6904 Help Shape the Future of IBM MQ

1:00-1:45pm 6903 Share your Experiences with the IBM MQ Experts

2:00-2:45pm 3695 How Banco Bradesco Manages IBM MQ for z/OS

3:15-4:00pm 6892 Availability and Scalability with MQ Clusters

4:15-5:00pm 6891 Making MQ Resilient across DCs and the Cloud

6893 Keep Out the Bad Guys by Securing MQ

Thursday

8:30-9:15am 6886 Simple, Resilient Messaging with MQ Appliance

9:30-10:15am 6878 You Need MQ Messaging!

6887 Managing MQ Messaging in the Hybrid Cloud

6890 New Tools and Interfaces to Manage IBM MQ

10:30-11:15am 6889 Help! Has Anyone seen my MQ Message?

Find us in the EXPO: Hybrid Integration Booth, Messaging Ped

72 3/19/2017

Notices and disclaimers

Copyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and

the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actionsthe customer may need to take to comply with such laws. IBM does not

provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

73 3/19/2017

Notices and disclaimers continued

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®,Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli® Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml

hhm 6894 messaging apis for cloud, enterprise and digital applications

Software