jim glenister ibm consultant, websphere software

®

IBM Software Group

© 2005 IBM Corporation

IBM Support for FIA (FIXML) Messaging Enterprise Service Bus View of Standards

The Evolution of Messaging for Financial Services

March 2005

Jim GlenisterIBM Consultant, WebSphere Software

IBM Software Group | WebSphere software

2

WMQ and FIXML Standards for Financial Services

POST TRADE VMU

- TRADE DATA (NOE) - CLEAR & SETTLEMENT

TRADING & REPORTING

MKT DATA ORDER REPORTING EXECUTION

MARKET DATA

RETRANSMIT REQUEST

VALUE ADDINGMKT INFORMATION PROVIDERS

PRIMARY FINANCIAL MARKETS SYSTEMS (EXCHANGES, NASD)

SECURITIES FIRM OFFICES

RETRANSMITTED DATA

LAST SALEBID - ASK

TRADES

SECURITIES FIRM

ORDER + TRADE PROCESSING

TRADESP + S

CONFIRMSMARGIN CALLS

ORDERS EXECUTIONS

ENTERPRISEMESSAGING

FINANCE INDUSTRY SHARED ENTERPRISE SERVICE BUS

ORDERS +INQUIRIES

REPORTS +CONFIRMS

MDS INQUIRIES

MKT INFO

TRADES COMPARISON

POSITION

CLIENT DIRECT MESSAGING

ORDERS +INQUIRIES

REPORTS +CONFIRMS

COLLATERALPOSITION

NET SETTLEMENTPOSITION

STP SERVICE PROVIDERCUSTODY + CLEARING

(OCC, DTCC, Other) Future SystemsT+1 (T+0) , STP

ORDERS EXECUTIONS

SECURITIES FIRM ACCOUNTINGCLEARANCE

SETTLEMENT

SCOPE OF FIXMLMESSAGING STANDARDS

INVESTMENT MANAGER

ACCESS GW

PRIVATE CLIENTACCESS GW


3

FIA to spearhead related efforts:

Message Descriptors

Standardize on the usage of Message Descriptors

Put and Get

Guidelines for setting fields and Options in an MQ-Put

Guidelines for setting fields and Options in an MQ-Get

Queue Definition

Queue Depth based on volume

Maximum Message Length

Message Data Formats

Standardize on the use of ASCII for FIXML messages

Dead Letter Queues

Processing messages from Dead Letter Queues

Channels Exit Routines

Hardware requirements / Software requirements

Establishing standards for message transmission

Exchanges and Firms to work with FPL to define a FIXML messaging API for re-transmission of messages

MessageDB

Origin

Destination

Date/Time

Sequence Nbr

Type


4

How WMQ (MQSeries) Works – FIA Standards

Message Queue Manager

Q1MessageQueuingService

MessageChannel

Agent

Requesting Application

PUT MSG

MQSeriesInterface

Message Queue Manager

MessageQueuingService

MessageChannel

Agent

Serving Application

GET MSG

MQSeriesInterface

MQSeriesInterface

Q1 Q2

XMIT Q Network

Serving Application

GET MSG

MSG MSG

DLQ


5

WMQ Queue Manager Structure – FIA Standards

QueueManagerKernel

Command Queue

Event Queues

Data Conversion Interface

Security Enabling Interface

Name Service Interface

PluggableMessageChannelAgents

PluggableTriggerMonitors

Channels

Queues

Performance

ExitsDLQ


6

Messaging Service Environment Characteristics

Multiple Platforms (zOS/OS390, UNIX , Windows NT/2000, AS/400, Tandem) "Store and Forward" of Messages Assured Delivery (Once, and only once) of Message Objects Persistent or Non-Persistent Messages (Designated by message) Priority Levels on Messages Purge of "Killer" Messages that Repeatedly Cause Failures Message Expiry (Time-to-live for a given message) 24 x 7 Continuous Operation Transactional Integrity as a Resource Manager and Recovery Coordinator Failover and Automated Recovery of Messaging Access to Business Services High Volume Performance with Peak Load Handling Mechanisms Flexible Message Resource Naming (Alias names that reflect business service ) Security (Protected message resources, data privacy via encryption, intrusion protection) Automated Management Controls (Programmed commands, monitoring, event alerts) Feedback: Confirmation and Notification Reports Message Logging (Optional redundant logging) Large Objects ( --> 100MB)


7

Messaging Application Characteristics

Multiple Platforms (zOS/OS390, UNIX , Windows XP/2000, AS/400, Tandem) Consistent APIs in Multiple Programming Languages (Java, C, Cobol, Others) "Business Service Name" Transparently Mapped to Message Destinations Transparent Placement of Messages on Local or Remote Queues Support for Synchronous and Asynchronous Style Shared or Exclusive Queue Usage Message Arrival Triggers Initiation of Server Processes Multiple Servers Concurrent Access to Queues with Preservation of Message Integrity Correlation of Requests and Corresponding Replies Selective "Get" of a Specific Message from the Message Queue Non-destructive Browsing of Message Queues Selective Commit/Rollback (Syncpoint) Controls for Unit of Work Integrity Support for XML Messages Applications Have No Communications Network Responsibility or Awareness Multiple Programming Models Supported by Messaging

Client-Server One-way Event Push (Asynchronous "Fire-and-Forget" Datagram) Publisher-Subscriber Object Oriented Design (e.g., JMS Message Driven Beans) Multi-step Workflow Processing


8

Messaging Architecture – Point-to-Point Patterns

Sv

Sv

Sv

Sv

Sv

Sv

Sv

Rq

Rq

Rq

Rq

Rq

Rq

Rq

Tp

MESSAGEHUB

Peer Applications 'One-way Event Push'

Peer Applications 'Requests-Replies'

Peer Applications 'Third-Party-Reply'

Peer Applications 'M x N via Mesh'

Peer Applications 'M x N via Hub'

SvRq M

Peer Applications 'One-way Event Push'(with Confirmation)


9

Messaging Architecture - Publish and Subscribe Patterns

Ms

Sb

Sb

Sb

Sb

Sb

Sb

Sb

Sb

Sb

Sb

Ms

B

Sb

Sb

Sb

Pb

Pm

Ps

Ps

Fs

Publish to Subscribers'Broadcast Datagram'

Publish to Subscribers'Multicast Datagram'

Publish to Subscribers'Selective by Topic'

Publish to Subscribers'Selective by Content via Subscriber Filters'


10

WMQ: Design for Performance

Exploitation of WMQ Capabilities Priority scheduling of messages and processes Thresholds on queues Load-responsive multiple server instantiation (e.g., Triggers) Alias queue name = service name (can inhibit G/P, change priorities) Message channel and queue activity (persistent, non-persistent) Logical unit of work (syncpoint) controls (MQCMIT, MQBACK) Batch transmissions across channels

Efficiencies for Throughput and Fast Response Long running tasks (eliminates startup/shutdown transients) Persistent-connection channels between queue managers Parallel processes (e.g., Logging can be asynchronous via MQ) 'Fast Path' techniques

Resources Allocated to Meet Critical Peaks (Chaos Control) Deferred processing of non-critical requests Deferred processing of secondary activities MQSeries Log on separate, high speed devices Page sets and buffer pools


11

WMQ High Availability - Cluster Groups

Transparent Access to QMs and Queues in a Cluster Workload Balancing via WLM Exit Failover to Surviving Members of Cluster Group Application Affinity (Requester-Server) if Required Administrative and Operational Support (e.g., Q Definitions, Suspend/Resume)

REQUEST

REPLYREPORT

REQUEST

REPLYREPORT

REQUESTER 1

REQUESTER n

REQUEST

REPLYREPORT

REQUEST

REPLYREPORT

REQUEST

REPLYREPORT

SERVER 1

SERVER 2

SERVER n

MESSAGE HUB SITE 1

MESSAGE HUB SITE 2

MESSAGE HUB CLUSTER GROUP SERVER CLUSTER GROUP

ROUTINGLOGGINGFORMATTINGASSURED DELIVERYRECOVERYREPORTING

ROUTINGLOGGINGFORMATTINGASSURED DELIVERYRECOVERYREPORTING

QMR1

QMRn

QMS2

QMSn

QMS1 SERVERAPPLICATIONREQUESTER

APPLICATION

QMH1

QMH2REQUESTERAPPLICATION

1 2

43

SERVERAPPLICATION

SERVERAPPLICATION


12

WMQ High Availability - z/OS V5.3 Shared Queue Groups

CF

Shared DB2

Shared Qs

WAS, CICS TS1

APPLICATIONS

zOS1

WMQ QM1DB21

Channel

WLM ARM RRS CS

WAS, CICS TS2

APPLICATIONS

zOS2

DB22WMQ QM2

Channel

WLM ARM RRS CS

SYSPLEX

Persistent shared messages kept in CF list structuresDefined as Shared Queue GroupsMessages logged and recoverable by MQBackup dataset is the log datasetPhysical message size still < 63KBPeer Recovery - Transactional recovery in concert with CS/390, CICS TS, DB2

Message Grouping supportAddresses 63KB shared message size limit in 5.2

SSL supportEnables MQ to MQ security

24x7 supportRemove the need for OS/390 IPLs with new MQ release


13

Message Flow Standards via WMQ (MQSeries) and ESB

FIXML ClientBusiness Transaction

Gateway Service API

FIXML InterfaceApplication Services

Message Queuing API

Messaging Services

Network Transport

BUSINESS SERVICE INTERFACEDIRECTORY NAME SERVICESMESSAGE LOGGING + ARCHIVINGSECURITY SERVICESEXCEPTION HANDLINGCONTENT BASED ROUTINGDATA TRANSFORMATIONRETRIEVAL (RESEND + RETRANSMIT)PUBLISH - SUBSCRIBE SERVICES

MESSAGE QUEUEINGASSURED DELIVERYPRIORITY DELIVERYCONFIGURATION DEFINITIONS (Qs)WMQ NETWORK DEFINITIONS (CLUSTERS)ROUTE AROUND FAILURES RECOVERYPROCESS TRIGGERINGEVENT MONITORINGTRANSACTIONAL INTEGRITYSECURE MESSAGE TRANSPORT

FIXML Service Provider

Message Queuing API

Messaging Services

Network Transport

WMQ TCP/IP (FIXML) Networks

WMQ FIXML Utility Interfaces Messaging Management and Monitoring Services

FIXML-MQ STANDARD SERVICES

Business Service API

WMQ (MQSeries)

WMQ Gateway

FIXML InterfaceApplication Services

WBI Message Broker

WMQ (MQSeries)MSG Q

MSG Q


14

WMQ and WBI Family Product Detail and Directions

Graham Oakes, IBM


15

Summary

IBM appreciates the opportunity to work with FIA to evolve the FIXML over WMQ standards needed for high performance and reliable communication.

The objective of an ESB approach is to transcend the WMQ transport layer with value add services that reduce the cost of developing and managing disparate interfaces. An FIA sponsored ESB can enable sharing of standards and components that implement them: A communications “architecture” that enables software applications that run

on different platforms and devices or which are written in different programming languages or which use different programming models or which use different data representations

to reliably communicate with no disruption to existing applications or interfaces.

IBM has the most comprehensive WMQ and ESB skills, experience and offerings that we can share with the FIA membership. The formation of a working group and named interfaces to pursue the standards is recommended as the next step.


16

FIA Standards Working Group Focus and Structure

FIA Enterprise Service Bus

BUSINESS SERVICES API

DIRECTORYSERVICES

SECURITYDATA

CONVERT

MESSAGELOGGING

RetransmissionArchive

EDUCATION+

TRAINING

MESSAGING NETWORKOPERATIONS

+SUPPORT

SYSTEMADMINISTRATION

+MANAGEMENT

LEGACYACCESS

MESSAGINGGATEWAYS

EXTERNALACCESS

APPLICATIONDESIGN

APPLICATIONSTANDARDS

MSG FORMATSTANDARDS

NAMINGCONVENTIONS

AVAILABILITYDESIGN

PERFORMANCEDESIGN +TUNING

SECURITYSTANDARDS

MQ NETWORKDESIGN

ARCHITECTURE+

DESIGN

BUILD+

TEST

INTEGRATION+

ROLLOUT

RULES BASEDROUTING

BUSINESSAPPLICATION

BUSINESSRECOVERYSERVICES

MAINTENANCE+

SUPPORT

MIGRATIONPLANNING

SYSTEM MGTINTEGRATION

APPLICATION DEVELOPMENT TOOLS + FACILITIES

REQUIREMENTSDEFINITION

WORKGROUPACCESS

WORKFLOWMANAGEMENT

PUBLISHSUBSCRIBE

DATATRANSFORM

WMQ (MQSeries)

WBI Message Broker

WBI Server (Flows)

WMQ Platforms z/OS

UNIX (Solaris, AIX, Linux, HP-UX)Windows XP/2003

TPFTandem

COMPLEXEVENTS

EVENTPROCESSING

FIA WMQ Standards


17

IBM WMQ and WBI Components of FIA (FIXML) ESB

FIXML Client Access Adapter

WBI Business Integration Services

WBI Message Broker - FIXML Gateway and Broker Services

BusinessEvent Rules

+Routing Logic

BusinessProcessManager

ServiceRegistry

RecoveryServices

MessageTransformation

Message Flow Management

Rules Based Routing

Transaction Standards

Data BaseStandards

RequesterInterface

WMQ Basic Infrastructure Services

Security

Standard Messages

LoggingServices

EventMonitor

ClientProfiles

ServiceMonitoring

+Recovery

Management

System Administration

+Resource

Management

Message Transformation

ServerInterface

Message Transformation

FIXML Service Provider Connector

Process State Management

APIServices

BusinessProcess

FlowRules

Business Process Interfaces(APIs, Objects, Adapters)

Message Formatting

Data Enrichment Services

Bridges and Adapters

FIXML Data(State, UDDI, Metadata)

Transaction State Management

ExceptionHandling

TriggerMonitor

Configuration Management

Channel Session State

Publish Subscribe

AlertNotification

ResendRetransmit

QueueManagement

Message Repositories

Data Transform

ServiceLocator

LogicalSession State

jim glenister ibm consultant, websphere software

Documents