Using JMS Queues with Oracle Data Integrator O R A C L E W H I T E P A P E R | N O V E M B E R 2 0 1 5
Using JMS Queues with Oracle Data Integrator
1 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
Table of Contents
Introduction 2
Main Article: Using JMS Queues with Oracle Data Integrator (ODI) 3
Configuring JMS Queues in Oracle WebLogic 12c 5
Configuring JMS Servers 7
Configuring JMS Modules 11
Configuring JMS Queues 16
Configuring Connection Factories 23
Configuring the XML and DTD Files for ODI 31
Configuring JMS Queues in Oracle Data Integrator 35
Configuration #1: JMS Queue Booking 35
Configuration #2: JMS Queue Revenue 35
Configuring JMS Queues in the ODI Topology 36
Configuring ODI Data Models for JMS Queues 42
Adding the ODI JMS Queue Knowledge Modules 49
Configuring an ODI Mapping to Consume JMS Queue Messages 51
Creating an ODI Mapping to Publish Messages to JMS Queues 55
Testing the JMS Queue Configuration 58
How to Add Messages to the JMS Queue 58
Executing the ODI Mapping to Publish Messages to the JMS Queue 66
Conclusion 68
Summary of Additional Resources 70
2 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
.Introduction
This article presents a demo on how to configure and use Java Message Service (JMS) queues with
Oracle Data Integrator (ODI). Two examples of how to use JMS queues with ODI are presented. The
first example demonstrates how to use ODI to consume JMS messages from JMS queues and load
their content into Oracle tables. The second example demonstrates how to use ODI to integrate data
from Oracle tables, transform it, and send it as a JMS message to a JMS queue.
In ODI, the JMS Queue XML technology will be used to demonstrate these two examples. The JMS
messages illustrated in this article are extensible markup language (XML) documents.
This article is divided into three main parts:
The first part shows how to configure two JMS queues using the WebLogic Server 12.1.3.0.0,
which includes creating the JMS servers, the JMS modules, the JMS queues, and the JMS
connection factories. ODI will consume JMS messages from the first JMS queue, and will
send JMS messages to the second JMS queue.
The second part shows how to configure these two JMS queues in ODI. The ODI discussion
includes how to configure the ODI physical data servers, the ODI data models, the ODI
knowledge modules, and the ODI mappings.
The third and final part shows how to test the JMS queues with ODI. JMS messages will be
added into a JMS queue using the WebLogic Console. Then, ODI will consume these JMS
messages, and load their content into an Oracle table. This Oracle table will then be used by
ODI as a source dataset to transform, and send a JMS message to another JMS queue.
This white paper was inspired by Oracle customers in the airline industry; they use JMS queues and
ODI to consume, transform, and send millions of JMS messages every day. Their JMS messages
include passenger bookings, reservations, ticketing, and other airline subject areas.
If you would like to download a free copy of this ODI demo, go to ODI 12.1.3 SmartExport Demo for
JMS Queues, and search for JMS.
https://java.net/projects/oracledi/downloads/directory/ODI/Repository%20Sampleshttps://java.net/projects/oracledi/downloads/directory/ODI/Repository%20Samples
3 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
Main Article: Using JMS Queues with Oracle Data Integrator (ODI)
Oracle defines the Java Message Service (JMS) Application Programming Interface (API) as a messaging standard
that allows application components to create, send, receive, and read messages. It enables distributed
communication that is loosely coupled, reliable, and asynchronous. The JMS API is based on the Java Platform
Enterprise Edition (Java EE).
The JMS API offers several advantages:
Neither the sender nor the receiver needs to know anything about each other; only the message format and
the destination are required. Thus, the JMS API enables distributed communication that is loosely coupled.
The sender and the receiver do not have to be available at the same time in order to communicate. The
sender can send a message to a destination, and the receiver can retrieve it at anytime.
A JMS provider can deliver messages to a client as they arrive; thus, the client does not have to request
messages in order to receive them. This is known as JMS Asynchronous Communication.
The JMS API enables reliable communication; the JMS API can guarantee that a message is delivered once
and only once. Additional levels of reliability are available within the JMS API architecture for applications
that cannot afford to miss or get duplicate messages.
The Oracle WebLogic 12c server is fully compliant with both the Java EE 5.0, and the JMS 1.1 Specification. The
WebLogic JMS architecture includes JMS servers that can host a defined set of JMS modules. JMS modules
contain destinations such as queues and topics - and resources such as connection factories. Client JMS
applications such as Oracle Data Integrator (ODI) can either produce messages to destinations, or consume
messages from destinations.
Figure 1 shows a conceptual representation of how the WebLogic JMS architecture can be used with ODI. ODI can
consume JMS messages from WebLogic JMS destinations. Likewise, ODI can send JMS messages to WebLogic
JMS destinations. In ODI, the content of JMS messages can be loaded into a SQL database and transformed by
any transformation engine, such as the Oracle database or a Hadoop cluster.
http://docs.oracle.com/javaee/6/tutorial/doc/bncdr.html
4 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
WebLogic JMS with ODI
Message Message
WebLogic JMS
Transformation
Engine
Consum
e Publish
ODI
Figure 1: WebLogic JMS with ODI
ODI supports two types of JMS destinations: JMS queues, and JMS topics. For each JMS destination, two types of
messages are supported:
5 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
JMS Messages This type of message contains rows of data known as text payloads and they can be
in the form of text, or binary documents. When JMS messages are processed by ODI, the JMS message is
mapped to a single ODI datastore. Each JMS message is stored as a record in the ODI datastore.
JMS XML Messages This type of message contains a complete XML data structure known as XML
payloads. In ODI, a JMS XML message is mapped to a relational schema known as the ODI XML
schema. The ODI XML schema contains - at a minimum - two ODI datastores representing the XML data
structure: the root element, and the child element.
JMS messages and JMS XML messages contain header fields and message properties such message ID, message
type, and message destination. ODI knowledge modules can filter messages based on header fields or message
properties; thus, reducing the number of messages retrieved from a JMS destination. Also, ODI can filter messages
using datastore columns from the ODI XML schema, but the filtering is applied after the messages are consumed
from the JMS destination.
This article focuses on how to create JMS queues in WebLogic 12c, and how to use ODI to consume and send JMS
XML messages from and to JMS queues, respectively.
For additional information on the JMS API and the ODI JMS technologies, go to the following resources:
Java Messaging Service (JMS) Specifications Introduction to the Java Message Service (JMS) Fusion Middleware Programming JMS for the WebLogic Server Using Java Message Services (JMS) in Oracle Data Integrator (ODI)
Configuring JMS Queues in Oracle WebLogic 12c
This section demonstrates how to configure JMS queues in WebLogic. Two JMS queues will be configured in this
section. The first JMS queue will be used as a source ODI will read and consume JMS XML messages from this
JMS queue. The second JMS queue will be used as a target ODI will send JMS XML messages to this JMS
queue.
Figure 2 illustrates the logical configuration of these two JMS queues:
https://jcp.org/en/jsr/detail?id=343http://docs.oracle.com/javaee/6/tutorial/doc/bncdq.htmlhttp://docs.oracle.com/cd/E24329_01/web.1211/e24387/fund.htm#JMSPG116https://docs.oracle.com/middleware/1212/odi/ODIKM/jms.htm#ODIKM1035
6 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
JMS XML Queues with ODI
WebLogic JMS
Oracle Data Server
ODI PublishConsume
7 | USING JMS QUEUES WITH ORACLE DATA INTEGRATOR
Figure 2: JMS XML Queues with ODI
Figure 2 above shows the configuration of two JMS servers: BookingServer, and RevenueServer. The first JMS
server, BookingServer, will host a JMS module called BookingModule. Two JMS resources will be created in this
JMS module: BookingQueue, and BookingConnection. BookingQueue is a JMS queue containing JMS XML
messages about passenger bookings. ODI will consume these JMS XML messages from this queue. The
BookingConnection is a JMS connection factory. A JMS connection factory is an object in WebLogic that
encapsulates connection configuration information. ODI will use a JMS connection factory to create a connecti