ibm websphere transformation extender (tx) high level and...

®

IBM Software Group

© IBM Corporation

IBM WebSphere Transformation Extender (TX) High Level and Technical Overview

IBM Software Group | WebSphere software

2

What Does IBM WebSphere TX Do?

It takes any kinds of data from their native forms

ANY Data

Processes them together, natively, with NO CODE

And outputs them into their native target formatsANY Data

Many-to-many Integration

Ex. - Hierarchical Data , Binary Data, Packed Data, Tabular Data, Relational Data, Nested Structures, Mixed-Type Data, and on and on…


3

Transformation Vision - Simplified

Transform Anything

Transform Anywhere

Everyone Transforms


4

What Makes WebSphere TX so Different?

Code-Free Design and Deployment

In-Process Data Validation

High Throughput Execution of Complex

Transformations and Enhancements

One Engine, Multiple Deployment Options


5

Examples of Complex Transformation Challenges Addressed by WebSphere TX

Data Enhancement

Lookups

Data Logic and Routing

Data Validation

Context Based Data and Usage Rules

Many to Many Transformation

Single-Transaction, interdependent data sets, conversions and logic

Mixed Data and Source/Target Types

Dependent Result Sets, Nested Structure Dependencies

Complex Data Transformation

Nested, Semi-structured And Hierachical Data Types

Dependent Inputs And Outputs

Binary, Packed, EBCDIC, ASCII, Mixed Character Data


6

One Engine, Multiple Deployment Options

WebSphere TX is the one transformation engine that can be reused

across all applications and infrastructures in the enterprise

Deploy the same WebSphere transformation asset across:

ESB

SOA

Web Services

SOAP

EJB

HTTP

Event driven

Batch

Java

C/C++

COBOL

Unix

Linux

Windows

z/OS

– Batch

– CICS

– IMS

– USS

30+ Adapters

ETL

Data quality

Enterprise

Applications

Application

Servers

WebSphere Message Broker

WebSphere ESB

WebSphere Process Server

WebSphere Partner Gateway

WebSphere Application Server

WebSphere DataStage


WTX Editions

WTX for Application Programming

Deploy transformations for

Custom programming environment

Third party App Servers

WTX with Command Server

Invoke transformation from command line, shell script or timer

WTX with Launcher

Activate transformation based on triggers such as –

File Creation, DBMS trigger, Message on a queue, timer etc

Graphical flow Editor to create Transformation flows.

WTX for Integration Servers

WTX for Message Broker merged with this edition

Supports integration with WESB to build BPM solutions

Simplifies installation and provides new bindings for WPS and WESB

7


WTX Enterprise Packs

Accelerator for key Enterprise Applications

WebSphere Transformation Extender Pack for PeopleSoft

WebSphere Transformation Extender Pack for SAP R/3

WebSphere Transformation Extender Pack for SAP XI

WebSphere Transformation Extender Pack for Siebel

8


WTX Industry Packs

Financial Services

FIX, NACHA, SEPA, SWIFTNet Fin and SWIFTNet Funds

Healthcare

HIPAA, HL7, NCPDP

Insurance

ACORD

EDI

X12, EDIFACT, Eancom, Odette, TRADACOMS

9


WTX Extension Pack

WTX Design Studio

Eclipse based tooling to develop, test, performance tune Transformations and metadata

WTX SDK

Programming API and documentations to invoke transformations from client application

WTX Trading Manager

Supports B2B Solution, complements Partner Gateway

WTX Launcher Agent

Inbound HTTP agent for WebSphere Transformation Extender with Launcher to listen

and respond to HTTP requests.

WTX Secure Adapter Collection

Extends technology adapters of the base edition to encrypt and decrypt data according

for popular standards

WTX SNMP Collection

Enable integration with network management applications

10


11

WebSphere TX –A Long And Storied Road

Jan 1994

V1.0

June 1995

V1.1

Nov 1995

V1.2

June 1996

V1.3

Aug 1997

V1.4

Dec 1999

V2.1

June 2000

V5.0

Sept 2001

V6.0

Jan 2002

V6.5

Jan 2003

V6.7

June 2004

V7.5

July 2005

V8.0

Q4 2007

V8.2

Proven

Reliable

Mature

Innovative

Q3 2006*

V8.1

*-Scheduled

V….

….

®

IBM Software Group

© IBM Corporation

Developing a WebSphere TX Solution


13

Adapter Type Tree

The Components of a WebSphere TX Solution

A WebSphere TX solution is composed of the following:

Rules Type Tree Adapter

MappingSource Destination

In other words…

An adapter is a technical connector which can be

―plugged‖ into a source or destination - without parsing!

A Type Tree is a graphical representation of meta-data

Rules define how a Source structure is transformed to

a Destination structure

Together these components form a WebSphere TX Map


14

Adapter Type Tree

Adapter Type Tree

Adapter Type Tree

However…many to many mapping is also possible:

Rules

MappingSources Destinations

A Map can perform many tasks :

Validation

Enrichment,

Content Based Routing

etc.

Type Tree

Type Tree

Adapter

Adapter

SOURCE FILE TRANSFORMED MESSAGE

ON OUTPUT

CROSS-REFERENCE

TABLE

DATABASE UPDATE

The Components of a WebSphere TX Solution


15

Developing a TX Solution

We usually talk about three simple steps:

Describe the data...

Data that is handled by IBM WebSphere TX, are described in "Type Trees‖

that define the data structure and semantics.

Transform the data...

Data is mapped between the source and destination using drag-and-drop.

Deploy the transformation

Multiple deployment models

Batch

Event Driven

Tightly-coupled (through APIs)

Loosely-coupled (RMI)


16

WebSphere TX Design Environment

• Flows

• Events

• Monitoring

Describe the Data

Interfaces Resources

Define Transformation Rules

Deploy and Manage

• Properties

• Structure

• Validation Rules

• Structure

• Formats

• Semantics

• Communication

• Adapters

Type Designer

Database Interface

Designer

Map Designer

Message

Broker

Launcher

Command

Server


17

DE

VE

LO

PM

EN

T

Data Definition

Transformation and Routing Rules

Reusable Object Library

Data Definition SupportsIndustry Standards

DESIGN

EXECUTION

DEVELOPMENT

Data Definition –

Built-in and

Automated

compliance toolsets–HL7

–ACORD

–SWIFT

–NCPDP

–RosettaNet, BizTalk

–X12

–Edifact

–HIPAA

Data Definition –

XML

COBOL

SQL (any ODBC

compatible)

SAP/IDOCS

Siebel

Oracle

PeopleSoft/PANELS


18

Describe the Data...

IBM WebSphere TX Type Trees The Type Trees defines the structure of the data to be handled in IBM WebSphere TX.

This XML structure is then represented as an IBM WebSphere TX Type Tree.


19

Integration: Metadata Automatic data definitions: Importers

XML DTDs, Schema

COBOL Copybooks

Database Structures

Application Definitions

SAP: BAPI, Idoc, DXOB, BDC

PeopleSoft 7: Business Document, Message Agent

PeopleSoft 8: Application Messaging, Component

Interface

Siebel: COM Business Object, EAI, EIM

WSDL (Web Services)

EJB API

Java Class

Java Messaging Service

Other: Tuxedo, text files

• Built-in data definitions

– EDI, HL7, SWIFT, ACORD

– All definitions are

modifiable through GUI


20

Built-in Validation (Restrictions)

Items have extensive validation options

• Enumerations• Exclusions (with substitution)

• Patterns• Ranges


21

Additional Built-in Validation

Component Rules allow cross-field validation

Incorporates semantic validation directly into the parsing process


22

Additional Built-in Validation

Component Rules support aggregate functions

Use component rules to specify conditions (business rules) that must

be met for a particular component to be valid


23

Transform the Data...

The Type Trees for input and output are added to IBM WebSphere TX

Map Designer.

For each input and output, or ―card‖, an adapter and a data source are defined.

IBM WebSphere TX can handle several inputs to one or more outputs, and vice-versa.

The mapping is a simple drag and drop from the input source to the output target.

For each field in the output card, rules can be added--for example, mathematical formulas, conversions, etc.

More than 150 different rules, similar to MS Excel formulas, are available through a self-describing wizard.


24

DE

VE

LO

PM

EN

T

Data Definition

Transformation and Routing Rules

Reusable Object Library

Object Design Delivers One Set of Logic

DESIGN

EXECUTION

DEVELOPMENT

Object Design –

One set of logic, regardless of

platform, protocol,

communication layer or

database

World-class transformation

with no 3GL coding

Recognized industry leader

with 140 partners imbedded

Object Design –

Highly reusable

Content-based and

dynamic routing

Many-to-many in a single

pass

Scaleable and Efficient


25

Map Designer Create maps to:

Validate data from data sources

Identify data targets

Specify data transformation logic (rules)


26

Identifying Data Sources and Targets

Identify type tree and type tree components that define

data target

Identify type of data and

location of data


27

WebSphere TX Adapters

WebSphere TX

Design Studio

Extended Adapters

CommunicationCICSCOMCORBAE-MAILFTP/FTPSHTTP/HTTPSJAVA ClassJCA GatewaySOAPSOCKETVAN

ApplicationsSiebel

- BIM, EIM, OI COMSAP R/3 & BW

- ALE, BDC, DMI, DXOB- BW, EDI, BAPI

PeopleSoft- Message Agent API, - Open Query, EDI

QualityStage

SecurityS/MIMEOpenPGPSSL Module

DatabaseDB2InformixODBCOLE-DBOracleMicrosoft SQL ServerSybaseHP NonStop SQL/MX & SQL/MP

MessagingBEA MessageQ & TuxedoCandle RomaIBM WebSphere MQ

(MQSeries)JMSMicrosoft MSMQOracle AQTIBCO Rendezvous

UtilityArchive (ZIP/TAR)Base 64BatchFileGZIP/ZLIBJNDILDAP

MIME

Quoted-Printable

Shell Script

Sink

SNMP

XML


28

• Drag and Drop

• No Procedural Coding!!

• Easily represent complex business rules

Creating transformation rules


29

Map Rules

Describe how to build an output data object

Color-coded for readability

Can combine hard-coded values (literals),

WebSphere TX functions, object names and

functional map references

Are required for every output object


30

Building a Map

Analyzes logical interfaces

within map rule

Build errors prevent map

from being compiled

Build warnings alert of

condition that might be

problematic

Creates compiled map file

For current platform

For selected platform, which

accounts for byte-order and

character set differences on

that platform


31

Multiple Deployment Options

Batch & Standalone

Event Driven

WebSphere Message Broker

WebSphere ESB

WebSphere Process Server

WebSphere Partner Gateway

WebSphere Application Server

WebSphere DataStage, etc.

Developing a TX Solution –An Example Putting It All Together

1. Describe the data

Design Studio

Z/OS

COBOL

Copybook

AIX

Map

Deploy

MQ

Importer

Importer

3. Deploy the transformation

2. Transform the data

Oracle

®

IBM Software Group

© IBM Corporation

WebSphere TX in Eclipse


33

Eclipse GUI

Uses the same Eclipse platform as other WebSphere Products

Ability to plug into other Eclipse WID

Take advantage of any Eclipse plug in

Application based on Project, making it easier to import and export all artifacts together

Shared Library can be used across projects

Invoking maps from WESB Mediation Module

Invoking maps from WPS Module


34

Eclipse GUI With Transformation Extender Perspective


35

Sharing WID with WESB or WPS

WTX Project

WTX Library

WPS Module


36

Wizard to Create Map Template based on Business Object


37

Executable Maps are Sharable in Library


38

Integration through Data Binding


39

Integration through Data Binding for JMS, MQ and HTTP exports and imports


40

Integration through Data Handler for FlatFile, FTP and Email imports and exports


41

Participating in WESB Mediation Module


42

Participating in WPS Module


43

Map Testing

1. Build the Map

2. Run the Map

3. View Results


44

Integrated Testing

1. Select Component to Test

2. Fill in the Test Data

3. Start the Test


45

Integrated Testing

4. Select Deployment Location

5. Wait for Operation to Complete

6. View Output Results from JMS Client


46

Second Implementation Option: Use Java API

// Load a mapMMap map = new MMap(mapName, workDir , mapBytes);// Get the adapter object handle for input card #1MCard card = map.getInputCardObject(1);

// Override the adapter in input card #1 to be a streamcard.overrideAdapter(null, MConstants.MPI_ADAPTYPE_STREAM);

// Get the handle to the stream objectMAdapter adapter = card.getAdapter();MStream stream = adapter.getOutputStream();// Send a single large pagestream.write(inBytes, 0, inBytes.length);

// Get the adapter object handle for output card #2card = map.getOutputCardObject(1);

// Override the adapter in output card #2 to be a streamcard.overrideAdapter(null, MConstants.MPI_ADAPTYPE_STREAM);// Run the mapmap.run();


48

WebSphere Universal Mapping Tool

WTX has become WebSphere’s Universal Mapping Tool

The Same Map can Plug into WebSphere Message Broker node

Plugs into WebSphere Partner Gateway

Runs in WebSphere Data Power

Data Transformation can now be done without any Coding


49

Chief Value Differentiators of WebSphere TX to alternative transformation methods

Powerful Transform capabilities with No CodingLowers cost to implement, maintain and re-use

High-Throughput, Complex Transformations

Cost per transaction very low

Natively Handles Any Data TypeNever resort to ―flattening‖ or ―re-structuring data‖

Data integrity and meaning are never lost

In Process Data ValidationGreatly reduces development efforts and implementation costs

Solve Really Hard problems, in less time, with one common design method


50

Summary—What WebSphere TX Provides

Simplified Complexity

• Graphical integration views

• Consistent approach to multiple types of

integration

• Reusable objects

Adaptability

• Leverage your proven IT infrastructure

• Fully support your choice of IT investments

(i.e. messaging middleware…)

• Non-invasive integration

• ―Snappable‖ modules

Speed

• Easy design and implementation

• No re-training

• Fast maintenance without coding

• Rapid adaptation to change

Interoperability

• Connect applications, databases, processes

within the enterprise

• Partners, processes across enterprises

• Powerful transformation and routing


51

Merci

Grazie

Gracias

Obrigado

Danke

Japanese

French

Russian

German

Italian

Spanish

Brazilian Portuguese

Arabic

Traditional Chinese

Simplified Chinese

Thai

ibm websphere transformation extender (tx) high level and...

Documents