Enterprise Application Integration

Paulo MarquesInformatics Engineering DepartmentUniversity of Coimbrapmarques@dei.uc.pt

2008

/200

9

7. Legacy Systems7.1. Integration Levels

2

What we have been discussing...

SystemA

ServiceInterface

SystemB

ServiceInterface

SystemC

ServiceInterface

SystemD

ServiceInterface

EnterpriseService Bus

ProcessOrchestrator

ServiceDirectory

Service OrientedArchitecture

3

Developing new systems

Systems are thought explicitly for enterprise integration It’s a fundamental concern to provide an external

integration interface That external interface should guarantee:

High granularity in terms of business functions Autonomy and independence from other services Clear and Consistent semantics for integration Security Ability to be evolved (e.g. versioning) (Meta-information and Self-documentation)

At the enterprise level, systems much be thought as a whole for wide integration

Standardization and common approach to service development at the entprise

Usage of a Service Bus Service registration on a common directory

4

According to this vision, the world looks like…

5

The “real world”...

6

BIG BALL OF MUD

«While much attention has been focused on high-level software architectural patterns, what is, in effect, the de-facto standard software architecture is seldom discussed. (...)

A BIG BALL OF MUD is a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design. Yet, its enduring popularity cannot merely be indicative of a general disregard for architecture. (...)»

B. Foote and J. Yoder, "Big Ball of Mud", in Proc. The 4th Pattern Languages ofProgramming Conference, Monticello, Illinois, USA, Sep. 1997

http://www.laputan.org/mud/mud.html

7

Ball-of-Mud regarding EAI

8

Application Types

“Simple Isolated Applications” (silo) “1-tier application” The interface may or may not be graphical; in many

enterprise applications it’s not

Business Logic

User Interface

Data Management

9

Application Types (2)

2-tier Applications Separation between data and business logic

Business Logic

User Interface

Database

10

Application Types (3)

3-tier Applications Separation between user interface, business logic and

data

Business Logic

User Interface

Database

11

Thick vs. Thin Clients

Business Logic

Thick Client

Database

BusinessAPI

Business Logic

Thin Client

Database

Graphical-orientedAPI

12

Packaged Applications

An application provider supplies customers with an application server capable of supporting a large number of business needs Most systems need to be heavily customized Plug-and-play of modules Integration with other systems is made through custom

APIs or adaptors (e.g. Java JCA)

Some examples: SAP, PeopleSoft, Oracle, Baan, Siebel, etc.

ERP

M1

M2

M3

13

Proxies and Wrappers

Fundamental Concept: Hide the legacy application behind a proxy or a wrapper Typically, a proxy consists in automatically generated code that

maps an existing interface into a more convenient one A bridge is a form of proxy that connects two different systems

A wrapper consists in a piece of core (or interface) that hides an existing application, allowing it to be seen as an autonomous system

Wrappers are normally “hand coded” When it’s not an application that is being hidden, but just an

interface, that it’s called an adapter

Business Logic

GUI

Database

WEB

SERVICE

(e.g. a simple wrapper thatmaps a whole applicationusing integration at the user interface level)

14

Integration Types (Traditional EAI)

Traditionally, EAI focused on Application-To-Application Integration Démodé: nowadays we use SOA and ESB! Even so, the learned-lessons are important!

15

Integration Types

User Interface Level(GUI or not)

Business Methods Level(Application-specific APIs; typically RPC-based)

External Application APIs Level(Integration-specific APIs; CORBA; MOM; “connectors”)

Data Level(Databases, Files)

16

Screen-Scraping

Integration systems using the user-interface

17

Screen-Scraping

It can be simple or complex!

Text-based Systems VT100, IBM 3270 (IBM mainframes), IBM 5250 (AS/400),

etc. In many cases, an operation depends on navigating

through several screens and menus. Problem: what to do if an operation fails?

Lack of documentation and consistency

Thick-client systems It’s easier: you can remove the user interface and interact

directly with the business logic

Thin-client systems (web) Relatively simple Fragile (web-based interfaces change quickly)

18

Screen-Scraping (2)

Some problems: Needs detailed knowledge of the application In many cases that are “calculated fields” which do not

map into the business logic or corresponding databases Difficult to evolve and maintain

Data extraction procedures: Static (line/column based) Dynamic (tag search based, regular expressions, etc.)

As a general rule, it shouldn’t be used while trying to integrate an application into a larger system. (It’s too fragile.)

19

Integration at the Business Methods Level

Some applications do provide external interfaces that can be used for EAI: Thick-client applications Applications thought for distributed systems Applications thought to be integrated

Thick-client / Distributed Applications Method granularity problem Interoperability problem Documentation problem / reverse engineering Data integrity problem Security problem

Application thought for being integrated Typically CORBA or DCE RPC interfaces Relatively easy to integrate

20

External Application APIs Level

“Packaged applications” are thought with integration with mind ... with modules of the SAME SUPPLIER In some cases, with modules or applications from other suppliers

Characteristics Adequate granularity Correct semantics for security and data consistency In many cases they use MOM / CORBA

(in others: COM/DCOM, Java RMI, XML-RPC, SOAP) In a lot of cases they use proprietary protocols In the Windows world... typically DCOM!

Different business areas try to standardize different procedures: SWIFT: Banking operations FIX: Financial operations HL7: Medical Systems ...

Try to keep your system standard-compliant!

21

Data-level Integration

Two common approaches Directly accessing the database tables! Export-Transfer-Import of data (file-based)

Direct access to Database Tables ODBC, JDBC, or native BD driver In some cases, there are still non-relational BDs being

used! Dangerous because of possible semantic violations and

data inconsistencies Reverse-engineering / documentation problem Fragile to changes Many times, it’s used for data replication among

databases Latency problem Consistency problem

22

Data-level Integration (2)

Export-Transfer-Import It’s quite common to dump a database to file, transform

the file in some way, and transfer the data into another system

“FTP-based integration” is quite common Extremely common in Data-Warehousing

Operational DB Data Warehouse Normally, theses systems have temporal deadlines that

must be met 24-h latencies are common Non-consistent data view throughout the whole system

Relatively secure and even resilient to some (limited) changes

23

EAI using files

24

Typical...

25

IMPORTANT CONCEPT: Evolving Systems!

26

Progressive Deployment

EnterpriseService Bus

Phase 1 Phase 2 Phase 3

27

Some Warnings...

Avoid introducing new technology when solutions can be created with what you have

Avoid introducing complex and hard-to-maintain processing logic

Avoid creating a fragmented environment where the business logic is distributed throughout the enterprise

Avoid creating tightly coupled channels which fail when the intervening systems change

Minimize the re-development of existing applications while performing enterprise application integration

Pay special care on how performance is affected while doing EAI

28

Bibliography

Enterprise Integration Patternsby Gregor Hohpe & Bobby Woolf Chapter 2

Enterprise Application Integrationby David LinthicumAddison-Wesley, ISBN 0201615835, Dez/1999 Chapters 1-5

Service-Oriented Architecture: A Field Guide to Integrating XML and Web Servicesby Thomas Erl Chapters 8 and 9

Enterprise Application Integration

Paulo MarquesInformatics Engineering DepartmentUniversity of Coimbrapmarques@dei.uc.pt

2008

/200

9

7. Legacy Systems 7.2. Brief introduction to JCA

30

J2EE

Web Server

DeployedJSP pages

EJB Server

EJB Container

Employee Service

ServiceAdmin

Employee Record

Payroll

PensionPlan

Database

EmployeeDatabase

ServiceDatabaseDeployed EJBs

Service Record

JDBC

31

J2EE on a larger world...

Web Server

DeployedJSP pages

EJB Server

EJB Container

Employee Service

ServiceAdmin

Employee Record

Payroll

PensionPlan

ERP

CRMDeployed EJBs

Service Record

?

32

Problemas

Sistemas CRM, ERP e legados não são... Bases-de-dados acessíveis via JODBC Código Java que suporte APIs Java

Normalmente os fabricantes oferecem adaptadores... Específicos!

Como é que se pode gerir, de forma uniforme: Obtenção e gestão de ligações?

(E.g. criação de ligações e connection pooling) Propriedades transaccionais?

(E.g. propagação transaccional) Segurança?

(E.g. autenticação e autorização...)

33

Java Connector Architecture (JCA)

Web Server

DeployedJSP pages

EJB Server

EJB Container

Employee Service

ServiceAdmin

Employee Record

Payroll

PensionPlan

MyERP

MyCRMDeployed EJBs

Service Record

MyERPJCA Adap

MyCRMJCA Adap

Registados no servidorJNDI do servidor J2EE

34

Java Connector Architecture (JCA)

Especifica um conjunto de contratos e interfaces entre: O servidor aplicacional e o adaptador Entre as aplicações cliente (que correm no servidor

aplicacional) e o adaptador ... é semelhante a um “device driver”

Esses contratos cobrem: A obtenção e gestão de ligações A gestão de segurança (autenticação) Propagação transaccional

Suporte alargado por parte da indústria

Actualmente na versão 1.5 (JCA 1.5) http://java.sun.com/j2ee/connector/index.jsp

35

Visão alargada dos contractos

36

Quem escreve os adaptadores?

Os fabricantes de sistemas alargados que desejem integração transparente dos seus sistemas com J2EE

Programadores que desejem “encapsular” sistemas legados presentes na sua organização num sistema J2EE No entanto, tal só é tipicamente feito quando o software é

reutilizado em diversos componentes (questão de custo vs. benefício)...

Os programadores tipicamente utilizamos adaptadores, não os escrevem...

37

Contratos – Gestão de Ligações

Oferece um API uniforme para obter uma ligação a um EIS (Enterprise Information System) assim como para realizar a sua gestão. Normalmente, o adapter é responsável por implementar

pooling de ligações de forma eficiente.

ConnectionFactory permite criar ligações, Connection representa uma ligação.

38

Contratos – Gestão de Transacções

Garante a utilização e propagação de contextos transaccionais para o EIS. Novamente, é responsabilidade do adaptador fazê-lo

correctamente

Três níveis transaccionais: Nulo ou nenhum Transacções locais (i.e. apenas envolvendo um EIS) Transacções distribuídas

39

Contratos – Gestão de Segurança

Permite a um servidor aplicacional ligar-se a um EIS usando um conjunto de credenciais correctas para um determinado “principal” Username/Password, Certificado X.509, etc.

São suportados dois modos de autenticação: “Container-managed Sign-on”: as credenciais são pré-

configuradas quando é feita a instalação da aplicação. Configured Identity: todos os recursos são acedidos com a

identidade configurada Principal Mapping: o “principal” que está a utilizar a

aplicação é mapeado num “principal” do EIS Caller Impersonation: o “principal” do servidor aplicacional é

o que é utilizado na chamada ao EIS Credentials Mapping: semelhante ao anterior mas é feito um

mapeamento de credenciais “Component-managed Sign-on”: o programador trata

disso no código

40

Contratos – Client Connection Interface (CCI)

Permite ao programador transferir e obter informação de um EIS de forma uniforme Algo semelhante ao interface JDBC – orientado aos dados O programador não necessita de o utilizar e o adaptador

não necessita de o providenciar: é possível utilizar um API definido à medida

O CCI divide-se em quatro partes: Connection Interfaces: o API para o estabelecimento das

ligações Interaction Interfaces: Controla a execução do EIS Record/ResultSet Interfaces: Resultados obtidos do EIS Metadata Interfaces: Permite reflexão sobre metadata do

EIS

41

Exemplo...

Ninguém disse que era “fácil” ou“bonito”...

42

JCA 1.5

O JCA 1.5 resolve algumas limitações do JCA 1.0. Em particular...

Em JCA 1.0 apenas é possível ter invocações J2EEEIS Em 1.5 é possível ter um “Message Inflow” em que um EIS envia

mensagens assincronamente para uma aplicação J2EE. Essas mensagens são entregues a “messaging beans” correctamente configurados.

Em 1.5 é possível propagar um contexto transaccional no sentido EISJ2EE (Transaction Inflow)

Em JCA 1.0 a gestão do adaptador era pobre... Em 1.5 é possível gerir o ciclo de vida de um resource adapter:

“Lifecycle Management Contract” (iniciar, parar, etc.) Em 1.5 um resource adapter por pedir ao servidor J2EE para

realizar trabalho (e.g. monitorizar a disponibilidade de um certo servidor, de um componente, etc.): “Management Contract”

Toda a gestão de threads associada corresponde ao servidor J2EE, não ao adaptador.

43

Questão...

Compare a utilização de SOA baseada em Web-Services e as especificações WS-* com a abordagem JCA para integração de aplicações num application server.

Em particular... há alguma vantagem nesta abordagem versus SOA/WS? ... se SIM, qual ou quais? ... se NÃO, então porque é que se utiliza JCA?

Compare a utilização de SOA baseada em Web-Services e as especificações WS-* com a abordagem JCA para integração de aplicações num application server.

Em particular... há alguma vantagem nesta abordagem versus SOA/WS? ... se SIM, qual ou quais? ... se NÃO, então porque é que se utiliza JCA?

44

Bibliografia

“J2EE Connector Architecture: Integrating Java Applications with Existing Enterprise Applications” whitepaper at the Sun’s Developers Network,

http://java.sun.com/j2ee/white/connector.html

Dirk Reinshagen, “Connect the enterprise with the JCA”, Parts 1 & 2 in the JavaWorld Jornal, Nov/2001 and Fev/2002

http://www.javaworld.com/javaworld/jw-11-2001/jw-1121-jca_p.html

http://www.javaworld.com/javaworld/jw-02-2002/jw-0201-jca2_p.html

45

IMPORTANT NOTICE

YOU ARE FREE TO USE THIS MATERIAL FOR YOUR PERSONAL LERNING OR REFERENCE, DISTRIBUTE IT AMONG COLLEGUES OR EVEN USE IT FOR TEACHING CLASSES. YOU MAY EVEN MODIFY IT, INCLUDING MORE INFORMATION OR CORRECTING STANDING ERRORS.

THIS RIGHT IS GIVEN TO YOU AS LONG AS YOU KEEP THIS NOTICE AND GIVE PROPER CREDIT TO THE AUTHOR. YOU CANNOT REMOVE THE REFERENCES TO THE AUTHOR OR TO THE INFORMATICS ENGINEERING DEPARTMENT OF THE UNIVERSITY OF COIMBRA.

(c) 2007 – Paulo Marques, pmarques@dei.uc.pt