adapters and eai

Technology, Tools and Trends

November 2015

Adapters & EAI

AgendaSession 1:

Background on software integrationBackground on adaptersOverview of adapter technology

Session 2:Overview of trends in adapter technologyJ2CA, Web Services, Semantic AdaptersOverview of adapter development tools

Background On Software Integration

Software integration is the #1 IT priority todayIt is also the most complex and expensive

problem to solveAbout 30% - 40% of all IT costs are integration-

relatedApproximately 1/2 of all integration costs are

adapter-relatedAdapters represent the highest incremental cost

of software integrationEach software system that needs to be

integrated requires an adapter

Background On Software IntegrationThe primary focus of software integration is

information systems (or business applications)The two main areas of software integration are:

DataFunction (Web Services)

The two main types of software integration environments are:Point to Point (Application Server based)Brokered (Integration Platform based)

The extended focus of software integration is business processesIntegrating information systems with business processesAutomating the integrated business processes

Background On Software IntegrationComponents of Software Integration Platform

Foundation componentsHost (Application Server, Integration Server, others)Messaging bus (JMS, others)Object brokers (OMG)Database engines (RDBMS, ODBMS, others)Transaction engines (XA compliant, others)

Value add integration componentsTransformer

Mapping, Object aggregation, Object decompositionBroker

Routing, FiltersAdapters

Current Perspective on EAIEAI is still a big, complex, expensive,

tedious and repeating problem faced universally by companies of all sizesApproximately 1/3 of all IT costs are EAI

relatedApproximately 1/2 of all EAI costs are

adapter-relatedThe definition of EAI is much broader today and

includes software systems within and outside the companies firewalls The term “EAI” has been changed and misunderstood with

the continuing evolution of EAI technologies

Current Perspective on EAIEAI is an ongoing initiative and requires

strategic thinking, strong risk management, and realistic goal oriented implementation plans

Industry studies have reported that the success rates of EAI project is very low < 5%

Existing methodologies, project management practices & processes are not enough to deal with the unique and specific risk management requirements of EAI

Current Perspective on EAINew tools have a strong commitment to

solve one of the toughest problems of EAI namely development of custom/customized adapters and lowering the total cost of ownership

The first step is to define EAI in a clear and simple form easy to understand, comprehend, and implement

Definition of EAIEAI is the process of integrating applications and database systems into a unified software system

capable of supporting:On-demand data exchange Collaboration of shared functionsConsolidation & Transformation of data into

informationAutomating data synchronizationAutomating business process

EAI DomainsThe EAI processes are grouped into a set of

logical processes (“Domains”) which simplifies the definition of specific EAI processesApplication To Application Domain (A2A - Domain)

The process of integrating two or more business applications

Application To Information Domain (A2I - Domain)The process of integrating a business application with

information producers (data warehouses), information access portals, etc.

Application To Process Domain (A2P - Domain)The process of integrating a business application with one

or more work flows, EDI gateways, supply chain automation networks, mobile workforce networks

EAI Domain Reference ModelsEvery EAI Domain is supported by a context

sensitive reference model that can be used to define, design, deploy a customized EAI solution

Each EAI domain reference model contains:A logical architecture that defines the components

and relationships of the EAI processA set of integration patterns that define the

different behavior and interactions of the EAI process

A technology stack containing layers of technologies and standards that enable the EAI process

Business ApplicationBusiness Application

Application to Application (A2A) EAI

Database DatabaseFunction Function

DataIntegration

BrokerAdapter Adapter

WebServicesBroker

Adapter Adapter

SOAP

XML

Transformer

UDDI

Information PortalBusiness Application

Application to Information (A2I) EAI


Adapter

EIIServer

Adapter

ETL

DataWarehouse

PortalServer

Business Application Partner / SupplierSystems

Business Application

Application to Process (A2P) EAI


Adapter

WorkflowEngine

Adapter

EDIGateway

XML

DatabaseFunction

Adapter

XML / EDIFACT

Firewall

WFMC

Data Replication Pattern

A2A PatternsData

ChangeDetector

DataExtractor

DataFilter

Data Source Data TargetDataLoader

Other PatternsData Synchronization (Bi-directional Data Replication)

Composite Data Extraction

Collaboration of Shared Services

Information Creation Pattern

A2I Patterns

DataRetriever

Meta-DataCreator

InfoAggregator

Data Source Information SinkInfoPresenter

Other PatternsExtract Transform Load (ETL)

Data Staging

Task Delegation Pattern

A2P Patterns

TaskAssignment

Agent

Task toFunction

MapWorkflow

Manager

Business Application

FunctionManager

Other PatternsBusiness Process Management

Generic EAI Solution Architecture

Foundation (Host, JMS, CORBA, Transaction and Database Engines)

Broker Transformer

Adapter

InformationSystem

Adapter

InformationSystem

Adapter

InformationSystem

Adapter

InformationSystem

Adapter

InformationSystem

Adapter Technology Status Quo

Adapters represent the highest incremental cost of integrationEach applications that needs to be integrated

requires a minimum of one to three adapters (A2A, A2I, A2P)

The average Fortune 1000 company1 has 48 applications and 14 databases for a total of 62 potential adapters

There are few specific tools and platforms available for development and maintenance of adapters

1IDC

Definition of an AdapterThere is no Single Definition of an AdapterBut there is a common understanding of

the adapters behavior and roleAdapters, Connectors, Plug-ins, Web

Services, Integration Components and other proprietary terms are used in different context to mean “Adapter Type Of Functionality”

Adapter definitions tend to be context sensitive

Generic Definition Of AdapterAn Adapter is a software component which

integrates two different instances of software systemsTypically the software systems integrated are

business applications (information systems)In a brokered integration environment a business

application is integrated with the broker by using an adapter

Adapter facilitates integration between the integrated software systemsProvide connectivity between the systemsManage the interactions between the systemsEnforce transformation and other business rules

Background On AdaptersThe concept of adapters was first introduced

by the Enterprise Application Integration (EAI) vendors in the later half of 1990’s

Adapters have become the most important aspect of software integrationEvery software that needs to be integrated

requires an adapterThe average Fortune 1000 company has 48

different business applications and 14 different databases¹

Adapters represent the highest incremental cost of software integration

¹ IDC

Background On AdaptersUntil recently there were no standards for

developing and supporting adaptersJ2EE:JCA is the first system level standard

designed to support adapters on the Java Application Server Major vendors are now supporting J2EE:JCA in their Java

application serversAdapters are becoming a commodity

System standards for adapters will result in more focus on integration functionality

Multiple providers of the same adapter will give users opportunity to compare adapter features

Background On AdaptersGenerally adapters are custom developed from

scratchMost information systems are proprietary or heavily

customizedThe average cost of developing and maintaining an

adapter for packaged applications like SAP, PeopleSoft, etc. over a five year period is over U$ 1 Million

The adapter market has few tools available for developersThe biggest challenge in creating adapter tools is the

expertise and experience required to design, develop, and maintain complex adapters

Adapter InteractionsAn adapter use case defines a specific interaction

between the two software systems integrated by the adapterAn adapter use case is a sequence of one or more

functional stepsThere are two types of steps in an adapter use case

Integration function (Connection, Parsing, Transformation, etc.)Support function (System Logging, Error handling, Audit Trail,

etc.)Two types of use cases

Simple Use Case Involves only one adapter (two software systems)

Complex Use Case Involves two or more adapters (four or more software systems)

Adapter ScenariosAn adapter scenario is a specific instance of

an adapter use case and is bound to a specific runtime environmentAtomic Scenario (Derived from simple use

case)Collaborative Scenario (Derived from

complex use case)It is possible for adapter scenarios to

change between atomic and collaborative scenarios depending on the runtime environment

Simple Use Case (Atomic Scenario)

Simple Adapter Use Case

SAPAdapter

SAP

APIClient

•Check customer•Add new order

Complex Use Case (Collaborative Scenario)

Simple Use Case

Complex Adapter Use Case

SAPAdapter

SAP

SiebelAdapter

Siebel

API

APIClient

Add neworder

Checkcustomer

Adapter Technical Architecture

Technology Stack

Client Interface

Service Gateway

Document Manager

Transformation Manager

Scenario Manager

Information System Manager

Host Environment Manager

Responsible for providing unidirectional or bidirectional access to adapter services

Responsible for parsing, filtering, and formatting data and documents

Responsible for managing single/multiple adapter scenarios involving one or more information systems

Responsible for defining and implementing service contracts and access control

Responsible for data type conversion, data mapping, object aggregation and object decomposition

Responsible for managing connectivity to the information systems

Responsible for managing the lifecycle and collaborating with the operating platform


Technology Stack

Client Interface

Service Gateway

Document Manager


Scenario Manager



Optional Layer

Optional Layer


Technology Stack

Client Interface

Service Gateway

Document Manager


Scenario Manager



SOAP/UDDI

J2EE:JCA

XML / XSLT

Simplifying Adapter ComplexityHow to manage the technical complexity of

adapters?Use model based software development

techniquesPre-defined software models based on integration and

design patterns will provide a robust framework for developing adapters

Developing multiple adapters for one applicationEncapsulate fewer interfaces into one adapter instance

Collaborate with external transformers instead of developing transformer functionality in each adapter

ImplementationAnalysis, Architecture and Design

Adapter Models

Abstract Adapter Model

Structural

Patterns

FunctionalPatterns

ConceptualPatterns

IntegrationPatterns

Concrete Adapter Model

Trends In Adapter TechnologiesAdapters are more sophisticated in functionality

Web services enable application function level collaboration

J2EE Connector Architecture enables plug-and-play support for adapters inside application servers

Adapters can function in a point-to-point or broker based integration environment

Standard adapter models will provide a consistent architecture and design for adapters

Adapter tools will enable faster, cheaper, and easier development and maintenance

Impact of Web ServicesThe most significant impact of web services is

the shift in software application architectureService Oriented Architecture “SOA” is

becoming the foundation for new internet based business applicationsThe concept of “Composing” business

applications from existing software servicesWeb services enables development of “SOA”

compliant applicationsAdapter have an even more critical role in

fulfilling “Services”

Impact of Web ServicesWhere is the Synergy between adapters and

web services?Web services provide a standardized platform for

defining, locating, and invoking business servicesWSDL, UDDI, SOAP

Adapter provide a standardized component mechanism for fulfilling the web services by accessing external software systemsJ2CA

Both web services and adapters are moving towards a common semantic environment enabling rapid “composition”UDEF, RDF

What is J2EE Connector Architecture?J2EE Connector Architecture (J2CA) is a sub-

specification of J2EEJ2CA is a system level interface for hosting

and accessing connectors to non-J2EE applicationsJ2CA connectors extend the J2EE servers ability

to interface with non-J2EE applicationsMost J2EE servers support J2CA specification

IBM WebSphere, BEA WebLogic, Jboss, Oracle ASJ2CA compliant adapters for popular applications

like SAP, PeopleSoft, are now available in the market place

What is J2EE Connector Architecture?J2CA Specification (Ver 1.5) defines system

contracts for managing security, transactions, and connections between J2EE application components and legacy systems

Support meta-data interfaces and bi-directional access between adapters and legacy systems

Message driven access to J2CA connectors is now possible

Moving Towards Semantic AdaptersWhat is a Semantic Adapter?

A Semantic Adapter provides an interface to the semantics of the adapterAdapter Behavior (Services provided by the adapter)Meta-data of the adapted systems data structuresMeta-data of the adapted systems service interfacesConfiguration properties of the adapters runtime

environmentA Semantic Adapter uses an Ontology based

approach to define the semanticsThis involves associating commonly understood

meaning to the definition of adapter properties, behavior, configuration, and associated meta-data

Adapter Development ToolsFundamental Requirements

Enable just in time adapter development and maintenanceCustomers do not purchase adapters ahead of requirementEnd points (information systems) change and therefore pre-

built adapters need customization tooExtend the model based design (MBD) paradigm to

adaptersModel based development significantly reduces

development time and cost, and increases adapter qualityDifferentiate and support abstract adapter models

and concrete adapter modelsEnables multiple implementations of a common abstract

adapter model

Adapter Development ToolsFundamental Requirements

Enable adapter developers to focus on the business requirements of integrating two software systemsDevelop and maintain platform controllers which

encapsulates the operating platform system interfaces from adapter developersEliminates the burden of learning and programming

system level interfaces and standardsE.g: J2EE:JCA controller which encapsulates J2EE:JCA

system contracts from the adapter developerGenerate adapter framework and component shell from pre-

defined adapter modelsEnsures design consistency, reduces bugs and enhances

the quality of adapters

Adapter TechnologyArchitecture:

Adapter architecture based on the concept of functional layersEach layer encapsulates specific adapter functionality

accessible as well defined servicesEach layer can be implemented as an isolated/distributed

component of the adapterArchitecture supports three types of interactions for

an adapterSynchronization

Keeping distributed data models in sync with each otherCollaboration

Enabling software systems to collaborate functionsDelegation

Enabling software systems to execute tasks on behalf of other systems

Adapter TechnologyAdapter Model

Is a collection of one or more adapter patterns managed by the E-Fusion framework Integrated Collaboration Environment (ICE)

E-Fusion supports two types of Adapter ModelsAbstract adapter model

Represents the architecture and design of an adapterAdapters cannot be created directly from abstract

adapter models as there are no implementation details defined

Concrete adapter modelRepresents the implementation reference model of an

adapter Includes one or more run-time scenariosAdapters can be created from concrete adapter models

Adapter Development EnvironmentAdapter Development Environment (ADE)

supports developers throughout the full lifecycle of adapter development and maintenanceFrom design to maintenance

Design, Develop, Assemble, Configure, Deploy, MaintainDevelop pre-built adapter models and support custom

adapter modelsEnable higher efficiencies in adapter development

without changing existing development workflow and processesDefine new adapter engineering and development

methodologies which can be applied with generic software development tools

Adapter Runtime EnvironmentAdapter Runtime Environment (ARE) supports

distributed and centralized adapter deployments across multiple platformsSupport dynamic distribution of adapter

components across the platforms in the adapter runtime environment

Support centralized administration, monitoring, and debugging of adapters at runtime

Support dynamic language (locale) translation for multi-lingual runtime environments

Support dynamic component code download and side-by-side runtime environments (production and QA)

SummaryAdapters are complex pieces of software

performing critical integration tasksThe definition of adapters continues to expand

and support new application architectures (“SOA”)

Adapter tools are now available for enabling rapid adapter development and deployment

The trend is towards developing semantically and functionally rich adapters will continue to strengthen the role of adapters