Chapter 6 – The EAI Process

The following activities should be considered when approaching EAI:1. Understanding the enterprise and problem domain2. Making sense of the data3. Making sense of the processes4. Identifying any application interfaces5. Identifying the business events6. Identifying the data transformation scenarios7. Mapping information movement8. Applying technology9. Testing10. Considering performance11. Defining the value12. Creating maintenance procedures

Step 1: Understanding the Enterprise and Problem Domain

This is the most complex and time consuming part of the entire process, but we have to do it no matter what. Understanding the problem domain requires working with many organization managers in order to get a handle on the structure and content of various information systems, as well as business requirements of each organization-how they do business, what is important, etc.

This is actually a basic requirements-gathering problem. It requires interfacing with almost all the levels and systems of an organization to determine the information that will allow the EAI problem to be defined correctly so that it can be analyzed, modeled and refined.

Step 2: Making Sense of the DataMost of the EAI projects exist only at the data level. The

implementation of data-level EAI comes down to understanding where the data exists, gathering info about the data (for example understanding the schema information) and applying business principles to determine which data flows where and why.

There are 3 basic steps that should be followed to prepare for implementing data-level EAI:

a) Identifying the datab) Cataloging the datac) Building the enterprise metadata model

Fig. 6.1 Implementing data-level EAI

Lets look now at each of the 3 steps showed before:Identifying the data: First step is to create a list of

candidate systems. This list will make it possible to determine which databases exist in support of those candidate systems. The next step will require determining who owns the databases and where they are physically located.

The Data Dictionary: We can get detailed info by examining the data dictionaries (if they exist) linked to the data stores being analyzed. We aim to get info such as:

-The reason for the existence of particular data systems-Ownership-Format-Security parameters-The role within both the logical and physical data structure

Integrity Issues: It is very important to understand the rules and regulations that were applied to the construction of the database. Most middleware fails to take into account the structure or rules built into the databases being connected. As a result there exists the very real threat of damage to the integrity of target databases. The lack of integrity controls at the data level could result in serious problems.

Data Latency: It’s the characteristic of data that defines how current the information needs to be. Such information will allow EAI architects to determine when the information should be copied, or moved, to another enterprise system and how fast. There are 3 different categories of data latency when it comes to EAI:

- Real time (updating data with a frequency approaching 0 latency)

- Near time (updating data at set intervals)- One time (updating data once)

Data Format: another component of data. How information is structured, including the properties of the data elements existing within that structure can be gleaned from knowledge of the data format. Resolution of data format conflicts must be accomplished within such EAI technologies as message brokers and/or application servers. Different structures and schemas existing within the enterprise must be transformed as information is moved from one system to another.

Data Cataloging: the process of gathering metadata and other data throughout the problem domain. Once accomplished, it is possible to create an enterprise-wide catalog of all data elements that may exist within the enterprise. The resulting catalog then becomes the basis of the understanding needed to create the enterprise metadata model-the foundation of data level EAI

Fig. 6.2 Data/Message Transformation via Message Brokers

Building the Enterprise Metadata ModelThis model defines not only all the data structures existing in

the enterprise but also how those data structures will interact within the EAI solution domain.

Logical Model: Creating the logical model is the process of creating an architecture for all data stores that are independent of a physical database model, development tool, or particular DBMS (for example Oracle, Sybase, etc). Logical model is a sound approach to an EAI project in that it will allow developers the opportunity to make objective data-level EAI decisions, moving from high level requirements to implementation details.

At the heart of the Logical model is the Entity Relationship Diagram (ERD) which is a graphical representation of data entities, attributes and relationships between entities for all databases existing in the enterprise. (see Fig. 6.3)

Fig. 6.3 Entity Relationship Diagram (ERD)

Physical Model: There is no clear way to create a physical model that maps down to object-oriented, multidimensional, hierarchical, flat files, and relational databases at the same time. But if those databases are to be integrated, some common physical representation must be selected. Only then can the model be transformed as required.

The input for the physical model is both the logical model and the data catalog.

But before the logical and physical database is completed it is desirable to normalize the model. This is a process of decomposing complex data structures in simple relations using a series of dependency rules.

Normalization means reducing the amount of redundant data that will exist in the database or, in the case of EAI, in the enterprise.

Step 3: Making Sense of the ProcessesOnce the enterprise data is understood, and baseline

information such as the enterprise metadata model has been created, the decision must be made as to how to approach the enterprise business model.

Thus, once the applications in the problem domain have been analyzed, the enabling technology employed by each application within the problem must be identified.

Once this identification has been made, the next step is to determine ownership of the processes and develop insight into those processes-for example, why information is being processed in one manner versus another, etc.

Process identification is important to the method-level EAI project lifecycle.

Process Cataloging: Method-level EAI requires a “process catalog” which is actually a list of all business processes that exist within an enterprise or, at least, within the problem domain.

It is very important not only to identify each process but be able to understand how the logic flows within that process.

Each process catalog must maintain its own set of properties, custom built for each specific EAI problem domain:

Fig. 6.4 Building a Process Catalog

Leveraging Patterns for Method-Level EAI: Patterns are useful in the context of method-level EAI because they enable the developer to identify common business processes among the many business processes that already exist within the enterprise.

Using patters in this way is simply borrowing them from the world of application architecture, and applying them to the world of enterprise architecture-a perfect fit.

Patters describe generic schemes for the solution of the problems, solution schemes created by defining their components, responsibilities and relationships.

A 3-part schema is part of every pattern:a) Context describes the circumstances of the situation

where the problem arises or which have to be true for the pattern to be valid.

b) Problem is what is addressed by the patternc) Solution is the well-proven and consistent resolution of

the problem.

Types of Patterns:a) Architectural Patterns complex patterns that tend to link

to particular domains. They provide developers with the basic system structure-subsystems, behavior and relationships.

b) Design Patterns are mostly problem domain independent and they provide developers with a means of refining the subsystems and components and the relationship between them.

The goal in using these patterns is to find a common structure that solves a particular problem.

We define an idiom as a low-level pattern coupled with a specific programming language. It describes how to code the object at a low level.

Patterns add value to EAI architecture and the software development life cycle. They have the potential to increase the quality of most system development and to provide effective integration without increasing the cost.

Step 4: Identifying Application InterfacesThe best place to begin with interfaces is with the creation of

an application interface directory. This directory is used, along with the common business model and the enterprise metadata model, to understand the points of integration within all systems of the EAI problem domain.

The Application Interface Directory expands on the enterprise metadata model, tracking both data and methods that act on the data.

Business Processes are listings of functions or methods provided by the application

Step 5: Identifying the Business EventsIt is important to understand what invoked a business event,

what takes place during an event and any other events that may be invoked as a consequence of the initial event.

Step 6: Identifying the Schema and Content Transformation Scenarios

This is necessary because data existing in one system won’t make sense to another until data schema and content is reformatted to make sense to the target system.

Also it’s necessary because it will assure the maintenance of consistent application semantics from system to system.

Step 7: Mapping Information MovementIt should be noted where the information is physically

located, what security may be present, what enabling technology exists, and how information is extracted on one side to be placed on the other. Or if no event is required, what other condition causes the movement of information from the source to the target.

Step 8: Applying TechnologyThis is one of the hardest part (to select the proper enabling

technology to solve the EAI problem).Many technologies are available, including application

servers, distributed objects and message brokersThis selection is very important as a bad choice can lead to

the failure of the EAI project.

Step 9: TestingTesting is time consuming and expensive in terms of men

hours and machine CPU time. But is of an utmost importance for the success of the EAI project.

To ensure proper testing, a test plan has to be put in place making sure the source and target databases are off line during the initial EAI tests. This is hard to do as most of the databases are business critical and need to be operational all the time.

Step 10: Considering PerformanceIn many cases performance is ignored until everything is done

already and it’s too late to change anything. If EAI systems are not performing well they will fail for sure (for example if extracting a record from a database takes 20 min, then it’s of no value to that organization).

Although zero latency for EAI systems is only in theory, most of EAI implementations have a very low latency and in most cases it’s not interfering with the normal business process.

But although this might be true for a single user, EAI performance has to be checked under heavy use also when multiple users query the systems at the same time.

Step 11: Defining the ValueUsually the value of an EAI system is in the money that’s saved

by a good EAI implementation (after the cost of the EAI is covered)

Step 12: Creating Maintenance ProceduresFuture maintenance of the EAI system has to be addressed

also. Full maintenance logging is critical for debugging future EAI problems by other project teams.

Also disaster recovery plans have to be studied and put in place, many times solutions like relocating the EAI in case of a disaster is something considered a normal contingency planning

But no matter how much care and planning is put into an EAI, the final implementation is so unique and customized to an organization that there are no two EAI implementations that are alike.

Much care has to be put in documenting every step of the design, implementation and maintenance so future IT people would be able to learn it w/out too many headaches.