impact analysis for cross domain enterprise applications · enterprise applications iftach ragoler,...

IBM Labs in HaifaWebSphere Studio Asset Analyzer (WSAA)�

Manage the application chaos: Impact Analysis for Cross Domain Enterprise Applications

Iftach Ragoler, Sara Porat, Avi Yaeli, Sharif Mograbi, Shlomit Shachor-Ifergan, Alex Akilov, Guy Rosman, Shiri Dori

IBM Labs in Haifa

2 WebSphere Studio Asset Analyzer (WSAA)�

A Toy Application

��

Enterprise Archive (Ear)

Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��

IBM Labs in Haifa


A Real Application

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class Utility Jar

Package

Class Utility Jar

Package

Class


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS RegionEntry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

EjbEjb

Ejb

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gifHTML

HTMLHTML

HTMLHTML

HTML

JSPJSP

JSPJSP

JSPgifgif

gifgif

CICS Transaction

Concatenation setConcatenation set


Entry PointEntry Point


IMS TransactionCICS Transaction

CICS Transaction

��

��

��

��

��

��

��

��

��

��

��

��

EjbEjb

Package

Class

Package

ClassClassClass

Class

Class

Class

IBM Labs in Haifa


Application Mining

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class Utility Jar

Package

Class Utility Jar

Package

Class


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction


Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

EjbEjb

Ejb

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gifHTML

HTMLHTML

HTMLHTML

HTML

JSPJSP

JSPJSP

JSPgifgif

gifgif

CICS Transaction






CICS Transaction

��

��

��

��

��

��

��

��

��

��

��

��

EjbEjb

Package

Class

Package

ClassClassClass

Class

Class

Class

What is the impact of changing this CICS Transaction?

Which other assets are impacted because of a change to this Ejb?

What happens when changing a column in this DB?

IBM Labs in Haifa


Application Mining

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class Utility Jar

Package

Class Utility Jar

Package

Class


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction


Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

EjbEjb

Ejb

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gifHTML

HTMLHTML

HTMLHTML

HTML

JSPJSP

JSPJSP

JSPgifgif

gifgif

CICS Transaction






CICS Transaction

��

��

��

��

��

��

��

��

��

��

��

��

EjbEjb

Package

Class

Package

ClassClassClass

Class

Class

Class

Is there a component that parses XML files? Where is it?

Which tests should I run because of a change in this Ejb Jar?

How many applications access this DB table?

IBM Labs in Haifa


Application Mining

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class Utility Jar

Package

Class Utility Jar

Package

Class


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction


Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

EjbEjb

Ejb

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gifHTML

HTMLHTML

HTMLHTML

HTML

JSPJSP

JSPJSP

JSPgifgif

gifgif

CICS Transaction






CICS Transaction

��

��

��

��

��

��

��

��

��

��

��

��

EjbEjb

Package

Class

Package

ClassClassClass

Class

Class

Class

What is the estimated cost of changing this IMS Transaction?

What is the Business Rule of this Application?

IBM Labs in Haifa


Application Mining

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class

��

��

��

��

��

��

��

��

��

��

��

!� !��

CICS Transaction

CICS Region

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

��

��


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

Utility Jar

Package

Class Utility Jar

Package

Class Utility Jar

Package

Class


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb


Web Archive (War)

HTML

JSP

gif

Ejb Jar

Ejb

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction

CICS Region

CICS Transaction


Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

Entry Point

Program

Concatenation set

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

IMS Transaction

IMS subsystem

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

Ejb

Ejb Jar

EjbEjb

Ejb

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gif

Web Archive (War)

HTML

JSP

gifHTML

HTMLHTML

HTMLHTML

HTML

JSPJSP

JSPJSP

JSPgifgif

gifgif

CICS Transaction






CICS Transaction

��

��

��

��

��

��

��

��

��

��

��

��

EjbEjb

Package

Class

Package

ClassClassClass

Class

Class

Class

How many assets do I have in my application? What is their distribution?

How do I migrate this War to be JSF based?

Does this Ejb jar have use any bad practices? Does it follow a J2EE pattern?

IBM Labs in Haifa


Anatomy of a Real Enterprise Application (WSAA customer)

� 404,237 source files� 185,884 Cobol files� 203,557 JCL files

� 124,430,225 lines of code� 24,657,412 comments lines� 99,755,617 non comment lines� 17,196 blank lines

� A ‘small’ CICS region (out of a dozen) has� 219 groups� 1,100 transactions� 105 data stores

IBM Labs in Haifa


Motivation

� Many development teams� Distributed across the organization � possibly multisite� Not aware of other teams responsibilities� Not managed on the same Source Configuration Management (SCM)� Possessing different skills

� Many programming languages� Integration problems

� Old applications� Their developers might have left/retired

� A change request may raise the questions:� What should be changed?� What is impacted?� Who owns the impacted components?

Most of the IT dollars goes for maintaining legacy code

Applications need to evolve

Application Mining Tools

IBM Labs in Haifa


WebSphere Studio Asset Analyzer (WSAA)

� Solution for the management of application software assets � Inventory

�Capture applications inventory�Display application assets and their relationships in a graphical manner

�Application mining and impact analysis�Visually mine the structure and dependencies of application artifacts to

assess the impact of changes�Enables to make changes to code with more confidence, greater

speed, and less risk

�Search and Reuse�Query the repository to locate (reusable) assets

IBM Labs in Haifa


WebSphere Studio Asset Analyzer (WSAA) ��

FileSystems

off-line

BEA WebLogic

WAS 5 z/OS

WAS 5 ND

Central Repository

��

�� !"�� #�� $"�� %��&�'�(

Analyzers

Analysis

)��%��*��*"

Repository Analysis

EnterpriseRepositories

Discovery

Crawlers"

"�

�

#

$

%&

'

(

)

"*

""

�#��

��

Services

��#��

XML

Client Formatting Services

HTML

SVG

Portlet

Client Integration

WSED IDE

WebSpherePortal Server

WEBBrowser

EclipseIDE

run-time

WAS 4 AE

underway

Legend

in plan

��

��

�� !"�� *��

+� �,��-

+� �.�///+��'"�

��"�'��"��

��#"��*"

available features

CVS

CMVC

PVCS

ClearCase

+�� "�� 0

WSxD

+-��12�� 2�'�"� �2�' � �� 3��(

��--��-�4 �+�4 �-�� -�� --��"��(

5�#��6�77 ��5�5� �2�� #"�� 8��%��

+� �2�%��'*�� 9�� %% ��(

�%%�"��"��"�"�'

�%��*"

��"�� "�"��

IBM Labs in Haifa


WSAA Quick Fact Sheet

� Powered by DB2 and WebSphere� Scans code from a variety of configuration management and runtime

systems � ClearCase, PVCS, CMVC, CVS, WebSphere Application Server, …

� Analyzes various languages and artifacts � Java, J2EE (Servlet, JSP, EJB, Tag Library), C/C++, XML, …

� Detect and understands complex semantic relationships and dependencies � Java to CICS, Java to MQ, …

� Extensible framework design� Use the SDK to extend its functionality

IBM Labs in Haifa


IBM Labs in Haifa


Assets MetadataJava Ejb War Jsp C/CPP

Servlet Cell

IBM Labs in Haifa


Details Pages

�Each asset type has its own details page

�Displays the asset metadata

IBM Labs in Haifa


Exploring the Topology�Topology exploration helps to understand what exists where

IBM Labs in Haifa


Compute Asset Relationship – Intra-Language

Java Classes Java Methods

�Computes dependencies between software assets within the same programming language

IBM Labs in Haifa


Compute Asset Relationship – Intra-Language

Hyperlinks Ejb References and implementing classes

�Computes dependencies between assets within the same language

IBM Labs in Haifa


Compute Asset Relationship - Inter-Language

Html�Servlet� Implementing class

War�Ejb

�Computes dependencies between assets in different languages

IBM Labs in Haifa


Compute Asset Relationship – Inter-Language

Ejb/Class�Jms connection Factory Jsp � Tag Library

�Computes dependencies between assets in different languages

IBM Labs in Haifa


Compute Asset Relationship - Higher level of abstraction

� Adds aggregation of relationships for higher level views�E.g. class to class reference may inspire Jar to Jar and package to package relationship

IBM Labs in Haifa


IBM Labs in Haifa


Closing the Gap between Distributed and zOS

��--6�"��"��--6�"��"��

�%% �� %% ��

-��-��-��-��-��+��%%

� �� 12�� )�!12�� )�!

� � ��

��

��"��

��--�4��--�4��--�4��

+� �9��+� �9��

��

��

��

��

Data Field

Data Record

Job Step Transaction

Program or Copy Source

Program

Statement

File, Database

Data Field

Data Record

Job Step Transaction

Program or Copy Source

Program

Statement

File, Database

:5 :5 ��

�� -4� �0

-� ��"��;��-� ��"��;��

WSAADistributed + zOS

+� ��

� The assets are organized in a hierarchical view� The relationships of each asset to other assets are maintained� But we still need to connect between the Distributed and the zOS assets

Problem: How do we compute the connector links?

IBM Labs in Haifa


Connector Representation

IBM Labs in Haifa


Context ctx = new InitialContext();

DataSource dataSource = (DataSource)ctx.lookup("java:comp/env/jdbc/BankDB");

Connection connection = dataSource.getConnection();

Statement statement = connection.createStatement();

ResultSet resultSet = statement.executeQuery("select NAME from CUSTOMERS");

while(resultSet.next())resultSet.getString("NAME");

Connector Composition� A connector is usually comprised of a few standard API calls that

�Defines the actual middleware or backend system�Activates/writes/reads to/from this middleware or backend system

Data Base access

IBM Labs in Haifa



QueueConnectionFactory qqcf = (QueueConnectionFactory)ctx.lookup("java:comp/env/jms/PriceChangeQCF");

Queue queue = (javax.jms.Queue)ctx.lookup("java:comp/env/jms/TriggerQ");

QueueConnection txconn = qqcf.createQueueConnection();

QueueSession sess = txconn.createQueueSession(true, 0);

QueueSender sender = sess.createSender(queue);

TextMessage tm = sess.createTextMessage("50");

sender.send(tm);

Connector Composition� A connector is usually comprise of a few standard API calls that

�Defines the actual middleware or backend system�Activates/writes/reads to/from this middleware or backend system

Jms access

IBM Labs in Haifa


Connector Composition

� A connector is usually comprise of a few standard API calls that�Defines the actual middleware or backend system�Activates/writes/reads to/from this middleware or backend system

CICS ConnectorInteractionSpec ceInteractionSpec =

new com.ibm.connector.cics.ECIInteractionSpec();

ceInteractionSpec.setTransactionName("OS2C");

ceInteractionSpec.setProgramName("TC591700");

IBM Labs in Haifa


Connector Detection

� Consists of two phases:�Pattern Detection

�Find occurrences of a pattern in code

� Inference Engine�Infer from those occurrences the relationship between the source

component and the backend system or middleware

IBM Labs in Haifa


What is a Pattern?� A pattern is a sequence of instructions and the relationship between them� We need to know certain data that flows in the program in order to use it in the

connector resolution�E.g., database JNDI name

� For example, the following two instructions result in a Data Base access


DataSource dataSource = (DataSource)ctx.lookup("java:comp/env/jdbc/BankDB");

Important DataMatched

IBM Labs in Haifa


Pattern Detector Language

� We developed an abstract XML based language and a tool for detecting the occurrence of patterns in code

� E.g., The same example in our language:

Resolved

Matched

IBM Labs in Haifa


Inference Engine

� Just discovering pattern occurrences is not enough�We want to know to which component this occurrence belongs�We want to know the actual backend system or middleware

� The Inference engine has the domain knowledge for inferring the components�E.g., J2EE based Inference Engine

IBM Labs in Haifa


Connector Detection Challenges

� Currently uses static analysis� In the future should combine with dynamic analysis

� Need interprocedural analysis for OO languages� Data flow analysis that is cross language

� Load data from external resources�E.g., property file

IBM Labs in Haifa


Related Tools

� Distributed Impact Analysis and Program Understanding�CAST Mining�MKS Discover�Headway Software - ReView

� zOS Impact Analysis and Program Understanding�SEEC – Mosaic Studio�Relativity – RescueWare

� Component Stores� LogicLibrary�FlashLine

� Impact Analysis in the IDE�Eclipse�Many More ….

IBM Labs in Haifa


impact analysis for cross domain enterprise applications · enterprise applications iftach ragoler,...

Documents