handling complexity (mess?)integration or federation

Stephen ToddIBM WebSphere MQ

e-Science Institute: Edinburgh14 October 2003

The opinions expressed here are those of the authorand do not necessarily represent those of IBM.

outline

what are the difficulties facing• our customers?• the industry?

how should we address these difficulties• integration?• federation?

?

customer difficulties

lots of departments• every customer address stored 5 times

•in 5 different technologies• don't even know if they are the same customer

mergers and acquisitions• complexity - scale - heterogeneity

i.e. .....

complexity

clean complexity• quantum theory• non first normal form

dirty complexity• islands of automation• heritage applications and systems

(smart complexity?)• (autonomics?)

bomb

the industry has a solution

let us sell you our• magic middleware

–database system–application server–messaging system

• application solution

even for legacy

we can even wrap your old one• eg relational front end to an IMS database

"It's easy to put a relational front end on a pure IMS database~~~~at least, it would be if there were any."

dirtycomplexity

we can all grow with your needs

1 2

34

and the result is

DB

2

Oracle

Syb

ase

IMS

WebSphere app

server

CICS

WebLogic

MQRendezvousMSMQ

different dirty complexity

luckily, we have a solution

let us sell you our systems management system

database

applicationserver

messagingsystem

systemsmanagementsystem

so ....

can't you give us a more integrated solution?

but ...

but ... middleware religion

corporate directive• databases are ...• application servers are ...• messaging system is ...• (no MS software, but 1000 VB programmers)

"We can't install your messaging system if it requires DB2 -- even if it is hidden.

Corporate directive is Oracle."

complexity and contradiction

so, what are our problemswhen providing middleware to

help?

database

applicationserver

messagingsystem

systemsmanagementsystem

many overlapping solutions• integrated islands• heritage products

how many transaction coordinators?how many databases?

• and even more persistent stores...

our own dirty complexity

databaseapplication

server

messagingsystem

systemsmanagementsystem

product growth example: MQ

'simple' point-to-point messaging/queuing• reliable, heterogeneous

resource manager not database because ...transaction coordinator not external because ...publish/subscribebroker

• message semantics and dictionary not schema because ...• transformations not SQL because ...• database interaction

-with many databases so no integration ...• almost an application server but not because ...

so, potential for integration

common toolingcommon systems administrationcommon data and programming modeletc etc

databaseapplication

server

messagingsystem

databaseapplication server

least affinity ~~ impedance mismatchsubsumption, not integration

• even back to CICS, IMSDB subsumes application server

• stored procedures & UDFs make DB an app serverapplications subsume database

• programming persistence or object DB–removes need for (explicit) DB–but loses much DB modelling and query power?

integration potential

application serversmessaging

increased 'active' component in messagingneed for wider reach in app server

• more heterogeneity• wider geographies

–implies distributed, async–linked transaction model

integration potential

database / messaginglow level

• persistence, resource management, transactionshigh level

• transformations, data models, streamsdata placement and replication

relation

input stream result stream

integration potential

the data you want• where you want it• when you want it• in the form you want it

integration potentialsame messages, same

pictures

but should we integrate, or federate, or ...?

integration• cleaner models• easier administration

federation• heterogeneity• choice• handle dirty complexity

Can componentization give us the best of both?How big must the components be?

How interdependent?

What does the future hold?Will it change anything fundamentally?

WebServices• same technology, another name• very strong federation credentials

•(how widely will it really work)Grid

• ??? ### ???Aspect programmingPickled chocolates

so, to summarizebig, horrid monsters

• dirty complexity

• face our customers• face the industrywhat's the solution?

(We know how to draw the picture)

• integration• federation

or ....

brand solution

customers want integrationbut it's impossible in the real world

so rebrand federation as integration• and give them what they want• AND what they need