dennis wisnosky cross talk presentation d wiz 09262008 for publication

10/24/2008

1

Founding Sponsors

This Presentation Courtesy of the

International SOA Symposium

October 7-8, 2008 Amsterdam Arena

www.soasymposium.com

[email protected]

Gold Sponsors

Platinum Sponsors

Silver Sponsors

CrossTalk:

Market Research into SOA

State of the Art

Dennis E. Wisnosky, DoD BMA

CTO &

Chief Architect

October 7, 2008

10/24/2008

2

Background

Reach of the Business Mission Area

DWiz DoD BMA CTO/CA 2

"The Secretary of Defense is responsible for a half-

trillion dollar enterprise that is roughly an order of

magnitude larger than any commercial corporation

that has ever existed. DoD estimates that business

support activities—the Defense Agencies and the

business support operations within the Military

Departments—comprise 53% of the DoD

enterprise.‖

The Challenge


Many DoD systems are a decade

old and strain to support today’s

operations.

Many new systems are so

massive a decade passes before

they show results.

Our enemies change at the pace

of technology.

WE MUST ALSO.

10/24/2008

3

DWiz DoD BMA CTO/CA

A Small Slice of the As-Is

This is only the BTA DBSAE systems and Transaction

flows.

Add to this all the DoD Business Systems in the Army,

Navy, Air Force, USMC and all of the Defense

Agencies.

5

Defe

nse B

usin

ess S

yste

ms

Acq

uis

itio

n E

xecu

tive

B usiness Operating E nvironment (B O E )

T O -B E

AS -IS

B usinessS ervices

B us iness C apabilities

P rioritization &Harmonization

B us inessC apabilityE xtraction

S 1

S 2

S 3

S 4

S 5

S 6

S 7

S 8

S 9

S 11

S 10

S 12

…

BM

A B

usi

nes

s O

per

atin

g En

viro

nm

ent (

BO

E)

DoD & BMA Federated PortalBTA

Business Transformation Infrastructure

DoD Enterprise Services

Info

rmat

ion

Ass

ura

nce

Enterprise Service Management

MediationService

Discovery

Interoperability ControllerDistributed Integration Brokers Connected by Robust Messaging (MOM – Message Oriented Middleware)

Broker Broker

DoD & BMA

FederatedServices Registry,Data Catalog, &

Metadata Registry

Broker Broker Broker Broker Broker

Machine-to-Machine Messaging

High Volume Batch & XML-

based Transformation

Services

Other BMA Common Services

Business Infrastructure

Development

Business Activity

Monitoring (BAM)

Service Orchestration:Business Process &

Workflow Automation

Data Services: Data Virtualization Layer

Business Transformation Engine

Service Wrapper

New & TransformedApplications

BusinessService

Business Applications and Services

Stand-alone Independent Business Services

BusinessService

Business

ServiceLegacy

Applications

Service Interface


Conceptually we have designed a common operating

environment, the Business Operating Environment (BOE).Services are Discoverable, Composable, Scalable,

Reusable, commonly Managed and Secured.

Over the lifecycle, capabilities are deployed from new and

transformed systems as services into this environment.

.1 2

3

4

Current State and Future Vision: Its Happening Now

10/24/2008

4

This is a Concept of Operations for Services

Included in our DoD BMA Guidance – Is it viable?

DWiz DoD BMA CTO/CA

Will SOA Work for DoD

This is the infrastructure. Can commercially

available technology support its operation in DoD?


DoD Enterprise ServicesEIEMA Computing Infrastructure Domain


MediationService

Discovery


Broker Broker

DoD & BMA

Federated

Services Registry,Data Catalog, &

Metadata

Registry



High Volume

Batch & XML-

based

Transformation

Services



Development

Business

Activity

Monitoring

(BAM)

Service

Orchestration:

Business Process &

Workflow

Automation


Business

Transformation

Engine

Or is it just another pretty picture?

We decided to conduct research

into the state of the art of SOA.

Here we will address the

concepts and the findings.

7

DWiz DoD BMA CTO/CA

BM

A B

us

ine

ss

Op

era

tin

g E

nvir

on

me

nt

(BO

E) DoD Federated Portal



Info

rma

tio

n A

ssu

ran

ce

MediationService

Discovery


Broker Broker

DoD & BMA

Federated

Services Registry,

Data Catalog, &

Metadata Registry



High Volume

Batch & XML-

based

Transformation

Services



Development

Business Activity

Monitoring (BAM)

Service

Orchestration:

Business Process &

Workflow Automation


Business

Transformation

Engine

Service Wrapper

New &

Transformed

Applications

Business

Service

Business Applications and Services

Stand-alone Independent Business Services

Business

Service

Business

ServiceLegacy

Applications

Service Interface


The BOE, includes the foundational SOA infrastructure, the

BTI, portal and service-presenting applications.This illustration shows the details of the BOE.

8

Busin

ess O

pera

ting E

nvir

onm

ent

(BO

E)

DoD Federated Portal

Application 1 Application 2 Application 3 Application 4 Application n

…


DoD Enterprise Services

Business Operating Environment

10/24/2008

5

DWiz DoD BMA CTO/CA



MediationService

Discovery


Broker Broker

DoD & BMA

Federated

Services Registry,

Data Catalog, &

Metadata Registry



High Volume

Batch & XML-

based

Transformation

Services



Development

Business Activity

Monitoring (BAM)Business Process &

Workflow Automation


Business

Transformation

Engine


1

2

3

4

5

6

7

1. Interoperability

Controller

Vendors were asked to demonstrate technologies and answer

specific inquiries against components of this infrastructure.

2. Mediation – Standard

& High Volume

3. Service Discovery,

Data Catalog & MDR

4. Business Activity

Monitoring

5. Enterprise Services

Management

6. Business Process &

Workflow Automation

7. Data Virtualization &

Data Services 9



BTI Components:

1. Interoperability Controller

2. Mediation

3. Service Discovery

3a. Service Registry

3b. Metadata Registry

3c. Enterprise Catalog

4. Business Activity Monitoring

5. Enterprise Services Management

6. Business Process and Workflow Automation

7. Data Virtualization and Data Services

BOE Components:

8. Information Assurance

9. Governance

We will present patterns and conclusions for each of these

components and our findings based on the research.Findings: Patterns and Conclusions

10/24/2008

6

Vision of Standardized Enterprise Information

Exchanges Through the BTI

AF

DWiz DoD BMA CTO/CA

1. Interoperability Controller

Interoperability Controller

Broker Broker Broker Broker BrokerBroker Broker

Canonical Messages across Machine-to-Machine Messaging

USTC

Mediation

Service

Etc.

System Messaging

Message Recipients

Mediation

Service

Navy

USAF

USAF Etc.

Mediation Transforms

Outbound

Messaging

Inbound

Messaging

Common

Message

Vocabulary

Local

Message

Vocabulary

Canonical

Messaging

Mn≠Mc Mo≠Mc

BMA Ontology

1

2

3

4

5

Standards:HTTP/HTTPS

XML/XSD

SOAP 1.1 (Note)

SOAP 1.2 (Specification)

SOAP with Attachments

MIME/SMIME

REST

Web Services Addressing

Web Services Notification (WS-BrokeredNotification,

WS-BaseNotification, WS-Topics

SOAP Message Transmission Optimization

Mechanism (MTOM)

UDDI 3.0

WSDL 1.1

WSDL 1.2

WSDL 2.0

Web Services Reliable Messaging

Web Services Security 1.0

Security Assertion Markup Language (SAML)

Business Process Execution Language (BPEL) for

Web Services V1.1

WS-BPEL Extension for People 11

DWiz DoD BMA CTO/CA

12

1. Interoperability Controller - Conclusions

SOA Infrastructure must provide for:

Extensible set of integration brokers interconnected on

the network by robust messaging middleware

Execution of brokering, routing, and processing

messages and service invocations

State of the Technology:

Products which most closely support the

Interoperability Controller pattern are Enterprise Service

Bus, and Enterprise Application Integration and

Message-Oriented Middleware components

Industry products are reasonably mature

Industry challenge is a standards-based SOA stack

leveraging Web technologies, rather than an ESB-based

solution

10/24/2008

7

Business Transformation Infrastructure invoking the Mediation Service

USAF

Navy

Army

Other

USAF

Navy

Army

Other

Canonical

Message

Format

Common

VocabularyTransform

Vocabulary

& Format

Route

Message

Use

Common

Vocabulary?

Source

Non-Standard

Transform

Format

Yes

Yes

No

No

In

Canonical

Format?

Transform

Vocabulary

& Format

Transform

Format

Need

Local Vocab.

& Format?

Yes

Yes

No

No

Need

Local

Format?

Business

Transformation

Engine

Business

Transformation

Engine

Joint COI CDM

Message Flow

1

3

5

Vision of Standardized BOE Canonical Message Model

and Mediator

Messages typically are not in a common format, and

require mediation/transformation.

If all use common vocabulary and format, no

transformation is needed: faster, cheaper.2. Mediation

DWiz DoD BMA CTO/CA

13

Standards:OWL OWL-S ISO

11179 ebXML RDF

XSLT

BTI Canonical Message Model and Mediator

2

4

Canonical Messages are in Joint COI Canonical Data Model

(CDM) message formats, using the Joint COI CDM common

vocabulary based on DoD Universal Core and Business Core

common vocabulary standards.

Defense Business Systems Program of Records send

messages in local formats specific to their component or

program tier architectures.

DWiz DoD BMA CTO/CA

14

2. Mediation – ConclusionsSOA Infrastructure must provide for:

Short term need for translating and transforming

(mediation) of information exchanges/messages

Dynamic generation of transformations on a semantic

basis


Good vendor support of this pattern in both the

standard and high volume variations.

Transformation engines focused on XML messaging

and the use of XSLT transformation engines quite

capable

Advanced parallel processing capabilities allows for

high performance straight-through mediation services

Semantic mediation / semantic technology is immature,

only build time currently supported..

10/24/2008

8

DoD Enterprise

Catalog

DoD Metadata

Registry

DoD Data

Asset/Service

DoD Service

Registry

Discovery via

Enterprise

Catalog

Discovery Mechanism

Discovery of

schemas via

Federated

Search

Discovery via

Service

Registry

DoD Metadata StoreRegistered Data Asset/Service

Standards:

OWL UDDI XML Schemas

RDF DDMS WSDLDWiz DoD BMA CTO/CA 15

Asset or service provides standardized descriptive info to facilitate

visibility and discovery by DoD users. This is done in the DoD Enterprise Catalog. It houses instances

of DDMS-compliant descriptions of DoD Data Assets & Services.

.

Asset or service publishes schema for storage,

transactions, and information exchange formats.

The DoD Metadata Registry (MDR) is a clearing-house for storage of

metadata schematic formats. It is built to the ebXML reg/rep spec, has

Web Services interface and Web GUI.

Service registers and publishes its interface in UDDI

format, using WSDL for Web Services.

The DoD Service Registry is a UDDI- and Web Services

compliant registry.

.

3. Service Discovery

1

2

3

DWiz DoD BMA CTO/CA

16

Local/Regio

nal

Share

d

Unsh

ared

Prioritize

Requirements

Model

against

BEA

Service

Adoption

or Promotion

Develop

Services

Service

Composition,

Orchestration

Service

Certification

Service

Deployment

Operate Service

Service Discovery

Services by Consumer

Reach

Local/RegionalEnterprise

Non-Core Shared UnsharedCore

Standards:Universal Description and Discovery

Interface (UDDI)

ebXML Registry Information Model (RIM)

Registry Services (RS)

Service discovery must transcend the lifecycle of

services.

Service discovery provides for identifying the state of

the service.Service discovery must provide for classification and

categorization.

Service are discoverable based on what reach we

allow a consumer to have (Information Assurance).3a. Service Registry

10/24/2008

9

Metadata

Registry

ebXML Web Services

ebXML Reference

Implementation

DWiz DoD BMA CTO/CA

17

Standards:SOAP/JAXR

ebXML Registry Information Model (RIM)

Both known and unanticipated data consumers can “Pull”

structural and semantic Metadata from the MDR.

The Metadata Registry provides a location for the

publishing of structural and semantic Metadata.3b. Metadata Registry

Data Consumers including

MDR Users/DevelopersServer-side

Application

Layer1

2

4

5

3

Metadata

Artifacts

Other Developers/

Applications

DWiz DoD BMA CTO/CA18

Capture terms and definitions at the beginning of the

architecture process from Authoritative Sources. Make

them discoverable and reusable through toolsets.

Standards:

OWL XML Schema

RDF DDMS

3c. Enterprise Data Catalog

10/24/2008

10

DWiz DoD BMA CTO/CA

19

3. Service Discovery Conclusions


Metadata registries and repositories must support

discovery of services and information assets

Ability to federate

DoD UDDI service registry, Metadata Registry of structural

and semantic metadata, and Enterprise Catalog

specification metadata are standards-based


Vendors provide UDDI Service Registries, or include UDDI

capability

Vendors provide metadata management capabilities and

repository, others specialize around semantic metadata

Many vendors registries and repositories don’t federate

DWiz DoD BMA CTO/CA

20

BPM:Business

Process

Monitoring

Level using

architectural

Primitives and patterns

ESM:Enterprise

Service Mgmt

Monitoring

Level

BAM:Business

Activity

Monitoring

Level

Business

Service

Business

Service

Business

Service

Business

Service

Business

Service

Business

Service

Business

Service

BAM Presentation;

Dashboards &

Action Screens

Business Management

Transform technical

events to process events

…

Transform process events

to business events

BAM can govern End-to End process flows with Enterprise,

Component or Program Services running in the BOE

End to End Business Processes described in the BEA used as

basis for abstraction into BPMN from BEA AV-2, OV-5, OV-6c, et. al

Business

Enterprise

Architecture

and Core

Business

Mission EA

Enterprise,

Component

and Program

Services from

Business

Systems

Standards:

Business Process Modeling Notation (BPMN)

Business Process Execution Language (BPEL)

XML Process Definition Language (XPDL)

1

2

3

4

5

4. Business Activity Monitoring

10/24/2008

11

DWiz DoD BMA CTO/CA

21


Modeling & execution of business processes through

orchestration

Monitoring of those business processes.


Tools provide for orchestration at build time and

monitoring of those processes at run time

While tools with semantic continuity from modeling to

execution are not ready today, they are under

development

No vendor included Universal Modeling Language

(UML) in either its list of product offerings

4. Business Activity Monitoring -

Conclusions

DWiz DoD BMA CTO/CA

22

ESM is closely linked to and enables Governance and

Information Assurance

Service 2 Service 3 Service n…Service 2

Human

Service Manager

ESM Agents Enterprise, Component or Program Services

running in the BOE

Alerts &

Notifications

Human Manager

Response

Automated

Response

1

2

3

4

ESM makes visible BOE services’ operations in terms

of health and also performance against SLAs. Agents be automated to act providing fail safe and error

handling mechanisms to ensure unbroken operations.

Dashboards provide

overall views of the health

and operation of the

services in the BOE.

5. Enterprise Service Management

Automated

ESM

Facilities

ESM

Presentation; Dashboards &

Action Screens

3

10/24/2008

12

DWiz DoD BMA CTO/CA

23

5. Enterprise Service Management Conclusions


Necessary to manage the service lifecycle

Is the foundation for SOA runtime governance

Must integrate with Information Assurance products


Limited number of ESM vendors

Vendors posses strong portfolios in traditional network

management and Integrated Service extending to ESM

Products differ on where in the OSI layer they focus, lower

level IT service management or higher level business

management

Solutions stacks often combine multiple company products

to provide end-to-end ESM

Processes can be built End to End from disparate piece

parts by engineers AND business analysts

DWiz DoD BMA CTO/CA

24

Infra-structureService

BusinessService

CompositeService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService






CompositeService

CompositeService

CompositeBusinessServices

BasicBusinessServices

InfrastructureServicese.g., DoD CES

OrchestrationEngine Executinga Business Process

Service

s Layer

From Business Analysts/Modelers

Workflow:Person ActionSteps

Standards: Business Process Modeling Notation

(BPMN)

Business Process Execution Language

(BPEL)

XML Process Definition Language (XPDL)

1

2

3

4

5


24

10/24/2008

13

DWiz DoD BMA CTO/CA

25


ConclusionsSOA Infrastructure must provide for:

Business process modeling execution engine or workflow

engine

Standards compliance very important

Round-trip from architecture to implementation and back


Most vendors provide modeling and workflow environment,

many are proprietary or proprietary extensions

Some modeling tools are business analyst usable (truly

WISWIG GUI), other tools require computer programmers

Round-tripping not a capability to be provided in near

future – remains the ―holy grail‖

Enterprise Applications (Data Consumer)

26

Business Systems

Business System

BusinessService

BusinessData Store

BusinessService

BusinessService

BusinessSystem

BusinessSystem

5BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

BusinessService

Data Services

Query Optimization and Processing Engine

Data Virtualization Layer

Caching

Language specific (Java, C#...)API Database (OLAP, etc.) Flat file Web Service

Enterprise Service Management (external to Data Services)

Information Assurance (Security) (maybe external to Data Services)

1

2

3 4

5

6

Provide virtualized data based on operational needs and representative of

the authoritative source in alignment with metadata and the BEA.

Standards:

SOAP OWL WSDL

ODBC JDBC

Consumer Consumer Consumer Consumer Consumer Consumer Consumer

7

Data Providers

DWiz DoD BMA CTO/CA

7. Data Virtualization and Data Services

10/24/2008

14

DWiz DoD BMA CTO/CA

27

7. Data Virtualization Conclusions


Virtualization of data sources not data warehousing

Data trust through presentation of operationally staged

data from authoritative sources

Real-time availability when necessary operationally


Vendor products provide data virtualization relying on

query optimization and or caching

Vendors can provide high performance and robust data

sources and services reducing latency in data availability

No need for periodic loads


Authorization Model Using Both Local and

Enterprise-level Attributes

Standards:

WS-* WS-Trust SOAP

SAML eXtensible

Access Control Markup Language

(XACML)

FROM DoD Security Reference

Architecture v1.0

1

2

345

6

7

8. Information Assurance

10/24/2008

15

DWiz DoD BMA CTO/CA

29

8. Information Assurance Conclusions


Net-Centric data sharing capabilities enabled by SOA are

themselves potential points of vulnerability

Standards and standard protocols to narrow the range of

network capabilities that an attacker must subvert

Survivability, reliability, availability and non-repudiation

Policy enforcement including authorization control, redaction

and auditing

Federation to Public Key Infrastructure enabling single sign-on,

and preservation of non-repudiation


Vendor products provide data virtualization relying on query

optimization and or caching

Vendors can provide high performance and robust data sources

and services reducing latency in data availability

No need for periodic loads

DWiz DoD BMA CTO/CA

30

Governance is closely linked to Enterprise Service

Management and Information Assurance

Local/Regio

nal

Share

d

Unsh

ared

Prioritize

Requirements

Model

against

BEA

Service

Adoption

or Promotion

Develop

Services

Service

Composition,

Orchestration

Service

Certification

Service

Deployment

Operate Service

Lifecycle

Governance

from Design in

Build Time

through Run

Time to

deprecation

Run

Time Build

Time

DoD Governance of DoD Core Data, Universal Core

Business Enterprise Common Core Metadata COI, Biz Core

FM COIData Governance

HRM COIData Governance

FM COI Info

Models

HRM COI Info

Models

WSLM COI Info

Models

MSSLM COI Info

Models

Warfighter

Communities

of Interest

WSLM COIData Governance

MSSLM COIData Governance

RP&ILM COIData Governance

RP&ILM COI Info

Models

Intelligence

Communities

of Interest

Data Governance

Atomic

Service 3

Atomic

Service 4

Atomic

Service n…Atomic

Service 1

Atomic

Service 2

Human

Service Consumer

HCI for

Service

(Web UI, Portlet)

Machine Process

Service Consumer

Service Providers

Service Consumers

Service Providers

Service Consumers

Anticipated and Unanticipated

Service Users, Human and Machine

Composite Service

Enterprise, Component

or Program Services

Enterprise, Component

or Program Services

Atomic or Base Services (non-composite)

Quality of Service

governed by SLA

agreed to by Service

Provider

SAE and PEO for a Service are

those of the program and system that

actually provides the service, whether

atomic or composite, whether

designated an enterprise or

component

Governance is set by the service provider and designed in Policies desired

for the Human and Machine Service Consumer against SLA specified QoS9. Governance

10/24/2008

16

DWiz DoD BMA CTO/CA

31

9. Governance Conclusions

SOA Infrastructure must provide for:Assure that laws, regulations and policies are met in IT operations

Build Time (Investment) Governance to allow for management of IT

development policy and service reuse

Run Time Governance

State of the Technology: Tools have limited interoperability with design and develop services

and no way of automated compliance checking or management of the

transition between EA models and service designs

Duplication Avoidance is met through tool access to service registries

and repositories

Service Verification for testing and verifying services against functional

requirements and Service Level Agreement (SLA) exist with tradition

testing tools.

Tools integrate with registries to provide developers with service

descriptions and specifications

Support for capturing SLAs exists development tools for SLAs is limited

Results of the Research


BOE Component

SOA Tool Vendor Bu

ild

Tim

e

Ru

n T

ime

De

ve

lop

-

me

nt

Infr

a-

str

uc

ture

Baseline: NCES CES ?

Ab Initio

Active Endpoints

AmberPoint

BEA

Composite Software

Corticon (Bus Rules) w/ Metastorm

Fiorano

GEFEG (9/12/07)

HP (Mercury / Systinet)

IBM - Telelogic (EA Tools) (9/19/07)

IBM Websphere

Information Builders / iWay

Iona

Lombardi

MetaStorm (BPM)

Microsoft

Mirius (9/14/07)

MuleSource

NexaWeb

OpenSpan

Oracle

Red Hat-Jboss-Metamatrix

Revelytix

RogueWave

SAP

SOA Software

Software AG-webMethods

Streambase

SUN

TIBCO

Me

ss

ag

ing

Se

rvic

e

Orc

he

str

ati

on

Re

gis

try

Re

po

sit

ory

Me

tad

ata

Ma

na

ge

me

nt

Service

Governance

Se

rvic

e

Co

mp

os

itio

n

IA A

uth

en

tic

ati

on

& A

uth

ori

za

tio

n.

Se

rvic

e

Imp

lem

en

tati

on

Se

rvic

e W

rap

pe

r

Inte

gra

tio

n

Se

rvic

e

Ma

na

ge

me

nt

Da

ta

Vir

tua

liza

tio

n

Inte

rop

era

bil

ity

Co

ntr

oll

er

Bro

ke

r

BA

M

BMA Common

Services

Co

mp

lex

Ev

en

t

Pro

ce

ss

ing

ED

A E

na

bli

ng

Hig

h V

olu

me

Tra

ns

form

ati

on

Me

dia

tio

n

10/24/2008

17

Conclusion

―The DoD BMA has embarked on an SOA strategy. The ―BMA

Architecture Federation Strategy and Roadmap‖ provides guidance

for the DoD BMA to quickly gain business value by delivering

capability to support the warfighter through an SOA, while using a

phased approach for transforming legacy systems. The market

research performed by the BMA Office of the CTO and CA has found

that industry capabilities to implement or enable the components

defined in the BMA Service-Oriented Infrastructure have matured in

the marketplace. While serious caution remains in the areas of IA

and security, and the need for significant cultural change for

successful SOA implementation cannot be overemphasized, it is

clear that it is feasible for an enterprise the size of the DoD to move

forward on implementing an SOA and to realize the business

benefits of agility, interoperability, and net-centric data sharing that

an SOA provides.‖ ~ CrossTalk, Oct. 2008 IssueFull article can be viewed at:

http://www.stsc.hill.af.mil/crosstalk/2008/10/0810WisnoskyFeldshteynMancusoGoughRiutortStrassman.html


DWiz DoD BMA CTO/CA

One Answer: Companies Supporting Standards

Object Management Group (OMG), Windows

Workflow Foundation, Organization for the

Advancement of Structured Information Standards

(OASIS), Workflow Management Coalition (WFMC),

World Wide Web Consortium (W3C) WS-C …WS-X

And, this requires World Wide collaboration!

33

10/24/2008

18

Thank You for Your Time!

DWiz DoD BMA CTO/CA

Any Questions?

[email protected]