keys to soa success draft - dovel...

1

Copyright © 2006, ZapThink, LLC 1

Keys to SOA Success

Jason BloombergZapThink, LLC

Take Credit Code: SOAKEY


Agenda

SOA Level Set

SOA Foundations

SOA Governance

The SOA Roadmap

Rolling Out SOA

Organizational Issues

Case Studies

2


SOA: Level Set


SOA: Definition

An approach to building and managing distributed computing infrastructures that considers IT resources as Services available and discoverable on a network.

3


SOA Features

• SOA is architecture – a set of best practices for the organization and use of IT

• Abstracts software functionality as loosely-coupled, business-oriented Services

• Services can be composed into business processes (which are also Services) in a declarative manner


Characteristics of SOA

• Services are loosely coupled – making a change to a service provider does not mandate changing any service consumers.

• Business processes are composed of services, and are in turn exposed as services.

• Services are policy-driven – business users can change how a service behaves.

• Systems are inherently integrated by virtue of composable services – not through layers of middleware.

4


Characteristics of SOA(Cont.)

• Services leverage legacy systems – SOA does not mandate replacement of runtime infrastructure.

• In SOA, metadata controls how the system behaves instead of code – business logic trumps application logic.

• In SOA, it’s the contracted interface that matters, not the underlying runtime environment.


SOA is Enterprise Architecture

What is Architecture?The fundamental organization of a system embodied by its

components, their relationships to each other and to the environment and the principles guiding its design and evolution. (IEEE P1471/D5.3)

Enterprise architecture includes:• An aggregated architecture of all the individual IT systems

within an organization • The human element within the enterprise• Systems, people, and organizational constructs at other

companies that have relationships with the enterprise• Individual consumers who are that enterprise’s customers• Corporate governance

5


CompositeApplication

Delivery

BusinessServices

BusinessProcesses

SOA Fabric AtomicServices

Data Integration

Legacy Applications

and Middleware

Databases

Governance and Security InfrastructureGovernance and Security Infrastructure

SecuritySecurity

ManagementManagement

RoutingRouting

TransformTransform

AA

BBCC

DD

EE

AA

BBCC

DD

EE

MessagingMessaging

Rich Rich ClientsClients

Composite Composite AppsApps

DiscoveryDiscovery

Source: MW2 Consulting

SOA: A Technology View


Service orientation…the next big thing?

Approach TimeframeProgramming

ModelBusiness

Motivations

Mainframe timesharing1960s –1980s

Procedural (COBOL) Automated business

Client/server 1980s-1990s

Database (SQL) and fat client (PowerBuilder, Visual Basic)

Computing power on the desktop

n-Tier/Web 1990s-2000sObject-oriented (Java, COM)

Internet/eBusiness

Service orientation 2000sService-oriented (SOAP, WSDL, UDDI)

Business agility

6


SOA shifts the way we think

Compose ServicesIntegrate Silos

Loosely Coupled, Agile and Adaptive

Tightly Coupled

Designed to changeDesigned to last

Favors Heterogeneous TechnologyFavors Homogeneous Technology

Architecture makes it workMiddleware makes it work

Interactive and iterative development

Long development cycle

Metadata OrientedCode Oriented

Service Oriented ArchitectureTraditional Distributed Computing


How to Think Service-Oriented

• Think of software functionality as Services

• Think of Services as sending and/or receiving messages

• Think of integration as a byproduct of Service composition

7


The Best Technology…

• Is complex on the inside yet simple on the outside

• The secret is the abstraction layer


What about Coupling?

• Coupling – the level of common knowledge necessary in a distributed computing exchange

• Tight coupling – one participant must have detailed knowledge about the other participant

• Full decoupling – the two participants need have no knowledge about each other in order to interact

• Loose coupling – the two participants may have specific, limited knowledge about each other

8


Service Contracts & Loose Coupling

• Service contract – a document external to each participant that provides the information each participant needs to interact with the other

• The service contract is the mechanism that enables loose coupling

• Web Services Description Language(WSDL) – standard Service contract language


What’s in a Contract?

Functional Requirements

Non-Functional Requirements

Service Operations Infrastructure URI

Security

ServiceLevel

Transactions Commerce / Biz Rules

Semantics Process

9


Service Model Drives Contract-First Development

• Service contracts specify required functionality to IT and provided functionality to the business

• Service model represents the clearinghouse for information about IT environment

• Contracts go beyond WSDL:– Usage policies– Security policies– Consumer delivery contracts– Service-level agreements, etc.


SOA Foundations

Business Model(Use Cases) Service Model Platform

Dependent Models

10


The SOA Metamodel

Business Model(Use Cases) Service Model Platform

Dependent Models

Deployment View

System Architects &System Engineers

Implementation View

Technical Architects & Developers

Process View

Business Analysts

Logical View

Line-of-Business Users

TechnologyViews

BusinessViews

Use-case View

Service-Oriented Architects

Copyright © 2003 ZapThink LLC


SOA Foundation: The Zachman Framework

11



SOA & Zachman

DATA FUNCTION NETWORK

SOA

12


IBM Service Integration Maturity Model (SIMM)

Source: IBM


HP SOA Maturity Model

Source: HP

13


SOA Governance


Corporate Governance

• Establishing and communicating the policiesthat employees must follow

• Giving employees the tools they need to be compliant with those policies

• Providing visibility into the levels of compliance in the organization

• Mitigating any deviations from established policy

From project to program to sustainable processFrom project to program to sustainable process

14


Creating the Governance Model

• Who is responsible for creating policies?• Which policies are within scope?• How will they create and communicate those

policies?• How will policies be represented?• What tools will people use to follow policies?• How will management get visibility into policy

compliance?• How will mitigation issues be addressed?


IT Governance Activity Types

• IT principles– Clarifying the business role of IT

• IT architecture– Defining integration and standardization requirements across IT, as

well as organizational principles & best practices that govern IT infrastructure

• IT infrastructure– Determining shared technology within the organization

• Business application needs– Specifying business need for purchased or internally developed IT

applications

• IT investment and prioritization– Choosing which initiatives to fund & how much to spend

15


Elements of IT Governance Strategy

• Architecture board– Establish a cross-organizational architecture board with

the backing of top management to oversee implementation of IT governance strategy

• Architectural principles– Delineate comprehensive set of architectural principles to

guide, inform and support organization

• Architecture compliance strategy– Adopt specific measures to ensure compliance with the

architecture, including project impact assessments, a formal architecture compliance review process, and the involvement of architecture team in product procurement


Architectural Governance Processes

• The architecture review and approval process– Defines structured approach for review process & approves

changes to the existing architecture– Makes decisions in accordance with architecture roadmap

• The architecture exceptions and escalation process– Provides for ability to appeal architectural decisions– Allows exceptions to the architecture to meet specific

business requirements

• The architecture maintenance process– Maintains the architecture & communicates architectural

changes to the stakeholders – Focuses on communicating new & changed Services to

stakeholders– Documents & communicates exceptions to architecture

16


Architectural Governance Processes

• The architecture communication process– Makes the architecture available to everyone who needs

access to it – Promotes understanding of the importance of SOA

• The architecture compliance review process– scrutinizes the compliance of a specific project against

established architectural criteria and business objectives. Questions cover system engineering, information management, security and systems management. Formal review processes form the core of an enterprise architecture compliance strategy.


SOA Governance

–Policy management• SOA configured & controlled via metadata,

including policy

–Visibility• Services abstract heterogeneous data sources,

providing necessary business intelligence

–Flexibility• Ability to build Services that address compliance

issues and adjust them as regulations or business needs change

Not just governance of SOANot just governance of SOA……governance in the context of SOAgovernance in the context of SOA

17


SOA Guiding Principles

• Reuse, granularity, modularity, composability, and “componentization”

• Compliance to standards, both horizontal and industry-specific

• Identification and categorization of Services• Service provisioning and delivery• Service monitoring and tracking


SOA Architectural Principles

• Encapsulation of application functionality• Separation of business logic from underlying

technology• Leveraging existing assets wherever an

opportunity exists• Service lifecycle management• Efficient use of system resources• Service performance

18


Enabling Service Domains

• A Service Domain is a logical grouping of shared Services with a common business context

• Examples: customer-facing Services,purchasing-related Services

• Manage Services by managingthe Domains

• Move away from traditional IT silos for the purposes of managingServices, but retain technical teamsas needed


Service Domain Roles

SOA governance introduces new roles that the company must provide for:

• The domain owner– Manages the direction of the domain and the business

relationships between the domain and business units, as well as other domains

– Helps business process owners in various business units understand the business application of the Services within the domain.

– Tracks the usage of Services for management purposes and ROI calculations

• Domain SO business analyst– Identifies abstracted, normalized business Services– Translates business requirements into Service definitions– Works closely with IT personnel to direct Service implementation

19


Service Domain Roles

More new service domain roles:

• Line of business representative– Communicates business requirements– Identifies business Services for each of the domains

• Domain developer– Builds and maintains Services consistent with the SOA

lifecycle– Implements Services consistent with implementation

guidelines and SOA

• Service tester– Certifies that each Service conforms to the business

requirements that apply to that Service– Builds test cases for Service interface


The SOA Roadmap

20


The ZapThink SOA Roadmap


ZapThink’s SOA Implementation Roadmap

Heterogeneous Systems with Proprietary Interfaces

Manage Services

Implement the SOA Metamodel

Service-Oriented ProcessSemantic Integration

Dynamic Service Discovery

Secure Service Interfaces

Wrap Legacy Systems in Services Interfaces

Service-Oriented Enterprise

Enterprise SOA Buildout

Mission-Critical SOA

SOA Pilots

“Grass Roots” Web Services Implementations

Create a Governance Framework

Contract-First Development

Cross-Departmental SOA

21


Emerging Best Practices

• Architecture is a discipline– SOA mandates new practices

and methods

• Best Practice: Iterative development– Start small, and let it grow

• Best Practice: Contract-first, top-down as well as bottom-up

• Best Practice: Implementing governance sooner than later

You can’t get SOA from a software vendor!


Starting Point: Architectural Visioning Session

• Attendees: architecture team, business analysts responsible for business processes

• Provide the SOA perspective

• Processes drive the Services, Services drive the technology

• Split into two initiatives: process definition followed by Service definition

• Balance “big picture” with realistic pilot starting point

22


Bring Together Different Mindsets

• Developer Mindset: “Bottom-Up”– Everything is a Service or an Interface– Goal: connect Services– Method: Use objects and App Servers– Problem: Too many things to connect!

• Business Mindset: “Top-Down”– Everything is a Process– Goal: Run business efficiently: manage

processes– Method: Use diagrams and flowcharts– Problem: How can you turn “shelf-

ware” into software?


SOA Pilots

• A few high ROI Services• Build acceptance for SOA• Get team up to speed• Work out the kinks• Pilot the governance model• Part of an iterative approach to SOA

• Piloting only the Services instead of the architecture• Remember, the pilot is one step on the roadmap

DANGER! Avoid the SOA Pilot PitfallDANGER! Avoid the SOA Pilot Pitfall

23


Cross-Departmental SOA

• Organizational issues of governance and control become paramount

• Long-term architectural plan critical• Increased focus on semantic issues


Rolling Out SOA

24


How do you manage change?

• SOA is all about continuous and sometimes unpredictable change

• Development issues

– How to handle versioning?

– How to handle metadata management?

– How to develop changing policies?

• Runtime issues

– Service availability

– Policy enforcement

– Guarantee Service-level agreement

– Maintain low TCO


Key Elements of Successful SOA Projects

• Building the right SOA team– A diverse group that bring different perspectives and

wide-ranging support to the implementation

• Handling organizational/people issues– Human resistance to change more challenging than

the technical issues

• Tackling the project iteratively– Won’t have full spec as you get started

• Managing the Service lifecycle– Very different from the traditional software

development lifecycle (SDLC)

25


Building for ongoing change:The Death of the SDLC

• Traditional Distributed Software Development– “Waterfall” – Gather requirements,

design, develop, test, deploy as separate steps

– Works great when things don’t change– But, over time, usually fails to meet

ongoing business requirements• What if things are never “done”?

– Need Iterative approaches– Same order, with overlapping cycles– Better, but still assumes project

completion• SOA – applications are never complete,

Services are always in flux– Traditional SDLC wholly inadequate


The Agile SOA Lifecycle

• Agile architectures demand agile development approaches

• The Agile Manifesto:– Working software over documentation– People over programming– Begin with metadata

• Agile SOA:– Iterate between architecture and

implementation– The business drives the applications– Contract-first development

Software lifecycle at the Service levelSoftware lifecycle at the Service level

26


Implementing SOA:Bottom-Up

Put Service wrappers around existing applications

• Pros:– Reduces cost of integration

• Cons: – May not be reusable– May be redundant– Management challenges


Implementing SOA:Top-Down

Create architectural plan & detailed design

• Pros:–Agility, reuse, flexibility

• Cons: –May not be implementable–Difficult to budget

27


SOA Must be Both

• Develop the vision (but not the details) ahead of time

• Decompose some processes to identify target Services

• Build modest set of Services• Compose applications to enable

flexible processes• Refine architectural plan• Repeat

SOA should be iterative


Challenge: Service Granularity

• The trick of building composable Services is building at the right level of granularity

• Challenges:– Engraining business logic into code– Decomposing legacy services that are not fine-

grained enough

• Method– Top-down process decomposition, vs. bottom-

up Service development– Must be iterative

28


Service Lifecycle Considerations

• Lines of business should work with metadata — no IT involvement

• Services go thru individual lifecycles – development, test, production, revision

• Service development driven by Service contracts

Service Model

Service Metadata

ExistingInfrastructure

Lines ofBusiness


The Roles of SOA

Use Case ViewSOAs

Use Case ViewSOAs

Use Case ViewSOAs

Logical ViewFunctional requirements

BusinessStakeholders

Information View

InformationSpecialists

Business AnalystsLOB Experts

Process ViewProcess ViewProcesses

Use Case ViewSOAs

Enterprise Architects

Deployment ViewPlatforms

Systems Experts

Data View

Data Experts

Implementation ViewComponents

Component ArchitectsDevelopers

Copyright (C) 2004, ZapThink, LLC

29


The Services Lifecycle Team

• Business analysts– Adjust Service specs depending on business requirements– Reconfigure Services & composite applications

• Enterprise/SO architects– Manage Service model

• Technical/project architects– Specify implementation changes

• Service provider/consumer developers– Implement requirements and policy

• Testers– Insure conformance, simulate Service providers & consumers

• Support personnel– Respond to user issues


An Instance of the Service Lifecycle in Action

New require-

ment

Simulatechanged Service

New/changedcontract

Implem’tchangedService

Problemresolution

Processconfig

Change?

Deploychanges

TestService

Businessuser

Support

Operations

Tester

Architect

Servicedeveloper

Businessanalyst

Consumerdeveloper

30


Organizational Issues


Building the right SOA team

• Shared Services cross organizational boundaries

• Siloed IT management styles are becoming obsolete

• The new role for enterprise architects

31


People, Change and Fear

• People are inherently resistant to change

• People consider job security, authority and responsibility when asked to share

• Fear is the strongest emotion of all!


Interaction Challenges

• Cultural Issues– Network Ops and Developers don’t talk to each other

• Budget issues– Who pays for Service Infrastructure / intermediaries?

• Responsibility issues– Who is in control of policy?

Developer/ArchitectDeveloper/Architect Network OperationsNetwork Operations

Services blur the Application / Network Boundary!Services blur the Application / Network Boundary!

32


More Interaction Challenges

• Management issues– People tend to avoid risk, stay within “comfort zone” - may appear stubborn

• Technical issues– Architecture is a difficult subject

• Cultural issues– The “Ivory Tower” problem…

ArchitectArchitect Development/TestingDevelopment/Testing

Architecture is difficult to mandateArchitecture is difficult to mandate


Convincing Technical Specialists

• Among the most risk-averse are technical specialists – mid to late-career experts in a (typically legacy) technology (e.g., “COBOL Jockeys”)

• Architectural change threatens their careers• Solution:

– Work with younger developers to build acceptance for SOA (eventually the TS’s will come around)

– Take a “leave and abstract” approach over “rip and replace”

33


Working with IT Middle Management

• Middle managers threatened by SOA because of the Service domain reorganization

• Solution:– Technical specialties still required– New opportunities for Service domain

management


The “Ivory Tower” Problem

• Architects create design and other artifacts, but don’t have the authority or mandate to require their use

• Development team considers them optional

• Business likes idea of architecture in principle, but short-term needs trump best practices

• When architects are external consultants, the “not invented here” syndrome makes the Ivory Tower worse

34


The Power of the SOA Center of Excellence

• SOA experts who maintain a knowledge base of best practices– General and company-specific– Design time and runtime

• Drives SOA policy (either explicitly or implicitly)

• Can unify approaches across a large organization


Case Studies

35


Telco Example

Verizon

Large regional telecommunications provider formed thru the merger of Bell Atlantic and GTE

• Eliminated redundant systems inherited from the merger of Bell Atlantic and GTE

• On average, each transaction had been developed five to 25 times; one was deployed 45 different times

Challenge

• 2.5 million to 3 million Web Services transactions a day

• Helped Verizon slash its IT budget by 50 percent

• Included managing and securing the Services, charging for reuse and monitoring the performance of Service-enabled transactions

Results/Benefits

• IT Workbench SOA project, operational in 2004

• Thousands of developers, .NET and Java

• Focused on 250 business transactions – incl. verifying customer credit histories & looking up customer info

• 57 Service-oriented applications with 200 transactions

Solution


Government Example

The Defense Finance & Accounting Service (DFAS)US Federal Government agency responsible for all Department of Defense accounting

• Very large, complex organization that handles accounting for Army, Navy, Air Force & Marines

• Process $1 Billion dollars of payments per day

• Challenge of how to bring together DoD finances

• “Analysis Paralysis” – too much time devoted to design

Challenge

• SOA effective in environments of extreme complexity

• Information architecture essential to resolve semantic issues in complex environments

• SOA appropriate in environments where there are many stakeholders with many needs

Results/Benefits

• Reduced hardware costs by streamlining operations

• Achieved “semantic alignment” across organizations

• Collated and rolled up information in the face of enormous complexity

• Focused on information architecture in the context of SOA

Solution

36


Energy Company Example

Large Global Petroleum Firm

• Different divisions had different concepts of SOA

• No unified approach to Enterprise Architecture (EA)

• Lines of business often had very little in common

Challenge

• Moved toward establishing enterprise-wide EA committee

• Undertaking SOA pilot projects

Results/Benefits

• Brought together EAs from across company for first time in 6 years

• Worked to develop a common vocabulary

• Identified areas of redundancy suitable for shared Services

• Developed a SOA maturity model

Solution


Read the Book!

• Service Orient or Be Doomed! How Service Orientation will Change your Business

• Published by Wiley

• AVAILABLE EVERYWHERE!