cbdijournaleverware-cbdi.com/public/downloads/8h4qm/journal2005-10.pdfa preview of the windows...

CBDIJournal3 Editorial

Addressing Architectural Complexity

5 Best Practice ReportDeveloping the Architectural Framework for SOA

–Part 5–The Business Service ConsoleThe “business service console” by itself is a rich client that uses a set of shared and managed services provided by the enterprise system. This article examines these shared services, and describes the architecture of a business service console and introduces the concept of IT Resource Planning (IRP). Along the way, we will see how service orientation is much more than merely implementing web services technology, and how it’s

central to the vision described in Part 1 of this series.By Oliver Sims

12 Best Practice Report Service Oriented Business Intelligence

In our June 2003 report on web services and business intelligence, we described the real-time enterprise in terms of a closed loop management control process, supported by web services and SOA. In this report we look further at how SOA affects Business Intelligence. We examine classes of BI services and techniques for managing the granularity and abstraction of data to support the issues of complex and unstructured

information needs.By Richard Veryard

21 Product Report Windows Goes With the Flow

A preview of the Windows Workflow Foundation. Announced at the Microsoft Professional Developers Conference (Los Angeles, September 2005) this is a platform for automating business processes in a service-oriented fashion that is extensible and embeddable. We anticipate the independent software vendors will invent novel workflow-based offerings for service assembly.

By John Dodd

Independent Insight for Service Oriented Practice

OCTOBER 2005

ISSN 1745–1884

Editorial

www.cbdiforum.com

SERVICE ENGINEERING

A CBDI FORUM MEETING

5th and 6th December 2005Radisson Edwardian Hotel, Heathrow, London

Meet the SOA experts. Share experiences with peers.A unique opportunity to explore in-depth issues around strategy, process and

techniques for implementing real world SOA.

• Independent guidance and advice from independent experts and practitioners• No hype or marketing presentations• In-depth open forum sessions • Opportunity for off-line advice and guidance meetings • Discounts for multiple delegates from one organization• Shared attendance with colleagues encouraged• Book early and bring a colleague and save 20%

BOOK NOW AThttp://www.cbdiforum.com/public/events/service_engineering.php3

SERVICE ENGINEERING

Background

There’s more to SOA than Web Services and an ESB. Just like every new silver bullet, the reality is much more complex. As enterprises experiment with and pilot SOA they fi nd that the real adaptability benefi ts are elusive, and that the hype around SOA has created exaggerated expectations. Further as they move into production scale they realize that the vendor hype has run far ahead of their platform and tool capabilities.

Yet SOA is a realistic and practical approach. What’s needed is a structured framework and process which guides both adoption and deployment that ensures success at all stages from early learning to maturity.

CBDI has been developing best practices in this area for over seven years. They have guided and worked with leading edge users and refi ned their approach and are documenting what they now refer to as a set of integrated methodologies – Service Engineering.

Meeting Objectives

This meeting presents a unique opportunity to dialog in-depth with expert analysts and practitioners who have been developing the thinking underlying service orientation, SOA and CBD.

Each of the sessions will provide expert level insight and allow ample time for exploration of the issues. There are no vendor speakers. This is an opportunity to interact at a practice level with experts and peers without feeling you are being sold to!

Speakers are all CBDI analysts who have been developing practices and guiding enterprises and can comment in depth on practical experiences from many environments and situations.

The meeting is open to Forum members and non members and proceedings will be available to attendees only.

Editorial

www.cbdiforum.com

Addressing Architectural Complexity

In Michael Crichton’s novel Timeline1 he tells an entertaining story based on the theory that quantum mechanics allows interaction between particles even though they are a million miles apart with no visible connection between them. The basic theory is that our universe – the universe we see, the universe

of rocks and trees and people and galaxies out in space – was just one of an infinite number of universes existing side by side. Each of these universes was constantly splitting, so there was a universe where Hitler lost the war, and another where he won, and another where Kennedy died and another where he lived. And so forth, on and on.

My maths is certainly not up to quantum theory, but it does occur to me that there are parallel universes at work. In the first universe the vendor community creates the usual noise and excitement by talking up technology issues, with constant streams of press releases and events intended to convince us that technology acquisition is a foundation stone of any SOA strategy. In the second universe there are hard decisions to make about practical enterprise problems, many, if not most of which have been created by inappropriate technology acquisitions! In this second universe the utopian SOA vision painted by marketing folk is clouded by the reality of multiple sub-optimal decisions taken over time that have

created layers of complexity that need to be addressed.

The simple view of SOA is that everything is published as a service. This allows us to reuse the underlying capability without worrying about what goes on under the covers. But in the real world there’s many problems that won’t go away with a wave of the magic wand. In many situations the “mid tier” is itself a façade which provides some orchestration of multiple core systems. Putting composite applications on top of the underlying mess is usually

increasing complexity and creating an even greater maintenance nightmare.

Of course exposing business abstraction services is important and useful. In this month’s CBDI Journal we look at Microsoft’s Window’s Workflow Foundation and we fully expect this will enable improved

SOA technologies must be regarded as a SOA technologies must be regarded as a SOA technologies must be regarded as a SOA technologies must be regarded as a SOA technologies must be regarded as a SOA technologies must be regarded as a

component of a broader architecture. Our reasoning component of a broader architecture. Our reasoning component of a broader architecture. Our reasoning component of a broader architecture. Our reasoning component of a broader architecture. Our reasoning component of a broader architecture. Our reasoning

behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going behind this position is that SOA is inevitably going

to stimulate enterprise specific decisions which to stimulate enterprise specific decisions which to stimulate enterprise specific decisions which to stimulate enterprise specific decisions which to stimulate enterprise specific decisions which

address the core service implementation layer. address the core service implementation layer. address the core service implementation layer. address the core service implementation layer.

1Timeline, Random House, 1999

4 CBDI Journal © CBDI Forum Limited, October2005

Editorial continued . . . Best Practice Report

CBDI Journal © CBDI Forum Limited, October 2005 5

flexibility and response time for solution assembly. The problem is that if we haven’t solved the issues under the covers we may find that changing a core service’s specification or service level is going to become very expensive if not impossible.

We have consistently argued that SOA technologies must be regarded as a component of a broader architecture. Our reasoning behind this position is that SOA is inevitably going to stimulate enterprise specific decisions which address the core service implementation layer. I was interested to meet with IBM’s ESB architects recently, and encouraged to hear their opinion that the ESB is simply a layer of a much bigger service oriented architecture, in which all application and infrastructure resources are managed as services. I think it’s very important at this stage to have this vision in mind because it encourages decision making in context with a longer term transformation roadmap.

This month Oliver Sims continues his series on the architectural framework, and introduces the concept of IT Resource Planning. Oliver compares core

business operations and ERP to the task of managing the complexity of IT functions and services and looks forward to the time when we will have a similar level of operational support for life cycle IT processes.

Today there is considerable development work happening in the area of IT infrastructure, which is focusing on process definition and automation and certainly provides some of the underlying infrastructure for IRP. In his article Oliver provides an integrated view of the Service Architecture that will allow IT resources to be manipulated through a service console.

The basis for Crichton’s novel was that the multiple universes are infinite in number and exist at all earlier times, which allowed a form of time travel. The novel concentrates on traveling backwards in time, but for our purposes it might be more useful to think about future states and to develop an architectural roadmap that manages the progressive transition towards the endpoint vision.

David Sprott, CBDI October 2005

DID YOU SEE? SOA FUNDAMENTALS

An understanding of the fundaments of SOA is essential. CBDI suggests the following sequence of reports as a core basis for understanding and adopting SOA that should be read by all those involved from managers to developers, both in IT and those in the business such as business analysts responsible for communicating requirements and ensuring IT support to their business domain.

The following individual reports are available to Silver and Gold members. They have also been consolidated into a single PDF that can be downloaded by gold members at http://www.cbdiforum.com/secure/SOA_Fundamentals.php

Understanding SOA

Principles of Service Orientation

The Business Case for SOA

Towards the Service Oriented Organization

Enterprise Framework for SOA

Establishing a Service Lifecycle

Service Supply – A Fundamental Shift in Development Practice?

Business Modeling for SOA

Composite Applications


Editorial continued . . . Best Practice Report


1As described in the previous article, the “bottom” is where a component implementation does not use the services of any other lower-level component.

Developing the Architectural Framework for SOAPart 5 – The Business Service Console

The “business service console” by itself is a rich client that uses a set of shared and managed services provided by the enterprise system. This article examines these shared services, and describes the architecture of a business service console and introduces the concept of IT Resource Planning (IRP). Along the way, we will see how service orientation is much more than merely implementing web services technology, and how it’s central to the vision described in Part 1 of this series.

IntroductionIn today’s world, services are often merely an interface to existing applications. They are certainly useful, and deliver value, but they are still merely an appendage to current monolithic application structures. Indeed, some see service orientation to be all about the use of web services technology if you’re using the technology, you’re doing service orientation. However, this is to ignore the great promise of service orientation. CBDI sees service orientation as much more. Service orientation is a whole way of thinking about both function and structure and what they are for. It is a way of thinking about how function is made available in properly-sized chunks with minimal side-effects and dependencies so that they can be exposed as services that can be composed to provide new function.

There is also a school of thought that contends that services are a single layer at the outside of a set of core business systems, and that service orientation applies to no more than that. We disagree. We strongly believe that effective service orientation requires services to be recursive; that is, the implementation of one service makes use of other finer-grained services, which in turn make use of even finer-grained services, until the bottom of the service hierarchy is reached.1 Hence all business function is provided by services, and each service not only provides an interface but also has an implementation of that interface. This implementation is itself a composition of services.

But we can apply this thinking not only to applications provided for users outside IT (such as CRM, ERP, etc.). We can also apply it within IT and within the wider development community that uses such capabilities as BPM tools to define their own business processes (which are then acceptance-tested and deployed by IT). With a raising of the virtual platform [5] we can expect some business

By Oliver Sims

6 CBDI Journal © CBDI Forum Limited, October2005 CBDI Journal © CBDI Forum Limited, October 2005 7

people, hitherto quite separate from IT, to become involved in defining and testing business function. The first article in this series [1] described near-future scenarios where this occurred, and subsequent articles described how this could work.

But the scenarios describe something quite different from today’s development practices: they describe an environment in which life cycle assets are actively managed and used to support specific service capabilities. The assets created and maintained are service and other specifications, along with their metadata. Metadata describes and permits modification to the usage of a service, which allows a 360 degree view of the status of the service, and a wide range of useful new services both directly operating on service status, as well as composing other services, so offering yet further services.

This environment is presented to users through a Business Services Console (BSC) technically a rich client using services to aid in the business of development (as well as the usual application services such as Order Entry). The BSC presents users with an environment where the business is in direct control of significant parts of their IT, and where the IT organization itself becomes an indispensable, fast-reacting, often proactive, provider of run-time services.

The environment described in the Part 1 scenarios depends on an effective virtual platform whose run-time part includes an ESB, on a solution to the business/IT gap problem, on the kind of architectures addressed in the pages of CBDI Journal over the past several years, and on the capabilities described in the last three articles in this series.

This article focuses on how the development services offered through a business services console are provided.

Behind the BSCAn important motivation for IT is to remove the “Business/IT Gap”, which so often bedevils development projects. But we can’t remove the business/IT gap without giving business people a direct view of the system. Our scenarios in Part 1 showed people using a BSC that provides such capabilities as:

• A user view of business with drill-down to the relevant application modules (components)

• The ability to define new business processes

• An ability to test new processes (the equivalent of unit test – an IT professional still does acceptance testing)

• Assessing the likely impact of a proposed change (see [2])

• A common service-oriented view of both the business and the corresponding IT components that implement it (see Business/IT Bridge 2003).

• Alerts (In one of our Part 1 scenarios, Fred Kozsinsky, process deployment analyst, saw an alert generated when Elaine started to define new process).

But what’s behind this capability? How does it work? Well, there are four essential factors that enable a BSC capability:

1. Architecture-driven development

2. Active model-driven development

3. A ubiquitous event management or notification service

4. An IT Resource Planning and Execution application (IRP).

CBDI Forum has discussed the first of these in depth over the past few years, where the architecture is based on the kind of SOA championed by CBDI Forum and others. The second is being aggressively developed by IBM, Microsoft, and others, while standardization is being addressed by for example the OMG. Within

the timeframe of our near-future scenarios, we believe that model-driven development will have become mainstream. The need for the third factor – a cross-enterprise notification or event service – has been discussed before in CBDI Forum reports [4].

In this report, we focus on the fourth factor – IT Resource Planning and Execution, or IRP. But what is IRP? Well, a good analogy is this: if (say) the supply chain management parts of a business use ERP to plan, manage, and execute their functions and processes, then why can IT not use something similar – IRP – to plan, manage, and execute its own functions and processes? And just as ERP depends crucially on a set of managed databases, so IRP has its own core database.

the timeframe of our near-

If the supply chain management parts of a business use ERP to plan, manage, and

execute their functions and processes, then why does IT not use something similar – an IRP, to plan, manage, and execute its own

functions and processes?

Developing the Architectural Framework for SOA continued . . .

6 CBDI Journal © CBDI Forum Limited, October2005 CBDI Journal © CBDI Forum Limited, October 2005 7

Figure 1 shows a logical view of what the heart of an IRP could look like. Most important is the IRP database. This contains:

• Models that comprise the set of information describing IT goals, plans (application portfolio), developments, (e.g. components, process definitions), and operational systems (e.g. live service compositions or “applications”). It contains “models” in much the same sense that an Inventory database contains a “model” of the real items stored in a warehouse.2

• Metadata whereby models can be queried, created, and modified through relatively simple but “agile” [6] service interfaces3.

• Metadata to describe business concepts (an ontology) as well as configuration parameters.

• A “registry” of services

Today, suppliers are moving in this direction. An example is HP’s OpenView [3] which primarily address runtime systems – the portfolio of deployed and operational applications.

At the bottom of Figure 1 is shown the life-cycle compressed into plan-develop-run. The feedback loop from live running (Operational Systems) to Planning represents all input to planning, from whatever source. “Planning” here represents the wide variety of different plans, from a small project plans to strategic portfolio planning.

2However, the wonderful thing about IT is that the models, unlike the records of inventory, can in some cases be the things themselves! For example, a BPM definition is a model of a process that can also – at least in principle – be executed at run-time. 3The OMG’s standard for repository interfaces has “agile” properties – see http://www.omg.org/technology/documents/formal/mof.htm.

Figure 1: The IT Resource Planning “Core”




Note that a real IRP Core will almost certainly be a federation, with more services than are shown (for example, the IT Assets Service may well be a number of asset services, each managing a set of similar asset types). However, for present purposes we make the simplifying assumption of its being a single service assembly with a single database.

Not shown in the diagram is one of the essential IT assets held in the IRP-DB: that is, the set of development metrics. CBSE provides an excellent structure for collecting metrics; we assume that this or something similar is being used.

Within the “IRP Core” are a set of suggested base services:

• IT Assets Service – an “entity” service that understands the structure of the repository database and provides CRUD and validation access. This service is likely to be metadata-driven.

• Planning, MDD (Model-Driven Development), and Active Apps services – services provided to support procedures and processes to do with planning, development and run-time. The implementations of these services use the IT Assets Service.

• Note that MDD is essential: an environment where models are just documentation won’t work.

• Event Management Service – represents an enterprise-wide generic function, not just an IRP-only function.

• Traceability Service – A key service that allows other services outside the IRP Core to find out how various assets are linked, or are dependent on each other. For example, if you wanted to know which currently-running service assemblies and services deliver against a particular business goal, then the Traceability Service would provide the dependency path (and nodes on that path) between the goals and the relevant components currently running on the live systems.

• ESOA (Enterprise Service-Oriented Architecture) – This is not a service. It is the conceptual, structural, and behavioral foundation for an IRP. It defines how services are assembled, dependency management patterns, the structure of assets in the IRP-BD, etc. etc. A mature ESOA is absolutely fundamental to an IRP – it is very difficult to see how an IRP could be created without it.

In summary, the IRP Core provides the ability to find out what’s going on (traceability), then modify it (development), put the modification into operation (deployment), and collect feedback on the live systems for input back to Planning.

Given an IRP Core, what capabilities can we provide “on top” of it, so as to implement a full IRP?

IRP CapabilitiesThe IRP Core is the basis on which we can build higher-level services (which will compose the Core services as art of their implementation). Figure 2 shows the IRP consisting of the IRP core, and the IRP services that use those of the IRP Core. IRP services are made available to authorized requestors – such as a rich client within the enterprise. Based on our scenarios in Part 1, the IRP services include:

• Impact Assessment Service For example, suppose planners/designers want to provide a new service composition. This may need new lower-level services to be provisioned, and/or existing services to be modified4. An Impact Assessment Service, using the metrics stored in the IRP-DB, can quickly provide an estimate of the time and cost required (this can apply to outsourced work as well as in-house).

• Business Alerts Service The Notification Service allows for business people to express interest in the occurrence of a specific business event (e.g. the arrival of a particular sales order) and to be informed when that event occurs. The IRP would use this service so that an event such as a business person starting work on a process definition could be brought to the attention of the IT person responsible for that business area.5

• Other examples include being informed when there’s a change in some data flow such as, in a fashion-sensitive business – when the ordering rate on some set of items change markedly, showing a change in demand for that fashion.6

• Activity Monitoring Service – The representation of running systems in the IRP-DB, together with real-time links to the real service assemblies, can provide the basis for an activity monitoring service. People could see what processes are currently running, and what resources in a server farm are being consumed.

4This is precisely where “agile services” [6] come in handy!5A number of applications provide an event notification function by implementing their own polling mechanism. But polling is a pernicious solution at best and the thought of multiple applications all doing their own polling is the stuff of IT nightmares!6Some of this kind of capability could be implemented by probes into the message flow (remember that web services carry metadata within messages). So web services technology plus ESB plus IRP effectively provides a view of the “nervous system” of the enterprise. Of course, a business ontology would be a pre-requisite – easy to say, difficult but not impossible to do.




• Process Monitoring Service – Based on Activity Monitoring, a person could see a process being executed. In effect, a Process Monitoring Service could provide the business with a real-time view of its activities.

• Process Simulation Service – This enables a process designer to see results of a change to a service implementation. Although very similar to the Process Monitoring Service, this service would allow the user to stop and start a process in order to look at intermediate steps. In essence, the development lifecycle for business processes becomes a small loop: design, deploy (to a test system of course), observe/measure, revise, deploy, etc.

• 360° View Service – The Traceability Service provides the foundation for a higher level “360° View”, from business goals through processes to hardware resources – and back. Today, while linkages from process to hardware can be made, they are often either at too gross a level, or provide a miasma of thousands of links. CBSE-driven modularization and granularities (of service implementations) makes for a set of links that are far more usable.

For professional IT people, the IRP provides the ability to manipulate IT resources in terms of their specification. The

second scenario in Part 1 of this series described precisely this – a modification to components within the core business system. In future, depending on advances in DSLs, users with appropriate aptitude may be able to do some of these kinds of modification themselves. The essential foundation is an architecture that defines patterns and constraints for such things as a pre-defined technical structure for a service implementation (component), and constraints on interactions between services for dependency management reasons.

Other services mentioned in Part 1 can also be provided. Some of these are: Business Definition Service (allows business people to define and test new business processes), a System Status Service (provides a filtered view of live system activity – and of development activity), and a Planning Service that would support the planning activities.

Because of the importance of the “data” in the IRP-DB, it is possible that an IRP may need an organization whereby a given manager is responsible for a specific set of assets throughout the life cycle, from requirements through development and live running to sunsetting.

Finally, the real value of an IRP is the ability to modify both specification and resources within a very short time (compared to today’s non-IRP development processes and tools). That is, there is the potential for a huge reduction in “time to market”. An IRP actively supports the idea

Figure 2: The BSC Architecture (including ESOA)




7Some UI infrastructure implementations make it difficult physically to separate user and workspace tiers, but it is still very useful for at least their overall design to be done in terms of this separation.

of constant evolution rather than build and then apply maintenance. What the IRP could do is essentially to apply the office revolution (remember secretaries?) to business systems evolution.

The title of this article is “Business Service Console” and now, having described some of the services available from an IRP, we turn to how the user perceives and makes use of these services.

The BSC ArchitectureESOA best practice clearly separates services provided by a shared enterprise system from function dedicated to user interaction. Thus we can think of the BSC as having three important areas of architectural concern:

• Presentation and Interaction (“user tier”)

• Session or Workspace (“workspace tier”)

• User Persistence (“user resource tier”).

These tiers7 are shown in Figure 3, as is the IRP and also other business systems whose services the user may need. While the user tier will normally be implemented physically close to the screen and interaction devices, the workspace and user resource tiers could be implemented on a local server. However, with increasing infrastructure support for rich clients, The BSC itself will probably be a rich client. The “rich client” approach will typically have all three architectural tiers implemented on a PC:

• User Tier: Deals with presentation (e.g. graphical screens) and interaction (keyboard, mouse, etc. Uses the services of the workspace tier.

• Workspace Tier: Handles “business logic” to do both with what the user is doing and also with issues arising when the user chooses to run multiple concurrent “sessions”, makes user-specific data persistent based on a user request to do so (or for example when the user

Figure 3: The BSC Architecture




powers down at 5 o’clock when half-way through some work that cannot yet be submitted to the enterprise tier). Uses services provided by enterprise tiers logically located on the shared enterprise system, and also uses the services of the user resource tier.

• User Resource Tier: When requested, this tier makes user session-related data persistent (perhaps, for example, on a LAN server through a mapped network drive).

Notice that service orientation is also a very useful concept in the user interaction area. This does not necessarily imply the use of web services (although, given appropriate PC-resident infrastructure, web services technology could be used). For example, the user persistence tier could be provided “under the covers” through persistent objects used within the workspace tier.

Finally, it is worth mentioning that, no matter how good the services provided by the IRP are, if their potential value is not realized because of a poor design and/or implementation of the BSC, then much of the value could be dissipated.

Back to the PresentIn the previous discussion, a number of assumptions have been made but have not been mentioned. These include a project management architecture for IT development, tied in to the SOA service provisioning architecture; implementation of the business-IT bridge, and the availability of DSLs.

It is clear that much must happen before the kind of BSC described here can be a reality. However, a capability along the lines of the BSC is arguably a useful vision, as it combines tools, technology, architecture, and processes into a single system-wide view of how it could be. And a very realistic first step towards this vision – which brings its own value regardless of a BSC – is a move to SOA. Not

just the web services technology, but SOA as a ubiquitous capability that can enable effective service composition, and the resulting IT agility and responsiveness that can be gained.

It’s also worth pointing out that out-sourced systems can also benefit if when commissioned they are required to be not only service-oriented but also modularized to enable effective service composition. This probably means that the requirements for an outsourced system should include a specification of the services, including a black-box definition of component responsibilities, so that their granularity maps to the business.

In addition, the assessment of future packaged applications can benefit from establishing whether their designers have applied ESOA design principles, so that they can provide composable services.

The next and final part of this series will suggest and architectural strategy that can enable the ESOA vision, and will discuss a process for making the journey.

References[1] O. Sims, CBDI Journal May 2005, “Service-Oriented Architecture Part 1 – The Vision”.

[2] O. Sims, CBDI Journal September 2005, “Service-Oriented Architecture Part 4 – Business Services”.

[3] See http://www.managementsoftware.hp.com/.

[4] O. Sims, CBDI Journal June 2005, “Developing the Architectural Framework for SOA Part 2 – Service Granularity and Dependency Management”.

[5] O. Sims, CBDI Journal June 2003, “Service-Oriented Architecture – Part 4 – The Platform”.

[6] O. Sims, CBDI Journal July/August 2005, “Service-Oriented Architecture Part 3 – Agile Services”.

12 CBDI Journal © CBDI Forum Limited, October 2005

Best Practice Report

By Richard Veryard


Service-Oriented Business Intelligence

In our June 2003 report on web services and business intelligence, we described the real-time enterprise in terms of a closed loop management control process, supported by web services and SOA. In this report we look further at how SOA affects Business Intelligence. We examine classes of BI services and techniques for managing the granularity and abstraction of data to support the issues of complex and unstructured information needs.

IntroductionIn general terms, management information (whether in the form of a traditional report or as an adhoc enquiry) can be conceptualized as a structured message from a computer system to a human agent, or to a community of agents; and we generally find it useful to model a larger human-computer system that includes the use of this report as well as its production. (There is a strong thread within SOA that deals with message-oriented systems – and we can apply this thinking not just to purely software systems but also to business organizations as systems.) A business case for moving to more real-time monitoring and control can generally be based on an analysis of the management control loop and its overall effectiveness.

From Information Needs to Abstract ServicesWhat is the granularity and abstraction level at which we define information services to satisfy management information needs?

In terms of the core principles of SOA, it generally doesn’t make sense just to equate individual reports with individual services. Even in legacy systems, few companies have a simple one-to-one mapping between individual reports and report-writing programs. In the past, many SMEs have used generalized report-writer packages that define the output format independently (or at least semi-independently) from the input format. And large global companies have often written highly complex applications for generating large numbers of reports from a vast array of heterogeneous source databases. Thus there is often already some layering / decoupling / abstraction in the legacy architecture, which we can regard as a step in the direction of SOA.

But from an SOA point of view, whatever layering / decoupling / abstraction there is in the typical legacy report-writing architecture is significantly constrained, and unlikely to be capable of accommodating the complexities faced by today’s adaptive enterprise. This almost certainly means more work for somebody and/or more problems for everybody. So there may be a real benefit (reduced cost, reduced risk) in taking further steps towards SOA even in the production of traditional tabular reports.

12 CBDI Journal © CBDI Forum Limited, October 2005

Best Practice Report

By Richard Veryard


ComplexityEven a traditional reporting function is now faced with increased complexity and volatility, and this seriously affects its ability to contain costs and maintain quality. We need to define services (within service layers) that manage this complexity/volatility more effectively and efficiently, which might therefore respond with greater efficiency and effectiveness to evolving reporting requirements.

We can classify this complexity into three main types, as follows:

1) Increased complexity of input. Increasing distribution, diversity and volatility in the sources of data from which the report is compiled, which entails a perpetual struggle (for IT and/or users) to maintain the consistency and completeness of reports from one period to the next. Changes in corporate structure and boundaries (including M&A and outsourcing) that complicate like-for-like comparison. Risk of poor data quality.

2) Increasing complexity of output. Demands for new and more detailed reports, aggregating and analyzing data in new ways to satisfy new management, governance and regulatory demands. Demand for visibility across the extended enterprise. Sarbanes-Oxley, Basle II, etc, etc. Stringent quality requirements: severe penalties for inaccuracy. Risk of escalating effort in report production.

3) Increasing complexity of use. Many reports need to be manually adjusted between production and publication, because of known inaccuracies or gaps in the computer records on which they are based. Reports are not just printed on paper, but are transmitted as electronic documents, scanned or parsed, and fed into (say) Excel for further consolidation/comparison/analysis. Many companies are now rendering traditional financial reports in XBRL or similar formats, to support a greater range of use-contexts. (But among other things, this means it is increasingly easy for financial journalists to uncover any anomalies in your accounts.) Risk of releasing information that may be used in inappropriate or misleading ways.

An SOA perspective is very helpful in thinking about these three types of complexity, because they can all be expressed and modeled in terms of the decomposition and recomposition of services, at different levels of granularity.

Five Styles of Business IntelligenceThe BI vendor Microstrategy has identified five types of business intelligence, which are (not surprisingly) covered by its own BI tools. These are differentiated in two ways: firstly by the number and range of users, and secondly by the degree of analytical sophistication and user interactivity. According to Microstrategy, these two are inversely proportional – thus the greater the number and range of users, the lower the sophistication and interactivity. Microstrategy’s five types of BI can therefore be arranged in a 2 2 matrix – as shown in Table 1 – with the top right quadrant empty.

This reflects a common attitude about BI – that because the complex stuff is hard to use, and costly to service, therefore it is only going to be used by a small number of highly skilled users working on stand-alone problems. One of the opportunities of SOA is that it potentially allows us to expose BI services to a much wider user community, and to embed BI services into composite end-user services. The challenge is to produce semantic models, at suitable levels of aggregation/granularity, that support interoperability and composition.

Traditional Enterprise ReportingAs we have already seen, traditional enterprise reporting is faced with:

• increased complexity of input.

• increasing complexity of output.

• increasing complexity of use.

Anal

ytic

al s

ophi

stic

atio

n an

d us

er in

tera

ctiv

ity

High

Statistical Analysis and Data Mining

Adhoc Query and Analysis

Cube Analysis

Low Enterprise Reporting Alerting and Report Delivery

Low High

Number and range of users

Table 1: Five Types of Business Intelligence (based on Microstrategy)


Developing the Architectural Framework for SOA continued . . .Service Oriented Business Intelligence continued . . .


SOA can therefore be used

• to decouple the output (formats) from the input (formats)

• to encapsulate the method for calculating the output from the input

• to deliver the results in a flexible way to a broader community of users

We can define a standard set of SOA architectural patterns for data extraction, composition and delivery.

Report Delivery and AlertingReport delivery and alerting contributes to a series of event-driven systems.

SOA can be used to design the performance of the whole monitoring and control system. We can use a standard set of patterns for this.

• we need to define the range of possible responses at appropriate levels of granularity/specificity.

• we need to decouple the events to allow for meaningful and independent action by different recipients.

• and/or we need to coordinate multiple responses to complex events.

Complex Analytics – Towards the Knowledge GridAt the present state-of-the-art, tools for complex analytics are used by a small community of specialist users, typically working on individual isolated problems. One significant challenge here is to achieve collaborative synergies between specialist users, to enable them to work effectively together on larger and more complex business problems.

We use SOA concepts to design data discovery and analysis processes with appropriate levels of semantic interoperability.

There are some known organizational / management patterns for collaborative knowledge work, but these are not yet fully represented in technological / architectural patterns.

Knowledge-Based ServicesTraditional BI tools offer nothing in the fourth quadrant. Why is this?

• Because there is no requirement?

• Because the technology has not been capable of providing adequate performance for a large user community?

• Because the available tools are difficult to use and require specialist training?

SOA provides opportunities to expose increasingly complex data analytics to an increasingly diverse community of users, including customers.

We can use business patterns to identify knowledge-based services with high potential added-value.

Analysis and Design ApproachA traditional Information Engineering approach seems okay for producing a set of information needs for the

Enterprise Reporting style of BI. If we extend this approach to include process and data models of the extended enterprise (e.g. supply chain/Kanban1), then it may also be suitable for the Event-Driven and Alerting style of BI. One of the icons of Information Engineering was the MIS pyramid, which was often interpreted as saying that the information needs of higher management could be constructed as simple aggregations of

operational data. This is only plausible if we restrict ourselves to low complexity enquiries (Enterprise Reporting, Alerting and Report Delivery).

But it is the remaining three styles of BI that are more interesting from an SOA perspective. A traditional information needs analysis is not going to help much here. As indicated above, we have to address the problem at a different level of abstraction, bringing to bear some of the “software factory” thinking.2

What are the factors that cause fluctuation in the market demand for this item? What are the technical drivers for customer purchasing decisions? What are the root causes of this observed variation in this manufacturing process? What are the effective differences between version i and version j of this manufacturing process? How sensitive is

What is less obvious is the relevance of SOA to those areas of business intelligence where the business question being

asked is not well defined

1See CBDI Report SOA for Automotive, July/August 2005 http://www.cbdiforum.com/secure/interact/2005 – 07/Busi_Model_SOA_Examples_Auto_Industry.php2See article by Brodie and Moss




this analysis to the frequency with which we record the measurements?

More generally, the primary task of data mining and OLAP can be understood as a series of meta-level enquiries: Which variables should we be tracking, at what level of granularity? Which variables should we be controlling in order to achieve these specified business outcomes? Which groups of variables need to be managed together?

Therefore, enquiries of this kind can be understood in terms of a second-order model, in which the instances correspond to types in the first-order (operational process) model. We need to define the BI requirements for high complexity enquiries in terms of higher-order information needs and higher-order management and control processes. The information objects that emerge from this analysis, such as VARIATION and CORRELATION, become business objects to be managed in their own right. Perhaps not all the composite information objects will be genuine business objects. But there are certainly some that occupy the minds of large numbers of line managers. For example, statistical process control (a widely practiced discipline within the manufacturing sector) is all about managing instances of PROCESS VARIATION.

The point of an SOA is to structure capabilities that allow ease of reuse of standardized functionality. Is there a case for using SOA in this area – for example common sets. In our work with organizations adopting SOA we have established the principle of assembly – collections of parts being assembled to support many different, concurrent business processes. We use this principle to identify similar set of parts that underlie the complex BI questions.

Complex enquiries are assembled from simple enquiries. We can define a hierarchy of reusable composite information objects. Composition involves not just aggregation and statistical processing, but also synchronization. Synchronization is necessary to support like-for-like comparison and analysis. With a traditional data warehouse, this is not an issue because the data warehouse represents a snapshot of the operational database at a particular time. As the enquiries get more complex, the demands of synchronization become more tricky. Analysis is required to define the composite information objects at the most useful (in other words, reusable) level of granularity.

Levels of aggregation cannot be completely predetermined, but a large organization should be able to go some way towards systematizing its planning capability (and perhaps also its strategic capability). The only way to get significant

levels of reuse is to carry out this analysis at a higher level of abstraction, using models of these higher-order information needs and control processes, customized to the specific circumstances of a specific organization.

Information and Meta-InformationThe relevance of SOA to routine management information is fairly easy to perceive. What is perhaps less obvious is the relevance of SOA to those areas of business intelligence where the business question being asked is not well defined, or to those areas where the question is reasonably well defined, but the method for determining the answer is not yet well defined.

If we want users to be able to compose enquiries dynamically, we basically have several choices. One possibility is to provide a general-purpose enquiry language such as SQL, with every enquiry being coded from scratch in SQL. This is highly inefficient and somewhat error-prone, dependent on high levels of user expertise. Much better is to produce a special-purpose enquiry language, with appropriate templates and patterns for complex enquiries within a given domain. We can regard this as a Domain Specific Language (DSL).

For example, it is possible to define a large class of data mining enquiries relating to statistical process control, or customer relationship management, in terms of the desired knowledge management outcomes, and then provide a set of abstract information services that support these outcomes.

One of the problems of data mining is that you have to find ways of linking different levels of abstraction within a single enquiry. For example: “which context variables are statistically correlated with these customer behaviors?” – where the object CONTEXT VARIABLE appears to belong to the metamodel rather than the model. Obviously there are ways of collapsing meta-level into the model itself, but these are fraught with danger for the logically naive: resulting in extremely clumsy class diagrams, and in error-prone or grossly inefficient implementations. In principle it should be possible to define a DSL that can be used to express specific data mining enquiries (e.g. for manufacturing or statistical process control) in a way that is safe (or at least safer) – reducing the potential for logical error and inefficiency, and increasing the potential for churning them out quickly.




Three Levels of Management CapabilitySee Table 2 overleaf.

Three Types of BI ServicesThese models lead to the identification of at least three types of business service, as shown in Figure 1.

1. business management services that correspond to the higher level capabilities (Command&Control and Strategic)

2. information services that establish mappings between first-order information (BI style 1 and 5) and second-order information (BI style 2, 3 and 4)

3. “factory” services that use second-order analysis to dynamically create first-order change – for example if data mining identifies that customer demand is dependent upon the weather, then

a “factory” service can automatically create a weather-tracking service as a plug-in to existing operational monitoring systems (BI style 1 and 5). In addition, the data mining will drive the instrumentation of the operational systems. This way of integrating BI with the operational systems is particularly attractive for agile security, since it allows emerging security threats to be identified using BI tools and instantly reflected in additional security traps in the operational systems. But it clearly has value for the whole spectrum of “on-demand”.

This implies a layered architecture of BI services that plugs into a general layered architecture.

MIS PyramidMany readers will be familiar with a simple MIS pyramid, where data are summarized upwards from large quantities

Capability Typical Enquiry Solution Options

Execution Capability

Is this customer creditworthy? The traditional solution is to have an attribute called Credit Limit on the customer record, which is updated manually or through some batch process. (Information Engineering would have regarded this as a basic attribute, because the derivation would have been external to the basic processing system.)

The SOA solution involves Embedded BI – invoking some BI functionality in the form of a service call from the transaction system to calculate the appropriate credit limit or policy for a specific customer in real time, based on a 360 degree picture of the customer and his recent behavior.

Command & Control Capability

Are we extending credit to the right customers?

This has traditionally been addressed with Stand-Alone BI – knowledge workers performing sophisticated statistical analyses, using data stores (warehouses, marts, . ..) that are usually decoupled and desynchronized from the operational systems, largely for performance reasons.

The SOA solution includes what people are now calling Integrated BI. The benefits of this include faster and more effective business response to external change, as well as greater consistency and coordination of planning effort across the enterprise. Among other things, this activity generates the mechanisms (policies, formulas or whatever) that are used in answering the first question. Given that these mechanisms are embedded in services, we can therefore regard this activity as a service factory.

Strategic Capability

Should we be extending credit to customers at all?

(This kind of question potentially leads to radical restructuring of the business relationships with customers and/or business partners.)

In the past, this kind of enquiry has been addressed by Stand-Alone BI, together with sophisticated business simulation models.

From an SOA perspective, this calls for Collaborative BI, which orchestrates BI services across a federated management/governance structure.

Table 2: Three Levels of Management Capability




Figure 1: Three Types of BI Services

Figure 2: Extended MIS Pyramid




of operational data to smaller quantities of highly condensed information, used for strategic purposes. The top level of this simple pyramid is sometimes called Strategic.

But of course strategic decisions cannot be made solely on the basis of this summary information. Top management need to combine information from many different sources – external as well as internal.

Figure 2 shows how this simple MIS pyramid can be set against another pyramid based on external data. Often the external data are only available in some summarized form.

The important analytical task is then to create intelligent support for business decisions by combining internal and external data, often in highly complex ways. So the challenging aspects of business modeling is to analyze the derived bits in the middle.

Four Modes of Business IntelligenceIn Table 2 we identified four modes of Business Intelligence, which we now summarize in Table 3, and this implies a roadmap for implementing Service-Oriented Business Intelligence.

From this table, I’ve singled out a couple of specific short-term and medium term opportunities for creating synergies between web services and BI.

Business Opportunity # 1Many organizations such as retail and banking collect large amounts of data about customer transactions, which are available for analysis within the organization. This analysis is used for a range of purposes from marketing and product management to customer relationship management.

But many consumers would like to analyze their own spending. A bank might provide a flat transaction file which you can analyze yourself, but a retail organization probably doesn’t even provide that. But with embedded BI, it becomes possible to provide trend analyses and other complex information services directly to customers. For example, a consumer may wish to compare the amount spent on children’s clothing with a benchmark of other households with the same number/ages of children, or track expenditure trends against various user-defined categories.

Here’s an opportunity for a commercial organization to provide some significant added value to its loyalty

Mode Description Timescale

Stand-Alone BI BI activities are decoupled and desynchronized from the business and from operational systems.

BI activities are performed by independent knowledge workers.

Current BI practice

Embedded BI BI enquiries are wrapped as services and invoked from the operational systems

For example, calculating the appropriate credit limit or policy for a specific customer in real time, based on a 360 degree picture of the customer and his recent behavior.

Includes Business Opportunity # 1 – wrapping BI enquiries as customer-facing services.

Short-term SOA opportunity

Integrated BI BI activities are coordinated with one another, and synchronized with business and operations.

• Faster and more effective business response to external change

• Greater consistency and coordination of planning effort across the enterprise.

Includes Business Opportunity # 2 – BAM (business activity modeling / monitoring / management

Medium-term SOA opportunity

Collaborative BI Orchestrate BI services across a federated management or governance structure Longer-term SOA opportunity

Table 3: Four Modes of Business Intelligence




scheme customers, at relatively little cost. Furthermore, the organization itself may gain valuable insight about its customers (both individually and collectively) from the nature of the analyses that they carry out.

Business Opportunity # 2The second area is BAM – which variously stands for Business Activity Management, Modeling or Monitoring. We are currently seeing initiatives from a range of vendors including Microsoft and Oracle.

This technology starts to introduce new modes of integration into the business process – including data collection (e.g. RFID and web services used to instrument physical processes such as manufacturing and logistics), orchestration and workflow (BizTalk, BPEL), analytics (statistical analysis) and information dissemination (dashboard, portal).

The next step will be to provide web-service-enabled management actions back into the operational business process. There is likely to be some resistance towards doing this for core business processes, at least in the short term, but we expect to see further experimentation in some non-critical business processes.

Summary of ApproachIn general terms, Information Needs Analysis is a very powerful technique because it is non rigorous and business oriented. It can be used to define derived Business Types (one set of information types) that form the basis for answering complex questions. Traditional approaches to Information Needs Analysis (such as Information Engineering) can be extended by referencing the business process world in richer and more diverse ways, relying on many more sources of analytical input as well as many more ways of constructing composite information objects.

AcknowledgementsThanks to Asoka Diggs (Intel), Duncan Lamb (Craig Lamb & Singletary) and Paul Sanders (Microsoft).

References

General Material

Data Rich, but Information Poor?, by Mike Brodie and Larissa Moss (DM Review Magazine, July 2002)http://www.dmreview.com/article_sub.cfm?articleId=5341

Vendor Material

Microstrategy, Five Types of Business Intelligence.ht tp://www.microstrategy.com/Solutions /5Styles /index.asp

CBDI Material

Web Services to Improve Business Intelligence, by Richard Veryard. CBDI Journal, June 2003.http://www.cbdiforum.com/secure/interact/2003 – 06/bi.php3 See extract next page

Capability Modeling (in preparation)


By John Dodd

Product ReportExtract from CBDI Journal June 2003

Best Practice Report – Web Services To Improve Business Intelligence

By Richard Veryard


By John Dodd

Product Report

1Managed code is an intermediate form of code, broadly corresponding to Java bytecode, which can be executed on a virtual machine termed the Common Language Runtime (CLR). Development language code such as C# , Visual Basic or C++ is compiled into Common Intermediate Language modules, which form the managed code. This approach enables an instance of a class written in one development language to invoke a method of a class written in another language.2This new content had previously been described as the Pillars of Longhorn!3We thought the name WinFX was a neat contraction of Windows Foundation Series. But Brad Abrams tells us the “Win” part of the name “speaks to the Win16 and Win32 heritage “ while the “FX” part “speaks to the .NET Framework heritage”.

Windows Goes With the FlowA preview of the Windows Workflow Foundation

The Windows Workflow Foundation announcement made at the Microsoft Professional Developers Conference (Los Angeles, September 2005) was one that rather intrigued us. This upcoming technology will become an integral part of the Windows Vista platform for developers, and will also be made available for Windows XP and Windows Server 2003. We believe it will be of special interest to CBDI Journal readers too, since it could be used to automate business processes in a service-oriented fashion and is extensible and embeddable, so we anticipate the independent software vendors will invent novel workflow-based offerings for service assembly.

Vista and WinFXMicrosoft recently announced that the next release of Windows for client machines would be known as Windows Vista, and general availability was expected by the back end of 2006.

One of Vista’s new features will be WinFX. This is an extensive collection of managed code1 offering programmable access to wide range of platform capabilities. Developers will be able to call upon these capabilities through a standardized object-oriented API. Table 1 provides a summary of the new content2. WinFX also subsumes a lot of old content too: for example, WinFX will “embrace and extend the .NET framework”, and include “all the namespaces you know and love” – such as Forms and ADO.NET [Abrams].

So WinFX3 is basically a bunch of class libraries, which can be called from many different programming languages through a uniform style of API. Microsoft says its capabilities will be made available on Windows XP and Windows Server 2003, and the next generation Windows Server release (a.k.a. “Longhorn Server”), as


Windows Goes With the Flow continued . . .


well as on Vista (formerly known as “Longhorn”). They have a lot of fun with codenames in Redmond. WinFX promises to offer an easier-to-use interface which exposes both the .NET functionality to developers, and a lot of new stuff which should make many aspects of presentation, communication and data handling much simpler to accomplish.

What is WWF?We shall now home in on the Windows Workflow Foundation (WWF) component of the WinFX family. There are three main parts to this:

• Developer Tools – There is a Workflow Designer that can hosted in Visual Studio 2005. This enables a developer to draw a workflow diagram, and attach properties and events to the depicted activities and flows.

• The Programming Model – A base library or “toolbox” of work flow activities is supplied. These activities can be included in a work flow diagram, and configured according to their role in the workflow. Developers are not limited to the supplied activity types and can invent their own.

• The Workflow Engine – The run-time code that executes an instance of a workflow, performing its activities and following its flows.

Windows Workflow Foundation is currently available in beta form for both the client and server versions of Windows. But before delving any further, it will be useful to understand the nature of WinFX workflows.

WinFX package Formerly known as Purpose

Windows Presentation Foundation

“Avalon” For building applications with rich visualizations, blending together application UI, documents and media content. Includes support for Tablet and other forms of input, a more modern imaging and printing pipeline, accessibility and UI automation infrastructure, data-driven UI and visualization.

Windows Communication Foundation

“Indigo” Aims to simplify the development of “connected systems”, by unifying various distributed systems capabilities into a composable, extensible architecture. This architecture spans transports, security systems, messaging patterns, encodings, network topologies, and hosting models. Built on the .NET Framework.

The Web services support offers secure, reliable, and transacted messaging along with interoperability. It supersedes WSE 3.0.

InfoCard “InfoCard” Provides a consistent user experience for identity management. Specifically designed to protect the end-user’s digital identities against tampering and spoofing, and to provide end-user control.

Win-FS Windows File System Bridges the gap between file systems and databases by providing a unified, rich programming platform for structured, semi-structured and unstructured data. Developers will be able to use WinFS’s unified Item Data Model across applications, present new relational organization constructs for data, and create applications that exploit data more powerfully.

Windows Workflow Foundation Provides functions and tools for building workflow-based applications. Supports both system workflow and human workflow. Consists of an in-process workflow engine, a class library and design tools for Visual Studio 2005.

WinFX Foundation Vista’s ubiquitous underlying class libraries such as System.IO, System.Threading, System Transactions, System.Security and System.Runtime. Some of these will be brand new versions.

Table 1: The newer WinFX capabilities




What is a Workflow?A workflow consists of a set of activities each of which perform some kind of work, and flows which pass control to the next activity that needs performing.

Both program flowcharts and business process (flow) diagrams are variants of such a workflow. Either of these could be drawn using the Workflow Designer supplied by WWF’s, though the notation may not be as rich as we might like. Figure 1 depicts a simple business process in UML notation. The Workflow Designer equivalent appears in Figure 2. This example illustrates several features we would expect the workflow for a business process to support:

• The process is often initiated by a human being (an employee in this example), though it could be a temporal condition or business situation that triggers the process.

• The ability for work to flow in alternative directions, based on a rule. In this example, the rule is very simple: if the expenses claim form is correct, control goes one way; if it is incorrect, it flows in another direction. The correctness checking could be pretty complex however.

• The need to interact with people during the course of the process. In this example, a manager (these are often human) is asked to approve the expenses, so they need to be notified, by an e-mail say, and communicate their response to the system. The analysis of this response results in another alternative route within the process.

• The ability to maintain information about the progress of the workflow. For example, the Sarbanes-Oxley act requires control points within a process, and a record of the checks that have been made needs to be maintained. Where

Figure 1: Business Process Flow Example



human interactions are involved, the process could be left in suspended animation for any length of time, and the process must remember where it has reached. So workflows (and business processes) often need to maintain their own “state” information.

• The workflow will need to communicate with software that lies outside the workflow. In this example, we see that a message is sent to the Payroll System, using a Web service offered by the Payroll system, which might run on a remote site, especially where payroll is outsourced to a specialist company.

• Business processes can also involve parallel processing, though Figure 1 does not illustrate this.

We understand (but have not proved) that WWF can handle all these kinds of requirement. We are pleased to see that the WWF designers have recognized both system workflows and human workflows. The former term refers to workflows that are fully automated, with no human involvement, which is sometimes termed orchestration in the SOA arena. Human workflows involve man-machine interactions during the course of the workflow, as well as fully automated segments of processing. Business processes will typically involve human interactions somewhere along the way.

The Designer supplied with WWF cannot draw the full diagram shown in Figure 1, since WWF is more focused on breaking down the systems work into software activities and connecting these using precise control flows, rather than a wider organizational view with interactions. The workflow diagram must be more exacting, since the workflow engine will execute it at run-time. Take a look at Figure 2 to see how we think the diagram would be drawn in the Workflow Designer. Independent Software Vendors (ISVs) may, of course, create design tools that display standard notations such as those defined by BPMN4 or UML5 activity diagrams, so long as these can be transformed into elements that WWF’s engine can execute.

ActivitiesIn WWF, activities are composite or basic:

• Basic activities are the atomic steps of a workflow. Some of the basic activities are flow controls that force the executing workflow to loop or to branch into parallel or alternative flows. Other activities will only flow in one direction when complete: they simply execute some developer supplied code, invoke a web service, wait for an event, or whatever.

• Composite activities are collections of several activities. That is, they contain their own mini-workflow.

Figure 2: Business Process Flow drawn by WWF Designer

4This is Business Process Modeling Notation, available from http://www.bpmn.org/.5This is the Unified Modeling Language, available from http://www.uml.org/



Windows Workflow Foundation will be shipped with a set of standard activities. These activities appear in the “toolbox” panel of the Workflow Designer (see Figure 3). The developer can drag any such activity onto the “design surface”, to incorporate it into the workflow design and then configure it.

Table 2 provides a summary of the shipped activities which can be included in a sequential work flow. (There can be other types of workflow such as the state machine workflow and we will introduce its activities later on.)

Hosting WorkflowsNeither the workflow nor the workflow engine represents an executable application. Workflows need to be “hosted” in an application. That is, the application has to invoke a workflow engine, which then creates a run-time instance of the workflow and sets it running. The engine manages the workflow’s state throughout its lifecycle. Some of the workflow’s activities may be required to interact with the host application code.

The host application may be any application that can execute managed code. This includes an ASP running under IIS, or a Windows Forms application, or a console application. ISVs are expected to host the workflow engine in their own products, and Microsoft themselves plan to do this. Depending on the requirements of the workflow, the host will be obliged to provide some of the “services” that the workflow needs. For example, the host may need to supply persistency services, to store the state of a long-running transaction within a workflow while it is waiting for an external event.

State MachinesTechnically, an activity is represented by a class. So is a workflow, since a workflow is simply a “root” composite activity. Thus far, we have been assuming the workflow’s root activity is a sequence container, which will then contain a sequence of basic and composite activities (which may be nested to any depth).

Interestingly, another kind of root activity is supplied in the base library, one which supports a state machine. Readers may be familiar with state machines, but not normally perceive them as workflows. At the top level, a state machine work flow consists of several state activities, connected by state transitions. On the Designer surface, the states are boxes, and the arrows are transitions.

The base activity library provides some additional activities, as shown in Table 3, which are only for use in state machine workflows.

Figure 3: The Activity Toolbox of Workflow Designer





Activity Name Description

IfElse Executes one of several alternative container activities, depending which condition is true.

While Repeats the contained activities as long as the given condition remains true.

Sequence A container for a sequentially executed set of activities.

Parallel Executes two or more sequence activities at the same time, and waits until each sequence has been completed before continuing.

Replicator Multiple copies of the same contained activity are executed, the number being decided at runtime.

Conditioned Activity Group A collection of child activities, which may each have an associated condition governing when it executes. This arrangement can be used to create parallel, dependent, and looping workflow behavior, which ceases when the activity group’s UNTIL condition is true.

Code A block of code, written in C# or Visual Basic, which is executed as a part of the workflow. Lives in the in the code-beside file that is complied with workflow.

Listen Waits for one of a set of events, and executes a branch according to which event occurred.

Delay Suspends a branch of the workflow for a specified duration, or until a specific date/time is reached.

Invoke Method Calls a method of the (host) application.

Event Sink Waits for a call from some method that is in the host application.

Invoke Workflow Causes another workflow instance to start executing asynchronously.

Invoke Web Service Sends a message to an operation offered by a Web service.

Web Service Receive (and Response)

These activities are used when a workflow is used to implement an operation of a web service.

Terminate Stops the workflow execution prematurely.

Data Activities Four activities that enable data exchange between the workflow and host application.

Policy Gives access to WWF’s rules engine. WWF provides a base class that is used to build a policy activity that can be added to the toolbox. Requires coding.

Exception Handling ExceptionHandler is a container for one or more exception-handling activities. Within one Handler activity, a Compensation activity could be called, and the Throw activity can be used to raise a new exception or rethrow the original.

Table 2: Base Library Activities, for use in a Sequential workflow




This style of workflow may take some getting used to. Where the steps of a business process can be performed in many different orders, subject to a few rules, then it could be easier to express as a state machine. It could enable more flexible processes, or more flexible designs for user transactions involving multiple screens or web pages. It is also likely that event-driven system work flows, which are common in real-time applications, are usefully expressed as a state machine workflow.

WWF in the MarketplaceWorkflow support within Windows has the potential to deliver considerable savings in development time, since the high-level business view of a business problem can be diagrammed, verified and generated with end-user and business analyst involvement. And it should be possible to make a limited range of application modifications graphically. At least the high-level logic is expressed in pictures: the developers can then focus on the UI design intricacies and business logic exactitudes, using other productivity aids in the Microsoft canon. Workflow offers a useful way to separate concerns, an intuitive diagram for non-developers to draw or review and a useful starting place for system enhancements.

Within the host application, and within the code of workflow activities, it could also be useful to express purely internal logic as a workflow. Furthermore, WWF includes the rules engine that can greatly simplify development where large numbers of business rules need to be applied. But more importantly, it makes it possible to change the current rules without any re-coding.

Because WWF is highly extensible, we anticipate a number of creative ISVs will bring out higher-level tools suitable for analyst/designer use, that offer a wider range of activities for the common information system actions, or workflow starter patterns for common business value chains, and the easier definition and use of business rules. More specialized workflow-based tools may emerge for specific industries. Domains like systems management, project management, task scheduling, wizard design and approvals processes can be made more buyer-customizable when a rules engine and workflow engine are embedded within the tools.

Other ISVs will make use of the workflow and rules engine to build products involving procedural or event-driven work streams, with the buyers being unaware that WWF was used within.

Table 4 lists some of the companies planning to exploit WWF. As you can see, Microsoft intends to eat its own dog food.

In CBDI’s view, workflow support, freely available within the MS platform, has significant potential in the SOA arena. Reuse of common services when combined with workflow tools and high-level user interface design tools would be a formidable combination, making commonly cited goals such as reuse, rapid development, interoperability, customization, virtualization and business change far more plausible than with one of these technologies alone.

Activity Name Description

State This activity can contain an optional State Initialization activity and one or more Event Driven activities6.A state machine always includes an initial state and a completed state, and is expected to own a number of intermediate states.

State Initialization A container for activities which are executed as soon as a transition occurs.

Event Driven (or Event Handler)

A container for activities which are executed only when (a) a specific event/message is received and (b) the workflow is in the state the event handler belongs to.

Set State Changes the state of the workflow. That is, it causes a state transition to occur.

Table 3: Base Library Activities, for use in a State Machine workflow

6Some readers may be more familiar with state machines in which the states are inactive periods, and it’s the transitions which involve activity/actions. It’s the opposite here: states represent (event-driven) activities, and no additional actions are associated with the transitions.




What are WWF’s strengths?It is embedded within the operating system:

• It becomes available within any Windows application that needs it – workflow becomes a standard part of the developer’s toolbox

• Workflow activities are not restricted to single programming language

• A rules engine is included

• API conventions for WWF are the same as for other WinFX components

• Consistent behavior exhibited across all the Windows platforms that support WWF.

Vendor Products Involved

Microsoft Office “12” will ship with its own set of workflow activities and a wizard-driven design tool in FrontPage that will enable power-users to orchestrate a set of pre-deployed activities – for task and document management for example – into workflows that involve human interactions.

The next release of Windows SharePoint Services will host the WWF runtime. The BizTalk Server Adapter for SharePoint will enable BizTalk solutions to interact with document-centric workflows hosted by SharePoint Services.

Eventually, WWF will be “deeply integrated” into BizTalk Server (in the same time frame as Longhorn Server), so as to provide improved support for business process management.

WWF will be employed within future versions of the enterprise applications known as Microsoft Dynamics – for example CRM and Axapta.

Epicor WWF will be embedded In ICE, the Epicor Internet Component Environment.

Siebel Systems, Inc. Mainly known for its CRM and business intelligence offerings, WWF is to be used in Siebel’s next generation of business process management capabilities. The aim is to give business users the ability to edit and modify business operations in real-time.

Fujistu Transaction Solutions Inc.

WWF to be incorporated in the GlobalSTORE point-of-sale product, and other solutions for consumer-transaction environments.

SourceCode Technology Holdings, Inc.

WWF to underlie the next-generation of SourceCode’s K.Net, an “end-to-end enterprise process automation platform”.

Captaris,Inc. Captaris Workflow, formerly Teamplate, is expected to integrate WWF. Captaris specialize in automating workflows involving unstructured documents.

ADERANT Will embed WWF in the underlying technology architecture of its products, which are aimed at the professional services market (e.g. law firms). The aim is to automate paper-based business processes and in so doing reduce their clients’ exposure to risk.

DevelopMentor New training courses on workflow technologies announced at PDC05. WWF is taught within the Essential Windows Workflow course.

Sapient WWF will be used In Sapient projects delivering business process improvements.

Solidsoft Ltd. Specialize in automating business process solutions and delivering integration solutions. Specialists in BizTalk, so WWF will be used in conjunction with BizTalk.

Table 4: Vendors announcing intention to use WWF




Enables ISVs to focus on their own differentiators, since the underlying workflow and rules engines are a given. They have no need to “re-invent the wheel” each time. Workflow can be brought to new markets, in a standardized, reduced-effort way.

It provides a rich collection of activities “out-of-the-box”. Furthermore, it includes support for atomic and long-running transactional behavior, and the synchronization (of data) used by concurrent activities.

It is highly extensible:

• New composite activities can be created from base activities; an activity designer is supplied to help do this.

• Includes a public activity component model that allows new root, composite and basic activities to be developed. The new activities can then be added to the toolbox.

• ISVs, or WinFX customers, can create their own graphical designers, to speed up and specialize workflow definition, the use of the rules engine, the inclusion of proprietary activities and interactions with the host.

• The workflow engine can be embedded in a wide variety of host applications or products, which may need to provide a number of services to the workflow, such as persistency, transactions and tracking.

Workflows can be changed during execution: in a temporary manner on one running instance, or as a permanent change to all new instances of a workflow. Conditions, as used in the IfElse and While activities, if stored in XML in separate files, can be changed at runtime. (These conditions can also be expressed in code, but cannot then be changed at runtime). Partner organizations and other Microsoft teams have been working with the Microsoft developers to validate the features within WWF.

Any worries?• Workflow tools are typically expected to support

to-do lists, understand roles and end-users and know the alternative recipients when someone is absent or overloaded. They should be able to schedule work for each user or organization unit. And provide reports and alerts on progress and throughput . WFF does not cover any of this, and some might argue WWF supports application and program flows rather than real workflows. Of course, the highly customizable nature of WWF

means products which embed WWF can provide these capabilities, but you might have expected to see them among the base activities.

• What about the overlap with BizTalk Server capabilities? [Chappell] proposes BizTalk Server is more appropriate for enterprise application integration problems (EAI), for business to business interactions (B2B), where business activity monitoring (BAM) is a strong requirement, and where BizTalk’s administration and scaling tools are crucial. On the other hand, WWF is more appropriate where human interactions are prevalent, where workflow is needed within an (existing) application and for client-only solutions.

• The relationship between WWF and the WS-BPEL7 standard is unexplained. We anticipate that WS-BPEL support will be implemented using underlying the WWF classes.

• The Workflow Designer supplied for hosting in Visual Studio does not support swim lanes, nor the established business process modeling notations, and it displays some pretty obscure mini-icons.

• Not suitable for use by business analysts or process architects, so does not achieve the holy grail of business process design without programming. But ISVs now have the possibility of supplying this, and Microsoft themselves are showing the way with the use of workflows in Office “12”.

• It is early days. We have yet to find out the degree of effort required to build robust, industrial-strength business processes with WWF. It is not so difficult to generate system workflows. It is process flows that incorporate human interactions which are more demanding.

• It is a tool built by developers for developers, rather than a tool built by developers for end-users, business analysts or solution designers. But that is the point. It is a set of managed code that creative ISVs can exploit. It provides a uniform base for a diverse range of products, including Microsoft’s.

• How well will the Microsoft Modeling Tools hook up with the capabilities of WWF? WWF is being developed by the runtime team. While it is appears the WWF Designer uses the same technology as the Modeling tools (for domain

7OASIS has a technical committee working on Web Services Business Process Execution Language (WS-BPEL) which is a continuation of the work on BPEL4WS originated by Microsoft, IBM and BEA.




specific languages) there is little evidence that workflow design has been integrated into any modeling tools, or has been adequately covered within the Microsoft Solutions Life Cycle8 aimed at Microsoft’s customers.

But don’t forget . . .• Windows Workflow Foundation is not just a set of

managed code like some of the components of WinFX.

• Microsoft is not rushing this. It is working with its ISV partners to remove rough edges and find out what doesn’t work. Slowly but surely, Windows will be including the base functions needed to reengineer business processes using workflow and SOA – within the operating system! On the other hand, the majority of business-facing developers and analysts will need higher level tools, built in-house, or by ISVs or Microsoft, to achieve the productivity potential of workflow support.

ConclusionsOver the years Microsoft, more than any other major vendor save Apple perhaps, has been consistently focused on making complex tasks simpler. WWF is a further demonstration of the outcome of this broad strategy that will potentially have a major impact in customer decisions to adopt the Microsoft platform, particularly in “edge of the enterprise” and composite application developments.

We expect the superficial overlap with BizTalk is merely a timing issue. Once again Microsoft has shown that it really understands the concept of “platform” and uses it to drive competitive advantage. The WWF is clearly designed as a platform for third parties, enterprises and Microsoft itself to develop specific capabilities that relate to their situation – to process characteristics, sector, market, and so on. With its experience in many BizTalk deployments Microsoft is now in an excellent position to generalize workflow capabilities and encourage others to expand its market by creating a wide range of more specialist workflow functionality and applications.

References[Abrams] What is WinFX – The New Programming Interface Introduced in Windows Longhorn, a presentation by Brad Abrams.

[Chappell] Introducing Windows Workflow Foundation: An Early Look, a booklet by David Chappell of David Chappell Associates.

[Andrew et al.] Presenting Windows Workflow Foundation Beta Edition, a book by Paul Andrew and colleagues, published by Sams Publishing 2006.

This and other material is available from http://msdn.microsoft.com/winfx/

I wish to thank Lawrence Wilkes for reviewing this article.

8Which we heard about in the PDC05 Symposium “Connected Business, Connected Systems: Solutions Architecture for a Service-Oriented World.




Service Engineering ServicesIntroducing SOA requires systematic approaches that span technology and application architectures,

processes and organization. CBDI are specialists in this entire topic area and are actively assisting major organizations to understand and embrace this thinking.

CBDI has deep and practical experience gained from our research and numerous consulting engagements and can guide organizations in this complex area.

Service Engineering – An Integrated Set of Methodologies

CBDI provides a comprehensive set of services for managing change including:• Roadmap Planning Workshops• Strategic Planning for Business and IT• Methodology Introduction and Customization• Business Design

Contact CBDI by email at [email protected] by telephone at +353 28 38073

www.cbdiforum.com

Subscribe to the CBDI Forum

The CBDI Journal is published monthly. In addition to the

Journal, subscription includes electronic access to current

and all back numbers that now represent a significant resource

library. There are individual and corporate subscription

schemes. Corporate subscription includes access

to Powerpoint libraries and our workshop materials.

For more details and to subscribe see:

www.cbdiforum.com

CBDI Raison d’etreWe aim to provide unique insight on component and service oriented technologies and processes for the software industry and its customers. To provide high quality analysis and other information resources on best practice in business software creation, reuse and management. To maintain the highest level of independence.

Modus OperandiThe CBDi Forum has a number of channels:

• Subscription services – provision of continuous commentary and information.

• Workshops and seminars – providing indepth briefing and guidance on advanced architectures, processes and practices.

• Consulting – including Adoption Roadmap workshops, methodology customization, architectural guidance and reviews, business design advice, strategy development.

How we compare with othersWe are not a mass market, media oriented organization. All material published by the forum is unique. The majority of material is published by our own analysts, or commissioned from others, and under strict editorial control to ensure accuracy. We rigorously exclude spurious marketing.

We provide depth insight which covers a narrow topic footprint in a deeper way than the other analysts, and in particular cover not just the technology, but also the architectures, usage, practices and processes.

Also we are unusual as analysts; we do not simply echo what the vendors say, we are a think tank, identifying new ideas, opportunities and providing stimulus for thinking. We are thought leaders providing ideas generation and a rich source of conceptual thinking based on practical, real world feedback.

Who Reads the CBDI JournalTechnical leaders including Technical and Application Architects, Business Analysts, Consultants, CTO’s, Designers, Product Strategists, Senior Developers, Project Managers, CIO’s etc. Subscription is split roughly 40% USA and 50% Europe.

Contact UsFor further information on any of our services contact us at: [email protected] or +353 28 38073 (international)

IMPORTANT NOTICE: The information available in CBDI publications and services, irrespective of delivery channel or media is given in good faith and is believed to be reliable. CBDI Forum Limited expressly excludes any representation or warranty (express or implied) about the suitability of materials for any particular purpose and excludes to the fullest extent possible any liability in contract, tort or howsoever for implementation of, or reliance upon, the information provided. All trademarks and copyrights are recognised and acknowledged.

Insight for Web Service &Software Component Practice

cbdijournaleverware-cbdi.com/public/downloads/8h4qm/journal2005-10.pdfa preview of the windows...

Documents