Process Modeling in Web Applications

Stefano CeriDipartimento di Elettronica, Politecnico di Milano

Stefano.ceri@polimi.it

Overture

Talk motivation & outline

Why the topic is important

Web applications are widespread and embody complex processes

Themes: Enterprise Application Integration (EAI), E-business, E-government, B2B.

Technology: Service Oriented Architecture (SOA)

Thousands of: Approaches, Products, “Emerging” Standards (Process Modeling, B2B Protocols, Technologies, Platforms).

“Trendy” research topics

Process Composition Process Verification + AlgebrasSemantic Web Services (automatic selection, composition, semantic annotation / registration / navigation)

The focus of this talk

A high-level, model-driven approach to the specification and design of Web applicationsStarting from high-level requirements (processes)Modeling the “essential” concepts for the Web:

ContentWeb InterfacesWeb ServicesProcesses

Platform independent (supports code generation)

Characterization of Web applicationsas workflow-driven hypertexts

An hypertext with constraints

Stemming from the process definition Spanning over multiple actors

WF-driven

hypertexts

Navigation richness

Co

nst

rain

ts

Pure hypertexts

Pure WFMS

Product selection

Performed by:Customer

Order payment

Performed by:Customer

Shipping

Performed by:Warehouse Mgr

Billing

Performed by:Secretary

Workflow example:

Computer-aided Web-oriented Workflow Design

Workflow specificationSemi-automatic generation of hypertext model

Peer B

Home page

PreliminaryValidation

End

Loan[Activity=

PreliminaryValidation]

Loans Index

W

PreliminaryValidation

Start

Disclaimer

This is NOT a tutorialIn this talk I focus on CONCEPTUAL MODELING aspects onlyI include a moderate amount of “advertising” (I’ll try to stay below 10%)

First movement: Andante con brio

WebML Primer

Web Modeling Language

A conceptual language for high-level design of data-intensive Web applicationsDefined in 1998, in use for more than seven yearsBook: Ceri, Fraternali at al. Designing data-intensive Web applications, Morgan Kauffman, Dec. 2002Adopted in many universities worldwide (www.webml.org)Commercially implemented (see: www.webratio.com) and used (see www.acer-euro.com)

WebML Diagrams

WebML: Conceptual high-level language for specification and design of data-intensive Web applications.

WebML diagrams:Data modelHypertext modelPresentation model

WebML: Hypertext Design

Siteview: set of pages and/or areas forming a coherent view of the site Area: a set of logically homogeneous pagesPage: a container of one or more pieces of information shown to the user at the same timeUnit: atomic information elementOperation: atomic action element

MailingList Subscription Area

Subscription Page

Products Area

ProductBrands Page Products Page

ProductBrands

ProductBrand

Products

Product[ProdBrand2Product]

ProductBrand

SelectedBrand

Insert dataModifySubscr

Distributor[ID=CurrentUser]

DATAUNIT INDEXUNIT MULTIDATAUNIT

ENTRYUNIT SCROLLERUNIT

Content Units in WebML

Author first name:XXX last name:YYY photo:

Index of Authors

• L. Tolstoj• T. Mann• M. Proust

All Authors

Browse Authors

5/12: go to

1/12

MODEL

Insert Your Data

•Fname•Lname

MULTICHOICE

Choose Authors

Tolstoj Mann Proust

HIERARCHICAL

Books&Authors

1. Compilers Aho Ullman2. Systems Tannenbaum

Unit input and output

Input is required to compute the unit Pre-defined parameters for the unit +Other parameters required by the selector of the unit

Output can be used to compute other unit(s) depending on the current unit

unitX

entity[selector (par 1, .., parN)]

IN OUT

MODEL

Navigation: contextual links

• An oriented connection between two units (source unit and target unit), normally rendered by means of anchors or submit buttons

• Purpose:

•Allowing the user to move from one place to another

•Transporting information from one place to another

•Activating a computation (side effect)

AuthorAuthor

MODEL

source unit target unit

Author first name:James last name:Joyce photo:

Author[OID=p1]

Book[author2book(p2)]

Book[OID=p3]

Book Title:Ulysses Price:23$ Cover:

Which author’s books? Which book?

Books of YYY

•Ulysses•The Dubliners•Portrait...

Example of linksMODEL

p1 p2 p3

WebML Incarnation: WebRatio

Our Interpretations of WebML

Units are “accidents”, what matters are:Links and parameter passingDefault parameters and parameter inference rules“Page” logics

Unit CollectionsContent DeliveryContent ManagementWeb Service InvocationXML TransformersProcess ManagementAdaptation & Personalization UnitsVOI and Digital TV Units

Content Management Units

Delete Unit

Entity

Create Unit

Entity

Modify Unit

Entity

CREATE DELETE MODIFY

Disconnect Unit

relationship

Connect Unit

relationship

CONNECT DISCONNECT

MODEL

WSDL units

WebML units for synchronous Web service interactions

Sync Solicit-Response Unit

Sync Request-Response Unit

XML management primitives

XML in: takes in input a canonical XML fragment and stores it into the E-R repositoryXML out: extracts a set of object from the E-R repository and build an XML canonical fragmentXML Adapter: takes XML fragments with a schema as input and builds an XML fragment with different schema as output

XML in XML out XML Adapter

XML data mapping

MODEL

A visual XSL generator tool to build XSL rules that transform XML fragments into an E-R structure (XML canonical form)

Third Movement: Andante Maestoso

Workflows

Workflows

Finally we turn to the main topic of this talk: process modelingApproach:

OverviewWorflow meta-data and primitivesWorkflow modelingWorkflow distributionAsynchronous workflow primitivesException Handling

Workflow & Web Service Design Steps (given)

Buyer

TransporterSupplier4:Commit transport

5:Notify shipping status

2:Send quote,7:Send recipit

1:Request quote,3:Order

6:Acknowledge

1: Actors and macro-operations1: Actors and macro-operations

Web ServiceWS-

Supplier

QuoteRequest (IN ProductsData, IN ReqID)

CommitOrder (IN OrderID, IN ReplyToAddr)

GetRecipit (IN OrderID, OUT Recipit)

3: Web service interface building3: Web service interface building

Buyer Supplier

Request quote

Reply with quote

Evaluate quote

Place order

Execute

Notify

CommitTransport

Elaborate quote

Update

Transporter

Receive order

Item info

Shippingstatus

Products data

Quote, OrderID

OrderID

Shipping ID

Ack Success

Record confirmation

Shipping ID

AcknowledgeSend

Recipit

OrderID

Record / Close process

Recipit

2: Conversation flow and operations identification2: Conversation flow and operations identification

TransporterTransporter(UPS, DHL)(UPS, DHL)

SupplierSupplier

BuyerBuyer

Our focus: From process design to sofware design & deployment

A global view: Quote request

RequestQuote

SendQuote

BuyerBuyer SupplierSupplier

Main Peer

Application Request

Preliminary Validation

Residence Check

x

JobCheck

FinancialCheck

+

FinalValidation

Choice

FinancialCheck+

Residence Check

CreditCheck

x

Applicant Manager Employee

Loan

Mortgage

Not Valid

Not Valid

x

x

Process Model: BPMN

“Click4aLoan”

Case Study

10/2005

BPMN notation

OGateways

Events Start End Intermediate

Or gateway

Activity Name Activity

Flow Sequence flow Message flow

Name Activity with event catching

Grouping Pool

Data Association

Lane

X +Xor gateway And gateway

Other grouping objects are provided: transaction,group, subprocess, ...

Looping, ad-hoc, and compensation activities are provided

Workflow metadata

ActivityType

Name

User

UsernamePassword...

Group

Name...

0:N

1:1

Activity Instance

StatusStartTimeStampEndTimeStamp

Case

StatusNameStartTimeStampEndTimeStamp

0:N 0:N

0:N 1:1 1:N

Assigned To

1:1

0:N

1:1

Process

Name1:1 1:N

0:N

1:N

Default

Assigned To

Type

PartOf

PartOf

Order

QuantityPriceDate

0:N

0:NAssigned

Payment

CreditCardAmount ...

0:N 1:1

PaymentToOrder

ProductGroup

NameDescription

Product

NameDescriptionImageTechnicalDetails

0:N

1:1

0:N 1:1

OrderToProduct

0:N

0:N

Created

CurrentStep

StepNo.StepUrl

Current

0:1

1:1

User metadata

Application data

WF metadata: WfMC concepts

Process management units

Activity delimiters

Start Activity End Activity

ActivityName ActivityName

Start Activity End Activity

ActivityName ActivityName

Requests

W

RefundingRequest[ActivityType="Approval"]

[User=CurrentUser]

CurrentUser

Assign

A

RefundingRequest[Activity="Approval"][User=CurrentUser][Case=CurrentCase]

ReqIDIf

Amount[true]

[false]

Amount>1000$

Process delimiters

WF indexes Assignements Conditions

Apply

CheckSalary

CheckSavings

Approve

Illustration in WebML: loan application process

Clients apply for a loan

Bank employees perform two checks in parallel

A manager approves or rejects the application

Generic hypertext for Apply

Client site viewApply

CheckSalary

CheckSavings

Approve

Client login page

Login

Create an Appl, connect it to the Apply instance

Client home page

Apply for loan

End Activity

Apply

Start Activity

ApplyStartCase

Complete hypertext for Apply

What happens inside the Client site view

Client login page

Login

Client home page

Apply for loan

End Activity

Apply

Start Activity

ApplyStartCase

Loan application page

LoanDataSumPeriod

New appl.

Application

W

Connects the new application

to the current Activity Instance

(Apply)

Generic hypertext for SalaryCheck

Employee site view

Apply

CheckSalary

CheckSavings

Approve

Employeelogin page

Login

Check salaryFinish check if OKAbort case if KO

Employee home page Start Activity

CheckSalary

CasesOK

Case [ready("CheckSalary")]End Activity

CheckSalary

OK

KOEnd Activity

CheckSalary

Generic hypertext for Approve

Apply

CheckSalary

CheckSavings

Approve

Manager site view

Managerlogin page

Login

Approve orreject loan application

Managerhome page

End Activity

Approve

Start Activity

Approve

CasesOK

Case [ready("Approve")]

Fourth Movement (Scherzo):Allegro

Push/Pull StylesDistribution ExceptionsAsynchrony

WebML Unit for Push style process control

Specific WebML unit: AssignCreates a new ActivityInstancesConnects it to the current case[Connects to a given user][Connects data to the ActivityInstance and to the user]

Example: document translation workflow

Transl.doc

Employee

Write doc

Manager

Assign

ActivityNameUserEntity

A

Pull vs Push Styles

Up to here, "pull work" style:Users choose their cases, inspecting the past activities

"Push" style also possible (push data and/or work):Push data: Manager assigns doc to Employee(s)Push work: Manager assigns Translate activity to Employee(s)

Manager site view in Push style

Managerhome page

Createdoc

End Activity

Create

Start Activity

CreateStartCase

Doccreation page

DocumentTitleBody

New doc.

Document

W

Managerlogin page

Login

Translator

User[type="employee"]

Document[UsedBy(UserID)]

[Activity="Translate“]

AssignDoc

A

Assignpage

Employee site view in Push style

Employeehome page

End Activity

Translate

Document translationpage

TranslateTitleBody

StoreTrans

Document

W

Employeelogin page

Login Docum.

Document

Start Activ.

Translate

No longer creates the ActivityInstanceSets status="Active"

MyDocs

W

Document[Translates(uID)]

[readyCase("Translate")]

1) Distribution of sub-processes with centralized control

Main Peer Peer A Peer B

1

2

3

4

5

Main Peer

Application Request

Preliminary Validation

x

JobCheck

+

FinalValidation

FinancialCheck

+

x

Applicant Manager Employee

Mortgage

Not Valid

Remote Peer

Employee

ResidenceCheck

CreditCheck

FinancialCheck

ResidenceCheck

x

Loan

Choice

Not Validx

Processes Distribution

Processes Distribution

Possibility for a peer to delegate a “well-parethesized" sub-process to an other peer

2) Distribution of sub-processes with nested co-ordination

Main Peer Peer A Peer B

1

2

3

6

5

4

Main Peer

x

JobCheck

+

FinancialCheck

+

x

Applicant Manager Employee

Loan

Mortgage

Remote Peer

Employee

ResidenceCheck

CreditCheck

FinancialCheck

ResidenceCheck

Manager

Application Request

Preliminary Validation Not Valid

x

FinalValidation

Choice

Not Validx

FinalCheck

FinalCheck

No constraints on the shape of the spawned sub-process.

3) Distribution of sub-processes with geralized co-ordination

Main Peer

Application Request

Preliminary Validation

JobCheck

+

ResidenceCheck

+

Applicant Manager Employee

Not Valid

Remote Peer

Employee

FinancialCheck

Manager

x

FinalValidation

Choice

Not Validx

Credit Check

Main Peer Peer A Peer B

1

2

3

5

4

Processes Distribution

Processes Distribution

Processes don’t have any information about the number of other processes involved and their structure in terms of activity.

The only point of contact between these processes is represented by the Web service calls.

4) Independent processes

Manufacture

Matriculation

For Sale

Demolition

User 1

Will Sell

Search Car

Usage

Purchase / Sale

Car

x

Will Sell

Search Car

Usage

User 2

10/2005Conceptual Modeling of distributed business processes based on Web Services ArchitectureConceptual Modeling of distributed business processes based on Web Services Architecture

Web-specific WF exceptions

User generated (Back and Forward browser buttons, …)

System failuresClient sideServer side

Inconsistencies due to critical situationsSemantic exceptionsIn the data or in the process status (metadata)

Improper inbound browsing

A3A2A1

Improper outbound browsingImproper internal browsingWait

Zzz..

Exception handling approach

User Metadata

ActivityType

NameDescription

User

UsernamePasswordFirstNameLastNameEmail

Group

Name...

0:N1:1

Activity Instance

StatusStartTimeStampEndTimeStamp

Case

StatusNameStartTimeStampEndTimeStamp

0:N

0:N

0:N 1

:1

1:N

1:1

0:N

1:1

Process

Name

1:1

1:N

0:N1:N

0:1

1:1

0:N Instan

ceOfInstanceOf

1:1

Process Metadata

Process & exception metadata

+ application information

Exception handling models

Handling model

Capturing model

Notifying model

Remote services

Web client

Capturing model

Notifying model

Handling model

Asynchronous Web services interactions

Asynchrony is mandatory in many B2B interactionsAsynchrony is a good solution for clients …

Requesting thread is not blockedReceived response is managed by a different thread

… and serversNo risks of traffic peaks (managed workload)Parallel spawning of requests management

Additional mechanisms are needed for connecting the two parts of the interaction

Asynchronous correlation

Correlation may be performed at 3 possible levels: Transport level- HTTP: only synch correlation- other protocols: not popular enough

Application semantics- Correlation is implicit in the exchanged data

Conversation metadata- auxiliary structures for automatic & transparent correlation

Conversation metadata

E-R model of the data (also for logging, tracking, data-warehousing)

1: ConversationID

2: CID + OpID

3: CID + OpID + MsgID

MessageInstance

MessageIDContent Out

In ConversationInstance

ConversationIDTimeStampStatus

1:N

PartOf

1:1

OperationInstance

OperationIDTimeStamp

1:1

1:1

0:1

1:1

MessageType

NameFormat Out

In OperationType

OpNamePortNameBindingServiceNameNetworkAddressTimeout

1:1

1:1

0:1

0:1

1:1 1:1

0:N 0:N

MsgType OpType

ConversationType

NameTimeout1:N

PartOf

1:1

0:N

ConvType

0:1 3 options

Asynchronous patterns

Describe possible asynchronous correlation structures

Combinations of synchronuos interactionsPattern: basic interaction schemas

Main patterns:PollingCallbackPublish-subscribe

Polling Pattern

Light client (no WS-publishing)

Low efficiency

Usage scenariosB2C portals

E-commerce services

2 variants: Client-driven polling attempts

Server-driver notification prior to polling attempts

Service Provider

Client

Request

Reply with CIDReq-Resp

Request with CID

Reply with ResponseReq-Resp

E-mailnotification

Callback Pattern

Client(Peer)

ServiceProvider

Request with:- ‘CID’- ‘Reply-to’ address

Response with ‘CID’to ‘Reply-to’ address

One way

One way

More efficient

Client must publish a WS to receive the response

Service provider must publish a service to receive requests

Usage scenariosInformation Retrieval (non business critical)

Publish-subscribe, event notification

Callback with acknowledgement

Client(Peer)

ServiceProvider

Request,Reply-to Address

Reply with CID Req- resp

Reply

Response with CIDto reply-to address

Req- resp

Acknowledgement can be applied to any pattern

Single-message operations becomes two-message operations

Usage scenariosB2B transactions

Critical interactions

Finale: maestoso

From concepts to implementation without writing code

Tools for workflow-driven hypertext design

WebFlowTool

Richiesta diprestito

+

Start

End

Validazionepreliminare

Controllofinanziario

Controllo sit .lavorativa

+

Approvazionefinale

Clie

nte

Ma

na

ger

Imp

ieg

ato

WebMLsite

skeletons

WebRatio RunningWeb

applicationStart Activity

End Activity

Activity A

Pages/Units

...

Activity B

Pages/Units

...

Pages/Units

...

Pages/Units

...

Start ActivityEnd Activity

Site View 1

AA

BB

Start Activity

End Activity

Activity A

Pages/Units

...

Activity B

Pages/Units

...

Pages/Units

...

Pages/Units

...

Start ActivityEnd Activity

Site View 1

AA

BB

“ToWebML” transformation “Manual” completion WebRatio automatic code generation

ACTIVITY B

ACTIVITY C

ACTIVITY A

Activity C

Pages/Units

...

Pages/Units

...

Start Activity

Site View 1

C

Pages/Units

...

Activity A

Pages/Units

...

Pages/Units

...

Start Activity End Activity

AA

Assign

A

Application[Activity=”B”][Activity=”C”]

[Case=CurrentCase]

Pages/Units

...

Activity B

Pages/Units

...

Pages/Units

...

Start Activity

B

Pages/Units

...

Application Index

W

Application[Activity=”B”]

Application Index

W

Application[Activity=”C”]

Application (written in JSP)running on arbitrary WebServer & DB Server+ a few additional annotations

HOW TO

VERIFY?

WebRatio WS edition

www.webratio.com

New release (September 2005)

Supports WebML extension for interacting with remote WS:

Request/response, solicit operations (1 or 2 way)

Automatic data import from XML

Automatic data export to XML

Visual orchestration of multiple web service calls

Supports service views for publishing Web Services

WSDL generation

Complete source code generation

www.webratio.com

Runtime architecture

Browser

WebML Runtime

JSP Engine

Web Server

Messagecomposer

Messagedecomposer

SOAP sender/listener

ConversationManager

Client Layer

Presentation Layer

BusinessLayer

DataLayer

JSP pages

<... ></...>

Descriptors

RDBMS LDAP XML

www.webratio.com

The product

www.webratio.com

Industrial experiences

Acer-Euro business portal: standardize the communication between the channel operators

Huelva Province selling point: manage a Business

Advising Units

Tiscover tourism services: manage heterogeneous

informations from hotels, tourist offices and tourists

MetalC: exchange business documents between

partners in a supply chain of a commercial district

Planned extensions

Finale

The “beauty” of conceptual modeling(modeling is “rock”, programming is “slow”)The “deep meaning” of WebML (units are accidents, the real power is links=control+flow and scope=page)The “power” of code generation (made possible by XML-related technology)

In general: Web applications and Web services Specifically: Workflow modeling (distributed and asynchronous cases), exceptions, pull vs push)

More on WebML/Webratio

The site: www.webml.orgThe product: www.webratio.comThe book: Ceri, Fraternali, Bongio, Brambilla, Comai, Matera, “Designing Data-Intensive Web Applications”, Morgan-Kaufmann, Dec. 2002.