model-based simulation of soap web services from temporal logic specifications (talk @ iceccs 2011)

Sylvain Hallé

Sylvain Hallé

Model-Based Simulation of SOAP WebServices From Temporal Logic Specifications

Université du Québec à ChicoutimiCANADA

NOSHOW

Fonds de recherchesur la natureet les technologies

CRSNGNSERC

Sylvain Hallé

A motivating scenario

SHOW

Sylvain Hallé


$

NOINC

Sylvain Hallé


?

NOINC

Sylvain Hallé

?


NOINC

Sylvain Hallé

!


NOINC

Sylvain Hallé

!

?


NOINC

Sylvain Hallé

!

?


NOINC

Web service Web client

Interaction

Sylvain Hallé


NOINC

Sylvain Hallé

We want to do, as automatically as possible...


...impersonate the client, send test sequences to the service

?Check if service does whatwe expect/understand

...impersonate the service, generate responses to the client

?Environment closedÞ model checking possible

DRIVER STUB

SHOW

Sylvain Hallé

Some reasons for creating a stub:

·Test a cilent under development,without performing real actions onthe actual service

·Provide a closed environment for model checking

·Alternative to sandboxes: the stub's responses areunder the developer's control


STUB

SHOW

Sylvain Hallé

A SOAP web service

< >

</CartAdd>

CartAdd <CartID> <Items>

</Items>

ID123

123 456 789

</CartID>

<ItemID> </ItemID><ItemID> </ItemID><ItemID> </ItemID>

...

Request message in format:

· Nested elements · Many occurrences of the same element name · Flexible structure

XML

SHOW

Sylvain Hallé

Requests and responses form a

...

transaction

A SOAP web service

SHOW

Sylvain Hallé

<ItemSearch>

</ItemSearch> <Term>abc</Term>

A SOAP web service


...

transaction

NOINC

Sylvain Hallé

<ItemSearch>


<ItemSearchResponse>

</ItemSearchResponse>

<Items>

</Items>

123 456<ItemID> </ItemID><ItemID> </ItemID>

...

A SOAP web service


...

transaction

NOINC

Sylvain Hallé

<ItemSearch>


< >

</CartAdd>


</Items>

ID123

123 456 789

</CartID>


...



<Items>

</Items>


...

A SOAP web service


...

transaction

NOINC

Sylvain Hallé


...

...but not allsequences arevalid!

transaction

<ItemSearch>


< >

</CartAdd>


</Items>

ID123

123 456 789

</CartID>


...



<Items>

</Items>


...

A SOAP web service

NOINC

Sylvain Hallé

<ItemSearch>


< >

</CartAdd>


</Items>

ID123

123 456 789

</CartID>


...



<Items>

</Items>


...

A SOAP web service

1. Cart operations must begin with a CartCreate message

SHOW

Sylvain Hallé

A SOAP web service

SHOW

Sylvain Hallé

<CartCreate></CartCreate>

A SOAP web service

NOINC

Sylvain Hallé


<CartCreateResponse>

</CartCreateResponse> ID123<CartID> </CartID>

A SOAP web service

NOINC

Sylvain Hallé




A SOAP web service

< >

</CartAdd>


</Items>

ID456

123 456

</CartID>

<ItemID> </ItemID><ItemID> </ItemID>

...

NOINC

Sylvain Hallé




A SOAP web service

< >

</CartAdd>


</Items>

ID456

123 456

</CartID>


...

2. Once a cart is created, the same CartID must be passedin all requests and responses

NOINC

Sylvain Hallé




A SOAP web service

< >

</CartAdd>


</Items>

ID123

123 456

</CartID>


...

SHOW

Sylvain Hallé




A SOAP web service

< >

</CartAdd>


</Items>

ID123

123 456

</CartID>


...

< >

</CartAdd>


</Items>

ID123

456

</CartID>

<ItemID> </ItemID>

. . .

NOINC

Sylvain Hallé




A SOAP web service

< >

</CartAdd>


</Items>

ID123

123 456

</CartID>


...

< >

</CartAdd>


</Items>

ID123

456

</CartID>

<ItemID> </ItemID>

. . .

3. The same item cannot be added via CartAdd twice tothe same shopping cart

NOINC

Sylvain Hallé

The real service's behaviour follows constraints on:

1. of operations only2. Parameter only3. at the same time

How can we create a realistic stub thatfollows these constraints?

Sequencesvalues

Both

Challenge

SHOW

Sylvain Hallé

Current solutions

: create web servicesmock

SHOW

Sylvain Hallé

Current solutions

Problem

Responses are

messages

.

hard-coded

SHOW

Sylvain Hallé

Current solutions

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

Problem

Responses are

messages

.

hard-coded

NOINC

Sylvain Hallé

Current solutions

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>

CartAddResponse <CartID> <Items>

</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

Problem

Responses are

messages

.

hard-coded

NOINC

Sylvain Hallé

Current solutions

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

456

</CartID>

<ItemID> </ItemID>

Problem

Responses are

messages

.

hard-coded

NOINC

Sylvain Hallé

Current solutions

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

456

</CartID>

<ItemID> </ItemID>

Problem

Responses are

messages

.

hard-coded

NOINC

Sylvain Hallé

Current solutions

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

123

</CartID>

<ItemID> </ItemID>

< >

</CartAdd>


</Items>

ID123

456

</CartID>

<ItemID> </ItemID>

Problem

Responses are

messages: for each request type, same response every time!

.

hard-coded

?!?

NOINC

Sylvain Hallé

Current solutions

Other way: program a realistic stub in a programming language

SHOW

Sylvain Hallé

Other way: program a realistic stub in a programming language

struct

if

else

;

( .item ( .id)

XML(

m_cart

m_cartm_itemadd

return "<

Current solutions

NOINC

Sylvain Hallé

LTL-FO+

and

: extension of LTL with quantifiers on messageparameters (Hallé & Villemaire, IEEE Trans. on Services Computing 2011)

Can be used to express constraints on sequences of messages their values

For example, constraint 2:

(" CartCreateResponse/CartID/x : (" CartAddResponse/CartID/y : x=y))

...detailed semantics in the paper!

.

.

.

GX G

Specification of service behaviour

SHOW

Sylvain Hallé

Using LTL-FO+ as the specification language, producing a webservice stub becomes an application of LTL-FO+

Given...

·A pre-existing trace of requests·An LTL-FO+ formula

Produce:

·An extension of the trace (by one message) that the formula

satisfiabilitysolving

satisfies

Problem

SHOW

Sylvain Hallé

A model checker can find a trace of a formula,if there is one

Create a Kripke structure whose first n transitions areunique (and correspond to the pre-existing trace)

Don't give any constraints for the (n+1)-th state

Run the model checker on that system with thenegated specification

The counter-example found gives us a possibleextension of the existing trace

* S. Hallé, WS-FM 2010

counter-example

.

.

.

.

Initial solution*

SHOW

Sylvain Hallé

Don't rely on external tools, devise an algorithm to produce sequences from a formula

Interpret on sequences of messages......as to produce sequences of messages

The trick: decompose the formula into a tree of

directly

assertionsdirections

nodes

New solution

SHOW

sub-formulas thatmust be true now

sub-formulas that mustbe true next time

= conditions onthe current message

to generate

= conditions onthe remainder of the

transaction

Sylvain Hallé

Decomposition rules for some operators:

Decomposition rules

SHOW

Sylvain Hallé


Decomposition rules

NOINC

"j must holdin every message"

Sylvain Hallé


Decomposition rules

NOINC


Create a message thatfulfills j

Sylvain Hallé


Decomposition rules

NOINC



And next time, makesure that G j holds

Sylvain Hallé


Decomposition rules

NOINC


"j must holdin the nextmessage"



Sylvain Hallé


Decomposition rules

NOINC



Create a message thatfulfills j (No condition on

current message)


Sylvain Hallé


Decomposition rules

NOINC



Create a message thatfulfills j (No condition on

current message)

And next time, makesure that G j holds Next time, make

sure that j holds

Sylvain Hallé

Multiple branches = alternatives

SHOW

Decomposition rules

Sylvain Hallé

Multiple branches = alternatives

NOINC

Decomposition rules

"j must holdeventually"

In the currentmessage... In a future

message...

Sylvain Hallé

SHOW

Example: G (a ® b)X

G (a ® b)X ?

a, X b G (a ® b)X?

a G (a ® b), bX?

Øa G (a ® b)X?

a ® bX G (a ® b)X?

Decomposition rules

Sylvain Hallé

NOSHOW


G (a ® b)X ?

a, X b G (a ® b)X?

a G (a ® b), bX?

Øa G (a ® b)X?

a ® bX G (a ® b)X?

- Create a message that fulfills Øa

- Next time (a ® b)must hold

G X

Decomposition rules

Sylvain Hallé

NOSHOW


G (a ® b)X ?

a, X b G (a ® b)X?

a G (a ® b), bX?

Øa G (a ® b)X?

a ® bX G (a ® b)X?



G X

- Create a message that fulfills a

- Next time (a ® b)and b must hold

G X

Decomposition rules

Sylvain Hallé

SHOW

Once we exhaust the decomposition rules to apply...

a G (a ® b), bX?

Øa G (a ® b)X?



G X



G X

Decomposition rules

Sylvain Hallé


...we pick (arbitrarily) one of the alternatives

NOINC



G X

a G (a ® b), bX?

Decomposition rules

Sylvain Hallé


...we pick (arbitrarily) one of the alternatives andcreate a message based on the conditions

NOINC



G X

a G (a ® b), bX?

Decomposition rules

a

Sylvain Hallé


...we pick (arbitrarily) one of the alternatives andcreate a message based on the conditions

NOINC

a G (a ® b), bX?

G (a ® b), bX ?

Decomposition rules

a

The right-hand side conditions become the starting base for thenext message to produce

Sylvain Hallé

The decomposition rule for the existential quantifier creates values inside messages

p

pÅ

Values inside messages

SHOW

Sylvain Hallé


p

pÅ


"There exists an x atthe end of path p such

that j(x) is true"

NOINC

Sylvain Hallé


p

pÅ



that j(x) is true"

"Add some value b ati

the end of path p"

NOINC

Sylvain Hallé


p

pÅ



that j(x) is true"


the end of path p""Make sure that

true when x=b "ij(x) is

NOINC

Sylvain Hallé


...and repeat this for all possible values of bi

p

pÅ



that j(x) is true"


the end of path p""Make sure that

true when x=b "ij(x) is

NOINC

Sylvain Hallé

The decomposition rule for the universal quantifier ranges over values that are present in the message + potentially new values

p

pÅ


SHOW

Sylvain Hallé


p

pÅ


"All values x at theend of path p are such

that j(x) is true"

NOINC

Sylvain Hallé


p

pÅ


"Let S = set of all valuesi

already added at the endof path p + any number of

other values"


that j(x) is true"

NOINC

Sylvain Hallé


p

pÅ


"Let S = set of all valuesi

already added at the endof path p + any number of

other values"

"Make sure that true for all values in S "i

j(x) is


that j(x) is true"

NOINC

Sylvain Hallé

Conditions may add up and contradict themselves

pÅpÅØ


SHOW

Sylvain Hallé


pÅpÅØ


NOINC

"Value b must be ati

the end of path p"

Sylvain Hallé


pÅpÅØ


NOINC


the end of path p""Value b must not be ati

the end of path p"

Sylvain Hallé


pÅpÅØ


"Stop exploring thatalternative"

NOINC


the end of path p""Value b must not be ati

the end of path p"

Sylvain Hallé

Problem

The rule for " checks all values that were added byprevious applications of the rule for $

What if we add new values ?

Example: (" p/x : x > 0) Ù ($ p/y : y = 0)

Consequence: soundness is guaranteed only if all

after

$ areprocessed before any "(cf. Theorem 1 in the paper)

Soundness and completeness

SHOW

Sylvain Hallé

Universal stub: web service that takes as input a declarative specification of its behaviour

The stub dynamically produces sequences of messages following the specification

Implementation

MESSAGES

DOMAINS

SPECS

move[ put[row,col], player];

player: X,O,empty; row,col: 1,2,3;

[move/board/A1 x] ((x) = ({empty})); ...

Range of valuesfor each element

LTL-FO+ formulas

Structure of eachpossible message

SHOW

Sylvain Hallé

·

·Based on a runtime monitor for LTL-FO+ "running inreverse"

·

Implemented in Java

The stub dynamically produces sequences of messagesfollowing the specification

Implementation

SHOW

STUB

SPEC

Sylvain Hallé

A model checker can find a trace of a formula,if there is one

Create a Kripke structure whose first n transitions areunique (and correspond to the pre-existing trace)

Don't give any constraints for the (n+1)-th state

Run the model checker on that system with the specification

The counter-example found gives us a possibleextension of the existing trace

* S. Hallé, WS-FM 2010

counter-example

negated

.

.

.

.

Earlier solution*

SHOW

Sylvain Hallé

We compared both approaches on the same input specification

Showdown

SHOW

MESSAGES m[p*];

DOMAINS p : 1,2,...;

SPEC (" m/p/x : ($ m/p/y : x=y))G X G

<m>

</m>

0 2 ... 

Messages of the form

"Every value occurring insome must reappearin all future messages"



}

Sylvain Hallé

Exhibit A: we vary the size of the (i.e. the set of possible values in message parameters)

domain

Experiments

SHOW

MESSAGES m[p*];

DOMAINS p : 1,2,..., ;

SPEC (" m/p/x : ($ m/p/y : x=y))

n

G X G

Sylvain Hallé


domain

01

100

10,000

1,000,000

2 4 6 8

Tim

e (m

s)

Domain size

10

Experiments

With model checker

» 1300 × 1.65x

NOINC

Sylvain Hallé


domain

01

100

10,000

1,000,000

2 4 6 8

Tim

e (m

s)

Domain size

10

Experiments

With model checker

» 1300 × 1.65x

This paper's algorithm

» 6.5 × 1.42x

NOINC

Sylvain Hallé


domain

01

100

10,000

1,000,000

2 4 6 8

Tim

e (m

s)

Domain size

10

Experiments

With model checker

» 1300 × 1.65x


» 6.5 × 1.42x

6:50

0:00.375

NOINC

Sylvain Hallé

Exhibit B: we vary the message (i.e. the maximum number of parameters in messages)

arity

Experiments

SHOW

MESSAGES m[p{0, }];

DOMAINS p : 1,2,...;

SPEC (" m/p/x : ($ m/p/y : x=y))

n

G X G

Sylvain Hallé

Exhibit B: we vary the message (i.e. the number of parameters in messages)

arity maximum

Experiments

01

100

10,000

1,000,000

2 4 6 8

Tim

e (m

s)

Message arity

10

» 8500 × 1.64xWith model checker

NOINC

Sylvain Hallé

Exhibit B: we vary the message (i.e. the number of parameters in messages)

arity maximum

Experiments

01

100

10,000

1,000,000

2 4 6 8

Tim

e (m

s)

Message arity

10

» 8500 × 1.64xWith model checker

= 375


NOINC

Sylvain Hallé

Exhibit C: we measure processing time for as the trace lengthens

each new message

Experiments

00

Message #

2 124 146 168 1810 20

Tim

e (m

s)

900

800

700

600

500

400

300

200

100

SHOW

Sylvain Hallé


each new message

Experiments

» 16x + 511

With model checker

00

Message #

2 124 146 168 1810 20

Tim

e (m

s)

900

800

700

600

500

400

300

200

100

NOINC

Sylvain Hallé


each new message

Experiments

» 16x + 511

With model checker

» -0.2x + 3.5


00

Message #

2 124 146 168 1810 20

Tim

e (m

s)

900

800

700

600

500

400

300

200

100

NOINC

Sylvain Hallé

Take-home points

SHOW

1. Long-running web service transactions involve constraintsover message , and

2. Typical web service stubs only allow basic, pre-recordedinteractions

3. The logic LTL-FO+ can model these constraints

4. Simulating a web service becomes a problem of over a set of LTL-FO+

formulas

5. An algorithm can generate realisticsequences of messages

structure values sequence

declaratively

satisfiability solving

.

.

.

.

model-based simulation of soap web services from temporal logic specifications (talk @ iceccs 2011)

Technology

specificationsylvain

possiblesylvain hall

papersylvain hall

soap web service id123

soap web servicesylvain

mock web servicessylvain

soap web service id123

current solutions id123