An Automata-based Approach to Testing Properties in Event Traces

H. Hallal, S. Boroday, A. Ulrich, A. Petrenko

Sophia Antipolis, France, May 2003

2

Outline

Motivation

Event traces

Problem

Our approach

Implementation

Case study

Conclusions and extensions

3

Motivation

Analysis of distributed systems is complex and costly Asynchrony

Lack of global timing

Absence of reference specification

A practical solution is to instrument the system to generate traces of events that can be visualized and analyzed further

This solution can be used to debug the system During development After deployment

4

Visualization Vs Analysis Tools

elaborate ad-hoc algorithms more efficiency more efforts

reuse an existing model checker more expressiveness less efforts

AnalysisTools

Monitoring Tool

Trace

distributed system of processes

Event

...

Event

VisualizationTools

Visualization tools facilitate the manual inspection of collected traces

Analysis tools automate the verification of properties in the traces

5

Trace Analysis Problem

Given A distributed system under test (SUT)

Some properties

Verify whether the SUT satisfies the properties

Solution Monitor the SUT and collect an execution trace

Model the collected trace

Use an existing model checker to verify the properties

6

Trace

Distributed processes generate local traces Local events: state update, parameter change

Communication events: message exchange, RMI, RPC

Local traces are sequential

Communication Asynchronous: send and receive events

Synchronous: rendezvous events

Point-to-point communication Each message has a send and a receive in the trace

Each rendezvous involves at least two parties

7

Event Traces

Event ordering induced by local orders i and point-to-

point communication

A trace is a partially ordered set E of all events

Causality relation on events If a i b then a b 

for every message m, send(m) receive(m)

is transitive: If a b and b c then a c

Event trace

a tuple of local traces with an irreflexive causality relation on all events

8

Lattice of Ideals

Offers an efficient way to

check properties

n1 = 3

pr1

pr2

n2 = 6

n1 = 4

n2 = 4

n1 = 5

n2 = 2

m1

m3

m2

time

Encodes all the possible

linearizations of E

9

Problem

Given An event trace of a distributed system A set of properties

How to build the lattice of ideals to verify the properties?

Monolithic approach

build the lattice explicitly

use a model checker

Modular approach

model the event trace as a system of communicating automata

build the composition of automata

prove it is isomorphic to the lattice

10

Our Approach

We use finite automata to model Local traces of processes

states are ideals transitions are events

Message delays

We build the composition of all automata We prove

composition of automata lattice of ideals

Use the composition automaton to verify the properties use an existing model checker avoid full state space search

{send } {send, receive}

send(m) receive(m)

11

Implementation

We use SDL and ObjectGEODE (OG)

We model the SUT as an SDL system Local traces: designated processes

Local events: SDL TASK

Communication: signal exchange

How to treat the message delay automata? Individual processes

Individual queues

SDL “SAVE”

Properties are specified in GOAL of OG

12

Workflow of the Approach

Front-End tool to ObjectGEODE System specification Pattern specification

Library of property patterns Parameterized GOAL observers State-based, event-based, mixed

Monitoring Tool

distributed system of processes

Event...Event

User

ObjectGEODE Simulator

Front End to ObjectGEODE

User Interface

PropertySpecification

SystemSpecification

Trace

GOAL ObserverSDL Model

Model results: 1. Property satisfied or not2. Scenarios

Pattern Library

13

Pattern Library

Property patterns already exist Repository of common properties Mappings to main formalisms used in finite state verification

LTL, CTL, INCA, QRE,…

Library of GOAL observersAddress finiteness of traces

Encode common patterns Class: order vs. occurrence Name: response, universality, ... Scope: global, before, after, ... Parameterized GOAL specification

parameters are predicates on states, events, or both

14

Pattern Template

Name and Intent Response Cause-effect relationship

ClassOrder

ScopeGlobal: the entire execution

Exampleresource granted after request

observer response

success state success;

error

state error;

waitp

true

P

true

S

true

last_state

true

success

false

waitp

false

last_state

true

error

false

waits

false

last_state

true

success

false

waitp

waits

true

S

true

last_state

true

success

false

waitp

false

last_state

true

error

false

waits

S responds to P in the execution

15

TRAYSIS

Input: XML logfile

Output: SDL model

Features Logfile conformance check Synchronous/asynchronous Statistics on the model

processes, channels, variables, signals,...

Model customization

scalability

Access to OG

16

Property Manager

event observer SWP

s0

msg := output TVoid

msg!sender = pb2

true

cnt1 := cnt1+1,cnt := cnt1 + cnt2,trncnt := cnt - cnt4

true

err

false

l1(cnt1) := pb2!Win_Send

s0

false

true

trncnt > limitand cnt4 >0

true

err

false

l2(cnt2) := pb3!Win_Send

s0

false

s0

msg := input TWindowAck

true

updtarray

(pb2!Win_Quit, l1, cnt1, cnt2, cnt)

s0

false

true

updtarray(pb3!Win_Quit, l2, cnt2, cnt1, cnt)

s0

false

s0

output TWindowAck from pb1

cnt4 := cnt4 + 1,trncnt := cnt - cnt4

(trncnt > limitand

cnt4 > 0)

true

err

false

s0

msg!sender = pb3

msg!sender = pb2

msg!sender = pb3

(trncnt > limitand

cnt4 > 0)

cnt := cnt1 + cnt2,cnt2 := cnt2+1,

trncnt := cnt - cnt4

Supports property specification

Easy access to library Customize observers

17

Case Study

An implementation of the Sliding Window Protocol Extension to the PROFIBUS protocol stack Supports communication in distributed power control system

Properties of interest Maximum window size is respected Total number of unacknowledged messages less than limit Total number of messages in transit less than limit

Execution traces are collected using protocol analyzers

We used out tool set to automatically analyze the system

We have analyzed large traces (15k –20k events)

18

Conclusions and Future Work

Formal definition of event traces

A framework to model mixed communication modes (GALS)

Automata-based approach to analyze event traces

A component based implementation of the approach

A case study: the SWP

Target more general logfiles

Enhancement of the tool set

Merci beaucoup!