Xiao Liu

CS3 -- Centre for Complex Software Systems and Services

Swinburne University of Technology, Australia

xliu@swin.edu.au

Key Research Issues in Scientific Workflow Temporal Verification

22

Scientific Workflows Key Research Issues in temporal verification

Temporal Verification A motivating example

Constraint Setting Checkpoint Selection Temporal Verification Temporal Adjustment

Temporal Verification Framework SwinDeW-V Project

Outline

3

Scientific Workflows Scientific Workflow Management System

A type of workflow management system aiming at supporting complex scientific processes in many e-science applications such as climate modelling, astronomy data processing. It may be built upon grid, cluster, P2P, Cloud infrastructure.

4

E-Science and E-Business

High-performance computing

Collaborative data-sharing

Collaborative design

Drug discovery

Financial modeling

Data center automation

High-energy physics

Life sciences

E-Business

E-ScienceNatural language processing & Data Mining

Utility computing

From www.gridbus.org

55

Scientific Workflows Key Research Issues in temporal verification

Temporal Verification A motivating example

Constraint Setting Checkpoint Selection Temporal Verification Temporal Adjustment

Temporal Verification Framework SwinDeW-V Project

Outline

66

Introduction: Temporal Verification

Scientific workflow verification: Structure, Performance, Resource, Authorisation, Cost and Time.

Temporal verification is to check the temporal consistency states so as to identify and handle temporal violations.

In reality, complex scientific and business processes are normally time constrained. Hence:

Time constraints are often set when they are modelled as scientific workflow specifications.

Temporal consistency states, i.e. the tendency of temporal violations from consistency to inconsistency, need to be verified and treated proactively and accordingly.

77

Definition: Temporal Consistency

88

Scientific Workflows Key Research Issues in temporal verification

Temporal Verification A motivating example

Constraint Setting Checkpoint Selection Temporal Verification Temporal Adjustment

Temporal Verification Framework SwinDeW-V Project

Outline

9

A Motivating Example

Question 1: Where and how much should we set temporal constraints?

10

Constraint Setting – A Solution Two basic requirements:

Temporal constraints should facilitate both overall coarse-grained control and local fine-grained control.

Coarse-grained constraints refer to those assigned to the entire workflow or workflow segments.

Fine-grained constraints refer to those assigned to individual activities.

Temporal constraints should be well balanced between user requirements and system performance.

A probabilistic setting strategy (X. Liu, BPM08) Aggregation: Setting coarse-grained constraints Propagation: Setting fine-grained constraints

11

Constraint Setting – A Challenge Where?

Currently, the locations of temporal constraints are normally assumed to be pre-defined. It is evident that the locations of temporal constraints have great impact on the efficiency control of workflow executions.

End ActivityDecision

Point

Critical Path

12

A Motivating Example cont.

Question 2: Where should we check the current temporal consistency state?

13

Checkpoint Selection – A Solution Two basic requirements:

Necessity: only those activity points where real temporal inconsistency states take place are selected

Sufficiency: there are no any omitted points. A minimum time redundancy based checkpoint

selection strategy (J. Chen, ACM-TASS2007)

14

Checkpoint Selection – A Challenge Efficiency

The criteria of necessity and sufficiency have significantly reduced the cost over the previous strategies, it is still huge especially in a scientific workflow of thousands of activities.

15

A Motivating Example cont.

Question 3: What is the current temporal consistency state?

Qualitative : {strong consistency/inconsistency, weak consistency/inconsistency }

Quantitative : {80% probability of consistency, 20% probability of inconsistency}

16

Temporal Verification – A SolutionMulti-States based temporal consistency (J. Chen, CCPE2007)

Temporal Dependency based Checkpoint Selection (J. Chen, Y. Yang, ICSE2008)

17

Temporal Verification – A Challenge Efficiency

The efficiency of temporal verification strongly related to checkpoint selection since they are always performed together.

The relationship between different temporal consistency can be helped to improve the efficiency.

18

A Motivating Example cont.

Question 4: What should we do if there are temporal violations?

19

Temporal Adjustment – A Solution Time deficit allocation (J. Chen CCPE2007) Time deficit allocation strategy (TDA) compensates

current time deficit by utilising the expected time redundancies of subsequent activities.

Based on expected time redundancies. Only delay the violations of local constraints. No effective on overall constraints.

20

Temporal Adjustment – A Challenge No effective solutions have been proposed yet. Different from conventional exception handling:

on the fault tolerance of functional failures; on non-functional QoS violations

triggered when true violations happened; triggered when expected violations detected

Possible solution: Recruiting additional resources Workflow scheduling Negotiation—amendment of temporal constraints ?...

2121

Scientific Workflows Key Research Issues in temporal verification

Temporal Verification A motivating example

Constraint Setting Checkpoint Selection Temporal Verification Temporal Adjustment

Temporal Verification Framework SwinDeW-V Project

Outline

22

Constraint Setting Setting temporal constraints according to temporal QOS

specifications. Checkpoint Selection

Selecting necessary and sufficient checkpoints to conduct temporal verification.

Temporal Verification Verifying the consistency states at selected checkpoints.

Temporal Adjustment Handling different temporal violations.

A Temporal Verification Framework

2323

Scientific Workflows Key Research Issues in temporal verification

Temporal Verification A motivating example

Constraint Setting Checkpoint Selection Temporal Verification Temporal Adjustment

Temporal Verification Framework SwinDeW-V Project

Outline

24

SwinDeW-V SwinDeW-V is an ongoing research project which focuses on

temporal verification and serves as one of the key functionalities in our SwinDeW-G, a peer to peer based scientific grid workflow system.

Scientific Workflow Execution

UKVPAC

HongKong

SwinburneCS3

· SwinDeW-G· GT4· CentOS Linux

BeihangCROWN

· SwinDeW-G· CROWN· Linux

SwinburneESR

· SwinDeW-G· GT4· CentOS Linux

AstrophysicsSupercomputer

· SwinDeW-G· GT4· SuSE Linux

Network Connection

Grid Node

SwinDeW-G Peer

PfC

25

Current States and Future Work Currently, as an important reinforcement for the overall

workflow QoS, temporal verification is being implemented in SwinDeW-G. It currently supports dynamic checkpoint selection and temporal verification at run-time.

In the future, SwinDeW-V will explore more on the two tasks of constraint setting and temporal adjustment. Our main objective is that SwinDeW-V can be developed as an independent software component which can be easily adopted by any workflow systems to facilitate the functionalities of temporal verification.

2626

Conclusion Temporal verification is important in scientific workflows Key research issues and challenges

Constraint Setting: the location of temporal constraints Checkpoint Selection: efficiency, computation cost Temporal Verification: efficiency, different consistency Temporal Adjustment: how to compensate time deficit

The research on scientific workflow temporal verification is still in its infancy and requires more efforts.

2727

The End

Thanks for your attention!