setting temporal constraints in scientific workflows
DESCRIPTION
Setting Temporal Constraints in Scientific Workflows. Xiao Liu, Jinjun Chen, Yun Yang CS3: Centre for Complex Software Systems and Services Swinburne University of Technology, Melbourne, Australia {xliu, jchen, yyang}@swin.edu.au. Content. Introduction - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/1.jpg)
Xiao Liu, Jinjun Chen, Yun Yang CS3: Centre for Complex Software Systems and Services
Swinburne University of Technology, Melbourne, Australia
{xliu, jchen, yyang}@swin.edu.au
Setting Temporal Constraints in Scientific Workflows
![Page 2: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/2.jpg)
2
Introduction Temporal Verification Temporal QOS Framework
Setting Temporal Constraints in Scientific Workflows Problem Statement A probabilistic strategy Evaluation
Conclusion
Content
![Page 3: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/3.jpg)
3
Introduction: Temporal Verification
Scientific workflow verification: Structure, Performance, Resource, Authorisation, Cost and Time.
In reality, complex scientific and business processes are normally time constrained. Hence:
Time constraints are often set when they are modelled as scientific workflow specifications.
Temporal consistency states, i.e. the tendency of temporal violations from consistency to inconsistency, need to be verified and treated proactively and accordingly.
Temporal verification is to check the temporal consistency states so as to identify and handle temporal violations.
![Page 4: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/4.jpg)
4
Temporal QOS Framework Constraint Setting
Setting temporal constraints according to temporal QOS specifications.
Checkpoint Selection Selecting necessary and sufficient checkpoints to conduct
temporal verification. Temporal Verification
Verifying the consistency states at selected checkpoints. Temporal Adjustment
Handling different temporal violations.
![Page 5: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/5.jpg)
5
Introduction Temporal Verification Temporal QOS Framework
Setting Temporal Constraints in Scientific Workflows Problem Statement A probabilistic strategy Evaluation
Conclusion
Content
![Page 6: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/6.jpg)
6
Problem Statement Most current work adopts only few overall user specified
temporal constraints without considering system performance.
Few overall constraints: not applicable for local verification and control.
User specified constraint: frequent temporal violations, huge exception handling costs.
![Page 7: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/7.jpg)
7
Two Basic Requirements Temporal constraints should facilitate both overall
coarse-grained control and local fine-grained control. Coarse-grained constraints refer to those assigned to the
entire workflow or workflow segments. Fine-grained constraints refer to those assigned to
individual activities.
Temporal constraints should be well balanced between user requirements and system performance.
![Page 8: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/8.jpg)
8
Probabilistic Strategy--Assumptions
Two assumptions on activity durations Assumption 1: The distribution of activity durations can be
obtained from workflow system logs. Without losing generality, we assume all the activity durations follow the normal distribution model, which can be denoted as N(µ,σ2) .
Assumption 2: The activity durations are independent to each other.
Exception handling of assumptions : Using normal transformation and correlation analysis, or moreover, ignoring them first and then adding up afterwards.
![Page 9: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/9.jpg)
9
Probabilistic Strategy--Definitions
Weighted Joint Normal Distribution Specification of Activity Durations Probability based Temporal Consistency
![Page 10: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/10.jpg)
10
Weighted Joint Normal Distribution The motivation for weighted joint normal distribution is to
estimate the overall completion time of the entire workflow by aggregating the durations of all individual activities.
However, they are not in a simple linear relationship. Our strategy is to model each activity duration as random
variables and aggregate them according to four basic control-flow structures, i.e. sequence, iteration, parallelism and choice. Since most workflow process models can be easily built by the compositions of the four building blocks, similarly, we can obtain the weighted joint distribution of most workflow processes.
![Page 11: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/11.jpg)
11
Specification of Activity Durations
Maximum Duration, Mean Duration, Minimum Duration The 3σ rule depicts that for any sample comes from
normal distribution model, it has a probability of 99.73% to fall into the range [µ-3 σ, µ+3 σ].
In our strategy, we have the following specification of activity durations:Maximum Duration D(ai)= µ+3 σ
Mean Duration M(ai)= µ
Minimum Duration d(ai)= µ-3 σ
![Page 12: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/12.jpg)
12
Probability based Temporal Consistency
![Page 13: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/13.jpg)
13
Probabilistic Strategy—Overview
![Page 14: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/14.jpg)
14
I Want the process be
completed in 48 hours
Let me check the probability
The negotiation process
Example: Setting Coarse-grained Constraints
![Page 15: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/15.jpg)
15
That’s not good, how
about 52 hours
Sir, its 70%, do you agree?
Adjust the constraint
Example: Setting Coarse-grained Constraints
![Page 16: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/16.jpg)
16
Err… how long will it take if I want to have
90%
Then, it increases to
85%
Adjust the probability
Example: Setting Coarse-grained Constraints
![Page 17: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/17.jpg)
17
Ok, that’s the deal! Let’s do
it!
It will take us 54 hours
Negotiation result
Example: Setting Coarse-grained Constraints
![Page 18: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/18.jpg)
18
Ok! But, sir, I need to remind you that this is only a guarantee from statistic sense. If we cannot make it, please
blame the guy who comes up with the strategy!
Sorry, statistically, no predictions can be 100% sure!
Example: Setting Coarse-grained Constraints
![Page 19: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/19.jpg)
19
Example: Setting Fine-grained Constrains
Setting fine-grained constraints for individual activities Assume the probability gained from the last step is θ% that is
with a normal percentile of λ. Then the fine-grained constraints for individual activities are (µi +λσi).
For example, if the coarse-grained temporal constraints are of 90% consistency, that is a normal percentile of 1.28, then the fine-grained constraint for activity ai with a distribution of N(µi, σi
2) is (µi +1.28σi).
![Page 20: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/20.jpg)
20
Evaluation—System Environment
Overview of SwinDeW-G environment Overview of SwinDeW-G environment
![Page 21: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/21.jpg)
21
Step1: Weighted Joint Distribution
![Page 22: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/22.jpg)
22
Step2: Coarse-grained Constraint Negotiation for coarse-grained constraint
6300s
6360s
6390s
6400s
66%
75%
79%
81%
WS~N(6210,2182)
U(WS)=6400s, λ=0.87
![Page 23: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/23.jpg)
23
Step3: Fine-grained Constraint
![Page 24: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/24.jpg)
24
Introduction Temporal Verification Temporal QOS Framework
Setting Temporal Constraints in Scientific Workflows Problem Statement A probabilistic strategy Evaluation
Conclusion
Content
![Page 25: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/25.jpg)
25
Conclusion Temporal verification is important in scientific workflows Setting temporal constraints is a prior task for temporal
verification. Two basic requirements: User requirements & System performance Coarse-grained & Fine-grained temporal constraints
A probabilistic setting strategy Aggregation: Setting coarse-grained constraints Propagation: Setting fine-grained constraints
Evaluation proves to be effective
![Page 26: Setting Temporal Constraints in Scientific Workflows](https://reader035.vdocument.in/reader035/viewer/2022062321/568130ed550346895d970fef/html5/thumbnails/26.jpg)
26
The End
Thanks for your patience and attention!