challenges in establishment of open cloud markets
TRANSCRIPT
![Page 1: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/1.jpg)
Challenges in Establishment of Open Cloud Markets
Ivona Brandic
Vienna University of Technology, Austria
HPC Advisory Council Switzerland Conference 2012
March 13-15, Lugano
![Page 2: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/2.jpg)
Goods, Standardization, Markets
Challenging scenarios in electronic marketplaces
specification of non-standardized goods
dynamic evolution of user requirements
high diversity of user requirements
Infrastructure
STANDARDIZED GOODS
VIRTUAL GOODS
Applications
2
![Page 3: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/3.jpg)
Characteristics of Electronic Markets: Market Liquidity
Credits: Marcel Risch, Jörn Altmann, Ivona Brandic
3
![Page 4: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/4.jpg)
Today’s Cloud landscape
4
low market liquidity, stability and sustainability
large variety in services fragmented dynamic participation
?
?
?
?
? ?
![Page 5: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/5.jpg)
Adding autonomic capabilities solves these problems!
5
1. Definition of virtual products 2. Self-aware markets
Infrastructure
STANDARDIZED GOODS
VIRTUAL GOODS
Applications
![Page 6: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/6.jpg)
Challenges
?
?
?
?
?
?
Market
place
?
?
(1) Definition of virtual products
(2) Self-aware markets Rules Size
Connectivity
….
Cloud
Product 1
Cloud
Product 2
Cloud
Product x
Goals
Characteristics
Liquidity
![Page 7: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/7.jpg)
… From Autonomic Computing to Green ICT and self-aware Markets
Knowledge
Sensor Actuator
monitoring
analysis planning
execution
QoS Metric
Protocol
QoS Metric
Protocol
Evaluation
Service
Compositions
Mapping Strategies
Negotiation using
VieSLAF framework
Autonomic Manager
QoS Example
![Page 8: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/8.jpg)
Definition of Virtual Products: The SLA mapping approach
Market Directory
Private template
Service consumer
Private template
Service provider
Public SLA template
MMMMM
Market Directory
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
MMMMM
Market Directory
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
MMMMM
MM
Step 2: Define mappings
Market Directory
Step 3: Lookup
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
MMMMM
MM
Step 2: Define mappings
Market Directory
Step 3: Lookup
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
MMMMM
MM
Step 2: Define mappings
MM
Step 4: Define mappings
Market Directory
Step 3: Lookup
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
SLA TemplateAdaptation Component
MMMMM
MM
Step 2: Define mappings
MM
Step 4: Define mappingsStep 5: Adaptpublic templates
Market Directory
Step 3: Lookup
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
SLA TemplateAdaptation Component
MMMMM
SLA MappingModification Component
MM
Step 2: Define mappings
MM
Step 4: Define mappingsStep 5: Adaptpublic templates
Step 6: ModifySLA mappings
Market Directory
Step 3: Lookup
Private template
Service consumer
Step 1: Assign service to a template
Private template
Service provider
Public SLA template
SLA TemplateAdaptation Component
MMMMM
SLA MappingModification Component
MM
Step 2: Define mappings
MM
Step 4: Define mappings
Step 7: Assign new templates to usersStep 7: Assign new templates to users
Step 5: Adaptpublic templates
Step 6: ModifySLA mappings
8
Changing properties or
adding/deleting parameters
Service requirements
Standardized goods
Credits: Ivan Breskovic
![Page 9: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/9.jpg)
<xsl:template match="/ns:SLA/
ns:ServiceDefinition/
ns:WSDLSOAPOperation/
ns:Metric[@name='calculatePrice']/
ns:Function">
<ns:Function xsi:type="Times
resultType="float">
<ns:Operand>
<xsl:copy>
<xsl:copy-of
select="@*|node()"/>
</xsl:copy>
</ns:Operand>
<ns:Operand>
<ns:FloatScalar>
0.68096718
</ns:FloatScalar>
</ns:Operand>
</ns:Function>
</xsl:template>
Local
WSLA
template
Rule
From local
to remote
Rule
From local
to remote
Rule
from local
to remote XSLT
transformation
Remote
WSLA
template
XSLT
transformation
Rule
From local
to remote
Rule
From local
to remote
Rule from
remote
to local
+
+
XSLT
transformation
Rule
From local
to remote
Rule
From local
to remote
Rule from
remote
to local
XSLT
transformation
Local
WSLA
template
Rule
From local
to remote
Rule
From local
to remote
Rule
from local
to remote
+
+
Service
consumer
Service
provider
SLA Mapping Scenario
![Page 10: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/10.jpg)
Step 1: Public SLA template management
Medical apps
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Step 2: Group by SLO values
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Step 2: Group by SLO values
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Step 3: Learn
Step 3: Learn
Step 2: Group by SLO values
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Step 3: LearnData-intensive surgical apps
Step 3: Learn
Step 2: Group by SLO values
Simplesurgical apps
Initial public
SLA template
Medical apps
Step 1:Group bytemplatestructures
Cardiology
Oncology
Surgery
Step 3: LearnData-intensive surgical apps
Step 3: Learn
Step 2: Group by SLO values
Simplesurgical apps
Initial public
SLA template
New public
SLA templates
10
Requirements: 1. function for finding distance between clustering items 2. methods for computing cluster centroids
Credits: Ivan Breskovic
![Page 11: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/11.jpg)
Evolution of Public SLA Templates: Clustering Algorithms
Clustering
Finding a structure in a collection of unlabeled data
Goal: determine the intrinsic grouping in a set of unlabeled data
Algorithms
DBSCAN based on density distribution
of nodes
finds an appropriate number of clusters
two input values: distance e and set of points that are not farther away than min distance
k-Means assigns observations to
clusters with closest means
New means to centroids
Stops when there a no significant changes
11 Source:
http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/
![Page 12: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/12.jpg)
Computing distance between items
A. distance between template structures
B. distance between template objectives
12
dp (T1,T2 ) =
0 if propertiesof p are same in T1 and T2
1 if T1 orT2 does not contain p
1 if only one property of p differs in T1 and T2
2 if both properties of p differ in T1 and T2
ì
í
ïï
î
ïï
dH (X,Y ) = max{supxÎX
infyÎYd(x, y),sup
yÎY
infxÎXd(x, y)}
dH (VT1,VT2
) = max( x1 - x2 , y1 - y2 )
dV (T1,T2 ) =dH (VT1
,VT2) - min(dH ,V )
max(dH ,V ) - min(dH ,V )
x1 y1 x2 y2
VT1 VT2
![Page 13: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/13.jpg)
Computing cluster centroids
13
Maximum method
selects the maximum candidate
Threshold method
selects the maximum candidate if it is used more than a given threshold
Significant-change method
selects the maximum candidate if the percentage difference between the candidate and the current value exceeds a threshold
name: Cost Charge Rate (Price)
# mappings: 15% 15% 40% (30%)
Name of an SLA parameter
![Page 14: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/14.jpg)
Computing cluster centroids
14
Maximum method
selects the maximum candidate
Threshold method
selects the maximum candidate if it is used more than a given threshold
Significant-change method
selects the maximum candidate if the percentage difference between the candidate and the current value exceeds a threshold
name: Cost Charge Rate (Price)
# mappings: 15% 15% 40% (30%)
Name of an SLA parameter
![Page 15: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/15.jpg)
Computing cluster centroids
15
Maximum method
selects the maximum candidate
Threshold method
selects the maximum candidate if it is used more than a given threshold
Significant-change method
selects the maximum candidate if the percentage difference between the candidate and the current value exceeds a threshold
name: Cost Charge Rate (Price)
# mappings: 15% 15% 40% (30%)
Name of an SLA parameter
![Page 16: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/16.jpg)
Step 2: Autonomic mapping modification
Input
SLA templates: Tuser, Tinitial, Tnew
SLA mappings: Tuser ↔ Tinitial, Tinitial ↔ Tnew
Result
SLA mapping(s): Tuser ↔ Tnew
p p p
Tuser Tinitial Tnew
Tuser ↔ Tinitial Tinitial ↔ Tnew
Tuser ↔ Tnew
16
![Page 17: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/17.jpg)
Input
SLA templates: Tuser, Tinitial, Tnew
SLA mappings: Tuser ↔ Tinitial, Tinitial ↔ Tnew
Result
SLA mapping(s): Tuser ↔ Tnew
Step 2: Autonomic mapping modification
Availability Availability Availability
Tuser Tinitial Tnew
17
![Page 18: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/18.jpg)
Input
SLA templates: Tuser, Tinitial, Tnew
SLA mappings: Tuser ↔ Tinitial, Tinitial ↔ Tnew
Result
SLA mapping(s): Tuser ↔ Tnew
Step 2: Autonomic mapping modification
Availability Availability Av
Tuser Tinitial Tnew
Availability ↔ Av
Availability ↔ Av
18
![Page 19: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/19.jpg)
Input
SLA templates: Tuser, Tinitial, Tnew
SLA mappings: Tuser ↔ Tinitial, Tinitial ↔ Tnew
Result
SLA mapping(s): Tuser ↔ Tnew
Step 2: Autonomic mapping modification
Availability ServiceAv Av
Tuser Tinitial Tnew
ServiceAv ↔ Av
Availability ↔ ServiceAv ↔ Av
Availability ↔ ServiceAv
19
![Page 20: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/20.jpg)
Clustering - Challenges
Case 1: SLA parameter deleted from the template
Is the new public SLA template good enough for the user?
Case 2: SLA parameter added to the template
How to recognize new parameter’s equivalent in user’s private SLA template?
20
![Page 21: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/21.jpg)
Simulation testbed
Serviceconsumers
Market repositoryAccounting
serviceQuerying
serviceSLA mappingservice
Adaptation service
AA
Administration service
Service provider
Initial public SLA template
1. Assign service to the template 2. Fetch the SLA template
4. Apply clustering andlearning methods
6. Assign templatesto consumers
7. Measure overallnet utility
New public SLA templates
MMMMM
5. Modify SLAmappings
aa3. Create SLA mappings
Private SLA templates
AA
21
![Page 22: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/22.jpg)
Utility and cost model
Utility 0 - parameter does not exist in one of the templates
1 - both parameter properties differ in templates
2 - only one property differs in templates
3 - templates do not differ with respect to the parameter
Cost 0 - user needs no new mappings
1 - user must create new mapping(s)
2 - user must create new adding/deleting wishes
5 - user must rectify autonomically created mappings
Overall net utility = overall utility – overall cost
22
![Page 23: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/23.jpg)
Simulation results
Overall net utility
Overall cost
23
![Page 24: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/24.jpg)
Adding autonomic capabilities solves these problems!
24
1. Definition of virtual products 2. Self-aware markets
Infrastructure
STANDARDIZED GOODS
VIRTUAL GOODS
Applications
![Page 25: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/25.jpg)
Autonomic market platform
A marketplace is autonomic if it, once established, can exist and operate without human intervention
Self-awareness:
knowledge of the self and the self’s state, such that autonomic capabilities can be introduced
Self-Healing
Self-Protecting
Self-Optimizing
Self-Configuring
25
![Page 26: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/26.jpg)
Market Adaptation
Market Platform Frontend
Market Implementation
Management App Cloud
Application Reqs &
Constraints
App
Service Directory Allocation/Pricing
Mechanism
Market Infrastructure
Middleware
Sensor
Mappings
M
K A
P
E
26 Credits: Ivan Breskovic
![Page 27: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/27.jpg)
Monitoring methodology
Liquidity
Provider revenue
Transaction volume
Platform revenue
No. of allocations
Execution costs
No. of active participants
Immediacy
Bid-ask-spread
Market depth
Immediacy of matching
Average price bids
Average price requests
No. of allocated resources
Price of matching
Participation costs on market platform
Computation time No. of platform resources
Costs for platform resources
Active traders in last observation period
Metrics Goals
27
![Page 28: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/28.jpg)
Simulation environment
28
![Page 29: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/29.jpg)
Case study: buyers stop bidding
Simulation properties
200 buyers, each with 100 requests
100 sellers
Applied mechanism: continuous double auction
29
![Page 30: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/30.jpg)
Infrastructure observations
Simulation properties
200 buyers, each with 100 requests
100 sellers
Used infrastructure
MacBook Pro
Dual Core 2.66 GHz
4 GB RAM
30
![Page 31: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/31.jpg)
0
20000
40000
60000
80000
100000
120000
140000
160000
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82
Iteration
Total No. of bids Total No. of asks Total No. of allocations
Goals affected
Liquidity
Provider Revenue
Transaction Volume
Platform Revenue
No. Of Allocations
Execution Costs
No. Of Active Participants
Immediacy
Bid-ask-spread
Market Depth
Goals
0
500
1000
1500
2000
2500
3000
3500
4000
4500
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82
31
![Page 32: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/32.jpg)
Applications: Scientific Computing
Data analysis in bioinformatics
Joint work with D. Kreil and P. Labaj, Chair of
Bioinformatics, BOKU Vienna.
![Page 33: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/33.jpg)
Resulting Workflow
33 33
Million reads
Mapping to genome
Aligned reads Not-aligned reads
Splitting to sub-reads
Sub-reads
Set 1
Sub-reads
Set n
Sub-reads
Set 2
Searching for splice-junctions
Monitoring Agent
Monitoring Agent
Monitoring Agent
Reads aligned to the reference
Cloud Infrastructure
Monitoring
Framework
Knowledge
Management
FoSII Cloud Management
+ =
![Page 34: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/34.jpg)
Green ICT
34
![Page 35: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/35.jpg)
Green ICT
35
![Page 36: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/36.jpg)
Green ICT
36
![Page 37: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/37.jpg)
Green ICT
37
![Page 38: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/38.jpg)
Green ICT
38
Joint work with Jean-Marc Pierson, Georges Da
Costa, Damien Borgetto, University Paul Sabatier,
Toulouse, France.
![Page 39: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/39.jpg)
Conclusions and future work
Self-aware market platforms are crucial for the future development of utility computing
First step towards the vision: a monitoring methodology a series of realistic market goals
sets of extractable metrics from a market platform
mappings for each goal as a proxy for platform performance
Future work consider additional case studies
explore the remaining adaptation phases
connect and integrate with existing platforms
M
K A
P
E
39
![Page 40: Challenges in Establishment of Open Cloud Markets](https://reader036.vdocument.in/reader036/viewer/2022081503/62a23eab50eae4327b4dd06a/html5/thumbnails/40.jpg)
Thank you for your attention!
Ivan Breskovic, Ivona Brandic, Vincent Emeakaroha, Michael Maurer
Vienna University of Technology, Austria
Christian Haas and Simon Caton Karlsruhe Institute of Technology, Germany
Jörn Altmann Seoul National University, Korea