optimis – towards holistic cloud management 2011-09-20 johan tordsson, department of computing...
Post on 19-Dec-2015
214 views
TRANSCRIPT
OPTIMIS – TOWARDS HOLISTIC CLOUD MANAGEMENT
2011-09-20
Johan Tordsson,
Department of Computing Science & HPC2N,
Umeå Universitet
2
OPTIMIS: BACKGROUND AND MOTIVATION
What? IP, Call 5, 10.4 M€ budget, 13 partners (8
academic) www.optimis-project.eu
Why? Multiple cloud models, definitions, etc. Our view:
Private clouds are common practice within the next few years Additional resources to handle load peaks etc. are provided by
public cloud(s) No one-size-fits-all solution to cloud provisioning Need for common abstractions, tools, and methods for various
scenarios
Roles & Challenges
New challenges– New customers– New business models– New collaboration forms– New requirements
4
FIVE CONCERNS FOR FUTURE CLOUDS
1. Dependable sociability Management based on non-functional aspects Foundation for eco-system of providers and
consumers of cloud services
2. Many cloud architectures Private, bursted, federated clouds, etc.
3. Service life cycle management optimization
Construction, deployment, operation
4. Adaptive self-preservation Self-* management with respect to functional
and non-functional aspects
5. Market and legal issues Identify business opportunities and legislative
concerns
5
1. DEPENDABLE SOCIABILITY Beyond cost-performance tradeoffs Tools for measuring & prediction of
TREC: Trust
Reputation-assessment of actors (SPs, IPs, etc.) Transitivity aspects
Risk Probability of something (bad) happening … … and the consequences Identification, assessment, monitoring, treatment
Eco-efficiency Monitor and predict power, PUE, CO2, etc. Compliance to standards and legislations
Cost Need for economical models beyond list prices Required to balance the above 3 factors
Multi-clouds
Federated Clouds
Infrastructure Provider
Bursted Private Clouds
2. MULTI-CLOUDS: THREE BASIC SCENARIOS
Infrastructure Provider
Infrastructure Provider
Infrastructure Provider
Infrastructure Provider
Service Provider
Broker
Infrastructure Provider
Service Provider
Service Provider
Infrastructure Provider
Private infra-
structure
Cloud providers Eco-System
– Programming Model
– Services Composition
[Legacy & New]
Construction
– Self-management– Risk Evaluation– Eco-efficiency–Data Management
Internal Cloud Operation
Optimization
Plus: – Multi-clouds– Federated clouds– License Management– Eco-efficiency Evaluation– Security
External Cloud Operation
Optimization
– Risk Assessment– Trust Circle– Eco-efficiency Evaluation
– Economic factor
Deployment Optimization
3. SERVICE LIFE CYCLE
8
4. ADAPTIVE SELF-PRESERVATION
Clouds are complex and environments change rapidly
We needAutomatic self-* management of infrastructure
self-configuration self-healing self-optimization
Holistic view Cannot do management of services, VMs, data, etc. in isolation
Self-management based also on non-functional aspects Trust, risk, eco-efficiency, and cost
Policy-driven management Adaptable and replacable policies
5. MARKET AND LEGAL ISSUES
Cloud eco-system new and currently evolving Opportunities for new roles, business models,
relationships, value chains, etc.
Legal concernsAcquisition, location, and transfer of dataAcross borders and legal domainsData protection and security mechanisms
needed (CS) Research problem
How to design mechanisms to be used to implement currently not known policies?
OUR APPROACH – THE OPTIMIS TOOLKIT
Addresses the five challengesGeneric toolset to support multiple cloud
architecturesReusable and configurable components Incorporates TREC-management and self-*
abilitiesSupports full service life cycleData protection capabilities
OPTIMIS SYSTEM MODEL What is a service (in OPTIMIS)?
Any functionality offered to clients over a network
Delivered through one or more VMsElastic
#VMs change dynamically during operationDefined by SP in a service manifest
VM images (OVF) SLAs w.r.t. elasticity (service-specific KPIs) Tresholds with acceptable levels of trust, risk,
eco-efficiency, and costDeployed by SP in IP(s)Operated by IP(s)
OPTIMIS TOOLKIT OVERVIEW
Four main groups of components Basic Toolkit SP tools IP tools Tools usable by both SPs and IPs
BASIC TOOLKIT Monitoring
Core functionality for self-managed systems 3 levels
Services Virtual infrastructure Physical infrastructure
Tools for measurement and prediction of Trust Risk Cost Eco-efficiency
Security Identity management, etc. to handle
interconnection of clouds
SP TOOLS Programming model
Implement new service components Integrate existing ones IDE + runtime for workflow style applications
License management Integrate license-protected software in servicesChallenges:
Elastic services Migrating services
SP TOOLS (CONT.)
Service Optimizer (SO)Overall management of services
Tracking state and deployment(s) Performance monitoring Re-deployment
Contextualization mechanismsDynamic runtime setup of VMs and services,
with respect to networking etc.Two step process:
Preparation Attach boot-scripts in ISO image and couple this
with VM image Self-contextualization
Booting from ISO-image
IP TOOLS Admission Control (AC)
Capacity planning and safe overbookingAccept incoming service request or not?
+ Increased revenue- Added provisioning costs? Implications for already hosted services
Services are elastic Degree of elasticity differ Time and duration of spikes differ
Similar problems Network bandwidth multiplexing Selling airline seats
Long-term capacity planning (cf. scheduling)
IP TOOLS (CONT.) VM Management
VM lifecycle managementScheduling: optimal mapping of VMs
to physical hosts in an IP across multiple clouds
Federation and bursting
When? Admission of new service, upon elasticity, faults,
periodically
Optimal? SP perspective:
Performance (hosts, VMs), cost, guarantees, TREC, etc.
IP perspective: Provisioning cost, consolidation, isolation, SLA
violations, etc.
IP TOOLS (CONT.)
Fault Tolerance EngineAutomatic VM checkpointing and restart Intervals configurable
Cloud Optimizer (CO)Combines monitoring and prediction with
IP-level engines to perform self-management
Overall decisions related to local vs. bursted/federated VM placement etc.
Policy reconfiguration
COMMON TOOLS FOR SPS AND IPS
Service Deployment Optimizer (SDO) Coordinates service deployment process
Discovers and filters IPs, negotiates SLAs, assesses TREC-factors, contextualizes services, uploads data, deploys services
Service deployment (SP to IP) Private cloud + multi-cloud service deployment
VM placement (IP to IP) Cloud bursting + federation
Data Management Transfer of VM images and service data for deployment;
SP to IP, and IP to IP Manages distributed file system for service applications Automatic re-location of service data across federated IPs
COMMON TOOLS (CONT.)
SLA Management (CloudQoS)Creation and monitoring of SLAsWS-Agreement term extensions for TRECNegotiation primitives (WS Agreement
Negotiation) Elasticity Engine
Feedback controller for automatic and proactive VM allocation to meet peaks and lows in demand
More about this one later…
OPTIMIS TOOLKIT DEPLOYMENT ILLUSTRATIONS
Bursted private clouds
IP
SPSDO SO
CO
AC SDO
SO
Private Cloud
CO
ACIP
CO
AC SDO
SO
IP
SPSDO SO
Federatedclouds
CO
ACIP
CO
ACIP
SPSDO
SO Multi-clouds
CO
ACIP
CO
ACIP
FURTHER FUTURE DIRECTIONS Use cases:
Programming model Service construction/composition Examples in ERP/CRM (SAP) and bio-informatics
Cloud bursting Outsourcing based on TREC Interoperation with OPTIMIS
and non-OPTIMIS Ips E-Education test cases
Cloud brokering, a broker: Acts as IP to SPs Acts as SP to IPs Is independent? Provides value-added
services?Infrastructure Provider
Infrastructure Provider
Infrastructure Provider
Service Provider
Broker
CURRENT & FUTURE DIRECTIONS (CONT.) OPTIMIS (the project) : June 2010 … May
2013 Basic plumbing in place Algorithmical improvements next focus
TREC-aware self-* management policies Holistic management, BLO-driven IP- and SP-
operation Experimentation needed Open for collaborations