Resources and Services Virtualization without Boundaries (RESERVOIR)
www.reservoir-fp7.eu
The research leading to these results has been partially funded by the EuropeanCommunity's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 215605.
Alex GalisUniversity College London
Virtualized Service Platform 22.9.08 Brussels
The RESERVOIR Vision
• The Next Generation Infrastructure for Service Delivery– Provide revolutionary foundation for a new European infrastructure where
resources and services can be transparently and dynamically managed, provisioned and relocated like utilities – virtually “without borders”
– No single facility/provider can create a seemingly infinite infrastructure capable of serving massive amounts of users at all times, from all locations
• Federation of clouds • Leverage the diversity factor to achieve economies of scale• Leverage locality
– Analogies exists in areas outside services:• Electrical power delivery: capacity can be shifted to guarantee supply and lower costs• Roaming cellular communications: Talk wherever you are
• Impact – Enable utility-like deployment of services, relieving the service consumer from
awareness of the IT attributes while assuring QoS and security guarantees– Aim to create the basis for future service products
Virtualized Service Platform 22.9.08 Brussels
Approach• Focus on technologies that enable to build cooperating computing clouds
– Connect computing clouds to create an even bigger cloud
• Integration of virtualization technologies with grid computing driven by new techniques for service management The Service Oriented Infrastructure (SOI) equation:
• Building on this equation we will architect and implement a platform for supporting complex services, which• Enables dynamic deployment of complex multi-tier services across
heterogeneous administration domains• Uses virtualization of servers, storage and network to allow migration
without borders• Supports service definition, SLA management, accounting and billing
Grid-aware Virtualization
Service-oriented capacity provisioning
across sites
Grid-aware Virtualization
Service-oriented capacity provisioning
across sites
Virtualization-aware Grid
Optimal placement of VMs on a federated
cloud
Virtualization-aware Grid
Optimal placement of VMs on a federated
cloud
Business & Service Management
Policy-based management of service-level agreements
Business & Service Management
Policy-based management of service-level agreements
SOI
Virtualized Service Platform – 22.9.08 Brussels
RESERVOIR Partners and Competencies
Partner Role Comment
IBM HRL Technology Project Lead, Virtualization/SOA Infrastructure
Telefonica I+D Technology Service Technology, Billing Infrastructure
UCM Technology Grid, Dynamic Allocation Technology
Thales Technology Security, Virtualization Infrastructure, Hosting
SAP Use-Cases Use-Cases, Contribution to Requirement an Standards
Sun Microsystems Use-Cases + Tech Contribution to Standards, Java Services, Monitoring
DATAMAT Technology Service Management Technologies
University Lugano Technology Partner, Monitoring and SLA Management
University UMEA Technology Monitoring, Measuring and Billing Technology
University Messina Technology Grid Experience, Testbed Development,
University College London
Technology Service Management, Virtualization Technology
CETIC Technology Security
OGF Standardization Grid and Virtualization Standards
Virtualized Service Platform – 22.9.08 Brussels
The Reservoir Architecture
Infrastructure Provider = Site/Domain/Cloud
VEE Management System
Service Manager
VEE Management Enablement Layer
Virtualized Physical Resource (e.g., Hypervisor)
Service Provider
SLA SLA
SD+SLA
• Monitor service and enforce SLA compliance by managing capacity of Service Components (VEEs) or/and size of Service Tiers
• Deals with translation/mapping of service concepts/metrics (response time) to infrastructure concepts/metrics (VEE size)
• Monitor VEEs and find best VEE placement that meet constraint satisfaction problem
• Deals federation of domains
Virtualized Service Platform – 22.9.08 Brussels
Project Structure
A4: Service Management
A2: VEE Infrastructure
A3: VEE Management
A1:
Arc
hite
ctur
e
A5:
Tes
tbed
and
Sce
nario
s
A6:
Dis
sem
inat
ion
Virtualized Service Platform 22.9.08 Brussels
A2: VEE Infrastructure
• Virtual Machine Technologies (IBM)– Improve performance of VEE execution for typical
RESERVOIR workloads – Provide VEEMS enablement layer for virtual machines
• Relocation Enablement (IBM)– Network Virtualization– Storage Virtualization
• Java Service Containers (Sun)– Provide VEEMS enablement layer for virtual java service
containers
Virtualized Service Platform 22.9.08 Brussels
A3: VEE Management
• VEE Provisioning and Supervision (UCM)– Image management– Monitoring
• Allocation Policy Management (Datamat)– Policy based placement and migration
• Federation of Management Domains (UCM)– Built atop WSRF interfaces to access remote VEE
Supervisors• Push new and leverage existing OGF/DMTF/OASIS standards
– Interoperability between administrative domains and scheduling heuristics on federated and utility architectures.
Virtualized Service Platform 22.9.08 Brussels
A4: Service Management• Service Definition (UCL)
– Design a new service description language that will allow the description of service interfaces, service lifecycle, interface bindings to implementations, service deployment, SLA requirements for a service, rules for VEEs (re)configuration and (re)organisation and service components distribution and configuration
– Revisit the service lifecycle definition to accommodate the influence of virtualisation– Extend tools available for service design (for example the Eclipse Web Tools Platform)– Standardize the service description language
• Service Management (TID, UCL)– SLA monitoring across administrative domains and service-oriented architectures.– Integrate monitoring with resource allocation and scheduling and take explicit account
of the potentially synchronous nature of service invocations.– Automatic deployment of services based on complex service definition
• Accounting, Billing and Payment (TID)– Accounting and billing arrangements for outsourced services are based on raw
machine resource consumption (CPU-time, storage capacity etc)– RESERVOIR will pursue the definition of a framework that allows accounting and
billing in terms of the services that were completed, taking into consideration the quality of service that was provided.
Virtualized Service Platform 22.9.08 Brussels
A5: Experimentation and Validation• Testbed (UniMe)
– Create the necessary environment for testing and validation– Support the execution of use cases
• Scenario 1: eGov application (Thales)– Automatic adjustment of resources and domains cooperation.
• Scenario 2: SAP business application (SAP)– Business application oriented use cases and the opportunities to execute
them on a flexible infrastructure.• Scenario 3: Utility computing (Sun)
– Deploy arbitrary operating system and application stacks on remote resources. Provide secure and seamless access to them. Adjust resource allocation on-demand without the end user noticing disruption of service
• Scenario 4: Telco application (TID)– Hosting web sites that deals with massive access (e.g., the Olympics games)– High degree of personalization and support for mashups
Virtualized Service Platform 22.9.08 Brussels
Results so far
• RESERVOIR Architecture
http://ec.europa.eu/information_society/events/cf/item-display.cfm?id=583
• 10 presentations at conferences & workshops
• 2 papers fully based on the IPR generated in the project + 5 papers partially based on the IPR generated in the project
www.reservoir-fp7.eu
Virtualized Service Platform – 22.9.08 Brussels12
Virtualized Service Platform – 22.9.08 Brussels
The Evolution of the Power Grid
http://www.pbase.com/rbenny/image/29116201
http://www.anl.gov/Media_Center/logos22-1/electricity.htmThe US National Power Grid
http://www.rootsweb.com/~nytigs/BurdenPayrollRecords.htmThe Burden Iron Works Water Wheel
http
://ie
ee-v
irtua
l-mus
eum
.org
/col
lect
ion/
even
t.ph
p?id
=34
5687
6
The Pearl Street Station
•Make your own infrastructure•Not the company’s main business but a considerable competitive advantage
•The utility industry•Metering•Limited reach•Reproducible (yet costly)
•Efficient distribution•Federation of providers•The diversity factor•Economies of scale
Virtualized Service Platform – 22.9.08 Brussels
The Evolution of the Compute Grid
•Make your own infrastructure•Not the company’s main business but a considerable competitive advantage
•The utility industry•Metering•Limited reach •Reproducible (yet costly)
•Efficient distribution•Federation of providers•The diversity factor•Economies of scale
http://www.by-star.net/techspeak/datacenter/
http://www.smcplus.com/applications.asp?id=32
http://www.informationweek.com/galleries/showImage.jhtml?galleryID=62&imageID=13
Google @ The Dulles, OR
R E S E R V O I R“… today’s commercial clouds have not been open and general purpose, but instead been mostly proprietary and specialized for the specific internal uses (e.g., large-scale data analysis) of the companies that developed them. The idea that we might want to enable interoperability between providers (as in the electric power grid) has not yet surfaced …”
“…will move towards a mix of microproduction and large utilities, with increasing numbers of small-scale producers co-existing with large-scale regional producers, and load being distributed among them dynamically …”
There’s Grid in then thar Clouds - Ian Foster
Virtualized Service Platform 22.9.08 Brussels
SOI: Grid ComputingGrid node or Service Site
Physical Resources
Service Tasks
Virtualized Service Platform 22.9.08 Brussels
SOI: Grid Computing + Virtualization
Improved isolation, Relax dependencies, Well defined billing units
Virtual Execution Environment
(VEE)
Virtualized Service Platform 22.9.08 Brussels
Policy 1:If possible keep VEEs fromthe same organization in the same physical box
SOI: Grid Computing + Virtualization + BSM
Virtualized Service Platform 22.9.08 Brussels
Policy 1:If possible keep VEEs fromthe same organization in the same physical box
SOI: Grid Computing + Virtualization + BSM
Policy 2:Turn off underutilized physical boxes
Virtualized Service Platform 22.9.08 Brussels
Policy 2:Turn off underutilized physical boxes
Policy 1:If possible keep VEEs fromthe same organization in the same physical box
SOI: Grid Computing + Virtualization + BSM
Local optimizations (within a single site): placement, power, etc.
Virtualized Service Platform 22.9.08 Brussels
Policy 3:If possible keep
VEEs in “owning”organization
SOI: Grid Computing + Virtualization + BSM – Boundaries
Virtualized Service Platform 22.9.08 Brussels
Policy 3:If possible keep
VEEs in “owning”organization Policy 4:If possible keep
VEEs in least number of external organizations
SOI: Grid Computing + Virtualization + BSM – Boundaries
Virtualized Service Platform 22.9.08 Brussels
Policy 3:If possible keep
VEEs in “owning”organization Policy 4:If possible keep
VEEs in least number of external organizations
SOI: Grid Computing + Virtualization + BSM – Boundaries
Virtualized Service Platform – 22.9.08 Brussels
Policy 5:“Follow” the service customer
SOI: Grid Computing + Virtualization + BSM – Boundaries
Migration across sites Global optimizations: placement, cost, bandwidth, etc.
Virtualized Service Platform 22.9.08 Brussels
Virtualize the Network
Create virtual networks connecting VEEs regardless of physical server location
Virtualized Service Platform 22.9.08 Brussels
Virtualize the Network and the Storage
Enable secure access to relevant data regardless of storage location