pierluigi ritrovato research and technology director [email protected] giuseppe laria
DESCRIPTION
Pierluigi Ritrovato Research and Technology Director [email protected] Giuseppe Laria Head of GRID Unit [email protected]. GRASP GRid based Application Service Provision http://www.eu-grasp.net Microsoft Academic Days Segrate 22-24 November 2004. Agenda. Brief Grid overview - PowerPoint PPT PresentationTRANSCRIPT
GRASPGRASP
GRid based Application Service GRid based Application Service ProvisionProvision
http://www.eu-grasp.net
Microsoft Academic DaysMicrosoft Academic Days
Segrate 22-24 November 2004Segrate 22-24 November 2004
Pierluigi RitrovatoResearch and Technology Director
Giuseppe LariaHead of GRID [email protected]
2
Brief Grid overview Introduction to the GRASP project GRASP infrastructure description .NET and GRASP subsystems
Agenda
3
The evolution of the Grid concept
Metacomputing: A “new” computational approach is
introduced To use geographically distributed
supercomputing resources as they were a single high powerful computer
Clearly arising from need of high performance computer in scientific research world
Middleware Development: Identification of common building blocks Development of several middleware based on
proprietary protocols Heavy customisation was neded in terms of
development for setting up a Grid system
Grid computing & SOA for enabling scalable Virtual Organization: The Grid meets the Web Services world The definition of the “physiology of the grid” concept
and the emerging of OGSA Flexible secure coordinated resource sharing among
dynamic collections of individuals, institutions, and resources arranged as a Virtual Organisation
Grid computing has becoming synonymous of high scale distributed resource sharing for execution of innovative applications oriented to business and, in same cases, with high performance orientation
Guaranteeing interoperability inside the VO and across VOs
The visionary idea of the NGG expert group http://www.cordis.lu/ist/grids
4
EU funded projects in 5FP
IST Grid Projects
EGSOAVO
GRIA
CrossGridGridLab
GRIP
DataTAG
EuroGrid
DAMIEN
DataGrid
GEMSS
MammoGrid
BioGridSeLeNe
OpenMolGrid
COG
FlowGrid
GRACEMOSES
GRASP
1-10-2000 1-10-2001 1-10-2002
GRIDSTART cluster
Wave 2 Projects
5
The Grid meets the Web Services world (1)
At the origin the marriage between web services and Grid led to the Grid Service: “A WSDL-defined service that conforms to a set of conventions relating to its interface definitions and behaviors. Thus, every Grid service is a Web service, though the converse of this statement is not true”
The Grid Services basilar characteristics have been defined in the OGSI specifications (started at the end 2001 and released in 2003)
6
The Grid meets the Web Services world (2)
Grid
Web
Started far apart in apps & tech
OGSI
GT2
GT1
HTTPWSDL,
WS-*
WSDL 2
Have beenconverging ?
Source: globus alliance CrossGrids’04
7
The Grid meets the Web Services world (2)
Grid
Web
Started far apart in apps & tech
OGSI
GT2
GT1
HTTPWSDL,
WS-*
WSDL 2
Have beenconverging
Source: globus alliance CrossGrids’04
WSRF
WSRF
8
Web Service Resource Framework
WSRF is a framework consisting of a number of specifications WS-Resource Properties WS-Resource Lifetime WS-Service Groups WS-Notification WS-BaseFaults WS-Renewable References (unpublished)
Other WS specifications : WS-Addressing
9
From OGSI to WSRF:Refactoring and Evolution
OGSI WSRF
Grid Service Reference WS-Addressing Endpoint Reference
Grid Service Handle WS-Addressing Endpoint Reference
HandleResolver portType WS-RenewableReferences
Service data defn & access WS-ResourceProperties
GridService lifetime mgmt WS-ResourceLifeCycle
Notification portTypes WS-Notification
Factory portType Treated as a pattern: WS-Resource
ServiceGroup portTypes WS-ServiceGroup
Base fault type WS-BaseFaults
Source: globus alliance CrossGrids’04
10
Context Services Info
Services
InfraServices
SecurityServices
Rsrc Mgmt Services
Execution Mgmt
Services
DataServices
PolicyMgmt
VOMgmt
Access
Integration
Provisioning
Cataloging
BoundaryTraversal
Integrity
Authorization
Authentication
WSRF WSN WSDM
EventMgmt
Trouble-shooting
Discovery
JobMgmt
Logging
ExecutionPlanning
WorkflowMgmt
WorkloadMgmt
Provisioning
ApplicationMgmt
DeploymentConfigurationReservation
Naming
Self MgmtServices
HeterogeneityMgmt
Service LevelAttainment
QoSMgmt
Optimization
Information Services
Infrastructure Services
SelfMgmtServices
SecurityServices
Resource Mgmt Services
Execution Mgmt Services
DataServices
Context Services
Release date June 2004Release date June 2004
11
Basic capabilities
Infrastructure WS-Resource Framework (WSRF), WS-Notification (WSN) and other WS-
Standards Execution Management
concerned with the problems of instantiating and managing tasks, that are defined as single units of work to be managed
Data concerned with the movement, access and update of data resources
Resource Management performs several forms of management on resources in a Grid
Security to facilitate the enforcement of the security-related policy within a VO, e.g.
message integrity and confidentiality, authentication of interacting entities, minimum authentication strength, secure logging and audit, separation of responsibilities
Self-Management reduce the cost and complexity of owning and operating an IT infrastructure, e.g.
self-configuring, self-healing and self-optimizing Information
to efficiently access and manipulate information about applications, resources and services in the Grid environment
12
OGSA Framework
services are built on Web service standards, with semantics, additions, extensions and modifications that are relevant to Grids, e.g. WSRF
services are loosely coupled peers that, either singly or as part of an interacting group of services, realize the capabilities of OGSA through implementation, composition, or interaction with other services
services may be part of, or participate in, virtual collections called virtual domains
13
Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET and GRASP subsystems
14
Project Goals
GRASP is an RTD project started in 2002 and funded by the EC in the frame of IST-FP5 with 3 major goals:
Overcome weakness of current ASP solutions enabling Effective access to potentially distributed resources Delivery of services respecting negotiable service level agreements Charging services on the basis of effective use and delivered quality Managed security across different administrative domains
Design, implement and validate a layered architecture by: Analysis of existing GRID middleware and COTS for applicability
for critical aspects of an ASP infrastructure Utilising the concepts of the OGSA for creating Virtual
Organisations Test beds in the medical attention management and e-learning
domains as driver of requirements and validation scenarios Innovative ASP collaboration models
Federated model: The ASP consists itself of many actors that combine their resources in order to provide services to their clients
Many to many model: allows the integration of provider and user resources
15
Evolution of ASP
Internet Based Service Provider Provision of network centric
applications Target economies of scale Run services in a multi-tenancy model Support thousands of customers on a
single code basis Reduce hardware and admin cost by
partitioning and distributing resources Reduce of client-side development
investment and maintenance use standard Web technology as their
client-side interface
Utility computing shared infrastructure provided on
demand to multiple customers. lets companies pay for IT services as
needed Charge customers the same way an
electric company does: When usage spikes, so does the bill.
Pay-per-use or metered use Available on demand Dynamic allocation and integration
GRASP brings together elements of GRASP brings together elements of IBSP and Utility computing to enable IBSP and Utility computing to enable the next Generation Grid based ASPthe next Generation Grid based ASP
GRASP brings together elements of GRASP brings together elements of IBSP and Utility computing to enable IBSP and Utility computing to enable the next Generation Grid based ASPthe next Generation Grid based ASP
16
Technical Choices
GRASP platform main characteristics: The Open Grid Service Architecture is the reference model Microsoft (.NET platform) is the chosen deployment
infrastructure Compliance with OGSI specification Based on the OGSI.NET framework implementing the OGSI
v1.0 (released on July 2003) specification on Microsoft platform
GRASP platform leverages on standards coming from the Web Services world: SOAP, UDDI, BPEL, WS-Security, WS-Manageability, WSLA
GRASP platform integrates COTS components and commodity technologies: BizTalk for orchestration of Grid Services UDDI for searching Grid Services WSE: Microsoft implementation of WS standards MS Performance Counter for resource monitoring
17
MS Operating System
GRASP platform anatomy
OGSI.NET (Virginia University)
OGSI extensions
DestroyManageability
NotificationSecurity
WSRF.NET (Virginia University)
SL
A
Service
Instatiatio
n
Secu
rity
Acco
un
ting
&
Ch
argin
g
Microsoft .NET Framework
BizTalk 2004, UDDI directory
Service
Lo
cation
OrchestrationService
18
HE2
HE3 HEn
GRASP Physiology
HE1
INTE
RNET
End use
r
Host ASP application
VO
HE1 HE2
HE3HEn
19
Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET & GRASP subsystems
20
Basic GRASP Interactions
ServiceLocator
ServiceRequestor
ServiceProvider
ServiceInstantiator
Request
Service
Loca
tion
Return
SP
ServiceHost
Request Istance with specified
SLA
Create Instances or reuse idle Instance
(Invoking factory)
Provide access details
ServiceHostService
Host
Request Services/negotiation
SLAMonitoring
AS (SLA)
Acc (SLA)
Notify violation
SLA
Choose the host
ServiceHost GS
Collect data
HE
Gateway
21
The VHE concept
Grasp
ISA
PI F
ilter
OGSI.NET container
GRASPHandler
[Http Handler]
Service Registry
Service Instantiator
ServiceFactory
ServiceInstance1.2
ServiceInstance1.1
ServiceFactory
ServiceInstance2.2
Gateway Server Hosting Server(s)
New
OGSI.NET
OGSI.NET
Container PUC [Windows Service]
Handler
22
The VHE concept
Grasp
ISA
PI F
ilter
Service Registry
Service Instantiator
ServiceFactory
ServiceInstance1.2
ServiceInstance1.1
ServiceFactory
ServiceInstance2.2
Gateway Server Hosting Server(s)
Invoke
OGSI.NET container
OGSI.NET
OGSI.NETGRASPHandler
[Http Handler]
Container PUC [Windows Service]
23
Locator Subsystem
General scenario
VO 1
SP1
SP2
Service Locator
HE1
publish
Look up
HE2
publ
ish
SP3 deploy
deploy
MS UDDI Registry
24
Orchestration: 1st phase deployment
BizTalk Server 2004
Componentinvokers
Location and instantiation
Locator
HE
Instantiator
HE
Instantiator
HE
Instantiator
HE
Instantiator
ProcessDescription
AdditionalMetadata
GRASPcomposer
EnhancedProcess
Description
Deployment
ComposedService
Orchestration 2nd phase: invocation
25
Accounting & Manageability
OGSI.NET
•SDE management•Lifetime Management
ReferenceReference Grid Service
Manageability
•Metrics value SDEs•Service Shutdown
•Other OGSI port types•GRASP related port types
GridService PortType
(mandatory)
Other Port Type(optional)
26
Starting service accounting - Dynamic view
Accounting Service
Agreement Service
Instantiator Service
Charging Service
Business Service
accountService
chargeService
Retrieve SLA document (based on WSLA)
Subscribe to metric values changes
Creates records for informations related to: User, Service Account,User-Service relationships and then starts charging
Searches for Charge and Penaltiesin the SLA document; initializes internal logic with these policies
Subscribe to be informed about violation
Mng
Mng
27
Charging service usage – Dynamic view
Accounting Service
Charging Service
deliverNotification(…)
addServiceCharges(…)
Applies the Charge policy (defined in the SLA) andcalculates a new cost for theservice instance
MNG
Manageability detects metric value changes
Metric Provider
Updates metric changes
WMI
28
GRASP SOAP pipeline related to ASP.NET pipeline
IIS GRASPGRASPHandlerHandler
OGSI Handler
OGSI dispatcher
GRASP SOAP Handler
WSERoleCheck
SOAP Grid ServiceGrid Service
IIS ASP .NET WSE Web ServiceWeb Service
Configures WSE pipeline in order to allow it the management of GRASP related security aspects
Redirect incoming message towards
OGSI Handler
29
Secure group infrastructure
TT
AT
AT AT
ATAT
GM
GT
HE 1
HE 2
LSM1
LSM2
GT
30
Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET & GRASP subsystems
31
GRASP and MS.NET features
Instantiator Service: MS ISAPI Filter MS HTTP Handler MS Windows Service
Locator Service: MS UDDI registries
Orchestrator Service: MS BizTalk Server 2004 Orchestrator engine
Accounting: Windows Management Instrumentation (WMI)
SLA Monitoring: Performance counters
Security: Web Service Enhancement SOAP Handler
32
Contact Details
GrASP Scientific coordinator: Pierluigi Ritrovato (CRMPA)
Urls: www.eu-grasp.net www.crmpa.it
Contact persons: Pierluigi Ritrovato, Giuseppe LariaE-mail: [email protected],
33
Thank you… and now questions and
the second part
34
Charging service level violation – Dynamic view
Accounting Service
Charging Service
Agreement Service
deliverNotification(…)addServiceCharges(…)
Detects a servicelevel violation
Applies the Penalty policy (defined inthe SLA) and calculates the implied discount for the service instance
35
Instantiator Subsystem Instantiator extends OGSI standard factory
functionalities: Manages the creation of service instance within the
HE Interacts with the Service Level Agreement
monitoring subsystem to choose the machine (within the HE) that addresses the required QoS
Instantiates the service on the chosen machine Implements Handler Resolver capabilities:
The instance requestor isn’t provided with the real endpoint of the created service instance
It invokes the instance using a virtual GSH (but this is transparent to the requestor)
A subcomponent of the instantiator forwards the incoming request to the right instance
Implementation Status: Implementation is finished Possible improvements:
solving potential bottleneck issues Support migration mechanism Implement a WSRF version
36
Locator Subsystem Locator extends classical UDDI directory
functionalities: Allows to publish factory services against the directory
through an automatic mechanism Allows to classify factory on the basis of metadata that
describes features of instances created with this factory Locator subsystem architecture:
It has a hierarchical architecture with registries at three different levels:
Host Hosting environment Virtual organization
The deployment of a factory on a host automatically starts a process that publishes info about the new available service across three different levels of the locator subsystem
Based on following MS components: MS UDDI registry
Implementation Status: Implementation can be considered finished Possible improvements:
To introduce more advanced research criteria (at the moment they are based on the provision of a SLA document).
37
The orchestrator uses BPEL4WS as business process definition language (with some extensions to manage grid services)
It is exposed as a service (currently a Grid Service) that:
Takes as input parameters an extended BPEL script, including some ad hoc metadata for allowing the orchestrator to manage the grid service invocation
Performs “on the fly” location and instantiation of grid service during the business process execution
Exposes the composite service as a grid service Provides a simple mechanism for event handling based on
policy definitions
Orchestrator subsystem
38
Charging service level violation – Dynamic view
Accounting Service
Charging Service
Agreement Service
deliverNotification(…)addServiceCharges(…)
Detects a servicelevel violation
Applies the Penalty policy (defined inthe SLA) and calculates the implied discount for the service instance
39
It supports use-based and instance level-based charging; It uses policies (supported by WSLA, plus business extensions) for
providing a flexible, dynamically-configurable accounting (taking in account possible violation of SLA;
It supports multi-purpose accounting: we can identify two basic accounting threads: the first one will account costs to the
client (financial accounting), the second one will trace the costs that the provider of the service supports for providing services (cost accounting); Currently the only available type of accounting is financial accounting
It supports accounting of compound services Implementation status:
The implementation is finished It is based on the manageability concepts It is based on the Windows Management Instrumentation to store metric values At the moment a limited number of metrics are available:
Total Nr of invocation Nr of invocation for each operation Nr of invocation for each SDE Response time Size of passed parameters
Accounting subsystem
40
In GRASP we have two security contexts: Between ASP and their clients Within the Virtual Organization, that involves:
Creation of secure groups for grid services collaboration within the group
Provision of mechanism to enforce the instance access restriction within groups and across groups
GRASP focuses on the second context providing the following functionalities:
Security across organizational boundaries Manages secure service collaboration through:
Group membership management Encryption schema Management of policies based security mechanism
Security subsystem