gerhard engelbrecht june 22, 2009 a service-oriented grid environment with on-demand qos support...
Post on 29-Dec-2015
216 Views
Preview:
TRANSCRIPT
Gerhard EngelbrechtJune 22, 2009
A service-oriented Grid environment with on-demand QoS support
Gerhard Engelbrecht
July 10, 2009ICWS & Services 2009, PhD SymposiumLos Angeles, CA, USA
2
Gerhard EngelbrechtJune 22, 2009
Research Context
Grid Computing — Emerging new infrastructures— Enables globally distributed
resource sharing— eScience Vision
Challenge of complexity— Simplify development of
Grid-based applications— Hide complexity of the Grid
International:• EGEE• EUAsiaGrid• NextGRID• Open Science
Grid• PRAGMA• TeraGridField specific
Grids:• AstroGrid• BioGrid• BioinfoGrid• BIRN• CaBIG• GRACE• GRASP• GRIA• MammoGrid• myGrid• OpenMolGrid• Virolab• WISDOM• FightAIDS |
Folding | LHC | SETI @home
Middleware:• Condor• GLOBUS• gLite• GRIDBUS• GridSphere • NAREGI• UNICORE
3
Gerhard EngelbrechtJune 22, 2009
Grid Computing
Internet computing: sharing, distribution, and access to information
Grid Computing: sharing, distribution, and access to computing power
“Computational Grid is hardware and software infrastructure that provides dependable, consistent, and pervasive access to high-end computational capabilities”Foster, Kesselman (1998)
“Grid computing is concerned with coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”Foster, Kesselman (2000)
„The Grid ...— uses standard, open, general purpose protocols and interfaces— coordinates resources that are NOT subject to centralized control — delivers non-trivial qualities of service” Foster, Kesselman (2002)
4
Gerhard EngelbrechtJune 22, 2009
Neurosurgery support
Maxillo Facial Surgery support
Radiosurgerysupport
Inhaled drug delivery simulation
HPCApplication
GridResource
GridServic
e
DataManagement
JobManagement
QoSManagement
hidden fromthe user
QoS enabling
Motivation
• Expose HPC applications as services
• Support dynamic negotiation of QoS
SPECT image reconstruction
5
Gerhard EngelbrechtJune 22, 2009
Grid QoS scenario
I want to pay less then 10 €,
I can start simulation today at noon and I need the results at 3
p.m…
SPECT ClientMedical
practitioner
• Remote HPC facilities to be used by many different customers/clients
• Guaranteed response times and price Resource reservation Capacity/resource estimation
SPECT image reconstruction
QoS-awareGrid
ServiceNegotiation
6
Gerhard EngelbrechtJune 22, 2009
Architecture— Service-oriented architecture— Based on Web Services standards— Generic Service interfaces/access patterns
Application Services— Virtualize HPC applications as Grid services— Facilities: Data transfer, job management, error recovery— Quality of Service: Response time and price guarantees
Multi phase access model
Application Service
Client
Vienna Grid Environment
CA
Client
Registry
Service
S. Benkner, I. Brandic, G. Engelbrecht, and R. Schmidt. VGE - A Service-Oriented Grid Environment for On-Demand Supercomputing. In Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (Grid 2004), Pittsburgh, PA, USA, November 2004. IEEE.
Selection
ServicesRegistriesCandidate Criteria
Administrative Quality of Service Job Execution
AuthenticationAuthorizationSelect Price Model
QoS NegotiationPerformance EstimationResource ReservationsAgreement Exchange
Upload input dataStart JobMonitor JobDownload Results
7
Gerhard EngelbrechtJune 22, 2009
Contributions
Grid Environment—Service component architecture—Service provisioning infrastructure
QoS Support—QoS Negotiation—Micro QoS Management—QoS Management Approaches—Practical Evaluation
8
Gerhard EngelbrechtJune 22, 2009
Service Component Architecture
• Data transfer/staging/streaming• Application Execution• Quality of Service Negotiation
• Error recovery
• Monitoring
• Security facilities
S. Benkner, G. Engelbrecht, M. Köhler, and A. Wöhrer. Virtualizing Scientific Applications and Data Sources as Grid Services. Junwei Cao (Ed.), Cyberinfrastructure Technologies and Applications, Nova Science Publishers, New York, USA, 2008.
• Follows WSRF (Web Service Resource Framework)
• Separation of stateless service and stateful resource
Services (Components)
ResourceA
Resources
ResourceB
ResourceM
CompositeWSDL
Service 1
Service 2
Service 3
Service n
Services (Components)
ResourceA
Resources
ResourceB
ResourceM
CompositeWSDL
Service 1Service 1Service 1
Service 2Service 2Service 2
Service 3Service 3Service 3
Service nService n
Gerhard EngelbrechtJune 22, 2009
Service Description
Application Grid Service Provision
Seri
vce
Con
figura
tion
Serv
ice D
ep
loym
ent
Component A Config A+
Component B Config B+
Component N Config N+
…
Service ComponentSelection and Customizing
Config
HostAddress
…
+ApplicationContainer
Hosting EnvironmentSelection and Customizing
ApplicationDescriptor
ServiceProperties
Instantiation WAR Deployment Preparation
Libraries
ANTScripts
Autom. Process
WARWeb Application
Archive
Comp A
Comp N
…
Host (machine)
Tomcat (hosting env)
Serv
ice
10
Gerhard EngelbrechtJune 22, 2009
Contributions
Grid Environment—Service component architecture—Service provisioning infrastructure
QoS Support—QoS Negotiation—Micro QoS Management—QoS Management Approaches—Practical Evaluation
11
Gerhard EngelbrechtJune 22, 2009
— Objectives:>Response time and price guarantees>Trade-off between performance and price
— Client-driven business-oriented QoS negotiation with service providers
— Service Level Agreement (WSLA/WSA) exchanged with winner— Generic QoS infrastructure
>Application performance model >Advance resource reservation
Quality of Service: Big picture
Application Service
QoS Manager
Sch
ed
ule
r
Resources
Service Provider 1
Client Application
QoS Negotiator
Request N=10000
QoS Start: 2:00 End: 2:20 Price: 10€
Ask forWSLA offers
Offer WSLA
Accept/Reject offers
Start: 2:00End: 2:30Price: 8€
WSLA offered
Application Service
QoS Manager
Sch
ed
ule
r
Resources
Service Provider N
Start: 2:00End: 2:18Price: 11€
WSLA offered
…
App
App
12
Gerhard EngelbrechtJune 22, 2009
QoS Implementation
— Performance Model> Application- & machine-specific performance estimation
— Resource Model> Advance reservation of resources & interface to concrete scheduling system
— Pricing Model> Advance predication of the cost for a specific application
— QoS Manager > Management of I/O and interaction with all models
> Guided by a QoS management approach/algorithm
S. Benkner, G. Engelbrecht, S. E. Middleton, I. Brandic, and R. Schmidt. End-to-End QoS Support for a Medical Grid Service Infrastructure. New Generation Computing, Computing Paradigms and Computational Intelligence, Special Issue on Life Science Grid Computing, Ohmsha, Ltd. and Springer, 25(4), 2007.
RequestDescriptor
QoSRequest
fromclient
Performance
Model
Resource Model
Pricing Model
QoS Manager
QoSOffer
toclient
13
Gerhard EngelbrechtJune 22, 2009
QoS Manager: Prime-time approach
Inputs: QoS Request Request Descriptor
QoS ManagerPerformance
Model
Output: QoS Offer
PricingModel
Inputs: Request Desc Machine Desc
Output: Performance Descriptor
Output: Price for Offer
Input: QoS Request temp. Reservation
ResourceModel
Inputs: QoS Request Machine Desc Perf Desc
Output: Resource Descriptor
S. Benkner and G. Engelbrecht. Generic QoS Support for Application Web Services. International Symposium on Web Services and Applications, Las Vegas, USA, June 2005.
14
Gerhard EngelbrechtJune 22, 2009
QoS Manager: Prime price approach
Inputs: QoS Request Request Descriptor
QoS ManagerPerformance
Model
Output: QoS Offer
PricingModel
Output: Performance Descriptor
Output: Resource Desc
Input: Resource Desc Perf Desc
ResourceModel
Inputs: QoS Request Machine Desc Perf Desc
Output: Resource Descriptor
S. Benkner and G. Engelbrecht. Generic QoS Support for Application Web Services. International Symposium on Web Services and Applications, Las Vegas, USA, June 2005.
Inputs: Request Desc Machine Desc
15
Gerhard EngelbrechtJune 22, 2009
Practical Evaluation: GEMSS & AneurIST
GEMSS – Grid Enabled medical simulation services— Project duration: 2002-2005 (FP 5), — 11 Partners, Budget: ~3,5 MEuro
AneurIST - Integrated Biomedical Informatics for the Management of Cerebral Aneurysms
— Project duration: 2006-2010 (FP 6)—33 Partners, Budget: ~17,5 MEuro
Contributions/Results— Design and development of the Grid middleware— QoS infrastructure for application/compute services— GEMSS-Testbed with 6 different medical applications exposed as Grid
services— Stress-Tests: 1.200 jobs in 56h with robustness >95%— Successful ongoing usage in AneurIST
S. E. Middleton, M. Surridge, S. Benkner, and G. Engelbrecht. Quality of service negotiation for commercial medical Grid services. Journal of Grid Computing, Springer Verlag, ISSN 1570-7873, 2007.
16
Gerhard EngelbrechtJune 22, 2009
Summary and Future Work
Summary of Contributions—Grid Environment
>Service component architecture>Service provisioning infrastructure
—QoS Support>QoS Negotiation>Micro QoS Management>QoS Management Approaches>Practical Evaluation
Future Work— Virtualization
>Transition path towards Cloud computing>Service Science (Software as a Service)
— SLA management >Full lifecycle support (planning, negotiation, monitoring,
…)
17
Gerhard EngelbrechtJune 22, 2009
Related Publications in Journals
S. Benkner, G. Engelbrecht, M. Köhler, and A. Wöhrer. Virtualizing Scientific Applications and Data Sources as Grid Services. Junwei Cao (Ed.), Cyberinfrastructure Technologies and Applications, Nova Science Publishers, New York, USA, 2008.
A. Arbona, S. Benkner, G. Engelbrecht, J. Fingberg, M. Hofmann, K. Kumpf, G. Lonsdale, and A. Wöhrer. A Service-oriented Grid Infrastructure for Biomedical Data and Compute Services. IEEE Transactions on NanoBioscience, 2(6), June 2007.
S. Benkner, G. Engelbrecht, S. E. Middleton, I. Brandic, and R. Schmidt. End-to-End QoS Support for a Medical Grid Service Infrastructure. New Generation Computing, Computing Paradigms and Computational Intelligence, Special Issue on Life Science Grid Computing, Ohmsha, Ltd. and Springer, 25(4), 2007.
S. E. Middleton, M. Surridge, S. Benkner, and G. Engelbrecht. Quality of service negotiation for commercial medical Grid services. Journal of Grid Computing, Springer Verlag, ISSN 1570-7873, 2007.
R. Schmidt, S. Benkner, I. Brandic, and G. Engelbrecht. Component-Oriented Application Construction for a Web Service Based Grid. Concurrency and Computation: Practice and Experience, 19(5):637 -- 650, April 2007.
S. Benkner, G. Berti, G. Engelbrecht, J. Fingberg, G. Kohring, S. E. Middleton, and R. Schmidt. GEMSS: Grid Infrastructure for Medical Service Provision. Methods of Information in Medicine, 44, 2005.
W. Backfrieder, M. Forster, G. Engelbrecht, and S. Benkner. Optimized design of VOR for 3D image reconstruction in SPECT in a service oriented parallel implementation. Nuclear Medicine Technology, 31(2):139, 2003.
18
Gerhard EngelbrechtJune 22, 2009
Related Publications in Conference ProceedingsH. Rajasekaran, P. Hasselmeyer, L. Lo Iacono, J. Fingberg, P. Summers, S. Benkner, G. Engelbrecht, A. Arbona, A. Chiarini, C. Friedrich, M. Hofmann-Apitius, B. Moore, P. Bijlenga, J. Iavindrasana, H. Müller, R. Hose, R. Dunlop, A. Frangi, and K. Kumpf. @neurIST - Towards a System Architecture for Advanced Disease Managment through Integration of Heterogeneous Data, Computing, and Complex Processing Services. In IEEE International Symposium on Computer-Based Medical Systems, Jyväskylä, Finland, June 2008. IEEE Computer Society Press. Copyright (C) IEEE Computer Society.
S. Benkner and G. Engelbrecht. A Generic QoS Infrastructure for Grid Web Services. In Proceedings of the International Conference on Internet and Web Applications and Services, Guadeloupe, French Caribbean, February 2006. IEEE Computer Society Press.
S. Benkner, G. Engelbrecht, S. E. Middleton, and M. Surridge. Supporting SLA Negotiation for Grid-based Medical Simulation Services. In Workshop on State-of-the-Art in Scientific and Parallel Computing, Umea, Sweden, June 2006.
S. Benkner and G. Engelbrecht. Generic QoS Support for Application Web Services. In International Symposium on Web Services and Applications, Las Vegas, USA, June 2005.
S. Benkner, I. Brandic, G. Engelbrecht, S. E. Middleton, and R. Schmidt. Application-Level QoS Support for a Medical Grid Infrastructure. In Life Sciences Grid Workshop, Grid Asia, Singapore, 2005.
S. Benkner, G. Berti, G. Engelbrecht, J. Fingberg, G. Kohring, S. E. Middleton, and R. Schmidt. GEMSS: Grid-infrastructure for Medical Service Provision. In In Proceedings of HealthGRID 2004, Clermont-Ferrand, France, January 2004.
S. Benkner, I. Brandic, G. Engelbrecht, and R. Schmidt. VGE - A Service-Oriented Grid Environment for On-Demand Supercomputing. In Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (Grid 2004), Pittsburgh, PA, USA, November 2004. IEEE.
S. Benkner, I. Brandic, A. Dimitrov, G. Engelbrecht, R. Schmidt, and N. Terziev. Performance of Java Web Services Implementations. In Proceedings International Conference on Web Services, Las Vegas, USA, June 2003. CSREA Press.
19
Gerhard EngelbrechtJune 22, 2009
Supplementary slides
20
Gerhard EngelbrechtJune 22, 2009
QoS Models
QoS Capability Model
M: β c P x A
Service Capabilities: Ac = { a 1c … a n
c }
Client Request: Ar = { a 1r … a n
r }
QoS offer generation:
Ao = Ar п Ac
P … QoS parameters (job specific meta info)
A … QoS attributes (response time, price, …)
n-dimensional matching
21
Gerhard EngelbrechtJune 22, 2009
Generic Offer Generation
a1
a2
a3
a3cmax
a1cmax
a2cmaxAi
r|cmin
“QoS Capability Cuboid”
“QoS Request Cuboid”
a3rmax
a1rmax
a2rmax
Ar п Ao
"Offer Cuboid"
top related