EGEE-II INFSO-RI-031688
Enabling Grids for E-sciencE
www.eu-egee.org
The European Grid Initiative -A View from the Austrian National Grid Initiative
Dieter Kranzlmüller (EGI Coordinator)
GUP – Institute of Graphics and Parallel ProcessingJoh. Kepler Univ. Linz, Austria
CGW’06Cracow Grid Workshop
17 October 2006Cracow, Poland
The European Grid Initiative 2
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
The European Grid Initiative 4
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Defining the Grid
• A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user.
The European Grid Initiative 5
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: Biomedicine
• Parallel simulationof blood flowon the Grid
Cooperation with University Amsterdam
The European Grid Initiative 6
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: Flooding Crisis
• Simulation of floodingon the Grid
Cooperation with Slowak Academy of Sciences
The European Grid Initiative 7
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: Scientific Visualization
Visualization on Sony PSP
The European Grid Initiative 8
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: WISDOM
• Grid-enabled drug discovery process for neglected diseases– In silico docking
compute probability that potential drugs dock with target protein
– To speed up and reduce cost to develop new drugs
• WISDOM (World-wide In Silico Docking On Malaria)– First biomedical data challenge – 46 million ligands docked in 6 weeks
Target proteins from malaria parasite Molecular docking applications:
Autodock and FlexX ~1 million virtual ligands selected
– 1TB of data produced – 1000 computers in 15 countries
Equivalent to 80 CPU years
• Significant results– Best hits to be re-ranked using Molecular Dynamics
The European Grid Initiative 9
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: Avian flu
• Avian Flu H5N1– H5 and N1 = proteins on virus surface
• Biological goal of data challenge– Study in silico the impact of selected point
mutations on the efficiency of existing drugs – Find new potential drugs
• Data challenge parameters:– 5 Grid projects: Auvergrid, BioinfoGrid, EGEE,
Embrace, TWGrid– 1 docking software: autodock– 8 conformations of the target (N1)– 300 000 selected compounds >100 CPU years to dock all configurations
on all compounds
• Timescale: – First contacts established: 1 March 2006– Data Challenge kick-off: 1 April 2006– Duration: 4 weeks
N1H5
Credit: Y-T Wu
Credit: Y-T Wu
The European Grid Initiative 10
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Example: Earthquakes
• Seismic software application determines epicentre, magnitude, mechanism
• Analysis of Indonesian earthquake (28 March 2005)– Seismic data within 12 hours after the earthquake– Solution found within 30 hours after earthquake occurred
10 times faster on the Grid than on local computers– Results
Not an aftershock of December 2004 earthquake Different location (different part of fault line further south) Different mechanism
Rapid analysis of earthquakes important for relief efforts
Peru, June 23, 2001Mw=8.4
Sumatra, March 28, 2005Mw=8.5
The European Grid Initiative 11
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE-II Overview
The European Grid Initiative 12
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Defining the Grid
• A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user.
The European Grid Initiative 13
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE Infrastructure
Country participating
in EGEE
Scale (July 2006):~ 200 sites in 40 countries
> 26 000 CPUs
> 11 PB storage
> 35 000 concurrent jobs per day
> 60 Virtual Organizations
The European Grid Initiative 14
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Defining the Grid
• A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user.
The European Grid Initiative 15
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
MiddlewareGlobus GT4 CondorAPST
PlatformInfrastructure
Unix Windows JVM TCP/IP MPI .Net Runtime
Environmental Sciences
Life & Pharmaceutical
Sciences
ApplicationsGeo Sciences
Building Software for the Grid
VPN SSH
Courtesy IBM
Slide Courtesy David Abramson
The European Grid Initiative 16
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
MiddlewareGlobus GT4 CondorAPST
PlatformInfrastructure
Unix Windows JVM TCP/IP MPI .Net Runtime
Environmental Sciences
Life & Pharmaceutical
Sciences
ApplicationsGeo Sciences
Building Software for the Grid
VPN SSH
Courtesy IBM,Lower Middleware
Upper Middleware & Tools
Bonds
Slide Courtesy David Abramson
The European Grid Initiative 17
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Defining the Grid
• A Grid is the combination of networked resources and the corresponding Grid middleware, which provides Grid services for the user.
The European Grid Initiative 18
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE Applications
• >20 applications– High Energy Physics– Biomedicine– Earth Sciences – Computational Chemistry– Astronomy– Geo-Physics– Financial Simulation– Fusion
• Further applications in evaluation
Applications now moving from testing to routine and daily usage
The European Grid Initiative 19
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE-II Overview
BUT …
The European Grid Initiative 20
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGEE and Sustainability
BUT …
• How does EGEE compare to other computing infrastructures?– Number of infrastructure users?– Number of application domains?– Number of computing nodes?– Number of years in service?
• What would happen, if EGEE is turned off?• What happens after April 2008
(End of EGEE-II)?
The European Grid Initiative 21
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Sustainability: Beyond EGEE-II
• Need to prepare for permanent Grid infrastructure– Maintain Europe’s leading position in global science Grids– Ensure a reliable and adaptive support for all sciences– Independent of project funding cycles– Modelled on success of GÉANT
Infrastructure managed centrally in collaboration with national bodies (in EGEE-II: JRUs)
The European Grid Initiative 22
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
e-Infrastructure for Europe
• A European Vision for a Universal e-Infrastructure for Research(1)
– “An environment where research resources (H/W, S/W & content) can be readily shared and accessed wherever this is necessary to promote better and more effective research”
(1) Malcolm Read (Ed.) http://www.e-irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf
The European Grid Initiative 23
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
e-IRG Recommendations on Sustainable e-Infrastructures
I: Governments and the Commission should develop policies and mechanisms to encourage increased investment in a more coherent and interoperable way across Europe
II: The existing e-Infrastructure projects must be superseded by integrated sustainable services at national and European levels
III: e-Infrastructures must be application-neutral and open to all user communities and resource providers. National funding agencies should be encouraged to fund multi-disciplinary and inclusive infrastructures rather than disciplinary-specific alternatives
IV: e-Infrastructures must inter-operate and adopt international standard services and protocols in order to qualify for funding
V: The Commission should, within the seventh Framework Programme, develop a pan-European e-Infrastructure which explicitly encourages the further integration of national e-Infrastructure initiatives
e-IRG Task Force on Sustainable e-Infrastructures (SeI) http://www.e-irg.org/publ/2006-Report_e-IRG_TF-SEI.pdf
The European Grid Initiative 24
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Grids in Europe
• Large European investment in developing Grid technology• Sample of National Grid projects:
– Austrian Grid Initiative– Belgium: BEgrid– DutchGrid – France: Grid’5000– Germany: D-Grid; Unicore– Greece: HellasGrid– Grid Ireland – Italy: INFNGrid; GRID.IT– NorduGrid– Portuguese Grid– Swiss Grid– UK e-Science: National Grid Service;
OMII; GridPP
• Multi-national, multi-science Grid infrastructures are a priority of the EC:– DEISA, EGEE plus several supporting projects
The European Grid Initiative 25
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
“…for Grids we would like to see the move towards long-term sustainable initiatives less dependent upon EU-
funded project cycles”
Viviane Reding,
Commissioner,
European Commission,
at the EGEE’06 Conference, September 25, 2006
26
Broad scale test-beds
Production quality
facilities
Sustainable
e-Infrastructures
(utility model)
Sustainable grid/data-based e-
Infrastructures (utility model)
Towards sustainable grid-empowered e-Infrastructures
Slide courtesy of Kyriakos Baxevanidis, EC
27
The first e-Infrastructure Call in FP7
Year 2009Year 2009Year 2008Year 2008Year 2007Year 2007
DRAFT
1. e-Science Grid Infrastructures
2. Scientific Digital Repositories
3. Deployment of e-Infrastructures for new Scientific Communities
4. New Research Infrastructures – Design studies
5. New Research Infrastructures – Preparatory phase
6. Support measures (studies, policy initiatives,international co-operation,…)
Publication: early 2007Closure: spring 2007
•Support conceptual design studies for new RI (or major upgrades of existing ones) of clear European dimension and interest; such studies will help to assess technical and financial feasibility of proposed new RI
•Action should also foster emergence of new organisational models to consolidate a sustainable approach to e-Infrastructures, in particular in the domain of grids and data repositories
•New service provisioning schemes to be more neutral and open to all user communities and resource providers
Slide courtesy of Kyriakos Baxevanidis, EC
The European Grid Initiative 28
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
European Grid Initiative: Vision
• … organizes the management and operation of a reliable production Grid infrastructure (heterogeneous resources, independently managed and configured) for the European Research Area (ERA)
• ... interoperates with different e-Infrastructures around the globe (incl. HPC, …), provides a common abstraction layer to grid resources, contributes to Grid standardisation and policy efforts
• … is composed as a federation of National Grid Initiatives (NGIs) – inclusive to those interested to join, open to those unable to join at the moment – and supports the formation of new NGIs and the improvement of current NGIs
• … supports applications from diverse communities, is driven by the user communities, establishes and maintains links to business and industry, and disseminates knowledge about the Grid through training and education.
Towards a European Infrastructure for e-Sciencehttps://edms.cern.ch/file/766455/1/EGIOverview.pdf
The European Grid Initiative 29
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Structure
Federated model bringing together National Grid Initiatives (NGIs) to build a European organisation
Each NGI should be a national body• Recognised at the national level
– Important to be single point of contact– Organizations will differ; e.g. can be national coordination body
• Mobilises national funding and resources– Funding schemes will differ from country to country
• Operates the national e-Infrastructure• Supports user communities• Application independent, open to new user communities and resource
providers • Contributes and adheres to international standards and policies
The European Grid Initiative 30
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGI/NGI Responsibilities
• Responsibilities between NGI and EGI are split to be federated and complimentary
• Main role of EGI is to operate of the common multi-national, multi-disciplinary Grid infrastructure– Federation of the national infrastructures– Enables and supports international Grid-based collaboration– Provides support and adds value to NGIs– Does not repeat functions done at NGI level (no “super NGI”)– European co-ordination body– Liaises with corresponding infrastructures outside Europe
The European Grid Initiative 31
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
EGI Responsibilities in Detail
• The exact sharing of responsibilities between NGIs and EGI is an ongoing discussion – what follows reflects our current understanding– Further details can be found in the D-Grid use-cases document
• Operations management– Operational coordination on European level, provision of best practices, status
monitoring, training material, documentation repository etc.
• Policies– Definitions of policies between NGIs, SLAs, etc.
• Standards– Definition of standards or interoperable components that are needed to ensure
successful operation of the infrastructure
• Middleware testing and certification– Interoperability of different stacks will be required
– Provide build and test infrastructure to foster interoperability
– Provide source-forge like repository
– Initially provide a base distribution comprising certified components
• Dissemination and outreach
The European Grid Initiative 32
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Evolution
European e-Infrastructure
Testbeds Utility ServiceRoutine Usage
National
Global
EGIEuropean
GridInitiative
The European Grid Initiative 33
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Austrian Grid
EGIEuropean
GridInitiative
The European Grid Initiative 34
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Austrian Grid Phase II (2007-2009)
Goals of Phase 2:
1. Continued expansion of expertise of Austrian Grid scientists, both middleware and applications
2. Transition from Prototype-Testbed into sustainable, service-oriented Grid architecture, with seamless integration into European Grid Initiative
3. Extension of existing application areas and exploration of new application areas
4. Increased information- and trainings efforts for research and other institutions, for which grids could be relevant
5. Implementation of focus points for information exchange and cooperation with business and industry
The European Grid Initiative 35
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Austrian Grid Phase II - Structure
4 Components (distributed over Austria)• Research Centre
– Basic Grid Research: Middleware Extensions, Higher Levels of Grid Middleware
– Application Support: Grid Application Development and Enabling
• Development Centre– Contact point for Industry– User Information and Training
• Service Centre– Management of the Grid Infrastructure (Hardware) and
Middleware Distribution– Certification Authority
• E-Infrastructure Coordination Committee
The European Grid Initiative 36
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688
Summary
• The need for a European e-Infrastructure has been widely agreed
• The current short-term project based models are reaching their limits
• A model committing the National Grid Initiatives and building a central coordination organisation is proposed
your input and feedback is actively sought
• Such a scheme will ensure a sustainable e-Infrastructure for research, help maintain Europe’s leading position, and prepare for a global interoperable Grid infrastructure
• An EGI Design Project will be proposed in FP7 to evolve the current thinking to arrive at a consensus of its structure, tasks and funding