joint aginfra and sci-bus workshop & aginfra open stakeholder day, 30 may 2013 - sztaki, hungary...
TRANSCRIPT
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid and Cloud Infrastructures for Agriculture
Antun BalazInstitute of Physics Belgrade, Serbia
Joint agINFRA and SCI-BUS workshop &agINFRA Open Stakeholder Day
30 May 2013, SZTAKI, Budapest, Hungary
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Overview
• agINFRA infrastructure• Hardware resources• Grid resources• Scientific Gateways• agINFRA RESTful gateway• Key results
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
agINFRA Infrastructure
• Technically divided into three main layers:– Hardware resources (Grid and Cloud)– Scientific Gateways (Catania SG, WS-PGRADE SG, RESTful gateway)– Integrates services (Organic.Edunet, AGRIS, CIARD RING, AgLR-TF)
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Hardware resources• Resource centers
– PARADOX cluster – 88 x (2 x 4-core Xeon [email protected] GHz) = 704 CPU (1 GB of RAM / core)50 TB of disk space, Gigabit Ethernet
– Roma Tre cluster – 350 CPUs; 100TB– Catania cluster – 800 CPUs; 700 TB – SZTAKI cluster – 8 CPUs– PARADOX ugrade – 106 x (2 x 8-core Xeon [email protected] GHz)
= 1696 CPU (2 GB of RAM / core)100 TB of disk space, Infiniband
• Virtualized resources– SZTAKI Desktop Grid
90 000 volunteer resources; 3 Tflops.– EDGeS@home - 15 000 CPUs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid resources [1/2]
• agINFRA Virtual Organization: vo.aginfra.euhttps://voms.ipb.ac.rs:8443/voms/vo.aginfra.eu https://voms.ct.infn.it:8443/voms/vo.aginfra.eu
• MyProxy service (PX) – myproxy.ipb.ac.rs
• Computing Elements (CREAM)– cream.ipb.ac.rs:8443/cream-pbs-aginfra – 704 CPUs– ce-01.roma3.infn.it:8443/cream-pbs-grid – 328 CPUs– ce-02.roma3.infn.it:8443/cream-pbs-grid – 208 CPUs– grid012.ct.infn.it:8443/cream-lsf-aginfra – 20 CPUs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Grid resources [2/2]• Logical File Catalogue (LFC)
– lfc.ipb.ac.rs• Storage Elements (SE) – 850 TB
– dpm.ipb.ac.rs– prod-se-02.ct.infn.it– storm-01.roma3.infn.it
• Metadata Catalogue (AMGA)– amga.ct.infn.it
• Workload Management System (WMS)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs
• Logging and Bookkeeping (LB)– wms.ipb.ac.rs– wms-demo.ipb.ac.rs– wms-aegis.ipb.ac.rs
• Information System (BDII)– bdii.ipb.ac.rs
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Scientific Gateways
• Catania Scientific Gatewayhttps://aginfra-sg.ct.infn.it/
• gUSE/WS-PGRADE Portal (workflows)http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
agINFRA RESTful gateway• Cataloging
– User’s configurations of Grid-ported applications– Datasets’ metadata information and locations– Arbitrary user-defined metadata information– Querying of these information
• Off-line processing– Collect new datasets – Transform existing datasets– Produce new datasets’ metadata information
• Data management– Automatic replication of datasets– Ensures existence of the same dataset on different storage systems– Exposure through the HTTP protocol
Joint agINFRA and SCI-BUS Workshop & agINFRA Open Stakeholder Day, 30 May 2013 - SZTAKI, Hungary
FP 7-INFRASTRUCTURES
Key results• Design, specification and deployment of the agINFRA infrastructure• Establishment and operation of agINFRA Grid infrastructure (VO:
vo.aginfra.eu, VOMS, BDII, LFC, WMS/LB, PX, AMGA, DPMs, CREAMs)• Establishment and deployment of Cloud infrastructure (CLEVER and
Okeanos)• Deployment of agINFRA Scientific Gateways (Catania and gUSE/WS-
PGRADE SGs)• Development of agINFRA RESTful gateway• Development and deployment of data infrastructure
– dataset exposure layer (RESTful gateway)– data management layer (Grid RESTful API, HTTP exposure)– data transformation layer (agDCtoLOM, agLOMtoAK, agLOMtoRDF)– data processing layer (agDataHarvest, agMetaExtract,
agRecommender)