10 april 2003deploy grid in israel universities1 deploy grid testbed in israel universities lorne...
Post on 18-Dec-2015
216 Views
Preview:
TRANSCRIPT
10 April 2003 Deploy Grid in Israel Universities 1
Deploy Grid testbed in Israel universities
Lorne Levinson
David Front
Weizmann Institute
10 April 2003 Deploy Grid in Israel Universities 2
The Grid Problem
• Seamless, flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources
• Enable virtual organizations (VO)s to share geographically distributed resources as they pursue common goals assuming the absence of:
central location, central control, trust relationships.
10 April 2003 Deploy Grid in Israel Universities 4
DataGRID Job submission
UIJDL
ResourceResourceBrokerBroker(Match requests to (Match requests to resources)resources)
Job SubmissionJob SubmissionServiceService (Condor G) (Condor G)
StorageStorageElementElement (SE) (SE)
ComputeComputeElement (CE)Element (CE)
Information Information ServiceService
Job Status
ReplicaReplicaCatalogueCatalogue
DataSets info
Author.&Authen.
Job S
ub
mit
Even
t
Job
Qu
ery
Job
Stat
us
Input “sandbox”
Input “sandbox” + Broker InfoGlobus RSL
Output “sandbox”
Output “sandbox”
Job Status
Pu
blis
h
grid
-pro
xy-in
it
Exp
and
ed J
DL
SE & CE info
Logging &Logging &Book-keepingBook-keeping
10 April 2003 Deploy Grid in Israel Universities 5
File replication
Read
StorageStorageElementElement
Information Information ServiceService
Replica Catalogue (RC)Replica Catalogue (RC)Translates logical FN to physical FNTranslates logical FN to physical FNAsk for a LFN
Ask for LFN
Return nam
e & protocol
Return PFN
Pub
lish
SE & CE info
Write Writes a file
RC only tells where files areIn order to get them, one may use Gridftp
10 April 2003 Deploy Grid in Israel Universities 7
Data Management future : Reptor: The Next Generation Replica Manager
Replica Manager
Replica Metadata
Replica Location
File Transfer
Optimization
Transaction
Consistency
Preprocessing
Postprocessing
Subscription
Client
Reptor
Giggle
RepMeC
Optor
GDMP
10 April 2003 Deploy Grid in Israel Universities 8
The Grid is NOT (yet)
• Not: Super computing– The trend: Many PCs rather than a big computer
• Not: Can not guarantee fast response, mainly because of network limitations
• Not (yet) : A total solution for distributed computing– Batch rather than interactive or stream oriented:
Examples: the grid is not geared to• Support games • Show a lecture on line
• Not (yet) : Ultimate parallel computing solution– Supported class: Embarrassingly Distributed (ED)
10 April 2003 Deploy Grid in Israel Universities 10
Assumptions
• Grid should be functional
• Users are scientists
• Prefer open source
• Run on Linux RedHat 7.3
10 April 2003 Deploy Grid in Israel Universities 11
How?
• Define priorities for required functionality and suppliers of relevant SW packages
• Support wide functionality scope. Each institution will select its subset
• Find best, up-to-date, reasonably tested, set of SW packages
• Honor Glue - USA-Europe collaboration • The actual content and versions will change
because SW is evolving• Stick to one SW provider that does exactly that
10 April 2003 Deploy Grid in Israel Universities 12
LHC Computing Grid - LCG
• LCG is the project that selects at tests computing grid SW for LHC experiments in CERN
• LCG will supply highly integrated Grid SW• Utilizes American and European Grid SW
• Honors GLUE: America-Europe collaboration
10 April 2003 Deploy Grid in Israel Universities 13
LCG main SW suppliers• Globus: A bag of Grid services
– GSI: Grid Security Infrastructure– GRAM: Grid Resource Allocation & Management– MDS: Monitoring and Discovery Service– GridFTP: file transfer over the net
• VDT:– Condor-G: Submit jobs and monitor job consistency
• EDG:– Resource Broker: Select on what Computing Element (CE) to submit job– Data replication– VOMS: Virtual Organization Management System– TBD: LCFG: Installation and configuration
• GLUE:– Make sure that USA and Europe grid SW work together
10 April 2003 Deploy Grid in Israel Universities 14
LHC Computing Grid - LCG
Software Delivery Process
Globus NMI VDT
EDG
LCG
LCGCertificationAnd Testing
LCG IntegrationVDT
+
+
USA Europe
10 April 2003 Deploy Grid in Israel Universities 15
LCG-1 Components 1/2003
VDT 1.1.6
Globus 2.0 + patches
ClassAds: 0.9.4
Condor: 6.4.7
Condor-G: 6.4.7
EDG Certificates 0.12-1
EDG CRL Update: 1.2.5-1
MDS 2.2 + patches + static GLUE schema
EDG mkGridmap
EDG 1.4.3
GDMP 3.2.6 ( to be migrated to 4.0)
Replica Catalogue Server ( to be migrated to RLS) edg-rc-server 3.1-2
Replica Manager edg-replica-manager
Replica Catalogue API /CLI ReplicaCatalogue 3.2.3 [4]
Resource broker: Workload Management (informationindex-1.2.9-1, jobsubmission-[glue-aware], lbserver-1.2.14-1, locallogger-1.2.12-1, proxy-1.2.8-1, userinterface-1.2.15-1)
LCG
lcg-version
Configuration solutions
DataTAG 1.0
GLUE Information Providers
Edt-monitor
10 April 2003 Deploy Grid in Israel Universities 16
LCG-1 Components 3-5/2003
VDT 1.x.x 3/2003
Globus 2.2.x
RLS pre-release
Condor: 6.4.7
Info Providers (partial Glue set)
EDG 2.0 3-5/2003
R-GMA x.x.x (To be evaluated)
Reptor x.x
RepMecC x.x
Resource Broker x.x (Glue)
VOMS x.x.x (To be evaluated)
LCG 3/2003
lcg-version
Test-suite
DataTAG 1.0 3/2003
Edt-monitor
Dilemma: Should the more advanced, but less tested version be used?
10 April 2003 Deploy Grid in Israel Universities 17
Grid functionality: SecurityFunctionality Supplied by Priority
Authentication:
Certificate Authority (CA):
Allocate user and host certificates
Europe, USA: Many certification authorities
Israel: Machba to be a certification authority
Essential
single sign-on Globus: Grid Security Infrastructure (GSI)
Essential
Certificate delegation Globus: GSI Essential
Authorization:Virtual Organization (VO)
management
Globus: Mapfile - Map local users to grid users
EDG mkGridmap
EDG 2.0: VOMS
High
Firewall:
Submit grid jobs securely through firewalls
Not supplied yet High
10 April 2003 Deploy Grid in Israel Universities 18
Grid functionality: Job submission
Functionality Supplied by Priority
Job submission Globus:
Grid Resource Allocation & Management (GRAM)
Essential
Job consistency Condor-G Very high
Resource Broker (RB):
Select a Computing Element (CE) for job submission
EDG (Glue) The bigger the number of CE’s,
The more a RB is needed
Scheduling policy:
Control local resource allocation according to policy
Not supplied yet High
Job dependencies EDG 2.0
based on Condor DAGMan
Support interactive jobs EDG 2.0
10 April 2003 Deploy Grid in Israel Universities 19
Grid functionality: Data management
Functionality Supplied by Priority
Copy files over the net Globus: GridFTP Essential
Data mirroring EDG: GDMP Very high
Replica
Catalog+ management
EDG:
Replica catalog
EDG 2.0: reptor
The bigger the number of (big) files to be read at many sites,
The more replica management is needed
Virtual Data Not available
10 April 2003 Deploy Grid in Israel Universities 20
Grid functionality: Information services
Functionality Supplied by Priority
Information services
LDAP based
Globus:
Monitoring and Discovery Service (MDS).MDS resource information models physical and logical components of a compute resource as a hierarchy of elements, using LDAP
Essential
RDB based EDG 2.0: R-GMA Essential
Job monitoring Not available? High
10 April 2003 Deploy Grid in Israel Universities 21
Grid functionality: Graphical User Interface (GUI)
Functionality Supplied by Priority
Graphical (web based) User Interface (GUI)
• Genius:EDG simple portal for job submission
• EDG 2.0 will have more GUI
High
10 April 2003 Deploy Grid in Israel Universities 22
Links
Globus Project™– www.globus.org
Global Grid Forum– www.gridforum.org
European Data Grid EDG– http://eu-datagrid.web.cern.ch/eu-datagr
id
/
top related