eu 2nd year review – 04-05 jan. 2003 – title – n° 1 wp1 speaker name (speaker function and wp...
TRANSCRIPT
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 1
WP1
Speaker name (Speaker function and WP )
speaker@
Presentation address e.g. https://edms.cern.ch/document/366530
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 2
Review of WP1 architecture
WP1 WMS architecture reviewed To apply the “lessons” learned and addressing the problems emerged with the
first release of the software, deployed in the EDG testbed E.g. reduce of persistent job info repositories, delegate some functionalities to
pluggable modules, etc.
To increase the reliability of the system To support new functionalities:
User APIs Job checkpointing
Shown at the WP1 demo Job partitioning Interactive jobs Dependencies of jobs Parallel jobs Grid Accounting Resource reservation and co-allocation
To make easier the interoperability with other Grid frameworks
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 3
WP1 reviewed architecture
Details in D1.4 …
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 4
Job submission example (for a “simple” job)
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 5
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
edg-job-submit myjob.jdlMyjob.jdl
JobType = “Normal”;Executable = "$(CMS)/exe/sum.exe";InputData = "LF:testbed0-00019";ReplicaCatalog = "ldap://sunlab2g.cnaf.infn.it:2010/rc=WP2 INFN Test Replica Catalog,dc=sunlab2g, dc=cnaf, dc=infn, dc=it";DataAccessProtocol = "gridftp";InputSandbox = {"/home/user/WP1testC","/home/file*”, "/home/user/DATA/*"};OutputSandbox = {“sim.err”, “test.out”, “sim.log"};Requirements = other. GlueHostOperatingSystemName == “linux" && other. GlueHostOperatingSystemRelease == "Red Hat 6.2“ && other.GlueCEPolicyMaxWallClockTime > 10000;Rank = other.GlueCEStateFreeCPUs;
submitted
Job Status
UI: allows users to access the functionalitiesof the WMS
Job Description Language(JDL) to specify job characteristics and requirements
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 6
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Input Sandboxfiles
Job
waiting
submitted
Job StatusNS: network daemon responsible for acceptingincoming requests
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 7
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
WM: responsible to takethe appropriate actions to satisfy the request
Job
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 8
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-maker
Where does thisjob must be executed ?
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 9
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-Maker/ Broker
Matchmaker: responsible to find the “best” CE where to submit a job
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 10
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-Maker/ Broker
Where are (which SEs) the needed data ?
What is thestatus of the
Grid ?
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 11
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
Match-maker
CE choice
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 12
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
waiting
submitted
Job Status
JobAdapter
JA: responsible for the final “touches” to the job before performing submission(e.g. creation of wrapper script, etc.)
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 13
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Job Status
JC: responsible for theactual job managementoperations (done via CondorG)
Job
submitted
waiting
ready
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 14
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
CE characts& status
SE characts& status
RBstorage
Job Status
Job
InputSandboxfiles
submitted
waiting
ready
scheduled
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 15
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
InputSandbox
submitted
waiting
ready
scheduled
running
“Grid enabled”data transfers/
accesses
Job
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 16
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandboxfiles
submitted
waiting
ready
scheduled
running
done
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 17
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandbox
submitted
waiting
ready
scheduled
running
done
edg-job-get-output <dg-job-id>
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 18
Job submission
UI
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ReplicaCatalog
Inform.Service
ComputingElement
StorageElement
RB node
RBstorage
Job Status
OutputSandboxfiles
submitted
waiting
ready
scheduled
running
done
cleared
EU 2nd Year Review – 04-05 Jan. 2003 – Title – n° 19
Job monitoring
UI
Log Monitor
Logging &Bookkeeping
NetworkServer
Job Contr.-
CondorG
WorkloadManager
ComputingElement
RB node
LM: parses CondorG logfile (where CondorG logsinfo about jobs) and notifies LB
LB: receives and stores job events; processes corresponding job status
Log ofjob events
edg-job-status <dg-job-id>
Job status