june 1, 2015 1 inter-operating grids through delegated matchmaking alexandru iosup, dick epema pds...
Post on 18-Dec-2015
213 views
TRANSCRIPT
April 18, 20231
Inter-Operating Grids through Delegated MatchMaking
Alexandru Iosup, Dick EpemaPDS Group, TU Delft, NL
Todd Tannenbaum, Matt Farrellee, Miron LivnyCS Dept., U. Wisconsin-Madison, US
April 18, 20232
Outline
1. Grid Inter-Operation: Motivation and Goals• Evaluation of the e-Science Computational
Demand• Why Grid Inter-Operation?• The Grid Inter-Operation Research Question
2. Alternatives to/for Grid Inter-Operation3. Inter-Operating Grids Through Delegated
MatchMaking4. Experimental Results5. Conclusion and Future Work
April 18, 20233
Current e-Science Computational Demand
For Every Grid (Cluster), Over 500k Jobs/Year
A. Iosup, H. Li, M. Jan, S. Anoep, C. Dumitrescu, L. Wolters, D.H.J. Epema, The Grid Workloads Archive, 2007 (submitted to FGCS).
The Grid Workloads ArchiveThe Grid Workloads Archivehttp://gwa.ewi.tudelft.nlhttp://gwa.ewi.tudelft.nl
>500 kjobs/year/trace1.5 yrs >525K
April 18, 20235
Current e-Science DemandBursty Demand Leads to High Wait Time
A. Iosup, C. Dumitrescu, D.H.J. Epema, H. Li, L. Wolters, How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications, Grid 2006.
April 18, 20236
The 1M-CPU Machine with Shared Resource Ownership
• The 1M-CPU machine• E-Science (high-energy physics, earth sciences,
financial services, bioinformatics, etc.)• Over-provisioning for any individual e-Science field• Provisioning for all e-Science fields at the same time
• Shared resource ownership• Shared resource acquisition• Shared maintenance and operation• Summed capacity higher (more efficiently used) than
sum of individual capacities
April 18, 20237
How to Build the 1M-CPU Machine with Shared Resource Ownership?• The number of clusters increases at high
pace• Top500 SuperComputers: cluster systems from 0% to
75% share in 10 years (also from 0% to 50% performance)
• CERN WLCG: from 100 to 300 clusters in 2½ years
Source: http://goc.grid.sinica.edu.tw/gstat/table.html
clusters
MPPs
April 18, 20238
How to Build the 1M-CPU Machine with Shared Resource Ownership?Cluster size distribution over time, Top500, 1997-2007
1
10
100
1000
10000
100000
Nov-97
May-98
Nov-98
May-99
Nov-99
May-00
Nov-00
May-01
Nov-01
May-02
Nov-02
May-03
Nov-03
May-04
Nov-04
May-05
Nov-05
May-06
Nov-06
May-07
Date
Clu
ster
siz
e
Median Average Q1 Q3 Max
Median: 10x
Average: 20x
Max:100x
Last 10 years
Data source: http://www.top500.org
Last 4 years
Now:<1.2x/yr
To build the 1M-CPU cluster:To build the 1M-CPU cluster:- At last 10 years rate, another 10 years- At last 10 years rate, another 10 years
- At current rate, another 40 years- At current rate, another 40 years
April 18, 20239
How to Build the 1M-CPU Machine with Shared Resource Ownership?• CERN’s WLCG cluster size over time
Median: +5 procs/yr
Avg: +15 procs/yr
Max: 2x/yrShared clusters grow on average Shared clusters grow on average
slower than Top500 cluster systems!slower than Top500 cluster systems!
Data source: http://goc.grid.sinica.edu.tw/gstat/
Year 1 Year 2
April 18, 202311
How to Build the 1M-CPU Machine with Shared Resource Ownership?• Number of clusters is growing, cluster size is not• Many small clusters in
one large distributed computing system• 6,000 clusters = 1M CPUs / 150 CPUs/cluster [CERN]• 30,000 clusters = 1M CPUs / 32 CPUs/cluster [Kee et al.,SC04]
How to Inter-Operate 10,000s of Clusters?
• Inter-operate 10,000s of clusters with grids• But the largest grid has 300 clusters, most grids
have 2-3…
How to Build the 1M-CPU Machine with Shared Resource Ownership?
Research Question:Research Question:How to inter-operate cluster-based gridsHow to inter-operate cluster-based grids
in a scalable and efficient way?in a scalable and efficient way?
April 18, 202312
Outline
1. Grid Inter-Operation: Introduction, Motivation, and Goals
2. Alternatives to/for Grid Inter-Operation3. Inter-Operating Grids Through Delegated
MatchMaking4. Experimental Results5. Conclusion and Future Work
April 18, 202313
Alternatives to/for Grid Inter-Operation
Independent Centralized
HierarchicalDecentralized
Condor
Globus GRAM Alien
Koala
OAR
CCS
Moab/Torque
OAR2
NWIRE
OurGrid
Condor Flocking
Load imbalance?
Resource selection? Scale?
Root ownership?
Node failures?
Accounting?Trust? Scale?
April 18, 202314
Outline
1. Grid Inter-Operation: Introduction, Motivation, and Goals
2. Alternatives to/for Grid Inter-Operation3. Inter-Operating Grids Through
Delegated MatchMaking• Architecture• Mechanism
4. Experimental Results5. Conclusion and Future Work
April 18, 202315
3
3
3
333
2
3. Inter-Operating Grids Through Delegated MatchMaking
The Delegated MatchMaking Architecture
1. Start from a hierarchical architecture2. Let roots exchange load3. Let siblings exchange load
Hybrid hierarchical/decentralized Hybrid hierarchical/decentralized architecture for grid inter-operationarchitecture for grid inter-operation
April 18, 202316
3. Inter-Operating Grids Through Delegated MatchMaking
The Delegated MatchMaking Mechanism
1. Deal with local load locally (if possible)
Local load lowResource request
Resource usage rights Cluster
April 18, 202317
3. Inter-Operating Grids Through Delegated MatchMaking
The Delegated MatchMaking Mechanism
1. Deal with local load locally (if possible)2. When local load is too high, temporarily bind resources from
remote sites to the local environment. • May build delegation chains. • Delegate resource usage rights, do not migrate jobs.
3. Deal with delegations each delegation cycle (delegated matchmaking)
Delegate
Local load too high
Resource request
Resource usage rights
Bind remote resourceThe Delegated MatchMaking Mechanism=The Delegated MatchMaking Mechanism=
Delegate Resource Usage Rights, Delegate Resource Usage Rights, Do Not Migrate JobsDo Not Migrate Jobs
April 18, 202320
Outline
1. Grid Inter-Operation: Introduction, Motivation, and Goals
2. Alternatives to/for Grid Inter-Operation3. Inter-Operating Grids Through Delegated
MatchMaking4. Experimental Results
• Experimental Setup• Performance Evaluation• Overhead Evaluation
5. Conclusion and Future Work
April 18, 202321
4. Experimental Results
Experimental Setup• Inter-operating DAS and Grid’5000: 20 clusters, >3000
processors
• Discrete event simulation• Independent (separated clusters+FCFS, Condor+MM)• Centralized (CERN+poll, centralized grid scheduler+WF+FCFS)• Decentralized (Condor with flocking+MM+FS)
• Workloads• Real traces• Realistic
DAS+Grid’5000 with DMM
• Metrics• WT, RT, SD, Goodput• FinishedJobs[%],
O’head
April 18, 202323
• DMM• High goodput• Low wait time• Finishes all jobs
• Even better for load imbalance between grids
• [see paper]
4. Experimental Results
Performance Evaluation
Independent
Centralized
Decentralized
DMM
The DMM delivers The DMM delivers good performancegood performance
April 18, 202324
4. Experimental Results
Overhead Evaluation
• DMM• Overhead ~16%• 93% more control messages• Constant number of
delegations per job until 80% load
• DMM Threshold to control o’head.
• [see paper]The DMM incurs reasonable overheadThe DMM incurs reasonable overhead
April 18, 202325
Outline
1. Grid Inter-Operation: Introduction, Motivation, and Goals
2. Alternatives to/for Grid Inter-Operation3. Inter-Operating Grids Through Delegated
MatchMaking4. Experimental Results5. Conclusion and Future Work
April 18, 202326
Conclusion and Future Work
• The Delegated MatchMaking architecture, mechanism, and policies
How to inter-operate cluster-based gridsHow to inter-operate cluster-based gridsin a scalable and efficient way?in a scalable and efficient way?
Evaluation of DMMEvaluation of DMM• High goodput• Low wait time• Reasonable overhead
Future WorkFuture Work• Fault-tolerant policies (built resource availability model: Grid’2007)• Larger systems (we promised the 1M-CPU machine)• Malicious participants, trust • Real environment evaluation (built testing tool: GrenchMark, CCGrid’06)
ContributionsContributions• Hybrid architecture• Delegate resource usage rights• Framework for policy investigation
April 18, 202327
Thank you! Questions? Remarks? Observations?
The Grid Workloads Archivehttp://gwa.ewi.tudelft.nl/ (or Google “The Grid Workloads
Archive”)
ContactContact
• http://www.pds.ewi.tudelft.nl/~iosup/ (or Google “iosup”)
Share your Job and Resource Availability Traces!Share your Job and Resource Availability Traces!