![Page 1: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/1.jpg)
Report from Amsterdam
Massimo SgaravattoINFN Padova
![Page 2: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/2.jpg)
Amsterdam events Global Grid Forum (GGF) 1 DataGrid workshop
![Page 3: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/3.jpg)
Global Grid Forum Focuses on the promotion and development of
Grid technologies and applications via the development and documentation of "best practices," implementation guidelines, and standards
Global GF efforts are also aimed at the development of a broadly based Integrated Grid Architecture that can serve to guide the research, development, and deployment activities of the emerging Grid communities
www.globalgridforum.org
![Page 4: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/4.jpg)
Global Grid Forum History
GF BOF (Orlando)
GF1 (San Jose — NASA Ames)
GF2 (Chicago — iCAIR)
eGrid and GF BOFs (Portland)
GF3 (San Diego — SDSC)
eGrid1(Posnan — PSNC)
GF4 (Redmond — Microsoft)
eGrid2 (Munich — at Europar)
GF5 (Boston — Sun)
Global GF BOF (Dallas)
1999 2000
Asia-Pacific GF Planning (Yokohama)
![Page 5: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/5.jpg)
Global Grid Forum 2001
GGF1 (Amsterdam) March 4-7, 2001w/ DataGrid “Kickoff” (March 7-9)Hosts: Amsterdam Science and Technology Center(SARA, Univ. Amsterdam, NIKHEF)
325 participants192 organizations28 countries
GGF2 (Washington, DC)July 15-18, 2001
GGF3 (Rome)October 7-10, 2001W/ DataGrid workshop (October 2-5)Frascati (Rome)
2001
![Page 6: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/6.jpg)
Global GF Working Groups Account Management Applications and
Testbeds Grid Information
Services Remote Data Access Security Advanced
Collaborative Environments
Grid Protocol Architecture
Advanced Programming Models
Grid Computing Environments
Grid Performance Scheduling User services JINI
![Page 7: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/7.jpg)
How to Be Involved Electronic Participation
Each working group uses email distribution lists for discussions
Contributing to Grid Working Drafts Volunteer to help write a planned paper Submit an abstract/outline to propose a paper
Architectures / Models API, Protocol, or other specification Survey or Current Practice Description
Lead a Group Volunteer to start a working group or a sub-
group within an existing working group
![Page 8: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/8.jpg)
Scheduling & Resource Management WG Charter: generate best practice scheduling
and resource management documents and API specifications to enable interoperability Co-chairs: Jenny Schops, Bill Nitzberg, Uwe
Schwiegelshohn Web site:
http://www.cs.nwu.edu/~jms/sched-wg Mailing list: [email protected] General meetings at Grid forum events and
teleconferences
![Page 9: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/9.jpg)
Scheduling WG @ GGF 1 3 documents discussed
10 actions for superscheduling Goal: list the basic steps and capabilities involved in
resource reservation, acquisition and use Basically done
Advanced reservation API Just an API because not yet advanced for a protocol Basically done
Attributes for communication about scheduling instances
Goal: define a set of attributes that a higher level scheduler can ask of a lower level scheduler
In progress
![Page 10: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/10.jpg)
Scheduling WG @ GGF 1 Suggested new documents
Advance reservation protocol Allow advance reservation between systems
Scheduling dictionary List of tokens and definitions to coordinate with
the dictionary group (common glossary of important terms across the Grid forum)
Already started (terms only, no definitions) (www.sdsc.edu/~mthomas/GF/terms/sched-terms.cgi)
Document to move this effort forward
![Page 11: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/11.jpg)
Distributed Accounting WG Purpose
Develop a prototype Grid accounting architecture such as is required for resource economies
Define interactions with other Grid components and existing systems
Create a specification that can be used to build and evaluate prototype systems
Co-chairs: Tom Hacker, William Thigpen Web site: http://www.nas.nasa.gov/~thigpen/accounts-wg Mailing list: [email protected] Working drafts we may be interested in:
Distributed accounting on the Grid An economy Grid architecture for service-oriented grid
computing
![Page 12: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/12.jpg)
GGF and WP 1 Should we/how do we contribute to
the GGF effort ? Dictionary ? Accounting ? … ?
![Page 13: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/13.jpg)
DataGrid ATF Progress Report Joint Parallel meetings
WP 1-5 (Middleware) WP 6-7 (Testbed) WP 8-10 (Applications)
![Page 14: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/14.jpg)
ATF Progress Report Documents
(http://gridatf.web.cern.ch/grid-atf/documents.html) DataGrid Architecture version 1 Month 9 release
We have to provide reactions Implementation plan for M9 release by
the next PTB (April 19)
![Page 15: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/15.jpg)
WP 1 –5 Information Services
Relational vs. LDAP Scope
WP 3 is going to provide only APIs (for producers and consumers) and interfaces
What will be there in M9 ? Current Globus GIS implementation (WP 3) WP 1: many shortcomings and open issues (performances,
incomplete schema,…) in the current Globus GIS implementation, and therefore it is probably unsuitable
Discussion with S. Tuecke (Globus team) Aware of problems We may use/test Globus alpha releases
Workshop (Convenior WP 3 ) April 28,29 @ Cern
![Page 16: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/16.jpg)
WP 1 - 5 Job
Partition/Checkpoint of jobs ? At GRID level but also inside fabric ? Applications have to help us, otherwise it is
difficult how to split/checkpoint jobs Job input/output
Files ? Objects ? …? Is the input/output known at schedule time ? Scheduling policies ?
Deadline? Best fit ?, …? Applications should provide their view
![Page 17: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/17.jpg)
WP 1 –5 Storage Model
Boundaries between WP2, WP4, WP 5 Global vs. local filenames Who does the house keeping … Workshop: Convenior WP 2
March 27, 2001 @ Cern ? Accounting
Economic model (WP 1) ? WP 1 to investigate and circulate thoughts
![Page 18: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/18.jpg)
WP 6 –7 Testbed organization
Permanent Group (partner representatives, technical committee, site contacts, testbed support)
Release team (one team for each release validation phase)
Working model WP test activity (WPx) WP release (WPx + testbed +appl.) Testbed release (WPs + testbed + appl.) Official testbed release (+ users)
Waiting for test requests
![Page 19: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/19.jpg)
WP 6 –7 Testbed 0
Grid basic services deployment, based on Globus toolkit
Resources Mainly Linux machines (also Suns, SGI) LSF, Condor, PBS, BQS > 100 people > 40 sites
Main issues Information Services Resource access (certification, security and policies
management) Installation toolkit
![Page 20: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/20.jpg)
GIS Topology EU GIIS
o=datagrid0
INFN (Italy)dc=infn,dc=it,o=datagrid0
Francedc=fr,o=datagrid0
LIP (Portugal) dc=lip,dc=pt,o=datagrid0
...
dc=in2p3,dc=fr,o=datagrid
... ...... ...
(root server at Cern)
–Support for GIS deployment in place
(Top level server)
![Page 21: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/21.jpg)
WP 6 –7 Globus installation toolkit
3 toolkits available (INFN, UK, FR) Working group to converge on common tools and common
environment (within 3 weeks) Security
CA group (chaired by D. Kelsey) already in place Support for certificates signed by the various national CA’s
Access policies for Testbed 0 Globus grid-mapfiles Only one account per certificate (no group accounts) Requests for accounts for Testbed 0 via WP managers to WP6
General security group (chaired by D. Kelsey) is going to set up Security Workshop (29, 30 March @ Cern, Convenior: D. Kelsey)
Representatives from WP’s Security experts/officers/reps from Testbed sites Other interested people
![Page 22: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/22.jpg)
WP 8 - 10
OS & Net services
Bag of Services (GLOBUS)
DataGRID middlewarePPDG, GriPhyn, EuroGRID
HEPVO common application layer
Earth Obs. Biology
ALICE ATLAS CMS LHCb
Specific application layer
WP9 WP 10
WP8-9-10 TWG
![Page 23: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/23.jpg)
WP 8 – 10 User cases for testbed 0 & 1
Basically ready for WP 8 Available for WP 9
User cases for testbed 2 & 3 Draft in one month for WP 8
Common (WP8-10) user requirements First draft in one month First release at M6
![Page 24: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/24.jpg)
WP 8 –10 Preliminary reactions to ATF
Objects or files when interacting with the middleware ? Access to middleware services from applications via API
(C++ API) RW databases Interactive jobs
They will propose use cases Required job resources unknown at schedule time Middleware portability
Linux and Solaris is not enough The code should be written to run on any machine with a
reasonable cc/java MPI and Corba Worries about the use of Condor (open source)
![Page 25: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/25.jpg)
Other issues (not tackled) Grid global queues or local queues Gateway functionality
Sharing of WP1 and WP4 tasks on the gateway Language translation Translation of Grid credentials to local credentials
WP4: Will other WPs only query for information of the current state or is there a need to query for future changes ?
Central middleware repository Coding convention and tools …
![Page 26: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/26.jpg)
Informal discussions WP 2
Agent-based query optimization WP 4
For month 9 they basically agree to consider queues as resources
Resource monitoring info We are consumer, they are producer We have to specify the attributes needed for M 9
release, and they will tell us if they can implement the corresponding information providers
… depends on what exactly WP 3 plans to deliver concerning information and monitoring service
![Page 27: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/27.jpg)
Condor In the first year we are allowed to
use the Condor stuff Then we can rely only on open
source software Probably possible for the Condor
matchmaking library More complicated for Condor-G
Possibility to have the source code
![Page 28: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/28.jpg)
Condor: what’s new New Condor grid-manager and old Globus
job manager (basically ready) Fixes for existing shortcomings (scalability in
the submitting machine, logging, …) Globus GRAM client APIs
New Globus job manager (in testing phase) 2-phase submit protocol Support for reattaching to an existing job
New Condor grid-manager (4-6 weeks) Able to exploits the new Globus job manager
![Page 29: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/29.jpg)
Agenda DataGrid topical (IS, DM, Security) workshops at
CERN (week of March 26) DataGrid technical visit to Globus at ANL April 5-7 Agent technology workshop in Trento (ITC/IRST)
May 17 INET 2001 5-8 June in Stockholm EuroGlobus 16-22 June EU-US coordination in Rome on 23/6/2001 Next project conference at Oxford 2-5 of July GGF2 in Washington 15-18 of July HPDC 7-10 of August in the US
![Page 30: Report from Amsterdam Massimo Sgaravatto INFN Padova](https://reader035.vdocument.in/reader035/viewer/2022081519/56649e885503460f94b8d72a/html5/thumbnails/30.jpg)
Agenda CHEP2001 in Beijing (Grid sessions) 3-7 of
September CERN school of Computing (Grid session) 16-29
September in Santander, Spain 6th Globus retreat and iDG-1 17-21 of September in
Chicago DataGrid+GGF3 in Frascati+Rome 2-5 + 7-10 of
October IST2001 in Brussels end of October-early November
(DataGrid booth, presentations and demos?) SC’2001 10-16 November (DataGrid participation?)