university of catania, engineering faculty department of computer science and telecommunications...

University of Catania, Engineering FacultyUniversity of Catania, Engineering Faculty

Department of Computer Science and TelecommunicationsDepartment of Computer Science and Telecommunications

Report about the Development of Report about the Development of

Scheduling Algorithm in the Resource BrokerScheduling Algorithm in the Resource Broker

S. Cavalieri and S. MonforteS. Cavalieri and S. Monforte

IntroductionIntroduction

State of the State of the implementatiimplementationon

Design IssuesDesign Issues

One ClassAd One ClassAd One LDAP One LDAP EntryEntry

All in OneAll in One

Final Final RemarksRemarks







Current GRIS/GIIS Information ArchitectureCurrent GRIS/GIIS Information Architecture

Information Services (i.e. grid-info-host, globus-version etc.) on GRISes publish Information Services (i.e. grid-info-host, globus-version etc.) on GRISes publish LDIF objects representing their resources in LDAP directories.LDIF objects representing their resources in LDAP directories.

None of these informations are sent to the GIIS.None of these informations are sent to the GIIS. The GIIS maintains a directory (grid-info-site-regdir) as an index of registered The GIIS maintains a directory (grid-info-site-regdir) as an index of registered

GRISes (reg.GRISes (reg.<port><port>..<hn><hn>.conf).conf)

GIISGIIS

GRIS gridct1.ct.infn.itGRIS gridct1.ct.infn.it



globus-versionglobus-version

grid-info-hostgrid-info-host

globus-gram-schedulerglobus-gram-scheduler







grid-info-site-regdirgrid-info-site-regdir

reg.2163.gridct1.ct.infn.it.confreg.2163.gridct1.ct.infn.it.conf

reg.2163.gridct2.ct.infn.it.confreg.2163.gridct2.ct.infn.it.conf

reg.2163gridct3.ct.infn.it.confreg.2163gridct3.ct.infn.it.conf





The Resource BrokerThe Resource Broker

Implementation of the scheduling functions performed by Implementation of the scheduling functions performed by the Resource Broker, is based on the following architecture:the Resource Broker, is based on the following architecture:

GIISGIIS

GRISGRIS

GRISGRIS

GRISGRIS

Resource Resource BrokerBroker


Resources Published Resources Published in LDAP treesin LDAP trees

ClassAd Description of ClassAd Description of GRISes ResourcesGRISes Resources

JDLJDLJDLJDL

ClassAd Description of ClassAd Description of Job RequirementsJob Requirements

Best Matching GRISBest Matching GRIS




State of the State of the ImplementatiImplementationon





Implemented Functions Implemented Functions

Search of all the available resources. Through the analysis Search of all the available resources. Through the analysis of the content of the directory “grid-info-site-regdir” where of the content of the directory “grid-info-site-regdir” where GIIS maintains the list of GRIS referring to it, information GIIS maintains the list of GRIS referring to it, information useful to localisation and access to each GRIS is obtained.useful to localisation and access to each GRIS is obtained.

For each GRIS that can be reached from the GIIS, For each GRIS that can be reached from the GIIS, information published are retrieved through LDAP.information published are retrieved through LDAP.

Collected resources are converted in terms of ClassAds Collected resources are converted in terms of ClassAds and maintained by an ad hoc data structure. and maintained by an ad hoc data structure.







Implemented Functions Implemented Functions

Two objects have been developed in order to provide an Two objects have been developed in order to provide an abstraction over the resources published in a LDAP abstraction over the resources published in a LDAP directory tree of a GRIS as well as regarding the directory tree of a GRIS as well as regarding the architecture designed for connecting GRISes and GIIS. architecture designed for connecting GRISes and GIIS.

A GRISResourceWrapper is instantiated for each GRIS A GRISResourceWrapper is instantiated for each GRIS supplying the GIISResourceWrapper with a ClassAd View supplying the GIISResourceWrapper with a ClassAd View of its resources.of its resources.

The GIISResourceWrapper provides for a coherent and The GIISResourceWrapper provides for a coherent and easy to query view over the entire site resources, ready to easy to query view over the entire site resources, ready to be used for matching.be used for matching.








What is the best design for the GRISResourceWrapper ?What is the best design for the GRISResourceWrapper ?

How many ClassAds should be mantained for each How many ClassAds should be mantained for each GRISResourceWrapper ?GRISResourceWrapper ?

Which attributes should be mantained ? Which attributes should be mantained ?

How and when a GRISResourceWrapper should be How and when a GRISResourceWrapper should be updated ? updated ?







What is the best design for the What is the best design for the GRISResourceWrapper ?GRISResourceWrapper ?

Three Solutions:Three Solutions:

One ClassAd, One LDAP EntryOne ClassAd, One LDAP Entry

One ClassAd for all the LDAP EntriesOne ClassAd for all the LDAP Entries

Nested ClassAd (not discussed in the presentation)Nested ClassAd (not discussed in the presentation)







One ClassAd, One LDAP EntryOne ClassAd, One LDAP Entry… …

A GRISResourceWrapper maintains a collection of ClassAds, one for A GRISResourceWrapper maintains a collection of ClassAds, one for each entry of the LDAP directory, which contains the resources each entry of the LDAP directory, which contains the resources published by a globus information service.published by a globus information service.

[ [ ……

schedulertype=”condor”schedulertype=”condor”

……]]

[ [ ……


……]]



[ …[ …

ostype=”linux”ostype=”linux”

……] ]

[ …[ …


……] ]


[ …[ …

releaseDate=2000/04/11releaseDate=2000/04/11

……] ]

[ …[ …


……] ] ClassAd View of ClassAd View of ResourcesResources








The ClassAd expression of the job requirements may contain attributes The ClassAd expression of the job requirements may contain attributes published by different information services, thus owned by different published by different information services, thus owned by different ClassAd of the collection the GRISResourceWrapper stores. ClassAd of the collection the GRISResourceWrapper stores.

[… […

requirements= requirements= other.ostype==”linux” && other.ostype==”linux” && other.schedulertype==”condorother.schedulertype==”condor

……]]

[… […


……]]

JobAdJobAd

[ [ ……


……]]

[ [ ……


……]]

[ …[ …


……] ]

[ …[ …


……] ]

[ …[ …


……] ]

[ …[ …


……] ]

GRISGRIS

ResourceAdsResourceAds








Since the MatchClassAd specialization performs a bilateral match Since the MatchClassAd specialization performs a bilateral match between two and only two ClassAds, it will return FALSE for every between two and only two ClassAds, it will return FALSE for every ClassAd describing the GRIS resources.ClassAd describing the GRIS resources.

GIISGIIS


Resource Resource BrokerBroker ????

[… […


……]]

[… […


……]]

JobAdJobAd

[ [ ……


……]]

[ [ ……


……]]

[ …[ …


……] ]

[ …[ …


……] ]

[ …[ …


……] ]

[ …[ …


……] ]

GRISGRIS

ResourceAdsResourceAds

No Such GRISNo Such GRISNo Such GRISNo Such GRIS







All in OneAll in One… …

The second approach is based on the use of just one ClassAd describing The second approach is based on the use of just one ClassAd describing all the resources published by a given GRIS. all the resources published by a given GRIS.

[ [ ……




……]]

[ [ ……




……]]

ResourceAdResourceAd











On the other hand, it should be kept in mind that two different entries of On the other hand, it should be kept in mind that two different entries of the LDAP directory might contain one or more attributes with the same the LDAP directory might contain one or more attributes with the same name, but different values.name, but different values.

Therefore, any new value for an attribute already inserted in the Therefore, any new value for an attribute already inserted in the ClassAd must appended, thus creating a list of values. ClassAd must appended, thus creating a list of values.


globus-gram-schedulerglobus-gram-schedulerCONDORCONDOR

[ [ ……releaseDate=2000/04/11releaseDate=2000/04/11schedulertype={”condor”,”lsf”}schedulertype={”condor”,”lsf”}ostype=”linux”ostype=”linux”……]]




globus-gram-schedulerglobus-gram-schedulerLSFLSF








Nevertheless,Nevertheless, the match between the ResourceAd and the the match between the ResourceAd and the JobAd does not work yet. JobAd does not work yet.

GIISGIIS



GRISGRIS

No Such GRISNo Such GRISNo Such GRISNo Such GRIS

[… […


……]]

[… […


……]]

JobAdJobAd






Design Issues Design Issues





The problem is within the description of the job requirements in The problem is within the description of the job requirements in ClassAd, that is, the operator == does not work when applied to ClassAd, that is, the operator == does not work when applied to an attribute whose value is a list. The member function must be an attribute whose value is a list. The member function must be rather used.rather used.

GIISGIIS



GRISGRIS

Match OkMatch OkMatch OkMatch Ok

[… […

requirements= other.ostype==”linux” && requirements= other.ostype==”linux” &&

Member(“condor”,other.schedulertype)Member(“condor”,other.schedulertype)

……]]

[… […

requirements= other.ostype==”linux” && requirements= other.ostype==”linux” &&

Member(“condor”,other.schedulertype)Member(“condor”,other.schedulertype)

……]]

JobAdJobAd




It should be pointed out that the JDL user must know which attribute It should be pointed out that the JDL user must know which attribute requires the member function for a feasible matching. requires the member function for a feasible matching.







Final RemarksFinal Remarks In the author’s point of view the best solution is to use only one In the author’s point of view the best solution is to use only one

ClassAd representing a subset of the more meaningful attributes, ClassAd representing a subset of the more meaningful attributes, rather than the entire attributes published.rather than the entire attributes published.

Problem to be discussed: what information is needed for Problem to be discussed: what information is needed for the Broker and who has to publish itthe Broker and who has to publish it

university of catania, engineering faculty department of computer science and telecommunications...

Documents

ldap entryone classad

ldap entriesnested classad

oneone classad

classad view

different classad

classad expression

ldap entryall

classad descr