MONITORING, ACCOUNTING AND REGISTRATION SERVICES FOR RUSSIAN
GRID NETWORKq
Sergey Belov, Tatiana Goloskokova, Vladimir Korenkov, Nikolay Kutovskiy, Danila Oleynik, Artem Petrosyan, Roman
Semenov, Alexander Uzhinskiy LIT JINR
The 5th International Conference "Distributed Computing and Grid-technologies in Science and Education“
(GRID’2012)20 July 2012, JINR, Dubna, Russia
RGN projectRussian National Grid Network
Development of grid-based networks for high performance computing
To create a network infrastructure and connection to it of largest supercomputer centers, enterprises, high-tech industries and research organizations
Computing power for high-priority industrial tasks and scientific projects
2010 – start of network backbone’s development2011 – core grid services development, application’s
interfaces, grid infrastructure prototype2012 – further system development, more
supercomputers, more applications
2/14
Grid and network infrastructureMain aim
Integration of unique, medium and small supercomputers into a unified distributed computing environment
Development of high-speed backbone network for the infrastructure
Provide secure and clear remote access for users to these resources
Heterogeneous grid environment (hardware, software)
Oriented to parallel tasks rather than single batch tasks
Workflow management Jobs consist of tasks
Technical points is a set of grid services with unified interfacespartially based on Globus Toolkit 4, VOMSGSI based security model, mainly RESTful grid services
3/14
Core grid servicesWeb User InterfaceResource Broker/metascheduler + Workflow
management (RESTful)Grid gateway to the resources (WS)Information Service (RESTful)Monitoring & accountingRegistration service (RESTful)GSI services: CA, MyProxy, VOMSGridFTP serversNon-grid: applications licensing service, billing
and audit, application solvers images management
4/14
Goals of monitoringTracking state of sites and services
AvailabilityReal operational state
Monitoring of user's jobs and tasks Keeping history on different system's
parameters Information representation
General infrastructure state in wholeRunning jobs and tasksSeparate sites and services (real-time and history)Visualization of job events
5/14
Monitoring of resources and servicesState of computational resources by site
(based on data from information indexes)Slots available for tasksStructure and properties of clusters
Subclusters, nodes, slots, operation system, architecture
Application softwareSupported VOs (with ACLs, Access Control
Lists)Monitoring of jobs running on sites (by
information from Job management servers)6/14
Accounting and job monitoringGoal: to get information, both real-time and
historical, on resources utilization and jobs running on RGN infrastructure (by users, VOs, sites)
Information sources: JMS servers, Grid Gateways and local resources managers
Collecting data on jobs and tasks in the systemAll jobs events timestamps, real consumed CPU time
Accounting information reports in different views:by sites, VOs and single users
Aggregation of actual job's execution time from all sites
7/14
Monitoring and accounting information flows
8/14
Monitoring andaccounting
datastorage
Informationcollector
JMSJob
managementservices
Monitoringwebsite
Monitoring dataprovisioning
(Web Services)
AccountingInformationpublisher
Infosyscentral
Informationindex
Registrationservice
Accounting and jobs monitoring screenshots
9/14
Resource centers on the mapMain sites of the
infrastructure prototype (2011)
Other supercomputers are in process of connection and testing
Will be more supercomputers from different Russian regionsby the end of 2012
10 resource centers are already connected to high-speed VPN
10/14http://grid.voskhod.ru
Infrastructure operation visualization with Google Earth
11/14
Visualization package from the ARDA Dashboard project was used
Registration serviceService for registration of resources and grid servicesMain task is to store and provide information about
resources and grid services in RGNSite contacts and other formal informationServices’ endpoints, type, DN, etc.Resources operational state
Registration and information modification procedures support
Web-interface for users and RESTful interface for system services
Authorization and authentication along with rights and roles of access to add, modify or access information
Registration service’s web interface
Conclusion and resultsWere developed architecture and software
solution for monitoring , accounting and registration service for the Russian National Grid Network
Experience of developing of the similar services within GridNNN project was very helpful and was heavily used
Created tools were successfully deployed and are in production
Russian National Grid Network project is currently developing, so further work on the services will be continued
14/14