Rochester Institute of TechnologyRochester Institute of Technology

Information Services

Andrew BrownJon Ludwig

Elvis Montero

http://blackrose02.rit.edu/wiki/lib/exe/fetch.php?media=grid:seminar1:lectures:seminar-grid-1-information-services.ppt

Overview

• Globuso MDSo WSRFo WS MDS

• Deploymento ESGo MonaLISAo Inca

• Gangliao Daemonso Web Frontend

MDS

• Monitoring and Discovery System• Versions

o MDS2 -> Pre-WS MDSo MDS4 -> WS MDS

• Capabilitieso Querieso Subscriptionso Triggers

[Globus01]

WSRF

• Backgroundo OASIS specifications for web serviceso Globus, IBM major contributors

• Modelo Clients communicate with resource serviceso Include identifier to specify resourceo Similar to objects and methods

[OASIS] , [Globus02]

WS MDS - Overview

• Implementation of MDS in WSRF• Components

o Foundation Aggregator Framework Information Providers

o Components Index Service Trigger Service WebMDS

[Globus01]

WS MDS - Foundation

• Aggregator Frameworko Aggregator Sources

Java interface to collect information What to get, where to get it from Types

Query Subscription Execution

o Aggregator Services Collection of Aggregator Sources

[Globus01]

WS MDS - Basic Components

• Index Serviceo Collects informationo Stores as resource propertieso Publishes in single locationo Supports queries and subscriptionso Indices can be linked together

• Trigger Serviceo Collects informationo Matches rules and fires actions

• WebMDSo View information in web browsero Used by end users

[Globus01]

WS MDS - Advanced Components

• Information Providerso External applications used to collect datao Collection of Aggregation Serviceso Examples

Condor - hosts, processors, memory Ganglia - hosts, processors, memory WS GRAM - queue, CPUs, memory RFT - server, transfers

[Globus01]

WS MDS - Use

• Index Serviceo wsrf-query http://url [expression]o wsrf-get-property http://url --key <name>

• Trigger Serviceo mds-servicegroup-add http://url config.xml

• WebMDSo http://hostname:port/webmd

[Globus01], [IBM]

Deployment - ESG

[ESG]

Deployment - MonALISA

[MonALISA]

Deployment - Inca User Level

[Inca]

Ganglia

• Why monitor your systems?o Healtho Performance / Bottleneckso Trends

• Remotely view live or historical statistical data• High performance

o Low overhead per nodeo High concurrencyo Scalable

• Designed for federated clusters• Graphical view via website• XML data from Ganglia Meta Daemon

Ganglia Monitor Daemon - gmond

• Runs on each node that you wish to monitor• Minimal footprint - In-memory hashtable• Responsibilities:

o Monitors changeso Listens for and records the state of other Ganglia nodeso Reports state of the node

Broadcast when the state changes Periodically, if the state > threshold

Replies to requests for state informationo XML + TCP (telnet?!)o XDR + UDP (Multicast / Unicast)

• Need a different metric? - gmetric• Fault Tolerance

Ganglia Monitor Daemon - gmond

[GDT2002]

Ganglia Meta Daemon - gmetad

• Periodically polls child data sources to gather information about cluster stateso Unicast TCP + XMLo Redundant sourceso gmond or gmetad

• Saves data in a databaseo rrdtool

• Collect data from individual nodes or from clusterso Hierarchical

Ganglia Meta Daemon - gmetad

Ganglia Web Frontend

• Collects the tree of data from gmetad and displays it to the user via a website

• Demoo http://monitor.millennium.berkeley.edu/o http://cluster.rit.edu/ganglia/

References

[Globus01] GT 4.0: Information Services.  http://www.globus.org/toolkit/docs/4.0/info/. [OASIS] OASIS Web Services Resource Framework (WSRF) TC.  http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf. [Globus02] WSRF: The Web Services Resource Framework.  http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html. [Atkinson] Atkinson, Malcolm et al.  Web Service Grids: An Evolutionary Approach.  http://www.omii.ac.uk/dissemination/paper_web_service_grids.pdf. [Wikipedia01] Web Services Resource Framework.  http://en.wikipedia.org/wiki/Web_Services_Resource_Framework.  

References cont.

[Foster] Foster, Ian.  A Globus Primer.  http://www.globus.org/toolkit/docs/4.0/key/GT4_Primer_0.6.pdf.

[Globus03] Deploying WS MDS in a Virtual Organization. http://www.globus.org/toolkit/docs/4.0/info/key/deployment_overview.html. [Globus04] GT 4.0 WS MDS WebMDS: System Administrator’s Guide. http://www.globus.org/toolkit/docs/4.0/info/webmds/admin-index.html. [Globus05] GT 4.0 WS MDS Trigger Service: System Administrator’s Guide. http://www.globus.org/toolkit/docs/4.0/info/trigger/admin-index.html. [Wikipedia02] Ganglia (software). http://en.wikipedia.org/wiki/Ganglia_%28software%29.

References cont.

[GDT2002] Ganglia Toolkit. http://www.msg.ucsf.edu/local/ganglia/ganglia_docs/index.html

[Massie2004Ganglia] "The Ganglia Distributed Monitoring System: Design, Implementation, and Experience". Matthew L. Massie, Brent N. Chun, and David E. Culler. Parallel Computing, Vol. 30, Issue 7, July 2004.

[ESG] The Earth System Grid (ESG) monitoring architecture. http://www.globus.org/solutions/system_monitoring/architecture.php.

[MonaLISA] MonaLISA's Architecture.  http://monalisa.cacr.caltech.edu/monalisa_e__System_Design.htm.

[Inca] Inca's architecture and screenshots. http://inca.sdsc.edu/drupal/node/20.

[IBM] WebMDS Screen Shot. http://www.ibm.com/developerworks/cn/grid/gr-gt4mds/webMDSRFT.jpg