panda monitor development
DESCRIPTION
PanDA Monitor Development. ATLAS S&C Workshop b y V.Fine (BNL). ATLAS User driven development. Maintenance: I mplementing the new “Feature Requests” Fixing the “known” bugs Development: - PowerPoint PPT PresentationTRANSCRIPT
PanDA Monitor Development
ATLAS S&C Workshopby V.Fine (BNL)
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
2
ATLAS User driven development• Maintenance:
– Implementing the new “Feature Requests” – Fixing the “known” bugs
Development: – high priority, short term developments to address high priority
needs of operations and analysis users– plotting system: “flot”-based plots (after investigating options) to
add requested plotting features– ”enhanced” job info and statistics pages– jQuery integrated and used as standard basis for implementing
'client intelligence' and high quality UI– ”json” introduced as standard basis for server->client data export– modular, OO design used in new developments for greater
maintainability, reliability, flexibility, and easier reuse5/4/2011
ADC Monitoring Feb 16, 2011:http://indico.cern.ch/conferenceDisplay.py?confId=127099
“User request driven” approach
DONE
DONE
DONEDO
NE
DONE
DONE
DONE
DONE
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
4
Monitor: Before and After
DB
Page Generators
Panda Monitor Web Server
URL QUERY
SQL QUERY
HTML
DB
Page Generators
Panda Monitor Web Server
URL QUERY
SQL QUERY
histograms json flothtml/jsonJQuery“generic”
code
Where:
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
5
Jaroslava’s feedbackhttps://indico.cern.ch/getFile.py/access?contribId=2&resId=0&materialId=slides&confId=130719
Slide 5
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
6
Related Savannah tickets
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
7
There were 72 Savannah tickets and a bunch of the E-mail communicationsTrend: the number of “Bug Report” declined , the number of the “Feature request” grew and right now it is on the level 2.5 request per week.There was no complain about quality /performance of the “plotting” approach and its implementation.The requests are coming from users, managers, shifters, PanDA developersAmong them:• Reprocessing jobs’ time distribution• Analysis “merging” jobs monitoring• Flexible statistic against of any imaginable filtering per user / date / site /
cloud region/ processingType … in the other words against of any job parameter the Panda DB records.
There was no new “plotting”-related request recently. There is the growing number of requests for the new “Monitoring” tables (requests (#79437 - “High priority tasks” , #79618 - “Page with list of multicloud sites”) . We are taking in account this trend to set our development priorities.
Related Savannah tickets
5/4/2011
DONE
DONE
DONE
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
8
Site performance issue analysishttps://savannah.cern.ch/bugs/index.php?80368
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
9
reco performance: Intel vs. AMD
From: Alexandre Vaniachine <[email protected]>Date: March 18, 2011 8:21:03 PM GMT+03:00To: david rousseau <[email protected]>Cc: Wensheng Deng <[email protected]>Subject: reco performance: Intel vs. AMD
….we observed a difference in HI reconstruction performance on Intel vs. AMD that is larger than the difference in the GHz.
Do you think it would be interesting to investigate if there is such a difference on the level of particular algorithms?
https://indico.cern.ch/getFile.py/access?contribId=10&sessionId=1&resId=0&materialId=slides&confId=119169
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
10
DOE/NSF Bi-weekly Operations Meeting
5/4/2011
11
ATLAS User Analysis in the US Cloud
5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
http://indico.cern.ch/getFile.py/access?contribId=6&resId=0&materialId=slides&confId=119945
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
12
Panda Monitor / DashboardPanda Monitor / Dashboard cross references
5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
13
Output Merging for Analysis Jobs
5/4/2011
https://indico.cern.ch/getFile.py/access?contribId=121&sessionId=13&resId=0&materialId=slides&confId=119169
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
14
SummaryAfter a month of the development followed by one month of use
Monitor is free of the “known” bug and has been enhanced with about 2 dozens of the new features requested by ATLAS community via Savannah system. It includes the graphical representation of the user selected information.
• Supporting large Monitor user communities in operations and analysis• Quick response to up to 4-6 requests/week from ops and analysis users• Incrementally upgrading monitor design/function in the context of the
production monitorNear term developments to come:• more work to identify common effort and common code with other
monitoring activities• improve flexibility of data summary histograms• client side intelligence, ajax to improve response time and minimize DB load• server side modular python framework for easier support and integration
with other monitoring5/4/2011
PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011
15
Questions?
5/4/2011