a log service package for the atlas tdaq/dcs group
DESCRIPTION
A Log Service Package for the ATLAS TDAQ/DCS Group. 2006 Conference for Computing in High Energy and Nuclear Physics 13-17 Feb. 2006 Mumbai, India. Benedetto Gorini on behalf of Raul Murillo-Garcia. Outline. Motivation Architectural overview Log Producers Log Manager Log Server Log Users - PowerPoint PPT PresentationTRANSCRIPT
1
A Log Service Package for the ATLAS TDAQ/DCS
Group
2006 Conference forComputing in High Energy and Nuclear Physics
13-17 Feb. 2006 Mumbai, India
Benedetto Gorini on behalf of Raul Murillo-Garcia
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
2
Outline
Motivation Architectural overview Log Producers Log Manager Log Server Log Users Test Analysis Conclusions
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
3
Motivation
Having a centralized mechanism whereby messages produced by TDAQ applications can be stored and later retrieved in an intuitive manner.
This is especially necessary for applications running in diskless and ‘consoless’ machines.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
4
Architectural overview
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
5
Log Producers
Log Producers do the actual message logging via the ERS (Error Reporting System)
The ERS offers a unified system for handling error, warning and debug messages and provides the logging front-end for the TDAQ applications.
The Message Reporting System (MRS) provides the transport mechanism following the Producer-Server-Consumer scheme.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
6
Log Manager
The Log Manager is a PHP-based standalone tool that provides a web-based user-friendly interface to the Log Server database. • Filter – this is a mechanism whereby messages are
retrieved from the log server according to a qualifier-based criteria.
• Display – allows the user to view messages.• Local archive – permits the storage of messages to a
local file. • Delete – this option allows messages to be removed
remotely according to a filter. To prevent unwanted users from removing messages, the Log Service shall access the Access Manager system to restrict this option to authorized users only.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
7
Log Manager
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
8
Log Manager
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
9
Log Servers
The message archival is based on the MySQL DB.
An Apache HTTP server has been deployed to let Log Managers access the MySQL database.
Scalability has been taken into consideration to accommodate a multiple Log Server system. A logging policy based on the severity level of the message is deployed to distribute the messages.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
10
Log Servers
An application (Log Receiver) subscribes to the MRS server with a given criteria, archiving onto the database the messages received. An intermediate buffer is used to decouple the thread that receives the messages and the thread that does the actual logging. This buffer is a C++ List Container with the SQL INSERT queries as the entries.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
11
Log Users
A Log Service C++ API mirrors the functionality provided by the Log Manager. Remote access to the database is done through the MySQL library.
The API is intended for those applications in the TDAQ system that require access to the Log Service databases (Log Users). An example of such application could build histograms based on the number of errors occurred during a specific Run.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
12
Measurement taking: set up
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
13
Test Results
Log Server 1: Pentium 4 – 2,8 GHz 256 MB RAM and Hard Disk PCI (~42 MB/sec)Transmission Rate: 2,000 msgs/sec.
0
500
1000
1500
2000
2500
3000
3500
0 45 91 136 182 227 273 318 364 409 454 500 545 591 636 682 728 773 819 864
Time (sec.)
Inco
min
g-I
nsert
Rate
(m
sg
s/s
ec)
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
Bu
ffer
Occu
pan
cy
Incoming Rate (msgs/sec) Msg. Insert Rate (msgs/sec) Buffer Occupancy (msgs)
31
2
Behaviour of the low spec. Log Server for a transmission rate of 2,000 msgs/sec. This figure also shows the influence of the Log Manager (three red points). Insert Rate Upper Limit ~1,800 msgs/sec.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
14
Test ResultsBehaviour of the high spec. Log Server for a transmission rate of 3,000 msgs/sec. This figure also shows the influence of the Log Manager (three red points). Insert Rate Upper Limit ~3,800 msgs/sec.
Log Server 2: Dual CPU AMD 250–2,4 GHz with 4 GB RAM Hard Disk SATA (~85 MB/sec)Transmission Rate: 3,000 msgs/sec
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 30 60 90 121 151 181 212 242 272 303 333 363 393 424 454 484 515 545 575 606 636
Time (sec.)
Inco
min
g-I
nse
rt R
ate
(msg
s/se
c)
0
20000
40000
60000
80000
100000
120000
Bu
ffer
Occ
up
ancy
Incoming Rate (msgs/sec) Msg. Insert Rate (msgs/sec) Buffer Occupancy (msgs)
3
1 2
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
15
Test results
The tests have hinted how influential the Log Server specifications are. Especially important is the amount of RAM available, as it is desirable to avoid swap ins/outs, which greatly slow the whole system.
The Log Manager has a visible impact on the Insert Rate, especially when the Transmission Rate is close to the Insert Rate upper limit.
The intermediate buffer is only intended to accommodate transitions during which the insert rate decreases, be it because the Log Manager is used or because the transmission rate is larger than normal.
Conference for Computing in High Energy and Nuclear Physics13-17 February 2006, Mumbai
Raul Murillo-Garcia - CERN
16
Conclusions
A Log Service has been developed to allow applications to log messages and users to browse/maintain them in an user-friendly manner.
Although the Log Server should be as powerful as affordable, one can also resort to use multiple less powerful Log Servers. The final implementation can only be devised once the scale of the information produced by the TDAQ system is known.
The Log Service has been released for general use in October 2005.