1

Configuration Database Review

David ForrestUniversity of Glasgow

CM24 @ RAL :: 1st June 2009

2

Configuration Database

Overview of the database Status Future Work

3

Overview :: Scope The configuration database deals with

cabling, calibration, geometry and set value information ONLY

Its use cases are useful for offline analysis, reconstructing the configuration of the experiment

Eg “Give me the configuration of the experiment at 19:15 on 31st May 2009”

4

Overview :: Temporal Data Imagine uploading some calibration valid for a given

month. Store the period of validity and label this the valid time. Then later we find a better calibration for that same valid

time. We want to keep both, with the same valid time. We also store the time the information was given to the

database, which is the transaction time. Now we can get the most up to date calibration by

default, but also recall the other calibrations by meta data.

Our database is bi-temporal Example holds for geometry (eg misalignment) and

cabling also. Not currently implemented for set values.

5

Overview :: API There are many applications which will read from and

write to the database The database should not make any impositions upon

their design and implementation When something is updated it would be easier to update

code in one place rather than in many places, versions, etc

So we have an API, which stands for Application Programming Interface

The API is also independent of the reading and writing applications and the database management software.

David Forrest will write the API and Database. Malcolm Ellis will write the code for G4MICE to interact with the API. Other people may be needed to write other client applications eg James Leaver for the EPICs interface.

6

API-2

RAL Firewall

Grid Users

DB APIDB

G4MICE

Database Developer

Control Room Apps

Note: Following discussions from Software session, DB will be hosted on a new machine in the control room which must be bought (see Paul K or Malcolm for details)

7

Overview :: Implementation We use the Postgres 8.3 Database

management software. Open source. The API is a server with code written in Java

using JDBC. C++ code, etc, can of course connect to it!

All the client has to do is interpret the feedback which will be of a guaranteed agreed format

Full database has been built up incrementally, testing and prototyping. It is based at RAL, and can be accessed remotely

8

XML It has been agreed that input/output from API will be

in XML This means we can always be backwards

compatible. XML is extendable. You can put more stuff in if you have to.

API can check XML is well formed, and call the right functions itself, so potentially all the user need do is pass an XML file (eg with a geometry) so you barely need to know even what functions the API has

A well defined interface between the database and the client application means that the chance of distributed bugs over many client applications is reduced.

9

Consistency A series of constraints will be included in

the final product to maintain consistency eg Step VI cannot have three trackers. These constraints have not been completely catalogued yet.

Failure on one constraint results in rolling back the database to a state just before the constraint was violated.

It is envisioned that there will be a log, of every single operation and query on the database

10

Backup

Envisaged to make daily backups of the database contents to a box on the mice network for which the disk contents are regularly backed up

This will piggy-back on the backup strategy for other systems

11

Status: Calibrations It has been challenging to tie down all

detector groups to a calibration format So we won’t. The idea now is to have a

flexible calibration format which does not require us to know the format of a calibration in advance

Calibrations can now be a vector of values of elements of different types – double floating point number, string, boolean, integer

Meta data included + Temporal overlap

12

Calibrations - 2

13

Status: Cablings Just getting to grips with this recently Take detectors as roots of cabling - unless theres a

good use case for the configuration DB storing detector independent cabling

Previously had a fairly loose approach We now have detector specific cabling schemes – are

these complete (so far as is related to a real use case of the config DB)?

Every component of cabling should have a unique ID, often the serial number. This is unique not only over all components of same type, but over all components. Eg so a splitter can be assigned an ID number for each channel without knowing in advance the type of device it is connecting to.

14

Cablings-2

Prelim – will likely have a couple of changes to TOF, KL, and arising from implementation issues.

15

Geometry

16

Status: Set Values

The following are known and recorded as being needed:

Magnet currents (quads, dipoles); RFCC currents; diffuser thickness; decay solenoid current(s); spectrometer solenoid currents; focus coil currents; target depth & delay

17

Set Values We have two types of configurations with regard to set

values. Run config and saved config. Run config: This is an XML file sent from EPICS which is

saved by the database. It is not parsed and read into the database structure, it is only tagged in terms of time, version and run number. It is used once for a run with a time validity.

Reference config: You can tag a configuration and use it many times. This is read into the database structure.

If some aspect of a run configuration is implemented in EPICs but not yet in the configuration database structure, this does not introduce delay – its stored as a flat file.

18

Status: API The API exists now, and sits at RAL It really should be moved to a box which is visible to

everyone. Any offers? It currently takes a geometry encoded in XML and

translates that into SQL and uploads it to the database

XML is standardised and extendable, and highly recommended as a means of communicating with the API, which has good XML parsing functionality already

In an ideal world, for writing data, just send an xml file and the API knows what to do eg if its geometry, calibration, cabling etc.

19

Early Testing Geometry almost fully implemented, with valid time (but

not transaction time yet) and full module hierarchy 60+ geometries were uploaded and typical questions

asked about the geometry at a given time. Valid time introduces no significant slow down to Postgres. Transaction time will increase complexity only slightly

This still has to be translated from SQL response to XML, and G4MICE reconstructing the hierarchy may or may not place further obligations on the database, but current results are promising, unlikely that there will be any grievous slowdown.

20

Current State + Future Work Geometry prototyping is going well G4MICE will very soon be able to, as well as send a geometry to

the db (already done), also reconstruct a geometry from database into memory and compare that with what it sent.

Would like to work on calibrations and, mainly, cablings, next Once cablings are fully prototyped, much of the hardest work

will be done We may still need people to write code which sends requests to

the API and digests response (no db knowledge required)

Progress at this collaboration meeting in defining seams with other systems, eg G4MICE, control room apps, EPICS, DAQ by meeting up with JSG, Malcolm, Pierrick, and others

It is likely that the db will be placed in a box in the control room – probably a new box unless a suitable one is available.

21

Project Plan

Human resource: David Forrest 50% Long term maintenance is an open issue

Scheduled milestones Reconstruction of Geometry: end of June Prototyping and Testing of Cabling and

Calibration : end of July Save and recall run configurations: end

of August

22

Temporal but not temperamental