Report from the Working Group "Detector Database"

W.Peryt (rapporteur), P.Buncic, D.Vicinanza, Y.Schutz.

This WG has an urgent task to achieve that is the design and implementation of detector construction databases for the different detector group in ALICE.

(from the Memo of F.C.)

March 20th, 2001

Detector Construction Database Working Group

ALICE week, off-line software meeting, CERN, May 28th, 2001 W.S. Peryt & Warsaw DB Group, WUT

Report from the Working Group "Detector Database"

Konrad Bohuszewicz undergraduate student

Maciej Czyzowicz undergraduate student

Michal Janik Ph.D. student

Rafal Holubowicz undergraduate student

Dawid Jarosz undergraduate student

Piotr Mazan undergraduate student

Marcin Mierzejewski undergraduate student

Mikolaj Olszewski undergraduate student

Wiktor S. Peryt

Sylwester Radomski undergraduate student

Piotr Szarwas Ph.D. student

Tomasz Traczyk

Dominik Tukendorf undergraduate student

Jacek Wojcieszuk undergraduate student

Faculty of Electronics and Information TechnologyFaculty of Mathematics and Information SciencesFaculty of Physics

Alice detector databases – architectureAlice detector databases – architecture

Central database

Placed at CERN

Contains

– central inventory of components

– copies of data from laboratories

Satellite databases

Placed in laboratories-participants

Contain source data

– produced at laboratories

– delivered by manufacturers

Communication

Passing messages in XML

Off-line (batch processing)

No satellite-satellite communication!

Central database

Satellite databases

Central database versus satellite databasesCentral database versus satellite databases

Central database

Usage characteristics

– very large data volume

– many concurrent users

– twenty-four-hour utilisation

– critical data (breakdown may be very costly)

Necessary features

– 7 × 24 availability

– high reliability

– continuous administration by professional DBA team

– on-line backup

– data partitioning

Proper solution

– high-end commercial DBMS softwareis necessary

Satellite databases

Usage characteristics

– average or small data volume

– only few concurrent users

– used only few hours per day

– not critical data (can be restored from backup or from central database)

Necessary features

– should not need complex administration (no trained personnel at labs-participants)

– should be monitored remotely by developers

– off-line backup procedures are sufficient

– data may not need to be partitioned

Proper solution

– non-commercial database can be used

Monitoring of componentsMonitoring of components

Central database Central inventory of components contains

– actual location and status of each components

– history of each component

Flow of components Creation of new component must be signalled to

central inventory

– a unique identifier for each component is generated by lab and registered in central inventory

Arrival and departure of each component to/from any laboratory must be recorded in the inventory

Destruction of the component must also be signalled to the inventory

Solution Components must be “checked-out” and “checked-

in” from/to central inventory before/after they are processed by satellite databases

“Create”, “Check-in”,”Check-out”, and “Destroy” messages are passed as XML documents

Central database

Satellite databases

2. Check-in

3. Check-out

1. Creation

4. Destruction

ProblemsProblems

Compound components

Components may be composed of other components

Compound components

– form hierarchy

– must be treated in a special way

Changes of the components’ properties must be “cascaded” up and down the hierarchy, e.g.

– “checked-in”, “checked-out” and “destroyed” statuses

– location

Communication problems

Breakdowns of telecommunications links cannot lock normal work procedures of satellite laboratories

There must be a possibility to ‘force’ check-in, check-out and creation of new component’s identifiers during telecommunication breakdowns

Changes made in this ‘force’ mode must be synchronised in central database as soon as the communication is recovered

Unique identifier (primary key) must be created for each new component to store its data in satellite database – even if during communication breakdown, so

– globally unique id should be created by satellite database as compound id: database id + locally unique identifier

– this identifier must be registered in central inventory as soon as possible

Development of the systemDevelopment of the system

Phase 1

Central database

– contains only inventory

– located in WUT

– technology

Oracle DBMS

Java + Oracle XDK

Satellite databases

– prototypes prepared concurrently on

MySQL

PostgreSQL

– applications written in PHP

– XML applications based on Oracle XDK

Central-satellite communication

– in XML – prototype versions of messages

– only inventory (Check-in/out, etc.) data

Phase 2

Central database

– contains copies of components’ data collected from satellite databases

– all the data in one integrated structure

– technology

Oracle DBMS (with use of object-relational technology)

Java technology (servlets, JSP)

Oracle XDK

dedicated Oracle technology (PL/SQL Web Agent, XSU, XSQL)

Satellite databases

– all on the same selected DBMS

– applications in PHP

– XML applications: Java + Oracle XDK

Central-satellite communication

– in XML – final versions of messages

– inventory data and detailed components’ data

Design RepositoryDesign Repository

What is needed?

Formalised representation of

– user requirements

– software requirements

Formalised description of

– databases’ structures

– applications

Extended data dictionary

– prompts and descriptions

– help texts

– standard values of application parameters

(fields’ sequence, dimensions, etc.)

Automated generation of

– SQL scripts

– parts of applications

– help and documentation

Possible solutions

CASE software

– very powerful

– with graphical interface

– difficult and costly

“Poor man’s CASE”

– proprietary formalisms and scripting

– may be implemented in XML

Taken up tasksTaken up tasks

Prototype model of interchange of measurement data between

labs-participants and central database

“Envelope” (control) protocol for data interchange between

central and satellite databases

Prototype system for control of components’ flow

Remote monitoring of satellite databases

Human interface

IMAGEIMAGE IIntegrated ntegrated MAMAnagement with nagement with GGraphical Interfacraphical InterfacEE

What it will be

• Tool for remote db control & administration

• Graphical user-database interface

• General for all local databases

IMAGE – system features

• OS independence • WWW user-interface• Possibility of remote-control &

administration• Multi-access• Divers levels of accessibility• On-line documentation• Integration with ROOT IMAGE – implementation

• Object-oriented technologies • Open source development • Modular structure of the application

Class structure for IMAGE Package

IHObjectIHContainerIHLinkIHLaballed

IHListIHDocIHTCellIHTableIHForm

IHAppletIHFrameIHImageIHStringIDDatabase

IDField

IDField

IDDatabaseCtl

IDMetaTable IDCollectionIDQuerryIDTable

ISAuthorization ISPerms ISSession ISUser

IMAGE – documentation

Communication server (LabSERVER) for LabVIEW

Originally developed by: Michał Janik & Piotr Szarwasjanik@if.pw.edu.pl , szarwas@if.pw.edu.pl

Currently maintained by Marek Szubacyberman@if.pw.edu.pl

Works at IReS, Strasbourg

Available for all interested ALICE members immediately

LabVIEW LabSERVER

DBMS(MySQL or PostgreSQL)

TCP/IP

TC

P/IP

Environment for test measurements

LabVIEW LabVIEW LabVIEW LabVIEW

NETWORK

DBMSLabSERVER

Features of LabSERVER

100% C, POSIX-compliant

Works as daemon

Support for multiple clients connected simultaneously

Access control

Reliable data handling

Highly configurable: TCP/IP port, connection timeout, maximum number of clients, allowed domains etc.

Works with MySQL and PostgreSQL.On the other side: LabVIEW, HP VEE…

Milestones till September

Installation of Oracle DMBS at WUT (Warsaw)

Development of the prototype of the Central Database

Beta-versions of satellite databases

– implementation for SDD (Torino and Trieste)

– implementation for SSD (partial – mainly for modules, microcables and object flow; Strasbourg, Utrecht, etc.)

– maintenance of existing db concerning ladders for SSD (Utrecht)

– Summer training for ~8 undergraduate students at SUBATECH (Nantes) July 15th .. August 20th