the hades experiment @ gsi
DESCRIPTION
The HADES Oracle database and its interfaces for experimentalists Ilse Koenig, GSI Darmstadt for the HADES collaboration. The HADES experiment @ GSI. A H i gh- A cceptance D i- E lectron S pectrometer. ~ 80000 readout channels first production beam time - PowerPoint PPT PresentationTRANSCRIPT
The HADES Oracle database
and its interfaces for experimentalists
Ilse Koenig, GSI Darmstadt
for the HADES collaboration
CHEP 2009 The HADES Oracle Database Ilse Koenig 2
The HADES experiment @ GSIThe HADES experiment @ GSI
A A HHiigh-gh-AAcceptance cceptance DDi-i-EElectron lectron SSpectrometerpectrometer
Expanded view
~ 80000 readout channelsfirst production beam time Nov 2002one prod. beam time each year~ 500 – 800 files/day (~0.8 TB/day)(10 times more expected in 2010)actually ~ 90 analysis parameter containers (analysis of real and simulation data, ~ 40 MB for single file analysis)
Expanded view
CHEP 2009 The HADES Oracle Database Ilse Koenig 3
Main usage of the HADES databaseMain usage of the HADES database
DAQ EPICS
users
Oracle
related to runs by time stamps
onlineanalysis
offline analysissimulation
file catalog slow control data beam time logbook
parametersrun validation geometryelectronics setup
ROOT macros
Run:
smallest data set individually treated
several files, one per event builder
web interface
CHEP 2009 The HADES Oracle Database Ilse Koenig 4
Analysis initialization conceptAnalysis initialization concept
HYDRA
Designed for automatic initialization Parameters (numbers, arrays, ROOT classes,…) are stored in parameter containers, automatically created by the analysis tasks and managed by the runtime database classes. User chooses one or two parameter sources. The analyzed files define the parameter versions.
separate shared libraries Oracle Precompiler
ROOT TGeoManager
Oracle
ASCII Files ASCII, ROOT Files
Web GUI
Geometry Interface Runtime Database
HGEANT
HYDRA
Version management
Oracle: complete history of all parameters since 2002
ROOT file: local version management parameter file for local access snapshots of versions in Oracle
Storage of parameters in Oracle: 1. Insert of parameters with a ROOT macro
2. Validation with web GUI
CHEP 2009 The HADES Oracle Database Ilse Koenig 5
Version management in OracleVersion management in Oracle
1. Parameters are not stable, but may change from run to run2. They may change for the same run over time (history)3. They may come in different flavors (depending e.g. on an algorithm)
3-dimensional version management
history
runs
valid_since valid_until (default year 4000)
date_create
invalid_since(year 4000 for actual version)
version
2 different flavors (contexts)
defined by user
automatically defined
CHEP 2009 The HADES Oracle Database Ilse Koenig 6
Version 1
Parameter versionsParameter versions
runs
historyyear 4000
year 4000
valid_since v1
During a beam time the userinserts a version 1.Each new run is initializedwith version 1.
date_create v1
real
tim
e
valid_until v1
valid_since v2
invalid_since v1
date_create v2
Later he adds a version 2.New runs are then initialized with version 2.
run start
The crossing point of run start and history datedefines the parameter version
version 1
version 1
version 2
history date
Later new versions 3 and 4replace the old ones
Before a DST production wedo a beam time specificparameter release “history date with name”
Old versions can be retrievedwith an old history date
version 4version 3parameter
release
CHEP 2009 The HADES Oracle Database Ilse Koenig 7
Layout of parameter containersLayout of parameter containers
Basically two groups of parameter containers:
1. Array of objects each containing the same list of parameters Data are stored in one or more individual tables in Oracle. Each parameter corresponds to a table column. Many records together form a version set of the parameter container.
Example: Lookup table for an unpacker(maps readout channels to detector cells)
Constraints guarantee for each parameter container version thatone readout channel maps to only one detector cell and both are unique.
Advantage: enforcement of data consistency via constraints and triggersDisadvantage: lack of flexibility
Adding or discarding parameters needs changes in the tables and interfaces
have to be done by an expert
Suitable only for stable code
Used for low-level analysis parameters
CHEP 2009 The HADES Oracle Database Ilse Koenig 8
Generic parameter containerGeneric parameter container
2. All parameters are stored as name – object pairs in the same set of tables
parameter containerparameter object
allows to add or discard (invalidate) a parameter
makes it (almost) code independent
different versions
nametype: Int_t, Float_t, Double_t, Char_t, Text_t, UChar_t, class type
stored as byte array (RAW or BLOB)number of values (single value or array)class versionstreamer info, root version for ROOT classesown version management
any class derived from TObject decoded in the analysis interface
by ROOT streamer
CHEP 2009 The HADES Oracle Database Ilse Koenig 9
Implementation and interfacesImplementation and interfacesfor generic parameter containersfor generic parameter containers
Requires only a few lines of dedicated C++ code in the container class
Generic read and write interfaces for Oracle, ROOT and ASCII files
implemented in base classes The Oracle interface stores a snapshot of parameters in a ROOT TList.
The parameter container copies only parameters defined in the actual code version. The user stores the parameter in Oracle by a ROOT macro.
At validation, new parameters are added automatically.Old parameters no longer needed stay valid until explicitly set invalid.
No database expert needed!
Web-based graphical interface to Oracle for validation, searches and comparisons
Suitable also for non-stable code
Used for conditions and high-level analysis parameters
CHEP 2009 The HADES Oracle Database Ilse Koenig 10
Web InterfaceWeb Interface
more than 100 dedicated applications in tree-organized folders
written in PL/SQL , uses Oracle application server
access to parameter sets in a beam time
CHEP 2009 The HADES Oracle Database Ilse Koenig 11
HYDRA parameter containersHYDRA parameter containers
all parameter containers grouped by shared libraries
CHEP 2009 The HADES Oracle Database Ilse Koenig 12
Parameter dataParameter data
Although stored as binary,C-type values and arrays are converted.
Classes are decoded only by the analysis interface
CHEP 2009 The HADES Oracle Database Ilse Koenig 13
Online storage of EPICS slow control dataOnline storage of EPICS slow control data
Program developed at SLAC by Lee Ann Yasuhawa and Bob Hall http://www.aps.anl.gov/epics/meetings/2002-11/talks/hall.pdfextension of the Channel Archiver, slightly modifiedstreaming with OCI Direct Loader
EPICS
Oracle
10 min partitions, not indexed
1 day partitions
local index (timestamp, channel)
once per day (Oracle scheduler jobs)data moved, old partitions dropped, new partitions created
Online storage of ~ 3000 channels, total rate ~100 Hz, 10-20 GB per beam time
Problem: not performing for fast scans on a
single channel over a long time range
Generation of run based summaries
For each run and channel: mean, min, max value, sigma, number of data, timestamp of first entry, ...
Interfaces:Web interface filtering graphics with JAVA applet
Analysis interface ROOT graphics Run validation Parameter containers to be used in the event loop
CHEP 2009 The HADES Oracle Database Ilse Koenig 14
Run based summaryRun based summary
Displayed by analysis interfaceExample: RICH HV summaries for one day of beam time
Blue points:
mean HV value for one run
Error bars:
sigma of mean value
Large error bars indicate a HV trip.
The summaries allow to scan the data of a whole beam time within seconds.
index of run summary
HV[V]
data for one run
CHEP 2009 The HADES Oracle Database Ilse Koenig 15
SummarySummary
Since start-up HADES successfully employs an Oracle database.Our overall design has been proven to be flexible enough
to adapt to new requirements with small modifications
and to be manageable with very restricted manpower.
The HADES initialization concept and its interfaces are also implemented in FAIRROOT,the analysis and simulation framework used by the FAIR experiments CBM and PANDA.
The HADES collaboration
More than 100 scientists
from 17 institutions in 9 European countries
http://www-hades.gsi.de
Special thanks to M. Dahlinger from GSI IT department for providing us a very stable server.