AGATALocal Level Processing

Present implementation

Daniele, Dino, Enrico, Francesco

Structure of EDAQ

Detectorpreamps

Digitisers

TrackingControl,Storage…

EventBuilder/Merger

Pre-processingPSACore +36 seg.

Gobal Trigger

Clock100 MHzT-Stamp

Ancillary

AncillaryDetectors

Ancillary

GlobalLevel

DetectorLevel

Goals of the local level processing

• Manage all processing that is done on the germanium detectors– Readout the front end electronics

• Local event builder to merge the data of master and slave ATCA carrier cards

• Write the events into Crystal Frame

– Preparation of data: -> calibrations, cross-talk corrections, timing,…

– Pulse Shape Analysis– Tracking (formally part of the global level processing)

Implementation issues• Extremely difficult to write programs in a distributed parallel

environment as Narval– Almost no debugging tools (up to printouts)

• Joa Ljungvall has developed an emulator (C_Test) based on shared libraries loaded at runtime (like in narval) major step forward allowing the the use of debug and profiling tools (gdb, valgrind, KDevelop…)

• Two-class approach:– Most usual tasks (set-up and decoding/recoding of the events from/to

adf frames ) taken up by the mother class – algo-specific parts (e.g. PSA, Tracking) can be done by an inherited

class that re-implements the Process() virtual method– The driver program (“Narval emulator”) loads the shared libraries and

takes care of passing the data from • Based on this, a self contained emulator was developed locally with

the driver program (WinCtest) extended to handle several crystals in parallel, an event builder and a (pseudo) merger for ancillary data

Emulator WinCtest

• Written using the free Microsoft IDE Visual Studio Express by (mainly because of its excellent Debugger)

• Runs on Cygwin and GNU Linux using the gcc toolchain – The makefiles for Linux produce the libXXX.so used in Narval

• Multi-core CPUs exploited using threads provided by the Boost library (version > 1.35)

• Joa’s C_Test is still used as a final test before going to Narval

Don’t use the emulator if you analyse more then 1 ATC Install and learn to use Narval

Organization of directoriessource code

narval_emulatormisc_actors1 filters filters/Preprocessing

WinCtestenvironment variable

ADF_CONF_PATH

Still using a “local” version of ADF

Readout CrystalProducer/CrystalProducerATCA

• The main problem to solve was the readout of the master and slave ATCA cards in such a way that they do not block each other

– Readout performed by 2 independent threads and bounded buffers (ring buffers)– Synchronization with the main program done using mutex and conditions

• All this can be excluded when reading data from files (by not defining the USETHREADS symbol)

• The local event builder is derived from previous work done to merge the 10 TNT2 digitizers used previously as DAQ of a single crystal

• The events are formatted into an ADF data:crystal frames• Estimated readout is ~50 kevents/s (not using adf methods), which is the

design goal of AGATA• This is still a Mother-Daughter class approach but will probably be merged

into a single class as it is unlikely to be changed much• The original data can be written to disk and replayed with the same program

– Very useful to develop and debug the programs– Usually we write the locally merged data– This is the bottleneck of the whole processing chain 1 event = 14 kiB

• Used also to read ancillary data (DANTE) in the 2009-week27 test

Preprocessing PreprocessingFilter/PreprocessingFilterPSA• Decoding and recoding of events done by the mother class using the

ADF methods (decoding the input could be done without)• Data processing (Process method) is done in the daughter class,

also heavily based on previous work with the single crystal– Energy calibrations and cross-talk corrections – Calibration and xTalk coefficient produced automatically by a set of

program and procedures prepared by Francesco. – Baseline removal and energy calibration of traces (no cross talk

correction yet)– Digital CFD applied to the core to better determine T0

• Loss of efficiency at low energy (which we tried to avoid in the front end electronics by using a LE discriminator)

• All traces are shifted event-by-event by the same (integer) amount• The subsample interpolation of T0 passed to the PSA but not presently used

– Several Energy and Time spectra displayed with GRU or saved to disk in a flat binary format.

• The output format is data:ccrystal frames with 80 samples as 4-byte floats instead of the original 160 2-byte shorts– ~same size as input

• Estimated throughput is 5 kevents/s/process– xTalk correction of traces could slow down significantly

PSA PSAFilter/PSAFilterGridSearch

• Input data decoded to local structures using adf methods• Grid Search algo by Roberto Venturelli ported originally to the the

daughter class by Joa– Reads the signal basis (binary format) in the initialization phase– Multi-segment events are decomposed in single-segment hits– Assumes only 1 interaction in the hit segments– Does not (normally) use the net-charge signal but only nearest

neighbours in the search – The actual grid search can done in parallel either by multiple instances

of the actor (Narval feature) or by assigning it to worker threads running in one process

– ~0.5 kevents/s/core with the full search– ~4 kevents/s/core with a 2-steps search– Several spectra or matrices to be displayed with ViGRU or written as flat

binaries to disk.• Output data formatted as data:psa frames (~200 B/hit which could

be reduced a lot by using float instead of double and removing the errors which are not calculated by this algorithm)

Ancillary readoutAncillaryProducerTCP

• Written by Enrico Calore– TCP server using the skstream– Non blocking– Data formatted as data:ranc0 (a plain array of integer numbers

plus timestamp and event number in the key part of the frame)– Not using ADF– Original data can be saved to disk and replayed like for the

CrystalProducers

Ancillary processingAncillaryFilter/AncillaryFilterDante

• Developed first for Dante (data taken with the AGATA electronics)• Being extended now by Daniele and Enrico for PRISMA • Mother-daughter structure similar to the other actors

– Decoding from ADF data:ranc0 frames done by the SetInput() in the mother class

– Analysis done by Process() in the daughter class• Calibrations• analysis of DANTE to obtain the recoil velocity vector (in the AGATA

reference frame)• analysis of PRISMA to select reaction channel and determine velocity vector

in the PRISMA/AGATA reference frame– Coding back to data:ranc1 done by SetOutput() in the mother class

Francesco has added the possibility to write the data into raw binary files or a root tree

• This is the most variable part of the online libraries and needs to be organized carefully

EventBuilder

• C++ version to emulate the Narval (ADA) event builder written by Xavier Grave– Builds events using timestamp or event number– Data formatted as event:data:psa– For ancillaries:

• First, data:ranc1 frames are packed together with data:psa frames into event:data:psa

• Then a merger function is called to reformat the event:data:psa frame into an event:data frame, containing the event:data:psa and the data:ranc1 frames

• The ancillary part is enabled by the AddMerger keyword in the setup-file EventBuilder.conf

TrackingTrackingFilter/TrackingFilterOFT

• Mother class in charge of set-up and decoding (SetInput) and recoding (SetOutput)

• Orsay Forward Tracking implemented in the daughter class (Process methods)

• This actor is presently used also to control the experiment/analysis but this stuff should be moved to other actors (e.g. PreTracking and PostTracking) or to the watchers

• With 2 ATCs can handle ~2 kevents/second/process Should run multiple instances in parallel (Narval)

Organization directoriesdata taking and offline analysis

zCurrentNarvalDataDir

DataOut

Conf Conf/1R

Conf/Ancillary

Standardized names for directories and (most of the) configuration filesShould introduce environment variables to point to the actual directories

Topology for 2 ATCs + AncillaryChain 3 1R/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 3 1G/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 3 1B/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 3 2R/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 3 2G/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 3 2B/Producer CrystalProducerATCAFilter PreprocessingFilterPSAFilter PSAFilterGridSearch

Chain 2 ./Builder EventBuilderFilter TrackingFilterOFT

~3 GiB of memory neededMake your tests with 1 or 2 crystalsand run the real analysis in Narval