30 jan 2009elizabeth gallas1 introduction to tags elizabeth gallas oxford atlas-uk distributed...

30 Jan 2009 Elizabeth Gallas 1

Introduction to TAGs

Elizabeth GallasOxford

ATLAS-UK Distributed Computing Tutorial

29-30 January 2009


Outline

Introduction: ATLAS TAGs in the ATLAS Computing model TAG Contents TAG Formats

ELSSI – Event Level Selection Service Interface A Web based interface to ATLAS event selection

Hands on tutorial Selecting events of interest, refining selection criteria Extracting that selection Using selection as input to Athena analysis

Important TAG related hyperlinks


The TAG Development Group• Argonne:

David Malon, Jack Cranshaw, Qizhi Zhang, Peter van Gemmeren• Glasgow:

MikeKenyon, HelenMcGlone, TomDoherty• CERN:

Florbela Viegas, Trivan Pal• Oxford:

Elizabeth Gallas• Chicago:

Marco Mambelli• Grenoble:

Solveig Albrand• Orsay LAL:

Julius Hrivnac• Sheffield:

Tulay Donszelmann (PAT)

Thanks to everyone above and to all users with feedback !


What’s a “TAG” (in this talk) ATLAS “TAG” overload:

Not an acronym (retrospectively TAGs Are Great !) Not a CVS tag or a release tag or a project tag Not tracked by the ATLAS Tag Collector Not a COOL (Conditions Database) Tag Not a configTag Not a B, flavor, or muon tag

In this talk: ATLAS TAG application is A system supporting Event-level metadata

About 200 indexed variables for each event Stored in a ‘database’ (Oracle or file based)

1 kB/event -> 1 TB/year PURPOSE: Facilitates event selection for analysis


ATLAS TAGs in the ATLAS Computing model

Stages of ATLAS reconstruction RAW data file

ESD (Event Summary Data) ~ 500 kB/event AOD (Analysis Object Data) ~ 100 kB/event

TAG (not an acronym) ~ 1 kB/event TAGs

Produced in reconstruction Can be re-produced in re-processing Customized TAGs can be defined/produced TAG formats:

File based: AthenaAwareNtuple format (AANT)

Database: available globally through network connection Used by the TAG Browser (ELSSI)


TAG related hyperlinks:

TWiki with TAG content in ATLAS release: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/T

agForEventSelection

TWiki for Tutorial: https://twiki.cern.ch/twiki/bin/view/Atlas/EventTagT

utorials 4 Tutorial Components

ELSSI (Browser based) File-based TAGs (Command line based) JiveXML with TAGs (Atlantis Event Display) TAG based selection with PAthena on the Grid

ELSSI - Event Level Selection Service Interface: https://voatlas18.cern.ch/tagservices/index.htm


“TAG content”: Your selection variables PAT (Physics Analysis Tools) group

supports ‘official’ TAG content based on input from physics groups (YOU !)

‘TAG Database’ Application includes ~200 variables/event:

Event identification (run, event, lumi block numbers, timestamp, …) Global quantities (e.g., missing eT) Trigger decisions at all three levels (now bit encoded) Numbers of electrons, muons, photons, taus, and jets

pT, eta, phi for highest-pT objects Detector status and quality words Physics and performance group status words

‘Run Metadata’ at Temporal, Fill, Run, Lumi-block levels references to AOD,ESD,RAW for back-navigation A variety of supporting tools and infrastructure

ELSSI is a web based supporting tool Helps you navigate through “TAG content“ Uses other supporting tools and infrastructure Retrieve results in a number of forms


ELSSI: TAG Browser Basics An interface to the TAG Database

Supported browsers: Firefox, Safari (not IE) Requires Grid Certificate

Allows pre-selection of events for analysis, guiding the user through the selection process

User Input

Stream Temporal Quality PhysicsTrigger

COUNT events

SELECT attributes

RETRIEVE events


ELSSI: Implementation & Features

Running on server at CERN PHP and Javascript on backend

Features: Formulate multi-dimensional queries

At Run/LB and Event level Query caching Event counting Display Results (graphical and tabular form) Retrieve events in ROOT file Register collection in AMI


https://voatlas18.cern.ch/tagservices/index.htm

ELSSI Gateway

ChooseFDR2C

link


Summary

Metadata in a database is a powerful tool A great variety of dynamic selection criteria and

reporting become possible Development is ongoing to improve

Interfaces Functionality Usability

Integrate infrastructure tools

For comments/suggestions:

ATLAS Physics Metadata Hypernews


Backup Slides


Browser front page for FDR2 – ‘Temporal Tab’


Browser – ‘Stream Tab’


Browser – ‘ Trigger Tab’


‘Physics Attributes Tab’


‘Review Query tab’


‘Perform Query’: Count tab

Count events meeting criteria … revise criteria… count again… revise criteria …


‘Perform Query’: Display Results tab

Get an idea of the attribute distributions

Draw histograms – new functionality from Julius (still in development)


Retrieve Events

After refining query, generate a ROOT TAG file with selected events


Left Frame: about your Session(s)

Save/Share Session(s) Can now save a query

session for later use Uses cookies

Browser specific Share saved sessions with

others Selection Summary

Displays selection criteria of your session


Working on … (1)

Incorporating Detector Status (DQ) Loading online Runs

Even in the absence of TAG production An ELSSI “skim” button (Skimming Service)

using GangaTNT: From your selected TAG events, generate an AOD

dataset with only those events Output registered as new dataset Runs GANGA on the server to run the job on behalf

of the user User proxy is delegated to the server Monitoring loop checks job status and notifies user

of job completion / failure Integrate GangaTNT into GangaGUI.


Ganga TNT


Working on … (2)

Using TAGs with PAthena: Steps create an event collection (events.root) with ELSSI setup Pathena and the Grid environment use a joboption like the one you'd use to run a local

analysis based on TAGs submit the Pathena job use Panda monitoring to check for completion use DQ2 enduser clients (dq2-get) to retrieve the

output files

Problem: AOD to AOD (copy) is problematic in some releases


TAG job with Pathena

PandaPanda

TAG DBTAG DB

ELSSI or POOL Collection

Utilities

ELSSI or POOL Collection

Utilitiesevents.rootevents.root

POOL Collection Utilities

POOL Collection Utilities

events.rootevents.rootevents.rootevents.root

events1.rootevents1.root

Output DatasetOutput Dataset

DQ2 Catalogues

DQ2 Catalogues

PathenaPathena

DQ2 ClientDQ2 Client

jobOptionjobOption

30 jan 2009elizabeth gallas1 introduction to tags elizabeth gallas oxford atlas-uk distributed...

Documents

release tag

atlas tag application

tag contentuses

cvs tag

atlas tag collectornot

tag database application

atlas event selectionhands

tag databasesupported