n ational e nergy r esearch s cientific c omputing c enter 1 c. leggett intelligent (or not...
TRANSCRIPT
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
1C. Leggett <CGLeggett@ lbl.gov>
Intelligent Intelligent (or not completely stupid)(or not completely stupid)
Unpacking Unpacking
Structure and Timing Studies of the Level 3
Calorimeter Unpacking Tool
l3tCalUnpl3tCalUnp
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
2C. Leggett <CGLeggett@ lbl.gov>
Calorimeter UnpackerCalorimeter UnpackerL3TCalUnpL3TCalUnp
• Level 3 unpacking tool.
• Selectively unpacks sections of the calorimeter according
to input directives:
– Level 1 & 2 seed towers
– Entire calorimeter
– By region
– Size of neighborhood to unpack
– Energy thresholds
– Hot cell flagging thresholds
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
3C. Leggett <CGLeggett@ lbl.gov>
Low Level StructureLow Level Structure
• The calorimeter unpacker will share many low level unpacking routines with other subdetectors.
• Reads in RawDataChunks, and accesses them on a per-module (ADC card) basis, which is the granularity of the unpacker. This means that if you ask for 1 channel in a module, the unpacker will unpack all 384 channel, reducing the time to access other channels in the same module.
• No method for getting the L2 trigger seed tower information exists, so artificially creates seed towers according to various RCP flags.
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
4C. Leggett <CGLeggett@ lbl.gov>
Data StructureData Structure
• static float CalUnpData[Neta][Nphi][Nlayer]• Neta = 74
• Nphi = 64
• Nlayer = 18
– array of cell energies and towers (layer = 0)
• static vector<float *> CalUnpEt
– pointers to CalUnpData[eta][phi][0] towers
– for new towers with energy deposits
• various helper routines to access position information, hotcells, to go between array indices, array addresses, and physics indices, etc
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
5C. Leggett <CGLeggett@ lbl.gov>
Memory RequirementsMemory Requirements
• CalUnpData: 85248 entries = 333.0 kB• only 47456 valid cell addresses
• CalUnpEt: ~18kB (10 % calorimeter occupancy)
• various other internal data: ~10kB
• memory is allocated at beginning of run, and reused. Very
little dynamic memory allocation occurs.
• Big, with lots of empty space.
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
6C. Leggett <CGLeggett@ lbl.gov>
Initial Setup : Initial Setup : MakeMe()MakeMe()
• Initialize CalUnpData, and other variables
• Reserve space for all stl objects
• Calculate and fill lookup tables
• Queries run_config_mgr to determine hardware configuration, then builds translation tables between (moduleID, crateID) and (crate number, slot number)
• Load CalGainTable
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
7C. Leggett <CGLeggett@ lbl.gov>
Unpacking Procedure : Unpacking Procedure : DoThisTool(---)DoThisTool(---)
• The DoThisTool() method has been overloaded, so as to tell the unpacker which regions of the calorimeter are to be unpacked. It will currently take:
• DoThisTool()• DoThisTool(vector<TowerAddress> &TA)• DoThisTool(vector<TriggerTowerAddress> &TTA)• DoThisTool(vector<short> &eta,
vector<short>&phi)• DoThisTool(vector<float> &eta,
vector<float>&phi)• DoThisTool(L3Region &)• DoThisTool(L2Seeds &)
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
8C. Leggett <CGLeggett@ lbl.gov>
Unpacking Procedure : Unpacking Procedure : DoThisTool()DoThisTool()
• Reset vector of CalUnpEt
• Loop over requested seed towers
– Identify neighbors (radius n), form list of towers to unpack
• Loop over selected towers
– Identify associated module(s) (ADC card)
• If the module is not already unpacked, unpack it into CalUnpData
– For all cells in module, calculate energy if has non-zero ADC count
• Add current tower to list of populated towers
• For all newly populated towers, calculate tower energy,
place in layer 0 of CalUnpData
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
9C. Leggett <CGLeggett@ lbl.gov>
Unpacking Procedure : Whole CalorimeterUnpacking Procedure : Whole Calorimeter
• Reset CalUnpEt• Get all modules in the calorimeter
• Hopefully, this type of unpacking will be called after directed unpacking has already occurred
• Loop over all (new) modules– For each cell, calculate energy
– Set flag showing tower has energy deposits
• For all (new) towers with energy deposits, calculate tower energy
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
10C. Leggett <CGLeggett@ lbl.gov>
Unpacking Procedure Unpacking Procedure (cont)(cont)
• Do something with hotcells
• When all selected towers have been processed, fill a vector
(CalUnpEt) of pointers to layer 0 of CalUnpData array
for all new towers that have energy deposited in them.
• Sort CalUnpEt according to Et of associated tower in a
decreasing order.
– Assumes vertex = (0,0,0) - may apply correction for other
vertices
– expensive - requires atan(e - ): uses a lookup table instead of
calculating it each time
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
11C. Leggett <CGLeggett@ lbl.gov>
Unpacking Procedure: Unpacking Procedure: Reset()Reset()
• Loop over all unpacked towers and cells, and zero CalUnpData
• Reset vector of all unpacked modules
• Reset vector of all unpacked towers
• Accumulate statistics for hot cell killer
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
12C. Leggett <CGLeggett@ lbl.gov>
Timing StudiesTiming Studies
• Majority of time spent in low level routines, such as extracting the module ID for a given seed tower, or translating physics cell addresses to electronic addresses. Already uses lookup tables for some of this, may use more.
• Significant time spent sorting and unique-ing vectors.
• Time does not rise linearly with number of cells/towers unpacked.
• Impossible to say if current implementation is satisfactory, as we don’t have an optimized version of the D0RunII software that runs on NT
• WAY OUT OF DATE
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
13C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
t-tbar sample, no lookup tables
running on d02ka
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
14C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Using lookup tables
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
15C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Unpacking Entire Detector
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
16C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Directed unpacking, trigger threshold = 0.5GeV
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
17C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Directed unpacking, trigger tower energy > 2 GeV
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
18C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Directed unpacking, trigger tower energy > 2 GeV, radius = 7 towers
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
19C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Directed unpacking, trigger tower energy > 5 GeV
NATIONAL ENERGY RESEARCH SCIENTIFIC COMPUTING CENTER
20C. Leggett <CGLeggett@ lbl.gov>
Timing ResultsTiming Results
Directed unpacking, trigger tower energy > 5.0 GeV, neighborhood = 7