hall d level 1 trigger dave doughty 1/10/2008 hall d collaboration meeting

Hall DLevel 1 Trigger

Dave Doughty1/10/2008

Hall D Collaboration Meeting

Christopher Newport University

Outline

I. Intro to the Trigger

II. Photon Energy (Tagger)

III. Track Count Processing

IV. Subsystem Processors and Global Trigger Crate

V. Global Trigger Processors

VI. Trigger Timing

VII. Conclusions


L1 Trigger – Current plans

• Five separate subsystems– Barrel Calorimeter - compute energy– Forward Calorimeter - compute energy – Forward TOF - compute number of tracks – Start Counter - compute number of tracks (?)– Tagger – compute photon energy (useful at lower lum)

• Each subsystem computes continuously– Goal - At speed of the FADC pipelines - 250 MHz

• Global Trigger Processor “combines” all five subsystems– 4 level hierarchy: Board -> Crate -> Subsystem -> Global


Level 1 Trigger Block Diagram

BCAL SUM

FCAL SUM

TOF TRACK COUNT

START COUNTERTRACK COUNT

TAGGER ENERGY

PAIRSPECTROMETER

ENERGY SUM PROCESSORSUM/TIME(8 INPUTS)

ENERGY SUM PROCESSORSUM/TIME(8 INPUTS)

FADC

FADC

FADC

FADC

FADC

FADC-VXS-

-VXS-

-VXS-

-VXS-

-VXS-

-VXS-

-Fiber links-12 Crates


ENERGY SUM PROCESSOR**SUM/TIME(8 INPUTS)

** Process Track Counts

-Fiber links-2 Crates* Longest Link *


-Fiber link-1 Crates

GTP

Select FCAL Energy,BCALEnergy,PhotonEnergy,ANDTrack CountsF(x)

TRIGGERSUPERVISOR

-----------------CLOCKTRIGGERSYNC

ROC CONTROL

Signal distribution toFront End Crates(Fiber Links)


The (VXS) Serial Backplane

• Convert Board Data to serial data– 16 bits @250 MHz -> 4 Gbit/s

• Link speed of 3.125 Gbps => data rate => 2.5 Gbps– Packet overhead also!

• Multiple “lanes” likely necessary– But two lanes will work fine

– With four lanes - redundancy for free!

– Higher speed in the future will also help

• Use VXS “Switch Fabric”– VXS has “familiarity advantage” of VME at JLAB


VXS Crate


VXS Serial Routing


photon beam exits fromtagger inside vacuum,continues 70 m downto collimator caveand Hall D

• hodoscope exit window is 1 mm Al1 mm Al• other walls of vacuum chamber are 1 cm Al1 cm Al• exposed side of electron exit channel is 5 mm Al5 mm Al

Tagger in the Trigger

• broad-band focal plane hodoscope

– 144 readout channels

– short scint. rods

– max rate/chan: 2x106 (107)

• focal plane microscope– 120 readout channels

– 600 scint. fibers

– max rate/chan: 2x105 (106)


Microscope scintillating fiber design

focal planeelectron trajectory

SiPM sensors

scintillating fibers

clear light fibers

Design parametersDesign parameters

square scintillating fibers

size 2 mm x 2 mm x 20 mm

clear light guide readout

aligned along electron directionalong electron direction

for reduced background sensitivity

readout with silicon photomultiplierssilicon photomultipliers

(SiPM devicesSiPM devices)


Microscope electronics

SiPM “bases”SiPM “bases”

Programmable constant fraction discriminators: 120 channels High resolution tdc (F1TDC): 120 channels Pulse height needed: Pulse height needed: FADC is a good choice: 120 channels

essential for setupessential for setup useful for monitoring performanceuseful for monitoring performance rates alone give incomplete informationrates alone give incomplete information USED TO PRODUCE TRIGGER!USED TO PRODUCE TRIGGER!

Scalers counting on all channels: 120 channels

provides bias voltage regulationbias voltage regulation: regulation range: 48 ± 8 V regulation stability: 0.1 V regulation control: 0.1 V steps

reasonable packing density: 600 channels on 20x103 cm2


Fixed array electronics

Conventional 1” phototubes, basesConventional 1” phototubes, bases

High voltage for phototubes: 144 channels

Timing needed:Timing needed: High resolution tdc (F1TDC): 144 channels for measuring coherent bremsstrahlung spectrum

time is compared with pair spectrometer trigger

at low intensities (107) can run concurrently with experiment can run concurrently with experiment

at high intensities (108) only for special low-intensity runs only for special low-intensity runs

Pulse height needed: Pulse height needed: FADC may not be a good choice: 144 channels(YES IT IS – if you want to ever trigger on it!)

essential for setupessential for setup

at high intensities, at high intensities, monitor spectrum in integrating modemonitor spectrum in integrating mode

Scalers counting on all channels: 144 channels


Tagger trigger electronics

• TaggerOR– fast logical OR of 120 channels from the microscope

– sent over fiber to GlueX trigger electronics

• Too primitive! We can do better!– Select channels in microscope

– Optionally include main hodoscope

– Without any extra hardware!


Microscope - “Board Tag Word”• For microscope, not interested in a “sum,” we want an

energy “tag” (or two)• Have one crate of FADCs for the microscope

– @16 Flash ADC channels/board -> 8 boards

• Use “hit bit” on each FADC channel as energy tag!– 16 bits – every 4 ns!– These 16 bits are exactly like the 16 bit board sum produced by

the FADC for the calorimeters, except for the interpretation!

• For calorimeters - 16 bit “board sum” every 4 ns• For microscope – 16 bit “board tag word” every 4 ns

• Board tag words from all boards sent to “crate tagger processor”– 120 bits into crate tagger (at 250 MHz)


Microscope Photon Energy Tag• Use one “Switch Slot” for Photon Crate Trigger Processor

– FADCs in payload slots– All “board tag words” sent serially to Photon Crate Energy Tagger– (Note that Photon Crate Trigger Processor is same hardware as

Crate Energy Summer – both are now called Crate Trigger Processors)

• Photon Crate Trigger Processor (CTP) produces a 120 bit composite energy tag word from all microscope channels

• Can download an energy “mask” to Photon CTP– CTP board is loadable via TI board!

• Address of two “highest” or “lowest” energy tag bits are sent to Tagger Subsystem Processor– Need 7 bits to encode 120 bits, with 16 bit word can encode 2 tag

bits– The “rate conundrum”


For Hodoscope – Duplicate the System?

• With 144 channels need 8 bits to encode each energy tag.

• Two hits still fit into one 16 bit word!• No new development!• Option to include in trigger (or use by itself)


Track Processing – at Front End• Easy in principle - just add up discriminators

– How many layers (are there multiple layers)?– Which disc setting (is there a separate for trigger?)– Programmable time window– At pipeline resolution of 250 MHz - 4 ns time stamp– Only one crate of electronics for TOF?– Use for Start Counter as well?

• Create crate track count– Use VXS backplane for communication– Can send any data desired over backplane– Track CTP computes total track count in crate

• Time stamped track count data sent to Track Subsystem Processor (TSP)


From Crates to Global Processor

• Must move crate data (energy, track counts, photon energy) to the global trigger processor crate.

• Use four lanes of fiber optics!


HFBR-7934 Fiber Module

- POP4 compliant Fiber Optic Module. Uses 12 fiber MTP/MPO, 50/125μm multi-mode.

- 8 Used Fibers: 4 Tx, 4 Rx @ 3.125Gbps each for a total bandwidth of 12.5Gbps in each direction

- Small footprint allows 8 modules to fit VME 6U slot

- up to 150m range with 500Mhz-Km fiber, or up to 350m range with 2000Mhz-Km fiber

- Interfaces directly with Xilinx/Altera gigabit transceivers on FPGA


Crate Sum -> Energy Sum Processor


The Global Trigger Crate

• Global Trigger Crate has eight SubSystem Processors (SSPs) on one logical “Side”

• Global Trigger Crate has eight Global Trigger Processors (GTPs) on other logical “Side”

• SSPs are connected to the GTPs via a custom backplane– “VXS-Like”

– 8 x 8 Fully Connected

• Each SSP talks to each GTP via a four-lane Aurora Backplane Link

• Each SSP sources eight four-lane links to the backplane

• Each GTP sinks eight four-lane links from the backplane


The Global Trigger Crate (logical view)

SSP Array

GTP ArraybcalESPs

fcalESPs

tofTSP

strtTSP

photESP

VME/VXS

Clk/TrigIn

TrigOut


The Global Trigger Crate (physical view)

SSP Array

GTP ArraybcalESPs

fcalESPs

tofTSP

strtTSP

photESP

VME/VXS

Clk/TrigIn

TrigOutGTP Array


Global Crate Backplane Layout

• In Logical Layout 64 4-lane signals must be routed across center

• In Physical Layout 32 4-lane signals must be routed across center

• In “Normal” VXS 18 4-lane signals are routed to each of two center boards

• Will route!


A Similar but Smaller Backplane (Bustronic)


Subsystem Processors (SSPs)• Each of the subsystem processors resides in Global Trigger Crate• All subsystem processors are same physical PC boards!• Each SSP can receive up to eight four-lane “crate data links”• Each SSP has multiple Xilinx FPGAs for receiving and processing

data from crates• Some SSPs use two boards (because of crate count)

– If so – both board “Partial Results” sent to global processor• Eight SSPs are needed:

– Two for BCAL – Energy Subsystem Processor (ESP)– Two for FCAL – Energy Subsystem Processor (ESP)– One for TOF – Track Subsystem Processor (TSP)– One for Start Counter – Track Subsystem Processor (TSP) ????– One for Tagger – Photon Energy Subsystem Processor– One spare!

• Each subsystem processor sends time-stamped Subsystem Event Reports (SER) to all Global Trigger Processors


Global Trigger Processors (GTPs)• Run at the same 250 MHz pipeline speed as the subsystems• Receive (via backplane) data from all subsystems

– TTOF - Tracks Forward TOF– EFCal1 - Energy Forward Calorimeter (partial sum 1)– EFCal2 – Energy Forward Calorimeter (partial sum 2) – EBCal1 – Energy Barrel Calorimeter (Partial sum 1)– EBCal2 – Energy Barrel Calorimeter (Partial sum 2)– Tagger – Photon Energy– STRT – Track count in Start Counter ?

• Adds partial sums (but can use independently)• Adjusts data to match time-stamps• Does “global trigger computation”

– Z >= TFM*TTOF + EFM*EFCal + RM*((EFCal +1)/(EBCal + 1))– Or something similar


GTPs - II

• Each GTP has multiple internal processors (FPGAs), each looking for one of four triggers

• With eight GTPs allows 32 different triggers • Output to electronics is “Programmable Delay”

after computed trigger event time– Produces a result with constant latency (relative to

time-stamped data)

– Simplifies pipeline “lookback” for front end electronics - it’s fixed

• Interfaces with TS


Trigger Timing

– Board sum 64 ns

– Transfer to crate sum board 512 ns

– Crate sum 64 ns

– Link to subsystem 512 ns

– Subsystem trigger processing 256 ns

– Transfer SER to GTP 512 ns

– GTP 256 ns

– Level 1 output to FEE 256 ns

TOTAL = 2.432 S - design FEE for 3.5 s


Conclusion – Where is Level 1?• In relatively good shape!

– Thanks to electronics group here at JLAB– Have plans and/or proof of concept for most of system– Minimal number of boards to be developed (but of course more

“software” versions)• Crate Trigger Processor (CTP – simple prototype done)• SubSystem Processor (SSP)• Global Trigger Processor (GTP)

• Things we still need– Re-do of L1 simulation

• Current simulation dates from 2001• Chinese post-doc here at the lab

– Track Count Processing


SSP ->Energy Sum Processors (ESPs)

• Used for both BCAL and FCAL• Logic of both ESPs is essentially same as crate

sum• Each ESP uses data from all (up to eight) its crates• Matches time-stamps and adds data

– Knows time-stamp offsets for its subsystem (if any)

• Passes energy sum results to all trigger processors– SER looks like:

<Hdr><Time><Energy>– Time resolution is 4 ns


Where the ESP sits in the Pipeline


ESP Block Diagram (from Ben)


SSP ->Track Subsystem Processor (TSP)

• Used for TOF and perhaps Start Counter

• Logic of both TSPs is essentially same as crate sum

• Each TSP uses data from all its crates

• Matches time-stamps and adds data– Knows time-stamp offsets (if any)

• Passes total track counts when over threshold to trigger processors

• SER looks like:<Hdr><Time><Count>

– Time resolution is 4 ns


SSP -> Tagger Energy Processor (TEPs)

• Process data from Microscope and Hodoscope (?) • Matches time-stamps to correlate (if desired) data• Pass tagger energy results to global processor (in

same crate)– SER looks like:

<Hdr><Time><Photon Energy>

– Time resolution is 4 ns

hall d level 1 trigger dave doughty 1/10/2008 hall d collaboration meeting

Documents