ftk: a fast track trigger for atlas
DESCRIPTION
FTK: a Fast Track Trigger for ATLAS. Anton Kapliy. Overview. FTK is a hardware system for fast online track finding in the ATLAS detector that will be installed in 2014 Chicago plays a central role in FTK collaboration: 3 professors, 4 postdocs, 5 graduate students, 2 engineers - PowerPoint PPT PresentationTRANSCRIPT
FTK: a Fast Track Trigger for ATLAS
Anton Kapliy
Anton Kapliy - Sugarman Award Presentation
2
Overview
6/4/2012
FTK is a hardware system for fast online track finding in the ATLAS detector that will be installed in 2014
Chicago plays a central role in FTK collaboration: 3 professors, 4 postdocs, 5 graduate students, 2
engineers Principal Investigator: Mel Shochet
I’ve been involved in FTK since 2006 and participated in design, simulation, hardware, and initial commissioning. Motivation for having FTK Conceptual description of the system Simulation and expected performance Electronics development and commissioning
Anton Kapliy - Sugarman Award Presentation
3
An LHC collision
6/4/2012
LHC collides bunches of protons, not individual protons Currently: ~25 interactions per collision. Future: >70 Most of these interactions are “boring”
Our job is to find interesting physics among this mess
April 15th 2012: a Z➛ 𝝻𝝻 candidate event
Anton Kapliy - Sugarman Award Presentation
4
Interesting physics
6/4/2012
LHC goals: Higgs, SUSY, other BSM This stuff is rare!
Tracking system is critical for selecting such events from vast backgrounds: Isolated electrons and muons
Produced in decays of W and Z bosons 3rd generation fermions: taus, b-quarks
Expect a lot of them because whatever participates in EWK symmetry breaking couples to particle mass
Rates of production:
new physics
Example I:Three collimated tracks in a triple-prong tau decay:
Example II:Displaced secondary vertex in B-hadron decay
Anton Kapliy - Sugarman Award Presentation
5
ATLAS Tracker
6/4/2012
Pixel and SCT tracking detectors are relatively small But due to high granularity, they contain a whopping
86M readout channels (>98% of the ATLAS total!)
2T solenoid B-field
A typical charged track bends its way out through 3 Pixel and 8 SCT layers:
Anton Kapliy - Sugarman Award Presentation
6
ATLAS Trigger System
6/4/2012
Trigger necessary to reduce event rate in real-time from 40 MHz to ~200 Hz Of course, we would like to keep all “interesting physics” and reject boring stuff
Using tracking detectors for this rejection is very challenging due to high occupancy Impossible in LVL1 hardware. In LVL2, it is only possible within narrow cones (even that is slow!)
Available data:• 40 million events/sec• Each event is O(1
MB)→ 1 Library of Congress / sec
Able to save on disk:~100 MB/s→ about 200 events/sec
Anton Kapliy - Sugarman Award Presentation
7
FTK comes to the rescue
6/4/2012
FTK receives a complete copy of tracking data at LVL1 rate (75-100 KHz) Quickly reconstructs all tracks with pT>1GeV and makes them available for LVL2 Tracks are ready by the time LVL2 starts its selection of interesting physics!
FTK
FTK Buffer
FTKA dedicated hardware system for track-finding that sits between LVL1 and LVL2 trigger stages
Anton Kapliy - Sugarman Award Presentation
8
FTK approach: pattern recognition
Superstrips: ~1mm
1. Given a collection of hits in an event, we gang them together to form coarse “superstrips” – about 1 mm wide.
Original hits: ~50 microns
2. Build a pre-calculated lookup table of all coarse paths (“patterns”) that a charged track might take through the tracking layers. For full ATLAS detector, we expect about 1 billion such patterns
Monte-Carlo tracks
3. Load these patterns into specialized hardware – Associative Memories – that can simultaneously compare the event with ALL stored patterns and quickly return only those that match.MATCH!
Anton Kapliy - Sugarman Award Presentation
9
FTK approach: track fitting4. Restore full-resolution hits inside each matched pattern.
Ultimately, we need to create a list of tracks with:• χ2 (track quality)• Curvature (~ 1/pT)• Phi• Impact parameter,
etc
5. Because our patterns are narrow, we have very few hits inside of them.The remaining combinatorial problem can be solved via the brute-force method – i.e., trying out every combination.
6. For each combination, perform a linearized fit to arrive at final track parameters.Since these fits involve only scalar products, they can be performed VERY quickly in modern FPGA chips: 1 fit / ns
Matched pattern from previous stage along with all of its hits
Pre-computed constants
hit coordinatestrack parametersand χ2 componentsThis pattern has two combinations:
Finally, we apply a χ2 cut to remove bad tracks, perform duplicate removal, and send all final tracks to LVL2 trigger.
Anton Kapliy - Sugarman Award Presentation
10
Simulation – challenges
6/4/2012
If we could simulate FTK very quickly, we wouldn’t need to build it – just stick the software algorithm into LVL2 trigger!
In reality, FTK simulation is very challenging: Associative Memories need a lot of RAM to store 1 billion patterns CPUs are not well-suited to perform millions of track fits → each event must be simulated in ~100 separate jobs
Requires unprecedented parallelism not seen anywhere else in ATLAS! To simulate 1 million FTK events for 2016-2017 luminosity
profile, we needed about 500k PC-hours This would take ~50 years to simulate on a single PC
We wrote an advanced simulation framework that fully exploits the parallelism afforded by the HEP grid. Simulation time reduced to about 1 week
Anton Kapliy - Sugarman Award Presentation
11
Simulated FTK performance
6/4/2012
FTK performance is comparable to offline reconstruction However, FTK hardware is 3 orders of magnitude faster!
Per-event time: ~25 µs for FTK, 10-100 ms for offline
Reconstruction of track impact parameter (D0) is particularly important to identify jets with a B-hadron.
D0
12
Example: FTK selects W boson events
6/4/2012Anton Kapliy - Sugarman Award Presentation
Identification of W→µν decays becomes challenging as the number of interactions per collision increases: Electromagnetic calorimeter isolation used in the trigger
becomes powerless due to other “stuff” contaminating the event Track isolation using FTK tracks restores good efficiency
Add FTK tracks
Anton Kapliy - Sugarman Award Presentation
13
FTK architecture
6/4/2012
FTK operates in 64 θxϕ towers divided between 8 core crates.Each crate consists of a number of boards, each with a dedicated function.
HOLAs
Finished and commissioned by ChicagoInstalled @ CERN
Design started @ Fermilab
Prototypes @ Italy
Design started @ Argonne
Design started@ Illinois
Design + prototypes @ Chicago
14
First FTK installation @ ATLAS
6/4/2012Anton Kapliy - Sugarman Award Presentation
to FTK
HOLA: a high-speed (2 GBps) link that provides a copy of all tracker hits to FTK
to ATLAS
Development & testing@ Chicago
FPGA
Highlights:• Data transmission protocol similar to IEEE 802.3
(Ethernet)• Transmission medium: optical fiber• Flow control from downstream cards on both
channels• If this card fails, ATLAS loses the tracker
• → extensive stress-testing to ensure reliable operation
• Produced 270 cards (total system bandwidth ~50 GB/sec):• 268 passed our stress tests; 2 had a soldering
problem• 31 installed at CERN and are being used now
Installation & commissioning@ CERN
Anton Kapliy - Sugarman Award Presentation
15
Conclusions
6/4/2012
As LHC luminosity increases, it will be essential to efficiently select interesting physics from the vast background of boring events
Tracking plays a key role in identifying isolated leptons, b-quarks, and taus
FTK brings the power of tracking-based event selection to LVL2 trigger
Schedule: Tested board prototypes & TDR – by Summer 2013 Full system ready after the 2013-2014 LHC shutdown
FTK is a challenging, exciting and very educational project: Development and simulation of a complex trigger system State-of-the-art electronics development, testing, and
commissioningTHANK YOU!