CMS June 20031
TriDAS UpdateDrew Baden
University of Marylandhttp://www.physics.umd.edu/hep/HTR/hcal_june_2003.pdf
USCMS HCALUSCMS HCALUSCMS HCALUSCMS HCAL
CMS June 20032
HTR StatusHTR StatusHTR StatusHTR Status
• Rev 1 run Summer 2002 testbeam Board worked well – all functional requirements met Big concern on mechanical issues for production
o Had a difficult experience with previous board manufacturing
• Rev 2 produced March 2003 Board production changes:
o New assembler, in-house X-ray, DFM review, QCo Gold plated (Rev 1 was white-tin) for better QC
Changes to HTR:o Change from Virtex 1000E FBGA (1.00mm) to Virtex2 3000 BGA (1.27mm)o Added stiffenerso Moved all SLB/TPG output to front-panel daughterboardso Modified Rx refclk scheme (the usual TTC/refclk clocking concerns)
Full 48 channel capability (Rev 1 was “half HTR”) As of this date, no issues – this board is functioning well
CMS June 20033
Dual-LC O-to-E
VME
Deserializers
Xilinx XC2V3000-4
Sti
ffen
er
s
6 SLBs
TTC mezzanine
HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)
CMS June 20034
HTR Rev 3HTR Rev 3HTR Rev 3HTR Rev 3
• 30 boards delivered April 21 Checkout consisted of
o All systems except connectivity to SLB
o Fiber links checked out at 1.7Gbaud bit rate (1.6Gbaud is CMS requirement)Frame clock up to 2.0Gbaud bit rate and it stays synchronizedNo BER yet…will do a lab measurement soon12 boards x 16 links ~200 links(~5% of total) with no problemsUsed both onboard crystal oscillator and external clock for REFCLK
Minor adjustments will be needed for front panels, stiffeners, etc.
Will battle test these boards this yearo May synchronous testbeam just completed
o July testbeam
o Vertical Slice tests to commence in the fall
CMS June 20035
Clocks and SynchronizationClocks and SynchronizationClocks and SynchronizationClocks and Synchronization
• Clocking considerations can be divided into 2 parts: Deserializers REFCLK: stability critical (80MHz frame rate)
o Stability: must have a very low jitter – 30 to 40ps pkpk spec
o Frequency: TI TLK2501 spec is 100ppm (8kHz) to lock Measured ~350ppm (30kHz) needed to establish link
LHC variation expected to be few kHz
Once link is established, just needs to be stable (it’s a REFCLK!!!)
o Phase: relationship to LHC clock totally irrelevant
Phase critical clock for pipeline synchronizationo Must be in phase with LHC clock
o Jitter spec is very lose – this clock is used inside FPGA for sequential logic
CMS June 20036
HCAL Clock FanoutHCAL Clock FanoutHCAL Clock FanoutHCAL Clock Fanout
• HTR clocks provided by a single 9U VME board Chris Tully/Jeremy Mans from Princeton Has fiber TTC input
• Signals fanned out over Cat6 twisted pair: TTC stream
o To be used by each HTR and by DCC to decode commands & L1A
BC0o To be used by SLBs to synchronize TPGs
“40MHz” clocko To be used by FPGA and SLBs to maintain pipeline
Comes from QPLL
“80MHz” clean clocko To be used for deserializer REFCLK
Comes from QPLL
CMS June 20037
Clock DistributionClock DistributionClock DistributionClock Distribution
HTR
TTC fiber
TTC
CLK80
BC0
CLK40
distributionto 6 SLBs
and to 2 Xilinx
Brdcst<7:0>,BrcstStr, L1A
O/E
BC0
TTCTTC
TTC
FPGA
..
..
TestPoints forRxCLKand RxBC0
..
..
..
..
80.18 MHz
..
..
..
..
TTCrx
to Ref_CLK of SERDES(TLK2501)
CLK40
CLK80
Princeton FanoutBoard
TTCrx
QPLL
HTRHTR
Cat6Eor Cat7Cable
CMS June 20038
TTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumd
• General purpose TTC receiver board (TTCumd)
TTCrx ASIC and associated PMC connectors
• Will be used to receive TTC signal by HTR, DCC, and clock fanout boards• No signal receivers!
Copper/fiber receivers must be on the motherboard
Signal driven through TTC connectors
• Tested successfully by Maryland, Princeton, BU groups
Princeton Fanout Card
CMS June 20039
May Testbeam SetupMay Testbeam SetupMay Testbeam SetupMay Testbeam Setup
• Lack of a QPLL or decent equivalent – had to improvise: Front-end used commercial Cypress PLL
o Matches GOL 100ps pkpk jitter spec
Fanout card o No clean 80MHz REFCLK, so provided 2 alternatives:
2xLHC clock from crystal oscillator High quality clock from HP signal and pulse generators Jumper selectable on mezzanine cards
o No clean 40MHz system clock Just used 40MHz output from TTCrx chip anyway
CMS June 200310
May Testbeam ExperienceMay Testbeam ExperienceMay Testbeam ExperienceMay Testbeam Experience
• To establish link: FE
o TTC 40MHz clock cleaned up by Cypress “roboclock” chip (Cy7B993)
o FE reset signal to GOL
Fanout cardo Fanout from onboard 80.1576MHz crystal oscillator for REFCLK
o Fanout TTC 40MHz clock for system clock
HTRo TLK2501 link circuitry always enabled
Result: Fiber 1.6GHz link established oko No problem locking – worked every time.
CMS June 200311
Testbeam Experience (cont)Testbeam Experience (cont)Testbeam Experience (cont)Testbeam Experience (cont)
• Link stability - VERY PRILIMINARY, STILL STUDYING Ran 10hr test on 48 fibers
o 3 x 1015 bitso 20% failed to maintain linko During synchronized beam running, sent reset between spills to ensure link
Similar tests at Maryland using TI eval board showed no link errors, similar number of bits sent
• Curret plan Study FE →HTR link at FNAL this month
o FNAL test stand setup this week Investigate noise characteristics of H2 environment
o H2 is clearly different than FNAL, Maryland and BU experience Review of HTR and Fanout card
o Will learn what we need to do from the above
• Best guess All tests in US indicate solid link, but experience in H2 disagree Probably some kind of new noise component – figure out and correct.
CMS June 200312
HCAL TPGHCAL TPGHCAL TPGHCAL TPG
• Nothing new since May Electronics Week• TPG under development…
Preliminary FPGA code for TPGs doneo LUT for linearization (downloadable), 0.5GeV steps, 255Gev max ET
o E to ET and sums over as many as 7 channelsNot implemented in code yet…TBD
o Muon window in Eo BCID filter algorithm TBD from testbeamso Compression LUTs for output to SLBs
Utilization is ~50% of Virtex2 3000o We are confident this chip will be sufficient
Simulation effort under way…
• Latency issue See below – we are working on this…
CMS June 200313
HTR TPG CommissioningHTR TPG CommissioningHTR TPG CommissioningHTR TPG Commissioning
• 2 Xilinx FPGAs per HTR 3 SLBs per Xilinx Each mounted on triPMC
connectors
• Will test internal connectivity to SLBs at UMD For signals and for
localbus connections
• Need a scheme to test HTR/RCT connectivity Not just electrical! Also
includes data integrity
SLB
SLB
SLB
SLB
SLB
SLB
Xilinx
Xilinx
RCT
CMS June 200314
HTR/RCT TestingHTR/RCT TestingHTR/RCT TestingHTR/RCT Testing
• Will build PMC tester card to mount onto HTR Host to 1 or more Wisconsin
RCT Vitesse receiver boards
• Will run the signals from this RCT tester card back into Xilinx Using 1 HTR, both FPGAs –
one source, one sink – to test sending data from HTR to RCT
• Will try to engineer 3-SLB test to test single Xilinx → SLB →RCT
SLB Xilinx
Xilinx
RCT
CMS June 200315
HTR ProductionHTR ProductionHTR ProductionHTR Production• Full contingent of HTRs: 260 boards
Includes 10% spares, 20% spares for parts
• Full production will begin after: Testbeam demonstrates I/O works under battle conditions Successful testing of the 6 SLB daughter card functions Understanding of how to meet latency issues
o We are still some clock ticks short, but firmware is still very immature for the TPG part of the HTR (see slides below)
• Best guess: fall 2003 There is no reason to hurry other than to finish with the R&D part
of the project Current board design will be final, perhaps some layout
adjustments based on conclusion of testbeam effort
CMS June 200316
Overall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning Schedule
• Summer 2003 testbeam Repeat previous test w/production prototype boards
• Fall 2003 Slice tests HCAL will join as schedule allows
• 2003/2004 HCAL burn-in Continue with firmware development/integration as needed
• 2004/2005 Vertical Slice and magnet test We will be ready All HCAL TriDas production cards involved
• October 05 beneficial occupancy of USC Installation of all racks, crates, and cards We do not anticipate any hardware integration
o Should be all firmware / timing / troubleshooting
CMS June 200317
TPG LatencyTPG LatencyTPG LatencyTPG Latency
“Minimizing the trigger latency”
Item LatencyTOF .5
HCAL Optics 1
FE (CCA+QIE) 8-9
GOL 2
Fiber Tx to HTRs 18
Deserializer 2-3
HTR Alignment 6
HTR TPG path 5-10
SLB 3
TPG Cables 4
TOTAL 50 - 57
• Current total 50 – 57 clocks Very rough guesses
o Many numbers have not been measured
• Optimizations: Fiber cables need to be 90m? HTR firmware needs optimization Deserializer random latency fix TPG cables changed to 15m will save
1 tick Others…main efforts over next 6
months
CMS June 200318
TPG PathTPG PathTPG PathTPG Path
ETcomp
7 10
Muon bit
SumConsecutive
Time-samples9
TP
8
QIE-dataINPUT
LUTLineariz.
and Et
ET[9:0]
2
Compression
LUT
2Muon LUT
1
Delay to synchronize
with BCID
10
L1 Filter
10
Sum in ET
PeakDetection
TP_Bypass
1
0
2 2
2
Mask &Reset
“NO-SHOWER” LUTtake care of cases where showers can leak into a cell and incorrectly set the muon bit.
BCID
BCID avoids to flag as a muon the tail of a
more energetic event