1

TOF Electronics Overview

J. Schambach

University of Texas

DoE Review, BNL, 25 Sep 2006

2

Outline

• Electronics “Essential Model”

• Trigger & DAQ Interfaces

• Board Status

• Production & Testing

3

TOF Electronics Top Level

DAQ Room

STAR Detector WestSTAR Detector East

Start Detector

Electronics

60 Trayswith RPCs

Start Detector

Electronics

Fiber

60 Trayswith RPCs

Fiber

HV HVLV LVControl PC

withCANBUS I/F

CA

Nbu

s

Rack Electronics

DAQ

22

2

CA

Nbu

s

2

TCD

Trigger

2 2

L0 Trigger

60

Slow Controls Slow Controls

HV HV

Slow Controls

L0 Trigger

60

Trigger

TriggerLevel-2

Fib

er

2

Fiber

2

4

Electronics for One Side

Start Detector with

19 Scintillators &19 PMTs

Start DetectorElectronics

Tray

32 RPCWith 6 channels

eachOn-Tray Electronics

Data, Clk, Ctrl

CANbus

30Trays

L0 Trigger

Data, Clk, Ctrl

Dual Fiber DAQTHUB

Master ClockC

AN

bus

Tray

32 RPCWith 6 channels

eachOn-Tray Electronics

Data, Clk, Ctrl

CANbus

30Trays

L0 Trigger

Dual Fiber DAQTHUB

Slave Clock

CA

Nbu

s

OneDetector Side

CA

Nbu

s

Clk

, Ctr

l

To other THUB’s

Clk, Ctrl

Clk, Ctrl

Control PCwith

CANBus I/F

TCD & BusyLogic

Dual Fiber

Dual Fiber

L2

L2

5

Tray Level Electronics

Start Detector

TDIGTDIGTPMDTPMDPMTsPMTs

TOF Tray

TDIG TDIG

TCPU

DATA, CLOCK& CANbus

TINO TINO

MRPC MRPC

96 CHAN

TRAY CAN BUS

MULTIPLICITY

THUB

COPPER: DATA, SAMPLE CLOCK, RESET TRIGGER STROBE & DATA

TOP LEVEL CAN BUS

COPPER LINKS TO 29 TRAYS

SIU FIBER

CAN BUS TO 29 TRAYS

TCD

DAQ

96 CHAN

4

L0 Trigger

4

PMTs TPMD TDIG TCPUData, Clock& CANbus

Cop

pe

r:D

ata

, S

am

ple

Clk

, R

eset

Trg

Str

obe

& D

ata

Reset & ClockFrom other THUBs

Reset & ClockTo other THUBs

To Control PC

L2TDIG TDIG

TINO TINO

MRPC MRPC

44

SIU FIBER

Clock & Commands

Busy

Fast Z-Vertex Electronics

3 3

6

Front-End Electronics “TINO”TINO

NINO C

NINO B

NINO A(8 ch Amp / Disc)

MRPC 4(6 ch)

MRPC 1(6 ch)

MRPC 3(6 ch)

MRPC 2(6 ch)

Multiplicity to TDIG PLD(1 bit LVDS)

Hits to HPDTC(8 LVDS)

Threshold(All Common from TDIG)

MR

PC

Inte

rfac

eM

RP

C In

terf

ace

MR

PC

Inte

rfac

eM

RP

C In

terf

ace

6

6

6

6

6

6

2

4

4

2

8

8

TD

IG Interface

TD

IG Interface

TD

IG Interface

8 per tray

960 total

7

CERN/LAA NINO Chipdeveloped for ALICE

Parameter Value

Peaking time 1ns

Signal Range 100fC – 2pC

Noise (with detector) < 5000 e- rms

Front edge time jitter <25ps rms

Power consumption 30 mW/ch

Discriminator threshold 10fC to 100fC

Differential Input impedance 40Ω< Zin < 75Ω

Output interface LVDS

8

Digitizer Board “TDIG”8 per tray

3 per start side

966 total

Altera FPGA

HPTDC(LE & TE)

HPTDC(LE & TE)

HPTDC(LE & TE)

TINO Interface

8

88

3

Inte

rfac

e to

pre

viou

s T

DIG

MicroController

CANbus

Temperature sensors

External memory

Out

put i

nter

face

to n

ext

TD

IG o

r T

CP

U

Upstream data outbound

For Multiplicity

PLD control and status monitoring

Trigger & clocks inbound

Config & Status (3 TDCs)

8 p

air

fa

st m

ulti

plic

ity,

seri

al d

ata

rd

o s

ign

als

, cl

ock

s,a

nd

re

sets

Sa

me

as

ou

tpu

t in

terf

ace

TEMP & Power

IDTray position

switch

9

INL Correction

• Sigma = 0.9 timebins = 22 ps

• Implies single channel resolution of 16ps

10

Tray Controller “TCPU”

1 per tray

1 per Start Detector

122 total

MCU

TRAY LEVEL CAN BUS I/F

TOP LEVEL CAN BUS I/F

THUB-TCPU Interface

Trigger Data, reset inboundDATA outbound

TDIG CABLE I/F

TDIG CABLE I/F Trigger & clock, reset outboundTDC & Mult. Data inbound

Clock select and distribute

Local osc

To TDIG cables

RAM

TRAY ID

Mu

ltip

licity

Su

m

AlteraPLD

L0 TriggerInterface

Sa

mp

le C

lock

11

DAQ/Trigger Interface “THUB”

2 per Detector side

4 total

AlteraFPGA

MCU

TCDInterface

External ClockInput

ClockGeneration

Clock Outputs To 3 other THUB

BufferRAM

TCPU – THUBInterface

TCPU – THUBInterface

TCPU – THUBInterface

31

DDL SIU

Local Oscillator

System LevelCANbusInterface

TDC & Mult DATA

3

PR

EC

ISIO

N C

LOC

KS

TO

TC

PU

s

31

Trigger Data

& Strobe, Resets

31

Command, Token & Clock

BUSY

3 Reset InputFrom other THUBs

3

Reset OutputTo other THUBs

DDL SIU

DAQ

L-2

12

THUB Design

SERDES0-3

SERDES0-3

SERDES0-3

SERDES0-3

SERDES0-3

SERDES0-3

SERDES0-3

SERDES4-7

SERDES0-3

SERDES0-3

SERDES0-3

SERDES8-11

SERDES0-3

SERDES0-3

SERDES0-3

SERDES12-15

SERDES0-3

SERDES0-3

SERDES0-3

SERDES16-19

SERDES0-3

SERDES0-3

SERDES0-3

SERDES20-23

SERDES0-3

SERDES0-3

SERDES0-3

SERDES24-27

SERDES0-3

SERDES0-3

SERDES0-3

SERDES28-31

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

0.5M RAM0.5M RAM0.5M RAM0.5M RAM

CYCLONE FPGA

672 pin

36

36

36

36

36

36

uC

SIU

TCD

CAN

CLOCKS

Power supplies

CPLD

RESET OUT(3)

RESET IN

36

36

Clocks OUT(3)

Clock IN

SIU

13

ALICE DDL LinkFront-end electronicsFront-end electronics

DetectorData Link

DDL SIUDDL SIU

DDL DIUDDL DIU

RORCRORC

SourceInterfaceUnit

DestinationInterfaceUnit

ReadOutReceiver Card

PCPCData

Acquisition PC

Optical Fibre~200 meters

14

PMT Input Board “TPMD”3 per Start Detector

6 total

DISCRIMINATOR(1/2 MAX9601)

PMT in (x8)

TDIG Connectors

8

From TDIG dac

THRESHOLD Buffer (& Internal set)

External In

ISOLATION BUFFER

(AD 8009)

Z Vertex out

Leading EdgeTrailing Edge

Z Vertex out

Terminator

**

x8

INPUT PROTECTION

CLAMP

15

Interface to L0 trigger

• Provides multiplicity at 9.4 MHz with <~700 ns latency.• The multiplicity range is 0-24 for each tray, where one is added to the sum if

any of the 8 TOF channels in a NINO chip is above threshold.• TDIG receives the multiplicity information from the NINO and passes it to

TCPU. Each TCPU will send the multiplicity over 5 pairs to the Level 0 DSMI on the south platform. These cables all need to be the same length (+- 1 ns).

• Each DSMI can handle 128 bits of input. We plan to use 100 bits on each DSMI, representing 20 trays. Each DSMI has 8, 32-pin connectors. 6, 32-pin connectors will each serve 3 trays, 1 connector will serve 2 trays, and 1 will be unused.

• We plan to time in the signal to trigger by trial and error. We plan to scan the trigger info in ~10 ns steps using various cable lengths to find the sweet spot. The logic pulse should be >30 ns.

• Initially (this year), we will “self-latch” the data. Each TDIG will send a value between 0 and 3 to TCPU, TCPU sums the data and sends it to the DSMI

• If “self-latching” doesn’t work, we are considering modifying the DSMI to send the RHIC strobe to each tray’s TDIGs, learn the phase of this clock at each TDIG, and use it to latch the data on each TDIG

16

Interface to Level-2 Trigger• Interface is the identical DDL link that will be used to

connect TOF to DAQ.• Initially, it was planned to provide a 23k bit map of the

TOF hits to L2 for each L0 trigger, 192 bits per tray, where each bit represents a hit channel.

• In principle, same (timing) data presented to DAQ can also be provided to L2, needs just a change in firmware

• Data volume is big:– Fully occupied TOF: 385,888 bits per event per THUB – Assuming <20% occupancy: 77,178 bits per event per THUB – 10kHz L0 rate: 736 Mbits/sec per fiber ~ 92MBytes/sec per fiber – Total for 4THUBs: 2944 Mbits/sec ~ 368 MBytes/sec, four fibers

• Data is uncorrected, needs INL correction (lookup table?) and slewing correction

17

Interface to DAQ

• The TOF system needs to be faster than the upgraded TPC so as not to introduce any additional dead time. The TOF information is only useful in a STAR event if the TPC is also readout in that event (for tracking).

• The system is able to handle L0 accept commands at >10 kHz.

• The system will not process L2-accept or Abort commands, but rather pass those on to DAQ (as separate events) over the fiber for DAQ to process.

• The current system design foresees sending all events to DAQ independent of any higher level trigger decisions for each Level-0 trigger. This design does not require much memory on THUB other than a small amount (arranged as a FIFO) to decouple the clock domain on the TCPU (SERDES) side from the clock domain of the SIU interface.

18

DAQ Data RatesOverall Specifications

Trigger Level-0 Input Rate 10,000 Hz Input Rate into Level-2 10,000 Hz Number of Fiber pairs to DAQ 4 Number of channels per MRPC 6 Number of MRPCs per tray 32 Number of Channels per tray 192 Number of trays 120 Total Number of MRPC channels 23,040 Number of Start Detector Channels 19 Number of Start Detectors 2

Tray Specifications

Data Volume per (black) Event per Tray in bits 12,448 (192*2*32 + 3*32 header + 2*32 geographical)

THUB Specifications

Number of TCPU-THUB Interfaces 31 L0-rate per tray @10kHz in bits/sec (“black event”) 124,480,000 (“Black”) Event Size per THUB (31 trays) in bits 385,888 L0-rate assuming < 20% occupancy (bits/sec) 771,780,000 DDL Fiber Bandwidth @ 66MHz 250MB/s = 2Gb/s DDL Fiber Bandwidth @ 40 Mhz (* 40/66) 150MB/s = 1.2Gb/s

19

Current System Status

20

TINO Status• Motivation: Replace Maxim Amplifier & Comparator of TAMP with

custom ASIC “NINO” incorporating both functions:– Lower Cost– Power (no negative supply & lower power, fewer power supplies)– Fully differential: better match to HPTDC– Pulse stretching: one TDC can measure both leading & trailing edge

• Decision to use TINO instead of TAMP was reached in Feb 2006 based on cosmic ray testing (see following slides) with both TAMP and TINO giving similar timing resolutions

• Automated TINO prototype production has been achieved, fabricated 35 circuit boards, 15 out of 15 boards successfully assembled

• Minor revisions made to design and layout for part availability and assembly reasons

• Mechanical compatibility with tray design verified• Will install & test 8 TINO on tray in Run 7

21

UT Cosmic Ray Test Setup

• Readout through TAMP/TINO, TDIG, and TCPU

• dT = t3 – (t1+t2+t4)/3• Same INL correction

for all HPTDCs

S1

S2

S3

Gasbox

MRPC 1

MRPC 2

MRPC 3

MRPC 4

3.00

2.00

22.00

8.50

22

UT Cosmic Ray Test Results

Time-over-Threshold (ToT)

Slewing Corrections

Final Timing Resolutions

average σ(delta t) ≈ 91ps

TA

MP

TIN

O

23

TINO Test Plan

• NINO chips were fully tested @ CERN after packaging• Confirm timing & crosstalk performance of TINO design

in TDIG RFI environment• Verify input to output integrity of each PCB assembly• Verify PCB assembly, current drain• Measure, with TDIG readout, for each channel:

– Threshold input voltage required to produce “0” discrimination threshold

– Overall gain, (input discrimination level @ max. threshold)– Output pulse width “stretch”

24

TINO Test Items Needed

• Power supply with accurate current metering• Pulser with

– MRPC input attenuator / shaping / connector jig– PC interface (for automated testing)

• Production TDIG• PC with readout / control software to implement

desired degree of test automation

25

TPMD (start side FEE)

• New replacement for “TPMT” to match start detector electronics to tray electronics

• Design considerations:– Map 4 channels max. per HPTDC to minimize

crosstalk effect

– Outputs provided for (“fast”) Z-vertex FEE (not part of the TOF project)

• Status:– Design and layout completed– Printed Circuit Boards and parts ordered

26

THUB Status• Needed to concentrate data before sending to DAQ in order to

reduce the number of DAQ-fiber interfaces• Also provides interface to Level-2• Prototype designed @ UT, first board delivered in April 2006 with 4

SERDES links assembled• TCPU-THUB interfaces implemented as SERDES on daughter

cards; distributes trigger, clock, and resets to TCPU, receives data from TCPU

• SERDES link works reliably up to ~25 feet @ 40MHz, > 50 feet @ 20 MHz. Need about 30 feet for start detector cable run

• A cable delay test performed with a TCPU clock driven by a SERDES derived clock and a Run 5 TDIG shows no difference in timing resolution. This indicates derived clock has small enough jitter to operate HPTDC reliably.

• Probably use 20MHz on SERDES link, multiply to 40MHz on TCPU. Will try to add buffer to see if cable length at 40MHz can be increased.

27

THUB Plans• Need another revision for final THUB

– Add Level-2 interface (SIU)– Correct minor issues with prototype– Possibly place SERDES on main board (no daughter cards)?

• Need about 3 weeks of circuit design work and about 4 weeks for layout of final revision

• Production and assembly takes about 4 weeks (only 4 boards are needed)

• Board locations on the STAR magnet have been documented within STAR

• Cable runs were discussed with R.Brown• Two prototype THUB (one for East and one for West

side) will be ready Nov 2006 for installation in Run 7.• Final revision of THUB is not needed until Run 8. Final

design review will depend on TCPU design, probably combined with TCPU Final Design Review

28

TDIG & TCPU Status

• New design for TDIG almost completed• Plan to build 4 to confirm functionality, then

order about 20 for Run 7 and bench testing• TCPU design will start as soon as TDIG design

is finished• Details presented in Lloyds talk• See also Geary Eppley’s talk regarding the Blue-

Sky contract for these boards

29

Production & Testing

30

Schedule

• TDIG Design close to being finished, prototype production to start soon.

• TINO design finished and first 15 boards produced and assembled. Final production can start as soon as compatibility with new TDIG has been bench tested

• TCPU R&D should be finished by Jan 07• TPMD design finished and production of 15 cards has

started. Delivery expected in 5 weeks.• THUB design should be ready for final design review in

Jan 07.• Plan for a system test with all electronics components• Final Design review shortly after first TCPU prototypes

have been bench tested (Jan 2007)

31

Production

• TDIG production rate: ~100 boards per month. Total production should start in Dec 06 and be finished in less than 18 months (~Apr 2008)

• TINO production will stay ahead of TDIG production at about the same rate

• TCPU production will be in parallel to TDIG production, but need fewer boards

• TPMD production can be finished with one order• THUB production can be finished in one order

32

Electronics Testing

• Bare board testing and simple stuffed board testing at Vendor

• Simple single board functionality testing

• Test board set of 8 TINO, 8 TDIG, and TCPU (all tray electronics) as a set on a real tray top

• Test installed electronics on a tray in a cosmic ray setup

33

Electronics Test Plans

Bare board test

Automated board stuffing

Board Level acceptance

test

Tray level board test (17 boards + cables)

Tray assembly

Tray test

Automated board test (“flying probe”)

Burn-in

Purchase bare boards and components

Vendor

Rice

UT

UT

Vendor

Vendor

Vendor