1

Electronics

• Status of the MEG Trigger system• Status and plans for DAQ• MSCB slow control system

• Status of the MEG Trigger system• Status and plans for DAQ• MSCB slow control system

2

The Trigger System of the MEG

Experiment

On behalf of

M. Grassi

D. NicolòF. Morsani S. Galeotti S. Giurgola

3

Expected Trigger RateAccidental background and

Rejection obtained by applying cuts on the following variables

• photon energy• photon direction• hit on the positron counter • time correlation• positron-photon direction match

4γ 102~ 97% MeV 45 fE

99%)5.3( 1.2 oo

1-s 20 4

ff

TRfRR

e1.0

4

10 18

sR

16105 sRe

100% 102 nsT5 ; 2 ff

The rate depends on R Re+ R2

ee

e e

4

The trigger implementation

Digital approach – Flash analog-to-digital converters (FADC)– Field programmable gate array (FPGA)

Final system Only 2 different board types Arranged in a tree structure on 3

layers Connected with fast LVDS buses Remote configuration/debugging

capability

Prototype boardCheck of: the FADC-FPGA compatibility chosen algorithms synchronous operation data transmission

5

The board type 0

PMT inputs

LVDS transm.

LVDS receiv.

FADC

FPGA

configuration EPROMS

Differential drivers

package error solved with a patch board

control signals.

6

Prototype system

configuration

Diff. driver Fadc Proc.

Algor.

LVDS Rx

LVDS Tx

Proc. Algor.

Circ.buff

Circ.buff

Circ.buff

Circ.buff

Diff. driver Fadc Proc.

Algor.

LVDS Rx

LVDS Tx

Proc. Algor.

Circ.buff

Circ.buff

Circ.buff

Circ.buff

16 PMT

16 PMT

input

input

output

output

LVDS in

final

Board 1

Board 0

Last BVR conclusionsThe prototype system

met all requirementsIt is available to trigger

the LP in future beam tests

7

Trigger system structure

LXe inner face(312 PMT)

. . . 20 boards

20 x 48

Type1Type1

Type116

4

2 boards

. . . 10 boards

10 x 48

Type1Type1

Type116

4

LXe lateral faces(208 PMT)

(120x2 PMT)(40x2 PMT)

1 board

. . . 12 or 6 boards

12 x 48

Type1Type1

Type116

4

Timing counters(160 PMT)

or(80 PMT)

2 or 1

boards

4 x 48

1 board

4 x 48

2 x 48

2 VME 6U

1 VME 9U

Located on the platform

Type2

Type2

Type2

Type2

Type2

Type2

8

Type 1

Diff. driver Fadc

LVDS TxOpt.

Proces. Algor.

Circ.buff

Circ.buff

Circ.buff

LVDS Tx

Proc. Algor.16

PMT

Type 2 LVDS TxOpt.

Proces. Algor.

Circ.buff

Circ.buff

Circ.buff

LVDS Tx

Proc. Algor.10

type1

LVDS Rx

LVDS Rx

LVDS Rx

9

Software items– New package ISE 6.2– Verilog/schematic implementation– Block transfer in A32D16 format (VME library to be modified)

Hardware items– JTAG programming/debugging through VME by modifying the

Type0– Analog receivers and with DACs for pedestal– FPGA selected: VirtexII PRO

• On Type 1 XC2VP20-5-FF1152 • On Type 2 XC2VP40-5-FF1152

– Other components are fixed: • FADC • LVDS Tx and Rx • Clock distributor• Analog input by 3M coaxial connectors• LVDS connection by 3M cables

– Ancillary logic components and scheme

10

FPGAVIRTEX II - PRO

• easily at 100MHz

• 60% of IO

• 40% of CLB

• 2 PowerPC (not used)

11

Analog receiver

AD5300AD8138

Differential driver

DAC pedestal control

12

Analog receiver

• 16 channels on a type1 board

• 1 unit wide

13

DC/DC converter

• 1.5 Volts

• 3.3 Volts

14

ANCILLARY: TREEANCILLARY: TREE

TRIGGERANCILLARY

#0

START

STOP

SYNC

CLK INT

ANCILLARY#1CLK EXT

ANCILLARY#8CLK EXT

…

START, STOP, SYNC, CLK

………………………………

(… 16)

VMESTOP

15

ANCILLARY: BLOCKSANCILLARY: BLOCKS

4 x SILICON DELAYS:START, STOP, SYNC, CLK

INPUTS & CLK GENCLK - START - STOP - SYNC

TTL2LVDS

4 x 8(16)-LVDS-FANOUT

VME INTERFACE

16

ANCILLARY: INPUTS & CLOCK ANCILLARY: INPUTS & CLOCK GENGEN

10MHz CLK GEN INT/EXT SELECT

INPUT CONNECTOR

LVDS-to-TTL

17

ANCILLARY: SILICON DELAYANCILLARY: SILICON DELAY

18

ANCILLARY: LVDS FANOUTANCILLARY: LVDS FANOUT

19

Full System

2002 2003 2004 2005

Test MilestoneAssemblyDesign Manufactoring

Prototype Board

Final Prototype

Trigger

Full System

Prototype Board

Final Prototype

part.inst.

full.inst.

20

summary

• Components selected • Algorithms implemented• PCB design ready to start

21

Status and plans for DAQStatus and plans for DAQ

22

Domino Chip Principles

DLL

Phase and Frequency Stabilization

Phase and Frequency Stabilization

ExternalCommonReference

Clock

Vspeed8

inpu

ts

shift register

Trigger Signal Sampling

Trigger Signal Sampling

Low-jitterclock

domino wave

FADC

MUX16-bitDAC

uCFreq.Cntr

23

Timing reference

signal

20 MHz clock

PMT hit

Domino stops aftertrigger latency

24

Recovery of Timing

Domino speed stability of 10-3 : 400ps uncertainty for full window25ps uncertainty for timing relative to edge

Domino speed stability of 10-3 : 400ps uncertainty for full window25ps uncertainty for timing relative to edge

50 ns

1) Trigger publishes phase of trigger signal f relative to clock in multiples of 10 ns

1) Trigger publishes phase of trigger signal f relative to clock in multiples of 10 ns

2) Each DAQ card determines and fits “Time-Zero-Edge” in clock signal and uses this as t=0

2) Each DAQ card determines and fits “Time-Zero-Edge” in clock signal and uses this as t=0

3) Measure pulse width of clock to derive domino speed

3) Measure pulse width of clock to derive domino speed

4) Timing of all PMT pulses is expressed relative to t=0 point

4) Timing of all PMT pulses is expressed relative to t=0 point

25

Current readout mode

• First implemented in DRS2• Sampled charge does not leave chip• Current readout less sensitive to cross-

talk etc.

• First implemented in DRS2• Sampled charge does not leave chip• Current readout less sensitive to cross-

talk etc.

write

read

C

. . .

RI

VoutVin

26

DRS2 Chip• DRS2 design

– Up to 4.5 GHz sampling speed– 8+2 channels, 1024 bins deep each– Readout speed up to 100 MHz (?)– Submitted to UMC in Nov. 18th, 58 chips received in Jan.

15th, packaging 3 weeks

• DRS2 chip arrived in Feb. 04– 50 packaged chips (400 channels)– 1.5 GHz – 4.5 GHz sampling speed– Current mode readout works– Jitter estimation: 40ps

• Plans– VME prototype board by Aug. ‘04

• DRS2 design– Up to 4.5 GHz sampling speed– 8+2 channels, 1024 bins deep each– Readout speed up to 100 MHz (?)– Submitted to UMC in Nov. 18th, 58 chips received in Jan.

15th, packaging 3 weeks

• DRS2 chip arrived in Feb. 04– 50 packaged chips (400 channels)– 1.5 GHz – 4.5 GHz sampling speed– Current mode readout works– Jitter estimation: 40ps

• Plans– VME prototype board by Aug. ‘04

27

DRS2 chipDRS2 chip

28

DRS2 testsDRS2 tests

29

Sampling Speed Measurement

• Obtained last week with USB-Mezzanine board

• Usable sampling range 0.6 GHz – 4 GHz

• Obtained last week with USB-Mezzanine board

• Usable sampling range 0.6 GHz – 4 GHz

30

Jitter estimation

• Oscilloscope triggered with Domino pulse

• Show 250 turns later• 11ns/250 = 44ps• Should be improved

with better board design

• Oscilloscope triggered with Domino pulse

• Show 250 turns later• 11ns/250 = 44ps• Should be improved

with better board design

31

Analog readout

• 4 pulses, 12ns wide, ~1ns rise time digitized at 2.5 GHz

• Readout at 40 MHz

• Reproduced rise time: 1.2ns

• Tests with FADC will follow

• 4 pulses, 12ns wide, ~1ns rise time digitized at 2.5 GHz

• Readout at 40 MHz

• Reproduced rise time: 1.2ns

• Tests with FADC will follow

32

VME boards with Mezzanine Cards

R. PaolettiINFN PisaMAGIC collaboration

PSIGVME board

33

PSI GVME Board

34

VME Transition Boards

• PMT/DC signals through front-panel connectors to CMC cards with DRS

• Low-jitter clock through front-panel ch. #17

• Trigger, reset, etc through transition board

• Read feedback to trigger thorough transition board

• PMT/DC signals through front-panel connectors to CMC cards with DRS

• Low-jitter clock through front-panel ch. #17

• Trigger, reset, etc through transition board

• Read feedback to trigger thorough transition board

PM

Ts

Clo

ckC

lock

35

DAQ System

PMTActiveSplitter

~11m~3m

area

monitor

~3mDRS

Board(32chn)+ CPU

DC Pre-Amp DRSBoard

(32chn)+ CPU

~7m

SIS3100

5 VME crates

800 + 160

1920

opticalfiber (~20m)

Trigger

GigabitEthernet

Front-End PCs

On-line farm

Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)

Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)Rack – PC (Linux)

storage

Fitted data:10 Hz waveform data -> 1.2 MB/sec90 Hz ADC / TDC data -> 0.9 MB/sec

Raw data:2880 channels100 Hz50% / 10% / 10% occupancy2kB / waveform -> 5 x 25 MB/sec.

3+1 VME crates

trigger

ready

backpressure

36

Rack LayoutRack Layout

SplitterTriggerDRSInterface

?

Clock distribution(front)

TriggerBoard readyFast clear

Event counter(transition cards)

DCCalo+TC

37

HVHV

Splitter

Trigger

DAQ Calo+TC

DAQ DC

38

Electronics in B-Field

4.2 6.8 9.1

6.7 18.5 83.8

9.0 33.5433.7 440.331.2 7.5

50.5

73.320.4 56.369.7 19.8

7.8 11.6 11.3

Fringing field measured at πE5 [Gauss]• B. Allongue (PH-ESS Group,

Cern)• Wiener PL500 Power Supply

works up to 300 Gauss (900 with water cooling)

• Fan works up to 80-100 Gauss

• Ordered “normal” crate for tests

• Need air ducts with external A/C if problems arise

• B. Allongue (PH-ESS Group, Cern)

• Wiener PL500 Power Supply works up to 300 Gauss (900 with water cooling)

• Fan works up to 80-100 Gauss

• Ordered “normal” crate for tests

• Need air ducts with external A/C if problems arise

39

Waveform analysis

• Zero suppression in FPGA• Single hit

– ADC/TDC derived in FPGA

• Multiple hit– Waveform compressed in

FPGA (2x12 bit -> 3 Byte)– Waveform fitted /

compressed in PC cluster

• Store ADC/TDC only for “calibration” events

• Store (lossless) compressed waveforms for MEG candidates

• Zero suppression in FPGA• Single hit

– ADC/TDC derived in FPGA

• Multiple hit– Waveform compressed in

FPGA (2x12 bit -> 3 Byte)– Waveform fitted /

compressed in PC cluster

• Store ADC/TDC only for “calibration” events

• Store (lossless) compressed waveforms for MEG candidates

OriginalWaveform

Difference Of Samples

Threshold in DOS

Region for pedestal

evaluationintegration area

ADC1/TDC1

ADC2/TDC2

T

40

Differential DRS channels

write

Vin

write

Vin +

Vin -

DifferentialDriver

cross-talk

signalscancel

Measured for 1ns rise-time:• 6% neighbor• 2% next nb

Measured for 1ns rise-time:• 6% neighbor• 2% next nb

41

DRS3

• DRS2 can probably initially be used for DAQ (have 400 channels, can produce 400 more)

• DRS2 limitations– Only two channels are fully differential (others show

larger crosstalk)– Some tests remain to be done …

• New DRS3 design– All channels differential– Additional shielding between channels (ground bond

wires)– Reduced readout time (5x) minimized dead time– Internal cascading allows for n x 1024 sampling bins

• DRS2 can probably initially be used for DAQ (have 400 channels, can produce 400 more)

• DRS2 limitations– Only two channels are fully differential (others show

larger crosstalk)– Some tests remain to be done …

• New DRS3 design– All channels differential– Additional shielding between channels (ground bond

wires)– Reduced readout time (5x) minimized dead time– Internal cascading allows for n x 1024 sampling bins

42

Plans

• DRS2 VME prototype board Aug. 04• Measure all parameters (cross-talk, resolution,

stability)• Produce VME boards and equip with DRS2 chip

(400 chn + 400 chn ?), install as much as possible in are in summer 2005

• Design DRS3 in parallel• Mass production of DRS3 in fall 2005• Replace installed DRS2 with DRS3

• DRS2 VME prototype board Aug. 04• Measure all parameters (cross-talk, resolution,

stability)• Produce VME boards and equip with DRS2 chip

(400 chn + 400 chn ?), install as much as possible in are in summer 2005

• Design DRS3 in parallel• Mass production of DRS3 in fall 2005• Replace installed DRS2 with DRS3

43

DRS (DAQ)

2002 2003 2004 2005

Test MilestoneAssemblyDesign Manufactoring

DRS2

DRS2 test board

DRS3

VME boards

400 chn 400 chn

Mass Production

3000 chn

DRS1

2nd Prototype

Tests

Boards & Chip Test

DRS1

DRS2

Full System

installation

OptionalDRS2 production400 chn

44

Slow Control System

• Unified control system for– Cryogenics (temperature, pressure,

valves, etc.)– Environment (temperatures, crates)– Drift chamber gas system (pressure,

mass flow, temperature)– High Voltage (Calo+TC+DC, ~1000

chn.)

• System must be fail-safe

• Unified control system for– Cryogenics (temperature, pressure,

valves, etc.)– Environment (temperatures, crates)– Drift chamber gas system (pressure,

mass flow, temperature)– High Voltage (Calo+TC+DC, ~1000

chn.)

• System must be fail-safe

45

Slow ControlHV

PC

RS

232

12345

Temperature, pressure, …

GP

IB

Valves

??? 15° C

heater

PLC

12:30 12.3

12:45 17.2

13:20 15.2

14:10 17.3

15:20 16.2

18:30 21.3

19:20 18.2

19:45 19.2

MIDASDAQ

Eth

ern

etTerminal Server

46

Slow Control BusHV

Temperature, pressure, … Valves

heater

MIDASDAQ

47

• CAN, Profibus, LON available• Node with ADC >100$• Interoperatibility not guaranteed• Protocol overhead• Local CPU? User programmable?• How to integrate in HV? (CAEN

use CAENET)

• CAN, Profibus, LON available• Node with ADC >100$• Interoperatibility not guaranteed• Protocol overhead• Local CPU? User programmable?• How to integrate in HV? (CAEN

use CAENET)

Field Bus Solutions

48

Hardware Overview

• 8051-compatible C with– ADC 12-bit– DAC 12-bit– Flash EEPROM– Timers– Watchdog– Temperature sensor– UARTs– Up to 100MHz clock

speed• External signal conditioning

if needed• Serial communication

• 8051-compatible C with– ADC 12-bit– DAC 12-bit– Flash EEPROM– Timers– Watchdog– Temperature sensor– UARTs– Up to 100MHz clock

speed• External signal conditioning

if needed• Serial communication

49

RS-485 bus

• Similar to RS-232 but– Up to 256 (1/8 load) units can

be connected to a single segment, use repeater for more

– Address space for 65536 nodes– single line, half duplex– differential twisted pair– Segment length up to km (20 m

tested)– MSCB system: 115kbit

– Single Master – Multiple Slaves (like USB)

– Power through bus (10-wire flat ribbon)

– PC USB interface

• Similar to RS-232 but– Up to 256 (1/8 load) units can

be connected to a single segment, use repeater for more

– Address space for 65536 nodes– single line, half duplex– differential twisted pair– Segment length up to km (20 m

tested)– MSCB system: 115kbit

– Single Master – Multiple Slaves (like USB)

– Power through bus (10-wire flat ribbon)

– PC USB interface

50

Generic node SCS-200• C8051Fxxx Micro

controllers with 8x12 bit ADC, 2x12 bit DAC, digital IO, 8051 C and 32kB Flash Memory

• RS-485 bus over flat ribbon cable

• Powered through bus• Costs ~CHF 50• Piggy back board for

signal conditioning cards• 32 kB for real time C

programs

• C8051Fxxx Micro controllers with 8x12 bit ADC, 2x12 bit DAC, digital IO, 8051 C and 32kB Flash Memory

• RS-485 bus over flat ribbon cable

• Powered through bus• Costs ~CHF 50• Piggy back board for

signal conditioning cards• 32 kB for real time C

programs

51

2 Versions

• Generic node with signal conditioning• RS232 node with protocol translator• PC connection to parallel port (USB

planned)• Integration on sensors, in crates

BUS OrientedBUS Oriented

Crate OrientedCrate Oriented

• 19” crate with custom backplane• Generic node as piggy-back• Cards for analog IO / digital IO / °C /

220V• Used in 3 experiments at PSI

Can be mixed

52

SCS Nodes•SCS 210: RS232 I/O•SCS 300: Centronics I/O (14 bit digital)•SCS 310: GPIB (IEEE-488) I/O•SCS 400: 8 chn. Thermocouple + 4 chn.

digital output (PWM), temp. regulation in SW

•SCS 500: 8 chn. differential analog input0…10V, 0…1V, 0…100mV, 0…

10mV-10…10V, -1…1V, -100mV…100mV0…100mA, 0…10mA, 0…1mA15V power, Lemo or screw terminal

•SCS 600: 8 chn. digital output 0…30V, 1ALEDs and buttons on front panel220V “power box”

•SCS 700: 8 chn. PT100/PT1000, 8 bit digital output

•SCS 800: 8 chn. capacitance meter•SCS 900: 8 chn. 24-bit ADC -10V…10V

8 chn. 16-bit DAC -10V…10V

•SCS 210: RS232 I/O•SCS 300: Centronics I/O (14 bit digital)•SCS 310: GPIB (IEEE-488) I/O•SCS 400: 8 chn. Thermocouple + 4 chn.

digital output (PWM), temp. regulation in SW

•SCS 500: 8 chn. differential analog input0…10V, 0…1V, 0…100mV, 0…

10mV-10…10V, -1…1V, -100mV…100mV0…100mA, 0…10mA, 0…1mA15V power, Lemo or screw terminal

•SCS 600: 8 chn. digital output 0…30V, 1ALEDs and buttons on front panel220V “power box”

•SCS 700: 8 chn. PT100/PT1000, 8 bit digital output

•SCS 800: 8 chn. capacitance meter•SCS 900: 8 chn. 24-bit ADC -10V…10V

8 chn. 16-bit DAC -10V…10V

53

Software overview

mscb.dll

msc.exeCommand line

interface

VI

LabViewApplication

VI VI VI

LPT USB

PC

SCS 250

SCS xxxFramework

User code

RS485

SCS 300

54

Remote access

mscb.dll

msc.exeCommand line

interface

LPT USB

PC

RS485

mscb.dll

msc.exeCommand line

interface

VI

LabViewApplication

VI VI VI

PCTCP/IP

Speed (commands/sec, 115kBaud):

~1500 local~900 remote

Speed (commands/sec, 115kBaud):

~1500 local~900 remote

SCS 250

55

Protocol

• Asynchronous 345 kBaud / 115kBaud• 16-bit addressing (65536 nodes)• CRC-code for error detection• Optional acknowledge• Concept of channels and configuration

parameters (256 each per node)• Optimized protocol: 1500 cmd/sec @ 345kB

• Asynchronous 345 kBaud / 115kBaud• 16-bit addressing (65536 nodes)• CRC-code for error detection• Optional acknowledge• Concept of channels and configuration

parameters (256 each per node)• Optimized protocol: 1500 cmd/sec @ 345kB

command channel value CRC

write data

node

param1

param2

param3

channel1

channel2

channel3

ADC

ADC

port

command LSB MSB CRC

address command

1 Byte

command CRC acknowledge

56

Node communication

msc.exe

LabView

57

Labview control of Large Prototype

Labview control of Large Prototype

58

How to make it fail-safe?

• Robust protocol– CRC code– Automatic reconnect

• UPS– SC crates on battery (~30 min.)– Use Laptop for control PC

• Redundancy– Operate two completely independent

nets– Switch between nets on failure

• Robust protocol– CRC code– Automatic reconnect

• UPS– SC crates on battery (~30 min.)– Use Laptop for control PC

• Redundancy– Operate two completely independent

nets– Switch between nets on failure

59

Redundancy

Temperature, pressure, … Valves

uC uC uC

uC uC uC

Switch box

ControlPC1

ControlPC2

• If uC fails, use other system• If PC fails, use other system (PC-PC watchdog)• For critical valves, use two (parallel or serial)• Avoid single point of failure !• Test failures

• If uC fails, use other system• If PC fails, use other system (PC-PC watchdog)• For critical valves, use two (parallel or serial)• Avoid single point of failure !• Test failures

System 1

System 2

Ethernet

60

New HV DesignVin

Vout

• • • 12 times • • •

3000V

0-3000V

ADC24 - bit Micro

Controller

DAC16 - bit

• Microcontroller optically decoupled from HV side

• Higher ADC+DAC resolutions

10 mV accuracy• Stable operation in lab

(weeks)• Newer test results will be

presented in review

• Microcontroller optically decoupled from HV side

• Higher ADC+DAC resolutions

10 mV accuracy• Stable operation in lab

(weeks)• Newer test results will be

presented in review

61

Integration of Slow Control into DAQIntegration of Slow Control into DAQ

• Run parameters will be written to MySQL database• Calibration parameters recalculated after every run and stored in DB• More frequent database update possible• Use of MIDAS history system (widely established since many years)

• Display through Web interface

• Gif images generated dynamically in memory

• Command line query with mhist

• Write speed: 2000 events/s

• Query over one month in ~10sec

• Display through Web interface

• Gif images generated dynamically in memory

• Command line query with mhist

• Write speed: 2000 events/s

• Query over one month in ~10secCombines Slow Control,

monitoring and calibration data

Combines Slow Control, monitoring and calibration data

62

Summary Midas Slow Control Bus

• 256 nodes, 65536 nodes with one level of repeaters• Bus length ~500m opto-isolated• Boards for voltage, current, temperature, Digital IO, 220V• Readout speed: 700 channels / sec. @ 115kBaud• C library, command-line utility, Midas driver, LabView

driver• Nodes are “self-documenting”• Configuration parameters in EEPROM on node• Node CPU can operate autonomously for interlock and

regulation (PID) tasks (C programmable, floating point library)

• Nodes can be reprogrammed over network

http://midas.psi.ch/mscb

• 256 nodes, 65536 nodes with one level of repeaters• Bus length ~500m opto-isolated• Boards for voltage, current, temperature, Digital IO, 220V• Readout speed: 700 channels / sec. @ 115kBaud• C library, command-line utility, Midas driver, LabView

driver• Nodes are “self-documenting”• Configuration parameters in EEPROM on node• Node CPU can operate autonomously for interlock and

regulation (PID) tasks (C programmable, floating point library)

• Nodes can be reprogrammed over network

http://midas.psi.ch/mscb

63

Conclusions

• Trigger system is on schedule• DRS2 chip works• Partial electronics installation until

mid-2005 planned• Slow control system works (as

tested in ~10 installations at PSI and TRIUMF)

• Trigger system is on schedule• DRS2 chip works• Partial electronics installation until

mid-2005 planned• Slow control system works (as

tested in ~10 installations at PSI and TRIUMF)