debugging of m2-m5 g. passaleva. 11/02/2009 muon system mini review g. passaleva 2 outline short...
TRANSCRIPT
Debugging of M2-M5
G. Passaleva
11/02/2009 Muon System mini review G. Passaleva 2
outline
Short summary of Muon readout channel organization
Connectivity tests, timing tests Status of debugging since Oct. 2008 Plans for the coming weeks Summary
11/02/2009 Muon System mini review G. Passaleva 3
Muon channel organization Physical channels (PC) are grouped in logical OR
to form Logical Channels (LC). Time of LC is measured by the SYNC chip inside the ODE
Logical channels are those stored on disk and sent to L0
They are crossed at software (L0) level to get the Logical Pads (LP) i.e. the muon hits
In some regions LCs are directly grouped into LPs
Logical channel
Logical channel
Logical pad Physical channels
11/02/2009 Muon System mini review G. Passaleva 4
Muon channel organization
Sync channel
Sync channel
Delays can be adjusted:1. at physical channel level inside the DIALOG
chips
2. at sync channel level
3. at ODE level (sync to sync synchronization)
1)
2)
2)
3)
11/02/2009 Muon System mini review G. Passaleva 5
The Muon Detector readout
Data chain& Tell1 boards
C. D
epla
no
11/02/2009 Muon System mini review G. Passaleva 6
Principle of the testsAll the physical channels are masked and then unmasked and pulsed one after the other. The corresponding logical channels are read out
Sync channel
Sync channel
11/02/2009 Muon System mini review G. Passaleva 7
Connectivity: FE to ODE Connection Complexity
12 different type of electronic chain connections (up to ~1k Front-End CH, ~200 Read-Out CH and 48 muon chambers)
Test Connectivity using noise: On Front End
set low thresholds: noisy signals On Read Out
Look if the noise histogram is filled in the right read-out channel, and only there!
About 99% of the channel are well connected and correctly reachable via PVSS control software
C. Deplano 23 Sept 2008 8
C. D
epla
no
11/02/2009 Muon System mini review G. Passaleva 8
Time Alignment using the FE pulse system (FE to ODE)
DIALOG
ASD
MWPC
FEE
ASD
to ODEfrom PDM
PDM: pulses to FEE synchronous to a given BX number • Use LHC Clock• Pulse reproduced by
DIALOG and sent to CARIOCA
• Chambers are not involved
• Pulse distribution cables length compensated
C. Deplano 23 Sept 2008
Time Measurements done on the ODE board (SYNCs).The delay parameters are calculated and
saved intoPVSS recipes
C. D
epla
no
11/02/2009 Muon System mini review G. Passaleva 9
Status of debugging since Oct. 2008 After the cancelation of LHC run we starteda systematic campaign of
debugging to fix the problems left open
October/November C. Deplano + GP; December F. Dettori + A. Sciubba and M. Palutan; January: GP
30/10/2008 20/12/2008 Today
Tot log. channels
Bad log. channels
fraction Bad log. channels
fraction Bad log. channels
fraction
Side A 8352 390 4.7% 322 3.9% 277 3.3%
Side C 8352 257 3.1% 208 2.5% 128 1.5%
Total 16704 647 3.9% 530 3.2% 405 2.4%
Fixed 117 18% 242 37%
11/02/2009 Muon System mini review G. Passaleva 10
The actual status The remaining problems need careful interpretation of
the tests and intervention on the hardware
Typically the problems are in the chain dialog (IB) TB
ODE
Debugging time is hard to predict… but is never short !
Access and modify hardware
Test the single part
If successful re-test the whole hardware unit
…hope that no new problem show up…
11/02/2009 Muon System mini review G. Passaleva 11
connectivity timing bad Sync ChannelsQ1 M2 R1 HU1 noisy channel 1Q1 M2 R2 HU1 Chamber Chamber 48Q1 M2 R2 HU2 warn Verificare L0 2Q1 M2 R3 HU2 noise noisy channel 3Q1 M2 R3 HU1 chaos ! to be verified 7Q1 M2 R4 HU2 Chamber Chamber 1Q1 M3 R2 HU2 conn missing channel! Expected in timing 1Q1 M3 R3 HU2 timing change ODE ? 56Q1 M4 R3 HU1 connectivity (expected) conn 3Q1 M4 R4 HU1 connectivity (expected) conn 8Q1 M5 R1 HU1 noisy channel noise connectivity (expected+unexpected) 11Q1 M5 R4 HU1 timing change sync 8Q2 M2 R2 HU2 noise VERIFY NOISE 11Q2 M2 R3 HU2 Chamber Chamber 35Q2 M2 R3 HU1 conn expected in timing 2Q2 M2 R4 HU2 CONNECTIVITY conn CONNECTIVITY 1Q2 M3 R2 HU2 conn CONNECTIVITY 48Q2 M3 R2 HU1 noise confirmed noisy FE channels in 16A3 3Q2 M4 R2 HU1 timing problems on CMB 13A1 16Q2 M4 R4 HU1 timing was OK before PATCH ! 2Q2 M5 R3 HU1 timing problems were different before PATCH 10Q3 M3 R3 HU2 noisy channel noise 1Q3 M3 R4 HU2 noisy channel noise CANALE RUMOROSO ACCERTATO 3Q3 M4 R2 HU1 conn CONNECTIVITY 1Q3 M4 R3 HU1 conn expected;additional timing problem after PATCH 3Q3 M4 R4 HU1 conn CONNECTIVITY 7Q3 M5 R1 HU1 Chamber Chamber 96Q3 M5 R3 HU1 conn was OK before PATCH ! 5Q4 M4 R3 HU1 connectivity (unexpected) noise VERIFY NOISE 7Q4 M5 R1 HU1 conn expected in CT 2Q4 M5 R3 HU1 expected in timing timing problems were different before PATCH 3
405
11/02/2009 Muon System mini review G. Passaleva 12
M2
Fixed
problem
chamber
Channel m
aps Sep. 2008
11/02/2009 Muon System mini review G. Passaleva 13
M3
Fixed
problem
chamber
Channel m
aps Sep. 2008
11/02/2009 Muon System mini review G. Passaleva 14
M4
Fixed
problem
chamber
Channel m
aps Sep. 2008
11/02/2009 Muon System mini review G. Passaleva 15
M5
Fixed
problem
chamber
Channel m
aps Sep. 2008
11/02/2009 Muon System mini review G. Passaleva 16
Debugging databaseThanks to G. Graziani we have set up a database to save the history of all the tests we are doing, including diagnostic reports, timing tables and TDC histograms produced by the ECS tests. This helps our experts to spot the problems and get indications on where to intervene on the hardware
11/02/2009 Muon System mini review G. Passaleva 17
Plans for the debugging All the problems have been categorized
Will attack first the cases were a big number of channels are
lost.
Tests are very time consuming: connectivity tests require
several hours per ODE; timing tests about 1 hour
Very complex system: to spot a problem we have to
investigate many boards, cables etc.
Four problematic chambers. Need to access them. However:
problematic chambers in M2 front(2) can be accessed without
opening the walls. The other two need opening
11/02/2009 Muon System mini review G. Passaleva 18
Plans for the debuggingIn the next months a task force will be dedicated full time to this task:
Francesco Dettori (Cagliari) full time
GP > 50% (10 weeks from now to june); Matteo Palutan (from March, t.b.q)
Remote assistance by the experts (Sandro, Adalberto)
Actively looking for another full time person
Contribution from other people coming for shorter time
Next week Adalberto will be at CERN for a full week of hardware debugging.
Support from a STRONG group of people very expert on the hardware and on the ECS system:V. Bocci, S. Cadeddu, M. Carletti, C. Deplano, M. Lenzi, R. Nobrega, D. Pinci, A. Sciubba
In particular the ECS group is actively working to make our software more robust, faster and well organized
11/02/2009 Muon System mini review G. Passaleva 19
Plans for the debugging
In principle, (some) residual dead regions can be “faked” at L0 level. Since the spatial correlation is small, ghosts should not be a big problem
HLT: with some care (performance, ghosts, efficiency !) dead channels can be faked also there
Software studies in progress. Muon ID: a lot of work already done to study a looser
muon selection How to deal with dead regions (Condition DB etc) Online monitoring of efficiency
11/02/2009 Muon System mini review G. Passaleva 20
Summary
Debugging started Nov 2008: 37% bad channels recovered
Still missing 2.4% of the channels mainly concentrated in a few large groups
Four chambers with problems: 2 of them can be accessed without opening the walls. The other 2 need opening. To be decided