1

The Muon ShifterThe Muon Shifter

October 2009

2

ForewordsForewordsThis pages are meant to initiate people who are not familiar with ECS (Experiment Control System), and Muon ECS in particular, to perform shifts in global data taking run. In this sense, information given here are very basic and in general refer to a non-expert use of the system: setting-up of the system for DAQ runs, first check of proper running conditions, first simple interventions in case of simple frequently encountered problems.This basic manual can also be a starting point for a richer documentation while the operator skill increases and the control system becomes more stable and complete. Please note also that the ECS system is still under development. Changes on the panels and even on the structure is still possible. This pages should not be meant as a user manual of a delivered project/system, but just to gain a first idea on system operation.

Muon ECS (basics of the) Muon ECS (basics of the) basicsbasics

The LHCb Experiment Control System (ECS) is based on the PVSS framework. You can find some documentation on PVSS and LHCb ECS at page http://lhcb-online.web.cern.ch/lhcb-online/ecs/default.htm and in particular the tutorial http://lhcb-online.web.cern.ch/lhcb-online/ecs/powerpoint/LHCb_ECS_Tutorial.ppt which we warmly suggest you to read.The ECS is based on so-called PVSS projects running on dedicated online PCs. Each project runs on independent partitions. The MUON ECS has several project types, controlling:

• FEB (Front End Boards, i. e. CARDIAC via the Service Boards)• ODE (Off Detector Electronics Boards)• LV and HV• Gas, Temperature, RAC power• Tell1• FSM (Finite State Machine & Hierarchy) projects

The last projects do not directly control any hardware but the hierarchical structure of the ECS itself. The FSM allows the ECS commands to circulate from top to bottom of the structure, thus making possible the communications among the complete system.

IntroductiIntroductionon

3

The Muon Detector DAQ and Control electronics chain

DIALOG is configured and read back by SB (Service Board). Service Boards are placed in an off detector rack. The same rack houses thePDM board (Pulse Distribution Module): it generates pulses in a chosen phase with the LHC machine clock for the time alignment of the muon system.

Front-end Boards (CARDIAC):2 ASIC-CARIOCA (ASD chip)-DIALOG (logic, timing and monitoring)

The IB (Intermediate Board) make logical combinations of the front-end channels (logical channels)(logical channels) where the combination is not possible at DIALOG level

The ODE (Off Detector Electronic Board)

receive signals from the FEBs (or IBs),

synchronize and displace them to the L0

trigger. Upon reception of a trigger signal data

are sent to the TELL1 board and from here to

the DAQ (next slide for picture)

4

OD

E

L0 Muon Trigger processors

Muon TELL1 DAQ

ODIN

The ODIN board distributes all orbit signals (clock, trigger, L0 reset,…) to the ODEs, Muon TELL1s and Trigger processors (to all boards that are synchronuos and have the TTCrx chip)

The Muon Detector DAQ and Control electronics chain (II)

5

Muon detector back side view: each station M1-M5 is divided in four quadrant Q1-Q4

z

Interaction point

C side A side

Q4

Q3

Q1

Q2

M2

M3

M4

M5

M1

6

The structure of ECS consists of 3 types of objects: Device Unit (DU), Control Unit (CU) and Logical Unit (LU).

DU is an object controlling a hardware component (e.g. a CARDIAC board) CU is just a hierarchical object, passing commands and states on a hierarchical node LU is useful to group different DU together and is rarely used in our case

The hierarchical structure allows also including and excluding portions of the system.

Following the general LHCb structure, also the Muon hierarchy is organized in two independent and totally symmetrical parts, MUONA and MUONC, controlled by a higher node, MUON. The deeper the hierarchical level, the wider the understanding needed of system and hardware details. However, in “normal” conditions it should not be necessary to go further than the MUONA/C level.

EACH PVSS PROJECT RUNS ON AN ONLINE PC. On Control Room PCs you will find shortcuts to the online PCs where really the projects run

The Muon ECS

7

PVSS Tree for Muon Detector

MUON

MUONA MUONC

MUONC_DAQ MUONC_DCS MUONC_HV

MUON_DCS_LV

The Green bubbles are Control Unit (CU) of the: •PVSS project MUSIDES •PC: mudaqa01w

Inside this CU there is the hierarchy of the Muon detector electronics: Chambers/ODE/Tell1 •PVSS project MUDAQC •PC: muecsc01w

Inside this CU there is the hierarchy of the Muon detector Low Voltage •PVSS project MUDCSLVC •PC: mulvc01w

Inside this CU there is the hierarchy of the Muon detector High Voltage •PVSS project MUHVC01 (PNPI - R3 R4)•PC: muhvc01•PVSS project MUHVC02 (CAEN – R1 R2)•PC: muhvc02

An example for the C side

8

PVSS Tree of Muon DetectorMUONC_DAQ

MUONC_DAQ_Q3 MUONC_DAQ_Q4 MUONC_DAQL1

MUONC_DAQ_Q3_M1C MUONC_DAQ_Q3M45CMUONC_DAQ_Q3_M23C

MUONC_DAQ_Q3_M23C_M2

MUONC_DAQ_Q3_M23C_M3

MUONC_DAQ_Q3_M23C_CRC

Inside this bubble there is the hierarchy of the DAQ tell1 boards witch take the data coming from the ODEs:•PVSS project MUDAQL1 •PC: mudaq01

Inside this bubble there is the SB/PDM crate for station M2&M3 Q3&Q4•PVSS project MUDAQC23 •PC: mudaqc03w

An example for the C side

9

MUONC_DAQ_Q3_M23C

MUONC_DAQ_Q3_M23C_M2 MUONC_DAQ_Q3_M23C_CRCMUONC_DAQ_Q3_M23C_M3

Q3_M3_R1 Q3_M3_R2 Q3_M3_R3 Q3_M3_R4

Q3_M3_R1_HU1

Q3_M3_R1_HU2ChambersChambers

An example for the C side

PVSS Tree for Muon Detector

HU: Hardware Unit is made by One ODE and all the Chambers connected to itChambers ----------------------•PVSS project MUDAQC23 •PC: mudaqc03w•ODE ---------------------------•PVSS project MUODEC01 •PC: muodec01w

Hardware Unit

ODE

Chambers

10

Use Comp name PVSS IdProject Name Feb 2&3/C MUDAQC03w 260 MUDAQC23 Feb 4&5/C MUDAQC04w 261 MUDAQC45 Feb 1/C MUDAQC01w 262 MUDAQC1 Feb 1/A MUDAQA01w 263 MUDAQA1 Feb 2&3/A MUDAQA03w 264 MUDAQA23 Feb 4&5/A MUDAQA04w 265 MUDAQA45

FSM/C MUECSC01W 279 MUDAQC FSM/A MUECSA01W 280 MUDAQA

LV/C MULVC01w 210 MUDCSLVC LV/A MULVA01w 211 MUDCSLVA

HV/C MUHVC01 212 MUHVC01 HV/C MUHVC02w 213 MUHVC02 HV/A MUHVA01 214 MUHVA01 HV/A MUHVA02w 215 MUHVA02

ODEQ3/C MUODEC01w 266 MUODEC01 ODEQ4/C MUODEC02w 267 MUODEC02 ODEQ1/A MUODEA01w 268 MUODEA01 ODEQ2/A MUODEA02w 269 MUODEA02

T&P MUDCS01w 270 MUDCSSEN

TELL1 mudaq01 216 MUDAQL1M5 : mutella01 (Q1/Q2) mutellc01 (Q4/Q3)M4 : mutella02 (Q1/Q2) mutellc02 (Q4/Q3)M3 : mutella03 (Q1/Q2) mutellc03 (Q4/Q3)M2 : mutella04 (Q1) mutellc04 (Q4) M2: mutella05 (Q2) mutellc05 (Q3)M1: mutella06 (Q1) mutellc06 (Q4)M1: mutella07 (Q2) mutellc07 (Q4)MUON TOP mudaqa01w MUSIDES

MUON on-line MUON on-line PCs and their PCs and their

useuse

Comments:

1. Access to this machines to start/stop/operate the PVSS projects should normally not be needed but for experts.

2. As a general rule, PC names ending with W are Windows-based PCS, Linux otherwise.

3. Some of the reference experts are:

D. Pinci for FEBs, S. Cadeddu for FSM, M. Lenzi for LV and HV, M. Carletti for ODE, G. Graziani for Tell1 and Muon Monitor,C. Deplano.

Here a complete list of the online PCs and the relative projects running on them is given

11

ECS - PVSS world19 PCs20 PVSS projects

MUON

A SIDE C SIDE

ODE

ODE

ODE

CMB

CMB

CMB

CMB

TELL1

DCS

LV

LV

HV

HV

HV

HV

ODE

5

12

Panels and Panels and StatesStatesECS operation is based on control

panels, giving information on system status and allowing intervention of the operator to trigger a status change. Although intuitive, it is important to give some explanations on the use of the panel buttons.

{Sub-System buttons: clicking with the left mouse button opens the corresponding CU

Sub-System state, visualizing the current state. Clicking with the left mouse button gives the list of possible actions which can be done from it.

MUONA CU PanelMUONA CU Panel

Lock button: the system can be accessed/controlled by different parts of ECS. This button says who is presently TAKING the system and its inclusion/exclusion in the hierarchy. This info can be gained by placing the mouse tip on the lock. Before operating on a given system/subsystem you need to TAKE it (if possible). During Global data taking “your” system must be controlled by the global manager and the muon operator must release it properly (see next pages).Using the left mouse button it is possible to change the lock status.

Presently loaded recipe. A recipe is the collection of all the configuration parameters/variables needed for a specific data-taking run. Usually the shifter should not choose one recipe rather than another (it is already loaded). If further information is needed on recipe handling, for the moment you have to refer to an expert.

NEVER use this button to dismiss the panelALWAYS USE the Close button

13

1. How to ...1. How to ...Prepare the MUON system for global DAQ run Prepare the MUON system for global DAQ run when when

everything is okeverything is ok..

In normal conditions, the PVSS projects (FEBs, ODE, FSM) are up and running. This condition should be reached automatically at each power-on of the machines where the projects are installed (see list on next page). At the moment (June 08), this is implemented only for the LV control projects, but will be done soon also for the FEBs and the ODEs.

Being in this condition, the steps to set-up the system for DAQ are the following:

i. On any computer of the on-line cluster*, go to directory G:\online\ecs\Shortcuts38\MUON

ii. Launch (double click) the two shortcuts MUSIDES_UI_DEN and MUSIDES_UI_FSM. This visualizes the two control windows for DEN (Device Editor Navigator) and FSM (Finite State Machine) Hierarchy (see next page). Launching the shortcuts does not start the DEN and FSM processes (which are already running within the PVSS project), but just allows using the corresponding control windows from your screen.

(*you need an account within the on-line cluster: ask Niko Neufeld if you haven’t any.)

14FSM panel

The FSM panel you should have opened allows visualizing the status of the full MUON hierarchy and opening panels related to any hierarchical level.

Concerning the DU/CU status, the color code is as follows:

grey for grey for DEAD/NOT INCLUDEDDEAD/NOT INCLUDEDyellow for yellow for NOT READYNOT READYorange for orange for UNKNOWN/NOT ALLOCATEDUNKNOWN/NOT ALLOCATEDred for red for ERRORERRORblue forblue forREADY or ACTIVE (TFC case)READY or ACTIVE (TFC case)green forgreen for RUNNINGRUNNING

Now, placing the cursor tip on the MUON box and clicking the RIGHT mouse button, opens the MUON CU, which represents the top level of our hierarchy, see next page(s).

This box accesses the TELL1 CU (refer to page 15)

15

MUON CU panelMUON CU panelIn order to allow the global manager driving the MUON System, the MUON must be NOT_READY and its subsystems NOT_ALLOCATED. All the locks should be grey and open, without any slash on them (which means Not Included).

The first operation is TAKING the MUON System under your control. Click (LEFT mouse button) on the lock and select TAKE. The lock goes green and closed.

A panel automatically pops up, giving the list of LockedOut nodes and of CU which were not possible to take (more details on page 12).

Now OPEN (double click) the two sides (A & C)

Although this condition appears to be verified in the shown panel, it is equally necessary to check the MUONA and MUONC CU. Checking deeper levels is not required to a shifter.

16

MUONA CU panelMUONA CU panel

If everything is correct, the panel appears in the configuration shown.

In this conditions, you can release the MUON system in order to make it possible the control by the global console (go to next page)

(The MUONC panel has similar face)

17

MUON CU panelMUON CU panel

In order to release the MUON system, click on the green lock and choose ReleaseAll in the pop-up menu (the Release button releases this level of CU but not those below it).

The MUON status goes to NOT_READY and the system is free to be used by the global console.

Note:Case of a subsystem taken by someone else. Placing the mouse on, you obtain info on owner and status. In Global DAQ, the global manager decides what is to be included/excluded. You have to take care only of MUON, putting it in regular shape.

18

2. How to ...2. How to ... Prepare the MUON system for global DAQ run Prepare the MUON system for global DAQ run when when something is not the right waysomething is not the right way..

Suppose that opening the MUONA panel, a different status is shown (see figure):

Some subsystems appear to be locked out and the TFC is ACTIVE, while the right status is NOT_ALLOCATED for the TFC and NOT_READY for the others

i. Release MUON (ReleaseAll)

ii. TAKE MUONA (where the problem was found)

iii. ACTIVE Stop run READY

iv. READY RESETv. NOT-READY

DEALLOCATEvi. ReleaseAll (in the

lock)This should take again the MUONA in the right shape

In such a case, these are the steps to follow:

{

19

DEN panel

The DEN panel (you got from page 7) is to be used in case of DEAD states: click on the Start/Restart All button for the relevant stage of the hierarchy (use the FSM tab). This activate the FSM only on the relevant stage and should cure the DEAD state.

DEAD statesDEAD states

When you TAKE a System, an Action Report appears.

In some cases a subsystem cannot be taken because its state is DEAD (i.e. The FSM cannot communicate with it, or the FSM is blocked there)

20

TELL1 CCPC restart (1)TELL1 CCPC restart (1)

When the the system is being reset or configured, it may happen that the system enter an ERROR state.

This state propagates from the DU/CU which originates it, up to the top level of the hierarchy (MUON in our case).

Frequently this is caused by the Tell1 CCard, needing to be restarted (see next page).

(see next slide also to know how to get to the MUONA(C)_DAQL1 panel).

Here “ERROR” appears

21

1. Descend the hierachy starting from the MUON CU panel: MUON MUONA (e.g.) MUONA_DAQ MUONA_DAQL1(see previous slide). Alternatively, you can access the Tell1 panel directly from the FSM (see page 8).

2. Spot the Tell1 which is in error and identify the corresponding CCPC (see subsystems in the panel and the given correspondance table).

3. Log in on the CCPC using your user/pwd and start the ccserver (using putty, for example).

4. Type the command “sudo service ccserv restart”. You should obtain a double ok. If not, try to repeat the command.

This restarts the ccserver and should solve the ERROR

Side A PC TELL1 mutella01 MUTELLA_M5Q12 mutella02 MUTELLA_M4Q12 mutella03 MUTELLA_M3Q12 mutella04 MUTELLA_M2Q2 mutella05 MUTELLA_M2Q1 mutella06 MUTELLA_M1Q1Mutella07 MUTELLA_M1Q2

Side C PC TELL1 mutellc01 MUTELLC_M5Q34 mutellc02 MUTELLC_M4Q34 mutellc03 MUTELLC_M3Q34 mutellc04 MUTELLC_M2Q4 mutellc05 MUTELLC_M2Q3mutellc06 MUTELLC_M1Q4mutellc07 MUTELLC_M1Q3

TELL1 CCPC restart (2)TELL1 CCPC restart (2)

Here is how to (possibly) cure a TELL1 ERROR state

22

Low Voltage panels (2)Low Voltage panels (2) Q1 panelQ1 panel

Take beforeAnd release all when finished

23

Low Voltage panelsLow Voltage panels The LV domain is reachable from the MUON panel, from the MUONA_DCS and MUONC_DCS hierarchy.

Click directly on the relevant CU (RIGHT mouse button), or start from the top level clicking on MUONA_DCS on the FSM panel. The corresponding panel appears.

From this panel you can descend the hierarchy across quadrants and stations reaching the LV controls (see next slides)

24

Q1M23A panel: DAQ tab Q1M23A panel: DAQ tab At this level, the different sub-systems of LV are visible and accessible

Act only on the FSM buttons to switch on/off the sub-systems.

This part of the panel should be used only to check the effects of the action, not to cause them

The DAQ tab refers to ODE, SB & PDM, IB crates

25

The FEE tab refers to the CARDIAC boards

Q1M23A panel: FEE tab Q1M23A panel: FEE tab

26

Starting the MuonMoni PresenterStarting the MuonMoni PresenterThe better way to check the global status of the detector is looking at the data quality by means of the online Presenter. To start the Presenter:

1. Launch Exceed on your PC. There is a small problem with the new Exceed: the default configuration settings are re-set by the installer. Host access is now per default *DISABLED*. To change this open Exceed -> Tools -> Configuration -> Security  and remove the access control.

2. Log on mudaq01 (*)3. type: . /group/muon/scripts/lhcbsetup.sh4. set local display: export

DISPLAY=pcname:0.0 (**)5. run presenter:

/group/online/presenter/run_presenter.sh(*) Log-in using putty:

- session: mudaq01- connection

SSHX11 (enable x11 forwarding)open and login

(**) PC name : see label glued on the monitor

See the twiki page https://twiki.cern.ch/twiki/bin/view/LHCb/MuonDaqMonitoring about the use of the Presenter

27

How open the PVSS panels for a linux PC How open the PVSS panels for a linux PC For example: tell1 PC

The PC is: mudaq01 (login with Putty)Command line to open the “gedi” panel: PVSS00ui –proj MUDAQL1 –m gedi

On “gedi” go:- Module- Vision- localdisk/PVSS/MUDAQL1/panel- /grup/muon/fwComponent_MUDAQL1/panelTo open the “DEN” panel- select the folder fwDeviceEditorNavigator- open fwDeviceEditorNavigator.pnl

28

How create or upload a recipe for CMBHow create or upload a recipe for CMB1. Go into MUON_DAQ panel

2. Create a new recipe• Select the parameters

needed(Thresholds, Masks, …)

• Select the detector regions/side

• Push “set the configuration to region …”

• Check “re-load timing result” if the timing parameter have been changed

• write the recipe name• Push “Create Recipe”• Wait until the process-bar

arrive at 100%

3. Upload an existing recipe• Select the detector

region/side• Push “set the configuration

to region …”• Select a recipe from the

“Recipe List”• Push “upload Recipe”• Wait until the process-bar

arrive at 100%

29

How create recipe for ODE starting from timing test fileHow create recipe for ODE starting from timing test file1. Configure the ODE starting from the timing file

1. Go into the HU you need from the Musides shortcut and reset the ODE board2. Push “go to expert” and “continue”3. Into the tab “commands” push “Load HW Conf File”

1. Select the file you want write into the hardware 2. Save a PULSING recipe

1. Go into the corresponding ODE PC2. Open the ODE panel you want to change from the DEN3. Push “recipes…”4. Select the recipe you want create or overwrite (for example PULSING )5. Create the recipe “Create from hardware”6. Select “PULSING” recipe and push “edit”

1. Check that all “GOLCTRL” value are 0x001. If not: push “edit” into “table Mode”2. Write 0x00 for all the column “GOLCTRL” 3. Push “enter” into the keyboard: the changes you made became yellow4. Push “save” and “close”

3. Save a PHYSICS recipe1. Go into the corresponding ODE PC2. Open the ODE panel you want to change from the DEN3. Push “recipes…”4. Select “PHYSICS” recipe and push “edit”5. Take note of the value of the TTCrx registers “FD1” and “CD” and “close”6. Create the recipe “Create from hardware”7. Select “PHYSICS” recipe and push “edit”

1. push “edit” into “table Mode”2. Wrote the value you have toke note for TTCrx “FD1” and “CD” registers3. Push “enter” into the keyboard: the changes you made became yellow4. Push “save” and “close”

30

Muon Chambers

In the Cavern:

Chambers/CARDIACSB/PDM IB ODEHV distributorLV Marathon

SB/PDM

ODE

In B1:LV distributor (RCM/ACDC)

In D3:Tell1 boards for DAQ and L0 trigger

In D2:PCs with PVSS projects and SYSTEC witch connect ECS signals from SB/PDM/ODE to the PC)

IB