VLVnT 2011S. Anvar, H. Le Provost, F. Château, F. Louis,

P. Sizun. Y. Moudden, V. Gautard, K. Ménager , E. Zonca & B. Vallage

CEA Irfu – Saclay

On behalf of the KM3NeT Consortium

Online Control and ConfigurationKM3NeT Design

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

KM3NeT Detection Units

2

Storey

Digital Optical Module

(DOM)

Complete telescope

~ 10000 DOMs

Ons

hore

Offs

hore

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Network Topology

3

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

Optical Gb Network Optical Gb Network

OnshoreElectronicsOnshore

ElectronicsOnshoreElectronics

OnshoreElectronicsOnshore

ElectronicsOnshoreElectronics

High Performance Multi-Gb Network

Computer FarmUtility Computers

See talk byF. Louis

See talk byJ. Hogenbirk

Ons

hore

Offs

hore

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Online Processes

4

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbeddedElectronics

DOMEmbedded

SoC

Optical Gb Network Optical Gb Network

OnshoreElectronicsOnshore

ElectronicsOnshoreElectronics

OnshoreElectronicsOnshore

ElectronicsOnshoreElectronics

High Performance Multi-Gb Network

Computer FarmUtility Computers

OffshoreDAQ Threads

(VxWorks)

OffshoreDAQ Threads

(VxWorks)

OffshoreDAQ Threads

(VxWorks)

OffshoreDAQ Threads

(VxWorks)

Control & Configuration Monitoring

(Linux, Smartphone…)

See talk byA. Papaikonomou

Data Routers(Linux) Data Filters

(Linux) Data Managers(Linux)

Ons

hore

Offs

hore

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Time-slice building

5

Performance Parallelism

ComputingFarm

Intrinsic Parallelism

DetectionNodes

Switch Fabric

Ons

hore

Offs

hore

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Data FilterData FilterData FilterData FilterData FilterData Filter

OffshoreDAQ Server

OffshoreDAQ Server

Online Process Architecture

Electronics Control & Configuration

Run ControlGlobal Configuration

Command Line User InterfaceGraphical

User Interface WebUser Interface

Data SenderData Sender

OffshoreDAQ Server

Data RouterData Router

Data Sender

Data Storage& Management

Full Client-ServerArchitecture

6

Ons

hore

DAQ

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Massively Distributed Application

7

Client-Server Architecture

TCP Protocol

IP Networking

Switched Ethernet

~10000 processes

Middleware: Internet Communication Engine

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

ICE Middleware

8

Client-Server Architecture

~10000 processes

Ported to VxWorksfor KM3NeT (5 days)

Open Source Embedded ObjectOriented Multi

Language:C++, Java, Python,Objective-C, C#,

Ruby, PHP…Network PerformanceControl & Acquisition

Middleware: Internet Communication Engine

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Client-Server over ICE

9

Client Server

Contract:Interface Defintion

Code:Client

Language

Code:Server

Language

Code:Server

Implementation

GeneratedImplemented

Ons

hore

Offs

hore

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Client-Server over ICE

10

Data Router

Data Sender

Electronics Control & Configuration

Run ControlGlobal Configuration

GraphicalUser Interface

OffshoreDAQ Server

Data Filter

Data Storage& Management

C++ VxWorksC++ VxWorks

C++ Linux

C++ Linux

Java JVM

C++ Linux

C++ Linux

C++ LinuxCommand Line User Interface

Python PVM

No NetworkProgramming

TCP/IPPerformance

Development byMultiple Teams

ManagementTools

See talk byA. Papaikonomou

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Acquisition Format

11

Multiframe MetaformatGeneric, binary, versionable DAQ data format

Allows backward & forward compatible format evolution

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Configuration Framework

12

Config ID

Configsave/restore

Database

CConfig Framework (KM3NeT)– C++ Framework (Host & embedded)– Multilanguage server access (using ICE)– Java wrappers– Automatic Config Trees DB mapping– Parameter default value mechanism– Automatic association with RUN ID

. . .

RunControl

Expert Access

ElectronicsControl & Config

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Configuration Framework

13

#include "CCfg/CConfig.h"#include "CCfg/Document.h"Ccfg::Document doc("/DataPath/KM3NetBench.xcfg");CCfg::CConfig cfg(doc.getConfig());

bool isActive = cfg("NodeConfig")("Node",102)("TDC_Chan",1);

Default value

Overriden value

VLVnT2011 S. Anvar, CEA Irfu Saclay – KM3NeT

Conclusion

• Complete Control & Configuration Framework• Tightly integrated with Data Acquisition• Multi-language / Multi-OS• Highly Modular– Client Server Components– Rigorous Interfacing– Multiple Development Teams– Progressive / Continuous Integration

• Ready by Spring 2012 for PPM-DU

14