A Software Solution for the Control, Acquisition, and Storage of CAPTAN

Network TopologiesRyan Rivera, Marcos Turqueti, Alan Prosser, Simon Kwan

Electronic Systems Engineering Department, Fermi National Accelerator Laboratory

Abstract — The Electronic Systems Engineering department of the Computing Division at the Fermi National Accelerator Laboratory is developing a data acquisition system flexible and powerful enough to meet the demands of a variety of pixel and strip detectors for high energy physics applications. To facilitate data acquisition and processing as well as system configuration and control, a stacked system was devised with support allowing it to be placed in an Ethernet network operating at gigabit per second data rates. The individual unit within the system is known as a Compact And Programmable daTa Acquisition Node, or CAPTAN. The platform’s flexibility is achieved through the ability to stack specialized boards, both vertically and horizontally, to create an integrated system that can be optimized for each user and application. The CAPTAN’s main interface is via the Universal Datagram Protocol of the Internet Protocol (UDP/IP). The software solution presented in this paper is at the other end of the UDP/IP interface, and must orchestrate communications. The software must take a modular approach to its command library to give the user a unique and extensible vocabulary with which to communicate with ever advancing readout chip technologies and varying configurations of the specialized boards within a CAPTAN. The software must also handle multiple CAPTANs, each producing gigabits of data per second, thus the solution presented here provides an option to employ distributed computing for CAPTAN network topologies involving large amounts of data.

This figure depicts an overview of the CAPTAN software system and highlights the role of the Global Master (GM) within the solution. The GM is the unique central point for the CAPTAN software solution. Commands generated by the users are sent to the GM which then decides the proper final destination for the commands and finally conducts the forwarding. In the other direction, the GM forwards data from the CAPTANs to the proper recipients on the user side for interpretation.

Above the CAPTAN and User sides are better defined. The CAPTAN Controller (CC) is shown as the go-between for the Global Master (GM) and a CAPTAN. The CAPTAN Controller also has access to permanent storage which may be local or remote.

The GUI is the access point for users to the control and acquisition features of the system. The GUI can either be a stand-alone application using TCP/IP or a web based application using HTTP to communicate with the GM.

The approach taken for the GUI was to employ a tab-based modular interface to facilitate quick changes to configure, control, and interpret readout chips that are not known a priori. This makes it easy for users to define their own unique command sets and data interpretation.

Shown here is an example of the CAPTAN hardware. It is one of the specialized boards that is a building block of the hardware system. The four large connectors at the North, South, East, and West portions of the circuit board facilitate vertical stacking. The two connectors at the North-East and South-West edges are for horizontal stacking.

The software solution interfaces to this and other boards through an Ethernet interface which is controlled on the hardware side by the Xilinx Virtex-4 FPGA at the center of the board.

Presented in this paper are design decisions made to achieve a complete software solution for interfacing with a network of CAPTANs. A network of CAPTANs could mean a single CAPTAN working in a simple test stand system conducting basic functionality tests. It could also mean a farm of CAPTANs handling the controls and data acquisition for an entire experiment

K

1

2

1

2

K

CAPTAN

Side

User

Side

Storage

CAPTAN

The software system is novel in that it gives the user the option of maintaining the same controls, acquisition, and analysis tools from a single unit test stand to final system production. The system scales by taking advantage of multi-process and distributed computing techniques, and by exploiting the design decision which allows for the replication of function simply by executing multiple instances of the processing blocks.

This figure represents the wide range of CAPTAN network topologies supported by the software solution. C stands for CAPTAN, N for Network, CC for CAPTAN Controller, SC for Storage Controller, GUI for Graphical User Interface, and GM for Global Master. There can be only one GM, but the rest of the processing blocks may be replicated.

The simplest test stand system would have K = 1, L1 = 1, and J1 = 0 within the topology. And the CC, GM, and GUI would all run on the same computer with a CAPTAN connected through the computer’s Ethernet interface.

Isolated networks are shown to demonstrate that CAPTANs need not be on the main network that may be shared by an entire facility. This helps allay potential network security concerns.

Fermilab