STAMP: A NEW DATA ACQUISITION SYSTEM FOR ESA’S LARGE SPACE

SIMULATOR

H. Guijt(1)

, A. Popovitch(2)

(1)

TERMA, Schuttersveld 9, 2316XG Leiden, The Netherlands, tel: +31 71 524 0835, fax: +31 71 514 3277, email:

hg@terma.com (2)

ESA/ESTEC, Keplerlaan 1, 2201AZ Noordwijk, The Netherlands, tel: +31 71 565 3321, fax: +31 71 565 6541,

email: alexandre.popovitch@esa.int

1 ABSTRACT

STAMP is an advanced data acquisition, analysis and presentation system for thermal testing. To meet the increasingly complex requirements for modern spacecraft testing, STAMP was designed to be a flexible system that supports thousands of sensors and hundreds of power supplies. A high-performance, feature-rich presentation system is used to manipulate and present the acquired data in real time. The system runs on cheap computer hardware and supports multiple users. STAMP is currently used with the Large Space Simulator at ESTEC (see Fig. 1) and several other test facilities. Key features are: � powerful data presentation capabilities, including

various table formats and graphs, alarm generation, equilibrium detection, and output to Excel;

� extensive formula system to manipulate acquired data;

� able to acquire data from many thousands of sensors of virtually any type, including thermocouples, thermistors, and digital lines;

� able to control hundreds of power supplies; � able to interface with customer EGSE and spacecraft

hardware; � high degree of flexibility with respect to thermal test

configuration.

Fig. 1: the Large Space Simulator at ESTEC

2 DESIGN

STAMP ("System for Thermal Analysis, Measurement, and Power supply control") is designed as a client-server system (see Fig. 2). The server is an HPUX or Linux system, which is used as the repository for two databases: the Measurement Database, which contains all acquired data, and the Configuration Database, which contains the test configuration. The server also provides a communication mechanism for various stand-alone client processes. Workstations running Windows, Linux, or HPUX are responsible for data acquisition and presentation, configuration, and monitoring of system messages. Since client processes run independently of one another, a high degree of separation and thereby reliability is achieved. Workstations can be added and removed at any time to meet specific customer needs. STAMP contains no hard-coded information about the presence or properties of client software. The advantage is that new client software, such as software for presentation or power supply control, can be added without having to change existing software. Similarly, no information has been hard-coded regarding the test facility itself. The server used at ESTEC is a clustered pair of HPUX machines running in hot standby configuration. Only one of the machines in the cluster is actively used by STAMP at any time. The other provides redundancy in case the active server breaks down. On such an event STAMP automatically reconfigures itself to allow the test to continue without interruption. STAMP uses TCP/IP for all inter-process communication. This makes it possible for STAMP clients to run over the internet, if required. Optionally the communication can be compressed to be used with low bandwidth connections such as mobile phones.

Fig. 2: STAMP topology

3 ACQUISITION

STAMP acquires data from multiple types of scanner. Each scanner type requires a specialised acquisition client, which consist of two parts: the acquisition library and the scanner-specific code. Because the scanner-specific code is relatively small and simple, new scanner types can be added to STAMP with minimal effort. The acquisition library is shared between all acquisition clients and is responsible for controlling the timing of the scans, retrieving and maintaining the scanning configuration, sending acquired data to the Measurement Database, and for temporary local data storage. It also provides a simple man-machine interface with STAMP status information to the user. The scanner-specific code is responsible for translating the generic scan commands of the library into scanner-specific commands, and for interpreting data returned from the scanners. STAMP can process any kind of scalar numerical data, regardless of the unit or type. This allows it to handle any type of sensor that produces such data. At the time of writing, the following scanner types are supported by STAMP: � ASL F200 scanner; � ASL F300 scanner; � EGSE acquisition link (used to acquire data from

EGSE systems); � File import client (for offline import of data from

unconnected systems); � HP34420A scanner; � HP34970A (see Fig. 3); � HP3497A; � Keithley 2182A scanner; � Keithley 2750 scanner; � OPC link; � PLC link; � TempDAS (see section 3.2).

The STAMP design allows for communication with other acquisition systems such as EGSE systems ("Electronic Ground Support Equipment"). In this configuration STAMP would, in addition to its native data, also acquire ‘second-hand’ data from the other system.

3.1 GPIB scanner interface

GPIB scanners are connected to STAMP using a generic acquisition program that can handle GPIB and VISA-compatible scanners. The configuration MMI and scanner interface commands are expressed using a simple language called DCL (“Device Configuration Language”). Although DCL has less than 20 commands it is a complete programming language, with support for variables, branching and loops, as well as powerful features like regular expressions, exceptions, and a sophisticated expression evaluator. A built-in debugger makes it easy to step through running DCL programs. Because DCL was specifically designed to support GPIB-interaction, it allows complete scanner-interface programs to be expressed in a minimal number of lines. Using DCL, operators can add new scanners to STAMP without requiring the assistance of a software engineer.

3.2 TempDAS

To increase the number of hardware channels available to STAMP, ESA developed TempDAS ("Temperature Data Acquisition System"). It is an A/D converter-multiplexer designed to overcome the limited number of channels normally available on the slip ring unit of the Large Space Simulator. TempDAS is located on the rotating part of the motion simulator, which is why it must be able to operate in the same extreme environment normally reserved for space hardware. TempDAS consists of a MIL1553 bus that connects up to 30 acquisition boards to one controlling PC. There are several types of acquisition board: � relay based thermocouple (54 channels); � thermistor (26 channels); � solid state thermocouple (54 channels).

4 DATA STORAGE

The data acquired by STAMP is stored in the Measurement Database, residing on the server. The database consists of numerous data files, one per acquisition client per day. Each client has a helper application on the server, which is responsible for

writing acquired data into the relevant data file. A handshaking protocol makes sure that no data is lost even when the helper application or server fails. To protect the data files from disk failure, the files are physically located on a RAID-1 (“Redundant Array of Inexpensive Disks”) disk chain. The RAID-1 concept guarantees that data are stored on at least two separate disks. During a prolonged server or network failure, acquisition continues normally, even though no data is transmitted to the server. When the server becomes available again, the client will retransmit all data from temporary local storage to the server in one batch. In this way, acquisition is assured even during major failure modes.

Fig. 3: a HP34970A scanner

To keep the acquisition subsystem of STAMP as simple as possible, only raw data is stored in the Measurement Database (in this context, “raw” means “still in the same unit the data was acquired in”). All unit conversions and virtual sensor calculations are instead performed by the presentation subsystem. This approach has proven to be efficient.

5 POWER SUPPLY CONTROL

An important feature of STAMP is that it can control the power supplies used in most thermal tests. This means that customers do not need to buy or support their own power supplies. Power supplies can be controlled using the built-in control modes of STAMP, or through an external link to an EGSE. Power supplies can be attached to guard heaters, instrument dissipation emulation heaters, infrared lamps and other equipment. To support these different applications, various control modes are available which can be mixed at will. Table 1 lists available control modes.

Table 1: power supply control modes

Mode Description

Manual Setpoints are entered manually by the user.

thermostatic The system selects the most appropriate setpoint from a user-defined set, based on the value of a reference sensor.

PID The system selects the most appropriate setpoint using a PID algorithm, based on the value of a reference sensor.

K Multiple power supplies are controlled using a single setpoint.

Profile The setpoint is selected from a predefined sequence.

Formula The setpoint is the output of a user-specified formula; see section 6.1

5.1 Command process

The command process brings together all information a user needs to command power supplies, e.g. setpoints, actual measured values, reference sensor values and historical data on a single display. It displays a general overview of all power supplies controlled by the user and allows him to zoom in on a specific power supply when intervention is required (see Fig. 4). A security system ensures that any power supply is controlled only by the user responsible, to avoid conflicting command instructions. To avoid loss of control during a network failure, STAMP provides a man-machine interface allowing direct, local control over the power supplies.

Fig. 4: detail view of power supply command process

Table 2 lists all currently supported power supply types.

Table 2: supported power supplies

Type Max I Max V Max P

Gossen MSP24 0.6 A 40 V 24 W

Gossen SSP240 6 A 80 V 240 W Hart-Scientific 7380 NA NA NA Keithley 2400 1.05 A 210 V 220 W

6 PRESENTATION

STAMP contains an extensive presentation subsystem to give users as many tools as possible for monitoring and analysis of the acquired data. In addition to displaying data on screen, the presentation subsystem can be used to export data to other systems, such as EGSEs. Like the acquisition subsystem, the presentation subsystem can be easily expanded as most underlying functionality is implemented in a shared presentation library. Features of the presentation library include retrieval of configuration, retrieval of sensor values, conversion of raw sensor values to specific engineering units, calculation of virtual sensors, and saving and loading of stored presentations.

6.1 Formula system

STAMP supports the use of user-defined formulas of arbitrary complexity. The formula system can parse and evaluate formulas that look and behave like C-style expressions, with STAMP-specific extensions. It is used in numerous places within STAMP to convert acquired values to a desired engineering unit and calculate setpoints and the values of virtual sensors. It has also proven to be highly useful in other projects. A total of 76 operators are available in the formula system, some adopted from C, some unique to STAMP.

In addition to the normal operators, the ?: operator is

supported to allow conditional processing.

Table 3: types of operators

Type Examples

functions sin, log, sqrt functions with multiple arguments

max, avg, poly, stddev

keywords pi, rand operators +, *, <, **

logical operators &&, ||

bitwise operators <<, &, ^

constants 4.28, 0x3F references to sensors “pressure_1”, #301.raw

local variables $foo, $bar timestamps @”25 jan 2004 12:00:00"

In formulas, both real-time and historic data can be used. This allows for example the calculation of delta values or averages over time. Groups of sensors can be specified using wildcards, numbered ranges or enumeration.

6.2 Sensor types

STAMP supports four types of sensors: real sensors, virtual sensors, on-the-fly virtual sensors and comment sensors.

Real sensors are associated with an acquisition channel on a scanner. Values of real sensors are received from the Measurement Database or from the real-time channel. Usually a conversion is required to transform the received value from its raw value to the desired engineering value. Virtual sensors are not associated with an acquisition channel, but are defined by a formula (see section 6.1). They are defined globally, but are calculated locally from real sensors or other virtual sensors. Virtual sensors have many uses, such as calculating the average of a number of sensors or performing data transformations. On-the-fly virtual sensors are the same as virtual sensors, but they are defined locally and not shared with other presentations. This is useful for local manipulation of the data without having to change the test configuration. Finally, comment sensors are used to provide on-screen labels in various presentations. They are never evaluated, nor do they produce values in any way. Values from all sensors are numeric. Some values however are better understood as text, such as switch or valve states. The presentation subsystem can accommodate user understanding by translating such numbers into text.

6.3 Alarms

STAMP supports multiple types of alarm limit on each sensor. Alarm limits are defined globally, but can be overridden locally if desired. They can be set on all types of sensor except comment sensors. The following alarm types are available: � upper and lower alarm limit; � extrapolated upper and lower alarm limit; � upper and lower warning limit; � upper and lower slope limit; � undefined values. An “extrapolated alarm” occurs when the extrapolated value of a sensor exceeds the specified upper or lower alarm limit. Undefined values may occur as the result of an acquisition error or of certain calculations like division by zero. Although alarms are principally the responsibility of the alarm presentation process (see section 6.6.3), alarm indicators are displayed for all on-screen sensor values that are in alarm (see Fig. 5).

Fig. 5: tabular presentation with alarm indicators

6.4 Security

If required, it is possible to set access restrictions on sensors. A restricted sensor is visible only to authorised users. This is commonly used to separate facility sensors from spacecraft sensors, but can also be used to separate groups of spacecraft sensors from each other.

6.5 Loading and saving

During the course of a thermal test users will create and save many presentations. To help users quickly locate previously saved presentations, STAMP allows the user to associate a physical component of the spacecraft under test with each presentation. When reloading the presentation at a later time, the user simply looks up the component and selects the appropriate presentation (see Fig. 6).

Fig. 6: loading from a synoptic page

6.6 Available presentations

STAMP currently supports 18 types of presentation. Table 4 summarises their capabilities.

Table 4: presentation capabilities overview

Presentation Mode(1) X-link(2) OS(3)

Tabular (scan) both yes all

Tabular (over time) both yes all Alarm Real-time yes all

Graph both yes all Scatter graph both yes all Distribution graph both yes all

Equilibrium Real-time yes all Excel Offline yes all 2D synoptic Real-time yes all

3D synoptic both yes all 2D waterfall both yes all 3D waterfall both yes all

Event Real-time no all Throughput verification

Real-time yes all

Dynaworks both no HP-UX

Teletest Real-time no all CASA/FTP Real-time no Linux MetOp Real-time no Windows (1) This is either real-time, offline (retrieval from the

Measurement Database), or both. (2) If cross-linking is available, data from this

presentation can instantaneously be displayed using one of the other cross-linked presentation styles.

(3) This can be Windows, Linux, HP-UX, or all.

6.6.1 Tabular (scan) presentation

The tabular presentation displays sensors in multiple newspaper-style columns. It supports both real-time data and retrieval of older data from the Measurement Database. When data from multiple timestamps is available, it is possible to browse through the data easily. Optionally, an extra column can be displayed showing the scanner channel the data was acquired on.

6.6.2 Tabular (over time) presentation

This presentation displays multiple values from any number of sensors over a period of time. Each column in the presentation represents a sensor and each row a timestamp.

6.6.3 Alarm presentation

This presentation alerts the users to sensors that exceed their alarm limits. When an alarm occurs, the following actions take place: � an alarm sound is played; � a message is written to a log file; � a message is printed to a line printer. If a sensor is continuously in alarm, but nothing can be done about it, it is possible to temporarily place it in reserve. No alarms are raised on sensors that are in reserve, but the user can still monitor their values and actual alarm state. Since alarm generation is a presentation like any other, different users can choose to monitor different sensors

for alarms. There is no need to have all alarms monitored by all users at all times.

6.6.4 Graph presentation

The graph presentation (see Fig. 7) displays data of multiple sensors in a graph. It supports many features, including manual scaling, auto-scaling, linear and logarithmic scale, horizontal, vertical, cross-shaped and arrow-shaped labels, multiple vertical scales, user-defined colours and line styles, and an auto-follow mode for real-time data.

Fig. 7: graph presentation

If a virtual sensor contains a significant amount of noise, an option is available to split the virtual sensor into its component sensors. These are then displayed as separate curves in a newly created graph. This way the user can quickly identify which real sensor is responsible for the noise and take steps to remove it. The graph presentation has numerous convenient zoom options, such as zooming in on an area, incremental zoom out, undo zoom and an option to show all available data. The graph presentation has been ported to SCOS2000 as part of the Herschell-Planck EGSE development project.

6.6.5 Scatter graph presentation

The scatter graph presentation is similar to the normal graph presentation, but instead of displaying the values of sensors over time, it plots pairs of sensors against one another. It supports the same features as the graph presentation.

6.6.6 Distribution graph

The distribution graph shows how the data of a set of sensors is distributed. The number of categories is user-selectable, and several derived curves (such as an inverse curve, 1st-order approximation, and 2nd-order approximation) may be plotted as well.

6.6.7 Equilibrium presentation

This presentation (see Fig. 8) is used to determine if the spacecraft has reached a thermal equilibrium. This is achieved by calculating the equilibrium value for a selected group of sensors. This value is calculated by comparing the average of two periods of time (typically ten minutes) several hours apart. The presentation orders sensors by equilibrium value, using colour coding to differentiate those that are close to being in equilibrium from those that are not. Optionally, it can suppress sensors that are already in equilibrium. This allows the total thermal equilibrium state of the test object to be assessed at a single glance. Once all sensors are in equilibrium, a sound is played to inform the users. In addition to displaying the equilibrium value, several other values are displayed, including the average value, minimum, maximum, and standard deviation of the recent time range and the average value of the eldest time range.

Fig. 8: equilibrium presentation

Fig. 9: 2D synoptic presentation

6.6.8 Excel presentation

The Excel presentation creates Excel-compatible .CSV (“Comma Separated Value”) files containing values from the Measurement Database. The .CSV format is a common format which can also be read by many other tools. Therefore the format is useful as an exchange format for thermal data between ESTEC and its customers.

6.6.9 2D synoptic presentation

In this presentation, sensor values are associated with specific locations on an image of the spacecraft, similar to how stored presentations are associated with specific components (see Fig. 9). The 2D synoptic presentation can present sensor data as a simple number, as a horizontal or vertical bar, or as a dial.

6.6.10 3D synoptic presentation

The temperatures of the spacecraft are projected onto a 3D model of the spacecraft. The 3D model is obtained by importing an existing CAD model of the spacecraft. Colour ranges indicate the actual temperatures, and shapes are used to indicate alarm states. In addition to projecting temperature ranges, other parameters such as equilibrium state and heater setpoints may be displayed as well.

6.6.11 2D waterfall graph

The 2D waterfall graph displays the values of a large group of sensors side by side in a graph.

6.6.12 3D waterfall graph

In this experimental presentation, values are plotted in a 3D graph that can be inspected from all sides.

6.6.13 Event presentation

The event presentation triggers pre-defined system actions when certain criteria are met. There are three types of action: � generating a warning; � starting an unscheduled scan; � activating an external program or script. Since the event presentation can start external scripts, there is almost no limit to what it can achieve. In combination with the “stampcmd” program (see section 8.5) it can also make changes to the STAMP configuration itself, allowing test steps to be automated.

6.6.14 Throughput verification presentation

This presentation raises an alarm when measurement data on a channel is not received for too long. It is used to verify the correct operation of the acquisition subsystem, Measurement Database and the network in general.

6.6.15 Dynaworks presentation

The purpose of this presentation is to transfer data from STAMP to the Dynaworks system. Dynaworks is a presentation and analysis system created by Intespace. For more information, see http://www.dynaworks.com.

6.6.16 Teletest

This presentation transfers the data to a MySQL server for remote presentation and use by other applications.

6.6.17 CASA FTP presentation

This presentation transfers data to a remote FTP server.

6.6.18 MetOp presentation

This presentation was created specifically to interface with the MetOp EGSE hardware. It provides a specially formatted serial line that sends real-time data from STAMP to the EGSE.

7 REMOTE MONITORING

STAMP supports presentation of data outside the Test Centre, using a mechanism called Remote Monitoring. Using the Remote Monitoring Client (which contains all the common presentations), authorized users can connect to a dedicated server (which is separate from the main STAMP server) and perform the same monitoring and data analysis tasks they can perform in the Test Centre. To maximize security, two firewalls protect the main server: one between the main server and the dedicated server, and one between the dedicated server and the internet. The dedicated server is not allowed to initiate any connections to the main server. Certificates are used to verify the identity of any connecting Remote Client, and encryption is used to safeguard the data sent to the Remote Client once a connection has been established.

8 OTHER STAMP TOOLS

STAMP provides tools to modify configuration, monitor overall system status and perform various maintenance and system verification activities. This section gives a short overview of these programs.

8.1 Configuration program

This program (see Fig. 10) is responsible for examining and modifying the system configuration. Configuration can be modified before, during and after a test. If configuration is modified during a test, the changes will take effect immediately without the need to restart other programs. To avoid undesired configuration changes, the configuration program can itself be configured to allow only specific changes by specific users during each test phase.

Fig. 10: the Configuration Program

The Configuration Program supports data import and export to and from other applications such as Excel. Using Excel, customers may prepare their test configuration in advance and import it to quickly create the actual test configuration.

8.2 System Alarm Monitor

As STAMP is a distributed system, system status messages can be generated in inconvenient locations (i.e. outside the control room). The System Alarm Monitor gathers these messages and displays them in a central location.

8.3 System Process Monitor

This program monitors other processes, generating alarms when required processes are not running.

8.4 Main Control Program

The Main Control Program (see Fig. 11) is used to launch the various client programs. The program is used both by the system itself as well as by the users.

8.5 Replication Agent

The Replication Agent can copy a complete test from one STAMP server to another, and keep the slave server up to date during the testing (forwarding all stored data,

configuration changes, etc.). The slave server can be used as a remote monitoring server, or as a backup for the main server.

8.6 stampcmd

This program is a command line program used to query and change the Configuration Database. It allows UNIX shell-scripts to be written that modify the configuration, without requiring user intervention.

Fig. 11: the Main Control Program

8.7 Maintenance Program

The maintenance program (see Fig. 12) is used to perform general maintenance on the Configuration Database, such as: � making and restoring backups; � moving tests from one STAMP server to another; � updating and verifying database definitions; � installing new Configuration Databases. Backups are stored in XML (“eXtensible Markup Language”) format.

Fig. 12: the Maintenance Program

9 CAPACITY

STAMP was written to run well on networks containing dozens of machines for acquisition, power supply control and presentation. It was designed to support tests that are distributed through multiple locations and it can gracefully deal with issues such as network lag or failure. The design has no hard-coded limits. Instead, soft limits are set by available memory, CPU power and network bandwidth. The following values have been measured using the equipment available at ESTEC in 2003: � STAMP can handle any number of tests at the same

time, as long as the cumulative capacity used by those tests does not exceed the total capacity as defined below. Data, configuration and alarms from different tests are kept separate at all times;

� it can handle over 30,000 acquisitions per minute; � it supports any number of presentations, as long as

the total number of values retrieved from the server stays below approximately a million values per minute in real-time mode;

� there is no limit to the duration of a test or to the total amount of data acquired;

� STAMP does not impose any limits to the number of sensors used, the number of different scan definitions, the number of virtual sensors, or indeed of any configuration item.

Despite this, STAMP can also run on a single machine that acts as Configuration Database, Measurement Database, acquisition client and presentation client.

10 SPACECRAFT TESTED

At the time of writing, STAMP has been used to conduct thermal tests on the following spacecraft:

Table 5: spacecraft tested by STAMP

Test Date Facility

Helios 2 Aug. 2000 LSS

Hubble (solar panels) Oct. 2000 LSS MetOp PLM/EM May 2001 LSS MetOp PLM/EM Jun. 2001 LSS

Astra CE reflector Jul. 2001 LSS Astra IB reflector Oct. 2001 LSS Rosetta PFM Feb. / Mar. 2002 LSS

Integral FM Apr. / May 2002 LSS ATV STM Jul. / Aug. 2002 LSS Smart-1 Oct. / Nov. 2002 LSS

MetOp PLM/FM1 Nov. / Dec. 2002 LSS Smart-1 Dec. 2002 HBF-3 ISS Glovebox Nov. 2003 Corona

MetOp PLM/FM2 Dec. / Jan. 2004 LSS ISS Glovebox Feb. 2004 Corona

11 ACKNOWLEDGEMENTS

The authors would like to thank J. van der Meulen (ETS / ESTEC) for his tireless support and effort during the development of STAMP. We would also like to thank the many people whose ideas helped to shape STAMP, in particular C. Fransen (ESA / ESTEC), who is one of the originators of the STAMP project. This is an update of the original paper, taking into

account new features developed for STAMP after the

conference took place.