bkh - 16/01/2007lhc pm workshop1 sdds to labview the path from client data to viewing and analysis...

BKh - 16/01/2007 LHC PM Workshop 1

SDDS to LabVIEWThe path from client data to

viewing and analysis

Greg Kruk ➔ 08/2006 SDDS converter package

Boris Khomenko 2005 ➔ PM module viewer/analyser

-PMX for SDDS

Hubert Reymond 2005 Graph sheets

Dmitriy Kudryavtsev 2006 ➔ Generic SDDS converter


SDDS

Converter

PMD SDDS➔

pmdata

PMServer

subscription

PMD name

dev-time.pmd sys-class-pmxNN-dev-time.pmdpmdata.bin

PMXSDDS_bin

SDDS_ascii

NB: Duplicated PMD files are recognized and saved differently.Reprocessing is possible

scan_pmbin scan_sddsscan_pmclog

log


SDDS Converteris an Event Builder

SYSClass

evdays

devlst

year

device

evd_day

dvm_month

ev_evid

dv_evid*.sdds

*_ascii.sdds…

pmdata.binpmx_*-vNN.txt

s-link


SDDS: Self Describing Data Set

H

e

a

d

e

r

SDDS1

&parameter name=PMM_System, type=string, &end

&parameter name=PMM_Class, type=string, &end

&parameter name=PMM_Source, type=string, &end

&parameter name=PMM_Timestamp, type=llong, &end

&array name=AQNTIME, type=llong, &end

&array name=MB.LOC:ST_PWR_PERM, type=byte, modifier=ST:, &end

&array name=MB.LOC:U_2, type=float, format_string=%.4f, &end

&array name=MB.LOC:U_1, type=float, format_string=%.4f, &end

&array name=MB.LOC:U_QS0, type=float, format_string=%.4f, &end

&data mode=ascii, &end

D

a

t

a

QPS

DQAMCMB

C16L2

1165859504818000000

2548

1165859503784000000 1165859503789000000 1165859503794000000 ……

2548

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 …

…….


Added features for PM ➔ SDDS+

New data type “llong’ (int-64, timestamp standard for PM) Additional description field “Modifier”, used for:

o Name definition for digital data ( states and bit flags )• “Modifier=ST:” - boolean (0=False, 1=True)• “Modifier=ST:/Name_0/Name_1/…” - names for states 0,1, …• “Modifier=FL:/Flag_0/Flag_1/…” names for bit flags 0,1,…

o Arithmetic operations on analog values while file reading • “Modifier=*1e-3” - f.ex. conversion mV -> V

o Individual customization of string data representation (examples) • “Modifier=u1” - string length format is byte (for binary SDDS)• “Modifier=sep:t” - set string separator to tab (for ascii SDDS)


PMX: PM data index and controlSimple text file which makes SDDS converter generic

A text file fully defining the conversion process of raw data into SDDS file(s). Is very simple for simple cases, is clear to treat special requirements, is open for new (reasonable) complications.

Must contain

o Raw data block description (formats and names)

May contain

o Additional info for SDDS object attributes

o Instructions for data generation/modification

o Instructions for SDDS file formation


PMX/Converter extended options

As it was asked by users, the conversion takes additional efforts to make SDDS files more comfortable to read by their applications possibly without additional operation on data. So such operations may be performed by converter:

• (Yet from Greg’s version) Table data are represented as column arrays, not rows as in standard SDDS practice.

• NZ segmentation: automatic detection and cut of leading and trailing zeros in timestamp array, then all table columns will be cut in the same way

• Timestamp column generation as N samples at rate F or Δt, then adjusting f.ex. last element to acquired value

• Generation and/or modification of columns via arithmetic and some others operations on existing ones

• Creation several SDDS files per data module


PMX examples$SET Endian LITTLE

i4 TYPE

i4 CIRCUIT

i8 QUENCHTIME

$TABLE Rows 2550 SegmNZ AQNTIME

i8 AQNTIME

u1 MB.LOC:ST_PWR_PERM {ST:}

u1 DQQDL.LOC:ST_COHER {ST:}

u1 MB.LOC:ST_MAGNET_OK {ST:}

u1 MB.LOC:ST_NQD0 {ST:}

+80

f4 MB.LOC:U_2 (%.4f)

f4 MB.LOC:U_1 (%.4f)

f4 MB.LOC:U_QS0 (%.4f)

f4 MB.LOC:U_HDS_1 (%.4f)

f4 MB.LOC:U_HDS_2 (%.4f)

+72

$TEND

$FILE

$SET Endian BIG

#

# Buffer 'status'

…….

$TABLE Rows 21000 SegmNZ TIMESTAMP_SEC

i4 TIMESTAMP_SEC

i4 TIMESTAMP_USEC {*1e-6}

......

u2 ST_MEAS_A {FL:I_MEAS_OK/V_MEAS_OK/...}

u2 ST_MEAS_B {=}

......

i2 I_DIFF_MA {*1e-3} (%.3f)

……

f4 V_REF (%.4f)

f4 V_MEAS (%.4f)

$TEND

$GENCOL f4 I_DIFF = I_DIFF_MA/1000;

$MAXCOL 30

$MINCOL -30

$FILE Suffx status


Conversion flowchart

PM server

unzip

PMD

bin

PMX

SDDS

ascii

-bin

Dataobjects

pmdata


Conversion at client site (proposal)

optional use of block build methods

PM serverPMX

SDDS

bin

i

mage

Dataobjects

Data buffer

initDataModule(…)addDataIntParam(…)addDataStrParam(…)addDataLlongArr(…)…sendDataModule(…)


SDDS Converter Status• Is in production for QPS and PC

• Full processing time for module:– QPS/MB - 44ms– FGC/51 - 440ms (9 SDDS files)

• All user requests are implemented

• Demo for “conversion at client site” is almost ready, then contacts with interested users, if any, might be useful


Are PMD and SDDS data in security ?

• PM Server and SDDS Converter run under “OPERA” account

• It is a garbage account, 20 years old• PM project needs to be normally legitimated in this

sense, f.ex. by creation PMS, PMA and PMU accounts under group “PM” with permission scheme like:

PMD SDDS PM Results

PMS W W -

PMA - R W

PMU - R R


PMM_VAPM Module data View and Analysis

PMM data

locator

SDDS Read

SYS spec data modif

SYSClassOption

Analysis

Result Views

Analog +

State Data View

Result Table


PMM_VA in automatic mode(reduced application)

SDDS Read

SYS spec data modif

SYSClassOption

Analysis

Result Views

A+S Data View

Result Table

PMM data

locator

File(s)Event

Result Out

call params


PMM Data Locator-1


PMM Data Locator-2


SDDS reading

SDDS

Data

Parameters

Analog data channels

State data channels

Timestamp channel

Internal object types


QPS MB Graphs


QPS analysis result

NB. QPS team runs its own analysis locally. Nothing requested from PMM_VA .The shown result is from older frosen version, just for impression


S-Map for FGC


PMM_VA status

• New data locator• OK to read SDDS binary or ascii for QPS and PC• OK to display limited number of analog and state

(A+S) channels• Is in progress to show many channels by scrolling,

selecting, sorting and regrouping channel lists• QPS analysis no more requested by the team• PC automatic analysis is in development• Version 2.00 to the end of January

bkh - 16/01/2007lhc pm workshop1 sdds to labview the path from client data to viewing and analysis...

Documents

pmx sdds

sdds files

sdds scan

bin sdds

ascii sdds slide

lhc pm workshop4 sdds

lhc pm workshop1 sdds

binary sdds modifier