BKh - 16/01/2007 LHC PM Workshop 1
SDDS to LabVIEWThe path from client data to
viewing and analysis
Greg Kruk ➔ 08/2006 SDDS converter package
Boris Khomenko 2005 ➔ PM module viewer/analyser
-PMX for SDDS
Hubert Reymond 2005 Graph sheets
Dmitriy Kudryavtsev 2006 ➔ Generic SDDS converter
BKh - 16/01/2007 LHC PM Workshop 2
SDDS
Converter
PMD SDDS➔
pmdata
PMServer
subscription
PMD name
dev-time.pmd sys-class-pmxNN-dev-time.pmdpmdata.bin
PMXSDDS_bin
SDDS_ascii
NB: Duplicated PMD files are recognized and saved differently.Reprocessing is possible
scan_pmbin scan_sddsscan_pmclog
log
BKh - 16/01/2007 LHC PM Workshop 3
SDDS Converteris an Event Builder
SYSClass
evdays
devlst
year
device
evd_day
dvm_month
ev_evid
dv_evid*.sdds
*_ascii.sdds…
pmdata.binpmx_*-vNN.txt
s-link
BKh - 16/01/2007 LHC PM Workshop 4
SDDS: Self Describing Data Set
H
e
a
d
e
r
SDDS1
¶meter name=PMM_System, type=string, &end
¶meter name=PMM_Class, type=string, &end
¶meter name=PMM_Source, type=string, &end
¶meter name=PMM_Timestamp, type=llong, &end
&array name=AQNTIME, type=llong, &end
&array name=MB.LOC:ST_PWR_PERM, type=byte, modifier=ST:, &end
&array name=MB.LOC:U_2, type=float, format_string=%.4f, &end
&array name=MB.LOC:U_1, type=float, format_string=%.4f, &end
&array name=MB.LOC:U_QS0, type=float, format_string=%.4f, &end
&data mode=ascii, &end
D
a
t
a
QPS
DQAMCMB
C16L2
1165859504818000000
2548
1165859503784000000 1165859503789000000 1165859503794000000 ……
2548
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 …
…….
BKh - 16/01/2007 LHC PM Workshop 5
Added features for PM ➔ SDDS+
New data type “llong’ (int-64, timestamp standard for PM) Additional description field “Modifier”, used for:
o Name definition for digital data ( states and bit flags )• “Modifier=ST:” - boolean (0=False, 1=True)• “Modifier=ST:/Name_0/Name_1/…” - names for states 0,1, …• “Modifier=FL:/Flag_0/Flag_1/…” names for bit flags 0,1,…
o Arithmetic operations on analog values while file reading • “Modifier=*1e-3” - f.ex. conversion mV -> V
o Individual customization of string data representation (examples) • “Modifier=u1” - string length format is byte (for binary SDDS)• “Modifier=sep:t” - set string separator to tab (for ascii SDDS)
BKh - 16/01/2007 LHC PM Workshop 6
PMX: PM data index and controlSimple text file which makes SDDS converter generic
A text file fully defining the conversion process of raw data into SDDS file(s). Is very simple for simple cases, is clear to treat special requirements, is open for new (reasonable) complications.
Must contain
o Raw data block description (formats and names)
May contain
o Additional info for SDDS object attributes
o Instructions for data generation/modification
o Instructions for SDDS file formation
BKh - 16/01/2007 LHC PM Workshop 7
PMX/Converter extended options
As it was asked by users, the conversion takes additional efforts to make SDDS files more comfortable to read by their applications possibly without additional operation on data. So such operations may be performed by converter:
• (Yet from Greg’s version) Table data are represented as column arrays, not rows as in standard SDDS practice.
• NZ segmentation: automatic detection and cut of leading and trailing zeros in timestamp array, then all table columns will be cut in the same way
• Timestamp column generation as N samples at rate F or Δt, then adjusting f.ex. last element to acquired value
• Generation and/or modification of columns via arithmetic and some others operations on existing ones
• Creation several SDDS files per data module
BKh - 16/01/2007 LHC PM Workshop 8
PMX examples$SET Endian LITTLE
i4 TYPE
i4 CIRCUIT
i8 QUENCHTIME
$TABLE Rows 2550 SegmNZ AQNTIME
i8 AQNTIME
u1 MB.LOC:ST_PWR_PERM {ST:}
u1 DQQDL.LOC:ST_COHER {ST:}
u1 MB.LOC:ST_MAGNET_OK {ST:}
u1 MB.LOC:ST_NQD0 {ST:}
+80
f4 MB.LOC:U_2 (%.4f)
f4 MB.LOC:U_1 (%.4f)
f4 MB.LOC:U_QS0 (%.4f)
f4 MB.LOC:U_HDS_1 (%.4f)
f4 MB.LOC:U_HDS_2 (%.4f)
+72
$TEND
$FILE
$SET Endian BIG
#
# Buffer 'status'
…….
$TABLE Rows 21000 SegmNZ TIMESTAMP_SEC
i4 TIMESTAMP_SEC
i4 TIMESTAMP_USEC {*1e-6}
......
u2 ST_MEAS_A {FL:I_MEAS_OK/V_MEAS_OK/...}
u2 ST_MEAS_B {=}
......
i2 I_DIFF_MA {*1e-3} (%.3f)
……
f4 V_REF (%.4f)
f4 V_MEAS (%.4f)
$TEND
$GENCOL f4 I_DIFF = I_DIFF_MA/1000;
$MAXCOL 30
$MINCOL -30
$FILE Suffx status
BKh - 16/01/2007 LHC PM Workshop 9
Conversion flowchart
PM server
unzip
PMD
bin
PMX
SDDS
ascii
-bin
Dataobjects
pmdata
BKh - 16/01/2007 LHC PM Workshop 10
Conversion at client site (proposal)
optional use of block build methods
PM serverPMX
SDDS
bin
i
mage
Dataobjects
Data buffer
initDataModule(…)addDataIntParam(…)addDataStrParam(…)addDataLlongArr(…)…sendDataModule(…)
BKh - 16/01/2007 LHC PM Workshop 11
SDDS Converter Status• Is in production for QPS and PC
• Full processing time for module:– QPS/MB - 44ms– FGC/51 - 440ms (9 SDDS files)
• All user requests are implemented
• Demo for “conversion at client site” is almost ready, then contacts with interested users, if any, might be useful
BKh - 16/01/2007 LHC PM Workshop 12
Are PMD and SDDS data in security ?
• PM Server and SDDS Converter run under “OPERA” account
• It is a garbage account, 20 years old• PM project needs to be normally legitimated in this
sense, f.ex. by creation PMS, PMA and PMU accounts under group “PM” with permission scheme like:
PMD SDDS PM Results
PMS W W -
PMA - R W
PMU - R R
BKh - 16/01/2007 LHC PM Workshop 13
PMM_VAPM Module data View and Analysis
PMM data
locator
SDDS Read
SYS spec data modif
SYSClassOption
Analysis
Result Views
Analog +
State Data View
Result Table
BKh - 16/01/2007 LHC PM Workshop 14
PMM_VA in automatic mode(reduced application)
SDDS Read
SYS spec data modif
SYSClassOption
Analysis
Result Views
A+S Data View
Result Table
PMM data
locator
File(s)Event
Result Out
call params
BKh - 16/01/2007 LHC PM Workshop 15
PMM Data Locator-1
BKh - 16/01/2007 LHC PM Workshop 16
PMM Data Locator-2
BKh - 16/01/2007 LHC PM Workshop 17
SDDS reading
SDDS
Data
Parameters
Analog data channels
State data channels
Timestamp channel
Internal object types
BKh - 16/01/2007 LHC PM Workshop 18
QPS MB Graphs
BKh - 16/01/2007 LHC PM Workshop 19
QPS analysis result
NB. QPS team runs its own analysis locally. Nothing requested from PMM_VA .The shown result is from older frosen version, just for impression
BKh - 16/01/2007 LHC PM Workshop 20
S-Map for FGC
BKh - 16/01/2007 LHC PM Workshop 21
PMM_VA status
• New data locator• OK to read SDDS binary or ascii for QPS and PC• OK to display limited number of analog and state
(A+S) channels• Is in progress to show many channels by scrolling,
selecting, sorting and regrouping channel lists• QPS analysis no more requested by the team• PC automatic analysis is in development• Version 2.00 to the end of January