19. june, 2006 bologna alice week svetozár kapusta 1/28 pvss oracle archiving involved people in...

19. June, 2006

BOLOGNA ALICE Week Svetozár Kapusta 1/28

PVSS Oracle archivingPVSS Oracle archiving

Involved people in the latest developments:

Ronald Putz (ETM) Chris Lambert (IT-DES) Eric Grancher (IT-DES) Lothar Flatz (Oracle consultant) Luca Canali (IT-PSS) Manuel Gonzalez (IT-CO) Nilo Segura (IT-DES) Piotr Golonka (IT-CO) Laura Fernandez Nocelo (IT-CO) Svetozár Kapusta (PH-AIT)

19. June, 2006


HistoryHistory 2004

– ETM (company developing PVSS) promises to its customers to be able to archive to different flavors of database systems

2005– First prototype comes to CERN with many functionality issues,

mainly loosing data and low client performance (100 changes/s) Q4 2005

– Oracle consultant (paid by CERN) comes to ETM to boost client performance

Q1 2006– IT-CO puts enormous effort to test the new RDB Mgr

(Relational DataBase Manager)– insertion rate of 1500 changes/s for a few minutes– Stability issues, memory leak, slow Event Mgr – Data Mgr

communication, server shows low performance– Huge client memory consumption (up to 1.5GB RAM)

19. June, 2006


History 2History 2 April 2006

– Tuning session at CERN with Oracle consultant, 23 clients, 2 node RAC (Real Application Cluster) Oracle server

– Many enhancements, new issues discovered– Archive groups not working – partitioning on server instead– ~24 000 changes/s inserted continuously– Bottleneck: Network

May 2006– Tuning session at CERN with fixed RDB Mgr (also Linux

version available), 56 clients, 2 (later 4) node RAC server– lower memory consumption (300MB RAM)– ~56 000 (later 100k) changes/s inserted continuously– Bottleneck: Server

19. June, 2006


History 3History 3 End of May 2006

– Tuning session at CERN with ETM employee, 150 clients, 6 node RAC server

– Schema enhancements, new issues discovered and fixed– ~150 000 changes/s inserted continuously

Last Week & weekend– Tuning session at CERN, 170 clients, 6 node RAC server– ~70 000 changes/s inserted continuously over the weekend

Right Now– The tests with 170 clients still continue until end of the month to

tune queries

19. June, 2006


Test ConfigurationTest Configuration Client side

– mixture of Windows ALICE DCS lab (~20 PC) bldg 12 IT-CO lab (~4 PC) bldg 13

– and Linux clients LHCb (44 PC) Point 8 LxBatch (120 PC) Computing Center

– PVSS 3.1, cumulative patch– patches 167, 203 (available from IT-CO web page)– framework 2.3.6– data generated with simulator drivers configured to change the values of

DPEs (DataPoint Elements) at the required rate. Server side

– Oracle RAC server with 1, 2, 4 and 6 nodes (PCs) respectively– 3 GHZ Dual Processor Xeon, 2GB RAM, Oracle Real Application

Cluster 10.2.0.2 on Linux Red Hat Enterprise– Fiber channel RAID10, 32 SATA disks (20 write, 32 read)

19. June, 2006


Performance ImprovementsPerformance Improvements

19. June, 2006


Disable job to analyze tables Disable job to analyze tables and indexesand indexes

PVSS server job running every 3 hours– gathers statistics on objects in the DB to

improve performance– degrades performance considerably (one order

of magnitude) if ran at inappropriate times– this is a typical task for the DB administrator

19. June, 2006


Disable indexes on TS and Disable indexes on TS and element_IDelement_ID

Indexes are used to boost queries which filter on the timestamp and/or element_ID (unique number for each DP element)

Slowed the server considerably Consumed a lot of disk space (2/3 of data volume

was in the indexes (+extra data in the redo logs)) Useless for ALICE Got disabled, let only the primary key

(composite index on columns TS and element_ID)

Once the data are inserted, if required the disabled indexes could be enabled

More efficient in CPU and storage usage

19. June, 2006


Use direct path for data Use direct path for data insertion instead of merge insertion instead of merge

Used to move data from the client temporary tables to the history table

Has a big impact on performance Changes completely the current approach

– Previously: check is done before inserting a new value to see if it is a correction value or a new value

– Currently: insert everything in the history table and treat the errors afterwards

More efficient since new values will occur much more frequently in comparison to correction values

This way of treating errors can only be used since Oracle 10.2.

19. June, 2006


Index Organized TablesIndex Organized Tables

Data and index data are kept togetherFits the design (many inserts, few deletes) Transparent for client

Not as performant as INSERT /*APPEND/

19. June, 2006


INSERT /*APPEND/INSERT /*APPEND/

Oracle hintDon’t search for an empty place, just

append it at the end of the tableBoosts performance in high insert rate

environments (more than IOT)

But only 1 client can write into 1 table in time

19. June, 2006


Partition of the history table Partition of the history table per clientper client

Comes as an implication of the previous point

The history table has to be partitioned– Each client accesses only one partition and

doesn't interfere with other clients– Due to the way insert append works– To allow for scalability in a RAC server

Special feature that doesn't come in the standard Oracle version

19. June, 2006


Initial size and extend of Initial size and extend of tablespaces tablespaces

It is important to preallocate enough spaceSpace allocations and extensions were

happening very often and penalizing performance

We used the PVSS data driven mechanism to tune the parameters for allocations and extensions

19. June, 2006


Requests & Open IssuesRequests & Open Issues Password has to be introduced in config file in plain

text– It is just a workaround to be able to easily start the RDB for the

first time– one should rely on the internal DP

Negative system ids – system numbers bigger than 127 appear negative in the database– Problem fixed (new version of patch 203).

Archive groups – We spent a lot of time trying to understand what the clients do

when groups are used– We have not been able to make the Archive groups work– Remains an open issue

19. June, 2006


Requests & Open Issues 2Requests & Open Issues 2 Queries separated per DPE

– One SQL statement, let the DB do the processing– Don’t issue several separated SQL statements with processing in client– ETM needs to modify the dpGetPeriod function (also used by the IT-CO

trending tool)– use dpQuery in your code, so PVSS will not split the SQL queries

Memory allocation at OCCI level – OCCI 10.1 has a bug that makes the RDB Mgr crash if the first block

sent over OCCI is big– Current workaround: The RDB Mgr starts with a small block size,

increased afterwards– Solutions

Put a logic in the RDB limiting the size of the first block sent through OCCI Use OCCI 10.2

– ETM claims it has a bug when running queries– requires newer version of compilers that the ones ETM is using)

19. June, 2006


Requests & Open Issues 3Requests & Open Issues 3 Number of blocks retained in overload/DB not

reachable condition – A parameter limits the number of blocks in

memory of the RDB Mgr when it is not able to send all data

– However the RDB Mgr buffers blocks without any limit

– Dangerous since it could make the PVSS project crash

– Happens because the thread that writes to the DB is the same that manages the blocks in memory

– Solution would be to do the management of the blocks in the same thread that receives data from the DM and writes data to the blocks

19. June, 2006


Requests & Open Issues 4Requests & Open Issues 4 Very slow start up of clients

– RDB Mgr does several queries at start up: Query to get maximum Time Stamp in the history table Query to get all elements for events Query to get all elements for alerts

– The bigger the history table, the longer it takes run an initial query– The more clients were connected the slower the start of each client was– Last 2 queries a client is iterating over all the elements but keeps only his

ones (this filtering is much better done by the server and was fixed)– First query still an issue– Used to decide which values stored in the local lastval archive have to be sent

to the DB– ETM claims it is faster than to send all the values and see if there are

duplicates– Query the whole history table =>more than 1 client, it will not be coherent

(client should ask for the latest timestamp of a value inserted by him and not any client)

– History tables will be big => more efficient to insert all the local lastval values and the correct duplicates afterwards

– Not clear how this will work in 3.5 (local lastval archive has been disabled)

19. June, 2006


Requests & Open Issues 5Requests & Open Issues 5 Slow/Deadlock when starting clients after reconfiguration

– concurrent start of the clients was very slow and in some cases lead to a deadlock

– solution is that each client takes a lock on all the tables it requires (this serializes the operations and will not produce any deadlock situation)

Site configuration in RAC servers– There seems to be a problem when having a RAC as DB server. It is not

foreseen to easily reflect the RAC configuration in the internal table arc_site

– This is an issue because there is a check (SQL join) between the information in arc_site and a system view (v$instance) in the code

Algorithm to calculate element_id limited to 255 systems – The current algorithm to calculates the element_id in the DB from the

dpe_id, dp_id and system_id, is limited to 255 systems– PVSS 3.5 will not be limited to 255 systems in a distributed system

Querying data from a remote system requires local connection to DB – When querying data from a remote system, the local system needs to be

connected to the DB even though the query is executed by the remote system

19. June, 2006


Requests & Open Issues 6Requests & Open Issues 6

Error monitoring & handling – monitor the number of blocks in the RDB with

an internal DPE– know which periods of time were stored to file

instead of the DB– timeout in the threads that interact with the

DB. If for any reason they get blocked, their operation should finish gracefully

19. June, 2006


Future ETM improvementsFuture ETM improvements

PVSS Set up script for Oracle server– Will be run from a panel in PVSS– First version available– Future release will contain advanced configuration

parameters for CERN that set up the database to Disable the job gathering statistics Disable useless indexes Use direct path for data insertion instead of merge Use INSERT /*APPEND/ Partitioning (it is not a standard feature and most of the ETM

customers will not have it) Default parameters for allocation and extension of

tablespaces

19. June, 2006


Future ETM improvements 2Future ETM improvements 2

Queries in distributed systems– Query on data from a remote system goes via the remote system

Degrades performance of the remote system Some PVSS systems will be mostly writing to the DB, others will

be mostly querying (load will be on the systems that write data to the DB)

If remote systems are not available it is not possible to query their data, even if all the information is in the DB

– We would like to have the following behavior Queries go always through the local system, even if they require

data from a remote system Remote system is running, use its DP identification to query the DB Remote system is not running, use the information stored in the DB

to query the data. (Given a DP element name the query to get the element id for the DB is very straight forward and more efficient that manipulating the dpe id, dp id and system id to calculate the element id

19. June, 2006


Future ETM improvements 3Future ETM improvements 3 Buffer to local disk when not possible to send data to DB

– If the buffers in the RDB exceed a given size either because of an overload or because the DB is not reachable, it writes to files in the local disk

– separate file for each block that could not be handled– format of the files is ASCII– File contains SQL statements that are required to insert data into Oracle– Scheduled to be released in August. – CERN Comments on the concept: – The insertion from the files is done value by value. This is very bad for

performance, as we already saw in the early versions of the RDB manager. – Bind variables are not used. This is dangerous because it could delete the parsed

queries in the Oracle cache. – It was suggested to explore other possibilities:

Use Oracle SQL loader to import files with the values. This is the fastest way to insert the data. Error handling is a bit more complex because one has to wait for some report files to be generated.

Use the same method as for the normal insertion: write to the client temporary table and let the PL/SQL code in the DB do the job. This may require some changes in the PL/SQL code because right now it can only insert in the current history table, not in previous ones

19. June, 2006


CERN tests results CERN tests results Inserts of 170 clients each at 1000

changes/sDB was not able to handle fully the loadClients were inserting the data at the same

time because of their simultaneous startStarting clients randomly has increased

performanceLong term stability: 170 clients each

inserting at 500 changes/s during 14 hoursCurrently the impact of queries is tested

19. June, 2006


Other News And RemindersOther News And RemindersSPD are configuring their FERO using the

Oracle DB sitting in the DCS labAt CERN

– Use CERN DB service for PVSS configuration DB

– Use CERN DB service for FERO configuration DB or ask ACC team

– Ask ACC team or IT for an accountAt home institute

– install your own (Oracle versions to use)

19. June, 2006


Oracle Versions to useOracle Versions to use

The latest! Server 10.2.0.2:

– Standard and Enterprise editions (advanced users) http://www.oracle.com/technology/software/products/databa

se/oracle10g/index.html

– Express edition 10.2.0.1 http://www.oracle.com/technology/software/products/databa

se/xe/index.html

Any other development PCs:– Oracle instant client 10.2.0.2

– http://www.oracle.com/technology/software/tech/oci/instantclient/index.html

19. June, 2006


Current organization of SPD Current organization of SPD FERO configuration dataFERO configuration data

SPDVersion Number(10)PK

Side Char(1) PK

Sector Number(1) PK

Halfstave Number(1) PK

MCMVer FK

DACVer FK

MBRVer FK

SPDVersion

DACVersion

DACVect Raw(440)

DACVersion Number(9) PK

MBRVersion Number(9) PK

MBRVect Varchar2(2460)

MBRVersion

MCMVersion

ACO Blob

APIVect Raw(6)

DPIVect Raw(8)

MCMVersion Number(9) PK

GOLVect Raw(4)

19. June, 2006


ConclusionsConclusions

Huge improvements in performance of the PVSS Oracle archiving accomplished

RDB Manager is getting matureApplication can and will scaleNo “showstopper” identifiedFull functionality and tests expected this

summer

19. June, 2006


Thank you for your attention

19. June, 2006


AppendixAppendix

19. June, 2006


Organization of the Organization of the Configuration DatabaseConfiguration Database

The ALICE Running mode (e.g. lead-lead, cosmics, p-p, etc.) is defined by the ECS– The mode is sent to the DCS at every change– A valid configuration must exist for every running

mode The same configuration might be re-used for several modes

FERO Configuration: lookup tables define the relationship between the ALICE Running mode and the corresponding configuration version for each sub-detector

19. June, 2006


xxxxx

xxxxx

Hierarchical organization of Hierarchical organization of DCS FERO configuration dataDCS FERO configuration data

SPD DataSDD Data

TypeOfRun PKSPDVersion PKSDDVersion PKSSDVersion PK

…………

TRDVersion PK

SPDVersion PK

SSDVersion PKSDDVersion PK

TRDVersion PK

TRD Data

SSD Data

SPDModeTypeOfRun PK SPDversion FK

… xxxxx

ppRunID xxxxxCosmicsRunID xxxxx

… xxxxx…… xxxxx

PbPbRunID xxxxx

number(4,0) number(7,0)

Table NameColumn NameOracle Datatype

ALIVersion

A view created from the individual detector versions

xxxxx

SPDRemarkSPDversion FK

xxxxx

xxxxxxxxxx

xxxxx

xxxxx

xxxxx

number(7,0)Remark

xxxxx

xxxxxxxxxx

xxxxx

xxxxx

xxxxx

varchar2(1000)

Actual configuration modes(Fixed number of rows, only updates)

19. June, 2006


Organization of the Organization of the Configuration DatabaseConfiguration Database

Separate schemas (users and their tables) are created for each detector

Data organization is hierarchical and follows the intrinsic detector segmentation– The “atomic” data records represent a group of

parameters, which should be loaded together (e.g. chip settings, sector settings)

– A new configuration data for any of the detector components results in a new configuration set for the whole detector

Hierarchical organization allows for re-using the information already stored in the database

19. June, 2006


Configuration DB hierarchical Configuration DB hierarchical organizationorganization

Configuration VersionGroup 1. Version

…Group 1 Version

Sub-group 1 Version

…

Sub-group n Version

Group n Version

Group n Version

Sub-group 1 Version

…

Sub-group n Version

Sub-group 1 Version

DATA

Sub-group 1 Version

DATA

Sub-group n Version

DATA

Sub-group n Version

DATA

19. June, 2006


Configuration DB example: Configuration DB example: SPDSPD

Configuration VersionSide A Version Side A Version

Sector 0 Version

…

Sector 9 Version

Side C Version

Side B Version

Sector 0 Version

…

Sector 9 Version

Half-Stave 0 Version

DATA


…

…



DATA

Sector 0 Version

New configuration data

19. June, 2006


There are two options for writing the data record type – Table containing individual parameters– Configuration data stored in a BLOB

It is the responsibility of detector groups to define the database schema and implement the client code in the FED server(s)

The DCS team will– Provide assistance (we can help you to create the

schema and implement the client code)– Verify the schema (in collaboration with IT)– Install, operate and maintain the database

19. June, 2006


General recommendationsGeneral recommendations

Development of Oracle applications -General advices from IT– https://

twiki.cern.ch/twiki/bin/view/PSSGroup/GeneralAdvices

– Reference slideshowhttps://twiki.cern.ch/twiki/bin/view/PSSGroup/Ref

erenceSlideshow

– Database designhttp://agenda.cern.ch/askArchive.php?base=

agenda&categ=a044825&id=a044825s0t2/moreinfo

19. June, 2006


Define keys, constraints and Define keys, constraints and indicesindices

Primary key: Column or set of columns that uniquely identifies table

rows Every table should have a primary key defined Primary key protects from entering duplicated rows to a

table Oracle implicitly creates a unique index on primary key

columns Foreign key:

Relation (e.g. master-detail) between 2 tables should be formalized by creating a foreign key constraint

Foreign key helps to keep data integrity and facilities cascade operations

Oracle DOES NOT create implicitly indices on foreign key columns so if related tables are supposed to be joined one needs to create such index by hand.

19. June, 2006


Define keys, constraints and Define keys, constraints and indices (2)indices (2)

Primary and foreign keys syntax examples:– ALTER TABLE employees ADD CONSTRAINT

employees_pk PRIMARY KEY (id);

– ALTER TABLE departaments ADD CONSTRAINT employees_pk PRIMARY KEY (dept_id);

– ALTER TABLE employees ADD CONSTRAINT departaments_fk FOREIGN KEY (dept_id) REFERENCES departaments (dept_id);

19. June, 2006



Unique keys – Column or set of columns other then

primary key that uniquely identifies rows in the table

– Oracle implicitly creates index on unique keys

– In contrary to primary keys, unique keys allow NULL values

ALTER TABLE employees ADD CONSTRAINT employees_unq UNIQUE (lname);

19. June, 2006



Other constraints – Usually it is better to enforce data integrity

constraints on database server side than on client/middle tier side:

– Central management – Easier maintenance – Better performance – Allowed constraints:

NOT NULL CHECK

ALTER TABLE employees ADD CONSTRAINT salary_chk CHECK (salary BETWEEN 1000 AND 50000);

19. June, 2006



Indices: – They are like indices in books -> they speed up

lookups

– BUT: insert into a table with 3 indices is ~10 times slower than into table without indices

– It is important to find for each table a minimal set of indices

– It is usualy better to not create indexed on small tables (< 50 rows)

CREATE INDEX emp_dept_id_idx ON employees (dept_id);

19. June, 2006


Use Bind VariablesUse Bind Variables Place-holder variables in SQL statements that are

substituted with concrete values before statement execution e.g:

SELECT "schedulerJobId" FROM CMS_DASHBOARD.JOB WHERE "jobId"= 2001; SELECT "schedulerJobId" FROM CMS_DASHBOARD.JOB WHERE "jobId” = :job_id; Bind variables help:

– To reduce CPU consumption (less hard parses) 5000 executions of query above take either 2.4 or 18.1 s

depending on whether bind variable has been used or not. – To improve application scalability (less Oracle internal

locking) – To decrease DBAs’ anger – To improve code security

Bind variables protect from SQL injections. – To make code more readable and easier to maintain

Concatenated strings with hard-coded values are usually less readable then strings with bind variables

19. June, 2006


Connection management Connection management Opening a connection to a database is very

expensive from Oracle point of view: – New OS process has to be forked – Session context has to be created inside Oracle

instance – User has to be authenticated and authorized

Client-server applications: – Database session should be created at the

beginning of the client program execution and reused during its lifetime

– EXCEPTION: if client is idle for a long period of time it is better to close the session and free server-side resources

19. June, 2006


Recapitulation of General Recapitulation of General RecommendationsRecommendations

Use bind variablesUse primary keysDefine foreign keys (and add indices)Add indices to columns which are referred

to often in the WHERE clause of queriesUse constrains to enforce the data

integrityUse a normalized schemaFocus on representing the modularity of

the detector when creating the tables

19. June, 2006


Application of General Application of General Recommendations for FERORecommendations for FERO

Implement constrains on table columns– Use Primary keys PK on columns which contain the

version number– Use Foreign keys FK on the columns which reference

the primary keys.– Use “NOT NULL” keywords for columns in order to

have for every version the corresponding data– Use constrains with “CHECK” in order to have the

configuration data within reasonable values Since no deletes and no updates will be run-

– Consider using IOTs (Index Organized Tables) (CREATE TABLE… …ORGANIZATION INDEX…) to assure for future performance

19. June, 2006


Once the schema is ready…Once the schema is ready…

When populating the tables (e.g. from configuration files) you might use the /*+APPEND/ hint (example: insert /*+append / into mytable values (1, 255);)

You can let the version column increment itself by creating a sequence and a trigger– create or replace sequence myseq increment by 1 start

with 1 nomaxvalue nocycle;

– create or replace trigger mytrigger before insert on mytable for each row begin select myseq.nextval into :new.version from dual;end;/)

19. June, 2006


SPDSPD

Side A

Side C

x

y

z

19. June, 2006


SPDSPD

Sector

Half-stave

Half-stave

19. June, 2006


SPDSPD

19. June, 2006


Hierarchical organization of DCS Hierarchical organization of DCS SPD FERO configuration data SPD FERO configuration data

version 2version 21

SPDVersion PK

SIDAVer FK

SIDCVer FK

SIDVersion PK

SEC0Ver FK

SEC1Ver FK

SEC2Ver FK

…

…

SEC9Ver FK

SECA0Version

Halfstave 0

Halfstave 1

Halfstave 2

…

…

Halfstave 5

SIDVersion PK

SEC0Ver FK

SEC1Ver FK

SEC2Ver FK

…

…

SEC9Ver FK

SECA1Version

Halfstave 0

Halfstave 1

Halfstave 2

…

…

Halfstave 5

SECA2Version

Halfstave 0

Halfstave 1

Halfstave 2

…

…

Halfstave 5

…

Halfstave 0

Halfstave 1

Halfstave 2

…

…

Halfstave 5

…

Halfstave 0

Halfstave 1

Halfstave 2

…

…

Halfstave 5

SECA9Version

SECVersion PK

HAS0Ver FK

HAS1Ver FK

…

…

HAS5Ver FK

120

SPDVersion

SIDAVersion

SIDCVersion

HASVersion PK

MCMVer FK

MASVer FK

DACVer FK

HASA90Version

2 20

19. June, 2006


Hierarchical organization of DCS SPD Hierarchical organization of DCS SPD FERO configuration data version 2 FERO configuration data version 2

continuedcontinued

DACVersion

DACVect CHAR(440)


HASVersion PK

MCMVer FK

MBRVer FK

DACVer FK

HASA90Version

120

1


MBR Varchar(1640)

MBRVersion

1

Version 1:508 Tables

507 Primary Keys3862 Foreign Keys

MCMVersion

ACO Blob

APIVect Char(6)

DPIVect Char(8)


1

GOLVect Char(4)

Version 2:147 Tables


Redesign

19. June, 2006


Hierarchical organization of DCS Hierarchical organization of DCS SPD FERO configuration data SPD FERO configuration data

version 3version 31

SPDVersion PK

SIDE Number(1) PK

SECTOR Number(1) PK

Halfstave Number(1) PK

MCMVer FK

DACVer FK

MBRVer FK

SPDVersion

DACVersion

DACVect Char(440)


1


MBRVect Varchar2(2460)

MBRVersion

1

MCMVersion

ACO Blob

APIVect Char(6)

DPIVect Char(8)


1

GOLVect Char(4)

Version 3:4 Tables


19. June, 2006


Connecting and disconnecting from a Connecting and disconnecting from a Oracle database server using OCCI 10.2.0.1 Oracle database server using OCCI 10.2.0.1

and MSVC++ .NET 2003 7.1.3088and MSVC++ .NET 2003 7.1.3088 On a dual Xeon HT 2.8GHz, 3GB RAM, 2GB RAM for Oracle

10gR2, Win2003: 0.184625s (1000 loops) 0.1836 s (100 loops) 0.186 s (10 loops) 0.266 s (0 loops)

P4 HT 3.4GHz, 3GB RAM, 2GB RAM for Oracle 10gR2, WinXP: 0.158344s (1000 loops) 0.15422 s (100 loops) 0.164 s (10 loops) 0.25 s (0 loops)

P4 HT 3.0GHz, 1GB RAM, 0.7GB RAM for Oracle 10gR1, WinXP: 0.325016s (1000 loops) 0.32735 s (100 loops) 0.3531 s (10 loops) 0.391 s (0 loops)

19. June, 2006


Results of Oracle server Results of Oracle server teststests

Configuration Data download:– Blobs

– Big

– Small

19. June, 2006


Data Download from Oracle Data Download from Oracle Server (BLOBs)Server (BLOBs)

Large BLOBS were stored on the DB server connected to private DCS network (1Gbit/s server connection, 100Mbit/s client connection)

Retrieval rate of 150MB configuration BLOBs by 3 concurrent clients was measured to be:

3-11 MB/s/client

– Upper limit of 11MB/s corresponds to client network connection

– results depend on Oracle cache status, first retrieval is slower, succeeding access is faster

Depending on detector access patterns, the performance can be optimized by tuning the server’s cache

19. June, 2006


Test With Small BLOBsTest With Small BLOBs

Test records consists of 10 Blobs with size 10kB each

260 configuration records were retrieved per test

Allocation of BLOBs between configuration records altered– From random– To shared (certain BLOBs were re-used

between configuration records) -> to test the ORACLE caching mechanism

19. June, 2006


BLOB Retrieval TestsBLOB Retrieval Tests

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0 20 40 60 80 100

Fraction of repeated BLOBs [%]

Tra

nsf

er r

ate[

kB/s

]

1client 2 clients 3 clients 4 clients

19. June, 2006


Data Insertion Rate to the DB Data Insertion Rate to the DB ServerServer

Data was inserted to 2 column table (number(38), varchar2(128), no index)

– Following results were obtained (inserting 107 rows into DB):

– OCCI autocommit ~500 values/s– PL/SQL (bind variables) ~10000 values/s– PL/SQL (vararrays):

>73000 rows/s (1 client)>42000 rows/s/client (2 concurrent

clients)>25000 rows/s/client (3 concurrent

clients)

19. June, 2006


Data Download From TablesData Download From Tables

The FERO data of one SPD readout chip consists of– 44 DAC settings/chip

There are in total 1200 chips used in the SPD

– Mask matrix for 8192 pixels/chip– 15x120 front-end registers

The full SPD configuration can be loaded within ~3sec

19. June, 2006


Results of Oracle server tests Results of Oracle server tests (Data download)(Data download)

150MB of configuration data, 3 concurrent clients, DCS Private 100Mbit/s network, 1Gbit switch:

– Bfile ~10.59 MB/s– Blob ~10.40 MB/s– Blob, stream ~10.93 MB/s– Blob, ADO.NET ~10.10 MB/s

150MB of configuration data, 1 concurrent client, CERN 10Mbit/s network:

– Bfile ~0.81 MB/s– Blob ~0.78 MB/s– Blob, stream ~0.81 MB/s– Blob, ADO.NET ~0.77 MB/s

19. June, 2006


AMANDAAMANDA

AMANDA is a PVSS-II manager which uses the PVSS API to access the PVSS archives

Developed in collaboration with the offline team AMANDA is the only interface to the DCS archive Archive architecture (files/RDB) is transparent to

AMANDA AMANDA can be used as an interface between the

PVSS archive and non-PVSS clients AMANDA returns data for requested time period

19. June, 2006


AMANDA statusAMANDA status

AMANDA version 4.0 releasedTests look promising (stable and smooth

operation)Still being tested by offlineTool is ready for pre-installation

19. June, 2006


Operation of AMANDAOperation of AMANDA

19. June, 2006


AMANDA in distributed systemAMANDA in distributed system

The PVSS can directly access only file-based data archives stored by its own data manager

In a distributed system also data produced by other PVSS can be accessed, if the connection via DIST manager exists– In case of the file-based archival DM of the remote system is

always involved in the data transfer – In case of RDB archival, the DM can retrieve any data provided

by other PVSS within the same distributed system without bothering other DMs

It is foreseen, that each PVSS system will run its own AMANDA– There will be at least one AMANDA per detector– Using more AMANDA servers overcomes some API limitations

– some requests can be parallelized (clients need to know which sever has direct access to the requested data)

19. June, 2006


AMANDAAMANDAUser Interface

Manager

Driver DriverDriver

User InterfaceManager

User InterfaceManager

EventManager

APIManager

ControlManager

AMANDAServer

Archive(s)

AMANDAClient

PVSS-II

ArchiveManager(s)Archive

Manager(s)ArchiveManager

DatabaseManager

19. June, 2006


AMANDA in the distributed environment AMANDA in the distributed environment (archiving to files)(archiving to files)

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

DIS

DISDIS

19. June, 2006


AMANDA in the distributed AMANDA in the distributed environment (archiving to ORACLE)environment (archiving to ORACLE)

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

ORACLE

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

UI

DM

DRV DRVDRV

UI UI

EM

APICTR

AMD

PVSS-II

DIS

DISDIS

19. june, 2006 bologna alice week svetozár kapusta 1/28 pvss oracle archiving involved people in...

Documents

server slide

oracle rac server

pvss oracle

svetozr kapusta phait

week weekend tuning

gb ram slide

indexes pvss server

low client performance