d ata a cquisition b ackbone c ore dabc/dabc/dabc.php 27.09.07 j.adamczewski, h.g.essel, n.kurz,...

27.09.07J.Adamczewski, H.G.Essel, N.Kurz, S.Linev

1Data Acquisition Backbone Core http://www-linux.gsi.de/~dabc/dabc/DABC.php

Work supported by EU RP6 project JRA1 FutureDAQ RII3-CT-2004-506078

Data Acquisition Backbone CoreJ.Adamczewski, H.G.Essel, N.Kurz, S.LinevGSI, Experiment Electronics, Data Processing group

• Motivation• Data-flow engine, control• Event building network• Performance• To do



CBM data acquisition

data dispatcher

FEEdeliver time stamped

data

CNetcollect data into buffers

Detector Detector Detector

collect

~50000 FEE chips

event dispatcher

switching networkBNetsort time stamped data

~1000 links a 1 GB/sec

HNethigh level selection

to high level computing

and archiving~1 GB/sec Output

processing

PNetprocess events

level 1&2 selection

subfarm subfarm subfarm ~100 subfarms

~100 nodes per subfarm

~10 dispatchers → subfarm

~1000 collectors

~1000 active buffers

TNettime distribution

W.F.J.Müller, 2004



PCIe

FEFEFront end board: sampling ADCs, clock distribution

Active Buffer board *: PCI express card

PC

ABBPCIe

2.5 GBit/s bi-directional (optical) link: data, clock

Use case example: Frontend components test

The goal:

• Detector tests• FEE tests• Data flow tests

FE: Frontend board

ABB: Active Buffer board

* A.Kugel, G.Marcus, W.Gao, Mannheim University Informatics V



GE switch

PC

ABBPCIe

DCB

FE

IB switch

FE

DC

Front end board: sampling ADCs, clock distribution

Data combiner boards, clock distribution to FE

Active Buffer board: PCI express card

8-20 PCs dual/quad

PC

ABBPCIe

8

4

Scales up to 10k channels, 160 CPUs

~625 Mb/s bi-directional (optical) link: data, clock

~2.5 Gb/s data linksThe goal:• Investigate critical technology• Detector tests• Replace existing DAQ

FE: Frontend boardDC: Data combiner boardABB: Active Buffer boardGE: Gigabit EthernetIB: InfiniBandMBS: Multi Branch System

MBS

Use case example: middle-size setup



Driving forces and motivation for DABC

2004 → EU RP6 project JRA1 FutureDAQ*2004 → CBM FutureDAQ for FAIR2005 → FOPI DAQ upgrade (skipped)2007 → NUSTAR DAQ

* RII3-CT-2004-506078

1996 → MBS future 50 installations at GSI, 50 external http://daq.gsi.de

• Detector tests• FE equipment tests• Data transport• Time distribution• Switched event building• Software evaluation• MBS event builder• General purpose DAQ

Data Acquisition Backbone Core

Intermediatedemonstrator

Requirements• connect (nearly) any front-ends• handle triggered or self-trigger front-ends• process time stamped data streams• build events over fast networks• provide data flow control (to front-ends)• interfaces to plug in application codes• connect MBS readout or collector nodes• be controllable by several controls frameworks



DABC key components

• Data-flow engine– memory and buffers management– threads and events management– data processing modules with ports, parameters, timers– transport and device classes, file I/O– back pressure mechanism

• Slow control and configuration– components setup on each node– parameters monitoring/changing– commands execution– state machine logic– user interface



Data flow engine

DABC Module

port

port

DABC Module

port

port

process process

Local transport

A module processes data of one or several data streams.Data streams propagated through ports, which are connected by transports

Queue



Data flow engine

DABC Module

port

port

DABC Module

port

port

process process

Device

Net transport

Device

Net transport

Queue Queue

Network

A module processes data of one or several data streams.Data streams propagated through ports, which are connected by transports and devices



Memory management

Main features:• All memory to be used in modules for transport organized in memory pools• Memory pool consists of one or several blocks of memory, divided on equal

peaces - buffers• Each buffer in memory pool can be referenced once for writing and any times

for reading• Several references may be combined in gather list

Use for transport:• Only data from memory pools can be transported via ports• Each module port associated with only memory pool• Transport between two modules in same application done via pointer• Zero-copy network transport where supported (InfiniBand)• Support of gather lists for all kind of transports



Threads and event management

DABC Module A

processInputprocessCommand

Event manager

Thread with modulesDevice thread

Commands are synchronized with thedata flow also through the Event manager

Data ready event

A device, i.e. socket device controls several ports (transports).Once a queue buffer is filled, the transport signals the Event manager,which in turn calls the processInput function of the associated module.

Transport

QueueDABC Module B

Manager thread

CommandCommand

event



Back pressure mechanism

Basic idea:• sender allowed to send packets only after receiver confirms (with special

acknowledge message) that it has enough resources to receive these packets • Implemented on transport layer (not visible for user)

Impact on module code:• No additional efforts is required• Can be enabled/disabled for any port• To block connection - just do not read packets from it

Pro: easy method for traffic control in small system

Con: easy way to block complete network when single node is hanging



Class diagram as currently implemented

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .10. .1

0. .1

0. .10. .1

0. .1

0. .1 0. .1

0. .1

0. .10. .1

0. .1 0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .10. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. . *

0. . 1

0. .1

0. .1 0. .10. .1

0. .1

0. .10. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

0. .1

dabc::CommandsQueue

#fQ ueue: stl _cmd_queue*#fRepl yQ ueue:bool#fCmdsMutex:Mutex*

+CommandsQueue+~CommandsQ ueue+Push:voi d+Pop:Command*+Front:Command*+Si ze:unsi gned+Cl eanup:voi d

dabc::StandaloneCommandClient

#fCmdsMutex:Mutex*#fNumSumbCommands: i nt#fRepl yCond:Condi ti on*#fRepl yedCmds:CommandsQ ueue

+Standal oneCommandCl i ent+~Standal oneCommandCl i ent+Wai tCommands:bool#Wai tCommandRepl y:bool#PostCommandProcess:bool#ProcessRepl y:bool#AttachCmd:voi d#D ettachCmd:voi d#I sCl i entWorki ng:bool

dabc::CommandThrdConnect

+CommandThrdConnect

dabc::MemoryPool

#fBuffer:voi d *#fTotal Si ze:ui nt64_t#fNumBuffers:BufferI d_t#fBufferSi ze:BufferSi ze_t#fD efaul tO ffset:BufferSi ze_t#fUsage:ui nt32_t*#fSeekCnt:BufferI d_t#fNoFreeBuffers:bool#fPool Mutex:Mutex*#fO wnMutex:bool

+MemoryPool+~MemoryPool+Al l ocate:bool+UseMutex:voi d+Rel ease:voi d+G etTotal Buffer:voi d *+G etTotal Si ze:ui nt64_t+G etNumBuffers:BufferI d_t+G etBufferSi ze:BufferSi ze_t+G etBufferLocati on:voi d *+TakeBuffer:bool+Unl ockBuffer:bool+NewReference:bool+Rel easeBuffer:voi d+I sBufferUsed:bool+TakeMemoryBuffer:MemoryBuffer*+CreateSegmentedBuffer:SegmentedBuffer*+I sFreeBuffers:bool

EBuffersUsage

dabc::ModuleItem

#fModul e:Modul e*#fI temType: i nt#fI temI d: i nt#fPri ori ty: i nt

+Modul eI tem+~Modul eI tem+G etModul e:Modul e*+G etType: i nt+I temI d: i nt+G etPri ori ty: i nt#SetI temI d:voi d#SetPri ori ty:voi d#Fi reEvent:voi d#D oStart:voi d#D oStop:voi d

dabc::CommandDirectConnect

+CommandD i rectConnect

dabc::A ctionListner

+Acti onLi stner+~Acti onLi stner+ProcessActi on:voi d

enumdabc::EModuleEvents

+evntNone: i nt+evntI nput: i nt+evntO utput: i nt+evntPool : i nt+evntCommand: i nt+evntRepl y: i nt+evntTi meout: i nt

dabc::CommandPortConnect

+CommandPortConnect

dabc::ModulesThread

- fQ ueues:EventsI dQ ueue*-fCondi ti on:Condi ti on*-fNumQ ueues: i nt-fModul es:Poi ntersVector-fWorki ng:bool-fCheckTi mers:bool-fTi mers:Poi ntersVector-fTi me:Ti meSource*-fSysCommands:CommandsQ ueue*-fSysMutex:Mutex-fSysCondi ti on:Condi ti on-fExpl i ci tLoop:bool

+Modul esThread+~Modul esThread+AddModul e:voi d+RemoveModul e:voi d+StartModul e:voi d+StopModul e:voi d+SysCommand:voi d+Fi re:voi d+Wai t:EventI d+TestWorki ng:bool+ThrdTi meStamp:doubl e#Worki ngLoop:voi d#D oWakeup:bool#ProcessThreadEvent:voi d#ProcessThreadCommand:bool#D efineNextTi meout: l ong#Fi reModul eCommand:voi d#UpdateModul eD ependentLi sts:voi d

dabc::CommandA llModules

+CommandAl l Modul es

dabc::Timer

#fMutex:Mutex*#fTi meout:doubl e#fLastFi re:doubl e#fTi meoutFi red:bool#fCounter: l ong#fSi ngl eShoot:doubl e

+Ti mer+~Ti mer+G etTi meout:doubl e+D efineNextTi meout: l ong+G etCounter: l ong#Fi reTi meout:voi d#Si ngl eShoot:bool#Compl eteTi meout:bool#D oStart:voi d#D oStop:voi d

dabc::MemoryBuffer

#fI d:BufferI d_t#fBuffer:voi d *#fFul l Si ze:BufferSi ze_t#fO ffset:BufferSi ze_t#fD ataSi ze:BufferSi ze_t

#MemoryBuffer#~MemoryBuffer+G etI d:BufferI d_t+G etD ataLocati on:voi d *+G etD ataSi ze:BufferSi ze_t+G etO ffset:BufferSi ze_t+SetD ataSi ze:voi d+G etMaxD ataSi ze:BufferSi ze_t+MakeReadO nl y:bool+NewReference:MemoryBuffer*+CopyFrom:voi d+G etTotal Si ze:BufferSi ze_t

dabc::Folder

- fChi l ds:Poi ntersVector-fO wner:bool

+Fol der+~Fol der+AddChi l d:voi d+RemoveChi l d:voi d+D el eteAl l Chi l ds:voi d+NumChi l ds: i nt+G etChi l d:Basi c*+Fi ndChi l d:Basi c*+I sChi l d:bool+I sO wner:bool#G etFol der:Fol der*-Chi l dCreated:voi d-Chi l dD estroyed:voi d

dabc::Iterator

#fTop:Basi c*#fCurrent:Basi c*#fI ndexes:std: :vector<i nt>#fFol ders:std: :vector<dabc: : Fol der*>#fFul l Name:Stri ng

+I terator+~I terator+next:Basi c*+current:Basi c*+l evel : i nt+ful l name:const char *+name:const char *+Pri ntH i eararchy:voi d

dabc::CommandCl ient

+CommandCl i ent+~CommandCl i ent#ProcessRepl y:bool#AttachCmd:voi d#D ettachCmd:voi d

enumdabc::EModuleItemType

+mi tPort: i nt+mi tPool : i nt+mi tParam: i nt+mi tTi mer: i nt+mi tCmdQ ueue: i nt

dabc::CommandStartModule

+CommandStartModul e

dabc::CommandReceiver

+CommandRecei ver+~CommandRecei ver+Submi tCommand:bool

dabc::CommandCreateModule

+CommandCreateModul e

dabc::Condition

#fI nternCondMutex:Mutex#fCondMutex:Mutex*#fCond:pthread_cond_t#fFi redCounter: l ong i nt#fWai ti ng:bool

+Condi ti on+~Condi ti on+D oFi re:voi d+D oWai t:bool+Reset: voi d+_D oFi re:voi d+_D oWai t:bool+CondMutex:Mutex*

dabc::A ctionSource

- fLi stners:Poi ntersVector-fMutex:Mutex*

+Acti onSource+~Acti onSource+AddLi stner:voi d+RemoveLi stner: voi d+D oActi on:voi d

enumdabc::EModuleThreadEvents

+mthrtevntH al t: i nt+mthrtevntCmd: i nt+mthrtevntExcl : i nt+mthrtevntCl ean: i nt

dabc::VerbsDevice

#fI bPort:ui nt8_t#fContext: i bv_context *#fPD : i bv_pd *#fD evi ceAttr: i bv_devi ce_attr#fPortAttr: i bv_port_attr#f_mtu:ui nt16_t#fO sm:VerbsO sm*#fRecvThrd:VerbsRecvThread*#fConnThrd:VerbsConnThrd*#fVerbsMutex:Mutex#fChannel : i bv_comp_channel *#fMai nCQ :VerbsCQ *

+VerbsD evi ce+~VerbsD evi ce+I sMul ti cast:bool+I bPort: i nt+context+pd+l i d:ui nt16_t+mtu:ui nt16_t+G etG i dI ndex: i nt+CreateAH+Regi sterMul ti CastG roup:bool+UnRegi sterMul ti CastG roup:bool+CreateMAH+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool#O penVerbs:bool#Cl oseVerbs:bool#CreatePortQP:VerbsQ P*#CreateTransport:voi d

dabc::VerbsTransport

#fVerbs:VerbsD evi ce*#fQ P:VerbsQP*#f_mr: i bv_mr *#f_rwr: i bv_recv_wr *#f_swr: i bv_send_wr *#f_sge: i bv_sge *#fH eadersPool :VerbsMemoryPool *#fSegmPerO per: i nt

#_Submi tSend:voi d#_Submi tRecv:voi d#G etRecH eader:voi d *+VerbsTransport+~VerbsTransport+MaxSendSegments: i nt

std: :vector<NetworkTransport*>dabc::NetTransportVector

bnet::B netManagerPlugin

+BnetManagerPl ugi n+CreateModul e:dabc: :Modul e*+CreateSubeventsG eneratorWorker:SubeventsG eneratorWorker*+CreateSubeventsCombi nerWorker:SubeventsCombi nerWorker*+CreateSenderWorker:SenderWorker*+CreateEventBui l derWorker:EventBui l derWorker*+ReadoutBufferSi ze:ui nt64_t+ReadoutPool Name:const char *+TransportBufferSi ze:ui nt64_t+TransportPool Name:const char *+EventBufferSi ze:ui nt64_t+EventPool Name:const char *+Control BufferSi ze:ui nt64_t+Control Pool Name:const char *

bnet::EventBui lderModule

#fI npPool :dabc: : Pool Envel ope*#fO utPool :dabc: :Pool Envel ope*#fNumSenders: i nt#fWorker:EventBui l derWorker*

+EventBui l derModul e+NumSenders: i nt+ProcessEvent:voi d

bnet::EventBui lderWorker

+EventBui l derWorker+AddPacket: voi d+NumPackets: i nt+I sCompl eteEvent:bool+ProduceEvent:bool

bnet::EventFi lterModule

#fI npPool :dabc: : Pool Envel ope*

+EventFi l terModul e+ProcessI nputEvent:voi d

bnet::GlobalDFCModule

#fNumNodes: i nt

+G l obal D FCModul e+NumNodes: i nt

bnet::LocalDFCModule

#fNodeI d: i nt

+Local D FCModul e+NodeI d: i nt

bnet::ReceiverModule

#fNumSenders: i nt#fPool :dabc: :Pool Envel ope*#fRecvRate:dabc: :Ratemeter#fLoopCounter: i nt

+Recei verModul e+ProcessEvent:voi d+AfterModul eStop:voi d+NumSenders: i nt

bnet::SenderModule

#fNumRecei vers: i nt#fPool :dabc: :Pool Envel ope*#fWorker:SenderWorker*#fTranportBufferSi ze: i nt#fSendRate:dabc: :Ratemeter

+SenderModul e+ProcessEvent:voi d+NumRecei vers: i nt

bnet::SenderWorker

+SenderWorker+D efineTargetNode: i nt

bnet::SubeventCombinerModule

#fNumReadout: i nt#fI npPool :dabc: : Pool Envel ope*#fO utPool :dabc: :Pool Envel ope*#fWorker:SubeventsCombi nerWorker*

+SubeventCombi nerModul e+ProcessEvent:voi d+NumReadout: i nt

bnet::SubeventsCombinerWorker

+SubeventsCombi nerWorker+AddPacket: voi d+NumPackets: i nt+I sCompl eteSubevent:bool+ProduceSubevent:boolbnet::SubeventsGeneratorWorker

+SubeventsG eneratorWorker+G enerate:bool

dabc::CommandStopModule

+CommandStopModul e

std: :excepti ondabc::Exception

#fWhat:std: :stri ng

+Excepti on+Excepti on+~Excepti on+what: const char *

dabc::SegmentedBuffer

#fCapaci ty:unsi gned#fNumSegments:unsi gned#fSegments:Segment*

#Rel l ocate:voi d#SegmentedBuffer#~SegmentedBuffer+AddSegment:bool+AddSegment:bool+G etTotal Si ze:BufferSi ze_t+NumSegements:unsi gned+G etI d:BufferI d_t+G etD ataLocati on:voi d *+G etD ataSi ze:BufferSi ze_t+G etO ffset:BufferSi ze_t

Segment

dabc::TimeSyncModule

#fMasterConn:bool#fNumSl aves: i nt#fSyncTi mes:doubl e *#fPool Name:Stri ng#fTi meSource:Ti meSource*#fWorkBuf:MemoryBuffer*#fCurrCmd:Command*#fSl aveCnt: i nt#fPktCnt: i nt#fLoopTi me:Average#fMasterToSl ave:Average#fSl aveToMaster:Average#fSetShi f t: doubl e#fSetScal e:doubl e

#Fi l l MasterPacket:bool#Fi l l Sl avePacket:bool#ProcessFastRecv:bool#ExecuteCommand:bool#ProcessTi merEvent:voi d#BeforeModul eStart: voi d#AfterModul eStop:voi d+Ti meSyncModul e+~Ti meSyncModul e

dabc::CommandChannelModule

#fPool : Pool Envel ope*#fI sMaster:bool#fNumSl aves: i nt#fCmdO utQ ueue:CommandsQ ueue*

+CommandChannel Modul e+~CommandChannel Modul e+I sMaster:bool+NumSl aves: i nt+G etPool :MemoryPool *+ExecuteCommand:bool+ProcessI nputEvent:voi d+ProcessO utputEvent:voi d#SendSubmi ttedCommands:voi d

dabc::StandaloneManager

#fNodeI d: i nt#fNumNodes: i nt#fCmdChannel :CommandChannel Modul e*#fCmdThread:Modul esThread*

+Standal oneManager+~Standal oneManager+NodeI d: i nt+NumNodes: i nt+ConnectCmdChannel : voi d+Submi tRemoteCommand:bool+H al tAl l Nodes:voi d+ParameterEvent:voi d#D efineNodeI d: i nt#D efineNodeName:std: : stri ng#D efineNumNodes: i nt#Val i dNodeName:bool#CanSendCmdToManager:bool#SendRemoteCommand:bool#SendRemoteRepl y:bool#D oCommandRecv:voi d

dabc::TimeSource

#fCl ockShi f t:cycl es_t#fTi meScal e:doubl e#fTi meShi f t:doubl e

+Ti meSource+Ti meSource+operator=:Ti meSource&+~Ti meSource+G etTi meStamp:doubl e+Cal cTi meStamp:doubl e+SetCal i br: voi d+Adj ustTi meStamp:voi d+Adj ustCal i br: voi d

dabc::CpuStatistic

#system1:unsi gned l ong#system2:unsi gned l ong#user1:unsi gned l ong#user2:unsi gned l ong#i dl e1:unsi gned l ong#i dl e2:unsi gned l ong#fStatFp:FI LE*

+CpuStati sti c+~CpuStati sti c+Reset:bool+Measure:bool+CPUuti l : doubl e#G et:bool

dabc::PoolEnvelope

#fPool :MemoryPool *#fRequi redNumber:ui nt32_t#fRequi redSi ze:ui nt64_t

+Pool Envel ope+~Pool Envel ope+SetRequi rements:voi d+I ncrementRequi rements:voi d+G etRequi rements:bool+I sPool Assi gned:bool+Assi gnPool :voi d+D i sconnectPool :voi d+TakeBuffer:MemoryBuffer*+ProcessActi on:voi d

pool :MemoryPool *

dabc::ManagerPlugin

+ManagerPl ugi n+~ManagerPl ugi n+CreateModul e:Modul e*

dabc::ManagerThread

#fManager:Manager*#fCondi ti on:Condi ti on#fEvents:EventsI dQ ueue

+ManagerThread+~ManagerThread+Worki ngLoop:voi d+D oWakeup:bool+Fi reEvent:voi d

dabc::EventsIdQueue

#fQ ueue: stl _eventsi d_queue*

+EventsI dQ ueue+~EventsI dQueue+Push:voi d+Pop:EventI d+Si ze:unsi gned

dabc::Module

#fModul eThrd:Modul esThread*#fWorki ng: i nt#fI tems:Poi ntersVector#fI nputPorts:PortsI ndexVector*#fO utputPorts:PortsI ndexVector*#fPorts:PortsI ndexVector*#fCommands:Modul eCommandsQ ueue*#fRepl yes:Modul eCommandsQ ueue*#fO wnMai nLoop:bool#fModul eI d: i nt

#i ni t:voi d+Modul e+Modul e+~Modul e+Assi gnToThread:voi d+G etModul eThread:Modul esThread*+Start: voi d+Stop:voi d+Submi tCommand:bool+NumI nputs: i nt+NumO utputs: i nt+NumPorts: i nt+I nput:Port*+O utput:Port*+I O Port: Port*+Fi ndParam:Parameter*+SetParamStr:bool+SetParamD oubl e:bool+G etPool sFol der:Fol der*+G etPortsFol der:Fol der*+G etO bj Fol der: Fol der*+G etParsFol der: Fol der*+G etTi mersFol der:Fol der*+G etI tem:Modul eI tem*+TakeBuffer:MemoryBuffer*#I sWorki ng:bool#I sO wnMai nLoop:bool#SetO wnMai nLoop:voi d#Mai nLoop:voi d#TestWorki ng:bool#BeforeModul eStart: voi d#AfterModul eStop:voi d#ProcessEvent:voi d#ProcessI nputEvent:voi d#ProcessO utputEvent:voi d#ProcessPool Event:voi d#ProcessTi merEvent:voi d#ProcessUserEvent:voi d#ProcessFastRecv:bool#ExecuteCommand:bool#Repl yCommand:bool#Assi gnTransport:voi d#CreatePool :Pool Envel ope*#CreateI nput: i nt#CreateO utput: i nt#CreatePort: i nt#CreateParameter:Parameter*#CreateTi mer:Ti mer*#Fi ndTi mer:Ti mer*#ShootTi mer:bool#G etPortI tem:Port*#I temCreated:voi d#I temD estroyed:voi d#ProcessRepl y:bool#ThrdTi meStamp:doubl e#G etThrdTi meSource:Ti meSource*-Modul eMai nLoop:voi d-Fi reEvent:voi d-PreprocessEvent:bool-D oTransportAssi gn:voi d-D oStop:voi d-D oStart: voi d

dabc::ObjectsQueue

#fQ ueue: stl _obj s_queue*#fI sO wner:bool#fQ ueueMutex:Mutex*

+O bj ectsQ ueue+~O bj ectsQueue+Push:voi d+Pop:Basi c*+Si ze:unsi gned

T: cl ass

dabc::CleanupEnvelope

#fO bj :T*

+Cl eanupEnvel ope+~Cl eanupEnvel ope

dabc::B asic

- fParent:Basi c*-fName:Stri ng-gNumI nstances: l ong

+Basi c+~Basi c+G etParent:Basi c*+MakeFul l Name:voi d+G etFul l Name:Stri ng+G etName:const char *+I sName:bool+Fi l l I nfo:voi d+G etManager:Manager*+NumI nstances: l ong#SetName:voi d#D el eteThi s:voi d-Chi l dCreated:voi d-Chi l dD estroyed:voi d

dabc::Runnable

+Mai nLoop:voi d *+~Runnabl e

dabc::Thread

#fThrd:pthread_t

+Thread+~Thread+Start: voi d+J oi n:voi d+Ki l l :voi d+SetPri ori ty:voi d+I sI tsel f :bool dabc::RunnableThread

- fWorkStatus:EWorkStatus-fO wnMutex:Mutex-fStartCond:Condi ti on*-fSyncCond:Condi ti on*-fWai ti ngJ oi n:bool-fWorkMutex:Mutex*

-Mai nLoop:voi d *#Worki ngLoop:voi d#D oWakeup:bool#SetWorkMutex:voi d#G etWorkMutex:Mutex*#_I sWorki ng:bool#ConfirmWakeup:voi d+Runnabl eThread+~Runnabl eThread+I sWorki ng:bool+G o:bool+Stop:voi d+D oThreadSync:bool+Wai tForJ oi n:bool+SetThreadPri ori ty:voi d

EWorkStatus

dabc::PointersVector

#fVector: stl _voi dstar_vector*

+Poi ntersVector+~Poi ntersVector+push_back:voi d+pop_back:voi d *+i ndex_of : i nt+remove:bool+remove:bool+at:voi d *+si ze: i nt+operator[]:voi d * &+cl ear: voi d

dabc::Manager

#fWorki ng:bool#fCmdThread:ManagerThread*#fUseThreadForRepl y:bool#fCommandsQ ueue:CommandsQ ueue#fRepl yesQ ueue:CommandsQ ueue#fCl eanupQ ueue:O bj ectsQ ueue#fSendCmdsMutex:Mutex*#fSendCmdCounter: i nt#fSendCommands:Poi ntersVector

#Manager#~Manager#ExtractManagerName:const char *+SetUseThreadForRepl y:voi d+Wai tCmdThreadJ oi n:bool+H al tManager:voi d+G etPl ugi nFol der: Fol der*+G etD evi ceFol der:Fol der*+G etThreadsFol der:Fol der*+G etModul esFol der: Fol der*+G etPool sFol der:Fol der*+G etConnFol der: Fol der*+Fi ndModul e:Modul e*+Fi ndPort:Port*+Fi ndNonconnectedPort: Port*+Fi ndD evi ce:G eneri cD evi ce*+Fi ndPool :MemoryPool *+CreateMemoryPool s:voi d+Submi tCommand:bool+Submi tCommand:bool+Modul eExecpti on:voi d+Sampl ePacket:voi d+ParameterEvent:voi d+StartModul e:voi d+StartAl l Modul es:voi d+D el eteO bj ect: voi d+D el eteAnyObj ect:voi d+Cl eanupManager:voi d+D el eteAl l Modul es:voi d+G etThread:Modul esThread*#ProcessRepl y:bool#ExecuteCommand:bool#ExecuteSpeci al Command:bool#ExecuteManagerCommand:bool#CreateStandardModul e:Modul e*#PostCommandProcess:bool#ProcessEvent:voi d#I sCl i entWorki ng:bool#Val i dNodeName:bool#CanSendCmdToManager:bool#SendRemoteCommand:bool#SendRemoteRepl y:bool#RecvRemoteCommand:voi d#RecvRemoteRepl y:voi d

MgrEvents

enumdabc::EMemoryBufferType

+mbt_G eneri c: i nt+mbt_I nt64: i nt+mbt_TymeSync: i nt+mbt_AcknCounter: i nt+mbt_User: i nt

dabc::B uffer

#fPool :MemoryPool *#fReadO nl y:bool#fTypeI d:ui nt32_t#fUserI d:ui nt32_t#gNumI nstances: l ong

#Buffer#~Buffer#SetReadO nl yFl ag:voi d+G etPool :MemoryPool *+I sReadO nl y:bool+SetTypeI d:voi d+G etTypeI d:ui nt32_t+SetUserI d:voi d+G etUserI d:ui nt32_t+G etTotal Si ze:BufferSi ze_t+Rel ease:voi d+NumI nstances: l ong

dabc::SocketConnThread

#fMai n:SocketD evi ce*#fServerH ost:Stri ng#fServerPort: i nt#fServerSocket: i nt#fConnCounter: i nt#fMutex:Mutex#fConnRecs:Poi ntersVector

+SocketConnThread+~SocketConnThread+TryServerPort:bool+ServerH ost:const char *+ServerPort: i nt+Worki ngLoop:voi d+D oWakeup:bool+StartServerConnect:bool+StartCl i entConnect:bool#ServerWorki ngLoop:voi d#Cl i entWorki ngLoop:voi d#Cl eanupTi meout:bool

SocketConnRec

dabc::SocketRecvThread

#fMai n:SocketD evi ce*#fWakeupMutex:Mutex#fWakeupCounter: i nt#fWakeupPi pe: i nt [2]

#Worki ngLoop:voi d#D oWakeup:bool+SocketRecvThread+~SocketRecvThread

dabc::SocketTransport

#fSocket: i nt#fSocketRecvQ ueue:SocketRecvQ ueue*

#_Submi tSend:voi d#_Submi tRecv:voi d#RecvD ataFromSocket: i nt#SendD ataO verSocket:bool+SocketTransport+~SocketTransport+MaxSendSegments: i nt

dabc::SocketSendThread

#fQ ueue:EventsI dQ ueue#fCondi ti on:Condi ti on*

#Worki ngLoop:voi d#D oWakeup:bool#Fi reEvent:voi d+SocketSendThread+~SocketSendThread

dabc::IntParameter

#fVal ue: i nt

+I ntParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+SetI nt:bool+G etI nt: i nt+G etD oubl e:doubl e+SetD oubl e:bool

dabc::DoubleParameter

#fVal ue:doubl e

+D oubl eParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+SetD oubl e:bool+G etD oubl e:doubl e

dabc::Parameter

#fFi xed:bool

+Parameter+~Parameter+Ki nd:EParamKi nd+G etPtr: voi d *+I sFi xed:bool+SetFi xed:voi d+G etStr:bool+SetStr:bool+G etD oubl e:doubl e+SetD oubl e:bool+Fi l l I nfo:voi d#Changed:voi d#Rai seEvent:voi d

dabc::StrParameter

#fVal ue:Stri ng

+StrParameter+Ki nd:EParamKi nd+G etPtr: voi d *+G etStr:bool+SetStr:bool+G etD oubl e:doubl e+SetD oubl e:bool

enumdabc::EParamK ind

+parNone: i nt+parStri ng: i nt+parD oubl e: i nt+parI nt: i nt

dabc::LocalTransport

#fPai r: Local TransportPai r*#fO ther: Local Transport*#fRecvQ ueue:Local TransportRecvQ ueue*

#PortChanged:voi d+Local Transport+~Local Transport+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+MaxSendSegments: i nt

dabc::PCITransport

#fMutex:Mutex#fRecvQ ueue:PCI TransportRecvQ ueue*#fRecvQ ueueLength: i nt#fD MARecvQueue:PCI D MATransportRecvQ ueue*#fPool :MemoryPool *

#PushBuffer:bool#PushD MABuffer:bool#PortChanged:voi d+PCI Transport+~PCI Transport+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+MaxSendSegments: i nt+D i rectSend:voi d

std: :queue<ui nt32_t>dabc::SocketRecvQueue

dabc::AbbDevice

- fD MAengi ne:mprace: :D MAEngi neWG *-fCurrentBuffer:mprace: :D MABuffer*-fExpl i ci tPol l i ng:bool

+AbbD evi ce+~AbbD evi ce+Wri teBuffer:bool+ReadBuffer:bool+SetExpl i ci tPol l i ng:voi d

std: :queue<mprace: :D MABuffer*>dabc::PCIDMATransportRecvQueue

dabc::GenericTransport

#fPortMutex:Mutex#fPort: Port*#fD evi ce:G eneri cD evi ce*#gNumTransports: l ong

#G eneri cTransport#Fi reI nput:voi d#Fi reO utput:voi d#PortChanged:voi d#ProcessFastRecv:bool+~G eneri cTransport+Assi gnPort:voi d+D ettachPort: voi d+I sPortAssi gned:bool+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+MaxSendSegments: i nt+NumTransports: l ong

dabc::CommandDoTimeSync

+CommandD oTi meSync

dabc::CommandPCIBoardDeviceConnect

+CommandPCI BoardD evi ceConnect

dabc::Command

#fParams:CommandParametersLi st*#fCl i ent:CommandCl i ent*

+Command+~Command+SetPar:voi d+G etPar: const char *+RemovePar:voi d+SetStr:voi d+G etStr: const char *+SetBool : voi d+G etBool :bool+SetI nt:voi d+G etI nt: i nt+SetPtr:voi d+G etPtr: voi d *+SaveToStri ng:voi d+ReadFromStri ng:bool+AddVal uesFrom:voi d+SetResul t: voi d+G etResul t:bool+Cl earResul t:voi d+SetCl i ent: voi d+I sCl i ent:bool+D oRepl y:voi d

std: :queue<MemoryBuffer*>dabc::LocalTransportRecvQueue

dabc::LocalTransportPair

#fO wnMutex:Mutex#fMutex:Mutex*#fMemCopy:bool#fTr1:Local Transport*#fTr2:Local Transport*

#CheckCl eanup:voi d+Local TransportPai r+~Local TransportPai r+I sActi ve:bool

dabc::LocalDevice

#fCounter: l ong

+Local D evi ce+~Local D evi ce+Submi tCommand:bool+Cl eanupD evi ce:voi d+ConnectPorts:bool

dabc::GenericDevice

#G eneri cD evi ce+~G eneri cD evi ce+Cl eanupD evi ce:voi d

dabc::NetworkDevice

#fD evi ceMutex:Mutex#fVector:NetTransportVector*#fVectorChanged:bool

#NetworkD evi ce#~NetworkD evi ce#AddTransport: voi d#G etThrdsFol der:Fol der*#I ni ti ateCl eanup:voi d#_VectorSi ze:unsi gned#_VectorAt:NetworkTransport*#Cl oseNetD evi ce:voi d#D el eteTransports:voi d+Cl eanupD evi ce:voi d+G etTransport:NetworkTransport*+Submi tCommand:bool+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool

dabc::VerbsConnThrd

#fMai n:VerbsD evi ce*#fUseMul ti :bool#fMul ti LI D :ui nt16_t#fMul ti AH : i bv_ah *#fQ ueueLength: i nt#fBufferSi ze: i nt#fConnCQ :VerbsCQ *#fConnQ P:VerbsQ P*#fConnCounter: i nt#fConnPool :VerbsMemoryPool *#fH andshakePool :VerbsMemoryPool *#fMutex:Mutex#fConnRecs:Poi ntersVector

#Fi ndRec:VerbsConnRec*#Fi ndRecH S:VerbsConnRec*#ProcessH SPacketSend:voi d#ProcessUD PacketRecv:voi d#ProcessH SPacketRecv:voi d+VerbsConnThrd+~VerbsConnThrd+Worki ngLoop:voi d+D oWakeup:bool+D oServer:bool+D oCl i ent:bool

VerbsConnRec

dabc::VerbsMemoryPool

#fUD :bool#f_mr: i bv_mr *#f_rwr: i bv_recv_wr *#f_swr: i bv_send_wr *#f_sge: i bv_sge *

+VerbsMemoryPool+~VerbsMemoryPool+mr+G etRecvWR+G etSendWR+G etRecvWR+G etSendWR+G etSendBufferLocati on:voi d *

dabc::VerbsQP

#fQ PCounter:ui nt32_t#fVerbs:VerbsD evi ce*#fType: i bv_qp_type#f_qp: i bv_qp *#f_l ocal _psn:ui nt32_t#f_remote_l i d:ui nt16_t#f_remote_qpn:ui nt32_t#f_remote_psn:ui nt32_t#fNumSendSegs: i nt

+VerbsQ P+~VerbsQ P+qp+qp_type: i bv_qp_type+i s_ud:bool+qp_num:ui nt32_t+l ocal _psn:ui nt32_t+NumSendSegs: i nt+Connect:bool+I ni tUD :bool+remote_l i d:ui nt16_t+remote_qpn:ui nt32_t+remote_psn:ui nt32_t+Post_Send:bool+Post_Recv:bool+AttachMcast:bool+D etachMcast:bool

dabc::VerbsCQ

#f_cq: i bv_cq *#f_channel : i bv_comp_channel *#f_ownchannel :bool#f_wc: i bv_wc#f_srq: i bv_srq *#f_pi pe: i nt [2]

+VerbsCQ+~VerbsCQ+channel+cq+wc+srq+wr_i d:ui nt64_t+i mm_data:ui nt32_t+i mm_data_host:ui nt32_t+qp_num:ui nt32_t+src_qp:ui nt32_t+WasRecvO per:bool+CreateWakeUpPi pe:voi d+WakeUp:voi d+Pol l : i nt+Wai t: i nt+arg:ui nt64_t+StatusStr: const char *

dabc::VerbsRecvThread

#fChannel : i bv_comp_channel *#fCQ :VerbsCQ *#fLoopBackQ P:VerbsQ P*#fLoopBackPool :VerbsMemoryPool *#fPol l i ngCnt: i nt

+VerbsRecvThread+~VerbsRecvThread+Worki ngLoop:voi d+D oWakeup:bool#TestPol l i ng:bool#Wai tChannel Event:bool

structVerbsConnectData

+ki nd:ui nt64_t+node_LI D :ui nt64_t+node_Port:ui nt64_t+conn_Q PN:ui nt64_t+conn_PSN:ui nt64_t+tr_Q PN:ui nt64_t+tr_PSN:ui nt64_t+hand_Q PN:ui nt64_t+hand_PSN:ui nt64_t+ConnType:ui nt64_t+ConnI d:char [512]

dabc::NetworkTransport

#fTransportI d:ui nt32_t#fI sI nput:bool#fI sO utput:bool#fPool :MemoryPool *#fUseAckn:bool#fI nputQ ueueLength: i nt#fO utputQ ueueLength: i nt#fMutex:Mutex#fNumRecs:ui nt32_t#fRecsCounter:ui nt32_t#fRecs:NetI ORec*#fO utputQ ueueSi ze: i nt#fAcknAl l owedO per: i nt#fAcknSendQ ueue:NetI O RecsQ ueue*#fAcknSendBuf:MemoryBuffer*#fAcknSendBufBusy:bool#fI nputQ ueueSi ze: i nt#fI nputQ ueue:NetI O BuffersQ ueue*#fFi rstAckn:bool#fAcknReadyCounter: i nt#fErrorState:bool

#NetworkTransport#~NetworkTransport#I ni t:voi d#Cl ose:voi d#ErrorCl ose:voi d#PortChanged:voi d#I sActi ve:bool#_TakeRec:ui nt32_t#_Rel easeRec:voi d#Fi l l RecvQ ueue:voi d#CheckAcknReadyCounter:bool#_Submi tAl l owedSendO perati ons:voi d#_Submi tRecv:voi d#_Submi tSend:voi d#PackH eader:bool#UnpackH eader:bool+G etI d: i nt+ProcessSendCompl :voi d+ProcessRecvCompl :bool+Provi desI nput:bool+Provi desO utput:bool+Recv:voi d+RecvQ ueueSi ze: i nt+Send:voi d+SendQ ueueSi ze: i nt+D i rectSend:voi d+ProcessActi on:voi d

ENetworkO perTypes NetI O Rec

dabc::NetworkDeviceThread

#fD evi ce:NetworkD evi ce*

+NetworkD evi ceThread+~NetworkD evi ceThread

dabc::SocketDevice

#fSendThrd:SocketSendThread*#fRecvThrd:SocketRecvThread*#fServerThrd:SocketConnThread*#fCl i entThrd:SocketConnThread*

+SocketD evi ce+~SocketD evi ce+ServerConnect:bool+Cl i entConnect:bool+Fi l l ConnServerI d:bool#CreateTransport:voi d#Rebui l dUFD S:bool

dabc::Port

#fPool : Pool Envel ope*#fI nputQ ueueCapaci ty: i nt#fI nputPendi ng: i nt#fO utputQ ueueCapaci ty: i nt#fO utputPendi ng: i nt#fUseAcknol edges:bool#fTransport:G eneri cTransport*

#D ecO utputPendi ng:voi d#I ncI nputPendi ng:voi d#D i rectSend:bool#ProcessFastRecv:bool+Port+~Port+Pool :Pool Envel ope*+I nputQ ueueCapaci ty: i nt+O utputQ ueueCapaci ty: i nt+I sUseAcknol edges:bool+ChangeUseAcknol edges:voi d+NumO utputBuffersRequi red: i nt+NumI nputBuffersRequi red: i nt+Assi gnTransport:voi d+I sConnected:bool+D i sconnect:voi d+I sI nput:bool+I sO utput:bool+Fi reI nput:voi d+Fi reO utput:voi d+O utputQ ueueSi ze: i nt+O utputPendi ng: i nt+O utputBl ocked:bool+MaxSendSegments: i nt+Send:bool+Send:bool+I nputQ ueueSi ze: i nt+I nputPendi ng: i nt+I nputBl ocked:bool+Recv:bool+Recv:bool

dabc::PCIB oardDevice

#fCancel I O :bool#fEnabl eD MA:bool#fBoard:mprace: :Board*#fPool :MemoryPool *#fTransport:PCI Transport*#fThread:PCI Worki ngThread*#fReadBAR:unsi gned i nt#fReadAddress:unsi gned i nt#fReadLength:unsi gned i nt#fWri teBAR:unsi gned i nt#fWri teAddress:unsi gned i nt#fWri teLength:unsi gned i nt-fD MABuffers:std: : vector<mprace: :D MABuffer*>-fFreeBuffersQueue: std: :queue<mprace: :D MABuffer*>-fMutex:Mutex

+PCI BoardD evi ce+PCI BoardD evi ce+~PCI BoardD evi ce+Submi tCommand:bool+Cl eanupD evi ce:voi d+Wri teBuffer:bool+ReadBuffer:bool+ProcessI O :voi d+Bui l dI O Buffers:voi d+NextD MABuffer:mprace: :D MABuffer*+Rel easeD MABuffer:voi d+SetReadBuffer:voi d+SetWri teBuffer:voi d+I sCancel I O :bool+Cancel I O :voi d#ForwardBuffer:bool#ForwardBuffer:bool std: :queue<MemoryBuffer*>

dabc::PCITransportRecvQueue

dabc::PCIWorkingThread

#fD evi ce:PCI BoardD evi ce*

#Worki ngLoop:voi d#D oWakeup:bool+PCI Worki ngThread+~PCI Worki ngThread

std: :queue<MemoryBuffer*>dabc::NetIOBuffersQueue

std: :queue<ui nt32_t>dabc::NetIORecsQueue

Test2SendModule

#fPool :dabc: :Pool Envel ope*#fCanSend:bool#fSendRate:dabc: :Ratemeter#fPortCnt: i nt

+Test2SendModul e+ExecuteCommand:bool+ProcessO utputEvent:voi d+ProcessTi merEvent:voi d+AfterModul eStop:voi d

Test2RecvModule

#fPool :dabc: :Pool Envel ope*#fSl eepTi me: i nt#fRecvRate:dabc: :Ratemeter#fReal Sl eepTi me:dabc: :Average

+Test2RecvModul e+ExecuteCommand:bool+ProcessI nputEvent:voi d+AfterModul eStop:voi d

Test2WorkerModule

#fCounter: i nt

+Test2WorkerModul e+ExecuteCommand:bool+Mai nLoop:voi d

bnet::ReadoutModule

#fSubI d:SubD etector_t#fPool :dabc: :Pool Envel ope*#fWorker:SubeventsG eneratorWorker*#fEventCnt:ui nt64_t#fAbbI nput:bool#fAbbI ndependent:bool#fLastTm:doubl e#fMaxD i ff:doubl e#fAverD i ff:dabc: :Average#fAbbRate:dabc: :Ratemeter

+ReadoutModul e+BeforeModul eStart: voi d+AfterModul eStop:voi d+Mai nLoop:voi d+SubI d:SubD etector_t+ProcessO utputEvent:voi d+ProcessI nputEvent:voi d#G eneratePacket:bool

dabc::Average

#num: l ong#sum1:doubl e#sum2:doubl e#mi n:doubl e#max:doubl e

+Average+~Average+Reset:voi d+Fi l l :voi d+Number: l ong+Mean:doubl e+Max:doubl e+Mi n:doubl e+D ev:doubl e+Show:voi d

dabc::Ratemeter

#firstoper:doubl e#l astoper:doubl e#numoper: l ong#total packetsi ze: l ong#fMeasureI nterval :doubl e#fMeasurePoi nts: l ong#fPoi nts:doubl e *

+Ratemeter+~Ratemeter+D oMeasure:voi d+Packet:voi d+Reset:voi d+G etRate:doubl e+G etTotal Ti me:doubl e+AverPacketSi ze: i nt+G etNumO per: i nt+SaveI nFi l e:voi d+SaveRatesI nFi l e:voi d

Test2Plugin

+Test2Pl ugi n+CreateModul e:dabc: :Modul e*

D ABC 10-J ul -2007(Note: Mutex and Lockguardhi dden)

SocketInfiniBand

PCIboard

Port

Bnet

Module



Slow control

Tools for control

• Configuration via XML files• State machines, Infospace, message/error loggers, monitoring• Communication: Webserver, SOAP, DIM• Connectivity through DIM to: LabView, EPICS, Java, any DIM client/server• Java with NetBeans (Matisse GUI builder) maybe soon in Eclipse• Front-end controls?• Mix of cooperating control systems• First LabView and Java GUIs operable



Controls & monitoring communication

XDAQ Executive (process, address space)

Web server (SOAP)

DIM client:Java GUILabView GUIEPICS GUI

SOAP client:Java GUI

Infospace*

XDAQ Application

Modules, command queue

DABC data-flow

DIM server

State machine

Web browser

GRIDCC

* Infospace – remotely accessible parameters



LabView-DIM Control GUI*

Generic construction of fixed parameter table from DIM servers

* Dietrich Beck, EE



Java-DIM Control

Generic construction from commands and parameters offered by DABC DIM servers

Applications create the commands and parametersand publish

Ratemeter, trending, statistics



Generic Java DIM GUI controls



Event building network (BNet)

data dispatcher

FEEdeliver time stamped

data

CNetcollect data into buffers

Detector Detector Detector

collect

~50000 FEE chips

event dispatcher

switching networkBNetsort time stamped data

~1000 links a 1 GB/sec

HNethigh level selection

to high level computing

and archiving~1 GB/sec Output

processing

PNetprocess events

level 1&2 selection

subfarm subfarm subfarm ~100 subfarms

~100 nodes per subfarm

~10 dispatchers → subfarm

~1000 collectors

~1000 active buffers

TNettime distribution



collectingsortingtagging

buildingfilteringanalysis

collectingsorting tagging

buildingfilteringanalysis

IB

Linux

Linux

GE

analysisarchive

archive

Linux

GE: Gigabit EthernetIB: InfiniBand

frontendDataDispatcher

frontendother

frontendMBS readout Sender

Sender

BNet dataflow – bidirectional approach

DABC

Receiver

Receiver



plugin

input

BNet – modules view

plugin*

Readout

plugin

SubeventCombiner

DataReceiver

DataSender

Network NxM

plugin

EventBuilder

plugin*

EventFilter

M sender nodes

6 different modules

N receiver nodes

N outputs

M inputs

Ana

lisys

/ S

tora

ge

* optional



InfiniBand – testbench for BNet

InfiniBand - reliable low-latency zero-copy high-speed data transport

Aim: Prove of principal as event building network candidate for CBM

Tests last year:

• GSI cluster - 4 nodes, SDR

• Forschungszentrum Karlsruhe* (March 2007) – 23 nodes DDR

• UNI Mainz** (August 2007) - 110 nodes DDR

DDR – double data rate (up to 20 Gb/s), SDR – single data rate (10 Gb/s)

SDR (GSI) DDR (Mainz)

Unidirectional 0.98 GB/s 1.65 GB/s

Bidirectional 0.95 GB/s 1.3 GB/s

Point-to-point tests

Rate per node

Gb Ethernet ~100 MB/s

InfiniBand ~800 MB/s

BNet prototype (GSI, 4 Nodes)

* thanks to Frank Schmitz, Ivan Kondov and Project CampusGrid in FZK

** thanks to Klaus Merle and Markus Tacke at the Zentrum für Datenverarbeitung in Uni Mainz



Scaling of asynchronous traffic



Chaotic (async.) versus scheduled (sync.)



DABC further tasks

Achieved:• Control infrastructure

– setup, configure– communication – very first Java GUI

• Data flow engine– multi-threading– InfiniBand– Sockets (Gigabit Ethernet)– first PCIexpress board– good performance– back pressure

To do:• Data formats• Error handling/recovery• MBS event building• Time-stamped data• Final API definitions• Documentation

d ata a cquisition b ackbone c ore dabc/dabc/dabc.php 27.09.07 j.adamczewski, h.g.essel, n.kurz,...

Documents