![Page 1: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/1.jpg)
FLARE Workshop November 4-6, 2004 1
Data AcquisitionData Acquisition
Mark Bowden, Margaret VotavaMark Bowden, Margaret Votava
![Page 2: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/2.jpg)
2FLARE Workshop November 4-6, 2004
ParametersParameters
250,000 channels
4 Bytes/chan
2 MHz sample rate
Front-end data rate = 2 TBytes/sec
![Page 3: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/3.jpg)
3FLARE Workshop November 4-6, 2004
Front-end (ICARUS)Front-end (ICARUS)
Caen V789 (Digital Board) + V791 (32 channel A/D Board) Rack 18 X 32 channels
![Page 4: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/4.jpg)
4FLARE Workshop November 4-6, 2004
FLARE Front-end AssumptionsFLARE Front-end Assumptions
• Newer components allow greater packaging density
128 or 256 channels (if new amplifier/shaper ASIC)
Digital board can be reduced to single FPGA andcombined with Analog board
FE Board count drops from 16,000 to 2,000
• Eliminate VME
VME is expensive and provides no added value
Direct Ethernet connections to FE boards
6U Eurocard or 1U (pizza box) packaging
![Page 5: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/5.jpg)
5FLARE Workshop November 4-6, 2004
FLARE Front-endFLARE Front-end
40 X 128 channels (6U packaging) 20 X 256 channels (1U packaging)
48V Power
![Page 6: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/6.jpg)
6FLARE Workshop November 4-6, 2004
Data RatesData Rates
1 MByte/sec
2 GByte/sec
1 KByte/sec
2 MByte/sec
1 MByte/sec
2 GByte/sec
1 GByte/sec
2 TByte/sec
Without FE Data Compression
With FE Data Compression
Spill Only*
Continuous
* 2 msec per 2 sec cycle ** assuming 128 channels/FEB
- per FEB**
- system total
Note: BTeV is 0.5 TByte/sec
![Page 7: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/7.jpg)
7FLARE Workshop November 4-6, 2004
Data NetworkData Network
…
FE switches (100) $15KGBE switches (50) $25KCables $10KTotal $50K
… … … … …
……
……
……
……
……
……
5 GByte/sec
![Page 8: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/8.jpg)
8FLARE Workshop November 4-6, 2004
Switch-based DAQSwitch-based DAQ
• Any size/bandwidth system can be built from low cost switch components.
• Cost scales as NlogN.
• Up to 100m (copper) between switches.
![Page 9: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/9.jpg)
9FLARE Workshop November 4-6, 2004
Processing & Data StorageProcessing & Data Storage
1U dual processor $2K
DVD writer $7K
(Delay purchase of computers to maximize performance/cost.)
![Page 10: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/10.jpg)
10FLARE Workshop November 4-6, 2004
““Event” ProcessingEvent” Processing
• An “Event” is an arbitrary time slice of data from all channels
2 msec 4 MBytes
20 msec 40 MBytes
200 msec 400 MBytes
2 sec 4 GBytes
• Choose largest time slice that fits in processor memory
• For this application, ~100 msec
![Page 11: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/11.jpg)
11FLARE Workshop November 4-6, 2004
““Event” ProcessingEvent” Processing
Front-end (2000 data sources)
Ethernet Network
Processors
1 2 2000
1 2 200
Source 1
Source 2
Source 3
1 2 3 200
1 2 3 200
1 2 3 200
1 2 3
1 2 3
1 2 3
Destination
![Page 12: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/12.jpg)
12FLARE Workshop November 4-6, 2004
““Event” ProcessingEvent” Processing
Front-end
Ethernet Network
Processors
1 2 2000
1 2 200
Data transfer time = ~40 MBytes/sec * time_slice [4 sec]
Processing time = Nprocessors * time_slice [20 sec]
![Page 13: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/13.jpg)
13FLARE Workshop November 4-6, 2004
Data StorageData Storage
Tape$300K/PByte
Hard Disk$800K/PByte
Blu Ray$900K/PByte
<$100K/PByte
DVD-R$100K/PByte
DVD-RW$300K/PByte
![Page 14: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/14.jpg)
14FLARE Workshop November 4-6, 2004
Data StorageData Storage
• continuous readout, without online tracking
- 2 GBytes/sec X 107 sec = 20 PBytes/yr
- $2-6M/yr media cost
• continuous readout, with online tracking
- 2 MBytes/sec X 107 sec = 20 TBytes/yr
- $10K/yr media cost
Note: BTeV includes 2 PBytes of hard disk (@ $600K/PByte) in baseDAQ construction cost. Offline storage is part of operating cost.
![Page 15: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/15.jpg)
15FLARE Workshop November 4-6, 2004
Ethernet Network(~$50K)
Processing Farm(~$500K)
Data Storage(~$50K @ 2MB/sec,~$1M @ 2GB/sec+ media cost)
DAQ CostsDAQ Costs
Note: DAQ based on all commercial hardware
![Page 16: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/16.jpg)
16FLARE Workshop November 4-6, 2004
Additional hardware costs
Slow Controls ~$200KTiming System ~$50KDevelopment System ~$100KInfrastructure ~$50K
DAQ CostsDAQ Costs
![Page 17: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/17.jpg)
17FLARE Workshop November 4-6, 2004
• with online tracking
DAQ hardware ~$1.0MDAQ software ~$2.0MDAQ management ~$0.5M
DAQ operating cost ~$100K/yr
• without online tracking
(fewer online processors, but more data recorders)
DAQ hardware ~$1.7MDAQ software ~$1.5MDAQ management ~$0.5M
DAQ operating cost ~$4M/yr
DAQ Cost SummaryDAQ Cost Summary
![Page 18: FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava](https://reader036.vdocument.in/reader036/viewer/2022062408/56649e365503460f94b24eb3/html5/thumbnails/18.jpg)
18FLARE Workshop November 4-6, 2004
• data compression at front-end is preferred(“necessary” for continuous readout)
• online tracking will significantly reduce operating cost by reducing storage media expense
• DAQ base cost estimate is ~$3.5M
• DAQ contingency and overhead estimate is ~$1M
ConclusionsConclusions