Threshold Scan

• Inject various charges and histogram number of hits seen (occupancy plot) for each pixel

• Vcal (charge) from 0V to 200V (101 steps)• For each Vcal 100 events triggered• 32 mask steps• Testbed: ROD21 in SR1• 26 (4 PP0) modules connected to one ROD

• Mean and sigma of threshold by S-Curve fit

RCE Meeting 10/26/09 M. WittgenPage 1

S-Curve Log Likelihood Fit

RCE Meeting 10/26/09 M. WittgenPage 2

Number of electrons

Num

ber o

f hits

DSP Scan Engine

RCE Meeting 10/26/09 M. WittgenPage 3

Set Vcal

Hardware Limitation in ROD Dataflow• Acquire data from modules (40MBit/s link per module)• Trapping of events to SDSP• 5 MBit/s serial port to send commands to modules• Download histograms with SLAVE/MASTER/VME

RCE Meeting 10/26/09 M. WittgenPage 4

Leading Order Scan Steps

• Master• Setup Mask Step• Setup Parameter (Vcal for Example)• Trigger Loop (100 events)

• Slave• Processing Loop• Fitting

• Corresponding slave steps for Setup Mask Step + Setup Parameter don't take much processing time

RCE Meeting 10/26/09 M. WittgenPage 5

Threshold Scan Timing

• Data processing (histogramming) enabled• Trigger Loop MASTER: 305 s (101 x 32)• Setup Mask Stage: 29 s (32)• Setup Parameter: 142 s (32 x 101)• Time in Processing SLAVE: 374 s• Time in Fitting: 50 s

• No Data Processing on SLAVE - just counting words seen via DMA

• Trigger Loop: 170 s• Time in Processing 128 s• Time in Fit: 15 s (flatlined S-Curve)

RCE Meeting 10/26/09 M. WittgenPage 6

Scan Data Throughput• Data Processing slows scan down by about a factor of 2

• 170 s vs 305 s for trigger loop• Bytes seen on one SLAVE (7 modules):

• 6512 MByte / 170 s = 38 MByte/s• 38 MByte/s / 7 modules = 5 MByte/s per module• Consistent with a 40MBit/s stream + format overhead

• Amount of Data• 6.5 GByte: 50 hits x 1 word per hit x 101 Vcal steps x 46080

pixel x 7 modules• Occupancy histograms: 31 Mbyte = 101 Vcal Steps x 1 byte per

pixel x 46080 pixel• After fit: 3.7 Mbyte = 3 floats per pixel x 46080 pixels x 7 modules

RCE Meeting 10/25/09 M. WittgenPage 7

Serial Port Throughput

• Bottleneck: Creation of command bitstream + sending it to modules

• Lots of bit manipulation• Setup Mask Stage: 180 KBit/s out of possible 5 MBit/s

• Dominated by bitstream setup but only takes 30 s• Setup Parameter: 3.5 MBit/s out of 5 MBit/s

• Could be improved by using two serial ports?• Twice as fast? 142 s / 2: could take of one min of scan

time?

RCE Meeting 10/26/09 M. WittgenPage 8

Scan Time + Conclusion• Performance limit

• 3 mins triggering + processing + 2 mins module configuration • 5 mins theoretical limit for pure scan?• Additionally 1 min for fitting• Another limit is VME download

• From SLAVE to MASTER to SBC (last step through VME)• At the moment 1 MByte/s out of 4 MByte/s possible• Block transfer doesn't work (yet)

• Sequential download (one ROD after another) • For threshold scan data (4 MByte per SLAVE)

• 4 s x 4 slaves x 16 RODs per crate = 256 s• Additional PixLib (infrastructure) overhead

RCE Meeting 10/26/09 M. WittgenPage 9

DSP Code Characteristics• 28K of C code

• Completely stand-alone (compiles with g++, TI cross compiler)• 15 header files shared with PixLib• Module configuration, primitives

• Pseudo-OO model (for example SLAVE Object – for communication with SLAVE from MASTER)

RCE Meeting 10/26/09 M. WittgenPage 10

typedef struct { int structType, id, fProc; int (*write)(void *slave, UINT32 addr, UINT32 *buffer, int count); int (*read)(void *slave, UINT32 addr, UINT32 *buffer, int count); int (*init)(void *slave); void (*proc)(void *slave); void (*start)(void *slave); void (*put)(void *slave, UINT32 addr, UINT32 word); UINT32 *(*sendCommand)(void *commandChannel, int id, void *buffer, int nWords, int fetchReply); UINT32 (*get)(void *slave, UINT32 addr); UINT32 *hpic, *hpia, *hpid, *hpidi; Xface *xface; /* points to a SlaveXface structure */ CommandChannel *commandChannel; RequestChannel *requestChannel; int (*sendPrimitive)(void *slave, void *primBuffer, int size); TextBuffer *textBuffer[N_TEXTBUFFERS]; UINT32 histogramTask; /* albeit a UINT32 for passing around, it is a pointer on the slave */} Slave;

DSP Code Characteristics

• Most of the code can be reused• Easily “converted” to pure OO/C++

• Some code obsolete• Task structures and queue• Co-operative multitasking – only queuing feature really used

• Loop structure useful• Very flexible – allows fast implementation of new scans• DSPs operate independent (parallelization)• Trend to implement loops on DSP rather than PixLib

• Important Bottleneck: Communication to SLAVE DSP from SBC only through MASTER

• Shared “address space” for SLAVE/MASTER tasks would be plus

RCE Meeting 10/26/09 M. WittgenPage 11