INFN-FE, Angelo Cotta Ramusino 2013-10-14

GTK_DataReceiver module descriptionlast modified 2013-10-24

entity GTK_DataReceiver isgeneric ( HITCOUNT_MAX_INDEX : integer := 12 ); port ( rx_datain: IN STD_LOGIC_VECTOR(15 DOWNTO 0); rx_ctrldetect: IN STD_LOGIC_VECTOR(1 DOWNTO 0); rx_clkout: IN STD_LOGIC; rx_freqlocked: IN STD_LOGIC_VECTOR (0 DOWNTO 0); rx_pll_locked: IN STD_LOGIC_VECTOR (0 DOWNTO 0); reset: IN STD_LOGIC; par_rx_data_out: OUT STD_LOGIC_VECTOR(47 DOWNTO 0); FIFOwr_out: OUT STD_LOGIC; hit_count_out: OUT STD_LOGIC_VECTOR(11 DOWNTO 0); CRC_calc_on_rcvd_frame:OUT STD_LOGIC_VECTOR(15 DOWNTO 0); crcbad_from_trueCRCchkr: OUT STD_LOGIC; crcvalid_from_trueCRCchkr: OUT STD_LOGIC; o_total_frame_counter: out std_logic_vector (31 downto 0); o_error_frame_counter: out std_logic_vector (31 downto 0); o_good_frame_counter: out std_logic_vector (31 downto 0); o_error_flag: out std_logic);end GTK_DataReceiver;

Clock domain(s) : “rx_clkout” (160,32MHz)The operation of this module is synchronous to the “rx_clkout” clock signal originating from the XCVR module

(at the top level) in the synthesis-targeted design hierarchy or from the testbench in the simulation-only design).

Overview :The module receives the 16bit words coming from the 3.2Gbps de-serializer, formats them into 48bit words

according to the TDCpix format and provides the “FIFOwr_out” signal to store them into the derandomizer input FIFO instantiated at the higher level of the design hierarcy. The “GTK_DataReceiver” recognizes the framing words (TWO, IN THIS DEBUG DESIGN) and calculates the CRC on the received data until and including the first framing word; the calculated CRC is compared to the value of the CRC field of the second frame word. The “GTK_DataReceiver” flags good data frames by issuing the combination crcvalid_from_trueCRCchk=’1’ and crcbad_from_trueCRCchkr=’0’ for one clock cycle, while the combination crcvalid_from_trueCRCchk=’1’ and crcbad_from_trueCRCchkr=’1’ flags a good data frame. The “GTK_DataReceiver” also outputs diagnostic informations such as the CRC error flag, the total count of received frames, the count of frames with matching CRC and the count of frames with a mismatch between calculated and received CRCs.

Detailed description :The module instantiates a CRC generator for 16 bit words which calculates the CRC-16-CCITT value on the

incoming packet up to and including the first trailer word. The “pace_maker” state machine starts the CRC calculator when, after coming out of reset, it detects the first data (= non-K) code output by the deserializer. The “pace_maker” state machine is reset when the high speed serial receiver is not in “lock-to-data” mode OR when the GTK_DataReceiver module’s “reset” input is activated: in the debug implementation the “reset” is controlled by the “ExtraRxRst_from_button” signal at the top level.

Fig.1 screenshot from functional simulation of “sim_test_assembly_oct_slave_ta1” (exploiting an abstract model for the UNIPHY controller)

Please note that in the “test_assembly_oct_slave_ta1” module the 16-bit wide output data port of the “GTK_DataGenerator” is directly connected to the input data port of the “GTK_DataReceiver” via the testbench “test_assembly_oct_slave_ta1_tb”.The timing diagram above shows that the CRC generator’s operation is controlled by the “reset_n”, “datavalid”, “startofpacket” and “endofpacket”. The “datavalid” is high for all data words except for the second trailer. The “endofpacket” signal is true for one clock period at the end of the first trailer word received. The “startofpacket” signals. The “startofpacket” signal is set for one clock period with the onset of the “datavalid” after that an “endofpacket” has been generated the first frame in this test design DOES NOT GET a “startofpacket” signal but it correctly computes the CRC anyway. The CRC generator’s “reset_n” signal is activated at the beginning and after the reception of the second trailer word.

Please note that according to the TDCpix manual there no CRC is trasmitted with the frame word so some of the details described here are important only to understand this debug setup.

The CRC check determines whether the signals “derand_FIFO_wrenable_link_0” and “derand_FIFO_wrenable_link_1”, defined in module GTK_DataLinksToDDRBuffer (which instantiates the GTK_DataReceiver modules), are independently set (in case of a positive CRC check) or cleared (in case the calculated CRC does not match the received CRC):FIFO_chx_wrenable_condition <= GTK_crcvalid_from_trueCRCchkr_chx and not(GTK_crcbad_from_trueCRCchkr_chx);FIFO_chx_wrskip_condition <= GTK_crcvalid_from_trueCRCchkr_chx and GTK_crcbad_from_trueCRCchkr_chx ;The “..wrenable..” and “..wrskip..” signals are shaped to cross to the “frontend_clk” domain ( 240MHz) and become: “FIFO_chx_wrenable_condition_fend_clk” and “FIFO_chx_wrskip_condition_fend_clk” which are actually used to set and clear the “derand_FIFO_wrenable_link_x” signals. ( this feature was originally added (acr 2012-07-18) to skip writing the input derandomizer fifo until at least one packet with correct CRC has been received -> first good frame is lost in this test setup). as of 2013-10-18 the ..wrskip… does not clear the ..wrenable.. anymore; once ..wrenable.. is set it stays on: it regains the original meaning of stopping the storage engine until both channels see a good frame

“FIFO_chx_wrenable_condition_fend_clk” and “FIFO_chx_wrskip_condition_fend_clk” are one “frontend_clk” cycle wide.

The two independent “derand_FIFO_wrenable_link_x” are linked in the signal “derand_FIFOs_wrenable”:“derand_FIFOs_wrenable” <= “derand_FIFO_wrenable_link_0” and “derand_FIFO_wrenable_link_1”;

The signal “derand_FIFOs_wrenable” is actually used in the process “even_odd_frame_toggle_for_writing” where it is used to enable the toggling between ODD and EVEN as the input FIFO to be used as buffer for an incoming data frame; in this way the toggling is disabled until the first good ( i.e. CRC check passed) data frame is received.

the first trailer word is included in the CRC calculation

OCCHIO CHE SE NON C’E’ WRENABLE_CONDITION NON C’E’ TOGGLING TRA ODD ED EVEN NON SONO SICURO CHE SIA GIUSTO: BISOGNA CHE PER IL FRAME PERSO VENGA SCRITTA UNA RIGA VUOTA NELLA CACHE DPRAM E QUINDI NEL BUFFER DDR2!!!

ADESSO PROVO A LASCIAR COMMUTARE ODD/EVEN E VEDERE SE SI SCRIVE IL BUFFER VUOTO!!

OH RICORDA CHE HAI INIETTATO UN ERRORE ARTIFICIALE NEL CRC CHECK PER LA FRAME 3!!!!

nel modulo GTK_DataLinksToDDRBuffer c’e’ questo codice che non capisco bene (perche’ compllementare invece che settare?

if (end_of_raw_event_extr_flag_dlyd1 = '1') then -- acr 2013-03-10

DPRAM_0_valid_for_reading <= not DPRAM_0_valid_for_reading;

DPRAM_1_valid_for_reading <= DPRAM_0_valid_for_reading; -- acr 2012-08-08 adjusting the "valid_for_reading" flags

end if;

-- acr 2013-10-22 modified the clear condition for the "end_of_raw_event_extr_flag" to make it last just one "frontend_clk" period: anticipate the clear condition at state "nn_xfer_sm_cushion_state_0"

devi agiustare le cose anche per ack checksum receiver perche’ deve partire l’acknowledge anche quando il crc e’ bad

from module “na62_hpcii_driver_vhd”: cache_to_dram_wr_en_flag: process( DRAM_wr_sm_rst, frontend_clk ) --acr 2013-02-27 begin

if (rising_edge(frontend_clk)) then -- acr 2013-03-08 DPRAM_x_data_valid is already registered at the generating module, in the same clock domain (fronted_clk) DPRAM_data_valid <= DPRAM_0_data_valid OR DPRAM_1_data_valid;

cache_to_dram_write_enable <= cache_to_dram_wen and not( reading_from_DRAM );-- acr 2013-04-12 cache_to_dram_write_enable <= '1' when ((cache_to_dram_wen = '1') and (reading_from_DRAM = '0')) else '0';

if(DRAM_wr_sm_rst = '1') then cache_to_dram_wen <= '0';

else if (DPRAM_data_valid = '1') then

cache_to_dram_wen <= '1'; elsif (clr_cache_to_dram_write_enable = '1') then

cache_to_dram_wen <= '0'; end if;

end if;end if;

end process;

OCCHIO A QUESTO GLITCH

DEVI CONTROLLARE LA LOGICA DEL CHECKSUM RECEIVED

la logica sotto serve a generare il segnale aclr per gli input derandomizer fifosderand_FIFO_aclr_process:

process (frontend_clk, drandFIFO_clear_all) begin if (rising_edge(frontend_clk)) then if (drandFIFO_clear_all = '1') then

drandFIFO_ch0_even_rst_fendclk <= '1'; drandFIFO_ch0_odd_rst_fendclk <= '1'; drandFIFO_ch1_even_rst_fendclk <= '1'; drandFIFO_ch1_odd_rst_fendclk <= '1'; else

if ( (FIFO_ch0_wrskip_condition_fend_clk = '1') AND (ODD_nEVEN_frame_ch0_for_wr = '0') ) then drandFIFO_ch0_even_rst_fendclk <= '1';

else drandFIFO_ch0_even_rst_fendclk <= '0';

end if; if ( (FIFO_ch0_wrskip_condition_fend_clk = '1') AND (ODD_nEVEN_frame_ch0_for_wr = '1') ) then

drandFIFO_ch0_odd_rst_fendclk <= '1';else

drandFIFO_ch0_odd_rst_fendclk <= '0';end if;

if ( (FIFO_ch1_wrskip_condition_fend_clk = '1') AND (ODD_nEVEN_frame_ch0_for_wr = '0') ) then drandFIFO_ch1_even_rst_fendclk <= '1';

else drandFIFO_ch1_even_rst_fendclk <= '0';

end if; if ( (FIFO_ch1_wrskip_condition_fend_clk = '1') AND (ODD_nEVEN_frame_ch0_for_wr = '1') ) then

drandFIFO_ch1_odd_rst_fendclk <= '1';else

drandFIFO_ch1_odd_rst_fendclk <= '0';end if;

end if; end if;end process;

e infatti qui si vede che succede cosi

in questa immagine a fianco si vede che il derand_fifo attivo al momento dell’arrivo della bad-crc-frame era l’odd che viene resettato dal segnale sul suo “aclr”:

“derandfifo_ch0_odd_rst_fendclk”. la frame successiva viene bufferizzata sull’even derandfifo etc. quindi il toggling even-odd non si ferma e questo va bene.

adesso sarebbe bello riuscire a fare comunque scrivere la linea del buffer ddr2 ma solo con i terminatori!!! come faccio? pensavo che resettando il derandfifo, nel momento in cui partiva la costruzione della dram cache la dram cache veniva scritta con solo i terminatori.. vediamo... dalla state machine sotto si capisce che la scrittura della DPRAM parte quando si verifica la condizione (start_ch0_FIFO_rd_condition = '1')

when nn_xfer_sm_chk_FIFOs => end_of_raw_event_extr_flag <= '0';-- acr 2013-03-09 if (ch0_FIFO_data_valid = '1') AND (active_inFIFO_rdside_empty_ch0 = '0') and (xfer_to_DPRAM_cache_dne_ch0 = '0') then if (start_ch0_FIFO_rd_condition = '1') then -- (ch0_FIFO_data_valid = '1') AND (active_inFIFO_rdside_empty_ch0 = '0') and (xfer_to_DPRAM_cache_dne_ch0 = '0') then

reading_from_linkFIFO_ch0 <= '1'; reading_from_linkFIFO_ch1 <= '0';

nn_FIFO_rd_ch0 <= '1'; checksum_received_ack_ch0 <= '1'; FIFO_data_valid_ack_ch0 <= '1';

xfer_sm_state <= nn_read_FIFO_ch0_waitcycle;a questo punto vediamo chi e’ “start_ch0_FIFO_rd_condition”: start_ch0_FIFO_rd_condition <= ch0_FIFO_data_valid AND not(active_inFIFO_rdside_empty_ch0) and not(xfer_to_DPRAM_cache_dne_ch0);

a questo punto vediamo chi e’ “ch0_FIFO_data_valid”: xfer_to_dpram_start_controller:

process (frontend_clk, data_merger_reset) begin if ( rising_edge(frontend_clk) ) then if ( data_merger_reset = '1' ) then

ch0_FIFO_data_valid <= '0'; ch1_FIFO_data_valid <= '0'; FIFO_ch0_wrenable_condition_fend_clk_dlyd1 <= '0'; FIFO_ch0_wrenable_condition_fend_clk_dlyd2 <= '0'; FIFO_ch1_wrenable_condition_fend_clk_dlyd1 <= '0'; FIFO_ch1_wrenable_condition_fend_clk_dlyd2 <= '0';

else FIFO_ch0_wrenable_condition_fend_clk_dlyd1 <= FIFO_ch0_wrenable_condition_fend_clk; FIFO_ch0_wrenable_condition_fend_clk_dlyd2 <= FIFO_ch0_wrenable_condition_fend_clk_dlyd1; -- acr 2012-07-25 delay the

strobe to allow for the switching of the "active buffer" flag FIFO_ch1_wrenable_condition_fend_clk_dlyd1 <= FIFO_ch1_wrenable_condition_fend_clk; FIFO_ch1_wrenable_condition_fend_clk_dlyd2 <= FIFO_ch1_wrenable_condition_fend_clk_dlyd1; -- acr 2012-07-25

delay the strobe to allow for the switching of the "active buffer" flag if ( FIFO_ch0_rdenable_condition_fend_clk = '1' ) then

ch0_FIFO_data_valid <= '1';

...a questo punto vediamo chi e’ “FIFO_ch0_rdenable_condition_fend_clk”:FIFO_ch0_rdenable_condition_fend_clk <= FIFO_ch0_wrenable_condition_fend_clk_dlyd2 and not active_inFIFO_rdside_empty_ch0;

a questo punto vediamo chi e’ “FIFO_ch0_wrenable_condition_fend_clk_dlyd2”:registering_the_FIFO_ch0_wrenable_condition:

process (DL_rx_clkout(0)) begin if ( rising_edge(DL_rx_clkout(0)) ) then FIFO_ch0_wrenable_condition <= GTK_crcvalid_from_trueCRCchkr_ch0 and not(GTK_crcbad_from_trueCRCchkr_ch0); FIFO_ch0_wrskip_condition <= GTK_crcvalid_from_trueCRCchkr_ch0 and GTK_crcbad_from_trueCRCchkr_ch0 ; FIFO_ch0_paritytoggle_condition <= GTK_crcvalid_from_trueCRCchkr_ch0; -- acr 2013-10-21 end if;end process;

eccoci qui finalmente: e’ di nuovo il segnale che studiavamo all’inizio; quindi se non voglio bloccare del tutto la scrittura di una riga nella DDR2 corrispondente al bad-crc-frame posso ad esempio usare il segnale “FIFO_ch0_paritytoggle_condition” al posto di “FIFO_ch0_wrenable_condition” TANTO HO GIA’ RESETTATO IL FIFO E QUINDI QUANDO SI PROVERA’ A TRASFERIRE IL CONTENUTO DEL FIFO SVUOTATO DAL RESET DOVREBBE FINIRE CHE SI SCRIVERANNO NELLA DPRAM CACHE SOLO I TERMINATORI.fatta la modifica:-- acr 2013-10-22 changed the condition to start the transfer of the derandomizer FIFOs contents to the DPRAM cache: use the simple detection of the end of CRC calculation, -- regardless the outcome of the CRC match test

vediamo che succede:mi sembra niente… perche’?-- acr 2013-10-23 changed the condition to start the transfer of the derandomizer FIFOs contents to the DPRAM cache; changed also the "ODD_nEVEN_frame_chX_for_rd" toggling

Fig.2 screenshot from functional simulation of “sim_test_assembly_oct_slave_ta1” (exploiting an abstract model for the UNIPHY controller): start of input data frame buffering for channel 0

please note that in the case portraited above while the “derand_fifo_wrenable_link_0” gets set, the “derand_fifo_wrenable_link_1” was not set yet due to a mismatch between the received CRC and the calculated CRC caused by differences in the initialization of the ch0 and ch1 receivers’ CRC generators the “derand_FIFOs_wrenable” is not set and thus also the second frame for ch0 is not buffered.

Fig.3 screenshot from functional simulation of “sim_test_assembly_oct_slave_ta1” (exploiting an abstract model for the UNIPHY controller): detail on the onset of signal “derand_FIFO_wrenable_link_0” at the end of the first good frame received

note that these are in the “frontend_clk” domain (240MHz)

nota: bisogna capire perche’ il segnale di reset del crc generator del data_generator e’ un’onda periodicanota: che nella simulazione la prima frame e’ sbagliata per ch1 perche’ il suo CRC checker non riceve lo start_packet

ACR 2013-10-18 questa versione quindi se un pacchetto ha un bad crc non produce toggling dell’odd even e quindi fa si che il pacchetto dopo sovrascriva la stessa fifo (CHE PERO’ DEVE ESSERE RESETTATA DA QUALCOSA! PRIMA DELL’ARRIVO DEL FRAME SUCCESSIVO) QUESTO TEORICAMENTE (DA CONTROLLARE) POTREBBE CAUSARE CHE LA ddr PER LA RIGA CORRISPONDENTE A QUELLA FRAME NON VENGA AGGIORNATA E TIRI FUORI DATI VECCHI: DEVO AGGIUNGERE UN MARCATORE DI DATO VECCHIOoppure semplicemente si scrive tutto e sempre pero’ quando si trova un mismatch si scrive la timestamp in un pacchetto che viene eventualmente spedito nell’interspill e resettato a inizio spill

“GTK_DataGenerator” is connected to the input data port of the “GTK_DataReceiver” via the testbench “GTK_DataGenerator” is connected to the input data port of the “GTK_DataReceiver” via the testbench “test_assembly_oct_slave_ta1_tb”.The timing diagram above shows that the CRC generator is never reset: its operation is controlled by the “datavalid”, the “endofpacket” which is true for one clock (“rx_clkout” from the top level XCVR module) period at the end of the first trailer word received and the “startofpacket” which is true for one clock period. Note for me: would it be better to reset it? Immidiately after the reception of the trailer word is ok, since in this debug test bench after the first trailer word there is a second trailer word which is not CRCchecked. Please note that according to the TDCpix manual there no CRC is trasmitted with the frame word.

how the packets are built in the real TDCpix (from the “TDCpix manual” Feb 24th 2013): data word

how the packets are built in the GTKRO simulator TDCpix: data word

In the simulator each channel produces 320 hits per frame and thus the hit counter is 9 bit wide. The constant LOOPCOUNT_MAX_INDEX is 8-- Data format-hit word normal mode (48 bit) ------------------------------------------------------------------bits variable name47..40: tx_data_data2_int( 15 downto 8) <= X"00";39..32: tx_data_data2_int( 7 downto 0) <= std_logic_vector(tx_trlr0_HI_byte);

31.. 29: tx_data_data1_int(15 downto 13) <= ('0','0','0'); 28: tx_data_data1_int(12) <= '0'; 27.. 16: tx_data_data1_int(11 downto 0) <= fake_coarse_time(11 downto 0);

15.. 9: tx_data_data0_int(15 downto 9) <= ('0','0','0','0','0','0','0'); 8.. 0: tx_data_data0_int( 8 downto 0) <= hit_count(LOOPCOUNT_MAX_INDEX downto 0);

NOTE: bits 8-5 are analyzed by the “GTK_DataReceiver” module of the GTKRO FPGA firmware to bucket-order the incoming hit data into the DPRAM input buffer

how the packets are built in the real TDCpix (from the “TDCpix manual” Feb 24th 2013): frame word

how the packets are built in the GTKRO simulator TDCpix: frame word

In the simulator each channel produces, AFTER THE END OF A FRAME IS DETECTED, two frame wordsThe tokens for channel_0 of the test_assembly_ta1 are tx_trlr0_HI_byte = X"FE" and tx_trlr0_HI_byte = X"DE".The tokens for channel_1 of the test_assembly_ta1 are tx_trlr0_HI_byte = X"CE" and tx_trlr0_HI_byte = X"CA".

-- Frame word 1 format: (48 bit) ------------------------------------------------------------------bits variable name 47: tx_data_trlr5_int(15) <=’1’;46..41: tx_data_trlr5_int( 14 downto 9) <= b”000000”;40..37: tx_data_trlr5_int( 8 downto 5) <= std_logic_vector(tx_trlr0_HI_byte);36..32: tx_data_trlr5_int( 4 downto 0) <= hit_count(LOOPCOUNT_MAX_INDEX downto 4);

31.. 28: tx_data_trlr4_int(15 downto 12) <= hit_count( 3 downto 0); 27.. 16: tx_data_trlr4_int(11 downto 0) <= frame_count (27 downto 16);

15.. 0: tx_data_trlr3_int(15 downto 0) <= frame_count (15 downto 0);

-- Frame word 0 format: (48 bit) ------------------------------------------------------------------bits variable name 47: tx_data_trlr2_int(15) <=’1’;46..44: tx_data_trlr2_int( 14 downto 12) <= b”000”;43..32: tx_data_trlr2_int( 11 downto 0) <= X"a5a";

31.. 16: tx_data_trlr1_int(15 downto 12) <= checksum_from_CRCgen[15..0];

15.. 8: tx_data_trlr0_int(15 downto 8) <= std_logic_vector(tx_trlr0_HI_byte); 7.. 0: tx_data_trlr0_int( 7 downto 0) <= std_logic_vector(tx_trlr0_LO_byte);

Idle commandIf no data are transmitted the idle command is sent over the link. It is identicalto the link synchronisation word and consists of 5 K28.5 symbols followed by oneK27.7.Frame wordEvery time the coarse counter rolls over after 2048 cycles of clk dll one frame wordis sent at the next available transmission slot. Clk dll upon which the TDC coarsecounter and the frame counter is advanced can in principle be di_erent from clk sync.The time interval between two frames depends on clk dll. However, once a frameis due to be sent outside the chip from the clk dll domain, the transmission requestis ported into the clk sync domain and is executed at the next word transmissioncycle which has a period of 6 clk sync cycles. In nominal conditions (clk sync &clk dll = 320 MHz) the frame is sent out in average each (1/320 MHz * 2048 =)6.4 _s. However, as 2048 cycles cannot be divided 6 (number of 8bit10bit words in

each data word) the number of word transmission cycles between two frame wordstoggles between 341 and 342 (2 * 341 and 1 *342; 2 * 6.39375 _ s and 6.4125 _ s).The frame word is de_ned in table 2.2.Data wordThe data word is sent whenever a hit is available and no word with higher priorityneeds to be sent. The word transmission priority is de_ned as follows.1. send k sync slot requ,2. send testpattern requ,3. send k word requ,4. send serial time requ,5. send frame requ,6. send data requ.The data word is de_ned in table 2.3.Veri_cation: The veri_cation routines verify that the serial frequency recovery block,8b10b decoder, 6 character word aligner, hit/data/idle/sync word decoder is lockedwithin a time-out delay,_ sees a prede_ned number of synchronisations words (in simulation this numberis reduced from 65536 to 64 cycles._ sees only valid idle or sync sequences, data words or frame words and_ sees that the frame words arrive periodically within the required time intervall_ sees that the frame counter increases (rollover check not yet done)_ sees that the hit counter is correct (rollover check not yet done)Automatic routines using the pixel group _fo as input send:_ 10 hits at the same time to each of the column inputs,_ 2 sequences of 5 hits to 5 columns,_ 1000 hits to all columns in parallel with maximum speed of clksync/60. (maxread-out frequency)_ 1500 hits random arrival time with 3.3 *1.5 MHz per column (nominal + 50%)and random hit data._ Automatic sending of hits to each single pixel out of 450 using the coarsetime-unitVeri_cation of these sequences include:_ number of sent hits,_ pixgroup address,_ group collision counts,_ address hit arbiter,_ address pileup,_ lead coarse time selector,_ lead coarse time,_ lead _ne time,_ trail coarse time selector,_ trail coarse time,_ trail _ne time.