methodology for validation of a complex hardware/software ...€¦ · iii abstract the positron...
TRANSCRIPT
[Type text]
Methodology for Validation of a Complex
Hardware/Software System for PEM
Joel Fernando Morais da Costa Rego
Dissertation submitted for obtaining the degree of Master in
Engenharia Electrotécnica e de Computadores
(Electrical and Computer Engineering)
Jury
President: Prof. Doutor José António Beltran Gerald
Scientific Advisor: Prof. Doutora Isabel Maria Cacho Teixeira
Co-Supervisor: Pedro Miguel da Conceição Santos Lousã
Expert: Prof. Doutor João Paulo Cacho Teixeira
October 2008
[Type text]
i
Acknowledgements
I would like to thank my family who never stopped believing in me and supporting me
throughout all my personal and professional projects.
I would like to express my deep gratitude to Professor Isabel Cacho Teixeira for her valuable
assistance and encouragement throughout the dissertation.
I would like to thank Professor João Varela for the opportunity to participate in a project of this
dimension, for his availability and for all the help and clarifications provided to the understanding
of the problem under study.
I would like to leave a word of appreciation to all the colleagues who belong to the consortium.
Finally, I thank the INOV – INESC Inovação, the company I work for, to proportionate the
essential material conditions for the implementation of this project.
I’d like to express a special appreciation and gratitude to my boss Pedro Lousã, who always
believed in my capacities.
ii
iii
Abstract
The Positron Emission Mammography (PEM) prototype is intended to evaluate PET technology
principle in the diagnosis of malign neoplasm in the breast and of ganglion loco-regional
invasion. Relative to whole body PET systems, dedicated equipment has potentially better
spatial resolution, obtained with fine-grain crystal segmentation, and allows tighter coverage of
the region under analysis, leading to better sensitivity. The PEM system intends to detect breast
and armpit cancer with size at least of 2 mm, improving ten times the resolution of current PET
systems, as an essential factor for an early detection of this type of cancer.
Within PEM system, cancerous cells reacts with a radioactive substance (named radioactive
tag), which being injected on the patient, is spread all over the body by the blood flow. It is
known that this liquid, essentially composed by glucose, is taken in more quantity by tumor cells
than normal cells, due to their higher metabolism. In its natural decomposition, the liquid’s
radioactive isotope emits positrons (electron’s anti-particle) which quickly recombine with
electrons generating, among others, two photons in the same line and opposite directions.
These photons can be detected by specific crystals (collision) that scintillate when hit. The
presence of cancerous cells is detected by the intersection of those photons’ paths. PEM
system uses, as light to electrical signal transducer, Avalanche Photo Diodes (APD) coupled
with the crystals. The electrical signals are amplified, sampled and digitized using a Front-end
Electronics (FE) system. The FE system sends data to the Data Acquisition Electronics (DAE)
system, which is responsible to filter the relevant information and send it to the image
reconstruction computer, providing the final image for medical examination. The work in this
thesis lies on the DAE system, which is a very complex hardware/software system, capable of
process and filter a huge quantity of information in order to obtain an higher resolution image, at
least 10 times the resolution of actual whole body PET systems. PEM system was implemented
taking use of the FPGA (Field-Programmable Gate Array) technology, with high capacity and
speed. FPGAs are responsible for filtering data to obtain relevant information, data organization
and delivery, and system calibration. The work described in this dissertation involves:
conception and design of the DAE system hardware; methodologies, environment and
documentation for test and prototype validation of the DAE system; the procedures for
monitoring the DAE system; bidirectional Universal Serial Bus (USB) data link, which allows the
routing of data between the DAE and the image reconstruction computer.
Keywords
Positron Emission Tomography/Mammography, Data Acquisition System, Test Methodology,
Prototype Validation and Universal Serial Bus (USB).
iv
v
Resumo
Usando o princípio da tecnologia PET (Tomografia por emissão de Positrões – Positron
Emission Tomography) para detecção de cancro em exames de corpo inteiro, pretende-se
construir e validar um sistema, cuja aplicação é a detecção de cancro mamário, denominado
PEM (Mamografia por Emissão de Positrões – Positron Emission Mammography). O Sistema
PEM pretende detectar cancros mamários e nas axilas com dimensões tão reduzidas quanto 1
mm, melhorando em cerca de dez vezes a resolução actual dos sistemas PET, visto ser
essencial neste tipo de cancro a detecção precoce dos mesmos.
No sistema PEM, as células cancerígenas reagem com uma substância radioactiva injectada
no corpo humano e para cada reacção são emitidos dois fotões que se deslocam numa
trajectória rectilínea mas em sentidos opostos, colidindo em duas plataformas de cristais que
cintilam quando atingidos. A presença de células cancerígenas é detectada pela intersecção
das trajectórias dos fotões. O sistema PEM utiliza como transdutores de luz em sinal eléctrico,
foto díodos de avalanche (APD - Avalanche Photo Diodes) acoplados aos cristais. Os sinais
eléctricos são amplificados, amostrados e digitalizados usando um sistema electrónico
denominado de Sistema Electrónico de Linha da Frente (FE - Front-end Electronics).O sistema
FE envia os dados para processamento para um sistema electrónico externo de aquisição de
dados (DAE - Data Acquisition Electronics) que filtra a informação relevante da não relevante e
que organiza e envia a informação relevante para um computador que reconstrói a imagem de
origem, fornecendo a imagem final para exame médico. O trabalho que se apresenta nesta
dissertação recai sobre a electrónica de aquisição de dados (DAE), que é um sistema de
hardware/software de grande complexidade, pois a quantidade de informação que necessita de
ser filtrada é bastante elevada, de forma a obter uma resolução cerca de dez vezes superior à
dos actuais sistemas PET de corpo inteiro. Para implementar o sistema PEM usou-se a
tecnologia FPGA (Field Programmable Gate-Array) de grande capacidade e velocidade. As
FPGAs são responsáveis pela organização e encaminhamento de dados, filtragem para obter a
informação relevante e calibração do sistema. O trabalho descrito nesta dissertação envolve:
desenho e concepção do hardware do sistema DAE; metodologias, ambiente e documentação
de teste e de validação de protótipo do sistema DAE; procedimentos de monitorização do
sistema DAE; e ligação bidireccional de dados USB (Universal Serial Bus) que permite o
encaminhamento de dados entre o sistema DAE e o computador de reconstrução de imagem.
Palavras-Chave
PET, PEM, Sistema de Aquisição de Dados, Metodologia de Teste, Validação de protótipo e
USB.
vi
vii
Table of Contents
Acknowledgements ........................................................................................... i
Abstract ............................................................................................................ iii
Resumo ............................................................................................................. v
Table of Contents ........................................................................................... vii
List of Figures .................................................................................................. xi
List of Tables ................................................................................................. xiii
List of Acronyms and Abbreviations ............................................................ xv
1 Introduction ................................................................................................ 1
1.1 Objectives ................................................................................................................ 4
1.2 Relevant contributions ............................................................................................ 5
1.3 Outline of the dissertation ....................................................................................... 5
2 PEM system ................................................................................................ 7
2.1 Operating principle of the PET systems ................................................................. 7
2.2 PEM System Architecture...................................................................................... 10
2.3 Front-end Electronics ............................................................................................ 11
2.3.1 Scanner (Radiation Detector) Organization....................................................... 12
2.3.2 Signals and information characteristics ............................................................. 13
2.3.3 Information Qualification and Types of Events .................................................. 15
2.4 Data Acquisition Electronics ................................................................................. 16
2.4.1 DAE system requirements ................................................................................ 17
2.4.2 DAE system modules ....................................................................................... 17
2.4.2.1 Data qualification and filter ........................................................................... 19
2.4.2.2 Data organization and routing ....................................................................... 19
2.4.2.3 Functional DAE Scenarios ............................................................................ 19
3 DAE system hardware specification and design .................................. 21
3.1 DAE crate and Backplanes specification and design .......................................... 23
3.1.1 DAE Crate........................................................................................................ 23
3.1.2 DAE Compact PCI Backplanes ........................................................................ 25
3.2 Data Acquisition (DAQ) board hardware specification and design ..................... 25
viii
3.2.1 DAQ control block ............................................................................................ 26
3.2.2 DAQ acquisition block ...................................................................................... 27
3.2.3 DAQ processing block ...................................................................................... 28
3.2.4 DAQ interface block ......................................................................................... 30
3.2.5 DAQ board mechanics ..................................................................................... 31
3.3 Trigger and Data Concentration (TGR/DCC) board hardware specification and
design 34
3.3.1 TGR/DCC control block .................................................................................... 35
3.3.2 TGR/DCC block ............................................................................................... 35
3.3.3 TGR/DCC USB transmission block ................................................................... 35
3.3.4 TGR/DCC special signals generation block ...................................................... 36
3.3.4.1 Clock ............................................................................................................ 36
3.3.4.2 Reset ........................................................................................................... 36
3.3.4.3 Synchronization ............................................................................................ 37
3.3.5 TGR/DCC interface block ................................................................................. 37
3.3.6 TGR/DCC board mechanics ............................................................................. 38
4 Data interface between the DAE system and the
Acquisition/Reconstruction Computer ......................................................... 41
4.1 Bidirectional Universal Serial Bus (USB) interface .............................................. 42
4.2 Hardware design of the USB interface .................................................................. 42
4.3 Firmware design of the USB interface .................................................................. 46
4.3.1 USB interface protocol with the TGR/DCC FPGA ............................................. 46
4.3.2 USB interface protocol with the image reconstruction computer ........................ 50
4.3.2.1 FX2LP microprocessor Serial Interface Engine (SIE) .................................... 50
4.3.2.2 FX2LP microprocessor endpoint buffers configurations................................. 51
4.3.2.3 FX2LP microprocessor Enumeration and Re-numeration .............................. 53
4.4 Establishing a high performance data rate for the USB interface ....................... 53
4.5 USB link data rate transfer results ........................................................................ 55
5 Test methodology and demonstration results ...................................... 57
5.1 DAE system test methodology and procedures for prototype validation ........... 57
5.2 Prototype Validation .............................................................................................. 59
5.2.1 Electrical Tests ................................................................................................. 59
5.2.1.1 Power Supply ............................................................................................... 59
5.2.1.2 Special Signals: Clock, Synchronization and Reset....................................... 60
5.2.1.3 Transceivers ................................................................................................ 61
5.2.1.4 LVDS Receiver............................................................................................. 61
ix
5.2.1.5 FX2LP Microcontroller .................................................................................. 61
5.2.2 JTAG Boundary Scan Chain............................................................................. 61
5.2.3 LVDS Receiver and DAQ Sync module Functional Tests .................................. 64
5.2.4 Transceivers and Backplane Communication Functional Tests ......................... 65
5.3 Test procedure to validate the DAE FPGAs BIST ................................................. 66
5.4 DAE system monitoring procedures..................................................................... 68
5.5 DAE system test documentation........................................................................... 73
6 Conclusions ............................................................................................. 75
7 Future Work .............................................................................................. 77
Backplane ....................................................................................................................... 77
Backplane Protocol ......................................................................................................... 77
Optical link connecting the DAE system and the acquisition computer ............................. 78
Optical link connecting the DAE and the FE system ......................................................... 78
Boundary Scan test in all DAE circuits ............................................................................. 78
BIST report through a connection between the FPGAs and the FX2LP microprocessor ... 78
Annex 1 – DAQ Board Electrical Schematics .............................................. 79
Annex 2 – TGR/DCC Board Electrical Schematics ...................................... 95
Annex 3 – DAQ and TGR/DCC FPGA Pin out Distribution ........................ 105
References .................................................................................................... 109
x
xi
List of Figures
Figure 1.1 – PEM examination result with 2 (2 mm) tumors. ...................................................... 3
Figure 2.1 – PET system physical interface. .............................................................................. 7
Figure 2.2 – Operating principle of PET detector. ...................................................................... 8
Figure 2.3 – Detection of the origin of a gamma-ray emission by a PET detector. ...................... 8
Figure 2.4 – PEM detector head. ............................................................................................... 9
Figure 2.5 – Crystal associated with two APDs. ......................................................................... 9
Figure 2.6 – Less error of parallax using different values of the electrical energy generated in
each APD [20]. ................................................................................................................ 10
Figure 2.7 – PEM System Architecture. ................................................................................... 10
Figure 2.8 – Module of 32 crystals. .......................................................................................... 12
Figure 2.9 – Matrix of 32 APDs. ............................................................................................... 12
Figure 2.10 – PEM detector modules....................................................................................... 13
Figure 2.11 – ASIC pulse shape. ............................................................................................. 13
Figure 2.12 – Format of a sampled ASIC pulse. ...................................................................... 14
Figure 2.13 – ASIC sampled Pulse. ......................................................................................... 14
Figure 2.14 – Compton Effect. ................................................................................................. 15
Figure 2.15 – DAE system high level architecture [20]. ............................................................ 17
Figure 2.16 – DAE system modules and subsystems. ............................................................. 18
Figure 3.1 – DAE system modules. ......................................................................................... 21
Figure 3.2 – DAE Crate design. ............................................................................................... 23
Figure 3.3 – DAE Crate ventilation. ......................................................................................... 24
Figure 3.4 – DAE Crate picture. ............................................................................................... 24
Figure 3.5 – DAE Crate pictures during DAE system test. ........................................................ 24
Figure 3.6 – DAE cPCI Backplanes configuration. ................................................................... 25
Figure 3.7 – DAQ board block diagram. ................................................................................... 26
Figure 3.8 – DAQ board dimensions and layout. ...................................................................... 31
Figure 3.9 – DAQ board front panel layout. .............................................................................. 32
Figure 3.10 – DAQ board picture. ............................................................................................ 33
Figure 3.11 – DAQ board test setup. ....................................................................................... 33
Figure 3.12 – TGR/DCC block diagram ................................................................................... 34
Figure 3.13 – TGR/DCC board dimensions and layout. ........................................................... 38
Figure 3.14 – TGR/DCC board front panel layout. ................................................................... 39
Figure 3.15 – TGR/DCC board picture. .................................................................................... 40
Figure 3.16 – TGR/DCC board picture during test procedures. ................................................ 40
Figure 4.1 – Cypress FX2LP microprocessor logic block diagram. ........................................... 42
Figure 4.2 – Cypress FX2LP simplified block diagram for USB transmission. ........................... 43
Figure 4.3 – FX2LP FIFOs interfaces. ..................................................................................... 44
Figure 4.4 – TGR/DCC FX2LP microprocessor connections. ................................................... 45
xii
Figure 4.5 – FX2LP synchronous FIFO write interface pins...................................................... 46
Figure 4.6 – FX2LP synchronous FIFO writes timing diagram. ................................................. 46
Figure 4.7 – FX2LP synchronous FIFO read interface pins. ..................................................... 47
Figure 4.8 – FX2LP synchronous FIFO read timing diagram. ................................................... 47
Figure 4.9 – FX2LP Synchronous FIFO write and read state machine diagram. ....................... 49
Figure 4.10 – USB Serial Interface Engine (SIE)...................................................................... 50
Figure 4.11 – FX2LP microprocessor endpoint buffers configurations. ..................................... 52
Figure 4.12 – Software application to test the USB link performance. ...................................... 54
Figure 5.1 – Test procedures diagram. .................................................................................... 58
Figure 5.2 – DAQ board JTAG circuit. ..................................................................................... 62
Figure 5.3 – TGR/DCC board JTAG circuit. ............................................................................. 62
Figure 5.4 – Software detection of the DAQ board JTAG circuit. .............................................. 63
Figure 5.5 – Connections for the transceivers and DAQ Sync test. .......................................... 64
Figure 5.6 – Labview software application to control the DAQ Sync module test. ..................... 65
Figure 5.7 – Labview software application to gather BIST report. ............................................. 68
Figure 5.8 – DAE system I2C block diagram............................................................................ 69
Figure 5.9 – Electrical schematics of the DAQ board monitoring system. ................................. 70
Figure 5.10 – Electrical schematics of the TGR/DCC board monitoring system. ....................... 71
Figure 5.11 – DAE monitorization application software developed in Microsoft Visual Studio
C#.NET. .......................................................................................................................... 72
Figure 5.12 – DAE monitorization application software developed in Labview. ......................... 73
Figure 7.1 – DAQ FPGA’s coarse pin distribution. ................................................................. 105
Figure 7.2 – DAQ FPGA pin distribution. ............................................................................... 106
Figure 7.3 – TGR/DCC FPGA’s coarse pin distribution .......................................................... 107
Figure 7.4 – TGR/DCC FPGA pin distribution. ....................................................................... 108
xiii
List of Tables
Table 4.1 – USB transmission tests results .............................................................................. 55
Table 5.1 – Power supply test processes and scenarios .......................................................... 59
Table 5.2 – Processes and scenarios for the special signals test in the DAQ board ................. 60
Table 5.3 – Commands to collect DAE FPGAs BIST status. .................................................... 68
xiv
xv
List of Acronyms and Abbreviations
ACK – Acknowledge/Acknowledgement/Acknowledged
ADI – Agência de Inovação (Agency for Innovation)
ADC – Analog-to-Digital Converter
APD – Avalanche Photo Diode
API – Application Programming Interface
ASIC – Application-Specific Integrated Circuit
ATCA – Advanced Telecom Computing Architecture (also AdvancedTCA)
BIST – Built-In Self-Test
CAD – Computer-Aided Design
cPCI – Compact PCI
CRC – Cyclical Redundancy Checking
DAE – Data Acquisition Electronics
DAQ – Data Acquisition
DBUS – Dedicated Bus
DCC – Data Concentrator
DfD – Design for Debug
DoI – Depth of Interaction
EMI – Electromagnetic Interference
FDG – SF-18-Deoxyglucose or Fluorodeoxyglucose (radiopharmaceutical)
FE – Front-end Electronics
FIFO – First In, First Out
FLTR – Filter
FPGA – Field Programmable Gate-Array
GBUS – Generic Bus
I2C – Inter Integrated Circuit
IBEB – Instituto de Biofísica e Engenharia Biomédica (Institute of Biophysics and Biomedical
Engineering)
IBILI – Instituto Biomédico de Investigação da Luz e da Imagem (Institute for Biomedical
Research in Light and Image)
IP – Intellectual Property
INEGI – Instituto de Engenharia Mecânica e Gestão Industrial (Institute of Mechanical
Engineering and Industrial Management)
INESC – Instituto de Engenharia de Sistemas e Computadores (Institute of System and
Computer Engineering)
INESC-ID – INESC – Investigação e Desenvolvimento (INESC-Research and Development)
IO – Input/Output
JTAG – Joint Test Action Group
xvi
LIP – Laboratório de Instrumentação e Física Experimental de Partículas (Laboratory of
Instrumentation and Experimental Physics of Particles)
LVDS – Low Voltage Differential Signal
MMF – Multi Modal Fiber
NACK – Negative Acknowledgement
OP – Operating System
PC – Personal Computer
PCB – Printed Circuit Board
PCI – Peripheral Component Interconnect
PCI-E – Peripheral Component Interconnect Express
PCI-X – Peripheral Component Interconnect eXtended
PEM – Positron Emission Mammography
PET – Positron Emission Tomography
RAID – Redundant Array of Inexpensive Disks
RAM – Random-Access Memory
SFP – Small Form-factor Pluggable module
SIE – Serial Interface Engine
TGR – Trigger
VLDO – Very Low Drop Out (VLDO)
1
1 Introduction
Concerning developed countries, breast cancer is considered today as one of the most deadly
diseases among women [1] [2]. The treatment for this type of disease, as well as the means of
diagnosis associated with it, are generally quite expensive. However, statistics show that 85% of
cases of breast cancer are treatable and curable, if the disease is detected in its early stage.
So, it is very important to have resources available to carry out the diagnosis in the early stages
of the disease. In the developed countries, the diagnosis and treatment costs represent a
considerable fraction in the corresponding budgets of their social services support.
It is a proven fact that the sooner the tumor is diagnosed, the better is the possibility of cure and
the lower are the costs associated with their prevention and treatment, becoming less painful
and traumatic for the patient. Considering these social and economic reasons we are assisting
in the world, considerable and growing efforts to develop systems for diagnosis that allow the
detection of this disease in more incipient stages of their appearance.
Currently, the diagnosis of breast cancer has been mainly based on the use of X-ray systems
[1] like the screening mammogram.
However, medical literature and professionals believe that the sharpness of these tests does not
draw conclusions completely satisfactory, especially for situations where the size of the tumor is
still very small, since the confidence level of these results is only slightly over 50% in more
unfavourable cases of human tissue texture under observation [2]. Using X-ray methods of
diagnoses it is not possible to detect tumors smaller than 10 mm [2].
Consequently, intense research effort is pursued for reaching new diagnosis procedures and
new systems of detection for this type of cancer. The PEM (Positron Emission Mammography)
system [3]-[8] is one of these new solutions that works as an alternative and complement to the
X-ray tests. These systems use the principle behind the whole body technological PET (Positron
Emission Tomography) systems [9] [10], but reduce the scope of the observation field to specific
areas of the human body, in this case, the breast. It is hoped that with these new systems,
dedicated to specific parts of the human body, it is possible to improve the accuracy of the
diagnosis in comparison with the results provided by PET systems for the entire body.
Preliminary studies and the results of simulations [3]-[11] reflect that expectation. These
expectations result from the fact that the scanner [10] is closer to the body in the PEM system
(few centimeters) than in the system of whole body PET (tens of cm since the body of the
person has to fit inside the ring of the scanner). Simulation results show that the accuracy
improves if the scanner is closer to the body under examination [7] – [11].
In fact, the whole body PET systems have much less resolution of diagnostics than the 2 mm
resolution that is the target to be achieved in PEM system.
2
The work presented in this dissertation was conducted in the context of the PEM system, which
has been designed, developed, implemented and tested by the consortium Clear-PEM 1 [12],
and has been funded by the ADI (Agência de Inovação – Agency for Innovation).
Lesion detection in PET systems is based on the detection of the occurrence of collisions
between positrons through clusters of scintillant crystals. Those positrons are generated by the
collision of a radioactive substance injected into the human body, with electrons from organic
cells. In this collision gamma rays are emitted. The gamma rays interact with scintillant crystals
emitting light. By use of appropriate transducers, the emission of light is converted into electrical
pulses. The energy of these pulses is directly related to the brightness of the crystal.
The scanner is composed by 2 PEM detectors of scintillant crystals. Studies have been
performed in Crystal Clear Collaboration 2 [13] for the choice of crystals that adapt better to the
PEM system. Based on these studies, the crystal LYSO [14] was chosen, because it has the
appropriate scintillation performance and time constants. Each crystal is a prism of 2 cm of
height and 2x2 mm2 of base area. The crystal transverse dimensions (2x2 mm
2) are determined
by the desired position resolution, whereas the longitudinal dimension (20 cm) is dictated by the
required detector sensitivity. In each of each crystal bases is connected an Avalanche Photo
Diode (APD) 3 [15], that converts the light into electrical pulses.
The information associated with the electrical pulses supplied by APDs, is used to identify the
presence of any cancer cells.
Since only less than 5% of the radiation is from the breast and the gamma rays may come
either from cancer cells or from healthy cells, the majority of data generated in this process is
irrelevant (noise) to the medical diagnosis [11] and it will be necessary to have large amounts of
raw data to achieve the intended resolution.
In the PEM system, the information associated with the electrical pulses is processed and
filtered in order to identify relevant information. The relevant data is used later in the
reconstruction of the image that reveals the presence of cancer cells in the breast under
1 This consortium is formed by the following institutions: Tagusparque, Instrumentation and Laboratory of
Experimental Physics of Particulates (LIP), INESC Innovation (INOV), INESC Research and Development
(INESC-ID), Institute of Industrial Engineering and Management (INEGI), Institute of Biophysics and
Biomedical Engineering (IBEB) of the University of Lisbon, Institute of Biomedical Research in Light and
Image (IBILI) of Medicine University of Coimbra, Hospital Garcia da Orta (HGO) and recently PETSys
joined the consortium.
2 A collaboration CERN, Crystal Clear, which promotes the exchange between physical, producers and
users of crystals of different fields, to develop new scintillant crystals for high-energy physics, medical
imaging and other scientific and industrial applications.
3 These APDs are manufactured by Hamamatsu, in fact, the only global manufacturer of this type of
diodes.
3
observation. The program used in image reconstruction works with millions of points which is an
additional reason for the huge amount of data that is required.
Figure 1.1 shows a reconstructed image of a PET examination showing 2 tumors located in the
breast.
Figure 1.1 – PEM examination result with 2 (2 mm) tumors.
In later sections of this document the PEM system is described in some detail. It has a
significant component of dedicated electronic in order to acquire and process data in the
shortest time possible.
Besides the degree of resolution to be achieved, pretended to be as high as possible,
something that was considered important, given the nature of the examination, is that this
should be done in a period of time as low as possible [16]. This means, in terms of the system
requirements, that the data processing should be as efficient and as fast as possible; hence the
need to implement in hardware much of this processing.
The electronic component of the PEM system contains a significant part of analogue electronics
near the scanner crystals. This constitutes the Front-End Electronics (FE) [17] [18]. Another part
of the PEM electronics is responsible for digital data processing. It carries out qualification,
routing and filtering data for the subsequent reconstitution image. It is referred to as Data
Acquisition Electronics (DAE) and also as Off-Detector Electronics.
To achieve the necessary requirements of the PEM system, large amount of data and fast data
processing are the major requirements do build the DAE system. Since considerable research
has been carried out during the system development a flexible technology has been chosen for
implementation. Thus the DAE system has been implemented in Field Programmable Gate
Arrays (FPGA).
The organization, routing, communication of data among various components of the system and
between the system and the outside world is the subject of the dissertation referred to in [19].
The data classification and filtering to separate the relevant and the irrelevant information as
well as the structures of Built-In functional test is the subject of the dissertation referred to in
[20].
4
The aim of this dissertation is to describe the concept, design, and the methodology and
environment for validate the complex hardware/software of the DAE system. It includes also the
development of methodologies to identify and specify the requirements and features to be
validated and tested. This dissertation describes also the development of the test, validation
and monitor environment of the DAE system and the documentation for prototype validation.
1.1 Objectives
The main objectives of this dissertation are the following:
Specification and design of the DAE hardware system:
1. Specification of the DAE crate.
2. Design of the hardware of the Data Acquisition (DAQ) Board.
3. Design of the hardware of the Trigger and Data Concentration (TGR/DCC)
Board.
Establishment of a bidirectional Universal Serial Bus (USB) interface between the DAE system
and the Acquisition/Reconstruction Computer:
1. Hardware design of the USB interface.
2. Firmware design of the USB interface.
3. Establishment of a high performance data rate for the USB interface.
Development of the methodology and environment to validate the complex hardware/software of
the DAE system:
1. DAE system test procedures and documentation for prototype validation.
2. Test procedure to validate the DAE FPGAs Built-In Self Test based on
operating scenarios, described in [19] and [20].
3. DAE system monitoring procedures.
DAE system improvement guidelines:
1. Guidelines for DAE system design improvement.
2. Guidelines for DAE system test procedures improvement for product validation.
5
1.2 Relevant contributions
Within all the work done under this dissertation, the following issues are considered as the most
important contributions:
Development and design of a Data Acquisition System Hardware: it is a highly complex system,
which guarantees the PEM system performance requirements, specially the FPGAs
functionalities described in [19] and [20].
Development of the methodology and environment used to validate the complex
hardware/software DAE system, including test procedures, monitoring functions for prototype
validation, and procedures to help the debug and validation of the functionality implemented in
the FPGAs of the DAE boards, whose functionality are described in [19] and [20].
Hardware and firmware design to establish a bidirectional Universal Serial Bus (USB) interface
between the DAE system and the Acquisition/Reconstruction Computer has been implemented
in order to achieve a sustained data rate of approximately 45 MBps.
1.3 Outline of the dissertation
The dissertation is organized as follows: after a brief introduction (Chapter 1), describing the
framework and the motivation of this study, the Chapter 2 contains a more detailed description
of the PEM system currently under physical implementation and testing. This description avoids
the systems details that do not contribute to the definition of the context of the work that is the
subject of this dissertation.
Chapter 3 describes in more detail the DAE system hardware specification and design.
In Chapter 4, the implementation of the USB interface between the DAE system and the
Acquisition/Reconstruction Computer is described. It includes a description of the hardware and
firmware design. Chapter 4 also describes the strategy followed in order to obtain a high
performance data rate for the USB transmission.
Chapter 5 presents the test methodology and the environment for prototype validation of the
DAE system, including the monitoring procedures, the test procedures and the test validation for
the Built-In Self Test included in the DAE FPGAs firmware, and the test documentation.
Finally, chapter 6 presents the conclusions and Chapter 7 the future work including guidelines
for improvement the DAE system design and test procedures.
6
7
2 PEM system
The PEM system uses the physical principles underlying the PET technology. PEM is applied in
the detection of tumors in the breast and armpit and unlike traditional PET systems that seeks
for the entire human body. That fact allows the PEM scanner to be closer to the body (breast in
this case), providing more accurate diagnoses. This results in a better image resolution in a
shorter exam time, in comparison with the whole body PET systems.
Theoretical studies and simulations have been carried out by the Clear-PEM consortium [21]
[22] [23] [24] [25] in order to obtain models of the behaviour of the human body. These models
combined with innovative image reconstitution algorithms allows the detection of cancerous
cells in size equal or greater than 1-2 mm, in a satisfactory examination time. Figure 2.1 shows
the PEM system physical interface.
Figure 2.1 – PET system physical interface.
Main parts of the PEM system are the robot, the Electronic Data Acquisition system and the
Detector Heads.
2.1 Operating principle of the PET systems
PET systems identify the location of cancer cells by detecting gamma rays emitted by the
human body when injected with an appropriate radioactive substance. In PEM system this
substance is the FDG – 18F-fluoro-deoxy-glucose [2] [11] [12] [26]. This radioactive substance
is injected and it distributes itself throughout the body reacting with all body cells. However, only
2% of the injected substance lodges in the breast area, according to the literature in [2] [10]
[26]. Although all human cells can absorb this substance, cancer cells have higher metabolism,
which means they absorb a greater amount of glucose and consequently of the radioactive
substance than the healthy cells [26] [27]. After their synthesis, the radioactive substance
already begun to decay and release positrons in the process. When a positron (anti-matter)
8
collides with a human body electron (matter) both are annihilated emitting two gamma ray
(energy) photons, with 511 keV of energy, which move on a straight path but in opposite
directions. To detect the radiation (photons) the PEM system uses LYSO [28] [29] scintillant
crystals. To convert the light energy of the crystals into an electrical signal, PEM system uses
Avalanche Photo Diodes (APD).
Figure 2.2 graphically represents the principle of operation which has just been described. In
this figure, a pair of photons is detected by two of the sensors that are the basis of the operation
of PET detector. The pair crystal/APD act as sensor of radiation.
Figure 2.2 – Operating principle of PET detector.
Using a pair of sensors (crystal/APD) it is possible to reconstitute the route of the 2 photons
generated in a collision of a human body electron and a positron. Using 2 plans of sensors,
multiple paths can be detected through the interception of the trajectories of several pairs of
photons issued by the same cells, as shown in Figure 2.3.
Figure 2.3 – Detection of the origin of a gamma-ray emission by a PET detector.
The crystals convert a single high energy photon in many low energy photons. These low
energy photons are then converted into electrical signals by the avalanche photo diodes
(APDs).
9
The distance between the interaction point of the gamma ray/crystal and the photo diodes
determines the more or less number of photons captured by each APD and consequently, a
higher or lower energy associated with the electric signal generated by the corresponding APD.
The PEM system uses two parallel plans of crystals, each one with 3072 crystals. Each crystal
is coupled in both of the bases by an APD.
Figure 2.4 shows the two plans of crystal array with the FE-end Electronics (referred to as
detector head or detector module) and Figure 2.5 shows a set of a crystal associated with two
APDs and the FE electronics.
Figure 2.4 – PEM detector head.
LSOAPD APD
Electrical Signal
Electrical Signal
Figure 2.5 – Crystal associated with two APDs.
Using two APDs per crystal, the photons generated by the interaction between the gamma-ray
and the crystal are divided by the two APDs. Therefore, in each crystal, two electrical signals
are generated, one per APD. Each of these signals is pre-amplified, filtered and pre-processed
by the Front-end electronics (FE).
The different values of the electrical energy detected by each APD determine the depth of
interaction (DoI) of the original photon in the crystal.
Knowledge of the DoI, improves the resolution of the system by reducing the parallax error, as
illustrated in Figure 2.6.
10
Figure 2.6 – Less error of parallax using different values of the electrical energy generated in each APD [20].
2.2 PEM System Architecture
The overall architecture of the PEM system is shown in Figure 2.7. The main modules are
represented: plans of crystals matrix, Front-end electronics (FE), Data Acquisition electronics
(DAQ) and image reconstruction computer.
Figure 2.7 – PEM System Architecture.
The FE system is responsible for the conditioning of the electrical signal generated in APDs as
the result of the scintillation crystal when hit by a photon.
The most challenging aspect of the FE is the processing of the analog signal generated in the
APD. Indeed, in order to obtain a good resolution of the reconstruction image in a short period
of time, it has been concluded [28] that the signal generated by the APD should be amplified
and processed.
11
After the FE builds the signal with the correct shape, it samples that signal and identifies the
signals with greater energy. Once identified these signals and the correspondent crystals, that
information is sent to the Data Acquisition (DAQ) system, responsible for the acquisition and
processing of digital signals received from the FE. An FE module consists of APDs, ASICs
(responsible for conditioning and amplification of signals generated by APDs), ADCs (to convert
the analog signals provided by the ASIC to digital signals) and serialization data transmission
circuits that are responsible for sending digitized data to the DAQ system. More detailed
information of the FE system is described in the next section. More details of the FE conception
and design can be found in [18].
The DAQ processing includes data classification and filtering. Only relevant data is then sent to
the image reconstruction computer. A detail description of the DAE architecture is provided in
section 2.4. Image reconstruction is performed by specific software that generates 2 and 3
dimensions images [29] [30] [31] [32].
2.3 Front-end Electronics
As mentioned before, the Front-end Electronics (FE) carries out the first process of filtering
information, which significantly contributes to increase the efficiency of the PEM system. Indeed,
when the human cells absorb the radioactive solution, all that cells can emit gamma rays. The
most part of this information is captured by the detector, but not all the gamma ray/crystal
reactions release significant energy, at least, above certain minimum values.
To identify what levels of energy should be used as threshold to identify relevant reactions,
extensive theoretical studies have been performed about the phenomena involved in the
reactions between gamma-ray and crystal. Energy levels resulting from a gamma-ray/crystal
reaction that are below threshold convey no relevant data and therefore should be considered
as noise and discarded.
Theoretical studies [28] results showed that for an observation time interval of 100 ns (referred
as observation window), the probability of three or more interactions occur in the set of 192
crystals is about 5%.
This value makes legitimate to consider that during an observation window of 10 ns, it does not
make sense to expect more than two gamma-ray/crystal interactions for each set of 192
crystals.
The FE identifies for an observation window of 10 ns, the two higher level signals out from each
set of 192 signals. These signals are digitized, serialized and finally sent to the DAE system.
The serialization is intended to significantly reduce the number of electrical cables between the
FE and the DAE systems, improving data transfer efficiency.
To achieve a good image reconstruction resolution, large amounts of data are required. Large
volume of data can be achieved at the cost of greater volumes of injected substance into the
patient, which is undesirable, or at the cost of increasing the time of observation, which is also
12
undesirable. It is desirable to have a system that detects and processes the data generated
from the collisions, with the maximum efficiency in the shortest period of time. This is the
solution that PEM system intends to implement.
The DAE processes all data received from the FE system and selects potentially relevant data
to be stored and processed by the reconstruction computer. In this context, relevant data is
defined as data that can be used by image reconstruction software, i.e., that can be used to
calculate the spatial origin of the gamma-ray photons pair.
2.3.1 Scanner (Radiation Detector) Organization
The scanner (radiation detector) is a critical element in the PET system. As explained in section
2.1, the crystals are grouped in sets of 192 crystals, referred to as super modules. Each super
module consists of 32 crystals modules as shown in Figure 2.8. Each plan of 3072 crystal
consists of 16 super modules and the PEM scanner has 2 crystal plans.
Figure 2.8 – Module of 32 crystals.
The ADPs are grouped into matrixes of 32 APDs that match the distribution of crystals in
modules. There are 6144 (3072x2) APDs per plan. Figure 2.9 shows a matrix of 32 APDs.
Figure 2.9 – Matrix of 32 APDs.
Figure 2.10 shows a picture of a set of detector module crystals.
13
Figure 2.10 – PEM detector modules.
Each super module of crystals is combined with 2 FE integrated circuits, associated with the
base and the top of the super module. The chip used is an ASIC (Application-Specific
Integrated Circuit) that was developed for this purpose [18], constituting an important task of the
PEM project. In each crystal plan there are 32 ASICs. Each integrated circuit provides 2 signals
to the DAE, thus, there are 128 signals available and therefore 128 channels are required to
establish data communication between the FE and DAE systems.
2.3.2 Signals and information characteristics
In the FE system, the ASICs carrying out an initial processing on the electrical pulses generated
on APDs in order to obtain pulses with the general shape represented in Figure 2.11.
Figure 2.11 – ASIC pulse shape.
According to theoretical studies, the ASIC should generate a signal with a rise time of 30 ns, as
shown in Figure 2.11. These requirements result from the fact that the working clock frequency
is 50 MHz and that catalogue algorithms demands that the highest value should be the third
sample [28].
These pulses are then sampled, organized into packages and sent to the DAE system. Each
package consists of 10 samples of 10 bits plus a digital identification word of 11 bits.
14
Figure 2.12 shows the format of a sampled pulse generated in the ASIC.
Figure 2.12 – Format of a sampled ASIC pulse.
Each identification package (11 bits) consists of 1 start bit, 8 bits for APD identification, 1 error
bit and 1 stop bit error. The identification is sent to the DAE system in serial. Each crystal has 2
APD (with the same spatial identification), thus, for each crystal, 2 packages with the format
which has just been described are sent to the DAE.
In addition to the relevant samples, the ASIC should also send two pre-samples, known as
pedestal. The pedestal corresponds to the power attributed to background noise. Thus, the
complete information sent by FE is constituted by 2 pre-samples and 8 samples of the signal.
In order to consider a data package relevant or useful, it is necessary that the pulses are
adjacent in time. If this condition cannot be verified it is impossible to evaluate the instant of the
interaction and hence the corresponding data is discarded. If the data is useful, a signature is
attached and used in further processing. That signature is composed by the time mark, the
crystal spatial identification and the associated signal energy quality.
The time mark is the time instant corresponding to the signal peak. It is composed of 9 bit
identification for clock cycle (called "time tag") and an identification of 8 bit for the clock cycle
subdivisions (called "delta") as shown in Figure 2.13.
Figure 2.13 – ASIC sampled Pulse.
The spatial identification is the crystal (or the 2 associated APDs) unique identification.
15
The intensity of the electrical signal generated by an APD, and hence the associated energy,
depends on the light generated within the crystal when the reaction between the crystal and the
gamma-ray photon occurs.
However, it may happen that a gamma-ray does not shed all its energy in a single crystal but in
several, as illustrated in Figure 2.14. This power share is called Compton Effect. Clearly, the
energy associated with an electrical signal generated in the APDs of the 2 crystals that share
the photon energy can be quite different.
Figure 2.14 – Compton Effect.
In this dissertation, the interaction between a crystal and a gamma-ray photon is referred as hit
(collision). The set of hits associated with the collision of pair positron/electron is referred as an
event. The PET technology is based on the identification and characterization of those events.
The energy associated with each hit is not necessarily the same in all circumstances. The hit
associated with a higher energy interaction is referred as Photo-event(s) and the others as
Compton(s).
To distinguish between Photo-events and Comptons, specific energy values, referred as
threshold values, are used. These threshold values are also used to establish the distinction
between relevant hits and discard hits. There are two values of threshold voltage, namely, the
Photo-event threshold and the Compton threshold. A hit resulting in a power below the
threshold of a Compton is considered noise.
Photo-events and Comptons can be associated with the same gamma-ray, if they occur within a
predetermined period of time.
The quality of signal energy is measured in three levels: poor (noise), weak (Compton) and
good (Photo-event). The noise packets are immediately rejected, but the Compton and Photo-
event packets continue to the next processing block.
2.3.3 Information Qualification and Types of Events
In the PEM system 3 types of events are considered, namely, Normal, Random and Single.
A normal event is associated with photons that collide with crystals in the two plans.
A single event occurs if only one single photon interacts with a crystal in one plan. The
information associated with these events is used to calibrate the PEM system [28].
A Random Event is an abstraction, which corresponds to a pseudo normal event characterized
by the occurrence of hits in one plan of crystals in a period of time (referred to as random delay)
16
after the occurrence of hits in the other plan of crystals. They are artificially generated with real
data and allow estimating the noise caused by the acceptance of (pseudo) events that were not
actually originated in the same positron/electron collision.
An event is detected as normal if we have a coincidence, i.e., by the simultaneous occurrence
of hits in both crystal plans, making it possible to identify trajectories of radiation. In this context,
simultaneous means within a given time interval. This time interval is defined in terms of
temporal windows of acceptance and rejection of hits. The acceptance window is defined as the
maximum time interval for which a pair of hits, occurring in 2 crystal plans, can be associated to
the same event. The reject window is defined as the minimum time interval between photo-
events in the same crystal plan, so that any of them could be eligible for a possible match with a
photo-event occurring in the other crystals plan.
Compton window is defined as the time interval on which different hits, occurring in a single plan
of crystals, can be associated to a given event.
The hits associated with the single event occurring in a single plan of crystals during a time
window, are defined by the parameters Compton window and Reject window.
The normal and random events are also referred to as double events since they correspond to
the occurrence of hits in both crystal plans.
2.4 Data Acquisition Electronics
The Data Acquisition (DAQ) electronics is responsible for the digital processing and filtering of
data received from the FE. It also dispatches the relevant information to the image
reconstruction computer. In the following sections DAE system requirements are presented and
the architecture of its constituent modules is briefly described.
The DAE system takes advantage of the modular and hierarchical nature of the scanner
detector and FE electronics.
As mentioned, the DAE system must process information from 12288 APDs. The scanner
detector is organized in modules and the FE modularity reflects this organization. The DAE
system should reflect also this modularity. Flexibility and hierarchical nature of the DAE
architecture is another aspect of the project that lead to the choice of FPGAs as the main part of
the DAE system.
This section describes the structure and the modules of the FPGA DAE system. It also briefly
describes the Built-In Self Test methodology implemented to verify the correct functionality of
the FPGAs modules and detect hardware faults.
17
2.4.1 DAE system requirements
The main requirements that the DAE system must meet are:
1. 128 entry channels corresponding to 2 crystals plans.
2. Clock Frequency of 100 MHz
3. Single Events rate of 10 MHz
4. Doubles Events rate of 1 MHz
5. Random Events detection
6. Programmable parameters like: Accept Window time, Reject Window time;
Delay for the Random Events (Delay Length); Photo-event Threshold voltage;
Compton Threshold voltage, Compton Window time
7. Acceptance of 2 Comptons per crystals plan
2.4.2 DAE system modules
In Figure 2.15, the high level architecture of the DAE system is represented.
Figure 2.15 – DAE system high level architecture [20].
In Figure 2.15, the DAQs + FLTR modules are responsible for the identification of data
associated with photon-events. The TGR module is responsible for identifying relevant events.
Data conditioning is made in DCC module. The DAQ module is responsible for the identification
of crystals and the determination of the time of occurrence of each interaction.
There are 4 FE plans that communicate with the DAE. Each of these plans is a face of one of
the plans of crystals. As these faces are functionally identical, the functionality associated with
these plans is the same for all of them.
The block diagram of the DAE is shown in Figure 2.16. This figure also illustrates the data flow
between DAE modules and the outside interfaces. White arrows are data flow and grey arrows
are control flow.
18
The DAE system is composed of two subsystems which correspond, in fact, to the
implementation of two major features:
1. Data qualification and filtering (more detail in [20]).
2. Data organization and routing (more detail in [19]).
The subsystem referred to as “Data qualification and filtering” is responsible for the identification
and acquisition of data from the FE electronics.
The subsystem “Data organization and routing” is responsible for managing the entire flow of
data in DAE system and prepare it for transmission to the image reconstruction computer.
Data bandwidth required for each module and sub-modules interconnection is also represented
in Figure 2.16. It can be observed that the required bandwidth is being reduced along the data
path towards the image reconstruction computer. This is due to data filtering.
Figure 2.16 – DAE system modules and subsystems.
19
2.4.2.1 Data qualification and filter
The subsystem “Data qualification and filtering” is composed of three main modules:
1. Data Acquisition Module (DAQ)
2. Filter module (FLT)
3. Trigger module (TGR)
The Data Acquisition Module is responsible for filtering data sent by the FE system and for the
normalization of input signals. This normalization is a calibration and it is intended to
compensate the different behaviour of the crystals in the presence of photons and the different
behaviour of all the components of the FE electronics that carry out the initial conditioning of the
signals generated by the APDs. This procedure is called calibration of the scanner. More detail
about this calibration can be found in [19].
The Filter module detects and filters all events that had more than 3 hits per crystal plan. To do
this task, this module must detect all hits associated with the same event, that is, all the hits that
have occurred simultaneously in one of the crystal plans.
Trigger module detects Normal and Random events and coincidences in both crystals plans. If
those hits have the same Time Tag, they are considered to belong to the same event. If not,
they are discarded.
2.4.2.2 Data organization and routing
The subsystem responsible for data organization and routing consists of two major modules,
Memory Unit and Data Concentrator.
The Memory Unit is designed to store all data that were not filtered by Data Acquisition and
Filter modules and to select, according to the Trigger module indications, the data that is
relevant to the image reconstruction, i.e., the data associated with the occurrence of
coincidences.
The Data Concentrator groups all samples belonging to an event. This data is then formatted so
it can be sent via a Universal Serial BUS (USB) for the image reconstruction computer. The
USB transmission to the image reconstruction computer is made using a microprocessor, the
Cypress FX2LP. The data interface between the DAE system and the
Acquisition/Reconstruction Computer is explained in detail in chapter 5 and is a part of the work
that this dissertation refers to.
2.4.2.3 Functional DAE Scenarios
The DAE system operates on different scenarios, each representing an implementation of a
given feature.
There are 6 modes of operation, or scenarios: Normal/Random, Single, Constant Loading,
Functional Loading, Errors and Test.
20
The Normal or Random scenario describes the set of functions that are executed by the DAE
system during a PEM examination, i.e., when the system detects the occurrence of normal and
random events. In this mode of operation Single events are treated as noise and are therefore
disregarded.
The Single mode describes the functions performed by the DAE system when the system is in
calibration. In Single mode the system should detect the occurrence of Single events, i.e., the
system only receives data from a single plan of crystals.
The Constant Loading mode refers to the set of functions required for loading, in DAE memory,
the calibration values for the APDs. The Functional Loading mode identifies the current DAE
system operational mode. In the Error mode the image reconstruction computer asks the DAE
system what errors have occurred in a given mode of operation and a given time interval.
Finally, in the Test mode, the DAE system performs the Built-in Self Test.
The communication between the image reconstruction computer and the DAE system is
bidirectional. In Normal/Random mode, the DAE sends to the computer the data associated with
the events already processed for the reconstruction of the image. In Single mode (calibration),
the DAE sends to image reconstruction computer the samples received from the FE electronics
about hits on a single plan of crystals.
The image reconstruction computer sends to the DAE system control signals, such as, defining
the mode of operation, error control and indications regarding the Built-in Self Test.
21
3 DAE system hardware specification and
design
The main purpose of the work reported in this document is to describe the physical (both
mechanical and electrical) implementation of the DAE Crate’s system, including the crate itself,
the power supply, the backplanes and the DAQ and TGR/DCC Boards.
Figure 3.1 shows the physical modules of the DAE system and its connections to the FE
electronics and the image reconstruction computer.
Dedicated/Test Bus
Generic Bus
LVDS
Buffers (transceivers)
LVDS
DAQ
DAQ
ROC
FilterDAQ
DAQ
ROC
Filter
FPGA
FPGA
DAQ Board x 4
TRG/DCC ROC
TRGDCC DCC
ROC
FPGA
TRG/DCC Board
Buffers (transceivers)
USB
Connection
Image Recontruction
Computer
cPCI Backplane
FE
ASIC
FE
ASIC
LVDS LVDS
ADC ADC
FE Electronics
APDs
DAE System
cPCI Backplane
Figure 3.1 – DAE system modules.
As can be observed in Figure 3.1 the most important electronic modules are:
1. FE electronics, responsible for the acquisition and digitization of the signals
originated at the APDs (Avalanche Photo Detectors)
2. DAE system, responsible for the digital processing and filtering of the data
provided by the FE
3. Image reconstruction computer also used for system configuration purposes.
The DAE system is composed by 4 Data Acquisition boards (DAQ boards), a Trigger and Data
Concentrator board (TRG/DCC board) and 2 Compact PCI (cPCI) backplanes, containing the
Dedicated Bus and Generic Bus [19][20] respectively. The TGR/DCC board also contains a
22
USB interface (it will be described in detail in Chapter 4) to transfer bidirectional data between
the DAE system and the image reconstruction computer.
Two distinct features characterize the DAE system: Date Qualification and Filtering made by
DAQ and Filter modules [20] and Data Routing and Organization composed by Trigger and
Data Concentrator modules. This explains the need to design 2 types of boards. The DAQ
boards are responsible for the Date Qualification and Filtering and the TGR/DCC board is
responsible for the Data Routing and Organization.
The constant alteration of the system requirements during the development period was decisive
to choice the FPGA technology as the most appropriate for the implementation of the DAE
system. Additionally, the vast complexity of the system forced the development and
implementation of methodologies of self embedded functional test [19] [20], given also that the
chosen FPGA platform has room enough to include this feature.
Due to the large number of data lines required to transfer data from the FE electronics to the
DAE system, it has been necessary to use serializers in order to decrease the amount of
interconnecting cables. As the data transfer is unidirectional, the choice of serializers fallen on a
pair, emitter and receiver, with LVDS technology. In section 3.2.2, a brief description of these
serializers’ operation is provided.
The number of DAQ boards (4) used in the DAE system has been a consequence of the large
number of lines that the DAE system receives from the FE electronics and the fact that there are
4 FE crystal plans. Section 3.2 contains the design description of the DAQ board.
The TGR / DCC board is the master of the system, responsible for the arbitration of data
transfer between the DAE boards. It is also responsible for relevant data transfer to the image
reconstruction computer, using USB technology. Section 3.3 has a description of the design of
the TGR/DCC board.
Communication among the modules of the DAE system is carried out through the Dedicated
Bus and the Generic Bus. To provide the physical connection between DAE boards 2 Compact
PCI (cPCI) with 5 slots have been chosen. According to [19] and [20] the 2 buses were required
because it is not possible to transmit, in a single package, the information related to more than
one photoelectric event. This results from the fact that photoelectric events occurs rarely (in
time) and randomly (in the crystals space).
The information that flows through the Dedicated Bus (DBUS) is related to one photoelectric
event, which occurs at 10 MHz rate. The number of bits associated with this information is small
in comparison with the width (in terms of number of bits) of the Generic Bus (GBUS). It is not
possible to associate the information related with several photoelectric events in order to take
advantage of the GBUS width (in terms of bit width). This circumstance, associated with the
GBUS communication protocol requirements, reduces the GBUS efficiency in this type of
transmission to values lower then 12,5%. These circumstances justify the use of 2 different
buses in the DAE.
23
The 2 cPCI backplanes have enough data lines to satisfy the DAE system [19] [20]
requirements and to support a data line rate (66 MHz) superior of the DAE requirements (50
MHz). Section 3.1.2 has a description the cPCI backplane.
To household the DAE system, a custom 6U crate has been manufactured, hereinafter called
DAE Crate. The description of the DAE crate is provided in section 3.1.1.
3.1 DAE crate and Backplanes specification and design
3.1.1 DAE Crate
The DAE crate was specified to support the complete DAE system, including the 4 DAQ and
TGR/DCC boards, backplanes, power supply and control signals, taking advantage of the
modularity and flexibility that a crate can provide and making it easier the DAE integration with
the other physical modules of the PEM system.
Since there were no mechanical limitations to this crate, neither in terms of volume nor in terms
of weight, the selection was based on electronics constraints only.
The backplanes have only 5 slots. Thus, in order to take advantage of the available space, it
was decided that the slots should be placed horizontally in the crate. Mechanically a 6U crate is
the best option.
A custom-made crate, from Elma, which is a modified version of the standard sub-rack Type 12
family, enabling the assembly of two 3U power supplies, two Compact PCI backplanes and
eight 6U slots was chosen.
Below, the front view and the front and back perspective views of the crate can be seen: Figure
3.2 shows some perspectives of the crate design.
Front View
Back Perspective Front Perspective
Figure 3.2 – DAE Crate design.
24
The DAE crate includes 2 3U 200W power supplies (Elma 308-0200AC), each capable of
delivering 36A at 3.3V and 25A at 5.0V, that perfectly suits the power requirements for the DAE
system.
The DAE crate also includes seven fans that guarantee homogeneous air distribution inside the
crate. The fan disposition inside the crate can be seen in the Figure 3.3.
Figure 3.4 shows a picture of the DAE crate with the cPCI backplanes and the DAE boards
mounted (4 DAQ boards and a TGR/DCC board). Figure 3.5 shows 2 pictures during the test
procedures of the DAE system.
Figure 3.3 – DAE Crate ventilation.
Figure 3.4 – DAE Crate picture.
Figure 3.5 – DAE Crate pictures during DAE system test.
25
3.1.2 DAE Compact PCI Backplanes
Figure 3.6 shows the 2 Compact PCI Backplanes configuration used in the DAE system.
Figure 3.6 – DAE cPCI Backplanes configuration.
The main requirements are a 5 slot backplane (DAE system requires 5 boards: 4 DAQs and 1
TGR/DCC) and data lines should support 50 MHz, since communication between DAE boards
is to be carried out at 50 MHz. 2 Hartmann Electronik 33LA054614 backplanes were chosen,
since it allows data lines support communication at 66 MHz and have 5 slots (1 master for the
TGR/DCC board and 4 slaves for the DAQ boards). The DAE system communication requires
more data lines than a single backplane can provide. Thus, 2 backplanes are need in the DAE
system.
However, none of the communication protocols adopted for this system has been developed
according to the standard PCI protocol. The backplane is only the physical support used.
Each backplane provide 100 data lines (64 lines represent the original PCI data bus of 64 bits)
and 15 dedicated lines arranged so that one master slot can independently control the 4 slave
slots.
3.2 Data Acquisition (DAQ) board hardware specification and
design
The DAQ Board is responsible for data qualification and filter. In the DAE system, the DAQ
board implements the heavy processing algorithmic.
Figure 3.7 shows the block diagram of the DAQ board. There are 4 main blocks:
1. Control block: responsible for power management and board monitorization
(voltages, currents, consumption and temperatures).
2. Acquisition block: interface between the processing block of the DAQ board and
the Front-End electronics.
3. Processing block: board’s main core.
26
4. Interface block: interface between the DAQ board and the other DAE boards
(inside the crate).
L
V
D
S
F
E
E
l
e
c
t
r
o
n
i
c
s
Acquisition
Block
E
P
R
O
M
Processing
Block
F
P
G
A
sTransceivers
Interface Block
D
A
E
B
U
S
E
S
Control Block
MonitoringPower Supply and
Regulators
DAE System
Figure 3.7 – DAQ board block diagram.
A detailed description of each main block of the DAQ board is provided in the following sections.
An even more detailed description of the DAE specification and design can be found in [33].
3.2.1 DAQ control block
The control block is responsible for the power regulating and management, including the power-
up time, where the system requires more power. It also provides an effective control and
monitorization of several parameters such as the FPGAs currents, voltages and temperatures.
For this task a microprocessor from the 8051 family has been chosen, specifically the Cypress
FX2LP.
The FX2LP microprocessor establishes an internal I2C bus communication to several control
devices, to monitor the 3.3V, 1.8V and 1.5V power supplies and the FPGAs currents,
consumptions and temperatures. The 3.3 V supply is required to feed virtually the whole board,
including the transceiver firewall, the LVDS circuits and the FPGAs. It is received from the
crate’s power supplies through the cPCI backplanes. The 1.8 V supply is needed to power up
the FPGAs’ E2PROM core and the 1.5 V supply is needed to feed the FPGAs’ core. These
supplies are obtained with the use of accurate Very Low Drop Out (VLDO) voltage regulators.
Both were designed to feed digital circuits only. These voltages are obtained using the 3.3V
supply from the crate thus lowering the power dissipation. Linear regulators were chosen mainly
due to the circuitry simplicity and lower Electromagnetic Interference (EMI).
Annex 14 contains electrical schematics of the DAQ board, including the control block provided
by the FX2LP microprocessor.
4 Public version of this dissertation has illegible schematics related to confidential cause.
27
The FX2LP microprocessor communicates with the TGR/DCC Board’s microcontroller (also a
FX2LP) through an external I2C bus (using the cPCI backplane) which serves as a gateway to
the image reconstruction computer, which also functions as monitoring computer.
The FX2LP microprocessor in the DAQ boards can also communicate directly the monitoring
values to an external computer through an USB or RS232 connection. Thus, this is only used to
debug during prototype validation.
3.2.2 DAQ acquisition block
The acquisition block is the interface between the DAE processing block and the Front-End
electronics. It is mainly composed by the LVDS receivers (that connect to the Front-End sub-
system drivers, i.e., LVDS emitters) and the respective connectors.
The DAQ Boards’ input signals come from the Front-End electronics through 8 cables.
Tests were made to choose the most adequate connectors and cables for this connection. Both
the tests and the results are described in [34]. From these results, the twisted-pair cable
reference 125-3007-026 from Spectra-strip was chosen.
Due to physical limitations and the needed mechanical resistance, the 054595 connector from
Erni was chosen. Each one of these cables is connected on the FE side to one LVDS driver
(two FE ASICs), and on the DAQ side, each connector serves one LVDS receiver.
The LVDS circuit used for this connection is based on the driver from National Semiconductor
DS90CR483 which multiplexes 48 inputs into 8 outputs. This implies a cable reduction to 1/6
and the increase in the transmission rate from 100Mbps to 600Mbps.
The DAQ Board then uses an LVDS receiver (the NS DS90CR484) to de-serialize the data and
converts it to LVCMOS levels.
From each one of these receivers, 44 signals are sent to the corresponding FPGA in 4 channels
having 11 lines each: 10 data lines/bits (corresponding to the data samples) and an
identification line (as shown in section 2.3).
The LVDS receiver also sends to the FPGA two additional signals: the CLK_OUT and the
SYNC which are the LVDS clock and a FE synchronization signal. These signals help to
compensate the potentially different delay time in the interconnecting cables.
Each FPGA connects to 4 LVDS receivers, enabling each the processing of 184 Front-End
signals, corresponding to 8 FE ASICs and 4 crystal areas. One crystal area matches a matrix of
2x3x2 APD’s (192 crystals).
28
3.2.3 DAQ processing block
The processing block is this board’s central block and it is responsible to process data from the
FE filter. It is composed by two FPGAs plus their configuration EEPROM.
Due to the amount of connections that are required, 4 million-gate Xilinx FPGAs were chosen,
the XC2V4000-BF957, coming in a 957 pin flip-chip BGA package. The FPGAs functionality is
described in [30] and [31].
The DAQ board design and CAD was projected using Xilinx recommendations, as can been
seen in [33]. The main constrains are detailed in the next lines.
The power supply for this block feeds basically both FPGAs and their EEPROM. These voltages
are supplied by linear regulators (section 3.2.1) and distributed using the PCB’s (Printed Circuit
Board) power planes, in order to reduce the path’s resistance and inductance as much as
possible, lowering the voltage drop and EMI (Electromagnetic Interference).
Due to the extremely low voltage of the FPGAs (1.5 V) and EPROM (1.8 V), the linear
regulators for these voltages should be placed close to the devices they serve. The high current
demand for the 1.5 V supply, leads to the use of 2 linear regulators (one for each FPGA).
The decoupling of the power supply is achieved mainly using capacitors. Xilinx application note
[38] has been used to calculate the ideal amount and capacitance for these capacitors. The
capacitors should be ideally placed right below the pins they serve, on the PCB’s bottom layer
to optimize their performance. The maximum ideal distance to the pins is 1/40 of the resonant
frequency wavelength:
The backup battery input was used to accomplish configuration encryption and it’s used to store
the encryption keys for the FPGAs firmware.
Since the FPGAs have volatile configuration, it must be loaded each time the FPGA is powered
up. The configuration to be loaded is usually stored in one Flash EPROM (or EEPROM) that
communicates via a serial or parallel port with the FPGA.
Xilinx recommends the Flash EPROM XCF16P for these FPGAs. Since the two FPGAs are
expected to have the same exact behaviour, one Flash EPROM is enough to carry the
configuration (the Xilinx tools state that we need 93% of total space available).
When the FPGA is powered-up, the configuration is loaded in either serial, parallel or via JTAG,
master or slave, depending on the configuration desired. In this project, the parallel mode was
chosen. This mode enables the simultaneous loading of both FPGAs, being the clock signal
EEPROM fed. However, for the first prototype, jumpers enabling other configurations were
added for debug purposes.
The EEPROM itself is programmed via its JTAG port, using the cables and software provided by
the manufacturer (Xilinx Parallel IV cable and Xilinx iMPACT software).
29
The processing block needs the special signals, generated in the TGR/DCC Board in order to
function properly. These signals are as follows:
1. Clock: the clock signal is received from the backplane (Dedicated Bus) using
the backplane’s dedicated lines, at ½ the nominal frequency – 50MHz, due to
backplane’s physical restrictions. To reconstruct this signal and enable it to
drive the DAQ Board’s internal circuits, the ICS87004 was chosen. This zero-
delay clock generator can multiply the input clock and regenerate the clock
signal coming from the bus. In our case, we use only one of its two inputs and
two of its four outputs to drive the two DAQ FPGAs with a multiplied-by-two
clock signal – 100MHz. These clock signals are source-terminated with 68Ω
resistors (the average board characteristic impedance is Zo=67 Ω). For debug
purposes a clock signal from the FPGA to the test connector was also added.
2. Reset: the reset signal is received from the backplane (Dedicated Bus), using
normal data lines. It is active low and each board has its own debouncing
circuit, thus avoiding spurious glitches. This signal resets both the FPGAs and
the microcontroller. In addition to this system reset signal, for each board, a
watchdog mechanism is provided.
3. Sync: the sync signal is received from the backplane (Generic Bus) using the
backplane’s dedicated lines. To increase the system’s time resolution, the DAQ
FPGAs receive also from each of the LVDS receivers, the signal LVDS
CLK/SYNC. Its clock signal enables the FPGA to know the phase difference
between all LVDS channels and thus compensate that phase difference. This is
important because the system works with a time resolution close to the clock
period. Additionally, the FPGA receives the LVDS_SYNC signal, but enables
the FPGAs to know the FE synchronization moment.
The clock and sync signals use special FPGAs’ pins for clock distribution inside the FPGA. The
reset signal uses a specific pin to reset the FPGA. So a very careful pin assignment had to be
performed [33].
These FPGAs’ I/O connections can be divided into several functions, namely, the FE processing
and the outputs to the Generic and Dedicated Buses plus some test and system pins.
Physically, the FPGAs are divided into 8 banks plus the power supply area.
An assigned bank connects to one LVDS chip, corresponding to the handling of 4 channels (2
front-end ASICs). A detailed pin distribution by function can be seen in Annex 3, being the exact
pin layout defined by the CAD.
30
3.2.4 DAQ interface block
The DAE interface block corresponds to the interface between the DAQ board and the other
DAE boards and handles the transceiving between the local FPGAs and the others located in
both the other DAQ boards and in the TGR/DCC board.
According to [33], the system requires 4 DAQ Boards, each one sustaining by 2 FPGAs.
To implement the communications between the DAQ FPGAs and the TGR/DCC FPGAs, two
communication ports are needed. This is due to the big amount of data to exchange. As
mentioned before, these two buses were designated “Dedicated/Test Bus” and “Generic Bus”.
The former is also divided into two communication channels.
Physically, each bus stands on a 3U CompactPCI backplane
To protect the DAQ Boards, data and control lines, transceiving is needed as line drivers. Due
to the short delays required, namely in the Dedicated Bus protocol, two high speed transceivers
from Texas Instruments and its latest logic family, LVTH, were chosen:
1. SN74LVTH18512DGG (or SN74LVTH182512DGG) (18-bit) on a DGGR64
package.
2. SN74LVTH18514DGG (or SN74LVTH182514DGG) (20-bit) on a DGGR64
package.
Annex 15 contains electrical schematics of the DAQ board, including the interface block.
According to the DAE specification [35], the Generic bus requires for each FPGA: 64
bidirectional Data lines, 2 Request lines, 5 bus Grant lines, 2 bus data acquisition signals, 1
transceiver bus Read control line and 1 transceiver bus Write control line.
The Dedicated bus is divided into 2 logical channels (due to limitations in the available
bandwidth) and requires for each FPGA: 38 (2 x19) data lines, 6 lines to control the direction of
the transceivers, 2 request lines and 2 bus grant lines.
The purpose of the Test bus is to enable a very flexible BIST to all the DAQ boards inside the
crate. Data is then collected by the TGR/DCC board and afterwards sent to the image
reconstruction computer. Chapter 5 has more details about the functional BIST test
implemented in the DAE system. The Test bus is composed of 6 lines: 3 lines for DAQ FPGA
identification, 1 line for test bus control line, 1 line for test result and 1 line to indicate that the
test is concluded.
Using the Texas Instruments transceivers, all the specifications, like the protocol and time
constrain, established in [35] are fulfilled.
5 Public version of this dissertation has illegible schematics related to confidential cause.
31
3.2.5 DAQ board mechanics
The Figure 3.8 shows the DAQ board dimensions and layout.
Dedicated/Test BusGeneric Bus
Control Block
Monitorization
Circuit
DAQ FPGA 2DAQ FPGA 1
LVDS LVDS LVDS LVDS LVDS LVDS LVDS LVDS
FPGA1
VCCINT
FPGA2
VCCINT
E2
PR
OM
Dedicated Bus
Transceiver Firewall
Generic Bus Transceiver
Firewall
Acquisition Block
LVDS Connectors LVDS Connectors
Interface Block
Processing Block
16
0 m
m
233 mm
Status LEDs
TestTest
Figure 3.8 – DAQ board dimensions and layout.
As mentioned before, physically the DAQ board has the standard 6U dimensions (233mm x
160mm), having two 3U cPCI connectors, one for the Dedicated and Test Bus and one for the
Generic Bus, as required in [35]. The placement and connectors’ type are defined in the cPCI
specification [36].
The layout shows the transceiver firewalls for each bus, placed near the buses they serve. The
acquisition block is on the other side of the board and is placed so that it can be accessible from
the front panel, enabling the connection of the FE cables. The processing block is on the middle
of the board between the acquisition and interface blocks and the FPGAs are geometrically
placed so as to be as far as possible from each other. The control block is placed near the
connectors. The test connectors, for each FPGA, are placed near them, helping the debug
during prototype validation phase.
32
Figure 3.9 – DAQ board front panel layout.
The front panel layout of the DAQ board is shown in the Figure 3.9 and accessible in the front
panel of the DAQ Board are the LVDS connectors and the status LEDs only. The LVDS
connectors enable the connection to the Front-End cables. There are eight per board.
The status LEDs are divided in three parts: the FPGAs LEDs; the microcontroller LEDs and the
system LEDs [33].
The detailed test implementation is completely described in [37]. Details will be provided in this
dissertation in the 5th Chapter. In order to implement the test requirements some features were
added in the DAQ board design, namely:
1. Test Points, to enable a quick check on some of the main electrical lines.
2. Test Connectors, for each FPGA, to enable a quick test setup.
3. Boundary Scan Test, in order to facilitate the test procedures production. It has
been decided that, where possible, all components would be BST or JTAG
(IEEE 1149.1) compliant.
The JTAG circuit inside the DAQ Board has 2 chains: one connecting the JTAG features of the
E2PROM and FPGAs, used for FPGA programming, and, the other connecting the two
transceiver firewalls. This JTAG chain is also used to download the FPGAs firmware to the
EPROM using the Impact Xilinx software.
Figure 3.10 shows a picture of complete and tested DAQ board. Figure 3.11 shows the
environment during the development of the DAQ board.
33
Figure 3.10 – DAQ board picture.
Figure 3.11 – DAQ board test setup.
34
3.3 Trigger and Data Concentration (TGR/DCC) board hardware
specification and design
The TGR/DCC board is the control centre of the DAE sub-system. This board is responsible for
the arbitration of the DAQ boards, data gathering and special signals generation. Due to its
central role, it has to be placed in the backplanes’ master slot.
The following figure ( Figure 3.12) shows a block diagram of the TGR/DCC board.
D
A
E
B
U
S
E
S
TGR/DCC BlockInterface BlockFPGA
TGR/DCC System
TransceiversEPROM
USB Block
FX2LP USB
Microprocessor
Special Signals Block
RESET SYNCCLOCK
Control Block
MonitoringPower Supply and
Regulators
FE Service Board
Image Reconstruction
Computer
Figure 3.12 – TGR/DCC block diagram
There are 5 main blocks in this board:
1. Control block: responsible for power manager and board monitorization
(voltages, currents, consumption and temperatures).
2. Special signals generator block: the reset, synchronization and clock signals
are generated in this block.
3. TGR/DCC block: the board’s main block, is responsible for DAQ trigger
generation and data concentration of the DAE sub-system. The information is
then flushed to the USB Block.
4. USB block: responsible for data transfer to the Acquisition PC using a USB 2.0
connection interface.
5. Interface block: interface between the DAQ board and the other DAE boards
(inside the crate).
A detail description of each main block of the TGR/DCC board is provided in the following
sections.
More detail about the DAE specification and design can be found in [33].
35
3.3.1 TGR/DCC control block
The TGR/DCC control block is very similar to the one in the DAQ Board (section 3.2.1), and it is
responsible for the power regulation and management, namely during power-up time when the
system requires more power. It also provides an effective control and monitorization of several
parameters, such as the FPGA current, voltage and temperature. For this task the same
microprocessor used in the DAE boards was chosen: the Cypress FX2LP (8051 family).
Annex 26 contains electrical schematics of the TGR/DCC board, including the control block
provided by the FX2LP microprocessor.
The internal I2C circuit is similar to the DAQ boards and provides monitorization for the 3.3V,
1.8 V and 1.5 V power supplies, FPGA current, consumption and temperature.
The main difference is in the external I2C circuit, since the TGR/DCC FX2LP microcontroller
communicates with the 4 DAQ board’s microcontrollers, as master, gathering monitorization
data for all the DAQ boards and sending that information to the image reconstruction computer,
which also functions as monitoring computer, through a USB or RS232 connection.
Chapter 5 provides a more detail description of the DAE monitorization circuits.
3.3.2 TGR/DCC block
The TRG/DCC block is the heart of the TGR/DCC board and the DAE sub-system. This block is
responsible for the coordination of all the DAQ boards and the decision on which data shall be
marked as valid or non-relevant. Valid data is afterwards sent, via USB block, to the Image
reconstruction computer for image processing.
This block is mainly composed by one FPGA and its configuration E2PROM.
This FPGA’s task is less complex and less connections are required than the DAQ FPGA’s. A 3
million-gate Xilinx FPGA was chosen, the XC2V3000-BG728, coming in a 728 pin BGA
package. This device works at 100MHz, has 484 I/Os and has basically the same needs as the
XC2V4000 used by the DAQ Boards. The FPGA functionality is described in [30] and [31].
The TGR/DCC board design and CAD was laid out using Xilinx recommendations, as can been
seen in [33] and the main constrains are similar to the DAQs boards FPGA as explained
previously in section 3.2.3.
3.3.3 TGR/DCC USB transmission block
The transmission block is responsible to send the relevant data from the DAE system to the
image reconstruction computer. For the prototype validation and the first PEM machine a USB
connection was chosen. Since the FX2LP microprocessor used in the control block has a Serial
Interface Engine (SIE), thus, it was used to provide the transmission block. Since this USB
6 Public version of this dissertation has illegible schematics related to confidential cause.
36
connection is a significant part of the work developed by the author, a detail description of the
DAE USB transmission system is provided in chapter 4. Basically, the TGR/DCC FPGA sends
data trough a 16 bit FIFO interface supported in the FX2LP microprocessor. Then, the FX2LP
microprocessor establishes the USB interface between his internal FIFO and the image
reconstruction computer.
3.3.4 TGR/DCC special signals generation block
Special signals are the control signals that are common to all the units inside this sub-system,
having a very important role in system’s functioning. These signals are the Clock, Reset and
Synchronization and are generated in this block and then broadcasted to the whole system. In
the next sections a description of the special signals design is provide.
3.3.4.1 Clock
Due to its nature, the PEM system critically depends on the synchronization of the various
constituting sub-systems. There are several ways to achieve this goal. However, being the
frequency deviation the main concern, it was decided to rely on a central clock and then
distribute the clock to every system needing to be in sync. This option easies the choice for the
oscillator itself (synchronizing different oscillators demands extremely precise oscillators and
further environmental compensations), but demands a very reliable distribution network. Having
this in mind, the distribution network was chosen to be an ICS family of clock chips.
The frequency of the system’s main components is 100MHz. The oscillator chosen was the
Saronix S1613BP-100.000. This oscillator has output frequency stability lower than 100 ppM,
rise and fall times lower than 2 ns and a total jitter of 40 ps pk-pk.
The oscillator is connected to a clock fan-out buffer that acts as the first stage system clock
driver. It feeds the clocks inside the TGR/DCC Board, the clocks to the DAQ Boards and the FE
sub-system driver.
3.3.4.2 Reset
The reset signal has to be distributed throughout the system from a central location. For
coherency purposes, the TGR/DCC board was chosen to be the source of reset signal as well.
There must be two sources of reset: the power-on-reset and a manual pushbutton enabling the
user to restart the whole system.
Looking for a simple supervisor circuit, the Maxim MAX811 was chosen. This supervisor has a
manual pushbutton input and features power-on reset and low-voltage reset. The circuit chosen
was factory-trimmed to give a reset time of, at least, 600ms, which is more than the specification
demands [35].
To broadcast the reset signal, a buffer circuit is used in order to higher the fan-out of the
supervisor circuit.
37
3.3.4.3 Synchronization
Having a similar approach to the clock signal, the synchronization circuit’s main function is to set
periodic synchronization points, common to all circuits, preventing mistakes due to eventual
counting errors inside the different processing units of the system. The synchronization signal
shall have sharp rising and falling times, but low frequency. This frequency shall be high enough
so it can frequently reset the counters and lower the probability of errors but low enough so it
does not interfere with the processing itself. A convenient value was found to be 1 kHz. The
precision of this frequency is not critical. However, according to [35], this signal shall have a
very low duty-cycle (the up-time shall be between 10ns and 20ns, which infers a duty-cycle as
low as 1/100 000).
Generating the required frequency is simple and doesn’t require any special component.
However, the ultra-low duty-cycle is very difficult to achieve because one needs to derive a very
precise pulse from a low frequency signal. To guarantee synchronization, it has been decided to
supply the TGR/DCC FPGA with the low-frequency signal and the FPGA would give the
synchronization signal. This can be seen in the TGR/DCC electrical schematics in Annex 27.
3.3.5 TGR/DCC interface block
The TGR/DCC interface block handles the transceiving between the TGR/DCC and DAQ
FPGAs. It’s very similar to the DAQ board interface block (section 3.2.4), since the transceiving
requirements has the same constraints, thus, the same transceivers are used.
According to [33], the system requires 1 TGR/DCC board, having 1 FPGA. Annex 2 contains
electrical schematics of the TGR/DCC board, including the interface block.
According to the DAE specification [35], the Generic bus requires for each FPGA: 64
bidirectional data lines, 8 request lines, 5 bus grant lines, 2 bus data acquisition signals, 1
transceiver bus read control line and 1 transceiver bus write control line.
The Dedicated bus is divided into 2 logical channels (due to limitations in the available
bandwidth) and requires for each FPGA: 2 x 19 data lines, 8 lines to control the direction of the
transceivers, 8 request lines and 8 bus grant lines.
The Test bus purpose is to enable a very flexible BIST to the TGR/DCC board. Chapter 5 has
more details about the BIST test of the DAE system. The Test bus is composed by 6 lines, with
the same characteristics of the DAQ board (section 3.2.4).
All the specifications, such as, the protocol and the time constrains, established in [35] are
fulfilled.
7 Public version of this dissertation has illegible schematics related to confidential cause.
38
3.3.6 TGR/DCC board mechanics
The Figure 3.13 shows the TGR/DCC board dimensions and layout.
Dedicated/Test BusGeneric Bus
TGR/DCC
FPGA
TGR/
DCC
FPGA
VCCINT
E2P
RO
M
Dedicated Bus
Transceiver Firewall
Generic Bus Transceiver
Firewall
Clock
Connectors
Interface Block
TGR/DCC Block
16
0 m
m
233 mm
Status LEDs
Te
st
Transfer Board Connectors for Mezzanine Board
Reset Generation
CircuitClock
Generation
Circuit
Sync
Generation
Circuit
Sync
Connectors
High-Speed ConnectorsReset
Connectors
RS232
Connector
Control Block
Monitorization
Circuit
USB Transfer Block
FX2LP
Microprocessor
USB
Connector
Figure 3.13 – TGR/DCC board dimensions and layout.
Similarly to the DAQ Board, physically the TGR/DCC board has the standard 6U dimensions
(233mm x 160mm), having two 3U connectors, one for the Dedicated and Test Bus and one for
the Generic Bus, as required in [35]. The placement and connectors type are defined in the
cPCI specification [36].
The layout in Figure 3.13 shows the transceiver firewalls, for each bus, placed near the buses
they serve. The clock and sync generation circuits, being the most critical, are placed near the
front panel. The reset generation and the control block circuits don’t have special constraints.
The front panel is basically composed by the connectors of the different generation circuits
(RS232 and USB). A set of status LEDS and a space for a Mezzanine connector for a possible
DAE system upgrade to a higher speed link than the USB 2.0 protocol is provided. In the
TGR/DCC board design, available space was left to insert a Mezzanine card, through the
“Transfer Board Connectors for Mezzanine Board” connector. This subject will be discuss in
chapter 7, were guidelines and future work improvements are presented.
The front panel layout of the TGR/DCC board is shown in the Figure 3.14.
39
233 mm
20
mm
TGR/DCC Board
Reset/Test
Connectors
Clock/Sync
Connectors
RS232
Connector Reset
Space for Mezzanine Board
System LEDsFPGA LEDs
uC LEDs
DBUS CH2
DBUS CH1
FPGA ON
GBUS FPGA ERR
SYS COMM
SYS RESET
FPGA SM
USB
Connector
Figure 3.14 – TGR/DCC board front panel layout.
Accessible in the front panel of the TGR/DCC board are the status LEDs, a manual reset button
of all DAE system and the following connectors:
1. RS232 Communication (RJ45 connector)
2. USB Communication (USB connector)
3. Reset/Test Pulse Signal (Lemo female connector)
4. Clock/Sync Signal (Lemo female connector)
The status LEDs are divided in three parts: the FPGAs LEDs; the microcontroller LEDs and the
system LEDs [33].
In order to implement the test requirements some features were added in the TGR/DCC board
design:
1. Test Points, to enable a quick check on some of the main electrical lines.
2. Test Connectors for the FPGA, to enable a quick test setup.
3. Boundary Scan Test, in order to facilitate the production test procedures, it was
established that, where possible, all components would be BST or JTAG (IEEE
1149.1) compliant.
The JTAG circuit inside the DAQ Board has 2 chains: one connecting the JTAG features of the
E2PROM and FPGAs, used for FPGA programming, and, the other connecting the two
transceiver firewalls. This JTAG chain is also used to download the FPGA firmware to the
EPROM using the Impact Xilinx software.
Figure 3.15 shows a picture of complete and tested TGR/DCC board and Figure 3.16 shows a
TGR/DCC board in development and under test.
40
Figure 3.15 – TGR/DCC board picture.
Figure 3.16 – TGR/DCC board picture during test procedures.
41
4 Data interface between the DAE system
and the Acquisition/Reconstruction
Computer
This chapter contains a description of the data interface between the DAE system and the
acquisition/image reconstruction computer.
After the DAE system organizes, filters and prepares relevant data, received from the FE
system, the DAE system requires a data transmission method to transfer the relevant data to
the image reconstruction computer.
Due to time limitations, to validate the first PEM prototype it was decided that a USB 2.0
connection could satisfy the system requirements. The main goal is to achieve a sustained data
rate of 30 MBps. For the second prototype and to improve PEM data rate transfer it has been
decided to build an optical link between the DAE system and the image reconstruction
computer.
The data transfer rate is an important issue, since a better data transfer between the DAE
system and the image reconstruction computer will shorten the time exam, thus improve the
comfort of the patient under examination. Studies [20] [21] reveal that the DAE system can
produce 220 MBps of relevant data with the PEM system configuration. If the data transfer rate
is less than the required 220 MBps, some relevant data has to be discarded and thus, more
time is required for the same exam quality. The huge amount of data to be transferred is far
from being a trivial task.
In fact, even for fast computers, to write in disk at such data rate demands that the computer
have fast hard disks. In chapter 7, that presents future work, a brief description of the optical link
envisioned to implement the required 220 MBps of data transfer rate between the DAE system
and the image reconstruction computer is also presented.
For the prototype validation, a 30 MBps data transfer rate was considered acceptable. It has
been decided to use a USB 2.0 connection for several reasons: It has the required data rate, an
easy integration with the current computers (since all of them have USB ports), does not require
a special custom card in the image reconstruction computer (like the optical solution), it is
cheaper and finally can be provided by an integrated circuitry already used in the DAE system
design, the Cypress FX2LP. As described in section 3.2.1 and 3.3.1 the Cypress FX2LP
microprocessor [39] [40] is used to monitor the temperatures, voltages and currents of the DAE
boards.
Since the TGR/DCC board has the data flow control of the DAE system, it was decided that the
FX2LP microprocessor in the TGR/DCC board is used to be the USB interface between the
42
DAE system and the image reconstruction computer. The following sections contain the
implementation description of the DAE system USB interface.
4.1 Bidirectional Universal Serial Bus (USB) interface
The USB interface is bidirectional, i.e., relevant imaging data is sent from the DAE system to the
image reconstruction computer and the image reconstruction computer sends calibration and
configuration commands to the DAE system. Despite the USB link bidirectional nature, the
demanding data flow is the relevant imaging data coming from the DAE system towards the
image reconstruction computer. Data flow transfer rate from the computer to the DAE system is
not an issue because those commands, calibration constants and configuration are sent in small
size packages.
4.2 Hardware design of the USB interface
It is possible to see in Annex 28 the electrical schematics of the TGR/DCC board, including the
USB data interface between the DAE system and the image reconstruction computer, provided
by the Cypress FX2LP microprocessor. Figure 4.1 shows the logic block diagram of the Cypress
FX2LP microprocessor.
Figure 4.1 – Cypress FX2LP microprocessor logic block diagram.
8 Public version of this dissertation has illegible schematics related to confidential cause.
43
FX2LP microprocessor has an enhanced 8051 microprocessor that works at 48 MHz clock
frequency with 4 clocks per cycle, instead of the regular 8051 microprocessor that works at 12
MHz and 12 clocks per cycle. It has a serial interface engine capable of provide a full USB 2.0
connection of 60 Mbyte/s (or 480 Mbit/s), an I2C master connection interface and a 4 Kbyte
FIFO with external interface of 16 bits length.
Figure 4.2 – Cypress FX2LP simplified block diagram for USB transmission.
Figure 4.2 shows a simplified block diagram of the FX2LP microprocessor with the parts used to
transfer the imaging relevant data to the external computer.
It is possible to set up the FX2LP slave FIFOs to a maximum of 4 FIFOs. The total amount of
available memory is 8 Kbytes for those FIFOs. Those FIFOs are associated with USB endpoints
(sort of USB data buffers), allowing data to be send directly from the TGR/DCC FPGA and the
USB SIE of the FX2LP with just a small control and configuration of the 8051 core. Since the
8051 core works only at 48 MHz, if data has to pass through the 8051 core, performance would
be degraded, thus the slave FIFOs are directly connected to the USB endpoints.
There are 2 types of data sources transferred between the DAE System and the
acquisition/image reconstruction computer:
1. Relevant imaging data provided by the TGR/DCC FPGA to the image
reconstruction computer and calibration and control commands that the
computers send to the TGR/DCC FPGA.
2. Monitoring data of the DAE system, like temperatures, voltages and currents,
from the monitoring circuits (provided also by the FX2LP microprocessor) sent
to the external computer for visualization.
The FX2LP can provide 4 endpoints to the USB SIE. Two of them are IN endpoints (data flows
to the computer direction) and 2 are OUT endpoints (data flows to the FX2LP direction). Data
direction is defined having the external computer as the reference because in USB connections
the computer is the host and master of the communication. Those endpoints are defined by
Cypress (FX2LP manufactory) as endpoint 2, 4, 6 and 8.
This is a fundamental USB concept. There is exactly one master in a USB system: the host
computer. USB devices respond to host requests. USB devices cannot send information
between themselves, as they would if USB had a peer-to-peer topology. There’s an excellent
44
reason for this host-centric model: the USB architects were keenly mindful of cost, and the best
way to make low-cost peripherals is to put most of the intelligence into the host side, i.e., the
computer. If USB had been defined as peer-to-peer, every USB device would have required
more intelligence, raising cost.
Since there are 2 types of data, the FX2LP uses all the 4 endpoints:
1. Endpoint OUT 2, houses data that come from the computer to the TGR/DCC
FPGA, through a slave FIFO.
2. Endpoint OUT 4, houses data that comes from the computer to the FX2LP
microprocessor, in this case, a command to ask for monitoring data.
3. Endpoint IN 6, houses data that come from the TGR/DCC FPGA to the image
reconstruction computer through a slave FIFO.
4. Endpoint IN 8, houses monitoring data that come from FX2LP to the image
reconstruction computer.
Figure 4.3 shows the FIFOs interface signals and the relation between them and the USB
endpoints.
Figure 4.3 – FX2LP FIFOs interfaces.
The FIFO interface is standard for this kind of operation, with data lines, clock line, write strobe
line, read strobe line, output enable line, and full/empty flags. The address lines selects which
endpoint is selected for the desirable FIFO interface and the packet end line informs the
endpoint that frame is finished and ready to be sent to the computer.
For the PEM project the clock is provided by the FX2LP microprocessor with 48 MHz and the
FIFOs work in synchronous mode, for fast data rate transmission, since the 48 MHz are the
maximum allowed frequency for the FX2LP slave FIFOs.
The TGR/DCC FPGA selects the endpoint 2 and endpoint 6 in the FIFOs addresses, if data is
coming from or being sent to the computer respectively.
45
The connections between the FX2LP microprocessor and the TGR/DCC FPGA and with the
image reconstruction computer are shown in the following figure (Figure 4.4).
SIE EngineUSB
Interface 8051 Core
TGR/DCC FX2LP
Endpoint 4 and 8
Endpoint 2 and 6
Slave FIFO
IN/OUT
TGR/DCC FPGA
Endpoint Control
Endpoint
Selection
DAQ Boards
I2C Circuits
I2C Circuits
TGR/DCC
Board
ControlClock
Imaging
Data
Monitoring Data
Monitoring
Data
Image
Reconstruction
Computer
Figure 4.4 – TGR/DCC FX2LP microprocessor connections.
The FX2LP uses 2 endpoints to establish the connection between the image reconstruction
computer and the TGR/DCC FPGA and uses the other 2 endpoints to send monitoring data.
The 8051 core establishes two I2C interfaces: one to the I2C devices into the TGR/DCC board
and the second to the other 4 FX2LP microprocessors each one belonging to a different DAQ
board. The TGG/DCC FX2LP compiles all monitoring data information from the DAE system
and sends the report to the acquisition/image reconstruction computer. In the 5th chapter a more
detailed description of the monitoring circuits will be provided.
The next section provides a detailed description of the firmware design of the FX2LP, including
timing operation, USB configuration, image reconstruction computer drivers, data frame
formatting and handshake protocol.
46
4.3 Firmware design of the USB interface
In this section the FX2LP microprocessor handshake protocol with the TGR/DCC FPGA and the
image reconstruction computer is presented.
4.3.1 USB interface protocol with the TGR/DCC FPGA
As described in the previous section, the TGR/DCC FPGA sends relevant imaging data through
a FIFO interface inside the FX2LP microprocessor. This FIFO is directly connected to 2 USB
endpoints (2 and 6), avoiding the 8051 core which would degraded the performance.
Figure 4.5 shows the interface pins and Figure 4.6 shows the timing diagrams for synchronous
writes in the FIFO.
Figure 4.5 – FX2LP synchronous FIFO write interface pins.
Figure 4.6 – FX2LP synchronous FIFO writes timing diagram.
47
Figure 4.7 and Figure 4.8 shows the interface pins and the timing diagrams for synchronous
FIFO reads, respectively.
Figure 4.7 – FX2LP synchronous FIFO read interface pins.
Figure 4.8 – FX2LP synchronous FIFO read timing diagram.
The FIFO interface is synchronous (could be set to asynchronous) and the EZ-USB is
configured to use an internal clock source, the IFCLK. This clock is set to its maximum
frequency at 48 MHz.
The FX2LP has 4 programmable flags. The handshake protocol uses 2 flags, providing
essential information to the TGR/DCC FPGA:
1. Endpoint 2 FIFO Empty Flag
2. Endpoint 6 FIFO Full Flag
When endpoint 2 FIFO Empty flag is not set, means that there is a command form the image
reconstruction computer to be attended. Basically, the TGR/DCC FPGA when idle, waits for a
48
command provided by the image reconstruction computer and thus the computer acts as master
of the communication.
The TGR/DCC FPGA answers to the image reconstruction computer commands through
Endpoint 6. If endpoint 6 FIFO Full flag is active, the TGR/DCC FPGA must wait for the image
reconstruction computer to read the previous commands that are in the FIFO waiting to be
attended.
The TGR/DCC FPGA controls the Slave Output Enable (SLOE) pin, Slave Read (SLRD) pin,
Slave Write (SLWR) pin, Packet End (PKTEND) pin and FIFO Select (FIFOADR[1:0]) pins.
SLOE and SLRD are active-low, and the FIFO pointer is incremented on each rising edge of
IFCLK while SLRD is asserted. The SLOE pin enables the data outputs. In synchronous mode,
when SLOE is asserted, the data lines are driven with the data that the FIFO pointer is currently
pointing to. The data is pre-fetched and is output only when SLOE is asserted. SLWR is active-
low and in synchronous mode, while SLWR is asserted, data is written to the FIFO (and the
FIFO pointer is incremented) on each rising edge of IFCLK. PKTEND is active-low and the
TGR/DCC FPGA asserts the PKTEND pin to commit an IN packet to USB regardless of the
packet’s length, signalizing the end of a packet/frame. Since the protocol has different frame
sizes this feature is mandatory. The FIFOADR[1:0] pins select which out of the four FIFOs is
connected to the data lines (FD bus). If the TGR/DCC FPGA is reading from the FIFO it must
select the endpoint 2 and if it is writing it must select the endpoint 6.
The following figure (Figure 4.9) shows the synchronous FIFO writes and reads state machine
diagram implemented in the TGR/DCC FPGA.
49
IDLE
STATE 1
Point to IN FIFO
Assert FIFOADR[1:0]
STATE 2
Endpoint 6
FIFO Full?
STATE 3
Drive data on the bus
Assert SLWR
De-assert SLWR
STATE 4
More Data to Write?
STATE 4
Assert PKTEND
Yes
No
Yes
No
Wait for a Write Event
IDLE
STATE 1
Point to OUT FIFO
Assert FIFOADR[1:0]
STATE 2
Endpoint 2
FIFO Empty?
STATE 3
Drive data on the bus
Assert SLRD
De-assert SLRD
De-assert SLOE
STATE 4
More Data to Read?
STATE 4
Yes
No
Yes
No
Wait for a Read Event
Read State MachineWrite State Machine
Figure 4.9 – FX2LP Synchronous FIFO write and read state machine diagram.
50
4.3.2 USB interface protocol with the image reconstruction computer
4.3.2.1 FX2LP microprocessor Serial Interface Engine (SIE)
Every USB device has a Serial Interface Engine (SIE) which connects to the USB data lines (D+
and D-) and delivers data to and from the USB device. Figure 4.10 illustrates the SIE’s role:
1. Decodes the packet data
2. Performs error checking on the data using the transmitted CRC bits
3. Delivers payload data to the USB device (FX2LP).
Figure 4.10 – USB Serial Interface Engine (SIE).
USB transfers are asynchronous, meaning that they include a flow control mechanism using
ACK and NAK handshake packet data. The SIE indicates busy to the host by sending a NAK
handshake packet. When the USB device has successfully transferred the data, it commands
the SIE to send an ACK handshake packet, indicating success. If the SIE encounters an error in
the data, it automatically indicates no response instead of supplying an handshake packet data.
This instructs the host to retransmit the data at a later time.
To send data to the host, the SIE accepts bytes and control signals from the USB device,
formats it for USB transfer, and sends it over via D+ and D-. Because USB uses a self-clocking
data format, the SIE also inserts bits at appropriate places in the bit stream to guarantee a
certain number of transitions in the serial data. This is called “bit stuffing” and is handled
automatically by the FX2LP USB’s SIE.
51
USB defines four transfer types. These match the requirements of different data types delivered
over the bus:
1. Bulk: bursty, travelling in packets of 512 bytes or lower, has guaranteed
accuracy (due to an automatic retry mechanism for erroneous data), has built-in
flow control provided by handshake packets.
2. Interrupt: have packet sizes of 1 through 1024 bytes, interrupt endpoints have
an associated polling interval that ensures they will be polled by the computer
on a regular basis.
3. Isochronous: time-critical and used to stream data like audio and video,
contain up to 1024 bytes. Time of delivery is the most important requirement for
isochronous data. In every USB frame, a certain amount of USB bandwidth is
allocated to isochronous transfers. To lighten the overhead, isochronous
transfers have no handshake (ACK/NAK/STALL/NYET), no retries and error
detection is limited to a 16-bit CRC.
4. Control: configure and send commands to a device, employ the most extensive
USB error checking and the computer reserves a portion of each USB frame for
Control transfers.
For the PEM project the FX2LP only uses the control transfer type, always necessary for FX2LP
configuration, and the bulk transfer type, for sending relevant imaging PEM data and DAE
monitorization data. The choice for bulk transfer instead of isochronous transfer is made
because bulk transfer has guaranteed accuracy through an error detection mechanism, absent
in isochronous transfer. Interrupt transfer is not bursty, thus, inadequate for fast data transfer
required for PEM system.
4.3.2.2 FX2LP microprocessor endpoint buffers configurations
FX2LP microprocessor contains three 64-byte endpoint buffers, plus 4 KB of buffer space that
can be configured 12 ways, as indicated in Figure 4.11.
52
Figure 4.11 – FX2LP microprocessor endpoint buffers configurations.
The three 64-byte buffers (endpoint 0 in/out, endpoint 1 in and endpoint 1 out) are common to
all configurations and are used for USB configuration purposes. Endpoints 2, 4, 6 and 8 can be
configurable in size. In simple way, the PEM system just needs 2 of those endpoints: one to
send data to the computer and one to get commands from the computer. Since the system has
2 types of data (imaging data and DAE monitoring data), it was decided to use the 4 endpoints:
2 for imaging data (direction in and out) and 2 for DAE monitoring data (direction in and out).
With this configuration, it is possible to simplify data formatting, at the expense of a buffer size
decrease. Some tests were made (section 4.5 for results) and the use of 512 bytes buffers
length, instead of 1024 bytes, didn’t cause degradation on the transfer rate. The selected
configuration was the first one, with 4 endpoints of 1 Kbyte each, divided in 512 bytes dual-
buffered. The use of the dual-buffer makes significant improvement over the data transfer rate,
because the TGR/DCC can be sending imaging data to a buffer while the image reconstruction
computer can collect data from the other buffer in parallel. If the endpoint had just 1 buffer,
these tasks could not be done in parallel. The use of a quad-buffer didn’t make any difference in
data transfer rate compared to the dual buffer configuration. Since the minimum size of a buffer
was 512 bytes, the quad-buffer configuration doesn’t allow 4 endpoints and so the option was
discarded.
53
4.3.2.3 FX2LP microprocessor Enumeration and Re-numeration
Inside every USB device is a table of descriptors. This table contains the device’s requirements
and capabilities. When you plug the USB, the computer goes through a sign-on sequence
called Enumeration:
1. Once a new USB device is attached, the computer sends a command to get the
new device descriptor.
2. The device responds to the request by sending identification data.
3. The computer assigns a unique address to the just-attached device so it may
be distinguished from the other devices connected to the bus.
4. The computer asks for additional device information (number of endpoints,
power requirements, required bus bandwidth, what driver to load, etc).
The FX2LP chip can take on the identities of multiple distinct USB devices.
When first plugged into USB, the FX2LP enumerates automatically and downloads firmware
and USB descriptor tables over the USB cable. Next, the FX2LP enumerates again, this time as
a device defined by the downloaded information. This two-step process, called Re-numeration,
happens instantly when the FX2LP is plugged in, with no sign to the user that the initial
download step has occurred. For the Re-numeration process the FX2LP has a small EEPROM
(Cypress 24LC00, see Annex 29) attached to the I2C bus containing the product identification,
allowing the computer to load the correct driver for the PEM project. Basically, the first driver
downloads the FX2LP firmware and the second driver recognizes the device for the image
reconstruction program.
Alternatively, FX2LP can also load its firmware from an external EEPROM, included in the
TGR/DCC board (Cypress 24LC128, see Annex 2). More information on the Enumeration and
Re-numeration process can be found in [40].
4.4 Establishing a high performance data rate for the USB
interface
The image reconstruction computer operates under Linux Operating System (OS). The image
reconstruction software is not the subject of this thesis. However, to test the USB connection
between the DAE system and the external computer several tests have been made to validate
the driver and the performance of the USB link. Drivers for Windows and Linux OS have been
developed and tested with software applications. This section describes the computer drivers
9 Public version of this dissertation has illegible schematics related to confidential cause.
54
and the software application and the next section (4.5) contains the data transfer rate
performance results obtained.
Cypress provides a device driver and Application Programming Interface (API) for Windows
platform. This device driver is used in the Re-numeration process and a custom driver was
developed for the enumeration process, to provide a mechanism of the firmware download of
the FX2LP microprocessor. As explained in previous section (4.3.2.3), when the FX2LP is
connected to a computer, it sends a unique identification. With this identification the computer
loads the custom device driver which downloads the firmware to the FX2LP (Enumeration).
After the Enumeration a reset is sent to the FX2LP microprocessor and then Windows loads the
Cypress default driver and the test windows application can use the Cypress’s API.
With Linux OS, a similar process is made, with a slight difference: the device driver is provided
by a third party company, called Jungo, with application software called WinDriver, because
Cypress (FX2LP microprocessor manufactory) doesn’t support Linux.
A software application was made to test the performance of the USN link. The software
application is shown in Figure 4.12.
Figure 4.12 – Software application to test the USB link performance.
In the next section some performance results of the USB link are presented, under different
configurations.
55
4.5 USB link data rate transfer results
Table 4.1 shows a compilation of the results obtain in the USB transmission tests, using
different FX2LP microprocessor firmware, endpoint and device driver configurations.
Table 4.1 – USB transmission tests results
Operating
System
Number of
Endpoints
Endpoints
Buffers
Size
(Bytes)
Endpoints
Buffers
Configuration
Device
Driver
Buffer
Size
(Kbytes)
Write
in
Disk
Data Rate
Transmission
(MBps)
Windows 2 1024 Dual 4 Yes 40
Windows 2 512 Quad 4 Yes 40
Windows 4 512 Single 4 Yes 28
Windows 4 512 Dual 4 Yes 40
Windows 4 512 Quad 4 Yes 40
Windows 4 512 Dual 2 Yes 35
Windows 4 512 Dual 16 Yes 40
Windows 4 512 Dual 4 No 42
Linux 4 512 Quad 4 Yes 45
The maximum data transfer rate was obtained under Linux OS, because with Linux, the
software application can access hardware buffers with maximum priority, while with Windows
that access has limitations. In Linux a 45 MBps transfer rate was obtained while in Windows 40
MBps data rate was achieved. The desired performance of 30 MBps was established in both
OS.
The best configuration was achieved with endpoint buffer of 512 bytes length, dual buffered
configuration and with a device driver buffer size of 4 Kbytes. Performance wouldn’t increase
with a larger buffer than 4 Kbytes.
A test was made in which no data was written to disk to verify the impact of disk access in
performance. With a normal hard disk drive, disk writing produces 5% of data transfer rate
degradation. With faster hard disks in RAID (Redundant Array of Inexpensive Disks), like the
configuration in the image reconstruction computer, the data rate degradation is imperceptible.
56
57
5 Test methodology and demonstration
results
In this chapter the DAE system test methodology and implementation is presented.
The DAE test concept is based on the verification of the correct functionality of the DAE system
in general and the major modules in particular, namely the DAQ boards, the TGR/DCC board,
the Compact PCI backplane (hardware support for the communications between the boards)
and the DAE system crate.
The tests to the DAQ and TGR/DCC boards can be divided into prototype validation internal
functionality tests of the FPGAs and monitorization of the system using the FX2LP
microprocessor that is inserted into each one of the DAE system boards.
The DAE module is a very complex electronic system. In order to deal with that complexity, the
test procedures should be modular and hierarchical. Testing has been separated in several test
modes: electrical test, functional and performance test and finally monitoring procedures that
can be used during lifetime.
In the following sections the methodology of the DAE system test and the test modes to achieve
prototype validation is described.
5.1 DAE system test methodology and procedures for
prototype validation
DAE system architecture is modular and highly hierarchical. Modularity and module reuse has
been adopted as a design strategy. This hierarchical and modular character of the design
significantly simplifies the verification and test of the DAE system. The developed test
methodology provides test experiments at the different hierarchical levels and takes advantage
at each level of module reuse, which simplifies and achieves more efficient test procedures. The
proposed methodology has three objectives: Prototype Validation, Diagnosis and Debug, FPGA
Built-In Self-Test (BIST) and finally Monitorization procedures.
Design and debug of the FPGA firmware (using software tools [19] [20] [41]) is used to validate
the software model.
Prototype validation is the process of verifying that the real hardware, for example configured
FPGA at component level, implements the specified functionality, at the desired performance
level. Component testing is carried out at board level, when the FPGAs are assembled in the
PCB. In fact, using programmable devices, usually such components are delivered with a very
low level of defects and consequently, when assembled in the PCB, such components are
assumed to be defect-free. Design and prototype validation is difficult for very complex systems
58
and to guarantee that there are no design flaws, the test solution must allow the design team to
build confidence that the design is thoroughly exercised, and thus complying with system
specifications and therefore the FPGAs hardware was implement with BIST [19][20]. Since the
DAE design specifications can change for future upgrades (as usually happens with highly
innovative products), reconfigurable platform-based design solution has been selected and
design incorporates Design for Debug (DfD) capability [19] [20].
Another relevant test objective is production testing at board level. Since the PEM system
prototype is to be used by medical staff in field experiments with patients, the demonstrator will
have to be fully evaluated and full compliance to biomedical security standards, and regulations
need to be guaranteed. As a consequence, a test solution to ensure that the prototype performs
its correct functionality and induces no harm needs to be devised. Such solution will consider
individual components with zero defects, and concentrate on testing individual boards as sub-
systems, and the prototype as a system.
A third test objective is lifetime testing. As the final system is to operate according to functional
and security specs over the product lifetime, periodic, off-line testing of the components, boards
and system is required. At FPGA level there are automatic procedures, like system self-test
during power-up. At board level there are monitoring procedures to check system integrity and
detect flaws.
Figure 5.1 shows the complete diagram of the tests procedures for a DAE board which is a 3
phase process: Prototype Validation, System Functionality and Performance, and finally
Monitorization.
Phase 1
Prototype Validation
Electrical Tests
Power Supply
1
JTAG Boundary
Scan Chain
2
LVDS Receiver
DAQ Sync Module
3
BIST
5
Transceivers and
Backplane
4
Monitor
Parameters
6
Special Signals
Transceivers
LVDS Receiver
FX2LP
Microcontroller
1A
1B
1C
1D
1E
Phase 2
System Functionality and
Performance
Phase 3
Monitorization
Figure 5.1 – Test procedures diagram.
59
In section 5.2 the prototype validation procedures at board level (DAQ and TGR/DCC boards)
can be found. Section 5.3 has a brief description of the functional BIST [19] and [20] and of the
work made to monitor and evaluate the BIST at FPGA and DAE system level.
Section 5.4 contains a description of the monitoring procedures for the DAE system.
5.2 Prototype Validation
Although the DAQ board is different from the TGR/DCC FPGA, prototype validation tests
described in this section will cover simultaneously both boards. The differences will be pointed
out as necessary.
The prototype validation tests include the electrical tests, LVDS receiver and DAQ Sync module
functional tests (only for the DAQ board), transceivers and backplane communications
functional tests and finally the functional tests of the FPGA modules. The following sections will
describe each one of these points.
5.2.1 Electrical Tests
The main goal of electrical tests is to detect, after the board is manufactured, flaws that cannot
be detected by the functional tests or through the BIST in the FPGAs.
The objective is to prepare the board for the functional and BIST tests and to guarantee that
from the electrical point of view the board is functioning.
5.2.1.1 Power Supply
The purpose of this test is to verify the power connections of the DAQ or TGR/DCC board. The
Table 5.1 shows the processes and scenarios for this test.
Table 5.1 – Power supply test processes and scenarios
Scenario Process Description
VDD test without the FPGAs
Generate VDD Power the DAQ or TGR/DCC board with 3.3 V before mounting the FPGAs
VDD test without the FPGAs
Check VDD Check the voltages (3.3 V, 1.8 V and 1.5 V) with a multimeter in several of the available points on the board.
Verify and take note of the power consumption for all voltages in two situations: Power up and On run.
VDD test with the FPGAs
Generate VDD Power the DAQ or TGR/DCC board with 3.3 V after mounting the FPGAs
VDD test with the FPGAs
Check VDD Check the voltages (3.3 V, 1.8 V and 1.5 V) with a multimeter in several of the available points on the board.
Verify and take note of the power consumption for all voltages in two situations: Power up and On run.
60
The first tests are done before the insertion of the FPGAs, preventing that a faulty condition
damages the devices.
5.2.1.2 Special Signals: Clock, Synchronization and Reset
The purpose of this test is to verify if the special signals (Clock, Synchronization and Reset) are
being properly provided into the DAQ or TGR/DCC board.
These signals are important for the DAE system functionality, therefore its correct behaviour
should be verified prior to functional tests.
Those signals are generated in the TGR/DCC board and therefore there is no need to externally
provide them in this board. On the other hand, to test if the special signals are flowing properly
in the DAQ board, they should be injected with a data generator through the cPCI backplane.
Table 5.2, shows the correct procedure to validate this test.
Table 5.2 – Processes and scenarios for the special signals test in the DAQ board
Scenario Process Description
CLK signal test Generate CLK Insert in the backplane the CLK signal (100 MHz square
wave) with the NI generator.
CLK signal test Check CLK Verify the CLK signal level in several of the available
points on the board with the NI Data Analyzer and take
note of the time delay for the signal inserted in the
backplane.
Sync signal test Generate Sync Insert in the backplane the Sync signal (1 KHz square
wave) with the NI generator.
Sync signal test Check Sync Verify the Sync signal level in several of the available
points on the board with the NI Data Analyzer and take
note of the time delay for the signal inserted in the
backplane.
Reset signal test Generate Reset Insert in the backplane the Reset signal with the NI
generator.
Reset signal test Check Reset Verify the Reset signal level in several of the available
points on the board with the NI Data Analyzer and take
note of the time delay for the signal inserted in the
backplane.
61
5.2.1.3 Transceivers
The purpose of these tests is to verify if the transceivers are functioning correctly. The DAQ
board should be powered and the DAQ FPGAs should not be inserted.
The FPGA connectors should be used to control the data direction of the transceivers if
possible. In some of the transceivers, a square wave of 50 MHz data inputs should be injected,
being then compared with the output wave of the transceivers and the time delay should be
registered. In the bidirectional transceivers the test should be done in both directions.
To generate and verify the necessary signals, a Generator/Data Analyzer should be used.
5.2.1.4 LVDS Receiver
This test is only applied in the DAQ board and is meant to verify if LVDS connections are
working correctly. The DAQ board should be powered and the DAQ FPGAs should not be
inserted and basically a Generator/Data Analyzer is used as LVDS transmitter. The LVDS input
signals are generated by the Generator and the output signals integrity and time delay at the
LVDS receiver should be verified by the Data Analyzer.
With the Generator a burst with a known length of digital signals is sent through the LVDS
transmitter and the output of the LVDS receiver is verified using the Data Analyzer to check the
burst length.
5.2.1.5 FX2LP Microcontroller
The purpose of this test is to verify if the FX2LP microprocessor in the DAQ or TGR/DCC board
is correctly connected and working properly.
The test consists in loading the firmware into the FX2LP microprocessor and check if it
responds to USB commands and if its main signals are present: clock and reset. If the
monitorization procedures of the FX2LP microprocessor are finalized with success, a led in the
front panel should be on.
5.2.2 JTAG Boundary Scan Chain
In order to facilitate the production test procedures, it was established that, where possible, all
components would be BST or JTAG (IEEE 1149.1) compliant.
The backplane chosen for this board already has a 5-pin connector and 5 bus lines enabling
this feature. Likewise, each DAQ and TGR/DCC board has its own JTAG connector, enabling
the use of the standard on a stand-alone basis. Figure 5.2 and Figure 5.3 show the DAQ board
and TGR/DCC board JTAG circuit respectively.
62
Figure 5.2 – DAQ board JTAG circuit.
Figure 5.3 – TGR/DCC board JTAG circuit.
The JTAG circuit inside the DAQ and TGR/DCC boards has 2 chains: one connects the JTAG
features of the E2PROM and FPGAs and the other connecting the two transceiver firewalls.
TDO
TDI
TMS
TCK TDI
TDO
TMS
TCK
Vref
GND
TDI
TDO
TMS
TCK
TDO
TDI
TMS
TCK
TDO
TDI
TMS
TCK
TDO
TDI
TMS
TCK
TDO
TDI
TMS
TCK
EN
E2PROMTGR/DCC
FPGA
DBUS
TRANSCEIVERS
GBUS
TRANSCEIVERS
GBUS
TRANSCEIVERS
VCCAUX
VCCAUX
TDI, TDO
TMS, TCK, Others* All resistors are 68R
All capacitors are 100pF
JP1
JP2
JP3
JP5
VCCAUX
TDO
TDI
TMS
TCK
Transfer Board
4k7
stand-alone
connector
backplane
connector
JP4
TDO TDI
TMS TCK TDI
TDO TMS TCK
Vref GND
TDI TDO TMS TCK
TDO TDI
TMS TCK
TDO TDI
TMS TCK
TDO TDI
TMS TCK
TDO TDI
TMS TCK
TDO TDI
TMS TCK
EN
E 2 PROM FPGA 1 FPGA 2
DBUS TRANSCEIVERS
GBUS TRANSCEIVERS
GBUS TRANSCEIVERS
VCCAUX
VCCAUX
TDI , TDO
TMS , TCK , Others
JP 1
JP 2
JP 3
JP 4
VCCAUX VCCAUX
4 k 7 4 k 7
stand - alone connector
backplane connector
63
This enables, by means of jumper configurations, the connection (and programming) of the
FPGAs only.
The backplane and stand-alone connector connect in parallel to the TDI-TDO circuit because
they are never used at the same time.
Since the complete circuit is in fact a very long chain (18 chips for the DAQ board and 12 chips
for the TGR/DCC board), it is advisable to use a buffer circuit for the TMS and TCK signals as
well as line termination RC networks.
The JTAG circuits as an important role in the DAE boards because provides the possibility to
load the firmware into the FPGAs or the respective EEPROM.
Figure 5.4 shows Xilinx (FPGAs manufacturer) software has been used to detect 3 devices on
the DAQ board JTAG chain, 1 E2PROM and 2 FPGAs. If the JTAG chain is detected properly
and if the EPPROM and the FPGA load successfully the firmware it’s highly probable that the
FPGA and EEPROM hardware circuits are working [37] [41] [42].
Figure 5.4 – Software detection of the DAQ board JTAG circuit.
The backplane chosen for this board already has a 5-pin connector and 5 bus lines enabling
this feature. Likewise, each DAQ and TGR/DCC board has its own JTAG connector, enabling
the use of the standard on a stand-alone basis.
Test documentation was produced in [42] [43] that explain in detail the procedures to load the
FPGAs firmware. This subject is also referred again in the last section (section 5.5) of this
chapter.
As previously referred, the JTAG chain complete configuration includes not only the FPGAs and
EEPROM circuit, but also the transceivers and buffers.
Since the cPCI backplane has a JTAG connector, the JTAG chain can be configured as board
internal or for all DAE boards. When the DAE system is complete the JTAG chain passes
through the backplane and it is possible to detect simultaneously all Boundary Scan integrated
64
circuits in the DAE system. As will be pointed out in chapter 7, this can be used to promote
boundary scan tests to the complete DAE system, in an efficiently way, making possible to
detect hardware flaws. This kind of test procedures will be done in future work.
5.2.3 LVDS Receiver and DAQ Sync module Functional Tests
The LVDS receiver and DAQ sync module functional test target the validation of the Sync
Module of the DAQ FPGA. As referred in [19][20] this test cannot be done in the DAQ FPGA
BIST, since it requires external signalling to be provided to verify if the DAQ Sync module is
working properly.
According to [9], the following signals: Clock, Synchronization, Data test vector that simulates a
digitized channel from the FE are required to validate the DAQ Sync Module, the DAQ FPGA.
Those signals are generated in a Data Generator and should be connected directly to the DAQ
FPGAs through the backplane and from 2 different LVDS chips.
To introduce phase difference between the 2 tested channels, 2 cables with different lengths
have been used.
It is assumed that the BIST scenarios are working correctly and that the FPGA hardware is
functionally correct. The figure below (Figure 5.5) shows the connections required to execute
the Transceivers/DAQ Sync test.
LVDSEmitter
LVDSReceiver
LVDSEmitter
LVDSReceiver
Cable A
Cable B
SyncDAQ
Module
SyncDAQ
Module
FPGA
ChX Data
ChX Data
ChX Sync
ChX Sync
ChX CLK
ChX CLK
Sync
CLK
DATA 10
Backplane
SyncCLKFrom Backplane
From Data
Analyser
Cable A Lenght <> Cable B Lenght
DAQ Board
Figure 5.5 – Connections for the transceivers and DAQ Sync test.
Despite the fact that the DAQ Sync test is limited to 2 fixed channels simultaneously, this is
enough to verify the correct Sync –> DAQ linking and counters. Those 2 input channels from
different LVDS channels are analyzed and compared with the corresponding outputs.
The pulse shape, in data test vector in the LVDS channels, has no restrictions. The test vector
has 11 bits per channel, but we can use only 1 variable bit and fix the other 10 bits. If the data
65
from the LVDS channels is periodic, the time period should be greater than the time resolution
of the DAQ Sync module (the resolution of the synchronism counters (default: 4bit)).
There are 2 testable scenarios:
1. Scenario 1: the active channels should belong to the same DAQ module but
associated with 2 different LVDS chips.
2. Scenario 2: the active channels should have the same identification but be
associated with different DAQ modules and different LVDS chips.
For each DAQ board there are 4 Sync modules and each Sync module receives data from 2
LVDS chips. Each LVDS chip has 4 channels. 4 LVDS chips are connected to each FPGA.
Each LVDS pair of chips cross connects the same modules.
To verify the test, the DAQ FPGAs make a report using the test connectors and a Data Analyzer
is used to read the correspondent results. For a "go/no go" test, the DAQ FPGA uses the led as
its output. A Labview application has been developed to control the entire test and to control
Data Generator, i.e., the test vectors.
Figure 5.6 show a picture of the Labview software application that controls the DAQ Sync
module test.
Figure 5.6 – Labview software application to control the DAQ Sync module test.
5.2.4 Transceivers and Backplane Communication Functional Tests
The purpose of these tests is to electrically verify if the tristate buffers and the transceivers that
interconnect the DAQ FPGAs and the Compact PCI backplane are functionally correct.
The connections between the DAE system FPGAs are made by 3 buses: Generic Bus,
Dedicated Bus and Test Bus. The objective is to test all the communications paths of the 3
buses.
In this case the test connector in the DAQ and TGR/DCC board is used to initiate the test.
Afterwards the DAE FPGAs generate the communication signals through all lines to establish a
communication protocol.
66
The DAE FPGAs know their position in the backplane and after the communications testing, a
report is sent through the test connectors. A Data Analyzer is used to read the correspondent
results. If all test vectors are sent and received correctly, the transceivers are working correctly.
If any signal is lost, the report will indicate it.
5.3 Test procedure to validate the DAE FPGAs BIST
As described in [19] and [20] the DAE FPGAs implements functional Built-in Self-Test (BIST).
The procedures to test and validate the DAE boards functionality (including the FPGAs BIST) is
in the scope this of dissertation.
The built-in test modules of the different FPGA must support the fulfilment of two main
objectives:
1. To verify the correct DAE system functionality and performance;
2. To diagnose errors and to debug the DAE system, or subsystems when errors
are detected.
The DAE system is tested in order to guarantee that system functionality (functional test) is
correctly implemented and that timing requirements (performance test) are verified. This
purpose is achieved in two steps:
1. Functional Test is carried out at low frequency. By doing so, it is expected that
no timing errors occur. Therefore, if errors are detected, most probably, they
correspond to functional errors.
2. Performance Test is carried out at working frequency. Once the functionality
correctness is verified at low speed (the first step), the correctness of the
system functionality is verified again at nominal speed. If errors are detected,
they correspond to timing errors.
In each step, a hierarchical test procedure is followed. By doing so, system functionality of the
entire system is tested as well as the most important constituting system modules. The result of
the test allows the identification of the sources of errors and the verification of the correctness of
critical information flow along the corresponding paths. Since sources of errors are identified,
diagnosis can take place at modular level.
A major problem with non-formal functional verification is the difficulty in guaranteeing its
completeness, since exhaustive verification is prohibitive. In order to guarantee that functional
verification is as complete as possible, the built-in Test Modules are designed to cover all the
different scenarios that correspond to the different operation modes of the DAE system.
67
At system level, 5 operation modes have been defined for the DAE and are associated with 5
scenarios:
1. Normal/Random Mode Scenario: the system detects Normal and Random
coincidence events and sends them to the image reconstruction computer.
2. Single Mode Scenario: the system detects Single Photoelectric events and
sends them to the image reconstruction computer.
3. Constant Parameters Loading Scenario: the system receives the constant
parameters from the image reconstruction computer and distributes them to the
relevant processes.
4. Operation Mode Loading Scenario: the image reconstruction computer sends
two control signals with the information of the operation mode.
5. Error Request Scenario: the image reconstruction computer sends a control
signal requiring the number of operational errors that occur and are functionally
identified by the system. These errors are: Front End Channel overflow, signal
strength overflow and GBUS communications error.
A scenario is defined as the set of processes and corresponding data control flow that represent
the complete execution of the functionality in a given operation mode. Scenarios can be
activated for prototype validation and BIST.
A process is defined as a set of functions that carry out a given complete functionality.
These tests were first thought to be used as BIST and the described scenarios are tested in
sequence. After the tests, a report is produced, indicating possible errors in some of the tested
modules. For a test of the type "go/no go", the FPGA uses the led as its output. To start the test,
a word with the coding of the test is input through the FPGA test connectors. Then, the FPGA
runs, in sequence, all the possible test scenarios and finally generates the correspondent report
by sending it to the test connectors. To input the test word and to read the report generated the
Generator/Data Analyzer is used.
A communication protocol was established between the FPGAs to gather the BIST report. A
Labview software application was made to communicate with the DAE FPGAs, to initialize and
show the BIST report. Figure 5.7 shows the Labview application, built to collect the BIST report.
68
Figure 5.7 – Labview software application to gather BIST report.
Details about the communication protocol established between the DAE FPGAs and the
Labview application and the BIST report can be found in [37] [42]-[47].
The following table (Table 5.3) illustrates the commands send to the DAE FPGAs to gather
BIST status.
Table 5.3 – Commands to collect DAE FPGAs BIST status.
Command Function
85 DAQ or TGR/DCC/FLTR sequential and deterministic test result
8C DAQ or TGR/DCC /FLTR random test result
A3 DAQ_ROC or TGR/DCC_ROC test result (part 1)
A4 DAQ_ROC TGR/DCC_ROC test result (part 2)
If the results are positive, it means that DAE board under test (DAQ or TGR/DCC) passed the
BIST. If not, there is a detailed report explaining what went wrong.
5.4 DAE system monitoring procedures
As described in sections 3.2.1 (DAQ control block) for the DAQ board and in section 3.3.1
(TGR/DCC control block) for the TGR/DCC board, the DAE system has a control system for the
monitorization of several parameters like the FPGAs and boards currents, voltages and
temperatures. That control system is centralized in the Cypress FX2LP microprocessor.
69
Each DAE board has a FX2LP microprocessor that establishes an internal I2C bus
communication to several control devices to monitor the 3.3V, 1.8V and 1.5V power supplies,
the FPGAs currents, consumptions and temperatures.
In the DAQ boards, the FX2LP microprocessor communicates with the TGR/DCC Board’s
FX2LP microcontroller (system master) through an external I2C bus (using the cPCI backplane
Dedicated bus), gathering monitorization data for all the DAQ boards and serving as a gateway
to the image reconstruction computer, which also functions as monitoring computer.
The FX2LP microprocessor in the DAQ boards can also communicate directly the monitoring
values to an external computer through a USB or RS232 connection. Thus, this is only used for
debug during the prototype validation period.
The I2C block diagram of the DAE system is shown below (Figure 5.8).
Figure 5.8 – DAE system I2C block diagram.
Electrical connections of the monitoring system for the DAQ and TGR/DCC boards are shown in
Figure 5.9 and Figure 5.10, respectively.
uC
ADC
Temp
FPGA1
NVRAM
Temp
FPGA2
Ext
Conn*
uC
ADC
Temp
FPGA1
NVRAM
Temp
FPGA2
Ext
Conn*
uC
ADC
Temp
FPGA1
NVRAM
Temp
FPGA2
Ext
Conn*
uC
ADC
Temp
FPGA1
NVRAM
Temp
FPGA2
Ext
Conn*
DBUS
uC
ADC
Temp
FPGA
NVRAM
Ext
Conn*Acquisition
PCUSB / RS232
DAQ Board #4DAQ Board #3DAQ Board #2DAQ Board #1
TGR/DCC Board
Notes:
- The external connectors on the DAQ Boards are
for debug purposes only;
- All the connections represented here (except the
connection to the Acquisition PC) are composed
by 3 wires: SDA, SCL and GND.
70
Figure 5.9 – Electrical schematics of the DAQ board monitoring system.
71
Figure 5.10 – Electrical schematics of the TGR/DCC board monitoring system.
2 EEPROM chips are connected to the microcontroller: a smaller one for storing some
identification data and a larger one for future monitoring data storage.
72
The microcontroller uses a 128 Kbyte external RAM (GS71108T form GSI Technology). In fact
the microprocessor only addresses 64Kbyte of that memory. The remaining part is unused.
To help the microcontroller in the current and voltages measurement, an analog-to-digital (ADC)
circuit is needed. The Maxim MAX1139, multiplexing 12 channels, with a 10-bit resolution and
up to 100k samples per second was chosen. This chip communicates with the microcontroller
via an I2C bus.
The current in each voltage branch is sensed using current sensors from Maxim, the MAX4172.
It measures the voltage drop on a very low value, high precision resistor inserted in the power
supply circuitry. The value measured is converted and amplified for the range to be compatible
with the MAX1139 ADC inputs.
The temperature measurement in both FPGAs is obtained through two Maxim chips
(MAX1617), one for each FPGA and sensed using the FPGA’s internal temperature sensing
diode. These chips are recommended by Xilinx.
The acquisition/monitoring computer gather monitorization data from the TGR/DCC board
FX2LP microprocessor using a USB or RS232 connection. 2 software applications were
developed to illustrate the monitorization data.
Figure 5.11 shows the layout of a software application developed in Microsoft Visual Studio C#
.NET.
Figure 5.11 – DAE monitorization application software developed in Microsoft Visual Studio C#.NET.
73
Figure 5.12 shows the layout of a software application developed in Labview.
Figure 5.12 – DAE monitorization application software developed in Labview.
In figures 5.11 and 5.12 the monitorization tests were made during the BIST. All the results are
within the required range [42] – [47]: Voltages with less than 3% of error, temperatures less than
50ºC and FPGAs currents less than 1,5 A for the DAQ boards and less than 0,8 A for the
TGR/DCC FPGA.
For the functional tests, one of these programs should be run to verify the functionality of the
power supplies and to check if the FPGA exhibits the expected temperature and consumption.
5.5 DAE system test documentation
In complex projects like the PEM system, where too many different entities, developers and
researchers are involved, documentation has an important role in the project management and
information sharing.
Because PEM system is a research project, its first intention is to accomplish a prototype
evaluation. However, it is intended by the consortium that the PEM system will become a
commercial product.
As a medical equipment, certification is required, even at a prototype evaluation process, since
during this stage it is planned that first medical test in human patients will be executed.
74
For those reasons, documentation is a key procedure, not only to reach prototype validation, but
also to attain medical certification and product documentation.
A big effort has been made by all partners in the PEM system consortium to produce and
update all the project documentation.
Concerning the DAE subsystem of the PEM system, several important documents have been
released during development stage with my contributions.
The first released document was the “DAE Specification Document”, where the DAE
requirements were established.
After the DAE requirements had been established, the “DAE Design Document” was produced
and that was where the DAE system has started to be built. The first board to be designed and
assembled was the DAQ board, followed by the DAE crate and finally the TGR/DCC board.
During this stage, several documents were released: “DAE Interfaces Document”, that
established the interfaces with the FE system and the acquisition computer; “DAE Configuration
Document”, holding the configuration description of the DAQ and TGR/DCC boards.
Regarding the DAE test, the “DAE Test Specification Document” [37] was written, stating the
test methodology and test procedures for the electrical tests, functional tests, FPGAs BIST and
prototype validation tests.
The “DAE Test Specification Document” has been updated while the DAE system has been
developed and concerning the DAE test several other documents have been released:
1. DAQ Board Electrical and Functional Tests
2. DAQ Board Production Test Specification Document
3. DAQ Board Test Report
4. DAE LVDS Transmission Tests
5. TGR/DCC Board Electrical and Functional Tests
6. TGR/DCC Board Production Test Specification Document
7. TGR/DCC Board Test Report
These documents are related with the “DAE Test Specification Document” [37] and most of
them represent reports of the work that has been developed.
The DAQ board and TGR/DCC board electrical and functional test and production documents
represents guidelines to test the DAE boards that come from production.
The test report document represents the test results details of the produced DAE boards.
The “DAE LVDS Transmission Tests” is a report of the results achieved with the LVDS data
transmission between the DAE sub system and the FE sub system.
Several other documents have been carried out, some not finish yet, mostly regarding data
transmission between the DAE system and the image reconstruction/acquisition computer.
75
6 Conclusions
The system described in this dissertation was conducted under an innovative project to develop
a system for diagnosis of malign neoplasm in the breast and of ganglion loco-regional invasion,
based on PET (Positron Emission Tomography) technology, with a very accurate spatial
resolution and sensitivity. The system is generally designated by PEM system (Positron
Emission Mammography).
The results of medical examinations performed by PET systems are images of cancerous cells
found inside the human body. These images are produced by computer software programs of
image reconstruction, which need millions of points to produce images with high resolution.
To achieve a better image resolution in PEM system, an electronic system of great complexity
has been developed allowing the processing of huge amount of data sent by PEM detectors
within a reduced time window.
The work in this dissertation focuses on the Data Acquisition Electronics (DAE) system of the
PEM, which is a very complex hardware/software system, capable of process and filter a huge
quantity of information and transmit it to an external computer with acceptable data rates.
The DAE system uses FPGA (Field-Programmable Gate Array) technology, with high capacity
and speed, being these the devices responsible for filtering data to obtain relevant information,
data organization and delivery and system calibration.
The work described in this dissertation involves the concept, design, test and validation of the
DAE hardware system, as well as the design and implementation of a bidirectional Universal
Serial Bus (USB) data link, which allows the routing of data between the DAE and the image
reconstruction computer.
This dissertation describes also the methodology and environment that has been developed in
order to validate the complex hardware/software of the DAE system, including the development
of methodologies to identify and specify the requirements and features to be validated and
tested. This dissertation also describes the development of the test, validation and monitor
environment of the DAE system and documentation for prototype validation.
An additional aspect that conditioned the development of the DAE hardware system was the
fact that the project has been conducted simultaneously with the theoretical studies on many
aspects of physics underlying the functioning of the PEM system. That fact has made the
system requirements, functional and performance rather unstable during virtually the entire
process of project development.
To cope with this situation, the DAE system was developed with a modular and hierarchical
architecture. This also motivated the choice of the FPGA (Field-Programmable Gate Array)
technology that allows the easy modification of project functionalities to meet shifting
requirements.
76
A very modular system was built with 4 Data Acquisition (DAQ) boards, each one having two 4
million gates FPGAs, and a Trigger and Data concentrator (TGR/DCC) board with a 3 million
gates FPGA. The boards where interconnected via 2 Compact PCI (cPCI) backplanes and the
entire system was enclosured in a complete crate.
LVDS connections were established between the DAE system (through the DAQ boards) and
the Front-End electronics system. A USB connection was used to establish the connection
between the DAE system (through the TGR/DCC board) and the image reconstruction
computer.
System functionality and desired performance were obtained with the developed system and
prototype validation has been achieved.
This modular and hierarchical nature also enabled the development of a test methodology of the
DAE system, becoming easier to attain prototype evaluation.
All the required specification, design and test documentation of the DAE system was designed
to simplify the future process of medical certification and product manufacturing.
77
7 Future Work
The PEM system described in this dissertation is a prototype. It has been developed to prove
the concept. The main idea is to turn this prototype into a product expandable to include new
functionalities. A modular architecture has been designed in order to easily improve system
features.
In this chapter, some guidelines to improve the PEM DAE system design and test procedures
are presented. These modifications are intended to improve performance and also to match the
PEM system’s design to other applications, namely:
High performance PEM system
Whole body high resolution PET
Brain PET
PEM system combined with Ultrassonography
Small animal PET
Some of these improvements are the result of technological advances and others are the result
of architecture improvement. In the following paragraphs some specific ideas for system
improvement in the areas of design and test are outlined.
Backplane
The 5 slot CompactPCI backplane (64 bit with 66 MHz) chosen for the PEM system, was in the
beginning of the project the best choice, because it was fast, low cost and had the necessary
slots for the DAE boards. It’s capable of handling 266 MBps data transfer. To achieve a larger
bandwidth, a PCI-X backplane (bandwidth of 1064 MBps in a 64 bit and 133 MHz configuration)
or a PCI-E backplane (bandwidth of 4 GBps in a 16x configuration) can be used. Therefore, for
future developments, the PCI-X and PCI-E solutions are obvious solutions. A main problem that
can emerge from these solutions is the limited number of slots available (5 slots). This may be a
problem for some new PEM applications that require more DAQ boards. The ACTA (or uTCA)
bus meets all those requirements, with a maximum bandwidth of 5 GBps and a virtually
unlimited number of slots. In fact, there are ATCA commercial backplane solutions with 30 slots.
Backplane Protocol
As described in this document, the present PEM DAE solution uses a CompactPCI backplane
for data communication between the DAE boards. Nevertheless, the protocol used was custom,
since the implementation of the PCI protocol in the DAE FPGAs would take too much
development time and most of its features were not to be used. However, future versions of the
PEM system should use the default protocol of the backplane, making it possible the full
integration of the backplane (and the whole DAE system) in a commercial computer system.
78
Optical link connecting the DAE system and the acquisition computer
The DAE system communicates with the image reconstruction computer through a USB link.
This link has a slow data rate. However, the TGR/DCC has a mezzanine connector to plug a
card, which provides the bridge between the TGR/DCC FPGA and the image reconstruction
computer. That card should be an optical link, providing more bandwidth and larger cable length
connection capability (in comparison with electrical connection).
Optical link connecting the DAE and the FE system
An optical link solution similar to the bridge between the DAE system and the image
reconstruction computer should be made between the FE and DAE system, instead of the
actual electrical LVDS connection. The benefits of the optical connection are the larger
bandwidth and the greater cable distance capacity provided.
Boundary Scan test in all DAE circuits
Boundary Scan is a method for testing interconnects (thin wire lines) on printed circuit boards or
sub-blocks inside an integrated circuit. The majority of the DAE board’s circuits have Boundary
Scan capability. This should be used to detect hardware flaws in the boards, during board
validation and lifetime use. Test vectors should be programmed to test interconnects and
clusters of logic in the DAE system, using the existing JTAG chain in the DAE boards.
BIST report through a connection between the FPGAs and the FX2LP
microprocessor
Grabbing the BIST report of the DAE FPGAs is a complex task, since it requires external
hardware. If a communication protocol is established between the DAE FPGAs and the FX2LP
microprocessor (a serial interface is hardwired and could be used), the BIST request could be
made, by the image reconstruction computer, using the FX2LP microprocessor as a bridge and
the report would be sent back to the computer. This solution can provide continuous monitoring
of the DAE FPGAs functionality and performance.
79
Annex 1 – DAQ Board Electrical Schematics
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
Annex 2 – TGR/DCC Board Electrical
Schematics
96
97
98
99
100
101
102
103
104
105
Annex 3 – DAQ and TGR/DCC FPGA Pin out
Distribution
Figure 7.1 – DAQ FPGA’s coarse pin distribution.
1 2 3 4 5 6 7 8 8 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
AH
AJ
AK
AL
1 2 3 4 5 6 7 8 8 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
AH
AJ
AK
AL
BANK 0BANK 1
BANK 5BANK 4
BANK 3
BANK 2
BANK 6
BANK 7
Top View
User I/O
Special Function (reserved)
PIN LEGEND:
Power Supply
No Pin
Dedicated Bus (DBUS)
Front-End (FE)
FUNCTION LEGEND:
Generic Bus (GBUS)
Test Connector
Power Supply
106
Figure 7.2 – DAQ FPGA pin distribution.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
AH
AJ
AK
AL
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
AH
AJ
AK
AL
BANK 0BANK 1
BANK 5BANK 4
BANK 3
BANK 2
BANK 6
BANK 7
LVDS4_CH3
LVDS4_CH4
LVDS4_CH2
LVDS4_CH1
LVDS4_SYNC/CLK
LVDS3_SYNC/CLKDBUS_RD1/2
DBUS_REQ
SYS_DAQ_ID
DBUS_GRT
DBUS_DATA
DBUS_WR1/2
DBUS_RJS_IN1/2
DBUS_DAS
DBUS_RJS_OUT1/2
TST_DATA
GBUS_DIN (0..47)
LVDS2_SYNC/CLK
LVDS1_SYNC/CLK
LVDS1_CH4
LVDS1_CH3
LVDS1_CH1
LVDS1_CH2
GBUS_DAS_D
GBUS_GRT
GBUS_REQ
GBUS_DIN(48..63)
GBUS_RD
SYS_SYNC
GBUS_WR
GBUS_DAS_T
LVDS2_CH4
LVDS2_CH3
LVDS2_CH2
LVDS2_CH1
LVDS3_CH4
LVDS3_CH3
LVDS3_CH2
LVDS3_CH1
DAQ FPGA PIN FUNCTIONS
Xilinx XC2V4000-BF957
LVDS 4
LVDS 3
LVDS 2
LVDS 1
Dedicated
Bus
Test
Connector
Generic Bus
Top View
Unused I/O
User Clock (GCLK)
PIN LEGEND:
Used I/O
GND
VCCINT
VCCO
VCCAUX
JTAG
Config / Special
No Pin
SYS_CLK
RFA_CLK2
SYS_CHIP_ID
SYS_I2C
SYS_STATUS_LED
SYS_TEST_BUS
To LVDS chip x, channel y
FUNCTION LEGEND:
LVDSx_CHy
LVDSx_SYNC/CLK
TST_DATA
To LVDS, sync/clk signals
To Dedicated Bus
To Generic Bus
DBUS_funct
GBUS_funct
SYS_funct DAQ System Function
To Test Connector
107
Figure 7.3 – TGR/DCC FPGA’s coarse pin distribution
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
BANK 1BANK 0
BANK 6
BANK 7
BANK 3
BANK 2
Top View
BANK 4BANK 5
User I/O
Special Function (reserved)
PIN LEGEND:
Power Supply
No Pin
FUNCTION LEGEND:
Dedicated Bus (DBUS)
Generic Bus (GBUS)
PCI Interface
108
Figure 7.4 – TGR/DCC FPGA pin distribution.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
A
B
C
D
E
F
G
H
J
K
L
M
N
P
R
T
U
V
W
Y
AA
AB
AC
AD
AE
AF
AG
BANK 1BANK 0
BANK 6
BANK 7
BANK 3
BANK 2
PCI_FIFO_DATA(0..31)
TGR/DCC FPGA PIN FUNCTIONS
Xilinx XC2V3000-BG728
PCI FPGA
Dedicated
Bus
Test
Connector
Generic Bus
Top View
Unused I/O
User Clock (GCLK)
PIN LEGEND:
Used I/O
GND
VCCINT
VCCO
VCCAUX
JTAG
Config / Special
No Pin
BANK 4BANK 5
PCI_INT
PCI_FIFO_DATA(32..63)
PCI_FIFO_FLAGS
PCI_CTRL*
PCI_FIFO_DATA_CTRL**
PCI_USER_CTRL***
PCI_USER_DATA(0..32)
TEST_BUS
DBUS_DAQ_1_REQ(1..4)
DBUS_DATA1(0..18)
(*) PCI_CTRL includes:
CTRL_ADDR[3..7]
CTRL_CS
SPCI_DONE
(**) PCI_FIFO_DATA_CTRL includes:
SRC_SEL(0..1)
DATA_BE(0..7)
DATA_BYTESEL(0..2)
DATAIN_BYTEID(0..1)
DATA_CS
RD_WR
(***) PCI_USER_CTRL includes:
USER_REG(0..2)
USER_STOP
USER_BEREQ(0..7)
USER_RDWR
USER_ADDRVAL
USER_REQ
USER_MULT
ADDR_SEL
TST_DATA(0..7)
SYS_STATUS_LEDs
TST_DATA(8..15)
To PCI FPGA
FUNCTION LEGEND:
PCI_funct
TST_DATA
To Dedicated Bus
To Generic Bus
DBUS_funct
GBUS_funct
SYS_funct TGR/DCC System Function
To Test Connector
TEST_BUS To Test Bus
GBUS_DAS_D
SYS_CLK_2
DBUS_DATA2(0..18)
DBUS_GRT(0..7)
DBUS_RJS1,2
SYS_I2C
SYS_SYNC_GEN_IN/OUT
DBUS_RD/WR
DBUS_DAS1,2
DBUS_DAQ_2_REQ(1..4)
GBUS_DAQ_1_REQ(1..4)
GBUS_DATA(0..31)
GBUS_RD/WR
GBUS_GRT(0..4)
GBUS_DATA(32..63)
GBUS_DAS_T
SYS_SYNC_IN
GBUS_DAQ_2_REQ(1..4)
SYS_STATUS_LED_GBUS
109
References
[1] Greenlee RT, Hill-Harmon MB, Murray T, et al. “Cancer Statistics”, 2001. CA cancer J Clin.
2001; 51:1526.
[2] Thomson CJ, et al., “Positron Emission Mammography (PEM): a promising technique for
detecting breast cancer”. IEEE Trans
[3] N. Matela et al, “System matrix for Clear-PEM using ART and linograms”, MIC/IEEE 2004,
Rome, Italy (Proceedings to be published), 2004.
[4] M. Martins et al., “Clear-PEM Data Reconstruction Using STIR”, Annual Congress of the
European Association in Nuclear Medicine, Istanbul, October 2005.
[5] H. Cordeiro et al, “Avaliação de um Algoritmo Exacto de Rearranjo de Dados para Clear-
PEM”, X Congresso Nacional de Medicina Nuclear, Novembro 2005, Lisboa, Portugal.
[6] N. Oliveira et al., “Visualização e Análise de Imagens para Clear PEM”, X Congresso
Nacional de Medicina Nuclear, Novembro 2005, Lisboa, Portugal.
[7] Weinberg I., et al., “Preliminary results for positron emission mammography: real-time
functional breast imaging in a conventional mammography gantry”. Eur J Nucl Med, 1996.
23(7): p. 804-6.
[8] Raylman RR, et al., “The potential role of positron emission mammography for detection of
breast cancer. A phantom study”. Med Phys, 2000. 27(8): p. 1943-54.
[9] Noh D et al., “Diagnostic value of positron emission tomography for detecting breast cancer”.
World J Surg. 1998; 22: 223-228.
[10] Bender H. E col. “Breast imaging with positron emission tomography”. In: Taillefer R,
Khalkhali I, Waxman AD e col. Eds. Radionuclide imaging of the breast. New York: Marcel
Dekker: 147-75, 1998.
[11] J. Varela, 2002 “Clear-PEM, A dedicated PET camera for mammography”, Nucl. Instrum.
Meth. vol. A 486, pp. 1-6, 2002.
[12] http://www.lip.pt/experiments/pet/
[13] http://crystalclear.web.cern.ch/crystalclear/
[14] C. Ortigão et al, “Experimental characterization of the ClearPEM Detector Module”,
European Physical Society, 19th Nuclear Physics Divisional Conference, New Trends in
Nuclear Physics Applications and Technology, Pavia (Italy), Sptember, 2005.
[15] http://www.hamamatsu.com/
[16] P.M. Lousã, A.I. Santos et al., “Diagnóstico do Cancro da Mama – Vantagens de um Novo
Sistema Dedicado”, II Fórum Ibérico de Telemedicina, Viseu, Outubro de 2005.
[17] E. Albuquerque, P. Bento, F. Goncalves, C. Leong, P. Lousã, J. Nobre, J. C. Silva, L. Silva,
M. M. Silva, J. Rego, P. Relvas, P. Rodrigues, I. C. Teixeira, J. P. Teixeira, A. Trindade, J.
Varela, “Performance Simulation Studies of the Clear-PEM DAQ/Trigger System”, Proc. of the
14th. IEEE NPSS Real Time Conference, Stockholm, Sweden, June, 2005.
110
[18] E. Albuquerque et al., “The Clear-PEM Electronics System”, accepted for publication in
Transactions on Nuclear Science, 2005.
[19] P. Bento, “Sistema Electrónico Para Organização e Encaminhamento de Dados de um
Sistema de Detecção de Cancro da Mama em Tecnologia PET (Electronic System to Data
Routing and Organization for Breast Cancer System Detection using PET Technology)”, MSc
degree in Electrical and Computer Engineering from IST-UTL, 2006.
[20] C. Leong, “Sistema Electrónico de Qualificação e Filtragem de Dados para Detecção de
Cancro da Mama em Tecnologia PET (Electronic System for Qualification and Filtering Data for
the Detection of Breast Cancer using PET Technology)”, MSc degree in Electrical and
Computer Engineering from IST-UTL, 2006.
[21] A. I. Santos et al., “Design and evaluation of the Clear PEM detector for positron emission
mammography”, MIC/IEEE 2004, Rome, Italy (Proceedings to be published)
[22] S. Agostinelli et al., 1995, “GEANT4 – A simulation toolkit”, Nucl. Instrum. Meth. A. vol A
506, 2003, pp. 250-303.
[23] Bender H. E col. “Breast imaging with positron emission tomography”. In: Taillefer R,
Khalkhali I, Waxman AD e col. Eds. Radionuclide imaging of the breast. New York: Marcel
Dekker, 1998, pp. 147-175.
[24] P. Rodrigues et al., “Geant4 applications and developments for medical physics
experiments”, IEEE Trans. Nucl. Sci. vol 51, 2004, pp. 1412-1419.
[25] S. Agostinelli et al., 1995, “GEANT4 – A simulation toolkit”, Nucl. Instrum. Meth. A. vol A
506, 2003, pp. 250-303.
[26] P. Lecoq and J. Varela, 2002 “Clear PEM, A dedicated PET camera for mammography”,
Nucl. Instrum. Meth. vol. A 486, 2002, pp. 1 6.
[27] Noh D et al., “Diagnostic value of positron emission tomography for detecting breast
cancer”. World J Surg. 1998; 22: 223-228.
[28] A. Trindade et al., “Design and Evaluation of the Clear-PEM Detector for Positron Emission
Mammography”, IEEE-Medical Imaging Conference, Rome, Oct 2004.
[29] M. Correia, H. Cordeiro et al., “Evaluation of Rebinning Methods in a 2D Reconstruction
Scenario with Linograms for Clear-PEM”, EANM 2005, Annual Congress of the European
Association in Nuclear Medicine, Istanbul, October 2005.
[30] N. Matela et al., “Algebraic Reconstruction Technique (ART) using Linograms”, EANM´04 –
Annual Congress of the European Association of Nuclear Medicine, Helsinki, September 04
[31] N. Matela et al., “Comparação de Algoritmos de Reconstrução de Imagem 2D para
ClearPEM”, X Congresso Nacional de Medicina Nuclear, Novembro 2005, Lisboa, Portugal.
[32] Mónica V. Martins et al., “Reconstrução de imagem 3D para Clear-PEM”, X Congresso
Nacional de Medicina Nuclear, Novembro 2005, Lisboa, Portugal.
[33] J. Rego et al., “DAE design document”, September 2005
[34] J. Rego et al., “DAE transmission document”, 2005”
[35] J. Rego et al.,”DAE specification document”, 2005
111
[36] PICMG 2.0 D3.0 CompactPCI Specification, September 1999
[37] J. Rego et al.,”PET-PEM System Data Acquisition Electronics Test Specification”, 2007
[38] Power Distribution System (PDS) Design: Using Bypass/Decoupling Capacitors (Xilinx:
www.xilinx.com)
[39] FX2LP Datasheet (Cypress: www.cypress.com)
[40] FX2LP Technical Reference Manual (Cypress: www.cypress.com)
[41] P. Bento, C. Leong, I. C. Teixeira, J. P. Teixeira, J. Varela, “Testability and DfT / DfD Issues
of the DAE System for PEM”, Technical Report, version 3.1, January 2005
[42] J. Rego, P. Lousã, C. Leong, P. Bento, “DAQ board Functional Tests”, 2005
[43] J. Rego, P. Lousã, C. Leong, P. Bento, “TGR/DCC board Functional Tests”, 2005
[44] J. Rego, P. Lousã, C. Leong, P. Bento, “DAQ Board Electrical and Functional Tests”, 2006
[45] J. Rego, P. Lousã, C. Leong, P. Bento, “DAQ Board Production Test Specification
Document v1.0”, 2006
[46] J. Rego, P. Lousã, C. Leong, P. Bento, “TGR/DCC Board Electrical and Functional Tests”,
2007
[47] J. Rego, P. Lousã, C. Leong, P. Bento, “TGR/DCC Board Production Test Specification
Document v1.0”, 2007
112