utca global architecture development status atca ......architecture stratix v fpga crossbar 144 x...
TRANSCRIPT
Orsay 16 November.2011 uTCA fo 21cm 1
Using xtca crate for 21cm correlator
Thanks to Jean-Pierre Cachemiche
uTCA Global architecture
Development status
ATCA architecture
ADC board
Conclusion
Orsay 16 November.2011 uTCA fo 21cm 2
Motivations
Why uTCA?
Dedicated backplane for high-speed data communication.Scalable system : between 1 to 10 communication board
(AMC)No backplane to design : Standard product were, clock
distribution, system control, power supply sequencing, are take into account.
Little board 72mm x 160 mmEvolutionary system.1 on the shelf crate controller (MCH).Up to 360 10Gbit optical linkLess expensive than ATCA standard.
Orsay 16 November.2011 uTCA fo 21cm 3
Mechanic
Orsay 16 November.2011 uTCA fo 21cm 4
Connectivité backplane :Full Mesh ou Dual Star
Orsay 16 November.2011 uTCA fo 21cm 5
Connectivity between board
4 bidirectional pairs
2 bidirectional pairs
ECS
Throttles,Clocks,TFC information
Relies on powerful connectivity coming with the ATCA standard
Orsay 16 November.2011 uTCA fo 21cm 6
Links architecture
AMC In
AMC In
AMC In
AMC In
AMC In
AMC In
AMC In
AMC In
CrossbarMCH1
AMC out
CrossbarMCH2
AMC out
44
44
44
44
44
44
44
44
44
44
36 x10 Gbits/s 8 x 10 Gbits/s
8 x10Gbits/s
10 x10 Gbits/s
Orsay 16 November.2011 uTCA fo 21cm 7
Architecture
Stratix VFPGA
Crossbar144 x 144
max6.5 Gbits/s
Cyclone IIIFPGA
3 x 12 channels receivers
1 Gigabit Ethernet transceiver
Acquisition board (AMC)
NAT MCH (Tongues 1 and 2)
Supervision
NIOS
Standard backplane(dual star layout)
Stratix VFPGA
1X12 channelsemiters
Output board (AMC)
NIOS
(36 - 10 Gbits/s)
(36 x 10 Gbits/s)
10 boards
36
1
mezzanine board
mezzanine board
Switch board (Tongues 3 and 4)
GigabitEthernetSwitch
CPU
Ethernet link
SMAClock
distribution
Tongue 1
Tongue 2
Tongues 3 & 4
External clock
4 to MCH14 to MCH2
PHY
4 to MCH14 to MCH2
Orsay 16 November.2011 uTCA fo 21cm 8
Control system
Integer in a FPGAs
Base on Gigabit Ethernet
1 input at MCH board and distribution by backplane to all AMC board.
NIOSII cpu + serveur integer on each AMC board
SwitchGbit Ethernet
Nios +µCLinux
FPGA
Carte MCH
Carte AMC
Nios +µCLinux
FPGA
Carte AMC
Nios +µCLinux
FPGA
Carte AMC
Nios +µCLinux
FPGA
Carte AMC
Backckplane
Carte MCH
Orsay 16 November.2011 uTCA fo 21cm 9
MCH board
Cyclone IIIEPC3C25
Cyclone IIIEPC3C25
72 x 72crossbar
72 x 72crossbar
6 x 4
To
ngu
e 4
Backplaneconnector
NIOS core
To
ngu
e 3
PCB 4
6 x 4
6 x 4
6 x 4
Command bus
NAT MCHconnector
NAT MCHconnector
I2C, SPIGbit Ethernet
MarvellPHYDDR2
VS3441
VS3441
I2C bus
CCPM development
Orsay 16 November.2011 uTCA fo 21cm 10
Mécanique
17
mm
12
,9 m
m
1,6
mm
22
,9 m
m
23
,76
mm
Maximum component height
22,9 mm
3,9
mm
24
,46
mm1,6
mm
24
,56
mm
Maximum height
NAT mother board
Prototype control board
Tongue 1
Tongue 2
Tongue 3
Tongue 4
63.38 mm
Orsay 16 November.2011 uTCA fo 21cm 11
Example of MCH board
Orsay 16 November.2011 uTCA fo 21cm 12
Status
Orsay 16 November.2011 uTCA fo 21cm 13
AMC Board
• Now days in development at CCPM (J.P Cachemiche IN2P3 Marseilles).• Up to 36 10Gbits optical link.• Custom by re design optical mezzanine.• ~ 6000Euro Tbp
Stratix V GXStratix V GX
FLASHFLASH DDR3DDR3
@ @Data Data
MMCMMC
Clock out
1 serial links 4 serial links
2 PCIe links
1 GBe link
PCIe Clock
PLLPLLClock In
Optical InterfaceOptical Interface 36 serial links
Jittercleaner
Jittercleaner
ECS path
AMCconnector
X_FPGA path
TFC path
Throttle in, out
Clock In and out
Orsay 16 November.2011 uTCA fo 21cm 14
First version of AMC board
181 mm
73 m
m
Orsay 16 November.2011 uTCA fo 21cm 15
Mezzanine
76 mm
68 m
m
Orsay 16 November.2011 uTCA fo 21cm 16
Mesures a 8 Gbits/s
Serial link at 8 Gbits/s with GBTprotocol
18 hours of ruining without error: BER ~ 2 10-15
Measure Jitter à 10-12
Total : 77 ps (p to p)Random : 3.0 psDeterministic : 36 ps
UI= 125 ps
Orsay 16 November.2011 uTCA fo 21cm 17
Bridge architecture
Stratix VFPGA
Crossbar144 x 144
max6.5 Gbits/s
Cyclone IIIFPGA
3 x 12 channels receivers
1 Gigabit Ethernet transceiver
Acquisition board (AMC)
NAT MCH (Tongues 1 and 2)
Supervision
NIOS
Standard backplane(dual star layout)
Stratix VFPGA
1X12 channelsemiters
Output board (AMC)
NIOS
(12 - 10 Gbits/sEthernet
10 boards
24
1
mezzanine board
mezzanine board
Switch board (Tongues 3 and 4)
GigabitEthernetSwitch
CPU
Ethernet link
SMAClock
distribution
Tongue 1
Tongue 2
Tongues 3 & 4
External clock
4 to MCH14 to MCH2
PHY
4 to MCH14 to MCH2
(12 - 5 Gbits/sSerial LiteII
(12 - 5 Gbits/sSerial LiteII
(12 - 10 Gbits/sEthernet
(12 - 5 Gbits/sSerial LiteII
(12 - 5 Gbits/sSerial LiteII
Orsay 16 November.2011 uTCA fo 21cm 18
ATCA standard
Orsay 16 November.2011 uTCA fo 21cm 19
ATCA standard
ATCA Standard: 2 solutions1) “custom“ use.2) Full standard compatibility
Orsay 16 November.2011 uTCA fo 21cm 20
ATCA generic board
CCPCCCPC
J2J2
J1J1
J3J3
SerialCrossbar
SerialCrossbar
Fabric Channel 1
Fabric Channel 2
Fabric Channels3 to 14
Base Channels3 to 14
ShMC port
Base Channel 2
AMCAMC
AMCAMC
AMCAMC
AMCAMC
12
24
GbE 1
GbEswitch
GbEswitch
12
12
ClockCrossbar
ClockCrossbar
1
TFC
4
4
16
Clock
Throttle
4
SerialCrossbar
SerialCrossbar
1
122
1
TFC
4
Clock
Throttle
1
TFC
4
Clock
Throttle
1
TFC
4
Clock
Throttle
4
1
PCIe
Input/output:144 links
Orsay 16 November.2011 uTCA fo 21cm 21
ATCA standard
Orsay 16 November.2011 uTCA fo 21cm 22
ADC board
Orsay 16 November.2011 uTCA fo 21cm 23
LAL ADC board
➢ 4 input channels (digitized at 500 MHz max, 8bits)
➢ Input band-with 1.7GHhz.
➢ Time resolution ~10ps
➢ FFT 8K on each channel.
➢ Input clock + control (start/stop…) ports
➢ USB, and VME communications port (for control)
➢ 2 high speed (~ 5 Gbit/s) optical transceiver (mono mod).
➢ ~ 4500Euro
Orsay 16 November.2011 uTCA fo 21cm 24
On the shelf ADC board
• ICS1650A-001• PCI Express 250MHz/ch Digital Receiver (analog input), • 4 channels, embeded FPGAVirtex-5• V5SX95T, 12-bit resolution, 16MB mémory• 8400Euro.
Orsay 16 November.2011 uTCA fo 21cm 25
Conclusion
uTCA plus CCPM AMC board interesting solution for link bridge and data massaging.
Beam forming and visibility capability.
Easily scalable for future evolution.
Limited hardware development (optical mezzanine)
Rugged uTCA.