CASPER Open Source CASPER Open Source HardwareHardware

Current and Future Current and Future ADC’sADC’s

Dan Werthimer and CASPER Collaborators Dan Werthimer and CASPER Collaborators

http://casper.berkeley.educasper.berkeley.edu

Using Moores Law for Telescope Using Moores Law for Telescope ArraysArraysInfrastructure – Construction Costs Infrastructure – Construction Costs

Telescopes & Drives, Feeds, Receivers, Telescopes & Drives, Feeds, Receivers, Analog Electronics, FibersAnalog Electronics, Fibers, Software, , Software, GatewareGateware (costs don’t decrease (costs don’t decrease

rapidlyrapidly) )

Operating Costs – Operating Budget Operating Costs – Operating Budget Digital Computing Hardware Digital Computing Hardware

(beamformers, correlators, spectrometers, (beamformers, correlators, spectrometers, VLBI, pulsar, RFI mitigation, computing, VLBI, pulsar, RFI mitigation, computing, storage) storage) Throw away hardware every 5-6 Throw away hardware every 5-6

years (cheaper, easier to years (cheaper, easier to maintain, 16X)maintain, 16X) year 6: roach 7, year 6: roach 7,

100Gbit switch year 12: Roach 100Gbit switch year 12: Roach 13, Tbit switch13, Tbit switch

Allen Telescope ArrayAllen Telescope Array•6.1-meter offset Gregorian (2.4-meter secondary)

•Designed with Moores Law in Mind

ATA-42 Operational October 2007ATA-42 Operational October 2007

Commercial off-the-shelfMulticast 10 Gbps (10GE

or InfiniBand) Switch

PFBADCFPGA DSP

Module

FPGA DSPModule

FPGA DSPModule

FPGA DSPModule

FPGA DSPModule

FPGA DSPModule

General-purpose CPUs

PFB

PFB

.

.

.

Correlator

Beamformers/Spectrometers

Pulsar timer

.

.

.

ReconfigurableCompute Cluster

ADC

ADC

PolyphaseFilter Banks

.

.

.

.

.

.

Beowulf Cluster Like General Purpose ArchitechtureDynamic Allocation of Resources, need not be FPGA based

Existing Multi-purpose Existing Multi-purpose HardwareHardware• BEE2 (5 2VP70-7, 20 GB, 18 CX4)BEE2 (5 2VP70-7, 20 GB, 18 CX4)

• iBOB (1 2VP50-7, 8 MB, 2 CX4)iBOB (1 2VP50-7, 8 MB, 2 CX4)

• MiniRoach (simple Virtex5 test board, MiniRoach (simple Virtex5 test board, spectrometer)spectrometer)

• Roach (Virtex5, 4x10Gbit CX4, QDR SRAM, Roach (Virtex5, 4x10Gbit CX4, QDR SRAM, DRAM, Linux CPU)DRAM, Linux CPU)

• Bee3 (4*Virtex5, CX4, DRAM, Microsoft, BeeCube)Bee3 (4*Virtex5, CX4, DRAM, Microsoft, BeeCube)

• SERENDIP 5 (2V6000, 2V1000, 4 ADC’s 128 SERENDIP 5 (2V6000, 2V1000, 4 ADC’s 128 Msps)Msps)

• 10 Gbe Switches (HP, Foundry, Fujitsu….)10 Gbe Switches (HP, Foundry, Fujitsu….)

• 10 Gbe cables (Gore…. – optical, copper)10 Gbe cables (Gore…. – optical, copper)

10 Gbit Ethernet Switches, 10 Gbit Ethernet Switches, NIC’sNIC’s• HP – Procurve 6 to 8 ports CX4, 24-48 1GbeHP – Procurve 6 to 8 ports CX4, 24-48 1Gbe

• Fujitsu Fujitsu – 12 port, 20 port, 28 port – 12 port, 20 port, 28 port

• Arista - 28 port, SFP+ ($300/port)Arista - 28 port, SFP+ ($300/port)

• Sun – Sun – 4300 port 4300 port - 20Gbit Infiniband - 20Gbit Infiniband ($300/port)($300/port)

(big enough for SKA already)(big enough for SKA already)

• Myricom – Myricom – 10 Gbe NIC boards 10 Gbe NIC boards (PCIe) ($600)(PCIe) ($600)

CX4 cables (future: SFP+)CX4 cables (future: SFP+)

• Gore CX4 10Gbe cables (1, 2, 3, 15 Gore CX4 10Gbe cables (1, 2, 3, 15 meter…)meter…)

about $65 eachabout $65 each

(not infiniband cables)(not infiniband cables)

(see cable length memo for V2Pro (see cable length memo for V2Pro

1-3 Meter max)1-3 Meter max)

Active CX4 Cables – 45 meter maxActive CX4 Cables – 45 meter max

Optical CX4 Cables Optical CX4 Cables

Where to get tested boardsWhere to get tested boards• Build and test them yourself (cheapest for large Q))Build and test them yourself (cheapest for large Q))

• iBOB, iADC, Roach, enclosures, power supply, DRAMiBOB, iADC, Roach, enclosures, power supply, DRAM

iBOB mounting plates, iBOB cables, iBOB front panelsiBOB mounting plates, iBOB cables, iBOB front panels

Digicom ElectronicsDigicom Electronics

Mo Ohady, Mo Ohady, mo@digicom.org

Upcoming Roach Enclosures:Upcoming Roach Enclosures:

KAT – 1.3U, RFI shielded, $100, nice! NRAO – KAT – 1.3U, RFI shielded, $100, nice! NRAO – 3U, optional synthesizer, attenuator/gain3U, optional synthesizer, attenuator/gain

(from digicom electronics)(from digicom electronics)

tt

Current ADC BoardsCurrent ADC Boards

ADC2x1000-8 (dual 1GSa/sec, single 2Gsps, 8 bit) (dual 1GSa/sec, single 2Gsps, 8 bit)

ADC1x3000-8 (3GSa/sec, 8 bit) ADC (3GSa/sec, 8 bit) ADC

64ADCx64-12 (64x 64MSa/sec, 12 bit)64ADCx64-12 (64x 64MSa/sec, 12 bit)

ADC4x250-8 (quad 250MSa/sec, 8 bit) ADC4x250-8 (quad 250MSa/sec, 8 bit)

katADC (dual 1.5GSa/sec, 8 bit, with gain, atten, katADC (dual 1.5GSa/sec, 8 bit, with gain, atten, synth) synth)

ADC2x550-12 (dual 550 Msps, 12 bit) ADC2x550-12 (dual 550 Msps, 12 bit)

ADC2x400-14 (dual 400 Msps, 14 bit) ADC2x400-14 (dual 400 Msps, 14 bit)

ADC1x5000-8 (1x5Gsps,2x2.5Gsps,4x1.25G sps – ADC1x5000-8 (1x5Gsps,2x2.5Gsps,4x1.25G sps – Taiwan)Taiwan)

ADC1x1000-12 (optically isolated 12 bit 1Gsps – JPL)ADC1x1000-12 (optically isolated 12 bit 1Gsps – JPL)

Misc. BoardsMisc. Boards

• Programmable Attenuator/Gain (ATA, NRAO)Programmable Attenuator/Gain (ATA, NRAO)

4 GHz BW, -16 to +16 dB in 0.5 db step, Notch Filter4 GHz BW, -16 to +16 dB in 0.5 db step, Notch Filter

(or minicircuits, $50, programable (or minicircuits, $50, programable atten/gain)atten/gain)

• 1 PPS distribution amplifier box1 PPS distribution amplifier box

• VSI connector (converts ZDOC to VSI (VLBI))VSI connector (converts ZDOC to VSI (VLBI))

• Dual DAC Board: Dual 16-bit, 1000 MspsDual DAC Board: Dual 16-bit, 1000 Msps

• DAC board, 9 bits, 1.5 GspsDAC board, 9 bits, 1.5 Gsps

• Dual Dac Board, 9 bits, 1.5 GspsDual Dac Board, 9 bits, 1.5 Gsps

Future ADC’s?Future ADC’s?1/3 of the cost of a wide band correlator is 1/3 of the cost of a wide band correlator is

breaking the IF band up into sub-bands breaking the IF band up into sub-bands with analog mixers, filters and with analog mixers, filters and frequency synthesizers before the frequency synthesizers before the digitizers. digitizers.

introduces calibration problems – introduces calibration problems – stitching the bands back together is stitching the bands back together is tricky tricky

Desire WIDE BAND DIGITIZERSDesire WIDE BAND DIGITIZERS

20 Gsps ADC-FGPA-10Gbe 20 Gsps ADC-FGPA-10Gbe boardboard

20 Gsps 8 bit Agilent ADC20 Gsps 8 bit Agilent ADC 20 Gsps 5 bit E2V ADC 20 Gsps 5 bit E2V ADC 80 Gsps 8 bit Fujitsu 80 Gsps 8 bit Fujitsu Hittite, National, Adsantec Hittite, National, Adsantec

Virtex 6 FPGA Virtex 6 FPGA 6 SFP+ ports 6 SFP+ ports 120 Gbps xaui120 Gbps xaui 100 Gbps infiniband 100 Gbps infiniband 60 Gbps 10 Gbe 60 Gbps 10 Gbe

e2V 20 Gsps 5 bit e2V 20 Gsps 5 bit ADC ADC

e2V 20Gsps ADC e2V 20Gsps ADC no drift interleaved no drift interleaved (SiGe)(SiGe)

e2V 20 Gsps ADCe2V 20 Gsps ADC• ADC with 5-bit Resolution Using Interleaved e2v Core TechnologyADC with 5-bit Resolution Using Interleaved e2v Core Technology

• 10 GHz Differential CML AC Coupled Symmetrical Input Clock Required10 GHz Differential CML AC Coupled Symmetrical Input Clock Required

• 600 mVpp Analog Input (Differential AC Coupled)600 mVpp Analog Input (Differential AC Coupled)

• ADC Synchronization Signal (CML), CML Output FormatADC Synchronization Signal (CML), CML Output Format

• Digital Interface (SPI) with Reset Signal:Digital Interface (SPI) with Reset Signal:

• – – Output Mode Selection (Data Only, PRBS Only, Scrambled Data)Output Mode Selection (Data Only, PRBS Only, Scrambled Data)

• – – Gain Control (±10%), Offset Control (±30 mV)Gain Control (±10%), Offset Control (±30 mV)

• – – Per Channel Sampling Delay Adjust (0-10 ps Range)Per Channel Sampling Delay Adjust (0-10 ps Range)

• – – Global Delay Adjust (0-25 ps Range), Test Modes (VOL, VOH)Global Delay Adjust (0-25 ps Range), Test Modes (VOL, VOH)

• Power Supplies: Single 3.3V (2.5V Outputs)Power Supplies: Single 3.3V (2.5V Outputs)

• Power Dissipation: 10WPower Dissipation: 10W

• EBGA240 Package (RoHS, 1.27 mm Pitch)EBGA240 Package (RoHS, 1.27 mm Pitch)

• • • Full Power Input Bandwidth (–3 dB) up to 8 GHzFull Power Input Bandwidth (–3 dB) up to 8 GHz

• • • Band Flatness: < 1 dB from DC to 4 GHzBand Flatness: < 1 dB from DC to 4 GHz

• • • Fsampling = 20 GSps, Fin = 4 GHzFsampling = 20 GSps, Fin = 4 GHz

• – – ENOB = 4.5 bitENOB = 4.5 bit

• – – SFDR = 35 SFDR = 35 dBcdBc

Preliminary Characterization Preliminary Characterization resultsresults• 4 channels acquisitions (preliminary, no calibration)4 channels acquisitions (preliminary, no calibration)

– 5 bits per channel5 bits per channel

– Each ADC core @ 5GSpsEach ADC core @ 5GSps

– Analog Input : -1dBFS @ 6.4GHz Analog Input : -1dBFS @ 6.4GHz

0

8

16

24

32

Port_A Port_B Port_C Port_D

Casper Advisory BoardCasper Advisory Board Matthew Bailes, John Ford, Yashwant Gupta, Justin Jonas, Glenn Matthew Bailes, John Ford, Yashwant Gupta, Justin Jonas, Glenn

Jones, Alan Langman, Jason Manley, Aaron Parsons, Jonathan Jones, Alan Langman, Jason Manley, Aaron Parsons, Jonathan

Weintroub, Dan WerthimerWeintroub, Dan Werthimer

Last Years’s Report On Casper web siteLast Years’s Report On Casper web site

Constructive CriticismConstructive Criticism

Ideas for Future Directions, Long Term StrategyIdeas for Future Directions, Long Term Strategy

How best to collaborateHow best to collaborate

Organizational StructureOrganizational Structure

Anything else Anything else REPORT REPORT

MOVIE!!! (9X speed up)MOVIE!!! (9X speed up)

• Building SpectrometerBuilding Spectrometer

• CorrelatorCorrelator