intel® fpga p-tile avalon® memory mapped (avalon-mm) ip for … · 2020. 12. 16. · intel® fpga...

Intel® FPGA P-Tile Avalon® MemoryMapped (Avalon-MM) IP for PCIExpress* User Guide

Updated for Intel® Quartus® Prime Design Suite: 19.3

IP Version: 1.0.0

SubscribeSend Feedback

UG-20237 | 2019.11.05Latest document on the web: PDF | HTML

https://www.intel.com/content/www/us/en/programmable/bin/rssdoc?name=aib1557867923977mailto:[email protected]?subject=Feedback%20on%20Intel%20FPGA%20P-Tile%20Avalon%20Memory%20Mapped%20(Avalon-MM)%20IP%20for%20PCI%20Express%20User%20Guide%20(UG-20237%202019.11.05)&body=We%20appreciate%20your%20feedback.%20In%20your%20comments,%20also%20specify%20the%20page%20number%20or%20paragraph.%20Thank%20you.https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_ptile_pcie_avmm.pdfhttps://www.intel.com/content/www/us/en/programmable/documentation/aib1557867923977.html

Contents

1. Introduction................................................................................................................... 41.1. Overview..............................................................................................................41.2. Features...............................................................................................................41.3. Release Information...............................................................................................51.4. Device Family Support............................................................................................61.5. Performance and Resource Utilization....................................................................... 6

2. IP Architecture and Functional Description.....................................................................72.1. Top-Level Architecture............................................................................................7

2.1.1. Avalon-MM Bridge Architecture.................................................................... 82.1.2. Clock Domains.......................................................................................... 9

2.2. Functional Description.......................................................................................... 102.2.1. Avalon-MM Bridge.................................................................................... 102.2.2. PMA/PCS................................................................................................ 142.2.3. Data Link Layer Overview..........................................................................152.2.4. Transaction Layer Overview.......................................................................17

3. Parameters................................................................................................................... 193.1. Top-Level Settings............................................................................................... 193.2. Core Parameters..................................................................................................19

3.2.1. Base Address Registers.............................................................................203.2.2. Device Identification Registers................................................................... 213.2.3. PCI Express and PCI Capabilities Parameters............................................... 223.2.4. Configuration, Debug and Extension Options................................................26

4. Interfaces..................................................................................................................... 274.1. Overview............................................................................................................ 274.2. Clocks and Resets................................................................................................28

4.2.1. Interface Clock Signals............................................................................. 284.2.2. Interface Reset Signals............................................................................. 29

4.3. Avalon-MM Interface ........................................................................................... 304.3.1. 512-bit Avalon-MM Interface..................................................................... 314.3.2. Avalon-MM DMA Operations.......................................................................44

4.4. Serial Data Interface............................................................................................ 464.5. Hard IP Status Interface....................................................................................... 464.6. Hot Plug Interface (RP Only)..................................................................................474.7. Power Management Interface................................................................................ 484.8. Configuration Output Interface.............................................................................. 504.9. Hard IP Reconfiguration Interface...........................................................................54

4.9.1. Address Map for the User Avalon-MM Interface............................................ 554.9.2. Configuration Registers Access...................................................................57

4.10. PHY Reconfiguration Interface..............................................................................59

5. Advanced Features....................................................................................................... 615.1. PCIe Port Bifurcation and PHY Channel Mapping....................................................... 61

5.1.1. Clock Sharing in Bifurcation Modes.............................................................615.1.2. Reset Sharing in Bifurcation Modes.............................................................62

Contents

Intel® FPGA P-Tile Avalon® Memory Mapped (Avalon-MM) IP for PCI Express*User Guide

Send Feedback

2

mailto:[email protected]?subject=Feedback%20on%20Intel%20FPGA%20P-Tile%20Avalon%20Memory%20Mapped%20(Avalon-MM)%20IP%20for%20PCI%20Express%20User%20Guide%20(UG-20237%202019.11.05)&body=We%20appreciate%20your%20feedback.%20In%20your%20comments,%20also%20specify%20the%20page%20number%20or%20paragraph.%20Thank%20you.

6. Quick Start Guide..........................................................................................................636.1. Design Components............................................................................................. 636.2. Directory Structure.............................................................................................. 646.3. Generating the Design Example............................................................................. 646.4. Simulating the Design Example..............................................................................666.5. Compiling the Design Example...............................................................................67

7. Design Example............................................................................................................ 697.1. Overview of the Design Example............................................................................69

7.1.1. Block Descriptions....................................................................................707.1.2. Programming Model for the Example Design................................................ 737.1.3. DMA Operations Using the Example Design..................................................74

7.2. Current Limitations of the Design Example.............................................................. 75

8. Troubleshooting/Debugging......................................................................................... 768.1. Hardware............................................................................................................76

8.1.1. Debugging Link Training Issues..................................................................778.1.2. Debugging Data Transfer and Performance Issues........................................ 83

8.2. Debug Toolkit...................................................................................................... 868.2.1. Overview................................................................................................ 868.2.2. Enabling the P-Tile Debug Toolkit............................................................... 878.2.3. Launching the P-Tile Debug Toolkit............................................................. 878.2.4. Using the P-Tile Debug Toolkit....................................................................90

A. Configuration Space Registers.................................................................................... 102A.1. Configuration Space Registers..............................................................................102

A.1.1. Register Access Definitions...................................................................... 104A.1.2. PCIe Configuration Header Registers.........................................................104A.1.3. PCI Express Capability Structures.............................................................106A.1.4. MSI-X Registers..................................................................................... 107

A.2. Intel-Defined VSEC Capability Registers................................................................ 109A.2.1. Intel-Defined VSEC Capability Header (Offset 00h)..................................... 109A.2.2. Intel-Defined Vendor Specific Header (Offset 04h)...................................... 110A.2.3. Intel Marker (Offset 08h)........................................................................ 110A.2.4. JTAG Silicon ID (Offset 0x0C - 0x18)........................................................ 110A.2.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D).......................111A.2.6. General Purpose Control and Status Register (Offset 0x30)..........................111A.2.7. Uncorrectable Internal Error Status Register (Offset 0x34)...........................112A.2.8. Uncorrectable Internal Error Mask Register (Offset 0x38)............................ 112A.2.9. Correctable Internal Error Status Register (Offset 0x3C)..............................113A.2.10. Correctable Internal Error Mask Register (Offset 0x40)..............................114

B. Document Revision History.........................................................................................115B.1. Document Revision History for the Intel FPGA P-Tile Avalon Memory Mapped

(Avalon-MM) IP for PCI Express User Guide.........................................................115

Contents

Send Feedback Intel® FPGA P-Tile Avalon® Memory Mapped (Avalon-MM) IP for PCI Express*

User Guide3


1. Introduction

1.1. Overview

The P-Tile Avalon® Memory Mapped (Avalon-MM) IP for PCIe consists of:

• Modules, implemented in soft logic, that perform Avalon-MM functions. Together,these modules form an Avalon-MM Bridge.

Note: The P-Tile Avalon-MM IP for PCIe does not include an internal descriptorcontroller for DMA operations. This descriptor controller should beimplemented in the user application logic.

• A PCIe Hard IP that implements the Transaction, Data Link, and Physical layersstack required by the PCI Express protocol. This stack allows the user applicationlogic in the Intel FPGA to interface with another device via a PCI Express link.

This IP provides support for an Avalon-MM interface with DMA and is designed tooptimize the performance of large-size data transfers. If you want to achievemaximum performance with small-size transfers, Intel recommends the use of the P-Tile Avalon-ST IP for PCIe.

1.2. Features

The P-Tile Avalon-MM IP for PCI Express supports the following features:

• Support for Gen3 x16 Avalon-MM operations in Endpoint mode.

Note: Gen4 Avalon-MM operations and Root Port mode will be available in a futurerelease of Intel® Quartus® Prime.

• 512-bit data path with 250 MHz interfaces to user logic to ease timing closure forGen3 x16.

• High-throughput Bursting Avalon-MM Slave.

— Byte enables with byte granularity.

• High-throughput Bursting Avalon-MM Master.

— Up to 7 BARs, including expansion ROM BAR.

— Byte enables with byte granularity.

• Data movers with high throughput for DMA support

— Move data using PCIe Memory Read and Memory Write packets.

— Bursting Avalon-MM Master interfaces for data path.

— Byte enables with dword granularity.

— Avalon-ST interfaces for control and status.

— DMA transfers of 1 dword to (1 MB - 1 dword) in 1 dword increments.

— All addresses are dword-aligned.

UG-20237 | 2019.11.05

Send Feedback

Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.

ISO9001:2015Registered

mailto:[email protected]?subject=Feedback%20on%20Intel%20FPGA%20P-Tile%20Avalon%20Memory%20Mapped%20(Avalon-MM)%20IP%20for%20PCI%20Express%20User%20Guide%20(UG-20237%202019.11.05)&body=We%20appreciate%20your%20feedback.%20In%20your%20comments,%20also%20specify%20the%20page%20number%20or%20paragraph.%20Thank%20you.https://www.intel.com/content/www/us/en/quality/intel-iso-registrations.htmlhttps://www.intel.com/content/www/us/en/quality/intel-iso-registrations.htmlhttps://www.intel.com/content/www/us/en/quality/intel-iso-registrations.html

• Bursts of up to 8 cycles (512 bytes) for the bursting Avalon-MM master, BurstingAvalon-MM Slave and the data movers.

• Support for Max Payload Size values of 128, 256 and 512 bytes.

• Support for Max Read Request Size values of 128, 256 and 512 bytes.

• Available as a Platform Designer component with standard Avalon interfaces.

• MSI and MSI-X support will be available in a future release of Intel Quartus Prime.

• ECC on all M20K memory blocks.

• Separate Refclk with Independent Spread Spectrum Clocking (SRIS).

• Supports Autonomous Hard IP mode.

— This mode allows the PCIe Hard IP to communicate with the Host before theFPGA configuration and entry into User mode are complete.

• Modular implementation allowing users to enable the required features for aspecific application. For example:

— Simultaneous support for DMA modules and high-throughput Avalon-MMslaves and masters.

— Avalon-MM slave for easy access to the whole PCIe address space.

1.3. Release Information

Table 1. P-Tile Avalon-MM IP for PCI Express Release Information

Item Description

IP Version 1.0.0

Intel Quartus Prime Version 19.3

Release Date October 2019

Ordering Codes No ordering code is required

IP versions are the same as the Intel Quartus Prime Design Suite software versions upto v19.1. From Intel Quartus Prime Design Suite software version 19.2 or later, IPshave a new IP versioning scheme.

The IP versioning scheme (X.Y.Z) number changes from one software version toanother. A change in:

• X indicates a major revision of the IP. If you update your Intel Quartus Primesoftware, you must regenerate the IP.

• Y indicates the IP includes new features. Regenerate your IP to include these newfeatures.

• Z indicates the IP includes minor changes. Regenerate your IP to include thesechanges.

Intel verifies that the current version of the Intel Quartus Prime Pro Edition softwarecompiles the previous version of each IP core, if this IP core was included in theprevious release. Intel reports any exceptions to this verification in the Intel IPRelease Notes or clarifies them in the Intel Quartus Prime Pro Edition IP Update tool.Intel does not verify compilation with IP core versions older than the previous release.

1. Introduction

UG-20237 | 2019.11.05


User Guide5


1.4. Device Family Support

The following terms define device support levels for Intel FPGA IP cores:

• Advance support—the IP core is available for simulation and compilation for thisdevice family. Timing models include initial engineering estimates of delays basedon early post-layout information. The timing models are subject to change assilicon testing improves the correlation between the actual silicon and the timingmodels. You can use this IP core for system architecture and resource utilizationstudies, simulation, pinout, system latency assessments, basic timing assessments(pipeline budgeting), and I/O transfer strategy (data-path width, burst depth, I/Ostandards tradeoffs).

• Preliminary support—the IP core is verified with preliminary timing models forthis device family. The IP core meets all functional requirements, but might still beundergoing timing analysis for the device family. It can be used in productiondesigns with caution.

• Final support—the IP core is verified with final timing models for this devicefamily. The IP core meets all functional and timing requirements for the devicefamily and can be used in production designs.

Table 2. Device Family Support

Device Family Support Level

Intel Stratix® 10 DX, Intel Agilex™ Advance support.

Other device families No support.Refer to the Intel PCI Express Solutions web page on the Intel website for supportinformation on other device families.

1.5. Performance and Resource Utilization

The Avalon-MM variants include an Avalon-MM DMA bridge implemented in soft logic.It operates as a front end to the hardened protocol stack. The resource utilizationtable below shows results for the Simple DMA dynamically generated design example.

The results are for the current version of the Intel Quartus Prime Pro Edition software.

Table 3. Resource Utilization of the Avalon-MM IP for PCI Express IP Core

DesignExample Used

LinkConfiguration

Device Family Typical ALMs M20K MemoryBlocks(1)

Logic Registers

DMA Gen3 x16, EP Intel Stratix 10DX 15546 120 40985

DMA Gen3 x16, EP Intel Agilex 15455 120 41362

(1) These results only include resources in the Avalon-MM IP partition in the design example asthe Table title indicates. They do not include resources for external blocks such as the on-chipmemory, DMA controller, and other interconnect logic.

1. Introduction

UG-20237 | 2019.11.05


Send Feedback

6


2. IP Architecture and Functional Description

2.1. Top-Level Architecture

The P-tile Avalon-MM IP for PCI Express* consists of the following major sub-blocks:

• PMA/PCS

• Four PCIe* cores (one x16 core, one x8 core and two x4 cores)

Note: In the 19.3 release of Intel Quartus Prime, the P-tile Avalon-MM IP onlysupports Gen3 x16. Therefore, only the x16 core is active.

• Embedded Multi-die Interconnect Bridge (EMIB)

• Soft logic blocks in the FPGA fabric to implement the Avalon-MM Bridge, whichtranslates the PCIe TLPs from the PCIe Hard IP into standard Avalon memory-mapped reads and writes.

Figure 1. P-tile Avalon-MM IP for PCI Express top-level block diagram

PHY P-Tile

P-Tile Avalon-MM PCIe IP Top Level

FPGA Fabric

PCIe x16 Lanes

PCIe ControllersPMA Quad 3PLL A/B

PMA Quad 2PLL A/B

PMA Quad 1PLL A/B

PMA Quad 0PLL A/B

x16 PCIe PCS

Bifurcation Mux

Avalon -MM

Bridge

User Logic

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x4

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x4

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x8

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x16

EMIB

refclk0 refclk1

pin_perst_n

Note: Each core in the IP implements its own Data Link Layer and Transaction Layer.

UG-20237 | 2019.11.05

Send Feedback




The four cores in the IP can be configured to support the following topologies:

Table 4. Configuration Modes Supported by the P-tile Avalon-MM IP for PCI Express

Configuration Mode Native Hard IP ModeEndpoint(EP) / RootPort (RP)

Active Cores

Configuration Mode 0 Gen3x16 or Gen4x16 EP/RP x16

Configuration Mode 1 Gen3x8/Gen3x8 or Gen4x8/Gen4x8 EP x16, x8

Configuration Mode 2 Gen3x4/Gen3x4/Gen3x4/Gen3x4 orGen4x4/Gen4x4/Gen4x4/Gen4x4 RP x16, x8, x4_0, x4_1

In Configuration Mode 0, only the x16 core is active, and it operates in Gen3 x16mode or Gen4 x16 mode.

In Configuration Mode 1, the x16 core and x8 core are active, and they operate as twoGen3 x8 cores or two Gen4 x8 cores.

In Configuration Mode 2, all four cores (x16, x8, x4_0, x4_1) are active, and theyoperate as four Gen3 x4 cores or four Gen4 x4 cores.

Note: Only Configuration Mode 0 in Gen3 x16 Endpoint mode is available in the 19.3 releaseof Intel Quartus Prime. Other configurations will be available in a future release ofIntel Quartus Prime.

2.1.1. Avalon-MM Bridge Architecture

In PCIe Gen3 mode, the PCIe Hard IP uses a 500 MHz clock and the Avalon-MM Bridgeuses a 250 MHz clock. The Width and Rate Adapter handles the necessary conversionbetween these two clock domains. An additional adapter converts between theembedded header format of the PCIe Hard IP and the separate header and dataformat used by the other modules in the Avalon-MM Bridge.

The Avalon-MM Bridge consists of four main modules:

• Bursting Master (BAM): This module converts memory read and write TLPsinitiated by the remote link partner and received over the PCIe link into Avalon-MM burst read and write transactions, and sends back CplD TLPs for read requestsit receives. It can also function in a non-bursting mode.

• Bursting Slave (BAS): This module converts Avalon-MM read and writetransactions initiated by the application logic into PCIe memory read and writeTLPs to be transmitted over the PCIe link. This module also processes the CplDTLPs received for the read requests it sent. It can also function in a non-burstingmode.

• Write Data Mover (WRDM): This module uses PCIe memory write TLPs andAvalon-MM read transactions to move large amounts of data from your applicationlogic in the Avalon-MM space to the system memory in the PCIe space. The WRDMalso supports immediate writes, which are enabled by a bit in the descriptors thatthe WRDM receives via one of its descriptor sink interfaces. For more details onimmediate writes, refer to Write Data Mover Avalon-ST Descriptor Sinks on page36.

• Read Data Mover (RDDM): This module uses PCIe memory read TLPs and Avalon-MM write transactions to move large amounts of data from the system memory inthe PCIe space to the FPGA memory in the Avalon-MM space.


UG-20237 | 2019.11.05


Send Feedback

8


Figure 2. P-Tile Avalon-MM Bridge Block Diagram

512 512

512 512

Avalon ConduitAvalon-ST Sink

Avalon-ST Source

Avalon-MM Master

Avalon-MM Slave

Hard

IP In

terfa

ce

Response Re-ordering

Embe

dded

/Sep

arat

edHe

ader

Adap

tor

Avalon-MM Bridge

O

I

BurstingMaster

ReadData Mover

O

I

IIOM

WriteData Mover O

IIOM

ConfigSlave

O

IS

M

BurstingSlave

OOI

S

P-Tile Avalon-MM IP for PCIe MS

I

O

C

250 MHz

500 MHz

P-Til

e PCle

Har

d IP

P-Til

e PCle

Avalo

n-ST

with

Widt

h and

Rate

Adap

tor

The grayed out modules are not active in the 19.3 release of Intel Quartus Prime.They will be active in a future release.

Descriptors provided to the Data Movers through one of their Avalon-ST sinkinterfaces control the data transfers. The Data Movers report the transfers’ statusthrough their Avalon-ST source interfaces.

In Root Port mode, there is a Configuration Slave module that converts single-cycle,32-bit Avalon-MM read and write transactions into PCIe configuration read and writeTLPs (CfgRd0, CfgRd1, CfgWr0 and CfgWr1) to be sent over the PCIe link. This modulealso processes the completion TLPs (Cpl and CplD) it receives in return.

Note: Root Port mode will be available in a future Intel Quartus Prime release.

The Response Reordering module assembles and reorders completion TLPs receivedover the PCIe link for the Bursting Slave and the Read Data Mover. It routes thecompletions based on their tags.

No re-ordering is necessary for the completions sent to the Configuration Slavemodule as it only issues one request TLP at a time.

Endpoint applications typically need the Bursting Master to enable the host to provideinformation for the other modules.

2.1.2. Clock Domains

The P-Tile Avalon-MM IP for PCI Express has three primary clock domains:


UG-20237 | 2019.11.05


User Guide9


• PHY clock domain (i.e. core_clk domain): this clock is synchronous to theSerDes parallel clock.

• EMIB/FPGA fabric interface clock domain (i.e. pld_clk domain): this clock isderived from the same reference clock (refclk0) as the one used by the SerDes.However, this clock is generated from a stand-alone core PLL.

• Application clock domain (coreclkout_hip): this clock is an output from the P-Tile IP, and it has the same frequency as pld_clk.

Figure 3. Clock Domains

User Logic

Avalon- MM Bridge

FPGA Fabric P-Tile

EMIB PCIe Hard IP

x16 core_clk

x8 core_clk

x4_0 core_clk

x4_1 core_clk

pld_clk

coreclkout_hip

PCS

PMA

Widthand Rate Adapter

The PHY clock domain (i.e. core_clk domain) is a dynamic frequency domain. ThePHY clock frequency is dependent on the current link speed.

Table 5. PHY Clock and Application Clock Frequencies

Link Speed PHY Clock Frequency Application Clock Frequency

Gen1 125 MHz

Gen1 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP

Parameter Editor. For the 19.3 release of IntelQuartus Prime, Gen3 x16 is the only availableconfiguration. Therefore, the application clock

frequency is 250 MHz.

Gen2 250 MHz

Gen2 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP

Parameter Editor. For the 19.3 release of IntelQuartus Prime, Gen3 x16 is the only availableconfiguration. Therefore, the application clock

frequency is 250 MHz.

Gen3 500 MHz 250 MHz

Gen4 1000 MHz

350 MHz / 400 MHz (Intel Stratix 10 DX)350 MHz / 400 MHz / 500 MHz (Intel Agilex)

Note: Gen4 support will be available in a futurerelease of Intel Quartus Prime.

2.2. Functional Description

2.2.1. Avalon-MM Bridge

The P-Tile Avalon-MM Bridge can support three modes of operation:


UG-20237 | 2019.11.05


Send Feedback

10


• DMA mode using the Data Movers.

• Bursting Slave mode.

• Root Port mode.

However, in the 19.3 release of Intel Quartus Prime, only the DMA mode using theData Movers will be available. The other two modes will be available in a futurerelease of Intel Quartus Prime.

Depending on the mode of operation, different modules in the IP core are enabled.

Table 6. Operating Modes of the Avalon-MM BridgeIn the following table, Yes means the block is enabled for that operating mode. No means the block is notenabled for that mode.

Modes

Modules

Read DataMover

(RDDM)

Write DataMover

(WRDM)

Bursting Avalon-MMMaster (BAM)

Bursting Avalon-MMSlave (BAS)

ConfigurationSlaveNon-

BurstingMode

BurstingMode

Non-Bursting

Mode

BurstingMode

DMA modeusing Data

MoversYes Yes Yes No No No No

BurstingSlave No No Yes No No Yes No

Root Port No No Yes Yes Yes No Yes

2.2.1.1. DMA Mode with Data Movers

When the Avalon-MM Bridge is used in this mode, the following modules are enabled:

• Read Data Mover (RDDM)

• Write Data Mover (WRDM)

• Bursting Master (BAM) in Non-Bursting Mode

The following figure shows how the DMA example design that you can generate usingthe Intel Quartus Prime software interfaces with the P-Tile Avalon-MM IP to performDMA operations. If you are not using the provided DMA example design, you need toimplement your custom DMA Controller and BAR Interpreter in your application logic.


UG-20237 | 2019.11.05


User Guide11


Figure 4. P-Tile Avalon-MM IP in DMA Mode with Data Movers Enabled

DMA Example Design P-Tile Avalon-MM IP

Qsys Interconnect

Memory

Writedata mover(512 bits)

S

S

S

SC

M

WASTOASTO

ASTIASTIASTO

M

Readdata mover(512 bits) ASTI

ASTOASTIASTIASTO

M

Bursting Master(non-bursting

mode)(512 bits)

ASTI

ASTOCM

WASTOWASTI

RASTORASTORASTI

DMA Controller

BARInterpreter

CompletionRe-ordering

TX

RX

Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512

The BAM is used in non-bursting mode for the host software to program the registersin the user Avalon-MM space where the DMA controller typically resides. The DMAcontroller, after being programmed by software, sends descriptor-fetching instructionsto the host via the RDDM. After the fetched descriptors are processed by the WRDMand RDDM, status and/or MSI-X messages are sent to the host via the WRDM in“Immediate” mode. In this mode, the data payload is embedded in bits [31:0] or[63:0] of the fetched descriptors that the WRDM receives (depending on whether aone- or two-dword immediate transfer is needed respectively). For more details onimmediate transfers, refer to Write Data Mover Avalon-ST Descriptor Sinks on page36.

The RDDM uses PCIe memory read TLPs and Avalon-MM write transactions (which canbe bursting transactions) to move large amounts of data from the host memory inPCIe space to the local FPGA memory in Avalon-MM space. On the other hand, theWRDM uses PCIe memory write TLPs and Avalon-MM read transactions to move largeamounts of data from the FPGA memory in Avalon-MM space to the host memory inPCIe space. The Data Movers' transfers are controlled by descriptors that are providedto the Data Movers through one of their Avalon-ST sink interfaces. The Data Moversreport the transfers’ status through their Avalon-ST source interfaces.

2.2.1.2. Bursting Slave Mode

Note: This mode is not supported in the 19.3 release of Intel Quartus Prime, but will beavailable in a future release.

In this mode, the external master sends memory reads and writes upstream via theBursting Slave. The following modules are enabled:

• Bursting Slave (in bursting mode)

• Bursting Master (in non-bursting mode)


UG-20237 | 2019.11.05


Send Feedback

12


Figure 5. P-Tile Avalon-MM IP in Bursting Slave Mode

DMA Application Logic P-Tile Avalon-MM IP

Qsys Interconnect

S Memory

S

M Custom DMA

ControllerM

SC

MBursting Master (non-bursting

mode)(512 bits)

CM

BurstingSlave

(512 bits) ASTI

ASTO

ASTI

ASTO

S

DMA Control/

BAR AccessLogic


Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512

TX

RX

The external Avalon-MM master can be a custom DMA controller that uses theBursting Slave in the IP core to send memory reads and writes upstream. Thesememory reads and writes can be up to 512-bytes long. The reordering buffer in the IPcore reorders the Completion TLPs received over the PCIe link and sends them to theBursting Slave.

Registers in the custom DMA controller can be programmed by software via theBursting Master port.

2.2.1.3. Root Port Mode

Note: This mode is not supported in the 19.3 release of Intel Quartus Prime, but will beavailable in a future release.

In this mode, the IP core needs to be able to process memory read and write TLPscoming from the DMA controller that resides on the Endpoint side. The followingmodules are enabled:

• Bursting Master (in bursting and non-bursting modes)

• Bursting Slave (in non-bursting mode)

• Configuration Slave

Figure 6. P-Tile Avalon-MM IP in Root Port Mode

Custom DMA Application Logic P-Tile Avalon-MM IP

Qsys Interconnect

S Memory

SLocal

ProcessorM

M

SC

MBurstingMaster

(512 bits) ASTIASTOC

M

ConfigurationSlave

(512 bits) ASTIASTOS

Bursting Slave(non-bursting

mode)(512 bits)

ASTI

ASTOS

S

S

BurstingSlave


Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512RX

TX


UG-20237 | 2019.11.05


User Guide13


The IP core must be able to generate and process configuration reads and writes tothe Endpoint and to the Hard IP configuration registers. This is done via theConfiguration Slave. Since the DMA controller resides on the Endpoint side, its controlregisters need to be programmed by the FPGA local processor. Using the BurstingSlave (in non-bursting mode), the local processor can program the Endpoint controlregisters for DMA operations. The Endpoint can also send updates of its DMA status tothe local processor via the Bursting Master.

2.2.2. PMA/PCS

The P-Tile Avalon-MM IP for PCI Express contains Physical Medium Attachment (PMA)and PCI Express Physical Coding Sublayer (PCIe PCS) blocks for handling the Physicallayer (PHY) packets. The PMA receives and transmits high-speed serial data on theserial lanes. The PCS acts as an interface between the PMA and the PCIe controller,and performs functions like data encoding and decoding, scrambling anddescrambling, block synchronization etc. The PCIe PCS in the P-Tile Avalon-MM IP forPCI Express is based on the PHY Interface for PCI Express (PIPE) Base Specification4.4.1.

In this IP, the PMA consists of up to four quads. Each quad contains a pair of transmitPLLs and four SerDes lanes capable of running up to 16 GT/s to perform the variousTX and RX functions.

PLLA generates the required transmit clocks for Gen1/Gen2 speeds, while PLLBgenerates the required clocks for Gen3/Gen4 speeds. For the x8 and x16 lane widths,one of the quads acts as the master PLL source to drive the clock inputs for each ofthe lanes in the other quads.

Note: The P-Tile Avalon-MM IP for PCI Express only supports the Gen3 x16 configuration inthe 19.3 release of Intel Quartus Prime.

The PMA performs functions such as serialization/deserialization, clock data recovery,and analog front-end functions such as Continuous Time Linear Equalizer (CTLE),Decision Feedback Equalizer (DFE) and transmit equalization.

The transmitter consists of a 3-tap equalizer with one tap of pre-cursor, one tap ofmain cursor and one tap of post-cursor.

The receiver consists of attenuation (ATT), CTLE, Voltage gain amplifier (VGA) and a5-tap DFE blocks that are adaptive for Gen3/Gen4 speeds. RX Lane Margining issupported by the PHY. The Lane Margining supports timing margining only. Theoptional voltage margining is not supported. Timing margining capabilities/parametersare as follows:

• Maximum Timing Offset: -0.2UI to +0.2UI.

• Number of timing steps: 9.

• Independent left and right timing margining is supported.

• Independent Error Sampler is not supported (lane margining may produce logicalerrors in the data stream and cause the LTSSM to go to the Recovery state).

The PHY layer uses a fixed 16-bit PCS-PMA interface width to output the PHY clock(core_clk). The frequency of this clock is dependent on the current link speed. Referto Table 5 on page 10 for the frequencies at various link speeds.


UG-20237 | 2019.11.05


Send Feedback

14


2.2.3. Data Link Layer Overview

The Data Link Layer (DLL) is located between the Transaction Layer and the PhysicalLayer. It maintains packet integrity and communicates (by DLL packet transmission) atthe PCI Express link level.

The DLL implements the following functions:

• Link management through the reception and transmission of DLL Packets (DLLP),which are used for the following functions:

— Power management of DLLP reception and transmission

— To transmit and receive ACK/NAK packets

— Data integrity through the generation and checking of CRCs for TLPs andDLLPs

— TLP retransmission in case of NAK DLLP reception or replay timeout, using theretry (replay) buffer

— Management of the retry buffer

— Link retraining requests in case of error through the Link Training and StatusState Machine (LTSSM) of the Physical Layer

Figure 7. Data Link LayerTo Transaction Layer

Tx Transaction LayerPacket Description & Data Transaction Layer

Packet Generator

Retry Buffer

To Physical Layer

Tx Packets

Ack/NackPackets

RX Datapath

TX Datapath

Rx Packets

DLLPChecker

Transaction LayerPacket Checker

DLLPGenerator

Tx Arbitration

Data Link Controland Management

State Machine

Control& StatusConfiguration Space

Tx Flow Control Credit Information

Rx Flow Control Credit Information

Rx Transation LayerPacket Description & Data

PowerManagement

Function

Note:(1) The L0s (Standby) or L1 (Low Power Standby) states are not supported.


UG-20237 | 2019.11.05


User Guide15


The DLL has the following sub-blocks:

• Data Link Control and Management State Machine—This state machine connects toboth the Physical Layer’s LTSSM state machine and the Transaction Layer. Itinitializes the link and flow control credits and reports status to the TransactionLayer.

• Power Management—This function handles the handshake to enter low powermode. Such a transition is based on register values in the Configuration Space andreceived Power Management (PM) DLLPs. For more details on the power statessupported by the P-Tile Avalon-MM IP for PCIe, refer to section Power ManagementInterface on page 48.

• Data Link Layer Packet Generator and Checker—This block is associated with theDLLP’s 16-bit CRC and maintains the integrity of transmitted packets.

• Transaction Layer Packet Generator—This block generates transmit packets,including a sequence number and a 32-bit Link CRC (LCRC). The packets are alsosent to the retry buffer for internal storage. In retry mode, the TLP generatorreceives the packets from the retry buffer and generates the CRC for the transmitpacket.

• Retry Buffer—The retry buffer stores TLPs and retransmits all unacknowledgedpackets in the case of NAK DLLP reception. In case of ACK DLLP reception, theretry buffer discards all acknowledged packets.

• ACK/NAK Packets—The ACK/NAK block handles ACK/NAK DLLPs and generates thesequence number of transmitted packets.

• Transaction Layer Packet Checker—This block checks the integrity of the receivedTLP and generates a request for transmission of an ACK/NAK DLLP.

• TX Arbitration—This block arbitrates transactions, prioritizing in the followingorder:

— Initialize FC Data Link Layer packet

— ACK/NAK DLLP (high priority)

— Update FC DLLP (high priority)

— PM DLLP

— Retry buffer TLP

— TLP

— Update FC DLLP (low priority)

— ACK/NAK FC DLLP (low priority)


UG-20237 | 2019.11.05


Send Feedback

16


2.2.4. Transaction Layer Overview

The following figure shows the major blocks in the P-Tile Avalon-MM IP for PCI ExpressTransaction Layer:

Figure 8. P-Tile Avalon-MM IP for PCI Express Transaction Layer Block Diagram

Data Link Layer +Physical Layer

RAS

RX

CONFIG

TX

Avalon-ST RX

User Avalon-MM

Avalon-ST TX

RASCPL TimeoutLogic

Avalon-MM TX

Avalon-MM RX

Avalo

n-M

M Br

idge

The RAS (Reliability, Availability, and Serviceability) block includes a set of features tomaintain the integrity of the link.

For example: Transaction Layer inserts an optional ECRC in the transmit logic andchecks it in the receive logic to provide End-to-End data protection.

When the application logic sets the TLP Digest (TD) bit in the Header of the TLP, the P-Tile Avalon-MM IP for PCIe will append the ECRC automatically.

The TX block sends out the TLPs that it receives as-is. It also sends the informationabout non-posted TLPs to the Completion (CPL) Timeout Block for CPL timeoutdetection.

The P-Tile Avalon-MM IP for PCI Express RX block consists of two main blocks:

• Filtering block: This module checks if the TLP is good or bad and generates theassociated error message and completion. It also tracks received completions andupdates the completion timeout (CPL timeout) block.

• RX Buffer Queue: The P-Tile IP for PCIe has separate queues for posted/non-posted transactions and completions. This avoids head-of-queue blocking on thereceived TLPs and provides flexibility to extract TLPs according to the PCIeordering rules.


UG-20237 | 2019.11.05


User Guide17


Figure 9. P-Tile Avalon-MM IP for PCI Express RX Block Overview

Avalon-ST

RX Buffer Queue Filter

TLP Filtering

Config TX

MSG ERR

CFG Data

Received CPLProcessing (*)

MessageProcessingCPL

NP

P

MSG

DataLinkLayer

Trash

Avalon-ST

User Avalon-MM

RoutingAvalon-MM Avalon-STLogical

PHYLayer

Avalon-MM

Bridge

WidthandRate

Adapter

Note: The Received CPL Processing block includes the CPL tracking mechanism.


UG-20237 | 2019.11.05


Send Feedback

18


3. ParametersThis chapter provides a reference for all the parameters that are configurable in theIntel Quartus Prime IP Parameter Editor for the P-Tile Avalon-MM IP for PCIe.

3.1. Top-Level Settings

Table 7. Top-Level Settings

Parameter Value Default Value Description

Hard IP Mode Gen3x16, Interface - 512-bit Gen3x16, Interface - 512-bit

Select the lane data rate andlane width. Only Gen3 x16 issupported in the 19.3release of Intel QuartusPrime.

Port Mode Native Endpoint Native Endpoint

Only Native Endpoint issupported in the 19.3release of Intel QuartusPrime.

Note:

Root Port will besupported in a futurerelease of IntelQuartus Prime.

Enable PHYReconfiguration

True/False False Enable the PHYReconfiguration Interface.

PLD ClockFrequency 250 MHz 250 MHz

Select the frequency of theApplication clock.

Note:

For the 19.3 releaseof Intel QuartusPrime, only 250 MHzis available for Gen3support.

Enable SRISMode True/False False

Enable the SeparateReference Clock withIndependent SpreadSpectrum Clocking (SRIS)feature.When this is disabled, thedefault mode of operation isSeparate Reference Clockwith no Spread SpectrumClocking (SRNS).

3.2. Core Parameters

Depending on which Hard IP Mode you choose in the Top-Level Settings tab, youwill see different tabs for setting the core parameters.

UG-20237 | 2019.11.05

Send Feedback




Figure 10. Intel P-Tile Avalon-MM Top-Level IP Parameter Editor for a x16 Hard IP ModeIf you choose a x16 mode (either Gen4 or Gen3), only the PCIe0 Settings tab will appear.

3.2.1. Base Address Registers

Table 8. BAR Registers

Parameter Value Description

BAR0 Type

Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory

If you select 64-bit prefetchable memory, 2contiguous BARs are combined to form a 64-bitprefetchable BAR; you must set the highernumbered BAR to Disabled.Defining memory as prefetchable allows contiguousdata to be fetched ahead. Prefetching memory isadvantageous when the requestor may requiremore data from the same region than wasoriginally requested. If you specify that a memoryis prefetchable, it must have the following 2attributes:• Reads do not have side effects such as

changing the value of the data read.• Write merging is allowed.

BAR1 TypeDisabled32-bit non-prefetchable memory32-bit prefetchable memory

BAR2 Type

Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory


BAR4 TypeDisabled64-bit prefetchable memory64-bit non-prefetchable memory

continued...

3. Parameters

UG-20237 | 2019.11.05


Send Feedback

20


Parameter Value Description

32-bit non-prefetchable memory32-bit prefetchable memory


BARn Size 128 Bytes - 16 EBytesSpecifies the size of the address space accessibleto BARn when BARn is enabled.n = 0, 1, 2, 3, 4 or 5

Expansion ROM

Disabled4 KBytes - 12 bits8 KBytes - 13 bits16 KBytes - 14 bits32 KBytes - 15 bits64 KBytes - 16 bits128 KBytes - 17 bits256 KBytes - 18 bits512 KBytes - 19 bits1 MByte - 20 bits2 MBytes - 21 bits4 MBytes - 22 bits8 MBytes - 23 bits16 MBytes - 24 bits

Specifies the size of the expansion ROM from 4KBytes to 16 MBytes when enabled.

3.2.2. Device Identification Registers

The following table lists the default values of the Device ID registers. You can use theparameter editor to change the values of these registers.

Table 9. Device ID Registers

Register Name Range Default Value Description

Vendor ID 16 bits 0x00001172

Sets the read-only value of theVendor ID register. This parametercannot be set to 0xFFFF per thePCI Express Base Specification.

Note: Set your own Vendor ID bychanging this parameter.Address offset: 0x000.

Device ID 16 bits 0x00000000

Sets the read-only value of theDevice ID register. This register isonly valid in the Type 0 (Endpoint)Configuration Space.Address offset: 0x000.

Revision ID 8 bits 0x00000001Sets the read-only value of theRevision ID register.Address offset: 0x008.

Class Code 24 bits 0x00000000Sets the read-only value of theClass Code register.Address offset: 0x008.

continued...

3. Parameters

UG-20237 | 2019.11.05


User Guide21


Register Name Range Default Value Description

This parameter cannot be set to0x0 per the PCI Express BaseSpecification.

Subsystem Vendor ID 16 bits 0x00000000

Sets the read-only value of theSubsystem Vendor ID register inthe PCI Type 0 ConfigurationSpace. This parameter cannot beset to 0xFFFF per the PCI ExpressBase Specification. This value isassigned by PCI-SIG to the devicemanufacturer.

Subsystem Device ID 16 bits 0x00000000

Sets the read-only value of theSubsystem Device ID register inthe PCI Type 0 ConfigurationSpace.Address offset: 0x02C.

3.2.3. PCI Express and PCI Capabilities Parameters

For each core (PCIe0/PCIe1/PCIe2/PCIe3), the PCI Express / PCI Capabilities tabcontains separate tabs for the device, link, MSI, MSI-X, power management andvendor specific extended capability (VSEC) parameters.

Figure 11. PCI Express / PCI Capabilities Parameters

3. Parameters

UG-20237 | 2019.11.05


Send Feedback

22


3.2.3.1. Device Capabilities

Table 10. Device Capabilities


Maximum payload sizessupported

128 bytes256 bytes512 bytes

512 bytes

Specifies the maximumpayload size supported. Thisparameter sets the read-only value of the maxpayload size supported fieldof the Device Capabilitiesregister.

3.2.3.2. Link Capabilities

Table 11. Link Capabilities


Link port number (RootPort only) 0 - 255 1

Sets the read-only value ofthe port number field in theLink Capabilitiesregister. This parameter isfor Root Ports only. It shouldnot be changed.

Slot clock configuration True/False True

When this parameter isTrue, it indicates that theEndpoint uses the samephysical reference clock thatthe system provides on theconnector. When it is False,the IP core uses anindependent clock regardlessof the presence of areference clock on theconnector. This parametersets the Slot ClockConfiguration bit (bit 12) inthe PCI Express LinkStatus register.

3.2.3.3. MSI Capabilities

Note: MSI support is not available in the 19.3 release of Intel Quartus Prime, but will beavailable in a future release.

Table 12. MSI Capabilities


PF0 Enable MSI True/False False

Enables MSI functionality forPF0.If this parameter is True,the Number of MSImessages requestedparameter will appearallowing you to set thenumber of MSI messages.

PF0 Number of MSImessages requested

124

1Sets the number ofmessages that theapplication can request in

continued...

3. Parameters

UG-20237 | 2019.11.05


User Guide23



81632

the multiple messagecapable field of the MessageControl register.

3.2.3.4. MSI-X Capabilities

Note: MSI-X support is not available in the 19.3 release of Intel Quartus Prime, but will beavailable in a future release.

Table 13. MSI-X Capabilities


Enable MSI-X (Endpointonly) True/False False

Enables the MSI-Xfunctionality.For SR-IOV: VFs and PFs arealways MSI-X capable.

MSI-X Table Size0x0 - 0x7FF (only values ofpowers of two minus 1 are

valid)0

System software reads thisfield to determine the MSI-Xtable size , which isencoded as .For example, a returnedvalue of 2047 indicates atable size of 2048. This fieldis read-only.Address offset:0x068[26:16]

MSI-X Table Offset 0x0 - 0xFFFFFFFF 0

Points to the base of theMSI-X table. The lower 3 bitsof the table BAR indicator(BIR) are set to zero bysoftware to form a 64-bitqword-aligned offset. Thisfield is read-only after beingprogrammed.

Table BAR indicator 0x0 - 0x5 0

Specifies which one of afunction's BARs, locatedbeginning at 0x10 inConfiguration Space, is usedto map the MSI-X table intomemory space. This field isread-only after beingprogrammed.

Pending bit array (PBA)offset 0x0 - 0xFFFFFFFF 0

Used as an offset from theaddress contained in one ofthe function's Base Addressregisters to point to the baseof the MSI-X PBA. The lower3 bits of the PBA BIR are setto zero by software to forma 32-bit qword-alignedoffset. This field is read-onlyafter being programmed.

PBA BAR indicator 0x0 - 0x5 0

Specifies the function's BaseAddress register, locatedbeginning at 0x10 inConfiguration Space, thatmaps the MSI-X PBA into

continued...

3. Parameters

UG-20237 | 2019.11.05


Send Feedback

24



memory space. This field isread-only after beingprogrammed.

3.2.3.5. Power Management

Table 14. Power Management


Enable L0s acceptablelatency

Maximum of 64 nsMaximum of 128 nsMaximum of 256 nsMaximum of 512 nsMaximum of 1 usMaximum of 2 usMaximum of 4 usNo limit

Maximum of 64 ns

This design parameterspecifies the maximumacceptable latency that theapplication layer can toleratefor any link between thedevice and the root complexto exit the L0s state. It setsthe read-only value of theEndpoint L0s acceptablelatency field of the DeviceCapabilities Register(0x084).This Endpoint does notsupport the L0s or L1 states.However, in a switchedsystem, there may be linksconnected to switches thathave L0s and L1 enabled.This parameter is set toallow system configurationsoftware to read theacceptable latencies for alldevices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).This setting is disabled forRoot Ports.The default value of thisparameter is 64 ns. This isthe safest setting for mostdesigns.

Endpoint L1 acceptablelatency

Maximum of 1 usMaximum of 2 usMaximum of 4 usMaximum of 8 usMaximum of 16 usMaximum of 32 usMaximum of 64 usNo limit

Maximum of 1 us

This value indicates theacceptable latency that anEndpoint can withstand inthe transition from the L1state to L0 state. It is anindirect measure of theEndpoint’s internal buffering.It sets the read-only value ofthe Endpoint L1 acceptablelatency field of the DeviceCapabilities Register.This Endpoint does notsupport the L0s or L1 states.However, a switched systemmay include links connectedto switches that have L0sand L1 enabled. Thisparameter is set to allowsystem configurationsoftware to read theacceptable latencies for all

continued...

3. Parameters

UG-20237 | 2019.11.05


User Guide25



devices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).This setting is disabled forRoot Ports.

3.2.3.6. Vendor Specific Extended Capability (VSEC) Registers

Table 15. VSEC Register


Vendor Specific ExtendedCapability 0/1 0

Enables the Vendor SpecificExtended Capability (VSEC).

User ID register from theVendor Specific ExtendedCapability

0 - 65534 0

Sets the read-only value ofthe 16-bit User ID registerfrom the Vendor SpecificExtended Capability. Thisparameter is only valid forEndpoints.

Drops Vendor Type0Messages 0/1 0

When this parameter is setto 1, the IP core dropsvendor Type 0 messageswhile treating them asUnsupported Requests (UR).When it is set to 0, the IPcore passes these messageson to the user logic.

Drops Vendor Type1Messages 0/1 0

When this parameter is setto 1, the IP core silentlydrops vendor Type 1messages.When it is set to 0, the IPcore passes these messageson to the user logic.

3.2.4. Configuration, Debug and Extension Options

Table 16. Configuration, Debug and Extension Options


Enable Debug Toolkit True/False False

Enable the P-Tile DebugToolkit for JTAG-basedSystem Console debugaccess.

Enable HIP dynamicreconfiguration of PCIeread-only registers

True/False FalseEnable the user Hard IPreconfiguration Avalon-MMinterface.

3. Parameters

UG-20237 | 2019.11.05


Send Feedback

26


4. Interfaces

4.1. Overview

The P-Tile Avalon-MM IP for PCIe includes many interface types to implement differentfunctions.These include:

• High performance bursting master and slave Avalon-MM interfaces to translatebetween PCIe TLPs and Avalon-MM memory-mapped reads and writes

• Read and Write Data Movers to transfer large blocks of data

• Standard PCIe serial interface to transfer data over the PCIe link

• System interfaces for interrupts, clocking, reset

• Optional reconfiguration interface to dynamically change the value ofConfiguration Space registers at run-time

• Optional status interface for debug

Unless otherwise noted, all interfaces to the Application layer are synchronous to therising edge of the main system clock coreclkout_hip running at 250 MHz (for Gen3operation). The frequency of this clock is exactly half the frequency of the pld_clkgenerated by the IP, with 0ppm difference. You enable the interfaces using thecomponent IP Parameter Editor.

Read Data Mover (RDDM) interface: This interface transfers DMA data from the PCIesystem memory to the memory in Avalon-MM address space.

Write Data Mover (WRDM) interface: This interface transfers DMA data from thememory in Avalon-MM address space to the PCIe system memory.

Bursting Master (BAM) interface: This interface provides host access to the registersand memory in Avalon-MM address space. The Busting Master module converts PCIeMemory Reads and Writes to Avalon-MM Reads and Writes.

Bursting Slave (BAS) interface: This interface allows the user application in the FPGAto access the PCIe system memory. The Bursting Slave module converts Avalon-MMReads and Writes to PCIe Memory Reads and Writes.

The modular design of the P-Tile Avalon-MM IP for PCIe lets you enable just theinterfaces required for your application.

UG-20237 | 2019.11.05

Send Feedback




Table 17. Avalon-MM Interface Summary

Avalon-MM Type Data Bus Width Max Burst Size Byte EnableGranularityMax Outstanding

Read Request

Bursting Slave 512 bits

Bursting Mode: 8cycles

Non-Bursting Mode: 1cycle

dword/byteBursting Mode: 64

Non-Bursting Mode: 1

Bursting Master 512 bits

Bursting Mode: 8cycles

Non-Bursting Mode: 1cycle

dword/byteBursting Mode: 32

Non-Bursting Mode: 1

Read Data MoverWrite Master 512 bits 8 cycles dword N/A

Write Data MoverRead Master 512 bits 8 cycles dword 32

Config Slave 32 bits 1 cycle byte 1

Note: In the 19.3 release of Intel Quartus Prime, the P-Tile Avalon-MM IP for PCIe onlysupports Gen3 x16. In this configuration, the user interface clock frequency is 250MHz.

Note: The number of read requests issued by the Write Data Mover's Avalon-MM ReadMaster is controlled by the assertion of waitrequest by the connected slave(s). TheRead Master can handle 128 outstanding cycles of data. You cannot set this parameterin Platform Designer. The slave needs to correctly back-pressure the master once itcannot handle the incoming requests.

Note: The 512-bit Bursting Slave interface does not support transactions where all byteenables are set to 0.

Note: All transfers of four bytes or more are done in multiples of dwords.

4.2. Clocks and Resets

4.2.1. Interface Clock Signals

Table 18. Interface Clock Signals

Name I/O Description EP/RP Clock Frequency

coreclkout_hip O

This clock drives the ApplicationLayer.

The frequency depends on the datarate and the number of lanes being

used.

EP/RP

Native Gen3: 250 MHzNative Gen4: 400 MHz(Intel Stratix 10 DX) /500 MHz (Intel Agilex)

Note:

Gen4 supportwill be

available in afuture release

of IntelQuartus Prime.

refclk[1:0] IThese are the input reference

clocks for the IP core. These clocksmust be free-running.

EP/RP 100 MHz ± 300 ppm

continued...

4. Interfaces

UG-20237 | 2019.11.05


Send Feedback

28


Name I/O Description EP/RP Clock Frequency

For more details on how to connectthese clocks, refer to the section

Clock Sharing in Bifurcation Modes.Note:

When theEnable SRIS

Modeparameter is

enabled in theIP ParameterEditor, the P-

Tile Avalon-MMIP can

communicatewith a link

partner whoseclock domain is

notsynchronized

to the refclkdomain of theP-Tile. In this

mode ofoperation, P-

Tile and its linkpartner can

both have theirown spreadspectrumclocks.

p0_hip_reconfig_clk

I

Clock for the hip_reconfiginterface. This is an Avalon-MM

interface. It is an optional interfacethat is enabled when the Enable

HIP dynamic reconfiguration ofPCIe registers option in the PCIe

Configuration, Debug andExtension Options tab is enabled.

EP/RP

50 MHz - 125 MHz(range)100 MHz

(recommended)

xcvr_reconfig_clk I

Clock for the PHY reconfigurationinterface. This is an Avalon-MM

interface. This optional interface isenabled when you turn on theEnable PHY reconfiguration

option in the Top-Level Settingstab. This interface is shared among

all the cores.

EP/RP

50 MHz - 125 MHz(range)100 MHz

(recommended)

p0_app_clk O

This is the application clockgenerated from coreclkout_hip

or from the same source asrefclk.

EP/RP

Native Gen3: 250 MHz

Note:

Gen4 supportwill be

available in afuture release

of IntelQuartus Prime.

4.2.2. Interface Reset Signals

Table 19. Interface Reset Signals

Signal Name Direction Clock EP/RP Description

pin_perst_n Input Asynchronous EP/RP This is an active-lowinput to the PCIe HardIP, and implements

continued...

4. Interfaces

UG-20237 | 2019.11.05


User Guide29


Signal Name Direction Clock EP/RP Description

the PERST# functiondefined by the PCIespecification.

p0_reset_status_n Output Synchronous EP/RP This active-low signalis held low untilpin_perst_n hasbeen deasserted andthe PCIe Hard IP hascome out of reset.This signal issynchronous tocoreclkout_hip.When port bifurcationis used, there is onesuch signal for eachinterface. The signalsare differentiated bythe prefixes pn.

p0_link_req_rst_n Output Synchronous EP/RP This active-low signalis asserted by thePCIe Hard IP when itis about to go intoreset.The Avalon-MM BridgeIP will reset all itsPCIe-related registersand queues includinganything related totags. It will also stopsending packets to thePCIe Hard IP until theBus Master Enable bitis set again. TheBridge will also ignoreany packet receivedfrom the PCIe Hard IP.

p0_pld_warm_rst_rdy

Input Synchronous EP/RP This active-high signalis asserted by the userlogic in response top0_link_req_rst_nwhen it has completedits pre-reset tasks.

ninit_done Input Asynchronous EP/RP A "1" on this active-low signal indicatesthat the FPGA deviceis not yet fullyconfigured. A "0"indicates the devicehas been configuredand is in normaloperating mode.

4.3. Avalon-MM Interface

In the 19.3 release of Intel Quartus Prime, the P-Tile Avalon-MM IP for PCIe onlysupports the Gen3 x16 configuration in Endpoint mode. The Avalon-MM interface inthis case is a 512-bit interface.

Note: Root Port mode will be available in a future release of Intel Quartus Prime.

Note: DMA is not supported in Root Port mode.

4. Interfaces

UG-20237 | 2019.11.05


Send Feedback

30


The figure below provides the top-level block diagram of the P-Tile Avalon-MM IP withall the interfaces. These interfaces are described in more details in following sections.

Figure 12. P-Tile Avalon-MM IP for PCIe Top-Level Block Diagram

P-Tile Avalon-MM IP for PCI Express

Clocksrefclk[1:0]

coreclkout_hip

p0_wrdm_read_o

p0_wrdm_address_o[63:0]p0_wrdm_readdata_i[511:0]p0_wrdm_burst_count_o[3:0]p0_wrdm_wait_request_ip0_wrdm_readdatavalid_i

Write Data Mover Interface:Writes Data from FPGA

Memory to Host Memory

tx_n_out[15:0]rx_p_in[15:0] Serial Data

p0_rddm_write_op0_rddm_address_o[63:0]p0_rddm_write_data_o[511:0]p0_rddm_burst_count_o[3:0]p0_rddm_byte_enable_o[63:0]p0_rddm_wait_request_i

Read Data Mover Interface:Writes Data from Host

Memory to FPGA Memory

Hard IPReconfiguration(Optional)

p0_hip_reconfig_clkp0_hip_reconfig_address[20:0]

p0_hip_reconfig_readp0_hip_reconfig_readdata[7:0]

p0_hip_reconfig_writep0_hip_reconfig_writedata[7:0]

p0_hip_reconfig_waitrequest

p0_hip_reconfig_readdatavalid

p0_bam_pfnum_o[1:0]p0_bam_bar_o[2:0]

p0_bus_master_enable_o[1:0]

p0_wrdm_desc_valid_i

p0_bam_waitrequest_i

Write Data MoverNormal Descriptor

Queue

p0_pld_warm_rst_rdy_ip0_reset_status_n pin_perst_n ninit_done

Reset

xcvr_reconfig_clkxcvr_reconfig_address[25:0]

xcvr_reconfig_readxcvr_reconfig_readdata[7:0]xcvr_reconfig_readdatavalid

xcvr_reconfig_writexcvr_reconfig_writedata[7:0]

xcvr_reconfig_waitrequest

PHYReconfiguration(Optional)

p0_wrdm_desc_ready_o

p0_wrdm_desc_data_i[173:0]

Write Data MoverPriority Descriptor

Queue

p0_wrdm_prio_ready_op0_wrdm_prio_valid_ip0_wrdm_prio_data_i[173:0]

Read Data MoverPriority Descriptor

Queue

Read Data MoverNormal Descriptor

Queue

p0_rddm_prio_ready_op0_rddm_prio_valid_ip0_rddm_prio_data_i[173:0]

p0_rddm_desc_ready_op0_rddm_desc_valid_ip0_rddm_desc_data_i[173:0]

Bursting Avalon-MMMaster Interface

p0_rddm_pfnum_o[1:0]

p0_wrdm_pfnum_o[1:0]

p0_wrdm_byteenable_o[63:0]p0_wrdm_response_i[1:0]

p0_rddm_tx_valid_op0_rddm_tx_data_o[31:0]

p0_wrdm_tx_valid_op0_wrdm_tx_data_o[31:0]

p0_pld_link_req_rst_o

p0_app_clk

dummy_user_avmm_rst

tx_p_out[15:0]

rx_n_in[15:0]

4.3.1. 512-bit Avalon-MM Interface

4.3.1.1. Read Data Mover

The Read Data Mover has four user-visible interfaces:

4. Interfaces

UG-20237 | 2019.11.05


User Guide31


• One Avalon-MM Write Master with sideband signals to write data to the Avalondomain.

• Two Avalon-ST Sinks to receive descriptors.

• One Avalon-ST Source to report status.

4.3.1.1.1. Read Data Mover Avalon-MM Write Master and Conduit

Table 20. Read Data Mover Avalon-MM Write Master and Conduit

Signal Name Direction Description Platform DesignerInterface Name

rddm_pfnum_o[1:0] O

Physical function number.• PF0:

rddm_pfnum_o[1:0] =2'b00

• Others:rddm_pfnum_o[1:0] =Reserved

rddm_conduit

rddm_waitrequest_i I

When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 16The master can still issue 16transfers afterrddm_waitrequest_i isasserted.

rddm_master

rddm_write_o OWhen asserted, indicates themaster is requesting a writetransaction.

rddm_address_o[63:0] OSpecify the byte addressregardless of the data widthof the master.

rddm_burstcount_o[3:0] O

The master uses thesesignals to indicate thenumber of transfers in eachburst.

rddm_byteenable_o[63:0]

O

Specify the valid bytes ofrddm_writedata_o[511:0]. Each bit corresponds to abyte inrddm_writedata_o[511:0].

rddm_writedata_o[511:0]

O Data signals for writetransfers.

4.3.1.1.2. Read Data Mover Avalon-ST Descriptor Sinks

The Read Data Mover has two Avalon-ST sink interfaces to receive the descriptors thatdefine the data transfers to be executed. One of the interfaces receives descriptors fornormal data transfers, while the other receives descriptors for high-priority datatransfers.

4. Interfaces

UG-20237 | 2019.11.05


Send Feedback

32


Table 21. Read Data Mover Avalon-ST Normal Descriptor Sink Interface


rddm_desc_ready_o O When asserted, this readysignal indicates the normaldescriptor queue in the ReadData Mover is ready toaccept data. The readylatency of this interface is 3cycles.

rddm_desc

rddm_desc_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the normaldescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.

rddm_desc_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : single destination (2)

[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address

Table 22. Read Data Mover Avalon-ST Priority Descriptor Sink Interface


rddm_prio_ready_o O When asserted, this readysignal indicates the prioritydescriptor queue in the ReadData Mover is ready toaccept data. The readylatency of this interface is 3cycles.

rddm_prio

rddm_prio_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the prioritydescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.

rddm_prio_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID

continued...

(2) When the single destination bit is set, the same destination address is used for all thetransfers. If the bit is not set, the address increments for each transfer.

4. Interfaces

UG-20237 | 2019.11.05


User Guide33



[151:149] : applicationspecific[148] : single destination[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address

The Read Data Mover internally keeps two queues of descriptors. The priority queuehas absolute priority over the normal queue. Use it carefully to avoid starving thenormal queue.

If the Read Data Mover receives a descriptor on the priority interface while processinga descriptor from the normal queue, it switches to processing descriptors from thepriority queue as soon as it has completed the current descriptor. The Read DataMover resumes processing the descriptors from the normal queue once the priorityqueue is empty. Do not use the same descriptor ID simultaneously in the two queuesas there would be no way to distinguish them on the Status Avalon-ST sourceinterface.

The Read Data Mover can accept up to 128 descriptors (which correspond to themaximum size of a descriptor table) at a time. Software should only send newdescriptors when the Read Data Mover has processed all previously sent descriptors.The P-Tile Avalon-MM IP indicates the completion of the Read Data Mover's dataprocessing by performing an immediate write to the system memory using its WriteData Mover. For more details, refer to Read DMA Example on page 74.

4.3.1.1.3. Read Data Mover Status Avalon-ST Source

Table 23. Read Data Mover Status -ST Source


rddm_tx_data_o[31:0] O [31:16]: reserved[15]: error[14:12]: application specific[11:9] : reserved[8] : priority[7:0]: descriptor ID

rddm_tx

rddm_tx_valid_o O Valid status signal

This interface does not have a ready input. The application logic must always be readyto receive status information for any descriptor that it has sent to the Read DataMover.

The Read Data Mover copies over the application specific bits in the rddm_tx_data_obus from the corresponding descriptor. A set priority bit indicates that the descriptor isfrom the priority descriptor sink.

4. Interfaces

UG-20237 | 2019.11.05


Send Feedback

intel® fpga p-tile avalon® memory mapped (avalon-mm) ip for … · 2020. 12. 16. · intel® fpga...

Documents