Intel® FPGA P-Tile Avalon® memorymapped IP for PCI Express* UserGuide
Updated for Intel® Quartus® Prime Design Suite: 20.2
IP Version: 3.0.0
SubscribeSend Feedback
UG-20237 | 2020.07.13Latest document on the web: PDF | HTML
Contents
1. Introduction................................................................................................................... 41.1. Overview..............................................................................................................41.2. Features...............................................................................................................51.3. Release Information...............................................................................................61.4. Device Family Support............................................................................................71.5. Performance and Resource Utilization....................................................................... 7
2. IP Architecture and Functional Description.....................................................................92.1. Top-Level Architecture............................................................................................9
2.1.1. Avalon-MM Bridge Architecture.................................................................. 102.1.2. Clock Domains.........................................................................................122.1.3. Refclk.....................................................................................................132.1.4. Reset..................................................................................................... 15
2.2. Functional Description.......................................................................................... 162.2.1. PMA/PCS................................................................................................ 162.2.2. Data Link Layer Overview..........................................................................172.2.3. Transaction Layer Overview.......................................................................192.2.4. Avalon-MM Bridge.................................................................................... 20
3. Parameters................................................................................................................... 243.1. Top-Level Settings............................................................................................... 243.2. Core Parameters..................................................................................................25
3.2.1. Base Address Registers.............................................................................263.2.2. Device Identification Registers................................................................... 273.2.3. PCI Express and PCI Capabilities Parameters............................................... 273.2.4. Configuration, Debug and Extension Options................................................32
3.3. Avalon-MM Settings..............................................................................................32
4. Interfaces..................................................................................................................... 344.1. Overview............................................................................................................ 344.2. Clocks and Resets................................................................................................35
4.2.1. Interface Clock Signals............................................................................. 354.2.2. Interface Reset Signals............................................................................. 36
4.3. Avalon-MM Interface ........................................................................................... 374.3.1. Endpoint Mode Interface (512-bit Avalon-MM Interface)................................ 384.3.2. Root Port Mode Interface (256-bit Avalon-MM Interface)............................... 55
4.4. Serial Data Interface............................................................................................ 584.5. Hard IP Status Interface....................................................................................... 584.6. Interrupt Interface...............................................................................................59
4.6.1. Legacy Interrupts.................................................................................... 604.6.2. MSI........................................................................................................604.6.3. MSI-X.....................................................................................................63
4.7. Hot Plug Interface (RP Only)..................................................................................664.8. Power Management Interface................................................................................ 674.9. Configuration Output Interface.............................................................................. 684.10. Hard IP Reconfiguration Interface.........................................................................72
4.10.1. Address Map for the User Avalon-MM Interface...........................................744.10.2. Configuration Registers Access.................................................................76
Contents
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
2
4.11. PHY Reconfiguration Interface..............................................................................78
5. Advanced Features....................................................................................................... 805.1. PCIe Port Bifurcation and PHY Channel Mapping....................................................... 80
6. Troubleshooting/Debugging......................................................................................... 816.1. Hardware............................................................................................................81
6.1.1. Debugging Link Training Issues..................................................................826.1.2. Debugging Data Transfer and Performance Issues........................................ 88
6.2. Debug Toolkit...................................................................................................... 916.2.1. Overview................................................................................................ 916.2.2. Enabling the P-Tile Debug Toolkit............................................................... 926.2.3. Launching the P-Tile Debug Toolkit............................................................. 926.2.4. Using the P-Tile Debug Toolkit....................................................................956.2.5. Enabling the P-Tile Link Inspector.............................................................1066.2.6. Using the P-Tile Link Inspector................................................................. 107
A. Configuration Space Registers.................................................................................... 110A.1. Configuration Space Registers..............................................................................110
A.1.1. Register Access Definitions...................................................................... 112A.1.2. PCIe Configuration Header Registers.........................................................112A.1.3. PCI Express Capability Structures.............................................................114A.1.4. Physical Layer 16.0 GT/s Extended Capability Structure...............................116A.1.5. MSI-X Registers..................................................................................... 116
A.2. Intel-Defined VSEC Capability Registers................................................................ 117A.2.1. Intel-Defined VSEC Capability Header (Offset 00h)..................................... 118A.2.2. Intel-Defined Vendor Specific Header (Offset 04h)...................................... 119A.2.3. Intel Marker (Offset 08h)........................................................................ 119A.2.4. JTAG Silicon ID (Offset 0x0C - 0x18)........................................................ 119A.2.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D).......................119A.2.6. General Purpose Control and Status Register (Offset 0x30)..........................120A.2.7. Uncorrectable Internal Error Status Register (Offset 0x34)...........................120A.2.8. Uncorrectable Internal Error Mask Register (Offset 0x38)............................ 121A.2.9. Correctable Internal Error Status Register (Offset 0x3C)..............................122A.2.10. Correctable Internal Error Mask Register (Offset 0x40)..............................122
B. Document Revision History.........................................................................................123B.1. Document Revision History for the Intel FPGA P-Tile Avalon memory mapped IP for
PCI Express User Guide....................................................................................123
Contents
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
3
1. Introduction
1.1. Overview
The P-Tile Avalon® memory mapped IP for PCIe combines the functionality of previousAvalon memory-mapped (Avalon-MM) and Avalon memory-mapped with directmemory access (DMA) interfaces. The IP core using the Avalon-MM interface removesmany of the complexities associated with the PCIe protocol. It handles all of theTransaction Layer Packet (TLP) encoding and decoding, simplifying the design task. Italso includes optional Read and Write Data Mover modules facilitating the creation ofhigh-performance DMA designs. Both the Avalon-MM interface and the Read and WriteData Mover modules are implemented in soft logic. This IP Core natively supportsEndpoint and Root Port configurations with Gen3/Gen4 data rates and x4/x8/x16 linkwidths. Gen1/Gen2 data rates and x1/x2 link widths are supported via link down-training.
The P-Tile Avalon memory mapped IP for PCIe consists of:
• Modules, implemented in soft logic, that perform Avalon memory mappedfunctions. Together, these modules form an Avalon memory mapped Bridge.
• A PCIe Hard IP that implements the Transaction, Data Link, and Physical layersstack that is compliant with PCI Express Base Specification 4.0 . This stack allowsthe user application logic in the Intel FPGA to interface with another device via aPCI Express link.
This IP provides support for an Avalon memory mapped interface with DMA and isdesigned to optimize the performance of large-size data transfers. If you want toachieve maximum performance with small-size transfers, Intel recommends the use ofthe P-Tile Avalon streaming IP for PCIe.
Note: The P-Tile Avalon memory mapped IP for PCIe does not include an internal descriptorcontroller for DMA operations. This descriptor controller should be implemented in theuser application logic. The design example provided for this IP includes an example ofa descriptor controller.
Related Information
Intel FPGA P-Tile Avalon memory mapped IP for PCI Express Design Example UserGuide
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
1.2. Features
The P-Tile Avalon memory mapped IP for PCI Express supports the following features:
• Configurations supported:
Table 1. Configurations Supported by the P-Tile Avalon memory mapped IP for PCIExpress
Gen3/Gen4 x16 Gen3/Gen4 x8 Gen3/Gen4 x4
Endpoint (EP) Yes Yes N/A
Root Port (RP) (1) N/A Yes
Note: Gen1/Gen2 configurations are supported via link down-training.
• Support for 256-bit and 512-bit data paths.
• 512-bit data path with 250 MHz interfaces to user logic to ease timing closure forGen3 x16.
• Support for a single function (PF0).
• High-throughput Bursting Avalon memory mapped Slave (BAS).
— Byte enables with byte granularity.
• High-throughput Bursting Avalon memory mapped Master (BAM).
• Support for up to 7 BARs, including expansion ROM BAR.
• Support for byte enables with byte granularity.
• Support for up to 64 outstanding Non-Posted requests.
• Summary of outstanding Non-Posted requests supported:
Table 2. Outstanding Non-Posted Requests Supported
Ports Active Cores Outstanding Non-Posted Requests
0 x16 64, 512 (*)
1 x8 64
2 and 3 x4 64
Note: (*) : 512 outstanding Non-Posted requests support may be available in afuture Intel Quartus Prime release.
• Data movers with high throughput for DMA support
— Move data using PCIe Memory Read and Memory Write packets.
— Bursting Avalon memory mapped Master interfaces for data path.
— Byte enables with dword granularity.
— Avalon streaming interfaces for control and status.
— DMA transfers of 1 dword to (1 MB - 1 dword) in 1 dword increments.
— All addresses are dword-aligned.
(1) These configurations may be available in a future release of Intel® Quartus® Prime.
1. Introduction
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
5
• Bursts of up to 8 cycles (512 bytes) for the Bursting Avalon memory mappedMaster, Bursting Avalon memory mapped Slave and the data movers.
• Support for Max Payload Size values of 128, 256 and 512 bytes.
• Support for Max Read Request Size values of 128, 256 and 512 bytes.
• Available as a Platform Designer component with standard Avalon interfaces.
• MSI and MSI-X.
• Separate Refclk with Independent Spread Spectrum Clocking (SRIS).
• You cannot change the pin allocations for the P-Tile Avalon memory mapped IP forPCI Express* in the Intel Quartus Prime project. However, this IP does supportlane reversal and polarity inversion on the PCB.
• Supports Autonomous Hard IP mode.
— This mode allows the PCIe Hard IP to communicate with the Host before theFPGA configuration and entry into User mode are complete.
• Modular implementation allowing users to enable the required features for aspecific application. For example:
— Simultaneous support for DMA modules and high-throughput Avalon memorymapped Slaves and Masters.
— Avalon memory mapped Slave for easy access to the whole PCIe addressspace.
• VCS is the only simulator supported in the 20.2 release of Intel Quartus Prime.Other simulators may be supported in a future release.
Note: Throughout this User Guide, the term Avalon-MM may be used as an abbreviation forthe Avalon memory mapped interface or IP.
1.3. Release Information
Table 3. P-Tile Avalon memory mapped IP for PCI Express Release Information
Item Description
IP Version 3.0.0
Intel Quartus Prime Version 20.2
Release Date June 2020
Ordering Codes No ordering code is required
IP versions are the same as the Intel Quartus Prime Design Suite software versions upto v19.1. From Intel Quartus Prime Design Suite software version 19.2 or later, IPshave a new IP versioning scheme.
The IP version (X.Y.Z) number may change from one Intel Quartus Prime softwareversion to another. A change in:
1. Introduction
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
6
• X indicates a major revision of the IP. If you update your Intel Quartus Primesoftware, you must regenerate the IP.
• Y indicates the IP includes new features. Regenerate your IP to include these newfeatures.
• Z indicates the IP includes minor changes. Regenerate your IP to include thesechanges.
Intel verifies that the current version of the Intel Quartus Prime Pro Edition softwarecompiles the previous version of each IP core, if this IP core was included in theprevious release. Intel reports any exceptions to this verification in the Intel IPRelease Notes or clarifies them in the Intel Quartus Prime Pro Edition IP Update tool.Intel does not verify compilation with IP core versions older than the previous release.
Related Information
P-Tile IP for PCI Express IP Core Release NotesThis document provides information about the new features and updates for eachIP release.
1.4. Device Family Support
The following terms define device support levels for Intel FPGA IP cores:
• Advance support—the IP core is available for simulation and compilation for thisdevice family. Timing models include initial engineering estimates of delays basedon early post-layout information. The timing models are subject to change assilicon testing improves the correlation between the actual silicon and the timingmodels. You can use this IP core for system architecture and resource utilizationstudies, simulation, pinout, system latency assessments, basic timing assessments(pipeline budgeting), and I/O transfer strategy (data-path width, burst depth, I/Ostandards tradeoffs).
• Preliminary support—the IP core is verified with preliminary timing models forthis device family. The IP core meets all functional requirements, but might still beundergoing timing analysis for the device family. It can be used in productiondesigns with caution.
• Final support—the IP core is verified with final timing models for this devicefamily. The IP core meets all functional and timing requirements for the devicefamily and can be used in production designs.
Table 4. Device Family Support
Device Family Support Level
Intel Stratix® 10 DX Preliminary support
Intel Agilex™ Preliminary support
Other device familiesNo supportRefer to the Intel PCI Express Solutions web page on the Intel website for support information onother device families.
1.5. Performance and Resource Utilization
The Avalon-MM variants include an Avalon-MM DMA bridge implemented in soft logic.It operates as a front end to the hardened protocol stack. The resource utilizationtable below shows results for the Simple DMA dynamically generated design example.
1. Introduction
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
7
The results are for the current version of the Intel Quartus Prime Pro Edition software.
Table 5. Resource Utilization of the Avalon-MM IP for PCI Express IP Core
DesignExample Used
LinkConfiguration
Device Family Typical ALMs M20K MemoryBlocks(2)
Logic Registers
DMA Gen3 x16, EP Intel Stratix 10DX 15956 120 42345
DMA Gen3 x16, EP Intel Agilex 17116 120 42940
DMA Gen4 x16, EP Intel Stratix 10DX 15967 120 42641
DMA Gen4 x16, EP Intel Agilex 16963 120 45425
DMA Gen4 x8, EP Intel Stratix 10DX 14533 97 42610
DMA Gen4 x8, EP Intel Agilex 16275 97 41025
RP Gen3 x4, RP Intel Stratix 10DX 61339 172 133999
RP Gen3 x4, RP Intel Agilex 62185 172 113475
RP Gen4 x4, RP Intel Stratix 10DX 61467 172 134365
RP Gen4 x4, RP Intel Agilex 61878 172 118199
(2) These results only include resources in the Avalon-MM IP partition in the design example asthe Table title indicates. They do not include resources for external blocks such as the on-chipmemory, DMA controller, and other interconnect logic.
1. Introduction
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
8
2. IP Architecture and Functional Description
2.1. Top-Level Architecture
The P-tile Avalon-MM IP for PCI Express consists of the following major sub-blocks:
• PMA/PCS
• Four PCIe* cores (one x16 core, one x8 core and two x4 cores)
• Embedded Multi-die Interconnect Bridge (EMIB)
• Soft logic blocks in the FPGA fabric to implement the Avalon-MM Bridge, whichtranslates the PCIe TLPs from the PCIe Hard IP into standard Avalon memory-mapped reads and writes.
Figure 1. P-tile Avalon-MM IP for PCI Express top-level block diagram
PHY P-Tile
P-Tile Avalon-MM PCIe IP Top Level
FPGA Fabric
PCIe x16 Lanes
PCIe ControllersPMA Quad 3PLL A/B
PMA Quad 2PLL A/B
PMA Quad 1PLL A/B
PMA Quad 0PLL A/B
x16 PCIe PCS
Bifurcation Mux
Avalon -MM
Bridge
User Logic
DataLink
Layer
Trans- actionLayer
PHYLayer(MAC)
x4
DataLink
Layer
Trans- actionLayer
PHYLayer(MAC)
x4
DataLink
Layer
Trans- actionLayer
PHYLayer(MAC)
x8
DataLink
Layer
Trans- actionLayer
PHYLayer(MAC)
x16
EMIB
refclk0 refclk1
pin_perst_n
Note: Each core in the IP implements its own Data Link Layer and Transaction Layer.
The four cores in the IP can be configured to support the following topologies:
Table 6. Configuration Modes Supported by the P-tile Avalon-MM IP for PCI Express
Configuration Mode Native Hard IP ModeEndpoint(EP) / RootPort (RP)
Active Cores
Configuration Mode 0 Gen3x16 or Gen4x16 EP/RP x16
Configuration Mode 1 Gen3x8/Gen3x8 or Gen4x8/Gen4x8 EP x16, x8
Configuration Mode 2 Gen3x4/Gen3x4/Gen3x4/Gen3x4 orGen4x4/Gen4x4/Gen4x4/Gen4x4 RP x16, x8, x4_0, x4_1
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
In Configuration Mode 0, only the x16 core is active, and it operates in Gen3 x16mode or Gen4 x16 mode.
In Configuration Mode 1, the x16 core and x8 core are active, and they operate as twoGen3 x8 cores or two Gen4 x8 cores.
In Configuration Mode 2, all four cores (x16, x8, x4_0, x4_1) are active, and theyoperate as four Gen3 x4 cores or four Gen4 x4 cores.
2.1.1. Avalon-MM Bridge Architecture
The P-Tile Avalon-MM Bridge can support three modes of operation:
• Endpoint mode with Data Movers.
• Endpoint mode.
• Root Port mode.
In the first two modes, the P-Tile Avalon-MM IP functions as an Endpoint (EP). In RootPort mode, it functions as a Root Port (RP).
The Avalon-MM Bridge consists of five main modules: Read Data Mover (RDDM), WriteData Mover (WRDM), Bursting Avalon-MM Master (BAM), Bursting Avalon-MM Slave(BAS) and Control Register Access (CRA). These modules are shown in Figure 2 onpage 11 and described below. Depending on the mode of operation, differentmodules in the IP core are enabled.
Table 7. Operating Modes of the Avalon-MM BridgeIn the following table, Yes means the block is enabled for that operating mode. No means the block is notenabled for that mode.
Modes
Modules
Read DataMover
(RDDM)
Write DataMover
(WRDM)
Bursting Avalon-MMMaster (BAM)
Bursting Avalon-MMSlave (BAS)
Control RegisterAccess (CRA)Non-
BurstingMode
BurstingMode
Non-Bursting
Mode
BurstingMode
Endpointmode with
Data Movers(EP)
Yes Yes Yes No No No No
Endpointmode (EP) No No Yes No No Yes No
Root Portmode (RP) No No No Yes Yes No Yes
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
10
Here is a block diagram of the P-Tile Avalon-MM Bridge showing the main modules:
Figure 2. P-Tile Avalon-MM Bridge Block Diagram
512 512
512 512
Avalon ConduitAvalon-ST Sink
Avalon-ST Source
Avalon-MM Master
Avalon-MM Slave
Hard
IP In
terfa
ce
Response Re-ordering
Embe
dded
/Sep
arat
edHe
ader
Adap
tor
Avalon-MM Bridge
O
I
BurstingMaster
ReadData Mover
O
I
IIOM
WriteData Mover O
IIOM
Control RegisterAccess O
IS
M
BurstingSlave
OOI
S
P-Tile Avalon-MM IP for PCIe M
S
I
O
C
250 MHz
500 MHz
P-Til
e PCle
Har
d IP
P-Til
e PCle
Avalo
n-ST
Widt
h and
Rate
Adap
tor
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
11
• Bursting Master (BAM): This module converts memory read and write TLPsinitiated by the remote link partner and received over the PCIe link into Avalon-MM burst read and write transactions, and sends back CplD TLPs for read requestsit receives. It can also function in a non-bursting mode.
• Bursting Slave (BAS): This module converts Avalon-MM read and writetransactions initiated by the application logic into PCIe memory read and writeTLPs to be transmitted over the PCIe link. This module also processes the CplDTLPs received for the read requests it sent. It can also function in a non-burstingmode.
• Read Data Mover (RDDM): This module uses PCIe memory read TLPs and Avalon-MM write transactions to move large amounts of data from the system memory inthe PCIe space to the FPGA memory in the Avalon-MM space. It fetchesdescriptors from the system memory through one of its two Avalon-ST sinkinterfaces. These descriptors define the data transfers to be executed. The RDDMalso reports the status of these data transfers via its Avalon-ST source interface.
• Write Data Mover (WRDM): This module uses PCIe memory write TLPs andAvalon-MM read transactions to move large amounts of data from your applicationlogic in the Avalon-MM space to the system memory in the PCIe space. The WRDMalso supports immediate writes, which are enabled by a bit in the descriptors thatthe WRDM receives via one of its Avalon-ST descriptor sink interfaces. For moredetails on immediate writes, refer to Write Data Mover Avalon-ST Descriptor Sinkson page 49. Similar to the RDDM, the WRDM also has its own Avalon-ST sourceinterface to report the status of its data transfers.
• Control Register Access (CRA) Avalon-MM Slave (Root Port only): This module isused in Root Port mode only to issue accesses to the Endpoint's configurationspace registers. It supports a single transaction at a time. It converts single-cycle,32-bit Avalon-MM read and write transactions into PCIe configuration read andwrite TLPs (CfgRd0, CfgRd1, CfgWr0 and CfgWr1) to be sent over the PCIe link.This module also processes the completion TLPs (Cpl and CplD) it receives inreturn.
The Response Reordering module assembles and reorders completion TLPs receivedover the PCIe link for the Bursting Slave and the Read Data Mover. It routes thecompletions based on their tags.
No re-ordering is necessary for the completions sent to the CRA module as it onlyissues one request TLP at a time.
Endpoint applications typically need the Bursting Master to enable the host to provideinformation for the other modules.
2.1.2. Clock Domains
The P-Tile Avalon-MM IP for PCI Express has three primary clock domains:
• PHY clock domain (i.e. core_clk domain): this clock is synchronous to theSerDes parallel clock.
• EMIB/FPGA fabric interface clock domain (i.e. pld_clk domain): this clock isderived from the same reference clock (refclk0) as the one used by the SerDes.However, this clock is generated from a stand-alone core PLL.
• Application clock domain (p<n>_app_clk): this clock is an output from the P-TileIP, and it has the same frequency as pld_clk. This is a per-port signal (i.e, n =0,1,2,3).
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
12
Figure 3. Clock Domains
User Logic
Avalon- MM Bridge
FPGA Fabric P-Tile
EMIB PCIe Hard IP
x16 core_clk
x8 core_clk
x4_0 core_clk
x4_1 core_clk
pld_clk
p<n>_app_clk
PCS
PMA
The PHY clock domain (i.e. core_clk domain) is a dynamic frequency domain. ThePHY clock frequency is dependent on the current link speed.
Table 8. PHY Clock and Application Clock Frequencies
Link Speed PHY Clock Frequency Application Clock Frequency
Gen1 125 MHz
Gen1 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP
Parameter Editor.
Gen2 250 MHz
Gen2 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP
Parameter Editor.
Gen3 500 MHz 250 MHz
Gen4 1000 MHz350 MHz / 400 MHz (Intel Stratix 10 DX)
350 MHz / 400 MHz / 500 MHz (Intel Agilex)
Note: Refer to the P-Tile IP for PCI Express IP Core Release Notes for the matrix ofconfigurations supported by the P-Tile Avalon memory mapped IP for PCI Express.
2.1.3. Refclk
P-Tile has two reference clock inputs at the package level, refclk0 and refclk1.You must connect a 100 MHz reference clock source to these two inputs. Dependingon the port mode, you can drive the two refclk inputs using either a single clocksource or two independent clock sources.
In 1x16 and 4x4 modes, drive the refclk inputs with a single clock source (througha fanout buffer) as shown in the figure below.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
13
Figure 4. Using a Single 100 MHz Clock Source in 1x16 and 4x4 Modes
PCIe x4
(Port 3)
PHY3(x4)La
ne 15
Lane
12 PHY2(x4)La
ne 11
Lane
8 PHY1(x4)La
ne 7
Lane
4 PHY0(x4)La
ne 3
Lane
0
PCIe x8/x4
(Port 1)
Refclk distribution on the package substrate
Refclk1 Refclk0
Fanout Buffer
100MHz
PCIe x16/x8/x4
(Port 0)
PCIe x4
(Port 2)
In 2x8 mode, you can drive the refclk inputs with either a single 100 MHz clocksource as shown above, or two independent 100 MHz sources (see the figure below)depending on your system architecture. For example, if your system has each x8 portconnected to a separate CPU/Root Complex, it may be required to drive these refclkinputs using independent clock sources. In that case, it is strongly recommended thatthe refclk0 input for Port 0 (lanes 0 - 7) be always running because it feeds thereference clock for the P-Tile core PLL that controls the data transfers between the P-Tile and FPGA fabric via the EMIB. If this clock goes down, Port 0 link will go down andPort 1 will not be able to communicate with the FPGA fabric. Following are theguidelines for implementing two independent refclks in 2x8 mode:
• If the link can handle two separate reference clocks, drive the refclk0 of P-Tilewith the on-board free-running oscillator.
• If the link needs to use a common reference clock, then PERST# needs to indicatethe stability of this reference clock. If this reference clock goes down, the entire P-Tile must be reset.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
14
Figure 5. Using Independent 100 MHz Clock Sources in 2x8 Mode
PCIe x4
(Port 3)
PHY3(x4)La
ne 15
Lane
12 PHY2(x4)La
ne 11
Lane
8 PHY1(x4)La
ne 7
Lane
4 PHY0(x4)La
ne 3
Lane
0
PCIe x8/x4
(Port 1)
Refclk distribution on the package substrate
Refclk1 Refclk0100MHz
PCIe x16/x8/x4
(Port 0)
PCIe x4
(Port 2)
100MHz
2.1.4. Reset
There is only one PERST# (pin_perst_n) pin on P-Tile. Therefore, togglingpin_perst_n will affect the entire P-Tile. If the P-Tile x16 port is bifurcated into twox8 Endpoints, toggling pin_perst_n will affect both x8 Endpoints. To reset each portindividually, use the in-band mechanism such as Hot Reset and the Function-LevelReset (FLR). Following are the guidelines for implementing the P-Tile pin_perst_nreset signal:
• pin_perst_n is a "power good" indicator from the associated power domain (towhich P-Tile is connected). Also, it shall qualify that both the P-Tile refclk0 andrefclk1 are stable. If one of the reference clocks becomes stable later, deassertpin_perst_n after this reference clock becomes stable.
• pin_perst_n assertion is required for proper Autonomous P-Tile functionality. InAutonomous mode, P-Tile can successfully link up upon the release ofpin_perst_n regardless of the FPGA fabric configuration and will send out CRS(Configuration Retry Status) until the FPGA fabric is configured and ready.
The following is an example where a single PERST# (pin_perst_n) is driven withindependent refclk0 and refclk1. In this example, the add-in card (FPGA and Soc)is powered up first. P-Tile refclk0 is fed by the on-board free-running oscillator. P-Tile refclk1 driven by the Host becomes stable later. Hence, the PERST# isconnected to the Host.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
15
Figure 6. Single PERST# Connection in Bifurcated 2x8 Mode
2.2. Functional Description
2.2.1. PMA/PCS
The P-Tile Avalon-MM IP for PCI Express contains Physical Medium Attachment (PMA)and PCI Express Physical Coding Sublayer (PCIe PCS) blocks for handling the Physicallayer (PHY) packets. The PMA receives and transmits high-speed serial data on theserial lanes. The PCS acts as an interface between the PMA and the PCIe controller,and performs functions like data encoding and decoding, scrambling anddescrambling, block synchronization etc. The PCIe PCS in the P-Tile Avalon-MM IP forPCI Express is based on the PHY Interface for PCI Express (PIPE) Base Specification4.4.1.
In this IP, the PMA consists of up to four quads. Each quad contains a pair of transmitPLLs and four SerDes lanes capable of running up to 16 GT/s to perform the variousTX and RX functions.
PLLA generates the required transmit clocks for Gen1/Gen2 speeds, while PLLBgenerates the required clocks for Gen3/Gen4 speeds. For the x8 and x16 lane widths,one of the quads acts as the master PLL source to drive the clock inputs for each ofthe lanes in the other quads.
The PMA performs functions such as serialization/deserialization, clock data recovery,and analog front-end functions such as Continuous Time Linear Equalizer (CTLE),Decision Feedback Equalizer (DFE) and transmit equalization.
The transmitter consists of a 3-tap equalizer with one tap of pre-cursor, one tap ofmain cursor and one tap of post-cursor.
The receiver consists of attenuation (ATT), CTLE, Voltage gain amplifier (VGA) and a5-tap DFE blocks that are adaptive for Gen3/Gen4 speeds. RX Lane Margining issupported by the PHY. The Lane Margining supports timing margining only. Theoptional voltage margining is not supported. Timing margining capabilities/parametersare as follows:
• Maximum Timing Offset: -0.2UI to +0.2UI.
• Number of timing steps in each direction: 9.
• Independent left and right timing margining is supported.
• Independent Error Sampler is not supported (lane margining may produce logicalerrors in the data stream and cause the LTSSM to go to the Recovery state).
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
16
The PHY layer uses a fixed 16-bit PCS-PMA interface width to output the PHY clock(core_clk). The frequency of this clock is dependent on the current link speed. Referto Table 8 on page 13 for the frequencies at various link speeds.
Related Information
PHY Interface for PCI Express (PIPE) Base Specification
2.2.2. Data Link Layer Overview
The Data Link Layer (DLL) is located between the Transaction Layer and the PhysicalLayer. It maintains packet integrity and communicates (by DLL packet transmission) atthe PCI Express link level.
The DLL implements the following functions:
• Link management through the reception and transmission of DLL Packets (DLLP),which are used for the following functions:
— Power management of DLLP reception and transmission
— To transmit and receive ACK/NAK packets
— Data integrity through the generation and checking of CRCs for TLPs andDLLPs
— TLP retransmission in case of NAK DLLP reception or replay timeout, using theretry (replay) buffer
— Management of the retry buffer
— Link retraining requests in case of error through the Link Training and StatusState Machine (LTSSM) of the Physical Layer
Figure 7. Data Link LayerTo Transaction Layer
Tx Transaction LayerPacket Description & Data Transaction Layer
Packet Generator
Retry Buffer
To Physical Layer
Tx Packets
Ack/NackPackets
RX Datapath
TX Datapath
Rx Packets
DLLPChecker
Transaction LayerPacket Checker
DLLPGenerator
Tx Arbitration
Data Link Controland Management
State Machine
Control& StatusConfiguration Space
Tx Flow Control Credit Information
Rx Flow Control Credit Information
Rx Transation LayerPacket Description & Data
PowerManagement
Function
Note:(1) The L0s (Standby) or L1 (Low Power Standby) states are not supported.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
17
The DLL has the following sub-blocks:
• Data Link Control and Management State Machine—This state machine connects toboth the Physical Layer’s LTSSM state machine and the Transaction Layer. Itinitializes the link and flow control credits and reports status to the TransactionLayer.
• Power Management—This function handles the handshake to enter low powermode. Such a transition is based on register values in the Configuration Space andreceived Power Management (PM) DLLPs. For more details on the power statessupported by the P-Tile Avalon-MM IP for PCIe, refer to section Power ManagementInterface on page 67.
• Data Link Layer Packet Generator and Checker—This block is associated with theDLLP’s 16-bit CRC and maintains the integrity of transmitted packets.
• Transaction Layer Packet Generator—This block generates transmit packets,including a sequence number and a 32-bit Link CRC (LCRC). The packets are alsosent to the retry buffer for internal storage. In retry mode, the TLP generatorreceives the packets from the retry buffer and generates the CRC for the transmitpacket.
• Retry Buffer—The retry buffer stores TLPs and retransmits all unacknowledgedpackets in the case of NAK DLLP reception. In case of ACK DLLP reception, theretry buffer discards all acknowledged packets.
• ACK/NAK Packets—The ACK/NAK block handles ACK/NAK DLLPs and generates thesequence number of transmitted packets.
• Transaction Layer Packet Checker—This block checks the integrity of the receivedTLP and generates a request for transmission of an ACK/NAK DLLP.
• TX Arbitration—This block arbitrates transactions, prioritizing in the followingorder:
— Initialize FC Data Link Layer packet
— ACK/NAK DLLP (high priority)
— Update FC DLLP (high priority)
— PM DLLP
— Retry buffer TLP
— TLP
— Update FC DLLP (low priority)
— ACK/NAK FC DLLP (low priority)
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
18
2.2.3. Transaction Layer Overview
The following figure shows the major blocks in the P-Tile Avalon-MM IP for PCI ExpressTransaction Layer:
Figure 8. P-Tile Avalon-MM IP for PCI Express Transaction Layer Block Diagram
Data Link Layer +Physical Layer
RAS
RX
CONFIG
TX
Avalon-ST RX
User Avalon-MM
Avalon-ST TX
RASCPL TimeoutLogic
Avalon-MM TX
Avalon-MM RX
Avalo
n-M
M Br
idge
The RAS (Reliability, Availability, and Serviceability) block includes a set of features tomaintain the integrity of the link.
For example: Transaction Layer inserts an optional ECRC in the transmit logic andchecks it in the receive logic to provide End-to-End data protection.
When the application logic sets the TLP Digest (TD) bit in the Header of the TLP, the P-Tile Avalon-MM IP for PCIe will append the ECRC automatically.
The TX block sends out the TLPs that it receives as-is. It also sends the informationabout non-posted TLPs to the Completion (CPL) Timeout Block for CPL timeoutdetection.
The P-Tile Avalon-MM IP for PCI Express RX block consists of two main blocks:
• Filtering block: This module checks if the TLP is good or bad and generates theassociated error message and completion. It also tracks received completions andupdates the completion timeout (CPL timeout) block.
• RX Buffer Queue: The P-Tile IP for PCIe has separate queues for posted/non-posted transactions and completions. This avoids head-of-queue blocking on thereceived TLPs and provides flexibility to extract TLPs according to the PCIeordering rules.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
19
Figure 9. P-Tile Avalon-MM IP for PCI Express RX Block Overview
Avalon-ST
RX Buffer Queue Filter
TLP Filtering
Config TX
MSG ERR
CFG Data
Received CPLProcessing (*)
MessageProcessingCPL
NP
P
MSG
DataLinkLayer
Trash
Avalon-ST
User Avalon-MM
RoutingAvalon-MM Avalon-STLogical
PHYLayer
Avalon-MM
Bridge
WidthandRate
Adapter
Note: The Received CPL Processing block includes the CPL tracking mechanism.
2.2.4. Avalon-MM Bridge
As described in Table 7 on page 10, the P-Tile Avalon-MM Bridge can support threemodes of operation:
• Endpoint mode with Data Movers.
• Endpoint mode.
• Root Port mode.
2.2.4.1. Endpoint Mode with Data Movers
When the Avalon-MM Bridge is used in this mode, the following modules are enabled:
• Read Data Mover (RDDM)
• Write Data Mover (WRDM)
• Bursting Master (BAM) in Non-Bursting Mode
The following figure shows how the DMA example design that you can generate usingthe Intel Quartus Prime software interfaces with the P-Tile Avalon-MM IP to performDMA operations. If you are not using the provided DMA example design, you need toimplement your custom DMA Controller and BAR Interpreter in your application logic.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
20
Figure 10. P-Tile Avalon-MM IP in Endpoint Mode with Data Movers Enabled
DMA Example Design P-Tile Avalon-MM IP
Qsys Interconnect
Memory
Writedata mover(512 bits)
S
S
S
SC
M
WASTOASTO
ASTIASTIASTO
M
Readdata mover(512 bits) ASTI
ASTOASTIASTIASTO
M
Bursting Master(non-bursting
mode)(512 bits)
ASTI
ASTOCM
WASTOWASTI
RASTORASTORASTI
DMA Controller
BARInterpreter
CompletionRe-ordering
TX
RX
Avalon-ST Interface
P-Tile PCIe
Hard IP
512
512
Avalon-ST SinkASTI
ASTO Avalon-ST Source
M Avalon-MM Master
S Avalon-MM Slave
C Avalon Conduit
In this DMA example design, the BAM is used in non-bursting mode by the host toprogram the Control and Status registers of the DMA controller in the user Avalon-MMspace. The DMA controller, after being programmed, sends descriptor-fetchinginstructions to the host via the RDDM. After the fetched descriptors are processed bythe WRDM and RDDM, status and/or MSI-X messages are sent to the host via theWRDM in “Immediate” mode. In this mode, the data payload is embedded in bits[31:0] or [63:0] of the fetched descriptors that the WRDM receives (depending onwhether a one- or two-dword immediate transfer is needed respectively). For moredetails on immediate transfers, refer to Write Data Mover Avalon-ST Descriptor Sinkson page 49.
The RDDM uses PCIe memory read TLPs and Avalon-MM write transactions (which canbe bursting transactions) to move large amounts of data from the host memory inPCIe space to the local FPGA memory in Avalon-MM space. On the other hand, theWRDM uses PCIe memory write TLPs and Avalon-MM read transactions to move largeamounts of data from the FPGA memory in Avalon-MM space to the host memory inPCIe space. The Data Movers' transfers are controlled by descriptors that are providedto the Data Movers through one of their Avalon-ST sink interfaces. The Data Moversreport the transfers’ status through their Avalon-ST source interfaces.
2.2.4.2. Endpoint Mode
In this mode, the external master (in user logic) sends memory reads and writesupstream via the Bursting Slave. The following modules are enabled:
• Bursting Slave (in bursting mode)
• Bursting Master (in non-bursting mode)
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
21
Figure 11. P-Tile Avalon-MM IP in Endpoint Mode
DMA Application Logic P-Tile Avalon-MM IP
Qsys Interconnect
S Memory
S
M Custom DMA
ControllerM
SC
MBursting Master (non-bursting
mode)(512 bits)
CM
BurstingSlave
(512 bits) ASTI
ASTO
ASTI
ASTO
S
DMA Control/
BAR AccessLogic
CompletionRe-ordering
Avalon-ST Interface
P-Tile PCIe
Hard IP
512
512
TX
RX
ASTI Avalon-ST Sink
ASTO Avalon-ST Source
M Avalon-MM Master
S Avalon-MM Slave
C Avalon Conduit
The external Avalon-MM master can be a custom DMA controller that uses theBursting Slave in the IP core to send memory reads and writes upstream. Thesememory reads and writes can be up to 512-bytes long. The reordering buffer in the IPcore reorders the Completion TLPs received over the PCIe link and sends them to theBursting Slave.
The Bursting Master provides the host with access to the registers and memory in theAvalon-MM address space of the FPGA. It converts PCIe memory reads and writes toAvalon-MM reads and writes.
Registers in the custom DMA controller can be programmed by software via theBursting Master port.
2.2.4.3. Root Port Mode
In this mode, the IP core needs to be able to process memory read and write TLPscoming from the DMA controller that resides on the Endpoint side. The followingmodules are enabled:
• Bursting Master (in bursting and non-bursting modes)
• Bursting Slave (in non-bursting mode)
• Control Register Access
Figure 12. P-Tile Avalon-MM IP in Root Port Mode
Custom DMA Application Logic P-Tile Avalon-MM IP
Qsys Interconnect
S Memory
SLocal
ProcessorM
M
SC
MBurstingMaster
(512 bits) ASTI
ASTOCM
Control RegisterAccess
(512 bits) ASTI
ASTOS
Bursting Slave(non-bursting
mode)(512 bits)
ASTI
ASTOS
S
S
BurstingSlave
CompletionRe-ordering
Avalon-ST Interface
P-Tile PCIe
Hard IP
512
512RX
TX
ASTO Avalon-ST Source
ASTI Avalon-ST Sink
M Avalon-MM Master
S Avalon-MM Slave
C Avalon Conduit
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
22
The IP core must be able to generate and process configuration reads and writes tothe Endpoint and to the Hard IP configuration registers. This is done via theConfiguration Slave. Since the DMA controller resides on the Endpoint side, its controlregisters need to be programmed by the FPGA local processor. Using the BurstingSlave (in non-bursting mode), the local processor can program the Endpoint controlregisters for DMA operations. The Endpoint can also send updates of its DMA status tothe local processor via the Bursting Master.
2. IP Architecture and Functional Description
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
23
3. ParametersThis chapter provides a reference for all the parameters that are configurable in theIntel Quartus Prime IP Parameter Editor for the P-Tile Avalon-MM IP for PCIe.
3.1. Top-Level Settings
Table 9. Top-Level Settings
Parameter Value Default Value Description
Hard IP Mode
Gen4x16, Interface - 512-bitGen3x16, Interface - 512-bitGen4x8, Interface - 256-bitGen3x8, Interface - 256-bitGen4x4, Interface - 128-bitGen3x4, Interface - 128-bit
Gen3x16,Interface - 512-bit
Select the lane data rate and lanewidth.
Note:
Refer to the P-Tile IP forPCI Express IP CoreRelease Notes for thematrix of configurationssupported by the P-TileAvalon memory mapped IPfor PCI Express.
Port ModeRoot PortNative Endpoint
Native Endpoint Specifies the port type.
Enable PHY Reconfiguration True/False False Enable the PHY ReconfigurationInterface.
PLD Clock Frequency
500 MHz400 MHz350 MHz250 MHz
250 MHz
Select the frequency of theApplication clock.
Note:
400 MHz / 350 MHz are forGen4 support in IntelStratix 10 DX. 500 MHz /400 MHz / 350 MHz are forGen4 support in IntelAgilex. 250 MHz is forGen3 support.
Enable SRIS Mode True/False FalseEnable the Separate ReferenceClock with Independent SpreadSpectrum Clocking (SRIS) feature.
P-Tile Sim Mode True/False False
Enabling this parameter willenable P-Tile simulations to runfaster.
Note: Do not enable this option ifyou need to run synthesis.
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
The following figure shows how to enable Root Port mode:
Figure 13. Intel P-Tile Avalon-MM Top-Level IP Parameter Editor for a Gen3x4 Hard IPin Root Port Mode
3.2. Core Parameters
Depending on which Hard IP Mode you choose in the Top-Level Settings tab, youwill see different tabs for setting the core parameters.
Figure 14. Intel P-Tile Avalon-MM Top-Level IP Parameter Editor for a x8 Hard IP ModeIf you choose a x8 mode (either Gen4 or Gen3), the PCIe0 Settings and PCIe1 Settings tabs will appear.
3. Parameters
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
25
3.2.1. Base Address Registers
Table 10. BAR Registers
Parameter Value Description
BAR0 Type
Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory
If you select 64-bit prefetchable memory, 2contiguous BARs are combined to form a 64-bitprefetchable BAR; you must set the highernumbered BAR to Disabled.Defining memory as prefetchable allows contiguousdata to be fetched ahead. Prefetching memory isadvantageous when the requestor may requiremore data from the same region than wasoriginally requested. If you specify that a memoryis prefetchable, it must have the following 2attributes:• Reads do not have side effects such as
changing the value of the data read.• Write merging is allowed.
BAR1 TypeDisabled32-bit non-prefetchable memory32-bit prefetchable memory
For a definition of prefetchable memory, refer tothe BAR0 Type description.
BAR2 Type
Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory
For a definition of prefetchable memory and adescription of what happens when you select the64-bit prefetchable memory option, refer to theBAR0 Type description.
BAR3 TypeDisabled32-bit non-prefetchable memory32-bit prefetchable memory
For a definition of prefetchable memory, refer tothe BAR0 Type description.
BAR4 Type
Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory
For a definition of prefetchable memory and adescription of what happens when you select the64-bit prefetchable memory option, refer to theBAR0 Type description.
BAR5 TypeDisabled32-bit non-prefetchable memory32-bit prefetchable memory
For a definition of prefetchable memory, refer tothe BAR0 Type description.
BARn Size 128 Bytes - 16 EBytesSpecifies the size of the address space accessibleto BARn when BARn is enabled.n = 0, 1, 2, 3, 4 or 5
Expansion ROM
Disabled4 KBytes - 12 bits8 KBytes - 13 bits16 KBytes - 14 bits32 KBytes - 15 bits64 KBytes - 16 bits128 KBytes - 17 bits256 KBytes - 18 bits512 KBytes - 19 bits1 MByte - 20 bits2 MBytes - 21 bits4 MBytes - 22 bits8 MBytes - 23 bits16 MBytes - 24 bits
Specifies the size of the expansion ROM from 4KBytes to 16 MBytes when enabled.
3. Parameters
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
26
3.2.2. Device Identification Registers
The following table lists the default values of the Device ID registers. You can use theparameter editor to change the values of these registers.
Table 11. Device ID Registers
Register Name Range Default Value Description
Vendor ID 16 bits 0x00001172
Sets the read-only value of theVendor ID register. This parametercannot be set to 0xFFFF per thePCI Express Base Specification.
Note: Set your own Vendor ID bychanging this parameter.
Address offset: 0x000.
Device ID 16 bits 0x00000000
Sets the read-only value of theDevice ID register. This register isonly valid in the Type 0 (Endpoint)Configuration Space.Address offset: 0x000.
Revision ID 8 bits 0x00000001Sets the read-only value of theRevision ID register.Address offset: 0x008.
Class Code 24 bits 0x00FF0000
Sets the read-only value of theClass Code register.Address offset: 0x008.This parameter cannot be set to0x0 per the PCI Express BaseSpecification.
Subsystem Vendor ID 16 bits 0x00000000
Sets the read-only value of theSubsystem Vendor ID register inthe PCI Type 0 ConfigurationSpace. This parameter cannot beset to 0xFFFF per the PCI ExpressBase Specification. This value isassigned by PCI-SIG to the devicemanufacturer.Address offset: 0x02C.
Subsystem Device ID 16 bits 0x00000000
Sets the read-only value of theSubsystem Device ID register inthe PCI Type 0 ConfigurationSpace.Address offset: 0x02C.
3.2.3. PCI Express and PCI Capabilities Parameters
For each core (PCIe0/PCIe1/PCIe2/PCIe3), the PCI Express / PCI Capabilities tabcontains separate tabs for the device, PRS (Endpoint mode), MSI (Endpoint mode),ACS capabilities (Root Port mode), slot (Root Port mode), MSI-X, and legacy interruptpin register parameters.
3. Parameters
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
27
Figure 15. PCI Express / PCI Capabilities Parameters
3.2.3.1. Device Capabilities
Table 12. Device Capabilities
Parameter Value Default Value Description
Maximum payload sizessupported
128 bytes256 bytes512 bytes
512 bytes
Specifies the maximumpayload size supported. Thisparameter sets the read-only value of the maxpayload size supported fieldof the Device Capabilitiesregister.
3.2.3.2. Link Capabilities
Table 13. Link Capabilities
Parameter Value Default Value Description
Link port number (RootPort only) 0 - 255 1
Sets the read-only value ofthe port number field in theLink Capabilitiesregister. This parameter isfor Root Ports only. It shouldnot be changed.
Slot clock configuration True/False True
When this parameter isTrue, it indicates that theEndpoint uses the samephysical reference clock thatthe system provides on theconnector. When it is False,the IP core uses anindependent clock regardlessof the presence of a
continued...
3. Parameters
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
28
Parameter Value Default Value Description
reference clock on theconnector. This parametersets the Slot ClockConfiguration bit (bit 12) inthe PCI Express LinkStatus register.
3.2.3.3. MSI Capabilities
Table 14. MSI Capabilities
Parameter Value Default Value Description
PF0 Enable MSI True/False False
Enables MSI functionality forPF0.If this parameter is True,the Number of MSImessages requestedparameter will appearallowing you to set thenumber of MSI messages.
PF0 MSI 64-bitAddressing True/False False Enables or disables MSI 64-
bit addressing for PF0.
PF0 MSI Extended DataCapable True/False False
Enables or disables MSIextended data capability forPF0.
PF0 Number of MSImessages requested
1248
1632
1
Sets the number ofmessages that theapplication can request inthe multiple messagecapable field of the MessageControl register.
3.2.3.4. MSI-X Capabilities
Table 15. MSI-X Capabilities
Parameter Value Default Value Description
Enable MSI-X (Endpointonly) True/False False
Enables the MSI-Xfunctionality.For SR-IOV: VFs and PFs arealways MSI-X capable.
MSI-X Table Size0x0 - 0x7FF (only values ofpowers of two minus 1 are
valid)0
System software reads thisfield to determine the MSI-Xtable size <n>, which isencoded as <n-1>.For example, a returnedvalue of 2047 indicates atable size of 2048. This fieldis read-only.Address offset:0x068[26:16]
MSI-X Table Offset 0x0 - 0xFFFFFFFF 0
Points to the base of theMSI-X table. The lower 3 bitsof the table BAR indicator(BIR) are set to zero bysoftware to form a 64-bit
continued...
3. Parameters
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
29
Parameter Value Default Value Description
qword-aligned offset. Thisfield is read-only after beingprogrammed.
Table BAR indicator 0x0 - 0x5 0
Specifies which one of afunction's BARs, locatedbeginning at 0x10 inConfiguration Space, is usedto map the MSI-X table intomemory space. This field isread-only after beingprogrammed.
Pending bit array (PBA)offset 0x0 - 0xFFFFFFFF 0
Used as an offset from theaddress contained in one ofthe function's Base Addressregisters to point to the baseof the MSI-X PBA. The lower3 bits of the PBA BIR are setto zero by software to forma 32-bit qword-alignedoffset. This field is read-onlyafter being programmed.
PBA BAR indicator 0x0 - 0x5 0
Specifies the function's BaseAddress register, locatedbeginning at 0x10 inConfiguration Space, thatmaps the MSI-X PBA intomemory space. This field isread-only after beingprogrammed.
3.2.3.5. Power Management
Table 16. Power Management
Parameter Value Default Value Description
Enable L0s acceptablelatency
Maximum of 64 nsMaximum of 128 nsMaximum of 256 nsMaximum of 512 nsMaximum of 1 usMaximum of 2 usMaximum of 4 usNo limit
Maximum of 64 ns
This design parameterspecifies the maximumacceptable latency that theapplication layer can toleratefor any link between thedevice and the root complexto exit the L0s state. It setsthe read-only value of theEndpoint L0s acceptablelatency field of the DeviceCapabilities Register(0x084).This Endpoint does notsupport the L0s or L1 states.However, in a switchedsystem, there may be linksconnected to switches thathave L0s and L1 enabled.This parameter is set toallow system configurationsoftware to read theacceptable latencies for alldevices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).
continued...
3. Parameters
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
30
Parameter Value Default Value Description
This setting is disabled forRoot Ports.The default value of thisparameter is 64 ns. This isthe safest setting for mostdesigns.
Endpoint L1 acceptablelatency
Maximum of 1 usMaximum of 2 usMaximum of 4 usMaximum of 8 usMaximum of 16 usMaximum of 32 usMaximum of 64 usNo limit
Maximum of 1 us
This value indicates theacceptable latency that anEndpoint can withstand inthe transition from the L1state to L0 state. It is anindirect measure of theEndpoint’s internal buffering.It sets the read-only value ofthe Endpoint L1 acceptablelatency field of the DeviceCapabilities Register.This Endpoint does notsupport the L0s or L1 states.However, a switched systemmay include links connectedto switches that have L0sand L1 enabled. Thisparameter is set to allowsystem configurationsoftware to read theacceptable latencies for alldevices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).This setting is disabled forRoot Ports.
3.2.3.6. Vendor Specific Extended Capability (VSEC) Registers
Table 17. VSEC Register
Parameter Value Default Value Description
Vendor Specific ExtendedCapability 0/1 0 Enables the Vendor Specific
Extended Capability (VSEC).
User ID register from theVendor Specific ExtendedCapability
0 - 65534 0
Sets the read-only value ofthe 16-bit User ID registerfrom the Vendor SpecificExtended Capability. Thisparameter is only valid forEndpoints.
Drops Vendor Type0Messages 0/1 0
When this parameter is setto 1, the IP core dropsvendor Type 0 messageswhile treating them asUnsupported Requests (UR).When it is set to 0, the IPcore passes these messageson to the user logic.
Drops Vendor Type1Messages 0/1 0
When this parameter is setto 1, the IP core silentlydrops vendor Type 1messages.
continued...
3. Parameters
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
31
Parameter Value Default Value Description
When it is set to 0, the IPcore passes these messageson to the user logic.
3.2.3.7. Legacy Interrupt Pin Register
Table 18. Legacy Interrupts Parameters
Parameter Value Default Value Description
Enable Legacy Interruptsfor PF0
True/False False Enable Legacy Interrupts(INTx) for PF0 of PCIe0.
Set Interrupt Pin for PF0 NO INTINTA
NO INT When Legacy Interrupts arenot enabled, the only optionavailable is NO INT.When Legacy Interrupts areenabled, the only optionavailable is INTA.
3.2.4. Configuration, Debug and Extension Options
Table 19. Configuration, Debug and Extension Options
Parameter Value Default Value Description
Gen 3 Requestedequalization far-end TXpreset vector
0 - 65535 0x00000004
Specifies the Gen 3requested phase 2/3 far-endTX preset vector. Choosing avalue different from thedefault is not recommendedfor most designs.
Gen 4 Requestedequalization far-end TXpreset vector
0 - 65535 0x00000270
Specifies the Gen 4requested phase 2/3 far-endTX preset vector. Choosing avalue different from thedefault is not recommendedfor most designs.
Enable Debug Toolkit True/False False
Enable the P-Tile DebugToolkit for JTAG-basedSystem Console debugaccess.
Enable HIP dynamicreconfiguration of PCIeregisters
True/False FalseEnable the user Hard IPreconfiguration Avalon-MMinterface.
3.3. Avalon-MM Settings
Table 20. Avalon-MM Parameters
Parameter Value Default Value Description
Endpoint Settings
Enable 512 PCIe ReadOutstanding Requests forRead Data Mover
True/False False When this is enabled, theRead Data Mover can issueup to 512 PCIe Readoutstanding requests.However, no completion
continued...
3. Parameters
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
32
Parameter Value Default Value Description
reordering is performedwithin the Avalon-MMBridge.When this is not enabled,the Read Data Mover canonly issue up to 64 PCIeRead outstanding requests.In this case, the Avalon-MMBridge performs completionreordering.Note: This feature is not
available in the 20.2release of IntelQuartus Prime. Only64 outstanding Readrequests can besupported in thisrelease. Support for512 outstandingRead requests maybe available in afuture release.
Address width of ReadData Mover
{10:64} 64 Address width of Read DataMover.
Address width of WriteData Mover
{10:64} 64 Address width of Write DataMover.
Export interrupt conduitinterfaces
True/False False Export internal signals tosupport generation ofLegacy Interrupts/multipleMSI/MSI-X.
Enable Bursting SlaveMode
True/False False Enable bursting Avalon-MMSlave Interface. This willenable the Endpoint mode(where the IP's BAS andBAM modules are enabled,but its Data Movers are notenabled).
Address width of BurstingMaster
{10:64} 64 Only present in Root Portmode.In Endpoint modes, thisparameter is set by thelargest BAR address width.
Root Port Settings
3. Parameters
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
33
4. Interfaces
4.1. Overview
The P-Tile Avalon-MM IP for PCIe includes many interface types to implement differentfunctions.These include:
• High-performance bursting master (BAM) and slave (BAS) Avalon-MM interfaces totranslate between PCIe TLPs and Avalon-MM memory-mapped reads and writes
• Read and Write Data Movers to transfer large blocks of data
• Standard PCIe serial interface to transfer data over the PCIe link
• System interfaces for interrupts, clocking, reset
• Optional reconfiguration interface to dynamically change the value ofConfiguration Space registers at run-time
• Optional status interface for debug
Unless otherwise noted, all interfaces to the Application layer are synchronous to therising edge of app_clk. For Gen3 operation, this clock runs at 250 MHz. Thefrequency of this clock is exactly half the frequency of the pld_clk generated by theIP, with 0ppm difference. You enable the interfaces using the component IP ParameterEditor.
Read Data Mover (RDDM) interface: This interface transfers DMA data from the PCIesystem memory to the memory in Avalon-MM address space.
Write Data Mover (WRDM) interface: This interface transfers DMA data from thememory in Avalon-MM address space to the PCIe system memory.
Bursting Master (BAM) interface: This interface provides host access to the registersand memory in Avalon-MM address space. The Busting Master module converts PCIeMemory Reads and Writes to Avalon-MM Reads and Writes.
Bursting Slave (BAS) interface: This interface allows the user application in the FPGAto access the PCIe system memory. The Bursting Slave module converts Avalon-MMReads and Writes to PCIe Memory Reads and Writes.
Control Register Access (CRA) interface: This optional, 32-bit Avalon-MM Slaveinterface provides access to the Control and Status registers. You must enable thisinterface when you enable address mapping for any of the Avalon-MM slaves or ifinterrupts are implemented. The address bus width of this interface is fixed at 15 bits.The prefix for this interface is cra*.
The modular design of the P-Tile Avalon-MM IP for PCIe lets you enable just theinterfaces required for your application.
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
Table 21. Avalon-MM Interface Summary
Avalon-MM Type Data Bus Width Max Burst Size Byte EnableGranularity
Max OutstandingRead Request
Bursting Slave 512 bits
Bursting Mode: 8cycles
Non-Bursting Mode: 1cycle
dword/byteBursting Mode: 64
Non-Bursting Mode: 1
Bursting Master 512 bits
Bursting Mode: 8cycles
Non-Bursting Mode: 1cycle
dword/byteBursting Mode: 32
Non-Bursting Mode: 1
Read Data MoverWrite Master 512 bits 8 cycles dword N/A
Write Data MoverRead Master 512 bits 8 cycles dword 32
Control RegisterAccess 32 bits 1 cycle byte 1
Note: The number of read requests issued by the Write Data Mover's Avalon-MM ReadMaster is controlled by the assertion of waitrequest by the connected slave(s). TheRead Master can handle 128 outstanding cycles of data. You cannot set this parameterin Platform Designer. The slave needs to correctly back-pressure the master once itcannot handle the incoming requests.
Note: The 512-bit Bursting Slave interface does not support transactions where all byteenables are set to 0.
Note: All transfers of four bytes or more are done in multiples of dwords.
4.2. Clocks and Resets
4.2.1. Interface Clock Signals
Table 22. Interface Clock Signals
Name I/O Description EP/RP Clock Frequency
p<n>_app_clk(where n = 0, 1,
2, 3)O
This is the application clock generatedfrom coreclkout_hip or from thesame source as refclk. This is a
per-port signal.
EP/RPNative Gen3: 250 MHz
Native Gen4: 350 MHz / 400MHz
coreclkout_hip O
This clock drives the ApplicationLayer.
The frequency depends on the datarate and the number of lanes being
used.
EP/RP
Native Gen3: 250 MHzNative Gen4: 350 MHz / 400MHz (Intel Stratix 10 DX) /
500 MHz (Intel Agilex)
refclk[1:0] I
These are the input reference clocksfor the IP core. These clocks must be
free-running.For more details on how to connectthese clocks, refer to the section
Clock Sharing in Bifurcation Modes.
EP/RP
100 MHz ± 300 ppmWhen the Enable SRIS Modeparameter is enabled in the IP
Parameter Editor, the P-TileAvalon-MM IP can
communicate with a linkpartner whose clock domain is
not synchronized to therefclk domain of the P-Tile.
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
35
Name I/O Description EP/RP Clock Frequency
In this mode of operation, P-Tile and its link partner canboth have their own spread
spectrum clocks.
p<n>_hip_reconfig_clk (where n = 0,
1, 2, 3)I
Clock for the hip_reconfiginterface. This is an Avalon-MM
interface. It is an optional interfacethat is enabled when the Enable HIPdynamic reconfiguration of PCIe
registers option in the PCIeConfiguration, Debug and
Extension Options tab is enabled.
EP/RP50 MHz - 125 MHz (range)100 MHz (recommended)
xcvr_reconfig_clk I
Clock for the PHY reconfigurationinterface. This is an Avalon-MM
interface. This optional interface isenabled when you turn on the EnablePHY reconfiguration option in the
Top-Level Settings tab. Thisinterface is shared among all the
cores.
EP/RP50 MHz - 125 MHz (range)100 MHz (recommended)
4.2.2. Interface Reset Signals
Table 23. Interface Reset Signals
Signal Name Direction Clock EP/RP Description
pin_perst_n Input Asynchronous
EP/RP This is an active-low input to the PCIe Hard IP, andimplements the PERST# function defined by the PCIespecification.
p0_reset_status_n Output Synchronous
EP/RP This active-low signal is held low until pin_perst_n hasbeen deasserted and the PCIe Hard IP has come out ofreset. This signal is synchronous to coreclkout_hip.When port bifurcation is used, there is one such signal foreach interface. The signals are differentiated by theprefixes pn.
p0_link_req_rst_n Output Synchronous
EP/RP This active-low signal is asserted by the PCIe Hard IPwhen it is about to go into reset.The Avalon-MM Bridge IP will reset all its PCIe-relatedregisters and queues including anything related to tags.It will also stop sending packets to the PCIe Hard IP untilthe Bus Master Enable bit is set again. The Bridge willalso ignore any packet received from the PCIe Hard IP.
p0_pld_warm_rst_rdy Input Synchronous
EP/RP This active-high signal is asserted by the user logic inresponse to p0_link_req_rst_n when it hascompleted its pre-reset tasks.Note: When not using this signal, set it to 1'b1.
ninit_done Input Asynchronous
EP/RP A "1" on this active-low signal indicates that the FPGAdevice is not yet fully configured. A "0" indicates thedevice has been configured and is in normal operatingmode.Intel recommends using the output of the Reset ReleaseIntel Fpga IP to drive this ninit_done input. For moredetails on this IP, refer to the Application Note AN891 at https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an891.pdf
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
36
4.3. Avalon-MM Interface
The figures below provide the top-level block diagrams of the P-Tile Avalon-MM IP withall interfaces while operating in Endpoint mode with Data Movers or in Endpoint mode.These interfaces are described in more details in following sections.
Figure 16. P-Tile Avalon-MM IP for PCIe in Endpoint Mode with Data Movers Top-LevelBlock Diagram
P-Tile Avalon-MM IP for PCI Express
Clocksrefclk[1:0]
coreclkout_hip
p0_wrdm_read_o
p0_wrdm_address_o[63:0]p0_wrdm_readdata_i[511:0]p0_wrdm_burst_count_o[3:0]p0_wrdm_wait_request_ip0_wrdm_readdatavalid_i
Write Data Mover Interface:Writes Data from FPGA
Memory to Host Memory
tx_n_out[15:0]rx_p_in[15:0] Serial Data
p0_rddm_write_op0_rddm_address_o[63:0]p0_rddm_write_data_o[511:0]p0_rddm_burst_count_o[3:0]p0_rddm_byte_enable_o[63:0]p0_rddm_wait_request_i
Read Data Mover Interface:Writes Data from Host
Memory to FPGA Memory
Hard IPReconfiguration(Optional)
p0_hip_reconfig_clkp0_hip_reconfig_address[20:0]
p0_hip_reconfig_readp0_hip_reconfig_readdata[7:0]
p0_hip_reconfig_writep0_hip_reconfig_writedata[7:0]
p0_hip_reconfig_waitrequest
p0_hip_reconfig_readdatavalid
p0_bam_pfnum_o[1:0]p0_bam_bar_o[2:0]
p0_bus_master_enable_o[1:0]
p0_wrdm_desc_valid_i
p0_bam_waitrequest_i
Write Data MoverNormal Descriptor
Queue
p0_pld_warm_rst_rdy_ip0_reset_status_n pin_perst_n ninit_done
Reset
xcvr_reconfig_clkxcvr_reconfig_address[25:0]
xcvr_reconfig_readxcvr_reconfig_readdata[7:0]xcvr_reconfig_readdatavalid
xcvr_reconfig_writexcvr_reconfig_writedata[7:0]
xcvr_reconfig_waitrequest
PHYReconfiguration(Optional)
p0_wrdm_desc_ready_o
p0_wrdm_desc_data_i[173:0]
Write Data MoverPriority Descriptor
Queue
p0_wrdm_prio_ready_op0_wrdm_prio_valid_ip0_wrdm_prio_data_i[173:0]
Read Data MoverPriority Descriptor
Queue
Read Data MoverNormal Descriptor
Queue
p0_rddm_prio_ready_op0_rddm_prio_valid_ip0_rddm_prio_data_i[173:0]
p0_rddm_desc_ready_op0_rddm_desc_valid_ip0_rddm_desc_data_i[173:0]
Bursting Avalon-MMMaster Interface
p0_rddm_pfnum_o[1:0]
p0_wrdm_pfnum_o[1:0]
p0_wrdm_byteenable_o[63:0]p0_wrdm_response_i[1:0]
p0_rddm_tx_valid_op0_rddm_tx_data_o[31:0]
p0_wrdm_tx_valid_op0_wrdm_tx_data_o[31:0]
p0_pld_link_req_rst_o
p0_app_clk
dummy_user_avmm_rst
tx_p_out[15:0]
rx_n_in[15:0]
p0_bam_response_i[1:0]p0_bam_address_o[BAM_ADDR_WIDTH-1:0]p0_bam_byteenable_o[63:0]p0_bam_read_op0_bam_readdata_i[511:0]p0_bam_readdatavalid_ip0_bam_write_op0_bam_writedata_o[511:0]p0_bam_burstcount_o[3:0]
p0_tl_cfg_func_o[2:0]p0_tl_cfg_add_o[4:0]p0_tl_cfg_ctl_o[15:0]
Configuration OutputInterface
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
37
Figure 17. P-Tile Avalon-MM IP for PCIe in Endpoint Mode Top-Level Block Diagram
P-Tile Avalon-MM IP for PCI Express
Clocksrefclk[1:0]
coreclkout_hip
p0_bas_waitrequest_o
p0_bas_address_i[63:0]p0_bas_readdata_o[511:0]
p0_bas_burstcount_i[3:0]
p0_bas_read_i
p0_bas_write_i
Bursting Avalon-MMSlave Interface
tx_n_out[15:0]rx_p_in[15:0] Serial Data
Hard IPReconfiguration(Optional)
p0_hip_reconfig_clkp0_hip_reconfig_address[20:0]
p0_hip_reconfig_readp0_hip_reconfig_readdata[7:0]
p0_hip_reconfig_writep0_hip_reconfig_writedata[7:0]
p0_hip_reconfig_waitrequest
p0_hip_reconfig_readdatavalid
p0_bam_pfnum_o[1:0]p0_bam_bar_o[2:0]
p0_bus_master_enable_o[1:0]
p0_bam_waitrequest_i
p0_pld_warm_rst_rdy_ip0_reset_status_n pin_perst_n ninit_done
Reset
xcvr_reconfig_clkxcvr_reconfig_address[25:0]
xcvr_reconfig_readxcvr_reconfig_readdata[7:0]xcvr_reconfig_readdatavalid
xcvr_reconfig_writexcvr_reconfig_writedata[7:0]
xcvr_reconfig_waitrequest
PHYReconfiguration(Optional)
Bursting Avalon-MMMaster Interface
p0_bas_byteenable_i[63:0]p0_bas_pfnum_i[1:0]
p0_bas_writedata_i[511:0]
p0_pld_link_req_rst_o
p0_app_clk
dummy_user_avmm_rst
tx_p_out[15:0]
rx_n_in[15:0]
p0_bam_response_i[1:0]p0_bam_address_o[BAM_ADDR_WIDTH-1:0]p0_bam_byteenable_o[63:0]p0_bam_read_op0_bam_readdata_i[511:0]p0_bam_readdatavalid_ip0_bam_write_op0_bam_writedata_o[511:0]p0_bam_burstcount_o[3:0]
p0_bas_readdatavalid_o
p0_bas_response_o[1:0]
p0_tl_cfg_func_o[2:0]p0_tl_cfg_add_o[4:0]p0_tl_cfg_ctl_o[15:0]
Configuration OutputInterface
4.3.1. Endpoint Mode Interface (512-bit Avalon-MM Interface)
Table 24. Avalon-MM Interface Summary
Avalon-MM Type Data Bus Width Max Burst Size Byte EnableGranularity
Max OutstandingRead Request
Bursting Slave 512 bits 8 cycles byte 64
Bursting Master 512 bits 8 cycles byte 32
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
38
Avalon-MM Type Data Bus Width Max Burst Size Byte EnableGranularity
Max OutstandingRead Request
Read Data MoverWrite Master
512 bits 8 cycles dword N/A
Write Data MoverRead Master
512 bits 8 cycles dword 128
Control RegisterAccess
32 bits 1 cycle byte 1
These interfaces are standard Avalon interfaces. For timing diagrams, refer to theAvalon Interface Specifications.
Note: The number of read requests issued by the Write Data Mover's Avalon-MM ReadMaster is controlled by the assertion of waitrequest by the connected slave(s). TheRead Master can handle 128 outstanding cycles of data. You cannot set this parameterin Platform Designer. The slave needs to correctly back-pressure the master once itcannot handle the incoming requests.
Note: The 512-bit Bursting Slave interface does not support transactions where byte enablesare set to 0.
Related Information
Avalon Interface Specifications
4.3.1.1. Bursting Avalon-MM Master and Conduit
The Bursting Avalon-MM Master module has one user-visible Avalon-MM Masterinterface.
You enable this interface by turning On the Enable Bursting Avalon-MM Masterinterface option in the Avalon-MM Settings tab of the IP Parameter Editor.
Table 25. Bursting Avalon-MM Master and Conduit
Signal Name Direction Description Platform DesignerInterface Name
bam_pfnum_o[1:0] O
Physical function number• PF0: bam_pfnum_o[1:0] = 2'b00• Others: bam_pfnum_o[1:0] =
Reserved
bam_bar_o[2:0] O
This bus contains the BAR address for aparticular TLP. This bus acts as anextension of the standard address bus.000: Memory BAR 0001: Memory BAR 1010: Memory BAR 2011: Memory BAR 3100: Memory BAR 4101: Memory BAR 5110: Reserved111: Expansion ROM BAR
bam_waitrequest_i IWhen asserted, indicates that the Avalon-MM slave is not ready to respond to arequest.
bam_master
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
39
Signal Name Direction Description Platform DesignerInterface Name
waitrequestAllowance = 8The master can still issue 8 transfers afterbam_waitrequest_i is asserted.
bam_address_o[BAM_ADDR_WIDTH-1:0]
O
The width of the Bursting Master’saddress bus is the maximum of thewidths of all the enabled BARs.For BARs narrower than the widest BAR,the address bus’ additional mostsignificant (MSB) bits are driven to 0.
bam_byteenable_o[63:0] O
Specify the valid bytes ofbam_writedata_o[511:0]. Each bitcorresponds to a byte inbam_writedata_o[511:0].For single-cycle read bursts and for allwrite bursts, all contiguous sets ofenabled bytes are supported.For multi-cycle read bursts, all bits ofbam_byteenable_o[63:0] areasserted, regardless of the First ByteEnable (BE) and Last BE fields of thecorresponding TLP.
bam_read_o O When asserted, indicates the master isrequesting a read transaction.
bam_readdata_i[511:0] I Read data bus
bam_readdatavalid_i I
Asserted by the slave to indicate that thebam_readdata_i[511:0] bus containsvalid data in response to a previous readrequest.
bam_write_o O When asserted, indicates the master isrequesting a write transaction.
bam_writedata_o[511:0] O Data signals for write transfers.
bam_burstcount_o[3:0] O The master uses these signals to indicatethe number of transfers in each burst.
bam_response_i[1:0] I
00 : OKAY - successful response for atransaction.01 : RESERVED10 : SLAVEERROR11 : DECODEERROR
4.3.1.1.1. Bursting Avalon-MM Master and Conduit in Non-Bursting Mode
In non-bursting mode, the Bursting Avalon-MM Master module has the same interfaceas in bursting mode, except for some limitations in the size of transactions asdescribed below:
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
40
• Burst count must be 1.
• The request size ranges from 1 to 16 dwords with the following limitations:
— The address and size combination must generate a TLP that fits in one 512-bitchunk of data. For example, if the address starts at dword 15 of a 512-bittransaction, only one dword of data transfer is allowed. If the address starts atdword 0, all data transfer sizes up to 16 dwords are possible. The same ruleapplies to read completions.
— Byte enables are supported for a transfer size of one dword. For largertransfer sizes, dword enables apply.
• If non-bursting mode is enabled, sending a TLP larger than 64 bytes targeting thisinterface causes the interface to misbehave. In this case, a reset is required toallow the interface to recover.
• One outstanding read at a time. Incoming RX read/write TLPs will be delayed whilea downstream outstanding read exists.
4.3.1.2. Bursting Avalon-MM Slave and Conduit
The Bursting Avalon-MM Slave module has one user-visible Avalon-MM slave interface.
You enable this interface by turning On the Enable Bursting Avalon-MM Slaveinterface option in the IP Parameter Editor.
For more details on these interface signals, refer to the Avalon InterfaceSpecifications.
Table 26. Bursting Avalon-MM Slave and Conduit
Signal Name Direction Description Platform DesignerInterface Name
bas_pfnum_i[1:0] I
Physical function number• PF0:
bas_pfnum_i[1:0] =2'b00
• Others:bas_pfnum_i[1:0] =Reserved
bas_waitrequest_o O
When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 0The master cannot issue anytransfer afterbas_waitrequest_o isasserted.
bas_master
bas_address_i[63:0] ISpecify the byte addressregardless of the data widthof the master.
bas_byteenable_i[63:0] I
Specify the valid bytes ofbas_writedata_i[511:0]. Each bit corresponds to abyte inbas_writedata_i[511:0].
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
41
Signal Name Direction Description Platform DesignerInterface Name
For single-cycle read burstsand for all write bursts, allcontiguous sets of enabledbytes are supported.For burst read transactions,thebas_byteenable_i[63:0]must be64'hFFFF_FFFF_FFFF_FFFF.
bas_read_i IWhen asserted, indicates themaster is requesting a readtransaction.
bas_readdata_o[511:0] OEnsure that disabled bytesdo not contain stale data ifSR-IOV is enabled.
bas_readdatavalid_o O
maximumPendingReadTransactions: 64The maximum number ofpending reads that theAvalon-MM slave can queueup is 64.
bas_response_o[1:0] O
These bits contain theresponse status for anytransaction happening onthe BAS interface:• 00: OKAY - Successful
response for atransaction.
• 01: RESERVED - Thisencoding is reserved.
• 10: SLAVEERROR - Errorfrom an endpoint slave.Indicates an unsuccessfultransaction.
• 11: DECODEERROR -Indicates an attemptedaccess to an undefinedlocation.
bas_write_i IWhen asserted, indicates themaster is requesting a writetransaction.
bas_writedata_i[511:0] I Data signals for writetransfers.
bas_burstcount_i[3:0] I
The master uses thesesignals to indicate thenumber of transfers in eachburst.
4.3.1.2.1. Bursting Avalon-MM Slave and Conduit in Non-Bursting Mode
The Bursting Avalon-MM Slave module supports bursting mode while operating in RootPort mode.
This module can also operate in non-bursting mode. In this mode, the moduleinterface is the same as in bursting mode except that it has limitations in the size oftransactions as described below:
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
42
• Burst count must be 1.
• The request size ranges from 1 to 16 dwords with the following limitations:
— The address and size combination must generate a TLP that fits in one 512Bchunk of data. For example, if the address starts at dword 15 of a 512Btransaction, only one dword of data transfer is allowed. If the address starts atdword 0, all data transfer sizes up to 16 dwords are possible. The same ruleapplies to read completions.
— Byte enables are supported for a transfer size of one dword. For largertransfer sizes, dword enables apply.
• One outstanding read at a time (back-pressures the Avalon-MM Master while theoutstanding read exists).
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
43
4.3.1.3. Data Movers
4.3.1.3.1. Read Data Mover
The Read Data Mover has four user-visible interfaces:
• One Avalon-MM Write Master with sideband signals to write data to the Avalondomain.
• Two Avalon-ST Sinks to receive descriptors. One acts as a queue for prioritydescriptors, and the other acts as a queue for normal descriptors.
• One Avalon-ST Source to report status.
Read Data Mover Avalon-MM Write Master and Conduit
This interface provides the Read data from the Host memory to the user application.The rddm_address_o value is set within the descriptor destination address.
Table 27. Read Data Mover Avalon-MM Write Master and Conduit
Signal Name Direction Description Platform DesignerInterface Name
rddm_pfnum_o[1:0] O
Physical function number.• PF0:
rddm_pfnum_o[1:0] =2'b00
• Others:rddm_pfnum_o[1:0] =Reserved
rddm_conduit
rddm_waitrequest_i I
When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 16The master can still issue 16transfers afterrddm_waitrequest_i isasserted.
rddm_master
rddm_write_o OWhen asserted, indicates themaster is requesting a writetransaction.
rddm_address_o[63:0] OSpecify the byte addressregardless of the data widthof the master.
rddm_burstcount_o[3:0] O
The master uses thesesignals to indicate thenumber of transfers in eachburst.
rddm_byteenable_o[63:0]
O
Specify the valid bytes ofrddm_writedata_o[511:0]. Each bit corresponds to abyte inrddm_writedata_o[511:0].
rddm_writedata_o[511:0]
O Data signals for writetransfers.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
44
Read Data Mover Avalon-ST Descriptor Sinks
The Read Data Mover has two Avalon-ST sink interfaces to receive the descriptors thatdefine the data transfers to be executed. One of the interfaces receives descriptors fornormal data transfers, while the other receives descriptors for high-priority datatransfers.
The descriptor format for the Read Data Mover is described in the section DescriptorFormat for Data Movers.
Note: The user application is responsible for performing the scheduling between priority andnormal queues. No arbitration is performed inside the Read Data Mover.
Table 28. Read Data Mover Avalon-ST Normal Descriptor Sink Interface
Signal Name Direction Description Platform DesignerInterface Name
rddm_desc_ready_o O When asserted, this readysignal indicates the normaldescriptor queue in the ReadData Mover is ready toaccept data. The readylatency of this interface is 3cycles.
rddm_desc
rddm_desc_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the normaldescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.
rddm_desc_data_i[173:0]
I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : single destination (3)
[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address
Table 29. Read Data Mover Avalon-ST Priority Descriptor Sink Interface
Signal Name Direction Description Platform DesignerInterface Name
rddm_prio_ready_o O When asserted, this readysignal indicates the prioritydescriptor queue in the ReadData Mover is ready to
rddm_prio
continued...
(3) When the single destination bit is set, the same destination address is used for all thetransfers. If the bit is not set, the address increments for each transfer.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
45
Signal Name Direction Description Platform DesignerInterface Name
accept data. The readylatency of this interface is 3cycles.
rddm_prio_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the prioritydescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.
rddm_prio_data_i[173:0]
I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : single destination[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address
The Read Data Mover internally supports two queues of descriptors. The priority queuehas absolute priority over the normal queue. Use it carefully to avoid starving thenormal queue.
If the Read Data Mover receives a descriptor on the priority interface while processinga descriptor from the normal queue, it switches to processing descriptors from thepriority queue as soon as it has completed the current descriptor. The Read DataMover resumes processing the descriptors from the normal queue once the priorityqueue is empty. Do not use the same descriptor ID simultaneously in the two queuesas there would be no way to distinguish them on the Status Avalon-ST sourceinterface.
The Read Data Mover handles one descriptor at a time. When a descriptor has beenprocessed (the memory command has been issued to the PCIe link), the Read DataMover will read the next descriptor from the priority or normal descriptor interface.
Note: There is no buffer to store descriptors inside the Read Data Mover. In Intel's DMAdesign example, the buffer is located in the external DMA controller and supports upto 128 descriptors.
Software should only send new descriptors when the Read Data Mover has processedall previously sent descriptors. The P-Tile Avalon-MM IP indicates the completion of theRead Data Mover's data processing by performing an immediate write to the systemmemory using its Write Data Mover. For more details, refer to the Read DMA Examplesection in the P-tile Avalon Memory Mapped (Avalon-MM) IP for PCI Express DesignExample User Guide (see the link in the Related Information below).
Related Information
Read DMA Example
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
46
Read Data Mover Status Avalon-ST Source
Table 30. Read Data Mover Status -ST Source
Signal Name Direction Description Platform DesignerInterface Name
rddm_tx_data_o[31:0] O [31:16]: reserved[15]: error[14:12]: application specific[11:9] : reserved[8] : priority[7:0]: descriptor ID
rddm_tx
rddm_tx_valid_o O Valid status signal
This interface does not have a ready input. The application logic must always be readyto receive status information for any descriptor that it has sent to the Read DataMover.
The Read Data Mover copies over the application specific bits in the rddm_tx_data_obus from the corresponding descriptor. A set priority bit indicates that the descriptor isfrom the priority descriptor sink.
A status word is output on this interface when the processing of a descriptor hascompleted, including the reception of all completions for all memory read requests.
4.3.1.3.2. Write Data Mover
The Write Data Mover has four user visible interfaces:
• One Avalon-MM Read Master with sideband signals to read data from the Avalondomain.
• Two Avalon-ST Sinks to receive descriptors. One Sink acts as a queue for prioritydescriptors, and the other acts as a queue for normal descriptors.
• One Avalon-ST Source to report status
Write Data Mover Avalon-MM Read Master and Conduit
This interface reads data from the Avalon-MM Read Master interface and writes it tothe Host memory.
The wrdm_address_o value is set within the descriptor source address.
Table 31. Write Data Mover Avalon-MM Read Master and Conduit
Signal Name Direction Description Platform DesignerInterface Name
wrdm_pfnum_o[1:0] O Physical function number.
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
47
Signal Name Direction Description Platform DesignerInterface Name
• PF0:wrdm_pfnum_o[1:0] =2'b00
• Others:wrdm_pfnum_o[1:0] =Reserved
wrdm_waitrequest_i I
When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 4The master can still issue 4transfers afterwrdm_waitrequest_i isasserted.
wrdm_master
wrdm_read_o OWhen asserted, indicates themaster is requesting a readtransaction.
wrdm_address_o[63:0] OSpecify the byte addressregardless of the data widthof the master.
wrdm_burstcount_o[3:0] O
The master uses thesesignals to indicate thenumber of transfers in eachburst.
wrdm_byteenable_o[63:0]
O
Specify the valid bytes ofwrdm_writedata_o[511:0]. Each bit corresponds to abyte inwrdm_writedata_o[511:0].
wrdm_readdatavalid_i I
Asserted by the slave toindicate that thewrdm_readdata_i[511:0]signals contain valid data inresponse to a previous readrequest.
wrdm_readdata_i[511:0] I Data signals for readtransfers.
wrdm_response_i[1:0] I
The response signals areoptional signals that carrythe response status.
Note:
Because the signalsare shared, aninterface cannotissue or accept awrite response and aread response in thesame clock cycle.
The following encodings areavailable:
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
48
Signal Name Direction Description Platform DesignerInterface Name
• 00: OKAY - Successfulresponse for atransaction.
• 01: RESERVED -Encoding is reserved.
• 10: SLAVEERROR - Errorfrom an endpoint slave.Indicates an unsuccessfultransaction.
• 11: DECODEERROR -Indicates an attemptedaccess to an undefinedlocation.
For read responses:• One response is sent
with each readdata. Aread burst length of Nresults in N responses. Itis not valid to producefewer responses, even inthe event of an error. Itis valid for the responsesignal values to bedifferent for eachreaddata in the burst.
• The interface must haveread control signals.Pipeline support ispossible with thereaddatavalid signal.
• On a read error, thecorresponding readdatais a "don't care".
Write Data Mover Avalon-ST Descriptor Sinks
The Write Data Mover has two Avalon-ST sink interfaces to receive the descriptors thatdefine the data transfers to be executed. One of the interfaces receives descriptors fornormal data transfers, while the other receives descriptors for high-priority datatransfers.
The descriptor format for the Write Data Mover is described in the section DescriptorFormats for Data Movers.
Note: The user application is responsible for performing the scheduling between priority andnormal queues. No arbitration is performed inside the Write Data Mover.
Table 32. Write Data Mover Avalon-ST Normal Descriptor Sink Interface
Signal Name Direction Description Platform DesignerInterface Name
wrdm_desc_ready_o O When asserted, this readysignal indicates the normaldescriptor queue in theWrite Data Mover is ready to
wrdm_desc
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
49
Signal Name Direction Description Platform DesignerInterface Name
accept data. The readylatency of this interface is 3cycles.
wrdm_desc_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the normaldescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.
wrdm_desc_data_i[173:0]
I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : reserved[147] : single source (4)
[146] : immediate (5)
[145:128]: number ofdwords to transfer up to 1MB[127:64]: destination PCIeaddress[63:0]: source Avalon-MMaddress / immediate data
Table 33. Write Data Mover Avalon-ST Priority Descriptor Sink Interface
Signal Name Direction Description Platform DesignerInterface Name
wrdm_prio_ready_o O When asserted, this readysignal indicates the prioritydescriptor queue in theWrite Data Mover is ready toaccept data. The readylatency of this interface is 3cycles. wrdm_prio
wrdm_prio_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the prioritydescriptor queue. On each
continued...
(4) When the single source bit is set, the same source address is used for all the transfers. If thebit is not set, the address increments for each transfer. Note that in single source mode, thePCIe address and Avalon-MM address must be 64-byte aligned.
(5) When set, the immediate bit indicates immediate writes. Immediate writes of one or twodwords are supported. For immediate transfers, bits [31:0] or [63:0] contain the payload forone- or two-dword transfers respectively. The two-dword immediate writes cannot cross a 4kboundary.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
50
Signal Name Direction Description Platform DesignerInterface Name
cycle where this signal isactive, the queue samplesthe data.
wrdm_prio_data_i[173:0]
I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : reserved[147] : single source[146] : immediate[145:128]: number ofdwords to transfer up to 1MB[127:64]: destination PCIeaddress[63:0]: source Avalon-MMaddress / immediate data
The Write Data Mover internally supports two queues of descriptors. The priorityqueue has absolute priority over the normal queue, so it should be used carefully toavoid starving the normal queue.
If the Write Data Mover receives a descriptor on the priority interface while processinga descriptor from the normal queue, it switches to processing descriptors from thepriority queue after it has completed processing the current descriptor. The Write DataMover resumes processing descriptors from the normal queue once the priority queueis empty. Do not use the same descriptor ID simultaneously in the two queues asthere would be no way to distinguish them on the Status Avalon-ST source interface.
The Write Data Mover handles one descriptor at a time. When a descriptor has beenprocessed, the Write Data Mover will read the next descriptor from the priority ornormal descriptor interface.
Note: There is no buffer to store descriptors inside the Write Data Mover. In Intel's DMAdesign example, the buffer is located in the external DMA controller and supports upto 128 descriptors.
Software should only send new descriptors when the Write Data Mover has processedall previously sent descriptors. The Write Data Mover indicates the completion of theits data processing by performing an immediate write to the system memory using thelast descriptor in the descriptor table. For more details, refer to the Write DMAExample section in the P-tile Avalon Memory Mapped (Avalon-MM) IP for PCI ExpressDesign Example User Guide (see the link in the Related Information below).
Related Information
Write DMA Example
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
51
Write Data Mover Status Avalon-ST Source
Table 34. Write Data Mover Status Avalon-ST Source
Signal Name Direction Description Platform DesignerInterface Name
wrdm_tx_data_o[31:0] O [31:16]: reserved[15]: error[14:12] : application specific[11:9] : reserved[8] : priority bit[7:0]: descriptor ID
wrdm_tx
wrdm_tx_valid_o O Valid status signal
This interface does not have a ready input. The application logic must always be readyto receive status information for any descriptor that it has sent to the Write DataMover.
The ready latency does not matter because there is no ready input.
The Write Data Mover copies over the application specific bits in the wrdm_tx_data_obus from the corresponding descriptor. A set priority bit indicates that the descriptorwas from the priority descriptor sink.
4.3.1.3.3. Descriptor Format for Data Movers
The Read and Write Data Movers uses descriptors to transfer data. The descriptorformat is fixed and specified below:
Table 35. Descriptor Format for Data Movers
Signals Description (forrddm_desc_data_i orwrdm_desc_data_i)
Read Data Mover Write Data Mover
[173:160]: reserved N/A N/A
[159:152]: descriptor ID ID of the descriptor ID of the descriptor
[151:149]: application-specific Application-specific bits.Example of an Intel application isprovided below.
Application-specific bits.Example of an Intel application isprovided below.
[148]: single destination When the single destination bit is set,the same destination address is usedfor all the transfers. If the bit is notset, the address increments for eachtransfer.
N/A
[147]: single source N/A When the single source bit is set, thesame source address is used for all thetransfers. If the bit is not set, theaddress increments for each transfer.Note that in single source mode, thePCIe address and Avalon-MM addressmust be 64-byte aligned.
[146]: immediate N/A When set, the immediate bit indicatesimmediate writes. Immediate writes ofone or two dwords are supported.
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
52
Signals Description (forrddm_desc_data_i orwrdm_desc_data_i)
Read Data Mover Write Data Mover
For immediate transfers, bits [31:0] or[63:0] contain the payload for one- ortwo-dword transfers respectively. Thetwo-dword immediate writes cannotcross a 4k boundary.This can be used for MSI/MSI-X forexample.
[145:128]: transfer size Number of dwords to transfer (up to 1MB).
Number of dwords to transfer (up to 1MB).
[127:64]: destination address Avalon-MM address PCIe Address
[63:0]: source address PCIe Address Avalon-MM address
Application-Specific Bits
Three application-specific bits (bits [151:149] ) from the Write Data Mover and ReadData Mover Status Avalon-ST Source interfaces control when interrupts are generated.
Table 36. Encodings for Application-Specific Bits
Bit [151] Bit [150] Bit [149] Action
0 1 1 Interrupt always
0 1 0 Interrupt if error
0 0 1 No interrupt
0 0 0 No interrupt and drop statusword
The External DMA Controller makes the decision whether to drop the status word andwhether to generate an interrupt as soon as it receives the status word from the DataMover. When the generation of an interrupt is requested, and the corresponding RI orWI register does enable interrupts, the DMA Controller generates the interrupt. It doesso by queuing an immediate write to the Write Data Mover's descriptor queue(specified in the corresponding interrupt control register) using the MSI address andmessage data provided in that register.
4.3.1.3.4. Avalon-MM DMA Operations
Avalon-MM DMA operations are used to transfer large blocks of data. The P-TileAvalon-MM IP for PCIe can support DMA operations with an external descriptorcontroller implemented in the user application.
To interface to the DMA logic included in the P-Tile Avalon-MM IP for PCIe, the customDMA descriptor controller must implement the following functions:
• It must provide the descriptors to the Read Data Mover and Write Data Mover inthe P-Tile IP.
• It must process the status that the DMA Avalon-MM Read and Write mastersprovide.
The following figure shows the Avalon-MM DMA Bridge when a custom externaldescriptor controller drives the Read and Write Data Movers.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
53
Figure 18. Avalon-MM DMA Bridge Block Diagram with Externally InstantiatedDescriptor Controller
Avalon-MM (bam_*) 512 Bits
Read Data MoverWrite Master
Read Data Mover Descriptor Sinks
Read Data MoverStatus Source
Read Data Mover
Avalon-ST (rddm_desc, rddm_prio)
174 Bits
Avalon-ST (rddm_tx)32 Bits
Avalon-MM (rddm_master)512 Bits
Write Data MoverDescriptor Sinks
Write Data MoverStatus Source
Write Data MoverRead Master
Write Data Mover
Avalon-ST (wrdm_desc, wrdm_prio)
174 Bits
Avalon-ST (wrdm_tx)32 Bits
Avalon-MM (wrdm_master)512 Bits
Non-Bursting Avalon-MM MastersMWr, Mrd, CpID
PCIeTX
PCIeRX
PCIeHard IP
X16
X16
P-Tile Avalon-MM IP for PCI Express IP Core
Intel FPGA
Avalon-MM DMA withexternal Descriptor
ControllerAvalon-MM DMA Bridge (Soft Logic)
Custom Descriptor Controller
(Implemented inFPGA Fabric)
MRd, CpID
MWr
Hard
IP In
terfa
ce
This configuration includes the PCIe Read DMA and Write DMA Data Movers. Thecustom DMA descriptor controller must connect to the following Data Mover interfaces:
• PCIe Read Descriptor Sinks: These are two 174-bit, Avalon-ST sink interfaces (fornormal and priority descriptors). The custom DMA descriptor controller drives readdescriptor table entries on this bus. For more details on this interface, refer to Read Data Mover Avalon-ST Descriptor Sinks on page 45.
• PCIe Write Descriptor Sinks: These are two 174-bit, Avalon-ST sink interfaces (fornormal and priority descriptors). The custom DMA descriptor controller driveswrite descriptor table entries on this bus. For more details on this interface, referto Write Data Mover Avalon-ST Descriptor Sinks on page 49.
• PCIe Read Data Mover Status Source: The Read Data Mover reports status to thecustom DMA descriptor controller on this interface. For more details on thisinterface, refer to Read Data Mover Status Avalon-ST Source on page 47.
• PCIe Write Data Mover Status Source: The Write Data Mover reports status to thecustom DMA descriptor controller on this interface. For more details on thisinterface, refer to Write Data Mover Status Avalon-ST Source on page 52.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
54
4.3.2. Root Port Mode Interface (256-bit Avalon-MM Interface)
In Gen3 x4 and Gen4 x4 Root Port modes, the IP core uses the 256-bit Avalon-MMbridge instead of the 512-bit Avalon-MM bridge for the performance purpose. In RootPort mode, DMA functionalities are not available. The table below shows the interfacesfor the P-tile 256-bit Avalon-MM bridge.
Table 37. Summary of Interfaces for the 256-bit Avalon-MM Bridge
Interface Name Data Width Burst Count Width Byte Enable Width Wait Request
Bursting Master 256 5 32 Yes
Non-Bursting Slave(optional)
32 N/A 4 Yes
Bursting Slave(optional)
256 5 32 Yes
Control RegisterAccess (CRA)
32 N/A 4 Yes
4.3.2.1. High Performance Avalon-MM Slave (HPTXS) Interface
The High Performance Avalon-MM Slave has a 256-bit-wide data bus. It supports up to16-cycle bursts with dword granularity byte enable on the first and last cycles of awrite burst and for single-cycle read bursts. It also supports optional address mappingwhen the address bus is less than 64-bit wide.
This interface is optional. You enable it by turning On the Enable Bursting Slaveoption in the GUI.
Table 38. High Performance Avalon-MM Slave (HPTXS) Interface
Signal Name Direction Description Platform DesignerInterface Name
hptxs_address_i[hptxs_address_width_hwtcl-1:0]
I Byte address. Bits [4:0] areassumed to be zeros.
hptxs_slave
hptxs_byteenable_i[31:0]
I Specifies the valid bytes fora write command.
hptxs_read_i I When asserted, specifies aTX Avalon-MM slave readrequest.
hptxs_readdata_o[255:0]
O This bus contains the readcompletion data.
hptxs_write_i I When asserted, specifies aTX Avalon-MM slave writerequest.
hptxs_writedata_i[255:0]
I This bus contains theAvalon-MM data for a writecommand.
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
55
Signal Name Direction Description Platform DesignerInterface Name
hptxs_waitrequest_o O When asserted, indicatesthat the Avalon-MM slaveport is not ready to respondto a read or write request.
hptxs_readdatavalid_o O When asserted, indicatesthat the read data is valid.
hptxs_burstcount_i[4:0]
I When asserted, the value onthe response signal is a validwrite response.Writeresponsevalid isonly asserted one clock cycleor more after the writecommand is accepted.There is at least a one clockcycle latency from commandacceptance to the assertionof writeresponsevalid.
4.3.2.2. High Performance Avalon-MM Master (HPRXM) Interface
The bursting Avalon-MM master is always enabled in Root Port mode and is notassociated with any BAR. Packets targeting addresses outside of the range of the baseand limit registers are forwarded to the host via the HPRXM master. The burstingAvalon-MM master has a 256-bit-wide data bus and supports up to 16-cycle burstswith dword granularity byte enable on the first and last cycles of a write burst and onsingle-cycle read bursts. Byte granularity access is supported for single-cycle one-dword or smaller transactions.
Table 39. High Performance Avalon-MM Master (HPRXM) Interface
Signal Name Direction Description Platform DesignerInterface Name
rxm_write_o O Asserted by the core torequest a write to anAvalon-MM slave.
hprxm_master
rxm_address_o[avmm_addr_width_hwtcl-1:0]
O The address of the Avalon-MM slave being accessed.
rxm_writedata_o[255:0] O This bus contains the RXdata being written to theslave.
rxm_byteenable_o[31:0] O These bits specify the validbytes for the write data.
rxm_burstcount_o[4:0] O The burst count, measuredin qwords, of the RX write orread request. The maximumamount of data in a burst is512 bytes.
rxm_waitrequest_i I When asserted by theexternal Avalon-MM slave,this signal indicates that theslave is not ready for thenext read or write request.
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
56
Signal Name Direction Description Platform DesignerInterface Name
rxm_read_o O Asserted by the core torequest a read.
rxm_readdata_i[255:0] I Read data returned from theAvalon-MM slave in responseto a read request. This datais sent to the IP corethrough the TX interface.
rxm_readdatavalid_i I Asserted by the systeminterconnect fabric toindicate that the read data isvalid.
4.3.2.3. 32-Bit Control Register Access (CRA) Slave (Root Port only)
The CRA interface provides access to the control and status registers of the Avalon-MMbridge. This interface has the following properties:
• 32-bit data bus
• Supports a single transaction at a time
• Supports single-cycle transactions (no bursting)
Note: When the Avalon-MM Hard IP for PCIe IP Core is in Root Port mode, and theapplication logic issues a CfgWr or CfgRd via the CRA interface, it needs to fill the Tagfield in the TLP Header with the value 0x10 to ensure that the correspondingCompletion gets routed to the CRA interface correctly. If the application logic sets theTag field to some other value, the Avalon-MM Hard IP for PCIe IP Core does notoverwrite that value with the correct value.
Table 40. Avalon-MM CRA Slave Interface
Signal Name Direction Description Platform DesignerInterface Name
cra_read_i I Read enable.
cra
cra_write_i I Write request.
cra_address_i[14:0] I
cra_writedata_i[31:0] I Write data. The currentversion of the CRA slaveinterface is read-only.Including this signal as apart of the Avalon-MMinterface makes futureenhancements possible.
cra_readdata_o[31:0] O Read data.
cra_byteenable_i[3:0] I Byte enable.
cra_waitrequest_o O Wait request to hold offadditional requests.
cra_chipselect_i I Chip select signal to thisslave.
cra_irq_o O Interrupt request. A portrequest for an Avalon-MMinterrupt.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
57
4.4. Serial Data Interface
The P-Tile Avalon-MM IP for PCIe natively supports 4, 8, or 16 PCIe lanes. Each laneincludes a TX differential pair and an RX differential pair. Data is striped across allavailable lanes.
Table 41. Serial Data Interface
Signal Name Direction Description
tx_p_out[<b>-1:0],tx_n_out[<b>-1:0]
O Transmit serial data outputs using theHigh Speed Differential I/O standard.
rx_p_in[<b>-1:0],rx_n_in[<b>-1:0]
I Receive serial data inputs using theHigh Speed Differential I/O standard.
Note: The value of the variable b depends on which configuration is active (1x16, 2x8 or4x4).
• For 1x16, b = 16.
• For 2x8, b = 8.
• For 4x4, b = 4.
4.5. Hard IP Status Interface
This interface includes the signals that are useful for debugging, such as the linkstatus signal, LTSSM state outputs, etc. These signals are available when the optionalPower Management interface is enabled.
Table 42. Hard IP Status Interface
Signal Name Direction Description Clock Domain EP/RP
link_up_o O When asserted, this signal indicates thelink is up.
coreclkout_hip
EP/RP
dl_up_o O When asserted, this signal indicates theData Link (DL) Layer is active.
coreclkout_hip
EP/RP
ltssm_state_o[5:0]
O Indicates the LTSSM state:• 6'h00: S_DETECT_QUIET• 6'h01: S_DETECT_ACT• 6'h02: S_POLL_ACTIVE• 6'h03: S_POLL_COMPLIANCE• 6'h04: S_POLL_CONFIG• 6'h05: S_PRE_DETECT_QUIET• 6'h06: S_DETECT_WAIT• 6'h07: S_CFG_LINKWD_START• 6'h08: S_CFG_LINKWD_ACCEPT• 6'h09: S_CFG_LANENUM_WAIT• 6'h0A: S_CFG_LANENUM_ACCEPT• 6'h0B: S_CFG_COMPLETE• 6'h0C: S_CFG_IDLE• 6'h0D: S_RCVRY_LOCK• 6'h0E: S_RCVRY_SPEED• 6'h0F: S_RCVRY_RCVRCFG
coreclkout_hip
EP/RP
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
58
Signal Name Direction Description Clock Domain EP/RP
• 6'h10: S_RCVRY_IDLE• 6'h11: S_L0• 6'h12: S_L0S• 6'h13: S_L123_SEND_EIDLE• 6'h14: S_L1_IDLE• 6'h15: S_L2_IDLE• 6'h16: S_L2_WAKE• 6'h17: S_DISABLED_ENTRY• 6'h18: S_DISABLED_IDLE• 6'h19: S_DISABLED• 6'h1A: S_LPBK_ENTRY• 6'h1B: S_LPBK_ACTIVE• 6'h1C: S_LPBK_EXIT• 6'h1D: S_LPBK_EXIT_TIMEOUT• 6'h1E: S_HOT_RESET_ENTRY• 6'h1F: S_HOT_RESET• 6'h20: S_RCVRY_EQ0• 6'h21: S_RCVRY_EQ1• 6'h22: S_RCVRY_EQ2• 6'h23: S_RCVRY_EQ3
surprise_down_err_o
O When active, indicates that a surpriselink down event is occurring.
coreclkout_hip
RP
4.6. Interrupt Interface
The P-Tile Avalon-MM IP for PCI Express supports Message Signaled Interrupts (MSI),MSI-X interrupts, and legacy interrupts. MSI and legacy interrupts are mutuallyexclusive.
Legacy interrupts, MSI, and MSI-X interrupts are all controlled and generatedexternally to the Avalon-MM IP to ensure total flexibility of allocating interruptresources based on the user’s application needs.
To support domain-isolation, legacy interrupt messages, MSI, and MSI-X TLPs need tobe sent with the appropriate source IDs.
The following figure shows an example integrating an external interrupt controller withthe P-Tile Avalon-MM IP. The interrupt controller takes interrupt requests from theexternal DMA controller as well as those from the user application.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
59
Figure 19. Example of an Interrupt Controller Integrated with an Endpoint P-TileAvalon-MM IP for PCI Express IP Core
SM
S
BurstingSlave
P-Tile Avalon-MM IP for PCIe
User’s Interrupt Controller
M
tl_cfg_ctl_o[15:0]
intx_req_i[3:0]
BAR, PF & VF Numbers
PF & VF Numbers
Avalon-MM master for MSI/MSI-X
generation
Interrupt to VFm, nMapping
tl_cfg_add_o[4:0]tl_cfg_func_o[2:0]
BurstingMaster
Interrupt to MSI/MSI-XMapping
MSI-X TableDMA ControllerInterrupts
Interrupts fromUser’s Application
Interrupt Status Registers
4.6.1. Legacy Interrupts
If legacy interrupts are enabled at IP configuration time, the user’s interrupt controllergenerates legacy interrupts by asserting the intx_req_i input signal which causesthe PCIe Hard IP to send the corresponding interrupt message. Use of legacyinterrupts to signal the completion of DMA transfers is not recommended as theirordering with respect to the DMA traffic is not guaranteed.
4.6.2. MSI
If MSI or MSI-X are enabled at IP configuration time, the external interrupt controllercan generate MSI/MSI-X transactions by issuing memory writes to the Bursting Slaveor using the immediate write feature of the Write Data Mover, especially if signalingthe completion of a DMA transfer by the Write Data Mover. The interrupt controllergets the address and data information to generate the MSI/MSI-X messages from theMSI or MSI-X capability registers in the Transaction Layer in the P-Tile IP.
MSI interrupts are signaled on the PCI Express link using a single dword Memory WriteTLP. The user application issues an MSI request (MWr) through the Avalon-ST interfaceand updates the configuration space register using the MSI interface.
For more details on the MSI Capability Structure, refer to Figure 54 on page 114.
The Mask Bits register and Pending Bits register are 32 bits in length each, with eachpotential interrupt message having its own mask bit and pending bit. If bit[0] of theMask Bits register is set, interrupt message 0 is masked. When an interrupt messageis masked, the MSI for that vector cannot be sent. If software clears the mask bit andthe corresponding pending bit is set, the function must send the MSI request at thattime.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
60
You should obtain the necessary MSI information (such as the message address anddata) from the configuration output interface (tl_cfg_*) to create the MWr TLP inthe format shown below to be sent via the Avalon-ST interface.
Figure 20. Creating a MWr TLP for an MSI Request
MSI Message Data
Message Control Next CapabilityPointer
MSI Capability Structure
MSI (Memory Write) Transaction
Capability ID(05h) DW0
DW1
DW2
DW3Message Data
Byte 16
Byte 12
Byte 8
Byte 4
Byte 0
MSI Message Address [31:0]
Message Address [63:32]
Message Address [31:0]
1631 15 8 7 0
MSI Message Address [63:32]
Tag
+3+2+1+0
Last DW
Length
First DW1 1 1 10 0 0 0
0 0 0 0 0 0 0 0 0 10 0 0 0 00 1 1
7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
Requester ID
Fmt Type TC TH
TD
EP ATAt
trAttr00
0000h Data
Header
0 0
R RR
Table 43. MSI Pending Bits Interface
Signal Name Direction Description Clock Domain EP/RP
msi_pnd_func_i[2:0] IFunction number select for thePending Bits register in the MSIcapability structure.
coreclkout_hip
EP
msi_pnd_addr_i[1:0] I
Byte select for Pending BitsRegister in the MSI CapabilityStructure. For example ifmsi_pnd_addr_i[1:0] = 00,bits [7:0] of the Pending Bitsregister will be updated withmsi_pnd_byte_i[7:0]. Ifmsi_pnd_addr_i[1:0] = 01,bits [15:8] of the Pending Bitsregister will be updated withmsi_pnd_byte_i[7:0].
coreclkout_hip
EP
msi_pnd_byte_i[7:0] I Indicate that function has apending associated message.
coreclkout_hip
EP
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
61
The following figure shows the timings of msi_pnd_* signals in three scenarios. Thefirst scenario shows the case when the MSI pending bits register is not used. Thesecond scenario shows the case when only physical function 0 is enabled and the MSIpending bits register is used. The last scenario shows the case when four physicalfunctions are enabled and the MSI pending bits register is used.
Figure 21. Example Timing Diagrams for msi_pnd* Signals
0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3
B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3
0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1
0x10x0
0x2 0x3
0x0 0x1
B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3
0x0
0x0
0x0
0x0
msi_pnd_byte_i[7:0]
msi_pnd_func_i[2:0]
coreclkout_hip
coreclkout_hip
msi_pnd_addr_i[1:0]
msi_pnd_byte_i[7:0]
msi_pnd_func_i[2:0]
msi_pnd_addr_i[1:0]
coreclkout_hip
msi_pnd_byte_i[7:0]
msi_pnd_func_i[2:0]
msi_pnd_addr_i[1:0]
There are 32 possible MSI messages. The number of messages requested by aparticular component does not necessarily correspond to the number of messagesallocated. For example, in the following figure, the Endpoint requests eight MSIs but isonly allocated two. In this case, you must design the Application Layer to use only twoallocated messages.
Figure 22. MSI Request Example
Endpoint
8 Requested2 Allocated
Root Complex
CPU
Interrupt Register
RootPort
InterruptBlock
The following table describes three example implementations. The first exampleallocates all 32 MSI messages. The second and third examples only allocate 4interrupts.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
62
Table 44. MSI Messages Requested, Allocated, and Mapped
MSI Allocated
32 4 4
System Error 31 3 3
Hot Plug and Power Management Event 30 2 3
Application Layer 29:0 1:0 2:0
MSI interrupts generated for Hot Plug, Power Management Events, and System Errorsalways use Traffic Class 0. MSI interrupts generated by the Application Layer can useany Traffic Class. For example, a DMA that generates an MSI at the end of atransmission can use the same traffic control as was used to transfer data.
The following figure illustrates a possible implementation of the Interrupt HandlerModule with a per vector enable bit in the Application Layer. Alternatively, theApplication Layer could implement a global interrupt enable instead of this per vectorMSI.
Figure 23. Example Implementation of the Interrupt Handler Block
Interrupt Enable 0
Interrupt Request 0
msi_req0
Interrupt Enable 1
Interrupt Request 1
msi_req1
app_int_i
Arbitration & TLP Generator
MSI info (from tl_cfg_ctl* /tl_cfg_addr* / tl_cfg_func*) & Master Enable
Avalon-MMsingle-dword MWR TLPs
msi_pnd_*
Vector 1
Vector 0
IRQGeneration
App Layer
IRQGeneration
App Layer
R/W
R/W
4.6.3. MSI-X
The P-Tile Avalon-MM IP provides a Configuration Intercept Interface. User soft logiccan monitor this interface to get MSI-X Enable and MSI-X function mask relatedinformation. User application logic needs to implement the MSI-X tables for all PFs andVFs at the memory space pointed to by the BARs as a part of your Application Layer.
For more details on the MSI-X related information that you can obtain from theConfiguration Intercept Interface, refer to the MSI-X Registers section in the Registerschapter.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
63
MSI-X is an optional feature that allows the user application to support large amountof vectors with independent message data and address for each vector.
When MSI-X is supported, you need to specify the size and the location (BARs andoffsets) of the MSI-X table and PBA. MSI-X can support up to 2048 vectors perfunction versus 32 vectors per function for MSI.
A function is allowed to send MSI-X messages when MSI-X is enabled and the functionis not masked. The application uses the Configuration Output Interface (address 0x0Cbit[5:4]) or Configuration Intercept Interface to access this information.
When the application needs to generate an MSI-X, it will use the contents of the MSI-XTable (Address and Data) and generate a Memory Write through the Avalon-STinterface.
You can enable MSI-X interrupts by turning on the Enable MSI-X option under thePCI Express/PCI Capabilities tab in the parameter editor. If you turn on theEnable MSI-X option, you should implement the MSI-X table structures at thememory space pointed to by the BARs as a part of your Application Layer.
The MSI-X Capability Structure contains information about the MSI-X Table and PBAStructure. For example, it contains pointers to the bases of the MSI-X Table and PBAStructure, expressed as offsets from the addresses in the function's BARs. TheMessage Control register within the MSI-X Capability Structure also contains the MSI-XEnable bit, the Function Mask bit, and the size of the MSI-X Table. For a picture of theMSI-X Capability Structure, refer to Figure 56 on page 115.
MSI-X interrupts are standard Memory Writes, therefore Memory Write ordering rulesapply.
Example:
Table 45. MSI-X Configuration
MSI-X Vector MSI-X Upper Address MSI-X Lower Address MSI-X Data
0 0x00000001 0xAAAA0000 0x00000001
1 0x00000001 0xBBBB0000 0x00000002
2 0x00000001 0xCCCC0000 0x00000003
Table 46. PBA Table
PBA Table PBA Entries
Offset 0 0x0
If the application needs to generate an MSI-X interrupt (vector 1), it will read the MSI-X Table information, generate a MWR TLP through the Avalon-ST interface and assertthe corresponding PBA bits (bit[1]) in a similar fashion as for MSI generation.
The generated TLP will be sent to address 0x00000001_BBBB0000 and the data willbe 0x00000002. When the MSI-X has been sent, the application can clear theassociated PBA bits.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
64
4.6.3.1. Implementing MSI-X Interrupts
Section 6.8.2 of the PCI Local Bus Specification describes the MSI-X capability andtable structures. The MSI-X capability structure points to the MSI-X Table structureand MSI-X Pending Bit Array (PBA) registers. The BIOS sets up the starting addressoffsets and BAR associated with the pointer to the starting address of the MSI-X Tableand PBA registers.
1. Host software sets up the MSI-X interrupts in the Application Layer by completingthe following steps:
a. Host software reads the Message Control register at 0x050 register todetermine the MSI-X Table size. The number of table entries is the <valueread> + 1.
The maximum table size is 2048 entries. Each 16-byte entry is divided in 4fields as shown in the figure below. For multi-function variants, BAR4 accessesthe MSI-X table. For all other variants, any BAR can access the MSI-X table.The base address of the MSI-X table must be aligned to a 4 KB boundary.
b. The host sets up the MSI-X table. It programs MSI-X address, data, andmasks bits for each entry as shown in the figure below.
Figure 24. Format of MSI-X Table
Vector ControlVector ControlVector Control
Vector Control
Message DataMessage DataMessage Data
Message Data
DWORD 3 DWORD 2Message Upper AddressMessage Upper AddressMessage Upper Address
Message Upper Address
DWORD 1Message AddressMessage AddressMessage Address
Message Address
DWORD 0 Host Byte AddressesEntry 0Entry 1Entry 2
Entry (N - 1)
BaseBase + 1 × 16Base + 2 × 16
Base + (N - 1) × 16
c. The host calculates the address of the <nth> entry using the followingformula:
nth_address = base address[BAR] + 16<n>
2. When Application Layer has an interrupt, it drives an interrupt request to the IRQSource module.
3. The IRQ Processor reads the entry in the MSI-X table.
a. If the interrupt is masked by the Vector_Control field of the MSI-X table,the interrupt remains in the pending state.
b. If the interrupt is not masked, IRQ Processor sends Memory Write Request tothe TX slave interface. It uses the address and data from the MSI-X table. IfMessage Upper Address = 0, the IRQ Processor creates a three-dwordheader. If the Message Upper Address > 0, it creates a 4-dword header.
4. The host interrupt service routine detects the TLP as an interrupt and services it.
Related Information
• Floor and ceiling functions
• PCI Local Bus Specification, Rev. 3.0
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
65
4.7. Hot Plug Interface (RP Only)
Note: The Hot Plug interface is not supported in the 20.2 release of Intel Quartus Prime. Itmay be supported in a future release.
Hot Plug support means that the device can be added to or removed from a systemduring runtime. The Hot Plug Interface in the P-Tile Avalon-MM IP for PCIe allows anIntel FPGA with this IP to safely provide this capability.
This section describes the signals reported by the on-board hot plug components inthe Downstream Port. This interface is available only if the Slot Status Registerof the PCI Express Capability Structure is enabled.
Refer to the Slot Status Register of the PCI Express CapabilityStructure for additional information.
Table 47. Hot Plug Interface
Signal Name Direction Description Clock Domain EP/RP
sys_atten_button_pressed_i I Attention ButtonPressed. Indicatesthat the systemattention button waspressed, and setsthe Attention ButtonPressed bit in theSlot StatusRegister.
coreclkout_hip RP
sys_pwr_fault_det_i I Power FaultDetected. Indicatesthe power controllerdetected a powerfault at this slot.
coreclkout_hip RP
sys_mrl_sensor_chged_i I MRL SensorChanged. Indicatesthat the state of theMRL sensor haschanged.
coreclkout_hip RP
sys_pre_det_chged_i I Presence DetectChanged. Indicatesthat the state of thecard presencedetector haschanged.
coreclkout_hip RP
sys_cmd_cpled_int_i I CommandCompleted Interrupt.Indicates that theHot Plug controllercompleted acommand.
coreclkout_hip RP
sys_pre_det_state_i I Indicates whether ornot a card is presentin the slot.0 : slot is empty.1 : card is present inthe slot.
coreclkout_hip RP
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
66
Signal Name Direction Description Clock Domain EP/RP
sys_mrl_sensor_state_i I MRL Sensor State.Indicates the stateof the manuallyoperated retentionlatch (MRL) sensor.0 : MRL is closed.1 : MRL is open.
coreclkout_hip RP
sys_eml_interlock_engaged_i
I Indicates whetherthe systemelectromechanicalinterlock is engaged,and controls thestate of theelectromechanicalinterlock status bit inthe Slot StatusRegister.
coreclkout_hip RP
sys_aux_pwr_det_i I Auxiliary PowerDetected. Used toreport to the hostsoftware thatauxiliary power(Vaux) is present.Refer to the DeviceStatus Registerin the PCI ExpressCapabilityStructure.
coreclkout_hip RP
4.8. Power Management Interface
Note: The Power Management interface is not available in the 20.2 release of Intel QuartusPrime. However, it may be available in a future release.
Software programs the device into a D-state by writing to the Power ManagementControl and Status register in the PCI Power Management CapabilityStructure. The power management output signals indicate the current power state.The IP core supports the two mandatory power states: D0 (full power) and D3(preparation for a loss of power). It does not support the optional D1 and D2 low-power states.
The correspondence between the device power states (D states) and link power states(L states) is as follows:
Table 48. Relationship Between Device and Link Power States
Device Power State Link Power State
D0 L0
D1 (not supported) L1
D2 (not supported) L1
D3 L1, L2/L3 Ready
P-Tile does not support ASPM.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
67
Table 49. Power Management Interface
Signal Name Direction Description Clock Domain EP/RP
pm_state_o[2:0] O Indicates the current powerstate. coreclkout_hip EP/RP
x16/x8:pm_dstate_o[31:0]
x4:pm_dstate_o[3:0]
O
Power management D-statefor each function.• 0001b : D0• 0010b : D1• 0100b : D2• 1000b : D3• 0000b : uninitialized or
invalid
Async EP/RP
x16/x8:apps_pm_xmt_pme_i[7:0]
x4: NA
I
The application logic assertsthis signal for one cycle towake up the PowerManagement Capability (PMC)state machine from a D1, D2,or D3 power state. Uponwake-up, the IP core sends aPM_PME message.
coreclkout_hip EP
x16/x8:app_ready_entr_l23_i
x4: NA
I
The application logic assertsthis signal to indicate that it isready to enter the L2/L3Ready state. Theapp_ready_entr_l23_isignal is provided forapplications that must controlthe L2/L3 Ready entry (in casecertain tasks must beperformed before going intoL2/L3 Ready). The core delayssending PM_Enter_L23 (inresponse to PM_Turn_Off) untilthis signal becomes active.This is a level-sensitive signal.
coreclkout_hip EP
x16:app_req_retry_en_i[7:0]
x8:app_req_retry_en_i
x4: NA
I
When these signals areasserted, the P-Tile Avalon-MMIP will respond toConfiguration TLPs with aConfiguration Retry Status(CRS) if it is not ready torespond with non-CRS statussince the last reset.For x4 ports, this signal is notused and needs to be driven tozero.
Async EP
4.9. Configuration Output Interface
The Transaction Layer configuration output (tl_cfg) bus provides a subset of theinformation stored in the Configuration Space. Use this information in conjunction withthe app_err* signals to understand TLP transmission problems.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
68
Table 50. Configuration Output Interface
Signal Name Direction Description Clock Domain EP/RP
tl_cfg_ctl_o[15:0]
O
Multiplexed data output from theregister specified bytl_cfg_add_o[4:0]. Thedetailed information for each fieldin this bus is defined in thefollowing table.
coreclkout_hip EP/RP
tl_cfg_add_o[4:0] O
This address bus contains theindex indicating whichConfiguration Space registerinformation is being driven ontothe tl_cfg_ctl_o[15:0] bits.
coreclkout_hip EP/RP
x16/x8:tl_cfg_func_o[2:0]
x4: NA
O
Specifies the function whoseConfiguration Space register valuesare being driven out ontl_cfg_ctl_o[15:0].• 3'b000: Physical Function 0
(PF0)• 3'b001: PF1and so on
Note:
In the 19.4 release of IntelQuartus Prime, the P-TileAvalon-MM IP only supportsPF0.
coreclkout_hip EP/RP
The table below provides the tl_cfg_add_o[4:0] to tl_cfg_ctl_o[15:0]mapping.
Table 51. Multiplexed Configuration Information Available on tl_cfg_ctl
tl_cfg_add_o[4:0] tl_cfg_ctl_o[15:8] tl_cfg_ctl_o[7:0]
5'h00
[15]: memory space enable[14]: IDO completion enable[13]: perr_en[12]: serr_en[11]: fatal_err_rpt_en[10]: nonfatal_err_rpt_en[9]: corr_err_rpt_en[8]: unsupported_req_rpt_en
Device control:[7]: bus master enable[6]: extended tag enable[5:3]: maximum read request size[2:0]: maximum payload size
5'h01
[15]: IDO request enable[14]: No Snoop enable[13]: Relaxed Ordering enable[12:8]: Device number
bus number
5'h02
[15]: pm_no_soft_rst[14]: RCB control[13]: Interrupt Request (IRQ) disable[12:8]: PCIe Capability IRQ messagenumber
[7:5]: reserved[4]: system power control[3:2]: system attention indicatorcontrol[1:0]: system power indicator control
5'h03 Number of VFs [15:0]
5'h04[15]: reserved[14]: AtomicOP Egress Block field(cfg_atomic_egress_block)
[7]: ARI forward enable[6]: Atomic request enable[5:3]: TPH ST mode[2:1]: TPH enable
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
69
tl_cfg_add_o[4:0] tl_cfg_ctl_o[15:8] tl_cfg_ctl_o[7:0]
[13:9]: ATS Smallest Translation Unit(STU)[4:0][8]: ATS cache enable
[0]: VF enable
5'h05
[15:12]: auto negotiation link speed.Link speed encoding values are:• Gen1 : 0x1• Gen2 : 0x2• Gen3 : 0x4• Gen4 : 0x8[11:1]: Index of Start VF [10:0][0]: reserved
5'h06 MSI Address [15:0]
5'h07 MSI Address [31:16]
5'h08 MSI Address [47:32]
5'h09 MSI Address [63:48]
5'h0A MSI Mask [15:0]
5'h0B MSI Mask [31:16]
5'h0C
[15]: cfg_send_f_err[14]: cfg_send_nf_err[13]: cfg_send_cor_err[12:8]: AER IRQ message number
[7]: Enable extended message data forMSI (cfg_msi_ext_data_en)[6]: MSI-X func mask[5]: MSI-X enable[4:2]: Multiple MSI enable[1]: 64-bit MSI[0]: MSI enable
5'h0D MSI Data [15:0]
5'h0E AER uncorrectable error mask [15:0]
5'h0F AER uncorrectable error mask [31:16]
5'h10 AER correctable error mask [15:0]
5'h11 AER correctable error mask [31:16]
5'h12 AER uncorrectable error severity [15:0]
5'h13 AER uncorrectable error severity [31:16]
5'h14 [15:8]: ACS Egress Control Register(cfg_acs_egress_ctrl_vec)
[7]: ACS function group enable(cfg_acs_func_grp_en)[6]: ACS direct translated P2P enable(cfg_acs_p2p_direct_tranl_en)[5]: ACS P2P egress control enable(cfg_acs_egress_ctrl_en)[4]: ACS upstream forwarding enable(cfg_acs_up_forward_en)[3]: ACS P2P completion redirectenable(cfg_acs_p2p_compl_redirect_en)[2]: ACS P2P request redirect enable(cfg_acs_p2p_req_redirect_en)[1]: ACS translation blocking enable(cfg_acs_at_blocking_en)
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
70
tl_cfg_add_o[4:0] tl_cfg_ctl_o[15:8] tl_cfg_ctl_o[7:0]
[0]: ACS source validation enable (RP)(cfg_acs_validation_en)
5'h15
[15]: reserved[14]: 10-bit tag requester enable(cfg_10b_tag_req_en)[13]: VF 10-bit tag requester enable(cfg_vf_10b_tag_req_en)[12]: PRS_RESP_FAILURE(cfg_prs_response_failure)[11]: PRS_UPRGI (cfg_prs_uprgi)[10]: PRS_STOPPED(cfg_prs_stopped)[9]: PRS_RESET (cfg_prs_reset)[8]: PRS_ENABLE (cfg_prs_enable)
[7:3]: reserved[2:0]: ARI function group(cfg_ari_func_grp)
5'h16PRS_OUTSTANDING_ALLOCATION(cfg_prs_outstanding_allocation) [15:0]
5'h17PRS_OUTSTANDING_ALLOCATION(cfg_prs_outstanding_allocation) [31:16]
5'h18
[15:10]: reserved[9]: Disable autonomous generation ofLTR clear message(cfg_disable_ltr_clr_msg)[8]: LTR mechanism enable(cfg_ltr_m_en)
[7]: Infinite credits for Posted header[6]: Infinite credits for Posted data[5]: Infinite credits for Completionheader[4]: Infinite credits for Completion data[3]: End-end TLP prefix blocking(cfg_end2end_tlp_pfx_blck)[2]: PASID enable(cfg_pf_pasid_en)[1]: Execute permission enable(cfg_pf_passid_execute_perm_en)[0]: Privileged mode enable(cfg_pf_passid_priv_mode_en)
5'h19
[15:9]: reserved[8]: Slot control attention buttonpressed enable(cfg_atten_button_pressed_en)
[7]: Slot control power fault detectenable (cfg_pwr_fault_det_en)[6]: Slot control MRL sensor changedenable (cfg_mrl_sensor_chged_en)[5]: Slot control presence detectchanged enable(cfg_pre_det_chged_en)[4]: Slot control hot plug interruptenable (cfg_hp_int_en)[3]: Slot control command completedinterrupt enable(cfg_cmd_cpled_int_en)[2]: Slot control DLL state changeenable (cfg_dll_state_change_en)[1]: Slot control accessed(cfg_hp_slot_ctrl_access)[0]: PF’s SERR# enable(cfg_br_ctrl_serren)
5'h1A LTR maximum snoop latency register(cfg_ltr_max_latency[15:0])
continued...
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
71
tl_cfg_add_o[4:0] tl_cfg_ctl_o[15:8] tl_cfg_ctl_o[7:0]
5'h1BLTR maximum no-snoop latencyregister(cfg_ltr_max_latency[31:16])
5'h1C [15:8]: enabled Traffic Classes (TCs)(cfg_tc_enable[7:0])
[5:0]: auto negotiation link width6’h01 = x16’h02 = x26’h04 = x46’h08 = x86’h10 = x16
5'h1D MSI Data[31:16]
5'h1E N/A
5'h1F N/A
Note: The information on the Configuration Output (tl_cfg) bus is time-divisionmultiplexed (TDM).
• When tl_cfg_func[2:0] = 3'b000, tl_cfg_ctl[31:0] drive out the PF0Configuration Space register values.
• Then, tl_cfg_func[2:0] are incremented to 3'b001.
• When tl_cfg_func[2:0] = 3'b001, tl_cfg_ctl[31:0] drive out the PF1Configuration Space register values.
• This pattern repeats to cover all enabled PFs.
• The P-Tile Avalon-MM IP for PCIe only supports PF0.
Figure 25. Configuration Output Interface Timing Diagram
coreclkout_hip
tl_cfg_add_0[4:0]
tl_cfg_ctl_0[15:0]
tl_cfg_func_0[2:0]
0x00 0x01 0x02 0x00 0x01 0x020x03
PF0 DATA0 PF0 DATA1 PF0 DATA2 PF1 DATA0 PF1 DATA1 PF1 DATA2PF0 DATA3
0 1
4.10. Hard IP Reconfiguration Interface
The Hard IP reconfiguration interface is an Avalon-MM slave interface with a 21-bitaddress and an 8-bit data bus. It is also sometimes referred to as the User Avalon-MMInterface. You can use this interface to dynamically modify the value of configurationregisters. Note that after a warm reset or cold reset, changes made to theconfiguration registers of the Hard IP via the Hard IP reconfiguration interface are lostas these registers revert back to their default values.
Note: This interface can be used in Endpoint and Root Port modes. It must be enabled ifRoot Port mode is selected.
In Root Port mode, the application logic uses the Hard IP reconfiguration interface toaccess its PCIe configuration space to perform link control functions (such as HotReset, link disable, or link retrain).
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
72
Table 52. Hard IP Reconfiguration Interface
Signal Name Direction Description Clock Domain EP/RP
hip_reconfig_clk I
Reconfigurationclock50 MHz - 125 MHz(Range)100 MHz(Recommended)
EP/RP
hip_reconfig_readdata_o[7:0] O Avalon-MM readdata outputs
hip_reconfig_clk
EP/RP
hip_reconfig_readdatavalid_o O
Avalon-MM readdata valid. Whenasserted, the dataonhip_reconfig_readdata_o[7:0] isvalid.
hip_reconfig_clk
EP/RP
hip_reconfig_write_i I Avalon-MM writeenable
hip_reconfig_clk
EP/RP
hip_reconfig_read_i I Avalon-MM readenable
hip_reconfig_clk
EP/RP
hip_reconfig_address_i[20:0] I Avalon-MM address hip_reconfig_clk
EP/RP
hip_reconfig_writedata_i[7:0] I Avalon-MM writedata inputs
hip_reconfig_clk
EP/RP
hip_reconfig_waitrequest_o O
When asserted, thissignal indicates thatthe IP core is notready to respond toa request.
hip_reconfig_clk
EP/RP
dummy_user_avmm_rst I
Reset signal. Youcan tie it to groundor leave it floatingwhen using the HardIP ReconfigurationInterface.
EP/RP
Reading and Writing to the Hard IP Reconfiguration Interface
Reading from the Hard IP reconfiguration interface of the P-Tile Avalon-MM IP for PCIExpress retrieves the current value at a specific address. Writing to the reconfigurationinterface changes the data value at a specific address. Intel recommends that youperform read-modify-writes when writing to a register, because two or more featuresmay share the same reconfiguration address.
Modifying the PCIe configuration registers directly affects the behavior of the PCIedevice.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
73
Figure 26. Timing Diagram to Perform Read and Write Operations Using the Hard IPReconfiguration Interface
4.10.1. Address Map for the User Avalon-MM Interface
The User Avalon-MM interface provides access to the configuration registers and the IPcore registers. This interface includes an 8-bit data bus and a 21-bit address bus(which contains the byte addresses).
There are two methods to access the configuration registers:
• Using direct User Avalon-MM interface (byte access)
• Using the Debug (DBI) register access (dword access). This method is useful whenyou need to read/write the entire 32 bits at one time (Counter/ Lane Margining,etc.)
The following diagram and table show the address offsets for physical function 0(PF0), User Avalon-MM Port Configuration Register and Debug (DBI) Register.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
74
Figure 27. Address Map for the User Avalon-MM Interface
User Avalon-MM Port Configuration Register
PF0 PCie Configuration Registers 0x000000
0x001000
0x104068
0x1fffff
Debug_DBI_Data/Debug_DBI_Addr0x104200
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
75
Table 53. Configuration Space Offsets
Registers User Avalon-MM Offsets Comments
Physical function 0 0x0000 Refer to Appendix A for more details ofthe PF configuration space. This PF isavailable for x16, x8 and x4 cores.
User Avalon-MM Port ConfigurationRegister
0x104068 Refer to User Avalon-MM PortConfiguration Register (Offset0x104068) for more details.
Debug (DBI) Register 0x104200 to 0x104204 Refer to Using the Debug RegisterInterface Access on page 76 for moredetails.
4.10.2. Configuration Registers Access
4.10.2.1. Using Direct User Avalon-MM Interface (Byte Access)
Targeting PF Configuration Space Registers
User application needs to specify the offsets of the targeted PF registers.
For example, if the application wants to read the MSI Capability Register of PF0, it willissue a Read with address 0x0050 to target the MSI Capability Structure of PF0.
Figure 28. PF Configuration Space Registers Access Timing DiagramHIP reconfig clk
HIP reconfig addr 0x000050
HIP reconfig read
HIP reconfig readdata PF0 MSI Cap
HIP reconfig readdatavalid
Targeting VSEC Registers
User application needs to program the VSEC field (0x104068 bit[0]) first. Then allaccesses from the user Avalon-MM interface starting at offset 0xD00 will be translatedto VSEC configuration space registers.
Figure 29. VSEC Registers Access Timing Diagram
HIP reconfig clk
HIP reconfig addr 0x104068 0xD00
HIP reconfig write
HIP reconfig writedata 0x01
HIP reconfig read
HIP reconfig waitrequest
HIP reconfig readdata VSEC Cap
HIP reconfig readdatavalid
4.10.2.2. Using the Debug Register Interface Access
DEBUG_DBI_ADDR register is located at user Avalon-MM offsets 0x104204 to0x104207 (corresponding to byte 0 to byte 3). For example, the d_done bit is bit 7 atbyte address 0x104207.
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
76
Table 54. DEBUG_DBI_ADDR Register
Names Bits R/W Descriptions
d_done 31 RO 1: indicates debug DBI read/write access done
d_write 30 R/W 1: write access0: read access
d_warm_reset 29 RO 1: normal operation0: warm reset is on-going
d_vf 28:18 R/W Specify the virtual functionnumber.
d_vf_select 17 R/W To access the virtual functionregisters, set this bit to one.
d_pf 16:14 R/W Specify the physical functionnumber.
reserved 13:12 R/W Reserved
d_addr 11:2 R/W Specify the DW address forthe P-Tile Avalon-MM IP DBIinterface.
d_shadow_select 1 R/W Reserved. Clear this bit foraccess to standard PCIeconfiguration registers.
d_vsec_select 0 R/W If set, this bit allows accessto Intel VSEC registers.
DEBUG_DBI_DATA register is located at user Avalon-MM offsets 0x104200 to0x104203 (corresponding to byte 0 to byte 3).
Table 55. DEBUG_DBI_DATA Register
Names Bits R/W Descriptions
d_data 31:0 R/W Read or write data for the P-Tile Avalon-MM IP registeraccess.
To write all 32 bits in a Debug register at a time:
1. Use the user_avmm interface to access 0x104200 to 0x104203 to write the datafirst.
2. Use the user_avmm interface to access 0x104204 to 0x104206 to set the addressand control bits.
3. Use the user_avmm interface to write to 0x104207 to enable the read/write bit(bit[30]).
4. Use the user_avmm interface to access 0x104207 bit[31] to poll if the write iscomplete.
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
77
Figure 30. DBI Register Write Timing DiagramHIP reconfig clk
HIP reconfig addr 0x104200 0x104201 0x104202 0x104203 0x104204 0x104205 0x104206 0x104207
HIP reconfig write
HIP reconfig writedata 0x01 0x23 0x45 0x67 ADDR CTRL 0x4
HIP reconfig waitrequest
HIP reconfig read
HIP reconfig readdata 0x4 0xC
HIP reconfig readdatavalid
To read all 32 bits in a Debug register at a time:
1. Use the user_avmm interface to access 0x104204 to 0x104206 to set the addressand control bits.
2. Use the user_avmm interface to write to 0x104207 to enable the read bit(bit[30]).
3. Use the user_avmm interface to access 0x104207 bit[31] to poll if the read iscomplete.
4. Use the user_avmm interface to access 0x104200 to 0x104203 to read the data
Figure 31. DBI Register Read Timing DiagramHIP reconfig clk
HIP reconfig addr 0x104204 0x104205 0x104206 0x104207 0x104207 0x104200 0x104201 0x104202 0x104203
HIP reconfig write
HIP reconfig writedata ADDR CTRL 0x0
HIP reconfig waitrequest
HIP reconfig read
HIP reconfig readdata 0x0 0x8 D0 D1 D2 D3
HIP reconfig readdatavalid
4.11. PHY Reconfiguration Interface
The PHY reconfiguration interface is an optional Avalon-MM slave interface with a26-bit address and an 8-bit data bus. Use this bus to read the value of PHY registers.Refer to Table 60 on page 87 for details on addresses and bit mappings for the PHYregisters that you can access using this interface.
These signals are present when you turn on Enable PHY reconfiguration on theTop-Level Settings tab using the parameter editor.
Please note that the PHY reconfiguration interface is shared among all the PMA quads.
Table 56. PHY Reconfiguration Interface
Signal Name Direction Description Clock Domain EP/RP
xcvr_reconfig_clk IReconfiguration clock50 MHz - 125 MHz (Range)100 MHz (Recommended)
EP/RP
xcvr_reconfig_readdata[7:0]
O Avalon-MM read dataoutputs
xcvr_reconfig_clk
EP/RP
xcvr_reconfig_readdatavalid
O
Avalon-MM read data valid.When asserted, the data onxcvr_reconfig_readdata[7:0] is valid.
xcvr_reconfig_clk
EP/RP
xcvr_reconfig_write I Avalon-MM write enable xcvr_reconfig_clk
EP/RP
continued...
4. Interfaces
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
78
Signal Name Direction Description Clock Domain EP/RP
xcvr_reconfig_read I
Avalon-MM read enable.This interface is notpipelined. You must wait forthe return of thexcvr_reconfig_readdata[7:0] from the currentread before starting anotherread operation.
xcvr_reconfig_clk
EP/RP
xcvr_reconfig_address[25:0]
I
Avalon-MM address[25:21] are used to indicatethe Quad.5'b00001 : Quad 05'b00010 : Quad 15'b00100 : Quad 25'b01000 : Quad 3[20:0] are used to indicatethe offset address.
xcvr_reconfig_clk
EP/RP
xcvr_reconfig_writedata[7:0]
I Avalon-MM write data inputs xcvr_reconfig_clk
EP/RP
xcvr_reconfig_waitrequest O
When asserted, this signalindicates that the PHY is notready to respond to arequest.
xcvr_reconfig_clk
EP/RP
Reading from the PHY Reconfiguration Interface
Reading from the PHY reconfiguration interface of the P-Tile Avalon-MM IP for PCIExpress retrieves the current value at a specific address.
Figure 32. Timing Diagram to Perform Read Operations Using the PHY ReconfigurationInterface
0x000006
0x01
xcvr_reconfig_clk
xcvr_reconfig_address
xcvr_reconfig_read
xcvr_reconfig_readdatavalid
xcvr_reconfig_readdata
xcvr_reconfig_waitrequest
4. Interfaces
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
79
5. Advanced Features
5.1. PCIe Port Bifurcation and PHY Channel Mapping
Note: Port bifurcation for Gen3 x8, Gen4 x8 and Gen4 x4 will be available in a future releaseof Intel Quartus Prime.
The PCIe controller IP contains a set of port bifurcation muxes to remap the fourcontroller PIPE lane interfaces to the shared 16 PCIe PHY lanes. The table below showsthe relationship between PHY lanes and the port mapping.
Table 57. Port Bifurcation and PHY Channel Mapping
Bifurcation Mode Port 0 (x16) Port 1 (x8) Port 2 (x4) Port 3 (x4)
1 x16 0 - 15 NA NA NA
2 x8 0 - 7 8 - 15 NA NA
4 x4 4 - 7 8 - 11 0 - 3 12 - 15
Note: For more details on the bifurcation modes, refer to the Architecture section in chapter2.
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
6. Troubleshooting/DebuggingAs you bring up your PCI Express system, you may face issues related to FPGAconfiguration, link training, BIOS enumeration, data transfer, and so on. This chaptersuggests some strategies to resolve the common issues that occur during bring-up.
You can additionally use the P-Tile Debug Toolkit to identify the issues.
6.1. Hardware
Typically, PCI Express link-up involves the following steps:
1. Link training
2. BIOS enumeration and data transfer
The following sections describe the flow to debug link issues during the hardwarebring-up. Intel recommends a systematic approach to diagnosing issues as illustratedin the following figure.
Additionally, you can use the P-Tile Debug Toolkit for debugging the PCIe links whenusing the P-Tile Avalon-MM IP for PCI Express. The P-Tile Debug Toolkit includes thefollowing features:
• Protocol and link status information.
• Basic and advanced debugging capabilities including PMA register access and Eyeviewing capability.
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
Figure 33. PCI Express Debug Flow Chart
No
No
Yes
Yes
Go to “8.1.2 Debugging data transfer and
performance issues”
Go to “8.1.1 Debugging Link training issues”
System Reset
Start
End
Is the data transfer
successful?
Is the link training
successful?
6.1.1. Debugging Link Training Issues
The Physical Layer automatically performs link training and initialization withoutsoftware intervention. This is a well-defined process to configure and initialize thedevice's Physical Layer and link so that PCIe packets can be transmitted.
Some examples of link training issues include:
• Link fails to negotiate to expected link speed.
• Link fails to negotiate to the expected link width.
• LTSSM fails to reach/stay stable at L0.
Flow Chart for Debugging Link Training Issues
Use the flow chart below to identify the potential cause of the issue seen during linktraining when using the P-Tile Avalon-MM IP for PCI Express.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
82
Figure 34. Link Training Debugging Flow
Observation: ltssm_state_o signal toggles between Detect.Quiet and Detect.Active. Check the Receiver detection status from the registers for successful receiver detection
Issue: Far end receiver not detected by the FPGA TX
Resolution: Check coupling capacitance, far end termination resistance and TX OCT values are in accordance to the spec
Observation: ltssm_state_o signal transitions from Detect.Quiet –> Detect.Active –> Polling.Active –> Polling.Compliance states.
Issue: Far end device failing receiver detection
Resolution: Check far end coupling capacitance, near end termination resistance and TX OCT valuesare in accordance to the spec
Observation: ltssm_state_o signal stuck at Detect.Quiet state
Issue: IP is in reset state
Resolution: Check if the pin_perst_n reset signal is in reset
No
NoNo
Yes
Yes
YesIs the linkout of reset?
Does the linktrain to L0 at
reduced speed?
Is therereceiver detected
at the far end?
A.Observation: Loop of Detect.Quiet –> Detect.Active –> Polling.Active –> Recovery.rcvrlock transitions observed on ltssm_state_o signal
Issue: Poor refclk quality
Resolution: Check the reference clock quality is good. (e.g. Jitter, phase noise, etc). Ensure that the clocks used are in accordance with the guidelines described in the User Guide
B.Observation: Timeout during EQ Phases on few lanes when monitoring ltssm_state_o signal
Issue: Signal Integrity issues/Sub optimal EQ settings on few lanes
Resolution: Redo the Equalization (*)
Yes Yes
No
No
No
Yes
A.Observation: Frequent transitions on ltssm_state_o signal between L0 and Recovery.rcvrlock states
Issue: Signal Integrity issues (or) sub optimal EQ settings
Resolution: Redo the Equalization (*)
A.Observation: Wrong lane numbers encoded in TS1/TS2 (Observed using Protocol Analyzer)
Issue: Improper lane reversal
Resolution: Check the lane routing
B.Observation: Timeout during EQ Phases on few lanes when monitoring ltssm_state_o signal
Issue: Signal Integrity issues/Sub optimal EQ settings on few lanes
Resolution: Redo the Equalization (*)
Begin
OR
OR
Does the LTSSM
enter L0?
Does link go to L0 at
advertised speed with frequent recoveries?
Does the link train to
L0 with reduced lane width?
End
Note: (*) Redo the equalization using the Link Equalization Request 8.0 GT/s bitof the Link Status 2 register for 8.0 GT/s or Link Equalization Request 16.0 GT/sbit of the 16.0 GT/s Status Register.
Use the following debug tools for debugging link training issues observed on the PCIExpress link when using the P-tile Avalon-MM IP for PCI Express.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
83
6.1.1.1. Generic Tools and Utilities
You can use utilities like lspci, setpci to obtain general information of the device likelink speed, link width etc.
Example: To read the negotiated link speed for the P-Tile device in a system, you canuse the following commands:
sudo lspci –s $bdf -vvv
-s refers to “slot” and is used with the bus/device/function number (bdf) information.Use this command if you know the bdf of the device in the system topology.
sudo lspci –d <1172>:$did -vvv
-d refers to device and is used with the device ID (vid:did). Use this command tosearch using the device ID.
Figure 35. lspci Output
The LnkCap under Capabilities indicates the advertised link speed and widthcapabilities of the device. The LnkSta under Capabilities indicates the negotiatedlink speed and width of the device.
6.1.1.2. SignalTapII Logic Analyzer
Using the SignalTapII Logic Analyzer, you can monitor the following top-level signalsfrom the P-Tile Avalon-MM IP for PCI Express to confirm the failure symptom for anyport type (Root port, Endpoint or TLP Bypass) and configuration (Gen4/Gen3).
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
84
Table 58. Top-Level Signals to be Monitored for Debugging
Signals Description Expected Value for Successful Link-up
pin_perst_n Active-low asynchronous input signalto the PCIe Hard IP.Implements the PERST# functiondefined by the PCIe specification.
1'b1
p0_reset_status_n Active-low output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Held low until pin_perst_n isdeasserted and the PCIe Hard IPcomes out of reset, synchronous tocoreclkout_hip.When port bifurcation is used, there isone such signal for each Avalon-MMinterface.
1'b1
ninit_done Active-low output signal from the PCIeHard IP. High indicates that the FPGAdevice is not yet fully configured, andlow indicates the device has beenconfigured and is in normal operatingmode.
1'b0
link_up_o Active-high output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Indicates that the Physical Layer link isup.
1'b1
dl_up_o Active-high output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Indicates that the Data Link Layer isactive.
1'b1
ltssm_state_o[5:0] Indicates the LTSSM state,synchronous to coreclkout_hip.
6'h11 (L0)
6.1.1.3. Additional Debug Tools
Use the Hard IP reconfiguration interface and PHY reconfiguration interface on the P-Tile Avalon-MM IP for PCI Express to access additional registers (for example, receiverdetection, lane reversal etc.).
Figure 36. Register Access for Debug
PCIe Controllers
Port N
Registers
MACPMA
Quad Nx16PCIePCS
Hard IP ReconfigInterface
PHY ReconfigInterface
PLLAPCIe
x16 Lanes
PHY
PLLBDLL TL
PHYRegisters
Using the Hard IP Reconfiguration Interface
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
85
Refer to the section Hard IP Reconfiguration Interface for details on this interface andthe associated address map.
The following table lists the address offsets and bit settings for the PHY statusregisters. Use the Hard IP Reconfiguration Interface to access these read-onlyregisters.
Table 59. Hard IP Reconfiguration Interface Register Map for PHY Status
Offset Bit Position Register
0x0003E9 [0] RX polarity
[1] RX detection
[2] RX Valid
[3] RX Electrical Idle
[4] TX Electrical Idle
0x0003EC [7] Framing error
0x0003ED [7] Lane reversal
Follow the steps below to access registers in Table 59 on page 86 using the Hard IPreconfiguration interface:
1. Enable the Hard IP reconfiguration interface (User Avalon-MM interface) using theIP Parameter Editor.
2. Set the lane number for which you want to read the status by performing a read-modify-write to the address hip_reconfig_addr_i[20:0] with write data oflane number on hip_reconfig_writedata_i[7:0] using the Hard IPreconfiguration interface signals.
• hip_reconfig_write_i = 1’b1
• hip_reconfig_addr_i[20:0] = 0x0003E8
• hip_reconfig_writedata_i[3:0] = <Lane number>, where Lane number= 4’h0 for lane 0, 4’h1 for lane 1, 4’h2 for lane 2, …
3. Read the status of the register you want by performing a read operation from theaddress hip_reconfig_addr_i[20:0] using the Hard IP reconfigurationinterface signals.
• hip_reconfig_read_i = 1’b1
• hip_reconfig_addr_i[20:0] = <offset>
Offset = Refer to Table 59 on page 86 for the offset mapping.
• hip_reconfig_readdata_o[7:0] = Refer to Table 59 on page 86 for thebit position mapping.
Example 1: To read the RX detection status of Lane0 using the registers
1. Enable the Hard IP reconfiguration interface using the IP Parameter Editor.
2. Perform read-modify-write to address 0x0003E8 to set the lane number to 0 usingthe Hard IP reconfiguration interface signals.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
86
• hip_reconfig_write_i = 1’b1
• hip_reconfig_addr_i[20:0] = 0x0003E8
• hip_reconfig_writedata_i[3:0] = 4'h0
3. Read the status of the RX detection register by performing a read operation fromthe address 0x0003E9[1] using the Hard IP reconfiguration interface signals.
• hip_reconfig_read_i = 1’b1
• hip_reconfig_addr_i[20:0] = 0x0003E9
• hip_reconfig_readdata_o[1] = 1'b1 (Far end receiver detected)
Using the PHY Reconfiguration Interface
Refer to the section PHY Reconfiguration Interface for details on how to use thisinterface.
Follow the steps below to access registers in Table 60 on page 87 using the PHYreconfiguration interface.
1. Enable the PHY reconfiguration interface using the IP Parameter Editor.
2. Set the Quad and address offset from which you want to read the status byperforming a read operation from the address xcvr_reconfig_addr_i[25:0]using the PHY reconfiguration interface signals.
• xcvr_reconfig_read_i = 1’b1
• xcvr_reconfig_addr_i[25:0] = {5-bit Quad mapping, 21-bit addressoffset}. Refer to Table 60 on page 87 for the address offset and bit mapping.
• xcvr_reconfig_readdata_o[7:0] = Refer to Table 60 on page 87 for theaddress offset and bit mapping.
Table 60. PHY Reconfiguration Interface Register Map for PHY Status
PHY Offset Bit Position Register
0x000006 [7] PLLA state output status signal.1'b1 indicates that PLLA is locked.
0x00000a [7] PLLB state output status signal.1'b1 indicates that PLLB is locked.
Example 2: To read the PLLA status using the registers
1. Enable the PHY reconfiguration interface using the IP Parameter Editor.
2. Perform a read from address 0x000006 to read the PLLA status output of Quad0using the PHY reconfiguration interface signals.
• xcvr_reconfig_read_i = 1'b1
• xcvr_reconfig_addr_i[25:0] = 0x000006
• xcvr_reconfig_readdata_o[7:0] = 8'h80
• xcvr_reconfig_readdata_i = 1'b1 (PLLA state output high indicating PLLlock)
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
87
6.1.2. Debugging Data Transfer and Performance Issues
There are many possible reasons causing the PCIe link to stop transmitting data. ThePCI Express base specification defines three types of errors, outlined in the tablebelow:
Table 61. Error Types Defined by the PCI Express Base Specification
Type Responsible Agent Description
Correctable Hardware While correctable errors may affectsystem performance, data integrity ismaintained.
Uncorrectable, non-fatal Device software Uncorrectable, non-fatal errors aredefined as errors in which data is lost,but system integrity is maintained. Forexample, the fabric may lose aparticular TLP, but it still works withoutproblems.
Uncorrectable, fatal System software Errors generated by a loss of data andsystem failure are considereduncorrectable and fatal. Software mustdetermine how to handle such errors:whether to reset the link or implementother means to minimize the problem.
Table 62. Correctable Error Status Register (AER)
Observation Issue Resolution
Receiver error bit set Physical layer error which may be dueto a PCS error when a lane is in L0, ora Control symbol being received in thewrong lane, or signal Integrity issueswhere the link may transition from L0to the Recovery state.
Use the Hard IP reconfigurationinterface and the flow chart in Figure34 on page 83 to obtain moreinformation about the error.
Bad DLLP bit set Data link layer error which may occurwhen a CRC verification fails.
Use the Hard IP reconfigurationinterface to obtain more informationabout the error.
Bad TLP bit set Data link layer error which may occurwhen an LCRC verification fails or whena sequence number error occurs.
Use the Hard IP reconfigurationinterface to obtain more informationabout the error.
Replay_num_rollover bit set Data link layer error which may be dueto TLPs sent without success (no ACK)four times in a row.
Use the Hard IP reconfigurationinterface to obtain more informationabout the error.
replay timer timeout status bit set Data link layer error which may occurwhen no ACK or NAK was receivedwithin the timeout period for the TLPstransmitted.
Use the Hard IP reconfigurationinterface to obtain more informationabout the error.
Advisory non-fatal Transaction layer error which may bedue to higher priority uncorrectableerror detected.
Corrected internal error bits set Transaction layer error which may bedue to an ECC error in the internalHard IP RAM.
Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information about the error.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
88
Table 63. Uncorrectable Error Status Register (AER)
Observation Issue Resolution
Data link protocol error Data link layer error which may be dueto transmitter receiving an ACK/NAKwhose Seq ID does not correspond toan unacknowledged TLP or ACKsequence number.
Use the Hard IP reconfigurationinterface to obtain more informationabout the error.
Surprise down error Data link layer error which may be dueto link_up_o getting deassertedduring L0, indicating the physical layerlink is going down unexpectedly.
Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information about the error.
Flow control protocol error Transaction layer error which can bedue to the receiver reporting morethan the allowed credit limit.This error occurs when a componentdoes not receive updated flow controlcredits with the 200 μs limit.
Use the TX/RX flow control interface,Hard IP reconfiguration interface toobtain more information about theerror.
Poisoned TLP received Transaction layer error which can bedue to a received TLP with the EP bitset.
Use the Hard IP reconfigurationinterface to obtain more information onthe error and determine theappropriate action.
Completion timeout Transaction layer error which can bedue to a completion not received withinthe required amount of time after anon-posted request was sent.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Completer abort Transaction layer error which can bedue to a completer being unable tofulfill a request due to a problem withthe requester or a failure of thecompleter.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Unexpected completion Transaction layer error which can bedue to a requester receiving acompletion that doesn’t match anyrequest awaiting a completion.The TLP is deleted by the Hard IP andnot presented to the Application Layer.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Receiver overflow Transaction layer error which can bedue to a receiver receiving more TLPsthan the available receive buffer space.The TLP is deleted by the Hard IP andnot presented to the Application Layer.
Use the TX/RX flow control interfaceand Hard IP reconfiguration interfaceto obtain more information on theerror.
Malformed TLP Transaction layer error which can bedue to errors in the received TLPheader.The TLP is deleted by the Hard IP andnot presented to the Application Layer.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
ECRC error Transaction layer error which can bedue to an ECRC check failure at thereceiver despite the fact that the TLP isnot malformed and the LCRC check isvalid.The Hard IP block handles this TLPautomatically. If the TLP is a non-posted request, the Hard IP blockgenerates a completion with acompleter abort status. The TLP isdeleted by the Hard IP and notpresented to the Application Layer.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
continued...
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
89
Observation Issue Resolution
Unsupported request Transaction layer error which can bedue to the completer being unable tofulfill the request.The TLP is deleted in the Hard IP blockand not presented to the ApplicationLayer. If the TLP is a non-postedrequest, the Hard IP block generates acompletion with Unsupported Requeststatus.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
ACS violation Transaction layer error which can bedue to access control error in thereceived posted or non-posted request.
Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Uncorrectable internal error Transaction layer error which can bedue to an internal error that cannot becorrected by the hardware.
Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information on the error.
Atomic egress blocked Use the Hard IP reconfigurationinterface to obtain more information onthe error.
TLP prefix blocked EP or RP only Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Poisoned TLP egress blocked EP or RP only Use the Hard IP reconfigurationinterface to obtain more information onthe error.
Use the debug tools mentioned in the next two sections for debugging link trainingissues observed on the PCI Express link when using the P-Tile Avalon-MM IP for PCIExpress.
6.1.2.1. Advanced Error Reporting (AER)
Each PCI Express compliant device must implement a basic level of error managementand can optionally implement advanced error management. The PCI ExpressAdvanced Error Reporting Capability is an optional Extended Capability that may beimplemented by PCI Express device functions supporting advanced error control andreporting.
The P-Tile Avalon-MM IP for PCI Express implements both basic and advanced errorreporting. Error handling for a Root Port is more complex than that of an Endpoint. Inthis P-Tile Avalon-MM IP for PCI Express, the AER capability is enabled by default.
Use the AER capability of the PCIe Hard IP to identify the type of error and theprotocol stack layer in which the error may have occurred. Refer to the PCI ExpressCapability Structures section of the Configuration Space Registers appendix for theAER Extended Capability Structure and the associated registers.
6.1.2.2. Second-Level Debug Tools
Use the following debug tools for second-level debug of any issue observed on the PCIExpress link when using P-Tile:
Using the Hard IP Reconfiguration Interface
• Refer to the section Hard IP Reconfiguration Interface on page 72 for details onthis interface and the address map.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
90
Using the PHY Reconfiguration Interface
• Refer to the section PHY Reconfiguration Interface on page 78 for details on thisinterface and the address map.
6.2. Debug Toolkit
6.2.1. Overview
The P-Tile Debug Toolkit is a System Console-based tool for P-Tile that provides real-time control, monitoring and debugging of the PCIe links at the Physical, Data Linkand Transaction layers.
The P-Tile Debug Toolkit allows you to:
• View protocol and link status of the PCIe links per port.
• View PLL and per-channel status of the PCIe links per port.
• Control the channel analog settings.
• View the receiver eye and measure the eye height and width.
• Indicate the presence of a re-timer connected between the link partners.
The following figure provides an overview of the P-Tile Debug Toolkit in the P-TileAvalon-MM IP for PCI Express.
Figure 37. Overview of the P-Tile Debug Toolkit
P-Tile Debug Toolkit
NPDME
hip-reconfig_*
intel_pcie_ptile_avmm
AVMMPCle Config Space Registers
(Port 0)
AVMMPCle Config Space Registers
(Port 3)
AVMM PHY Registers (Quad 0)
AVMM PHY Registers (Quad 3)
...
...
System Console
GUI
xcvr_reconfig_*
When you enable the P-Tile Debug Toolkit, the intel_pcie_ptile_avmm module ofthe generated IP includes the Debug Toolkit modules and related logic as shown in thefigure above.
Drive the Debug Toolkit from a System Console. The System Console connects to theDebug Toolkit via an Native PHY Debug Master Endpoint (NPDME). Make thisconnection via an Intel FPGA Download Cable.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
91
This PHY reconfiguration interface clock (xcvr_reconfig_clk) is used to clock thefollowing interfaces:
• The NPDME module
• PHY reconfiguration interface (xcvr_reconfig)
• Hard IP reconfiguration interface (hip_reconfig)
Provide a clock source (50 MHz - 125 MHz, 100 MHz recommended clock frequency) todrive the xcvr_reconfig_clk clock. Use the output of the Reset Release Intel FPGAIP to drive the ninit_done, which provides the reset signal to the NPDME module.
Note: When using the port bifurcation feature, always connect the xcvr_reconfig_clk ofPort0 to a clock source. This signal is used to provide the clock to the Debug Toolkit.
Note: When you enable the P-Tile Debug Toolkit, the Hard IP reconfiguration interface isenabled by default.
When you run a dynamically-generated design example on the Intel Development Kit,make sure that clock and reset signals are connected to their respective sources andappropriate pin assignments are made. Here are some sample .qsf assignments forthe Debug Toolkit for Intel Stratix 10 DX devices:
• set_location_assignment PIN_A31 -to p0_hip_reconfig_clk_clk
• set_location_assignment PIN_C23 -to xcvr_reconfig_clk_clk
• #set_instance_assignment -name VIRTUAL_PIN ON -to *p0_hip_reconfig*
• #set_instance_assignment -name VIRTUAL_PIN ON -to *xcvr_reconfig*
6.2.2. Enabling the P-Tile Debug Toolkit
To enable the P-Tile Debug Toolkit in your design, enable the option Enable DebugToolkit in the PCIe Configuration, Debug and Extension options tab of the IntelFPGA P-Tile Avalon-MM IP for PCI Express.
When using bifurcated ports, you can enable the Debug Toolkit for each bifurcatedport by enabling the option Enable Debug Toolkit on each of the bifurcated ports.
Note: When you enable the P-Tile Debug Toolkit in the IP, the Hard IP reconfigurationinterface and the PHY reconfiguration interface will be used by the Debug Toolkit.Hence, you will not be able to drive logic on these interfaces from the FPGA fabric.
6.2.3. Launching the P-Tile Debug Toolkit
Use the design example you compiled by following the Quick Start Guide to familiarizeyourself with the P-Tile Debug Toolkit. Follow the steps in the Generating the DesignExample and Compiling the Design Example to generate the SRAM Object File, (.sof)for this design example.
To use the P-Tile Debug Toolkit, download the .sof to the Intel Development Kit. Then,open the System Console and load the design to the System Console as well. Loadingthe .sof to the System Console allows the System Console to communicate with thedesign using NPDME. NPDME is a JTAG-based Avalon-MM master. It drives Avalon-MMslave interfaces in the PCIe design. When using NPDME, the Intel Quartus Primesoftware inserts the debug interconnect fabric to connect with JTAG.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
92
Here are the steps to complete these tasks:
1. Use the Intel Quartus Prime Programmer to download the .sof to the Intel FPGADevelopment Kit.
Note: To ensure correct operation, use the same version of the Intel QuartusPrime Programmer and Intel Quartus Prime Pro Edition software that youused to generate the .sof.
2. To load the design into System Console:
a. Launch the Intel Quartus Prime Pro Edition software.
b. Start System Console by choosing Tools, then System Debugging Tools,then System Console.
c. On the System Console File menu, select Load design and browse to the .soffile.
d. Select the .sof and click OK. The .sof loads to the System Console.
3. The System Console Toolkit Explorer window will list all the DUTs in the designthat have the P-Tile Debug Toolkit enabled.
a. Select the DUT with the P-Tile Debug Toolkit you want to view. This will openthe Debug Toolkit instance of that DUT in the Details window.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
93
b. Click on the ptile_debug_toolkit_avmm to open that instance of the Toolkit.Once the Debug Toolkit is initialized and loaded, you will see the followingmessage in the Messages window: “Initializing P-Tile debug toolkit –done”.
c. A new window Main view will open with a view of all the channels in thatinstance.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
94
6.2.4. Using the P-Tile Debug Toolkit
The following sections describe the different tabs and features available in the DebugToolkit.
A. Main View
The main view tab lists a summary of the transmitter and receiver settings perchannel for the given instance of the PCIe IP.
The following table shows the channel mapping when using bifurcated ports.
Table 64. Channel Mapping for Bifurcated Ports
Toolkit Channel X16 Mode 2X8 Mode 4x4 Mode
Lane 0 Lane 0 Lane 0 Lane 0
Lane 1 Lane 1 Lane 1 Lane 1
Lane 2 Lane 2 Lane 2 Lane 2
Lane 3 Lane 3 Lane 3 Lane 3
Lane 4 Lane 4 Lane 4 Lane 0
Lane 5 Lane 5 Lane 5 Lane 1
Lane 6 Lane 6 Lane 6 Lane 2
Lane 7 Lane 7 Lane 7 Lane 3
Lane 8 Lane 8 Lane 0 Lane 0
continued...
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
95
Toolkit Channel X16 Mode 2X8 Mode 4x4 Mode
Lane 9 Lane 9 Lane 1 Lane 1
Lane 10 Lane 10 Lane 2 Lane 2
Lane 11 Lane 11 Lane 3 Lane 3
Lane 12 Lane 12 Lane 4 Lane 0
Lane 13 Lane 13 Lane 5 Lane 1
Lane 14 Lane 14 Lane 6 Lane 2
Lane 15 Lane 15 Lane 7 Lane 3
B. Toolkit Parameters
The Toolkit parameters window has 2 sub-tabs.
B.1. P-Tile Information
This lists a summary of the P-Tile PCIe IP parameter settings in the PCIe IP ParameterEditor when the IP was generated, as read by the P-Tile Debug Toolkit when initialized.
When using bifurcated ports, you will see all the P-Tile information for each port forwhich the Debug Toolkit has been enabled.
All the information is read-only.
Use the Get P-tile Info button to read the settings.
Table 65. P-Tile Available Parameter Settings
Parameter Values Descriptions
Intel Vendor ID 1172 Indicates the Vendor ID as set in the IPParameter Editor.
Protocol PCIe Indicates the Protocol.
HIP Type Root Port, End Point Indicates the Hard IP Port type.
Intel IP Type intel_pcie_ptile_ast,intel_pcie_ptile_avmm Indicates the IP type used.
Advertised speed Gen3, Gen4 Indicates the advertised speed asconfigured in the IP Parameter Editor.
Advertised width x16, x8, x4 Indicates the advertised width asconfigured in the IP Parameter Editor.
Negotiated speed Gen3, Gen4 Indicates the negotiated speed duringlink training.
Negotiated width x16, x8, x4 Indicates the negotiated link widthduring link training.
Link status Link up, link down Indicates if the link (DL) is up or not.
Retimer 1 Detected, not detectedIndicates if a retimer was detected
between the Root Port and theEndpoint.
Retimer 2 Detected, not detectedIndicates if a retimer was detected
between the Root Port and theEndpoint.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
96
Figure 38. Example of P-Tile Parameter Settings
B.2. PCIe Configuration Space
This lists a summary of the P-Tile PCIe configuration settings of the PCIe configurationspace registers, as read by the P-Tile Debug Toolkit when initialized.
All the information is read-only.
Use the Read cfg space button to read the settings.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
97
Figure 39. Example of P-Tile PCIe Configuration Settings
C. Channel Parameters
The channel parameters window allows you to monitor and control the transmitter andreceiver settings for a given channel. It has the following 2 sub-windows.
C.1. TX Path
This tab allows you to monitor and control the transmitter settings for the channelselected. Use the TX Refresh button to read the settings, TX Apply Ch to apply thesettings to the selected channel, and TX apply all to apply the settings to allchannels.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
98
Table 66. Transmitter Settings
Parameters Values Descriptions
PHY Status
Refclk enable Enable, Disable
Indicates reference clock isenabled for the PHY.
Enable: Reference clock isenabled for the PHY.
Disable: Reference clock isdisabled for the PHY.
PHY reset Normal, Reset
Indicates the PHY is in resetmode.
Normal: PHY is out of reset.Reset: PHY is in reset.
TX Status
TX Lane enable Enable, Disable
Indicates if TX lane isenabled in the PHY.
Enable: TX lane is enabled inthe PHY.
Disable: TX lane is disabledin the PHY.
TX Data enable Enable, Disable
Indicates if TX driver isenabled and serial data is
transmitted.Enable: TX driver for the
corresponding lane isenabled.
Disable: TX driver for thecorresponding lane is
disabled.
TX Reset Normal, Reset
Indicates if TX (TX datapath,TX settings) is in reset ornormal operating mode.Normal: TX is in normal
operating mode.Reset: TX is in reset.
TX PLL
TX PLL enable Enable, Disable
Indicates if the TX PLL ispowered on or powered
down. This is dependent onthe PLL selected as indicated
by TX PLL select.There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL
status corresponding to thatQuad.
• TX path for Ch0 to 3:Status of PLLs in Quad0
• TX path for Ch4 to 7:Status of PLLs in Quad1
• TX path for Ch8 to 11:Status of PLLs in Quad2
• TX path for Ch12 to 15:Status of PLLs in Quad3
Enable: TX PLL is poweredon.
Disable: TX PLL is powereddown.
TX PLL selectPLLA: Gen1/Gen2PLLB: Gen3/Gen4
Indicates which PLL isselected.
continued...
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
99
Parameters Values Descriptions
There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL
status corresponding to thatQuad.
• TX path for Ch0 to 3:Status of PLLs in Quad0
• TX path for Ch4 to 7:Status of PLLs in Quad1
• TX path for Ch8 to 11:Status of PLLs in Quad2
• TX path for Ch12 to 15:Status of PLLs in Quad3
TX PLL lock Green, Red
Indicates if TX PLL is locked.This is dependent on the PLLselected as indicated by TX
PLL select.There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL
status corresponding to thatQuad.
• TX path for Ch0 to 3:Status of PLLs in Quad0
• TX path for Ch4 to 7:Status of PLLs in Quad1
• TX path for Ch8 to 11:Status of PLLs in Quad2
• TX path for Ch12 to 15:Status of PLLs in Quad3
Green: TX PLL is locked.Red: TX PLL is not locked.
TX VOD
Iboost levelGen3: 15Gen4: 15
Indicates the transmittercurrent boost level when theTX amplitude boost mode is
enabled.
Vboost enGen3 EnableGen4 Enable
Indicates if the TX swingboost level is enabled.
Enable: TX swing boost isenabled.
Disable: TX swing boost isdisabled.
Vboost levelGen3: 5Gen4: 5
Indicates the TX Vboostlevel.
TX Equalization
Pre-shoot coefficientGen3: 20 (Preset 8)Gen4: 0 (Preset 0)
Indicates transmitter driveroutput pre-emphasis (pre-
shoot coefficient).
Main coefficientGen3: 30 (Preset 8)Gen4: 30 (Preset 0)
Indicates transmitter driveroutput pre-emphasis (main
coefficient).
Post coefficientGen3: 20 (Preset 8)Gen4: 40 (Preset 0)
Indicates transmitter driveroutput pre-emphasis (post
coefficient).
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
100
Figure 40. Example of Transmitter Settings
C.1. RX Path
This tab allows you to monitor and control the receiver settings for the channelselected. Use the RX Refresh button to read the settings, RX Apply Ch to apply thesettings to the selected channel, and RX apply all to apply the settings to allchannels.
Table 67. Receiver Settings
Parameters Values Descriptions
RX Status
RX Lane enable Enable, Disable
Indicates if RX lane isenabled in the PHY.
Enable: RX lane is enabledin the PHY.
Disable: RX lane is disabledin the PHY.
RX Data enable Enable, Disable
Indicates if RX driver isenabled and serial data is
transmitted.Enable: RX driver for the
corresponding lane isenabled.
Disable: RX driver for thecorresponding lane is
disabled.
continued...
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
101
Parameters Values Descriptions
RX Reset Normal, Reset
Indicates if RX (RX datapath,RX settings) is in reset ornormal operating mode.Normal: RX is in normal
operating mode.Reset: RX is in reset.
RX LOS <1,0>
Indicates if the receiver haslost the signal.
1: Receiver loss of signal.0: Receiver has a data
signal.
RX CDR
CDR Lock Green, RedIndicates the CDR lock state.
Green: CDR is locked.Red: CDR is not locked.
CDR Mode Locked to Reference (LTR),Locked to Data (LTD)
Indicates the CDR lockmode.
LTR: CDR is locked toreference clock.
LTD: CDR is locked to data.
RX Equalization
Adapt Mode
Gen3: Gen3 adaptationmode.
Gen4: Gen4 adaptationmode.
Indicates the RX adaptationmode.
Adapt ContinuousGen3: 1Gen4: 1
Indicates if the receiver is incontinuous adaptation.
• 0 - continuous adaptationoff.
• 1 - continuous adaptationon.
RX ATTGen3: 0Gen4: 0
Indicates the RX equalizationattenuation level.
RX CTLE BoostGen3: 12Gen4: 16
Indicates the RX CTLE boostvalue.
RX CTLE PoleGen3: 2Gen4: 2
Indicates the RX CTLE polevalue.
RX VGA1Gen3: 5Gen4: 5
Indicates the RX AFE firststage VGA gain value.
RX VGA2Gen3: 5Gen4: 5
Indicates the RX AFE secondstage VGA gain value.
RX FOM <0-255>
Indicates the Receiver Figureof Merit (FOM) / quality ofthe received data eye. A
higher value indicates betterlink equalization, with 8'd0
indicating the worstequalization setting and
8'd255 indicating the bestequalization setting.
DFE Enable Enable, Disable
Indicates DFE adaptation isenabled for taps 1 - 5.
Enable: DFE adaptation isenabled for taps 1 - 5.
continued...
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
102
Parameters Values Descriptions
Disable: DFE adaptation isdisabled for taps 1 - 5.
DFE Tap1 adapted value <-128 to 127>
Indicates the adapted valueof DFE tap 1. This is a
signed input (two'scomplement encoded).
DFE Tap2 adapted value <-32 to 31>
Indicates the adapted valueof DFE tap 2. This is a
signed input (two'scomplement encoded).
DFE Tap3 adapted value <-32 to 31>
Indicates the adapted valueof DFE tap 3. This is a
signed input (two'scomplement encoded).
DFE Tap4 adapted value <-32 to 31>
Indicates the adapted valueof DFE tap 4. This is a
signed input (two'scomplement encoded).
DFE Tap5 adapted value <-32 to 31>
Indicates the adapted valueof DFE tap 5. This is a
signed input (two'scomplement encoded).
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
103
Figure 41. Example of Receiver Settings
Eye Viewer
The P-Tile Debug Toolkit supports running eye tests for Intel devices with P-Tile. TheEye Viewer tool allows you to set up and run eye tests, monitoring bit errors.
1. In the System Console Tools menu option, click on Eye View Tool.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
104
Figure 42. Opening the Eye Viewer
2. This will open a new tab Eye View Tool next to the Main View tab. Choose theinstance and channel for which you want to run the eye view tests.
Figure 43. Opening the Instance and Channel
3. Choose the eye vertical step setting from the drop-down menu. The eye view toolallows you to choose between vertical step sizes of 1, 2, 4, 8.
Note: The time taken for the eye view tool to draw the eye varies with differentvertical step sizes (1 results in a faster eye plot when compared to 8).
Figure 44. Choosing the Step Size
4. The messages window displays information messages to indicate the eye viewtool's progress.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
105
Figure 45. Eye View Tool Messages
5. Once the eye plot is complete, the eye height, eye width and eye diagram aredisplayed.
Figure 46. Sample Eye Plot
6.2.5. Enabling the P-Tile Link Inspector
To enable the Link Inspector, enable the option Enable Debug Toolkit and EnablePCIe Link Inspector in the PCIe Configuration, Debug and Extension Optionstab.
Figure 47. Enabling the Link Inspector
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
106
6.2.6. Using the P-Tile Link Inspector
The Link Inspector is found under the PCIe Link Inspector tab after opening theDebug Toolkit:
Figure 48. View of the Link Inspector
When the Dump LTSSM Sequence to Text File button is initially clicked, a text file(ltssm_sequence_dump_p*.txt) with the LTSSM information is created in thelocation from where the System Console window is opened. Depending on the PCIetopology, there can be up to four text files. Subsequent LTSSM sequence dumps willappend to the respective files.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
107
Figure 49. Example LTSSM Sequence Dump (Beginning)
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
108
Figure 50. Example LTSSM Sequence Dump (End)
Each LTSSM monitor has a FIFO storing the time values and captured LTSSM states.The FIFO is written when there is a state transition. When you want to dump theLTSSM sequence, a single read of the FIFO status of the respective core is performed.Depending on the empty status and how many entries are in the FIFO, successivereads are executed.
6. Troubleshooting/Debugging
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
109
A. Configuration Space Registers
A.1. Configuration Space Registers
In addition to accessing the Endpoint's configuration space registers by sendingConfiguration Read/Write TLPs via the Avalon-ST interface, the application logic canalso gain read access to these registers via the Configuration Output Interface(tl_cfg*). Furthermore, the Hard IP Reconfiguration Interface (a User Avalon-MMinterface) also provides read/write access to these registers.
For signal timings on the User Avalon-MM interface, refer to the Avalon InterfaceSpecifications document.
The table PCIe Configuration Space Registers describes the registers for each PF. Tocalculate the address for a particular register in a particular PF, add the offset for thatPF from the table Configuration Space Offsets to the byte address for that register asgiven in the table PCIe Configuration Space Registers.
Table 68. Configuration Space Offsets
Registers User Avalon-MM Offsets
Physical function 0 0x00000
Physical function 1 0x10000
Physical function 2 0x20000
Physical function 3 0x30000
Physical function 4 0x40000
Physical function 5 0x50000
Physical function 6 0x60000
Physical function 7 0x70000
Port Configuration and Status Register 0x104000
Debug (DBI) Register 0x104200, 0x104204
Table 69. PCIe Configuration Space Registers for x16/x8/x4 Controllers
Byte Address Hard IP Configuration SpaceRegister
Corresponding Section in PCIeSpecification
x16 (Port 0) = 0x000 : 0x03Cx8 (Port 1) = 0x000 : 0x03C
x4 (Ports 2,3) = 0x000 : 0x03C
PCI Header Type 0/1 ConfigurationRegisters
Type 0/1 Configuration Space Header
x16 (Port 0) = 0x040 : 0x044x8 (Port 1) = 0x040 : 0x044
x4 (Ports 2,3) = 0x040 : 0x044
Power Management PCI Power Management CapabilityStructure
continued...
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
Byte Address Hard IP Configuration SpaceRegister
Corresponding Section in PCIeSpecification
x16 (Port 0) = 0x050 : 0x064x8 (Port 1) = 0x050 : 0x064
x4 (Ports 2,3) = 0x050 : 0x064
MSI Capability MSI Capability Structure, see also PCILocal Bus Specification
x16 (Port 0) = 0x070 : 0x0A8x8 (Port 1) = 0x070 : 0x0A8
x4 (Ports 2,3) = 0x070 : 0x0A8
PCI Express Capability PCI Express Capability Structure
x16 (Port 0) = 0x0B0 : 0x0B9x8 (Port 1) = 0x0B0 : 0x0B9
x4 (Ports 2,3) = 0x0B0 : 0x0B9
MSI-X Capability MSI-X Capability Structure, see alsoPCI Local Bus Specification
x16 (Port 0) = 0x0BC : 0x0FCx8 (Port 1) = 0x0BC : 0x0FC
x4 (Ports 2,3) = 0x0BC : 0x0FC
Reserved N/A
x16 (Port 0) = 0x100 : 0x144x8 (Port 1) = 0x100 : 0x144
x4 (Ports 2,3) = 0x100 : 0x144
Advanced Error Reporting (AER) Advanced Error Reporting CapabilityStructure
x16 (Port 0) = 0x148 : 0x164x8 (Port 1) = 0x148 : 0x164
x4 (Ports 2,3) = 0x148 : 0x164
Virtual Channel Capability Virtual Channel Capability Structure
x16 (Port 0) = 0x178 : 0x17Cx8 (Port 1) = 0x178 : 0x17C
x4 (Ports 2,3) = N/A
Alternative Routing-ID Implementation(ARI)
ARI Capability Structure
x16 (Port 0) = 0x188 : 0x1B4x8 (Port 1) = 0x188 : 0x1A4
x4 (Ports 2,3) = 0x188 : 0x1A4
Secondary PCI Express ExtendedCapability Header
PCI Express Extended Capability
x16 (Port 0) = 0x1B8 : 0x1E4x8 (Port 1) = 0x1A8 : 0x1CC
x4 (Ports 2,3) = 0x1A8 : 0x1C8
Physical Layer 16.0 GT/s ExtendedCapability
Physical Layer 16.0 GT/s ExtendedCapability Structure
x16 (Port 0) = 0x1E8 : 0x22Cx8 (Port 1) = 0x1D0 : 0x1F4
x4 (Ports 2,3) = 0x1CC : 0x1E0
Margining Extended Capability Margining Extended CapabilityStructure
x16 (Port 0) = 0x230 : 0x26Cx8 (Port 1) = 0x1F8 : 0x234
x4 (Ports 2,3) = N/A
SR-IOV Capability SR-IOV Capability Structure
x16 (Port 0) = 0x270 : 0x2F8x8 (Port 1) = 0x238 : 0x2C0
x4 (Ports 2,3) = 0x1E4 : 0x26C
TLP Processing Hints (TPH) Capability TLP Processing Hints (TPH) CapabilityStructure
x16 (Port 0) = 0x2FC : 0x300x8 (Port 1) = 0x2C4 : 0x2C8
x4 (Ports 2,3) = N/A
Address Translation Services (ATS)Capability
Address Translation Services ExtendedCapability (ATS) in Single Root I/O
Virtualization and Sharing Specification
x16 (Port 0) = 0x30C : 0x314x8 (Port 1) = 0x2D4 : 0x2DC
x4 (Ports 2,3) = 0x280 : 0x288
Access Control Services (ACS)Capability
Access Control Services (ACS)Capability
x16 (Port 0) = 0x318 : 0x324x8 (Port 1) = 0x2E0 : 0x2EC
x4 (Ports 2,3) = N/A
Page Request Services (PRS) Capability Page Request Services (PRS) Capability
continued...
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
111
Byte Address Hard IP Configuration SpaceRegister
Corresponding Section in PCIeSpecification
x16 (Port 0) = 0x328 : 0x32Cx8 (Port 1) = 0x2F0 : 0x2F4
x4 (Ports 2,3) = N/A
Latency Tolerance Reporting (LTR)Capability
Latency Tolerance Reporting (LTR)Capability
x16 (Port 0) = 0x330 : 0x334x8 (Port 1) = 0x2F8 : 0x2FC
x4 (Ports 2,3) = N/A
Process Address Space (PASID)Capability
Process Address Space (PASID)Capability Structure
x16 (Port 0) = 0x338 : 0x434x8 (Port 1) = 0x300 : 0x3FC
x4 (Ports 2,3) = 0x2AC : 0x3A8
RAS D.E.S. Capability (VSEC)
x16 (Port 0) = 0x470 : 0x478x8 (Port 1) = 0x438 : 0x440
x4 (Ports 2,3) = 0x3E4 : 0x3EC
Data Link Feature Extended Capability
x16 (Port 0) = 0xD00 : 0xD58x8 (Port 1) = 0xD00 : 0xD58
x4 (Ports 2,3) = 0xD00 : 0xD58
Intel-defined VSEC
A.1.1. Register Access Definitions
This document uses the following abbreviations when describing register accesses.
Table 70. Register Access Abbreviations
Abbreviation Meaning
RW Read and write access
RO Read only
WO Write only
RW1C Read write 1 to clear
RW1CS Read write 1 to clear sticky
RWS Read write sticky
Note: Sticky bits are not initialized or modified by hot reset.
A.1.2. PCIe Configuration Header Registers
The Corresponding Section in PCIe Specification column in the tables in theConfiguration Space Registers section lists the appropriate sections of the PCI ExpressBase Specification that describe these registers.
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
112
Figure 51. PCIe Type 0 Configuration Space Registers - Byte Address Offsets and Layout
0x0000x0040x0080x00C0x0100x0140x0180x01C0x0200x0240x0280x02C0x0300x0340x0380x03C
Device ID Vendor IDStatus Command
Class Code Revision ID
0x00 Header Type 0x00 Cache Line SizeBAR RegistersBAR RegistersBAR RegistersBAR RegistersBAR RegistersBAR Registers
ReservedSubsystem Device ID Subsystem Vendor ID
ReservedReserved
Reserved
Capabilities Pointer
0x00 Interrupt Pin Interrupt Line
31 24 23 16 15 8 7 0
Figure 52. PCIe Type 1 Configuration Space Registers - Byte Address Offsets and Layout
0x00000x004
Device ID31 24 23 16 15 8 7 0
0x0080x00C0x0100x0140x0180x01C0x0200x0240x0280x02C0x0300x0340x038
0x03C
Vendor ID
BIST Header Type Primary Latency Timer Cache Line Size
Status CommandClass Code Revision ID
BAR RegistersBAR Registers
Secondary Latency Timer Subordinate Bus Number Secondary Bus Number Primary Bus NumberSecondary Status I/O Limit I/O Base
Memory Limit Memory Base
Prefetchable Base Upper 32 BitsPrefetchable Limit Upper 32 Bits
I/O Limit Upper 16 Bits I/O Base Upper 16 BitsReserved Capabilities Pointer
Expansion ROM Base AddressBridge Control Interrupt Pin Interrupt Line
Prefetchable Memory Limit Prefetchable Memory Base
Related Information
PCI Express Base Specification 4.0
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
113
A.1.3. PCI Express Capability Structures
The layouts of the most basic Capability Structures are provided below. Refer to thePCI Express Base Specification for more information about these registers.
Figure 53. Power Management Capability Structure - Byte Address Offsets and Layout
0x040
0x04C
Capabilities Register Next Cap Ptr
Data
31 24 23 16 15 8 7 0Capability ID
Power Management Status and ControlPM Control/StatusBridge Extensions
Figure 54. MSI Capability Structure
0x050
0x0540x058
Message ControlConfiguration MSI Control Status
Register Field DescriptionsNext Cap Ptr
Message AddressMessage Upper Address
Reserved Message Data
31 24 23 16 15 8 7 0
0x05C
Capability ID
Figure 55. PCI Express Capability Structure - Byte Address Offsets and LayoutIn the following table showing the PCI Express Capability Structure, registers that are not applicable to adevice are reserved.
0x070
0x0740x0780x07C0x0800x0840x0880x08C0x0900x0940x0980x09C0x0A00x0A4
0x0A8
PCI Express Capabilities Register Next Cap Pointer
Device CapabilitiesDevice Status Device Control
Slot Capabilities
Root StatusDevice Compatibilities 2
Link Capabilities 2Link Status 2 Link Control 2
Slot Capabilities 2
Slot Status 2 Slot Control 2
31 24 23 16 15 8 7 0PCI Express
Capabilities ID
Link CapabilitiesLink Status Link Control
Slot Status Slot Control
Device Status 2 Device Control 2
Root Capabilities Root Control
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
114
Figure 56. MSI-X Capability Structure
0x0B0
0x0B4
0x0B8
Message Control Next Cap Ptr
MSI-X Table Offset
MSI-X Pending Bit Array (PBA) Offset
31 24 23 16 15 8 7 0Capability ID
3 2
MSI-XTable BAR
IndicatorMSI-X
PendingBit Array
- BARIndicator
Figure 57. PCI Express AER Extended Capability Structure
0x100
0x104 Uncorrectable Error Status RegisterPCI Express Enhanced Capability Register
Uncorrectable Error Severity Register
Uncorrectable Error Mask Register0x108
0x10C
0x110
0x114
0x118
0x11C
0x12C
0x130
0x134
Correctable Error Status Register
Correctable Error Mask Register
Advanced Error Capabilities and Control Register
Header Log Register
Root Error Command Register
Root Error Status Register
Error Source Identification Register Correctable Error Source Identification Register
31 16 15 0
Related Information
PCI Express Base Specification 4.0
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
115
A.1.4. Physical Layer 16.0 GT/s Extended Capability Structure
Figure 58. Physical Layer 16.0 GT/s Extended Capability Structure
A.1.5. MSI-X Registers
This section describes the registers previously shown in the MSI-X capability structure.
Table 71. MSI-X Control Register
Bit Location Description Access Default Value
31 MSI-X Enable: This bit mustbe set to enable the MSI-X
interrupt generation.
RW 0
30 MSI-X Function Mask: Thisbit can be set to mask allMSI-X interrupts from this
function.
RW 0
29:27 Reserved RO 0
26:16 Size of the MSI-X table(number of MSI-X interruptvectors). The value in this
field is one less than the sizeof the table set up for thisfunction. Maximum value is
0x7FF (2048 interruptvectors).
This field is shared amongall VFs attached to one PF.
RO Programmed via theprogramming interface.
15:8 Next Capability PointerPoints to the PCI Express
Capability.
RO Programmed via theprogramming interface.
7:0 Capability ID assigned byPCI-SIG.
RO 0x11
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
116
Table 72. MSI-X Table Offset Register
Bit Location Description Access Default Value
2:0 BAR Indicator Register:Specifies the BAR
corresponding to thememory address range
where the MSI-X table ofthis function is located (000= VF BAR0, 001 = VF BAR1,
…, 101 = VF BAR5).This field is shared amongall VFs attached to one PF.
RO Programmed via theprogramming interface.
31:3 Offset of the memoryaddress where the MSI-X
table is located, relative tothe specified BAR. Theaddress is extended by
appending three zeroes tomake it Qword aligned.
This field is shared amongall VFs attached to one PF.
RO Programmed via theprogramming interface.
Table 73. MSI-X Pending Bit Array Register
Bit Location Description Access Default Value
2:0 BAR Indicator Register:Specifies the BAR
corresponding to thememory address range
where the Pending Bit Arrayof this function is located
(000 = VF BAR0, 001 = VFBAR1, …, 101 = VF BAR5).This field is shared amongall VFs attached to one PF.
RO Programmed via theprogramming interface.
31:3 Offset of the memoryaddress where the PendingBit Array is located, relativeto the specified BAR. Theaddress is extended by
appending three zeroes tomake it Qword aligned.
This field is shared amongall VFs attached to one PF.
RO Programmed via theprogramming interface.
A.2. Intel-Defined VSEC Capability Registers
Table 74. Intel-Defined VSEC Capability Registers (0xD00 : 0xD58)
31 : 20 19 : 16 15 : 8 7 : 0 PCIe Byte Offset
Next Cap Offset Version PCI Express Extended Capability ID 00h
VSEC Length VSEC Rev VSEC ID 04h
Intel Marker 08h
JTAG Silicon ID DW0 0Ch
JTAG Silicon ID DW1 10h
JTAG Silicon ID DW2 14h
continued...
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
117
31 : 20 19 : 16 15 : 8 7 : 0 PCIe Byte Offset
JTAG Silicon ID DW3 18h
CvP Status User Configurable Device/Board ID 1Ch
CvP Mode Control 20h
CvP Data 2 24h
CvP Data 28h
CvP Programming Control 2Ch
General Purpose Control and Status 30h
Uncorrectable Internal Error Status Register 34h
Uncorrectable Internal Error Mask Register 38h
Correctable Error Status Register 3Ch
Correctable Error Mask Register 40h
SSM IRQ Request & Status 44h
SSM IRQ Result Code 1 Shadow 48h
SSM IRQ Result Code 2 Shadow 4Ch
SSM Mailbox 50h
SSM Credit 0 Shadow 54h
SSM Credit 1 Shadow 58h
A.2.1. Intel-Defined VSEC Capability Header (Offset 00h)
Table 75. Intel-Defined VSEC Capability Header
Bits Register Description Default Value Access
[31:20] Next Capability Pointer.Value is the starting addressof the next CapabilityStructure implemented, ifany. Otherwise, NULL. Referto the Configuration AddressMap.
Variable RO
[19:16] Capability Version. PCIespecification-defined valuefor VSEC Capability Version.
0x1 RO
[15:0] Extended Capability ID. PCIespecification-defined valuefor VSEC ExtendedCapability ID.
0x000B RO
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
118
A.2.2. Intel-Defined Vendor Specific Header (Offset 04h)
Table 76. Intel-Defined Vendor Specific Header
Bits Register Description Default Value Access
[31:20] VSEC Length. Total length ofthis structure in bytes.
0x5C RO
[19:16] VSEC Rev. User configurableVSEC revision.
k_vsec_rev_i RO
[15:0] VSEC ID. User configurableVSEC ID. The default valueis 0x1172 (the Intel VendorID), but you can change thisID to your own Vendor ID.
0x1172 RO
A.2.3. Intel Marker (Offset 08h)
Table 77. Intel Marker
Bits Register Description Default Value Access
[31:0]
Intel Marker - An additionalmarker for standard Intelprogramming software to beable to verify that this is theright structure.
0x41721172 RO
A.2.4. JTAG Silicon ID (Offset 0x0C - 0x18)
This read-only register returns the JTAG Silicon ID. Intel programming software usesthis JTAG ID to ensure that is is using the correct SRM Object File (*.sof).
These registers are only good for Port 0 (PCIe Gen4 x16). They are blocked for theother Ports.
Table 78. JTAG Silicon ID Registers
Bits Register Description Default Value(6) Access
[127:96] JTAG Silicon ID DW3 Unique ID RO
[95:64] JTAG Silicon ID DW2 Unique ID RO
[63:32] JTAG Silicon ID DW1 Unique ID RO
[31:0] JTAG Silicon ID DW0 Unique ID RO
A.2.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D)
This register provides a user configurable device or board ID so that the user softwarecan determine which .sof file to load into the device.
This register is only available for Port 0 (PCIe Gen4 x16). It is blocked for the otherPorts.
(6) Because the Silicon ID is a unique value, it does not have a global default value.
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
119
Table 79. User Configurable Device and Board ID Register
Bits Register Description Default Value Access
[15:0] This register allows you tospecify the ID of the .soffile to be loaded.
From configuration bits RO
A.2.6. General Purpose Control and Status Register (Offset 0x30)
This register provides up to eight I/O pins each for Application Layer Control andStatus requirements. This feature supports Partial Reconfiguration of the FPGA fabric.Partial Reconfiguration only requires one input pin and one output pin. The otherseven I/Os make this interface extensible.
Table 80. General Purpose Control and Status Register
Bits Register Description Default Value Access
[31:16] Reserved. N/A RO
[15:8] General Purpose Status. TheApplication Layer can readthese status bits. These bitsare only available for Port 0(PCIe Gen4 x16). They areblocked for the other Ports.
0x00 RO
[7:0] General Purpose Control.The Application Layer canwrite these control bits.These bits are only availablefor Port 0 (PCIe Gen4 x16).They are blocked for theother Ports.
0x00 RW
A.2.7. Uncorrectable Internal Error Status Register (Offset 0x34)
This register reports the status of the internally checked errors that are uncorrectable.When these specific errors are enabled by the Uncorrectable Internal ErrorMask register, they are forwarded as Uncorrectable Internal Errors.
Note: This register is for debug only. Only use this register to observe behavior, not to drivelogic custom logic.
Table 81. Uncorrectable Internal Error Status Register
Bits Register Description Default Value Access
[31:13] Reserved 0x0 RO
[12] Debug Bus Interface (DBI)access error status fromConfig RAM block.
0x0 RW1CS
[11] Uncorrectable ECC errorfrom Config RAM block.
0x0 RW1C
[10:9] Reserved 0x0 RO
[8] RX Transaction Layer parityerror reported by the IPcore.
0x0 RW1CS
continued...
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
120
Bits Register Description Default Value Access
[7] TX Transaction Layer parityerror reported by the IPcore.
0x0 RW1CS
[6] Uncorrectable Internal Errorreported by the FPGA.
0x0 RW1CS
[5] cvp_config_error_latched: Configuration errordetected in CvP mode isreported as an uncorrectableerror. Set wheneverssm_cvp_config_error ofthe SSM Scratch CvP Statusregister bit[1] rises in CvPmode. This bit is onlyavailable for Port 0 (PCIeGen4 x16), but not for theother Ports.
0x0 RW1CS
[4:0] Reserved 0x0 RO
Note: The access code RW1CS represents Read Write 1 to Clear Sticky.
A.2.8. Uncorrectable Internal Error Mask Register (Offset 0x38)
This register controls which errors are forwarded as internal uncorrectable errors.
Table 82. Uncorrectable Internal Error Mask Register
Bits Register Description Default Value Access
[31:13] Reserved 0x0 RO
[12] Mask for Debug BusInterface (DBI) access error.
0x1 RWS
[11] Mask for Uncorrectable ECCerror from Config RAM block.
0x1 RWS
[10:9] Reserved 0x0 RO
[8] Mask for RX TransactionLayer parity error reportedby the IP core.
0x1 RWS
[7] Mask for TX TransactionLayer parity error reportedby the IP core.
0x1 RWS
[6] Mask for UncorrectableInternal error reported bythe FPGA.
0x1 RWS
[5] Mask for Configuration Errordetected in CvP mode. Thisbit is only available for Port0 (PCIe Gen4 x16), but notfor the other Ports.
0x0 RWS
[4:0] Reserved 0x0 RO
Note: The access code RWS stands for Read Write Sticky, meaning that the value is retainedafter a soft reset of the IP core.
A. Configuration Space Registers
UG-20237 | 2020.07.13
Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide
121
A.2.9. Correctable Internal Error Status Register (Offset 0x3C)
The Correctable Internal Error Status register reports the status of theinternally checked errors that are correctable. When these specific errors are enabledby the Correctable Internal Error Mask register, they are forwarded ascorrectable internal errors. This register is for debug only. Only use this register toobserve behavior, not to drive custom logic
Table 83. Correctable Internal Error Status Register
Bits Register Description Default Value Access
[31:12] Reserved 0x0 RO
[11] Correctable ECC error statusfrom Config RAM.
0x0 RW1CS
[10:7] Reserved 0x0 RO
[6] Correctable Internal Errorreported by the FPGA.
0x0 RW1CS
[5] cvp_config_error_latched: Configuration errordetected in CvP mode (to bereported as correctable) -Set whenevercvp_config_error riseswhile in CvP mode. This bitis only available for Port 0(PCIe Gen4 x16), but not forthe other Ports.
0x0 RW1CS
[4:0] Reserved 0x0 RO
A.2.10. Correctable Internal Error Mask Register (Offset 0x40)
This register controls which errors are forwarded as internal correctable errors.
Table 84. Correctable Internal Error Mask Register
Bits Register Description Default Value Access
[31:12] Reserved 0x0 RO
[11] Mask for Correctable ECCerror status for Config RAM.
0x1 RWS
[10:7] Reserved 0x0 RWS
[6] Mask for CorrectableInternal Error reported bythe FPGA.
0x1 RWS
[5] Mask for Configuration Errordetected in CvP mode. Thisbit is only available for Port0 (PCIe Gen4 x16), but notfor the other Ports.
0x1 RWS
[4] Reserved 0x1 RWS
[3:0] Reserved 0x0 RWS
A. Configuration Space Registers
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
122
B. Document Revision History
B.1. Document Revision History for the Intel FPGA P-Tile Avalonmemory mapped IP for PCI Express User Guide
Document Version Intel QuartusPrime Version IP Version Changes
2020.07.13 20.2 3.0.0
Added support for the Gen3 x8 Endpoint andGen4 x8 Endpoint modes to the Features chapter.Updated the resource utilization numbers in the
Resource Utilization chapter.Added description for the Link Inspector in the
Debug Toolkit chapter.
2020.06.22 20.2 3.0.0
Added the lane reversal and polarity inversionsupport to the Features section.
Updated the bit ranges for the Next CapabilityOffset and Version fields in the Intel-Defined VSEC
Capability Registers section.
2020.04.29 20.1 2.0.0
Added clarification that VCS is the only simulatorsupported in the 20.1 release of Intel Quartus
Prime. Also added that PIPE mode simulations arenot supported in this release.
Changed the operation mode names from DMAMode with Data Movers to Endpoint Mode withData Movers, and from Bursting Slave Mode to
Endpoint Mode.
2020.04.28 20.1 2.0.0
Updated the document title to Intel FPGA P-TileAvalon memory mapped IP for PCI Express User
Guide to meet new legal naming guidelines.Updated the list of configurations supported in the
Features section.Replaced the Configuration Slave Interface with
the Control Register Access Interface.
2019.12.16 19.4 1.1.0Added parameters in Intel Quartus Prime to
control PASID and LTR.Added MSI extended data support.
2019.11.05 19.3 1.0.0
Added resource utilization numbers for the DMAdesign example in Intel Stratix 10 DX devices.Added the step to choose Intel Stratix 10 DX
devices to the Generating the Design Examplesection.
2019.10.28 19.3 1.0.0Removed a note containing a restriction on which
normal descriptors cannot be interrupted bypriority descriptors from the section Write Data
continued...
UG-20237 | 2020.07.13
Send Feedback
Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.
ISO9001:2015Registered
Document Version Intel QuartusPrime Version IP Version Changes
Mover Avalon-ST Descriptor Sinks, because allnormal descriptors being processed cannot be
interrupted.
2019.10.23 19.3 1.0.0 Initial release.
B. Document Revision History
UG-20237 | 2020.07.13
Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback
124