FPGA Based PCI Express Gen4 NVMe

SSD Platform

Amit Saxena, VP, Engineering

“The IP enabled solutions provider”

Santa Clara, CA

August 2016

AGENDA

• FPGA Based PCI Express Gen4 NVMeSSDC

• Design Challenges• Configurable IP Components• Mobiveil PCI Express Gen4 NVMe SSD

Platform• Summary

FPGA Based PCI Express Gen4 NVMe

SSDC Platform

FPGA Based PCI Express Gen4 NVMe SSDC Platform

On Chip Data Path

On Chip Control Path

PCIe IPNVMe

ControllerPCIe

PHY

PCIe

DDR4

Flash/Relia

bility (ECC)

ControllerFlash

Device FW

ARM Cortex A9DDR4

ControllerGPIO

HPS

Design Challenges

• Reference Design should be flexible to support different hardware configurations

• Reference Design should be able to handle various target technologies• It should be able to handle performance targets across technologies

• Reference Design should allow customization for individual implementations

• Reference Design should provide hooks for Statistics gathering, Error recovery, Reclaim and other value added Enterprise SSD functions

• Design Reuse is Critical

Feature Configurability

6Copyright Mobiveil, Inc.

PCIe

• Number of Virtual Channels

• SRIOV Needed

• Bifurcation Support

• DMA Needed

• INT/MSI/MSIX Support

• SRNS/SRIS

ECC

• BCH/LDPC

• Code Rate

• Programmable Padding/Puncturing

• User Defined LDPC Matrix

• Single/Dual Core

• Soft Read Procedure

NVMe

• Multipath IO Support

• LBA Size

• Number of IO Queues

• Vendor Command Support

• Optional Feature Support

• Vendor specific Arbitration

Flash Controller

• ONFI/Toggle IF

• Full Rate/Half rate/Quarter Rate Support

• Custom Command Support

• Number of LUNs

• Command Arbitration LUN Based

• Suspend/Resume Support

Implementation Challenges

7

HW/SW Partitioning

Data Path Width

Support

Clock Frequency

Processor

Dependency

Interfacing with 3rd

Party IPs

Efficient Buffering

Throughput

Balancing

Interfacing with 3rd

Party VIPs

Copyright Mobiveil, Inc.

Configurable IP Blocks

8

Address all Features

Design, Implementation,

Verification Effort

Latency,

Bandwidth, QOS

Area, Frequency

Copyright Mobiveil, Inc.

Configurable IP Components

PCI Express Controller (GPEX)

Copyright Mobiveil, Inc.

PCI Express (GPEX)

11

MRX MCTL MTX

DTXDCTLDRX

TRX TRX TTX

Receive Control Transmit

Transaction Layer

Data Link Layer

MAC Layer

Logic

Layers

PCS

PMA

PHY

Layers

PIPE 4

Application Interface

Serial Interface

Compliant to PCI Express Base 4.0

• PCISIG Certified IP• Supports Gen4, Gen3, Gen2 and Gen1

rates• Compliant to PIPE 4.3 specification• Supports x1, x2, x4, x8 and x16 lanes• Supports INTR, MSI, MSIX• 32, 64, 128, 256 and 512 Bit Data path

support• Supports 8, 16, 32 and 64 bit PIPE

Optional Feature Support• SR-IOV• ATS• ARI• FLR• LTR• SRIS

Provides PCS

Advanced Power Management• ASPM L1, Auxiliary power, beacon• Clock and Power Gating• L1 PM Substate

Value Added Features

• End to End Data parity protection

• Device specific control and status Registers, Loopback and test features

• Supports up to 8 virtual channels and traffic classes

• Supports isochronous transfer

• Supports RR, WRR and strict priority VC arbitration scheme

• Infinite credit option for selected types of traffic

• Supports all in-band messages including vendor defined message and broadcast message

Copyright Mobiveil, Inc.

LDPC Encoder/Decoder

LDPC Encoder/Decoder• LDPC scalable IP

• Supporting wide range of data-rates

• 50MB/s to 4.0GB/s for a single LDPC instance

• Major scalability parameters are (before IP instantiation):

• Codeword size (0.5KB vs. 1KB vs. 2KB, 4KB or 8KB)

• Maximum amount of supported parity

• Several parameters for degree of parallelism

• Memory access options

• IP Core Programmability Parameters (after IP instantiation):

• Simultaneous support for different amounts of parity

• Simultaneous support for several LDPC codes

• On-the-fly switching from one LDPC code to another

• User can program its own LDPC code/Matrix

13

▪ Lowest power LDPC decoder

▪ Code word size is 1KB+parity

▪ Total power is measured for TT, 0.9V, 25C

▪ TSMC 28 HPC process

Copyright Mobiveil, Inc.

Throughput @ EOL (MBPS)

Freq(MHz)

CellArea

(um2)

Memory(KB)

Power

Dynamic(mW)

Leakage(mW)

405 500 55,247 58 27.65 0.027

1183 500 127,880 57 66.79 0.059

1581 500 137,183 36 78.94 0.063

2214 700 135,604 57 102.82 0.206

UnH Certified NVM Express Controller IP

(UNEX)

NVM Express (UNEX) Controller

15

Command

Fetch &

Completion

Update

Interrupt

Logic

NVMe

Controller

RegistersAdmin & IO

Queue_0

Arbiter and

Command

Decoder

FW RQ

DMA

Data Buffer

DMA

Engine 1

DMA

Engine M

DMA

NVMe

Controller

RegistersAdmin & IO

Queue_1

Arbiter and

Command

Decoder

FW CQ

NVMe

Controller

RegistersAdmin & IO

Queue_N

Arbiter and

Command

Decoder

DMA CQ

Compliant to NVM Express 1.3a specification

• UnH Certified IP

• Supports Multi-Port Controller for Multi-path IO Support

• Supports configurable number of IO Queues

• Supports configurable Queue depth • Supports Round Robin or Weighted Round

Robin with Urgent Priority arbitration mechanism

• Host memory page size support of 128MB

• Efficient and Streamlined Command handling

• Supports Fused Operations

• Supports All Optional Admin Commands

Value Added Features

• Supports All Optional NVM Commands

• Supports Multi-Path IO and Namespace Sharing capabilities

• Supports Reservations

• Supports multiple name spaces

• Optional AXI interfaces for NVMe implementation in SoC

• Well defined Command Interface for local CPU to perform subsystem initialization and to handle all non-hardware accelerated commands

Copyright Mobiveil, Inc.

Enterprise Flash Controller

Enterprise Flash Controller

17

Cmd Controller

Command buffer

Write buffer

Read buffer

Status accumulator

SEQUENCER

Program memoryRegisters block

APB i/f

ONFI PHY i/f

CMD i/f

Read i/f

Status i/f

Write i/f

• Supports NAND interface standards such as ONFI and Toggle

• Supports LUN based CMD arbitration to maximize parallel execution

• Supports suspend/resume commands per LUN for executing software overridden priority IO commands.

• Programmable command sequences

• Supports up to 64 LUNs per channel

• Supports following modes• 1:1 (Full-rate Mode)• 1:2 (Half-rate Mode)• 1:4 (Quarter rate Mode)

• Supports data path width upto 256 bits on user interface

• Supports software based programming of commands

• Supports programming custom command definitions

• Supports randomization requirements

• Supports integration of per channel ECC.

Copyright Mobiveil, Inc.

UMMC Memory Controller

UMMC Memory Controller

19

Configuration & Status Registers (CSR)

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_0

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_1

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_M

PortArbitration

(QoS)

WRITEDATA

MUX

READDATA

DEMUX

Bank CTRL N

WRITE DATA PATH

READ DATA PATH

Bank CTRL 2

Bank CTRL 1

Bank CTRL 0

RE

C

S

SYS

IF

SYS

IF

Multi Port Controller (MPC) Base Controller (BC)

D

Q

M

F

I

YS

IF

APB

D

DEMUX

SCH

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_0

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_1

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_M

Compliant to AXI4 and DFI3.1

• Supports all major memory standards – DDR3, DDR4, LPDDR2, LPDDR3, HBM2

• Supports Multiport Configuration

• Supports QoS through various arbitration schemes

• Configurable and programmable address mapping

• Supports up to 4 ranks

• Supports following BC Clock to PHY Clock ratio• 1:1 (Full-rate Mode)• 1:2 (Half-rate Mode)• 1:4 (Quarter rate Mode)

• Supports Burst Length 4, 8, 16

• Supports Active/Precharge Power down

• Supports ECC Checking and Correction

• Supports Inline ECC

Copyright Mobiveil, Inc.

Mobiveil PCI Express Gen4 NVMe SSDC

Platform

Mobiveil PCI Express Gen4 NVMe SSDC Platform

On Chip Control Path

On Chip Data Path

PCIe

PHYPCIe

DDR

Flash

ARM Core

DDR4 controller

EFC-0

EFC-1

EFC-2

EFC-n

L

D

P

C

Media DMA

Media DMA

Media DMA

Media DMA

MCC

ONFI

/Tog

gle

PHY

DDR

PHYGPIO

Device FW

NVMe Controller

Command

Fetch &

Completion

Update

Inter

rupt

Logi

c

NVMe

Contr

oller

Regis

tersAdmin & IO

Queue_0

Arbiter

and

Comma

nd

Decoder

FW RQ

DMA

Data

Buffer

DMA

Engin

e 1

DMA

Engin

e M

DMA

NVMe

Contr

oller

Regis

ters Admin & IO

Queue_1

Arbiter

and

Comma

nd

Decoder

FW CQ

NVMe

Contr

oller

Regis

ters Admin & IO

Queue_N

Arbiter

and

Comma

nd

Decoder

DMA CQ

M

A

C

D

L

L

T

L

PCIe Controller

Configuration & Status Registers (CSR)

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_0

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_1

WR REQ Queue

RD REQ Queue

WR DATA BUF

RD DATA BUF

Port Queue_M

PortArbitration

(QoS)

WRITEDATA

MUX

READDATA

DEMUX

Bank CTRL N

WRITE DATA PATH

READ DATA PATH

Bank CTRL 2

Bank CTRL 1

Bank CTRL 0

RE

C

S

SYS

IF

SYS

IF

Multi Port Controller (MPC) Base Controller (BC)

D

Q

M

F

I

YS

IF

APB

D

DEMUX

SCH

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_0

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_1

SYS RD REQ

SYS WR REQ

SYS RD DP

SYS WR DP

SIC_M

Mobiveil PCIe Gen4 Configurable NVMe SSDC Platform

22Copyright Mobiveil, Inc.

Unique Subsystem Development Solution

23

• Provides Full NVMe Based Reference Design Using Mobiveil’s Controllers

• PCIe Gen4.0 PCIe Controller (GPEX)

• Multiport NVMe (UNEX)

• LDPC

• Enterprise Flash Controller (EFC)

• UMMC

• Media Control Cluster

• Reference FW is also provided

• Allows various Flash parts to be used

• Customer can add their custom value add in SW or HW

Copyright Mobiveil, Inc.

The Mobiveil TeamVision

Provide Technology, Platform and value added services to developstorage solutions

▪ RTL IP

▪ FPGA Platform

▪ Engineering services for Custom Designs

Leadership

Management with 30+ years experience in Semiconductor/ Silicon IP/Product Engineering Services

Team working together developing Silicon IP & Engineering Services for15+ years

Previously founded GDA (1997) developed several IP blocks

6 Patents in Flash Storage and Reliability

Location : Headquarters in Milpitas, Engineering Centers in Chennai,Bangalore and Hyderabad. Total headcount ~180

Mobiveil IP Advantages

Market leading & most exhaustively proven cores in the market: Industry leaders are using these cores

Consortium Participation : RIO – Member, PCISIG – Member, HMC - Member

Superior Technical Solution: Most Feature rich IP, Complete Customization and delivery Solution

Support: Clear IP Focus & Worldwide Support

3rd Party Partnerships for complete Solution: (Verification and PHY IPs)

Standard Body Certified Cores: All Mobiveil IPs are validated and certified: PCI Plug fest, UNH, RTA

MILPITAS : 890, Hillview Court, Suite 250, Milpitas, CA 95035

Ph: 408-212-9512 | Fax: 408-457-0406

INDIA: EMM YES PARK, 3rd Floor, #69 (Old #137), Jawaharlal Nehru Road,

Ekkattuthangal, Chennai 600032, Ph: +91-44-4208 6803, +91-44-4208 6804

#60 Adharsh Regent, 2nd Floor, Near Domlur Flyover, Koramangala Inner Ring road

Bangalore 560071, Ph: +91-80-4208 7666

#302, Second Floor, KTC Illumination, Gaffoor Nagar,

Madhapur, Hyderabad, Telangana 500081, Ph: +91-40-4854 3559

www.mobiveil.com | amit@mobiveil.com