applied research laboratoryapplied research laboratoryDavid E. Taylor
Users Guide:Fast IP Lookup (FIPL) in the FPX
Gigabit Kits Workshop 1/2002
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL System Design• Each FIPL Engine performs a longest matching prefix
lookup on a single 32-bit IPv4 destination address• FIPL Engine Controller scales to required lookup
throughput with minimal hardware resource usage– Instantiate required number of parallel lookup engines– 4 engines in current configuration (2.4 Gb/s link)– Pipeline memory accesses
Packet I/O
PP
Switch Fabric
Physical Links
CP
TI
PP
TI TI
FIPL Engine
FIPL Engine Controller
FIPL Engine
FIPL Wrapper
ProcessorControl
SRAM Interface
Packet I/O
• FIPL Wrapper– Buffers packets– Supports up to 4
virtual ports• Control Processor
– Handles data structure updates
applied research laboratoryapplied research laboratoryDavid E. Taylor
Design OverviewSRAM1
SRAM2
IP LookupEngine
counter
On-Chip Cell Store
SRAM1 Interface
Control CellProcessor
PacketReassembler
RAD FPGA
NID FPGA
ExtractIP Headers
Remap VCIsfor IP packets
LC SW
RequestGrant0 1
0
0 0
0
0
0
1 1
1
1 1
1
1
1
1
applied research laboratoryapplied research laboratoryDavid E. Taylor
Performance Evaluation• Used gate-level simulation with ModelSim
– 100 MHz system clock • Configured a FIPL Engine Controller to enable one to
eight FIPL engines based on the contents of a control cell• Initialized tree bitmap data structure with 16,564 entries
from the Mae-West routing table (July 12, 2001 snapshot)
• Measured lookup latency and throughput for test sequences of 2048 random destination addresses– Addresses stored in on-chip memory read by FIPL Engine
Controller• Measured lookup latency and throughput for various
update loads
applied research laboratoryapplied research laboratoryDavid E. Taylor
Throughput and latency performance
0
1
2
3
4
5
6
7
8
9
10
11
1 2 3 4 5 6 7 8# of FIPL engines
Mill
ion
s of
look
up
s p
er s
econ
d
0
100
200
300
400
500
600
700
800
900
1000
1100
Ave
rage
Loo
ku
p L
aten
cy (
ns)
Theoretical Worst-case ThroughputMae West ThroughputTheoretical Worst-case Avg. Lookup LatencyMae West Avg. Lookup Latency
applied research laboratoryapplied research laboratoryDavid E. Taylor
Update performance
0
1
2
3
4
5
6
7
8
9
10
11
1 2 3 4 5 6 7 8# of FIPL engines
Mill
ion
s of
look
up
s p
er s
econ
d
1,000 updates per second10,000 updates per second100,000 updates per secondNo updates
applied research laboratoryapplied research laboratoryDavid E. Taylor
Performance on WU Research Platform• Based on results, a 4 engine configuration was targeted to
the WUGS/FPX research platform– Sustained 1.988 Gb/s throughput on single-cell packets = 4.7 M
packets/sec Limited by 2 Gb/s switch interface of FPX (32-bit at 62.5 MHz) Verified using bandwidth monitoring software, the cell multiplying
feature of the WUGS, and four traffic sources sending at different rates with corresponding 24-bit prefix entries in the route table
– Utilizes only 8% of available logic resources and 12.5% of on-chip memory resources
4 FIPL Engines and FIPL Engine Controller utilizes 6% of logic resources
FIPL Wrapper utilizes 2% of logic resources and 12.5% of on-chip memory resources
applied research laboratoryapplied research laboratoryDavid E. Taylor
Current Work: MSR Integration
Control PathData Path
SRAMUpdates
DQ Status &Rate Control
Register SetUpdates & Status
CCP
SRAMSRAM Register Set
FIPL Q-MgrMgmtFilters
ISAR OSARPacket Storage Manager(includes free space list)
SDRAM SDRAM
Pkt-ptrShimHeader
Hdr updateRef. counterDiscard pkt.
LC
SW
LC
SW
AAL0
O-SW
AAL5
applied research laboratoryapplied research laboratoryDavid E. Taylor
Default FIPL Configuration• Current FIPL Wrapper configured for future MSR
integration (all parameters modifiable via control cell)• Listens for IP traffic on 4 sub-ports (SP0 – SP3)
– Sub-port VCI determined by an input base VCI (Ibase_VCI) and a sub-port index (SPI)
Sub-port VCI = Ibase_VCI + SPI
– Defaults: Ibase_VCI = 0x80 (128) SP0 = 0, SP1 = 1, SP2 = 2, SP3 = 3 SP0_VCI = 0x80 (128), SP1_VCI = 0x81 (129), …
• Similar operation for outgoing VC resolution– For current use, explicitly specify outgoing VCI as Next Hop
applied research laboratoryapplied research laboratoryDavid E. Taylor
References• Scalable IP Lookup for Programmable Routers , David E. Taylor, John W.
Lockwood, Todd Sproull, Jonathan S. Turner, David B. Parlour, WUCS-01-33, 10/01.
• Generalized RAD Module Infrastructure of the Field Programmable Port Extender (FPX) Version 2.0 , David E. Taylor, John W. Lockwood, Naji Naufel, WUCS-TM-01-16, 7/01.
• Generalized RAD Module Interface Specification of the Field Programmable Port Extender (FPX) Version 2.0 , David E. Taylor, John W. Lockwood, Sarang Dharmapurikar, WUCS-TM-01-15, 7/01.
• FPX Website: www.arl.wustl.edu/arl/projects/fpx
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL Switch Initialization
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL Switch Initialization• Switch Configuration -> GBNSC Restart• Switch Configuration -> Switch Reset• Switch Configuration -> Configure all VCIs• Switch Configuration -> Set ALL ports to Hardware Mode• Switch Configuration -> Configure a Unidirectional VC
– Incoming Port: 3 Incoming VC: 128– Outgoing Port: 2 Outgoing VC: 128
• Switch Configuration -> Configure a Unidirectional VC– Incoming Port: 2 Incoming VC: 154– Outgoing Port: 3 Outgoing VC: 154
• Switch Configuration -> Configure a Unidirectional VC– Incoming Port: 2 Incoming VC: 155– Outgoing Port: 3 Outgoing VC: 155
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Control Software• FPX Applications
– Start Application NCHARGE on all ports
• FPX Applications– Start Application
FIPL Memory Manager
(Port 2, Stack 0)
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Upload bitfile• Download
rad_fipl_msr.v2000e.CCLK.bit from:
http://www.arl.wustl.edu/~det3/talks/
• Save to Desktop• FPX Applications
– Upload a file: Browse to Desktop and select file
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Programming the RAD• Configuration Memory Updates
– Complete Configuration (filename given after upload)
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Adding Routes• Fast IP Lookup -> Route Add
– IP Address: 11.22.33.44 Net Mask: 24 Next Hop: 154
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Sending Test Packets (1)• Create Cells
– IPv4 IP Address:
11.22.33.44 Protocol: 4 TTL: 255 VCI: 128
– Random Data
– Create Cell
– Receive on 154
applied research laboratoryapplied research laboratoryDavid E. Taylor
FIPL – Sending Test Packets (2)
applied research laboratoryapplied research laboratoryDavid E. Taylor
Route Modify and Delete• Modify Route 11.22.33.44/24 155
• Create Cell 11.22.33.44 (transmit on 128)– Receive on 155– Receive on 154 (should timeout, no cell received)
• Add Route 11.22.55.66/16 154– Create cell 11.22.33.44
Should still receive on 155 (not 154)
• Delete Route 11.22.33.44/24– Create cell 11.22.33.44
Should now receive on 154