1

An Open Source Hardware Module for High-Speed Network Monitoring on

NetFPGA

NetFPGA European Developers Workshop 2010

Gianni Antichi, Stefano GiordanoEmail: <first.last>@iet.unipi.it

Department of Information EngineeringUniversity of Pisa

David J. MillerEmail: <first.last>@cl.cam.ac.uk

Computer LaboratoryUniversity of Cambridge

2

Outline

• Introduction

• Motivations

• Our solution Hardware plane Software plane

• Results

• Future work

3

Introduction

We present a passive network measurement solution based on the low-cost NetFPGA — suitable for network research,

security applications, and traffic engineering and management.

Network measurement and monitoring has been anactive area of research for at least the past 15 years.

Applications include academic research, security, and traffic engineering and management.

Key features:– Accurate timestamping.– Ability to filter traffic based on flow.

4

Motivations

• The ideal measurement and monitoring solution:– Accurate.– Guarantee no loss of information (or at least records exactly where records have

been lost).– Inexpensive.

• Software solutions:– Cheap.– Work well for low-speed networks or when timestamp accuracy is not too important.– Don’t scale to high speed networks.

• Hardware solutions:– Very accurate.– Expensive.

5

Motivations

We use NetFPGA platform, which is open and low-cost, to achieve the performance of hardware-based solutions but at

costs closer to that of software-only solutions.

In-series or In-parallel monitoring?

• In-parallel with the link to be monitored:– Copper network links require an expensive active tap.– Passive optical splitters are inexpensive.

• In-line with the link to be monitored:– Cheap.– Offers possibility of building an Intrusion Prevention System.– Extra latency.– Risk of interruption of the link.

6

Our Solution (Hardware Plane)

Our monitor system adds two new modules to the standard NetFPGA pipeline.

7

Our Solution (Hardware Plane)Timestamping module:

• Attaches to the RGMII as near as possible to the MAC.• The RGMII asserts its “data valid” signal when the SFD of a frame is

received at a physical interface.• We sample the free-running timestamp counter on the low-to-high

transition of the “data valid” signal.• Timestamps are sampled from a 64-bit, free-running counter driven

by the 125 MHz system clock, which increment by 8 once every 8 ns.• The timestamp counter can be reset.

8

Our Solution (Hardware Plane)

Filtering module:

• All packets received are retransmitted.• Packets that match one of up to 32 filter rules are also copied

verbatim, with their timestamp prepended, to the host.• We use the TCAM modules available in Xilinx CoreGen.• TCAMs are fast and permit on-the-fly rule updates.• Owing to problems with timing closure, we found it necessary to

implement the filter using 16-entry TCAMs, rather than one 32-entry TCAM.

9

Our Solution (Hardware Plane)

Pipeline:

• Timestamps pass in a side channel parallel with the main packet data path.

10

Our Solution (Software Plane)

• Auxiliary command line tool for TCAM rule management • Initialisation of the hardware timestamp• Libpcap-based capture programme:

– Converts and remove the hardware timestamp.– Overwrite the PCAP timestamp.– Record a standard PCAP trace.

11

Results

seconds

seconds

• X axis: DAG behaviour.• Y axis: NetFPGA behaviour.• Comparison of the two absolute drift (1000 samples).

12

Results

packets

milliseconds

• Relative drift: Comparison between the two oscillator.• We lose 1.7 ms in 53.7 seconds (32 ppm).

13

Future Work

We present a flexible and cheap passive NetFPGA-based monitoring system.

• Use of Direct Digital Synthesis, together with an external time-base to provide error-corrected timestamps in a convenient format.

• Re-implementation of the flow filter using Bloom Filters in order to support substantially more than 32 flows.

• Optional in-band markers for packets belonging to unmatched flows.• Refactor to include timestamp in a module-header.• Live libpcap support with extended precision timestamps• Endace ERF format and libtrace support.

14

Demo!

15

Nf-test13 Nf-test12

from 12.eth2 to 13.nf2c0