Efficient Multi-match Packet Classification with TCAM
Fang Yu Randy H. KatzEECS Department, UC Berkeley{fyu, randy}@eecs.berkeley.edu
Outline
New applications demand multi-match classification
Multi-Match classification using TCAMOrder rules in TCAM Remove negations
Simulations results Conclusions
Today’s Packet Classification Systems
A classifier consists of N rules, each with F fields Next hop routing using destination IP (F=1) Filters from firewall (F=5)
Single-Match Classification: Assumption: all the rules are associated with priorities Only the highest priority match matters E.g., longest prefix match
Source IP Destination IP Source Port Destination Port Protocol Action Priority
128.59.67.100 128.* * 15 Tcp drop 2
128.* 128.2.3.1 * 25 tcp allow 1
New Applications
Intrusion Detection Systems (e.g., SNORT) Rule header: a 5-field
classification rule for the packet header
Rule options: specify intrusion patterns for the entire packet scanning.
udp $EXTERNAL_NET any-> $HOME_NET 1434
content:"|04|"; depth:1;content:"|81 F1 03 01 04
9B 81 F1 01|";content:"sock";content:"send"
udp $EXTERNAL_NET any -> $HOME_NET any
content:"|00 01 86 A9|";offset:12; depth:4;
content:"|00 00 00 01|";distance:4; within:4;
byte_jump:4,4,relative,align;byte_jump:4,4,relative,align;byte_test:4,>,64,0,relative;
content:"|00 00 00 00|";offset:4; depth:4; sid:2027;
rev:4;
A rule for MS-SQLWorm detection.
A rule for RPC oldpassword overflow attempt
Packet header
Match
A packet may be related to multiple rules (matching rule headers)
Multi-Match Classification: Identify all the matching rule headers
Packet Payload
Scan
In some edge networks
Each box introduces extra delay Common functions like classification are repeatedly applied Highly inefficient!
Programmable Network Element Support multiple functions in one device Each packet may related to different set of functions
E.g., HTTP packets related to firewall and HTTP load balancer E.g., VPN packets related to encryption / decryption
Multi- Match Classification: identify the all the relevant functions
New Applications (cont.)
Multi-Match v.s. Single-Match Classification A classifier consists of N rules, each with F fields
Single-Match: Report the highest priority rule Multi-Match: Report all the matching rules
Single-match classification Software solutions: O(logN) query time with O(NF) storage Real-world rule sets are simpler than theoretical worst case
State of art heuristic algorithms: 20-30 memory accesses Multi-Match classification
More complex than single-match Complex follow-up processing Tighter time requirements
20-30 memory accesses slow Can hardware solution help?
Ternary-CAM (TCAM)
Fully associative memory: compares input string with all the entries in parallel If multiple matches, report index of
the first match Each cell takes one of three logic
states ‘0’, ‘1’, and ‘X’(don’t care)
Current TCAM technology Fast Match Time: 4 ns Size: 1-2MB Commercially used for single-match
classification
k bytes
> 1K
entries
192.128.101.100
168.100.xxx.xxx
192.128.xxx.xxx
Match192.128.101.xxx
Input
TCAM
Use TCAM for Multi-Match Classification Problem: TCAM only reports the
first matching result Return a bit vector of matched
results? Not a good solution because
processing cost for bit vector is O(N) Solution: Add additional
intersection rules Upper bound of intersections O(NF) Real-world rule set has far less
intersections Retrieve all matching results solely
based on the first matched result--- TCAM Compatible Order
tcp $SQL_SERVER 1433$EXTERNAL_NET 139
tcp any any any 139
Match
tcp $SQL_SERVER 1433$EXTERNAL_NET any
Input
TCAMStores Rules
Rule 1
Rule 2
SRAMStores Match list
(Index of rule)
1
2
3
n
1st entry
nth entry
tcp $SQL_SERVER 1433$EXTERNAL_NET 139
Rule 1,2
TCAM Compatible Order
Relationship between rules Ei and Ej , with corresponding matched list Mi and Mj
Exclusive (Ei Ej= ): i and j can have any order.
Subset (Ei Ej): i<j and Mi Mj .
Superset (Ei Ej): j<i and Mi Mj .
Intersection (Ei Ej= ):
add a rule El =(Ei Ej) , (l<i, l<j), (Mi Mj ) Ml.
Pseudo-Code for Generating TCAM Compatible Order Extend_rule_set(R){
E = ;for all the rule Ri in R
E=Insert(Ri, E);return E;
}
Insert(x, E){ for all the rule Ei in E { Switch the relationship between Ei and x: Case exclusive:
continue; Case subset:
Mi = Mx Mi; continue; Case superset:
Mx = Mx Mi; add x before Ei ;
return E; Case intersection:
If (Ei x E and M x Mi) add t = Ei x before Ei ;
Mt = Mx Mi } add x at the end of E and return E;}
Example
Original rule set
Extended rule set in TCAM compatible order
1 Tcp $SQL_SERVER 1433 $EXTERNAL_NET any
2 Tcp $EXTERNAL_NET 119 $HOME_NET Any
3 Tcp Any Any Any 139
Extended rules Matched List
Tcp $SQL_SERVER 1443 $EXTERNAL_NET 139 1,3
Tcp $SQL_SERVER 1433 $EXTERNAL_NET any 1
Tcp $EXTERNAL_NET 119 $HOME_NET 139 2,3
Tcp $EXTERNAL_NET 119 $HOME_NET any 2
Tcp any any any 139 3
$EXTERNAL_NET
$EXTERNAL_NET
$EXTERNAL_NET
$EXTERNAL_NET
$EXTERNAL_NET=!$HOME_NET
Representing Negation with TCAM
80’s binary form 0000 0000 0101 0000 Negation of 80 (!80=[0,79], [81,2^16-1])
0000 0000 0101 0000 =
1111 1111 1010 1111 = 65375 is only a subset of !80
Need 16 TCAM entries
Multiple negations in one rule tcp $EXTERNAL_NET any $EXTERNAL_NET !
80
requires up to 32*32*16=16384 TCAM entries
1xxx xxxx xxxx xxxx
x1xx xxxx xxxx xxxx
xx1x xxxx xxxx xxxx
xxx1 xxxx xxxx xxxx
xxxx 1xxx xxxx xxxx
xxxx x1xx xxxx xxxx
xxxx xx1x xxxx xxxx
xxxx xxx1 xxxx xxxx
xxxx xxxx 0xxx xxxx
xxxx xxxx x1xx xxxx
xxxx xxxx xx0x xxxx
xxxx xxxx xxx1 xxxx
xxxx xxxx xxxx 1xxx
xxxx xxxx xxxx x1xx
xxxx xxxx xxxx xx1x
xxxx xxxx xxxx xxx1
Remove Negation
Regions generating negation: A, B, D
Regions with no negation C, A C, C D,
A B C D
1 Tcp $SQL_SERVER 1433 $EXTERNAL_NET any
2 Tcp $EXTERNAL_NET 119 $HOME_NET Any
3 Tcp Any Any Any 139
C
A
D
B
Home Net
Home Net External Net
External Net
Rule 3
Rule 2, 3
Rule 1, 3
Rule 3
Source IP
Destination IP
Remove Negation
Can we extend rules in D to D C? Yes, We can!
With a first match TCAM
1 Tcp $SQL_SERVER 1433 $EXTERNAL_NET any
2 Tcp $EXTERNAL_NET 119 $HOME_NET Any
3 Tcp Any Any Any 139
C
A
D
B
Home Net
Home Net External Net
External Net
Rule 3
Rule 2, 3
Rule 1, 3
Rule 3
Source IP
Destination IP
TCAM entries Matched List
tcp $HOME_NET any $HOME_NET 139
3
any $HOME_NET any $HOME_NET any
Tcp $SQL_SERVER 1443 any 139
1,3
Tcp $SQL_SERVER 1433 any any
1
Removing Negation Rules in region C: “* $HOME_NET+ * $HOME_NET+ *” Separator rule 1: “any $HOME_NET any $HOME_NET any” Rules in region D,
specified in the form of region C and D: “* $HOME_NET+ * any *”
Rules in region A, specified in the form of region A and C: “* any * $HOME_NET+ *”
Separator rule 2: “any $HOME_NET any any any”
Separator rule 3: “any any any $HOME_NET any”
Rules applying to region B, specified in the form of region A, B, C and D: “* any * any *”
C
A
D
B
Home Net
Home Net External Net
External Net
Rule 3
Rule 2, 3
Rule 1, 3
Rule 3
Source IP
Destination IP
Extended rules Matched List TCAM entries needed
Tcp $SQL_SERVER 1443 $EXTERNAL_NET 139 1,3 32
Tcp $SQL_SERVER 1433 $EXTERNAL_NET any 1 32
Tcp $EXTERNAL_NET 119 $HOME_NET 139 2,3 32
Tcp $EXTERNAL_NET 119 $HOME_NET any 2 32
Tcp any any any 139 3 1
TCAM
Index
TCAM entries Matched List
1 tcp $HOME_NET any $HOME_NET 139 3
2 any $HOME_NET any $HOME_NET any
3 Tcp $SQL_SERVER 1443 any 139 1,3
4 Tcp $SQL_SERVER 1433 any any 1
5 Tcp any 119 $HOME_NET 139 2,3
6 Tcp any 119 $HOME_NET any 2
7 Tcp any any any 139 3
94.5% of TCAM entries saving
Analysis of Negation Removing Scheme More than one negations in each field
Both !80 and !90 in the source port field !subnet1 and !subnet2 in the destination IP field
Generation of algorithm For one field Fi,
Ki unique negations with disjoint non-negation forms Si =Ki separator rules
Ki unique negations with intersected non-negation forms Si =2Ki
separator rules Total separator rules:
removing $EXTERNAL_NET from source and destination IP addresses, S1= S2=1 a total of 3 separator rules
1-1))(S( i
Simulation Results
SNORT intrusion detection rule set
Version Rule SetSize
# of rulesin extended set
Singlenegation
Double negations
Triplenegations
2.0.0 240 3,693 62.334% 0.975% 0
2.0.1 255 4,009 62.484% 1.422% 0.025%
2.1.0 257 4,015 62.540% 1.420% 0.025%
2.1.1 263 4,330 62.332% 1.363% 0.023%
Performance of Negation Removing Scheme Snort
versionWith Negation Negation Removed TCAM
SpacesavedExtended
rule setsize
TCAM Entriesneeded
Extendedrule set
size
TCAMEntriesneeded
2.0.0 3,693 120,409 4,101 7,853 93.4%
2.0.1 4,009 145,208 4,411 8,124 94.4%
2.1.0 4,015 145,352 4,420 8,133 94.4%
2.1.1 4,330 151,923 4,797 8,649 94.3%
Fit all Snort rule headers into a 256KB TCAM Retrieve multi-match classification result with one TCAM lookup and
one SRAM lookup (<10ns)
Effect of Negation
Performance of Negation Removing Scheme
0
20000
40000
60000
80000
100000
00.
080.
160.
240.
32 0.4
0.48
0.56
0.64
0.72 0.
80.
880.
96
% of single Negation
TC
AM
En
trie
s n
eed
ed
With NegationNegation Removed
Conclusions
New applications demand for multi-mach classification
TCAM-based solution to solve the multi-match classification problem Report all the matching results with a single
TCAM lookup and a SRAM lookup Negation removing scheme can save 93%
to 95% of the TCAM space