CEDARCounter-Estimation Decoupling for
Approximate Rates
Erez Tsidon (Technion, Israel)
Joint work with Iddo Hanniel and Isaac Keslassy
(Technion)
1
Network Flow Counters Usage
Network management applications require per-flow counters, for example: Congestion Control Detection of Denial of Service Attacks Detection of Traffic Anomalies
Counter types: Packet counting Byte counting Rate measurement
2
Switch Example
40MB memory 32nsec for single counter access
DRAM is too slowSRAM is too expensive
3
SWITCH106 flows
Total Packet CountTotal Byte Count
Packet Rate
Count event A
Count event B
per-flow counters
64-bit width
Min packet size = 40B
Link Rate 10Gbps
Suggested Solutions
Hybrid SRAM-DRAM counters [Shah, Iyer, Prabhakar and McKeown ’02] Cannot support fast reading
Counter Braids – compress counters into small SRAM [Y. Lu et al ’08] Cannot decompress in real time
Heavy Hitters – store only high counters [Estan and Varghese ’03] No records of small counter values
4
Counter Estimation Solutions
Intuitively we want counters to be as precise as possible, unbiased whenever possible, and scalable
SAC – R. Stanojevic, “Small Active Counters”, 2007 Exponent-Magnitude representation Unbiased estimation Scalable
Restricted to specific representation which prevents error optimization
DISCO – C. Hu et al, “DISCO: Memory Efficient and Accurate Flow Statistics for Network Measurement”, 2010 Convex conversion function that reduces increment values Unbiased estimation
Restricted to a close function representation. No scaling
5
Our Contributions
CEDAR – decoupling counters from estimators
Optimal estimators for the min-max relative error
Dynamic up-scale algorithm Exponential-averaged rate estimation
6
Counter-Estimators Decoupling
7
995,784
1.2
1,000,000
1.2
Counter estimates
FN-1
FN-2
F1
F0
1,000,000
995,784
1.2
0
p(L-2)
p(1)
p(L-1)
p(1)
AL-1
AL-2
A1
A0
3.7 A2
Flow pointers
Shared estimators
log2 L
q
FN-1
FN-2
F1
F0
CEDAR Structure
N flows L estimators Flow array:
Estimation array:
Estimator differences:
,0jF j N
0,0 , 0iA i L A
1 , 1i i i iD A A D 8
1,000,000
995,784
1.2
0
p(L-2)p(1)
p(L-1)p(1)
AL-1
AL-2
A1
A0
3.7 A2
Flow pointers
Shared estimators
log2 L
q
FN-1
FN-2
F1
F0
01
4 4
CEDAR Increment Algorithm
941
A3
A2
A1
0A0
21113254.711
A7
A6
A5
A4
9410
21113254.711
9410
21113254.711
9410
21113254.711
time
p=1
p=1/3p=1/5
t=0 t=1 t=2 t=3
9
Upon packet arrival: with probability1j jF F 1
jF
pD
Performance Measures
Traffic Amount – - a random variable that represents
the number of real counter increments in order for to hit estimator
Relative error – Coefficient of Variation:
( )T a
X̂ a
10
2
( ) ( )( )
( ) ( )
Var T a T aT a
E T a E T a
Min-Max Relative Error
Problem: assuming L and AL-1 are given, find an estimation array that minimizes the guaranteed relative error δ such that
Solution: equal relative error
Estimation values
2
( ), ( )
( )l
ll
Var T Al T A
E T A
11
12
01 2
1 2
1
l
ii
l l
DA A
, ( )ll T A
Equal Relative Error Example
12
Estimation Values
Relative Error
δ
δ
A1 A2 A3
A1 A2 A3
δ
A1 A2 A3
Capacity Region of Static CEDAR
13
Example:• 10-bit counters• Max value 10^6min-max relative error 8%
4.5
1
Up-Scale Procedure
3
1
A3
A2
A1
0A0
211
132
54
11
A7
A6
A5
A4
24
5
2
0
517
314
156
93
54
11
p=0.5 0
p=0.43=(54-24)/(93-24)
93
14
A’ A’’
upscalethreshold
Initial relative error
δ0
Increase the relative error
δ0+ δstep
CEDAR Unbiasedness
15
CEDAR Equal Error
16
CEDAR Vs. SAC & DISCO 12-bit
17
4096 estima
tors
CEDAR Vs. SAC & DISCO 8-bit
18
256estima
tors
CEDAR Error Adjustment 12-bit
19
CEDAR Implementation on FPGA
20
5.4 Gbps
12K gates
Exponential-Average Rate Estimation
A3
A2
A1
A0
A7
A6
A5
A4
time
x0.98 down-scale
33.316.60
99.983.366.649.9
119.5
5134170
1221028568
0
P = 0.02x100/(85-68)
Incoming packet
513417
1221028568
t0 t0 + 1 t0 + 2
x0.98 down-scale
32.616.30
97.981.665.348.9
117.1
6885 83.3 81.683.383.3
21
Exponential-Average Rate Estimation
A3
A2
A1
A0
A7
A6
A5
A4
time
t0 + 10 t0 + 2
32.616.30
97.981.665.348.9
117.1
41.627.7
0
99.683.369.455.5
13.8
up-scaling
0
513417
1221028568P =
(85-69.4)/(85-68)
After 8 more down-scaling cycles
81.6 69.468
99.6
22
EXP-CEDAR 7-bit
23
Real exp-average with 64 bits
For 10^(-2) precision – 14 bits are required
EXP-CEDAR 9-bit
24
CEDAR Summary
Decoupling - flexible estimators Unbiased and scalable estimation Attains the min-max relative error FPGA supports link rate of 5.4Gbps and may
increase to tens of Gbps on ASIC Exponential-average rate estimation
25
Thank you.
26