cedar counter-estimation decoupling for approximate rates erez tsidon (technion, israel) joint work...

CEDARCounter-Estimation Decoupling for

Approximate Rates

Erez Tsidon (Technion, Israel)

Joint work with Iddo Hanniel and Isaac Keslassy

(Technion)

1

Network Flow Counters Usage

Network management applications require per-flow counters, for example: Congestion Control Detection of Denial of Service Attacks Detection of Traffic Anomalies

Counter types: Packet counting Byte counting Rate measurement

2

Switch Example

40MB memory 32nsec for single counter access

DRAM is too slowSRAM is too expensive

3

SWITCH106 flows

Total Packet CountTotal Byte Count

Packet Rate

Count event A

Count event B

per-flow counters

64-bit width

Min packet size = 40B

Link Rate 10Gbps

Suggested Solutions

Hybrid SRAM-DRAM counters [Shah, Iyer, Prabhakar and McKeown ’02] Cannot support fast reading

Counter Braids – compress counters into small SRAM [Y. Lu et al ’08] Cannot decompress in real time

Heavy Hitters – store only high counters [Estan and Varghese ’03] No records of small counter values

4

Counter Estimation Solutions

Intuitively we want counters to be as precise as possible, unbiased whenever possible, and scalable

SAC – R. Stanojevic, “Small Active Counters”, 2007 Exponent-Magnitude representation Unbiased estimation Scalable

Restricted to specific representation which prevents error optimization

DISCO – C. Hu et al, “DISCO: Memory Efficient and Accurate Flow Statistics for Network Measurement”, 2010 Convex conversion function that reduces increment values Unbiased estimation

Restricted to a close function representation. No scaling

5

Our Contributions

CEDAR – decoupling counters from estimators

Optimal estimators for the min-max relative error

Dynamic up-scale algorithm Exponential-averaged rate estimation

6

Counter-Estimators Decoupling

7

995,784

1.2

1,000,000

1.2

Counter estimates

FN-1

FN-2

F1

F0

1,000,000

995,784

1.2

0

p(L-2)

p(1)

p(L-1)

p(1)

AL-1

AL-2

A1

A0

3.7 A2

Flow pointers

Shared estimators

log2 L

q

FN-1

FN-2

F1

F0

CEDAR Structure

N flows L estimators Flow array:

Estimation array:

Estimator differences:

,0jF j N

0,0 , 0iA i L A

1 , 1i i i iD A A D 8

1,000,000

995,784

1.2

0

p(L-2)p(1)

p(L-1)p(1)

AL-1

AL-2

A1

A0

3.7 A2

Flow pointers

Shared estimators

log2 L

q

FN-1

FN-2

F1

F0

01

4 4

CEDAR Increment Algorithm

941

A3

A2

A1

0A0

21113254.711

A7

A6

A5

A4

9410

21113254.711

9410

21113254.711

9410

21113254.711

time

p=1

p=1/3p=1/5

t=0 t=1 t=2 t=3

9

Upon packet arrival: with probability1j jF F 1

jF

pD

Performance Measures

Traffic Amount – - a random variable that represents

the number of real counter increments in order for to hit estimator

Relative error – Coefficient of Variation:

( )T a

X̂ a

10

2

( ) ( )( )

( ) ( )

Var T a T aT a

E T a E T a

Min-Max Relative Error

Problem: assuming L and AL-1 are given, find an estimation array that minimizes the guaranteed relative error δ such that

Solution: equal relative error

Estimation values

2

( ), ( )

( )l

ll

Var T Al T A

E T A

11

12

01 2

1 2

1

l

ii

l l

DA A

, ( )ll T A

Equal Relative Error Example

12

Estimation Values

Relative Error

δ

δ

A1 A2 A3

A1 A2 A3

δ

A1 A2 A3

Capacity Region of Static CEDAR

13

Example:• 10-bit counters• Max value 10^6min-max relative error 8%

4.5

1

Up-Scale Procedure

3

1

A3

A2

A1

0A0

211

132

54

11

A7

A6

A5

A4

24

5

2

0

517

314

156

93

54

11

p=0.5 0

p=0.43=(54-24)/(93-24)

93

14

A’ A’’

upscalethreshold

Initial relative error

δ0

Increase the relative error

δ0+ δstep

CEDAR Unbiasedness

15

CEDAR Equal Error

16

CEDAR Vs. SAC & DISCO 12-bit

17

4096 estima

tors

CEDAR Vs. SAC & DISCO 8-bit

18

256estima

tors

CEDAR Error Adjustment 12-bit

19

CEDAR Implementation on FPGA

20

5.4 Gbps

12K gates

Exponential-Average Rate Estimation

A3

A2

A1

A0

A7

A6

A5

A4

time

x0.98 down-scale

33.316.60

99.983.366.649.9

119.5

5134170

1221028568

0

P = 0.02x100/(85-68)

Incoming packet

513417

1221028568

t0 t0 + 1 t0 + 2

x0.98 down-scale

32.616.30

97.981.665.348.9

117.1

6885 83.3 81.683.383.3

21

Exponential-Average Rate Estimation

A3

A2

A1

A0

A7

A6

A5

A4

time

t0 + 10 t0 + 2

32.616.30

97.981.665.348.9

117.1

41.627.7

0

99.683.369.455.5

13.8

up-scaling

0

513417

1221028568P =

(85-69.4)/(85-68)

After 8 more down-scaling cycles

81.6 69.468

99.6

22

EXP-CEDAR 7-bit

23

Real exp-average with 64 bits

For 10^(-2) precision – 14 bits are required

EXP-CEDAR 9-bit

24

CEDAR Summary

Decoupling - flexible estimators Unbiased and scalable estimation Attains the min-max relative error FPGA supports link rate of 5.4Gbps and may

increase to tens of Gbps on ASIC Exponential-average rate estimation

25

Thank you.

26