on scalable attack detection in the network
TRANSCRIPT
Ramana Rao Kompella IMC 2004, Sicily, Italy
On Scalable Attack Detection in theNetwork
Ramana Kompella,Sumeet Singh,
George Varghese
Ramana Rao Kompella IMC 2004, Sicily, Italy
Motivation
ÿ Impact: Attacks such as Worms, DoS => billionsof dollars in damageÿCollateral Damage: Detecting attacks early in
network reduces collateral damageÿScalable Detection: High speed attack detection
in the network requires scalability in stateÿThis Talk: Issues and algorithms for scalable
attack detection using DoS as an example.
Ramana Rao Kompella IMC 2004, Sicily, Italy
Collateral Damage
NetworkCore
Attacker ISP
Attacker 1
AttackerISP
VictimISP
Attacker n
Victim
Good
Least collateral damage
Most collateral damage
Less collateraldamage
Detecting and Defending inthe network reducescollateral damage
Ramana Rao Kompella IMC 2004, Sicily, Italy
Existing Approaches
ÿ End host based approaches (e.g, Syn-cookies) can avoidproblem but‡Deployment: Need to patch all end-nodes‡Network bandwidth: wasted
ÿ Perimeter Approaches (e.g., Riverhead, Michigan) detectand discriminate spoofed sources via TTLs but:‡Zombies: TTL based approaches cannot handle DDoS attacks via
non-spoofed sources such as zombies‡Network Bandwidth: Wasted (upto point of detection)
ÿ Other scalable approaches‡Multistage filters, sketches, MULTOPS‡Traditionally only bandwidth based attacks
Ramana Rao Kompella IMC 2004, Sicily, Italy
Challenges with attack detection inthe network
ÿMain Challenge: Scalability‡Large number of flows, hence per-flow state difficult‡High speed links, hence less processing time
ÿWhy Scalability in State is Important‡High speed memories (SRAM) don’t scale in density as
well as lower speed memory (DRAM)‡Even if available, high speed memory is expensive
How to scalably detect attacks in the network ?
Ramana Rao Kompella IMC 2004, Sicily, Italy
Scalability via Aggregation
ÿAnalogies‡IP addresses are aggregated using prefixes‡Flows are aggregated into DiffServ Classes
ÿKey Insight‡Maintain state across aggregates (instead of per-flow)‡Example : Subnet aggregation, Hash buckets
ÿBut Aggregation introduces issues:‡Behavioral Aliasing
• Good behaviors aggregate to look bad (False Positives)• Bad behaviors might look good (False Negatives)
‡Spoofing : More details in the paper• Attack behavior can be spoofed to look benign
Ramana Rao Kompella IMC 2004, Sicily, Italy
Contributions
ÿFramework: Any scalable attack detectionalgorithm must address behavioral aliasing &spoofing.
ÿConcrete technique: Data structure called PCF(Partial Completion Filter)
ÿEvaluation of PCFs:‡Behavioral Aliasing using Gaussian approximation
‡Spoofability
Ramana Rao Kompella IMC 2004, Sicily, Italy
Attack Classes we Address
ÿPartial Completion Attacks‡TCP SYN Flooding, Naptha attacks
ÿAttacks that use scanning‡Portscans, worm scanning, backdoor probes
ÿMuch existing work on these problems, butlittle focus on scalability.
Ramana Rao Kompella IMC 2004, Sicily, Italy
Naïve Solution
ÿSimple solution‡Maintain state (count of number of SYN packets - count
of number of FIN packets)‡Requires per-flow state‡Not feasible in the network
ÿWe use a Partial Completion Filter to‡Identify a behavior: Identify flows (destinations) that
have a high imbalance in SYN and FIN packets in agiven interval‡Perform Aggregation: We use randomized hashing
(unbiased) to aggregate flows
Ramana Rao Kompella IMC 2004, Sicily, Italy
Partial Completion Filter: Idea
SYN
FIN
SYN
SYN
SYN
FIN
SYN
FIN
SYN
SYN
SYN
Hash Buckets4 1
SYN
AttackStream
BenignTraffic
Subset of Packets mappedTo Bucket 2
Subset of Packets mappedTo Bucket 6
PCF
Ramana Rao Kompella IMC 2004, Sicily, Italy
Behavioral Aliasing: ModelingBenign Traffic
Interval 1 Interval 2 Interval 3 Interval 4
Long Lived Connection
SYNy Retransmissions
FINzRetransmissions
SYNx FINx
ÿ Long-lived connectionsÿ Retransmissionsÿ Route churn
Benign behaviorProb (Packet = SYN) = 0.5Prob (Packet = FIN) = 0.5
Ramana Rao Kompella IMC 2004, Sicily, Italy
Behavioral Aliasing: ModelingBenign behavior
SYN
FIN
SYN
FIN
SYN
Hash Buckets
SYNBenign behaviorProb (Xi = 1) = 0.5Prob (Xi = -1) = 0.5
Mean (Xi) = 0Variance (Xi) = 1
What is the distribution of X = ∑ Xi ?
X2
X3
X4
X5
X1
X6
ModelledRandom Variables
Ramana Rao Kompella IMC 2004, Sicily, Italy
Behavioral Aliasing: GaussianApproximation
Hash Buckets
X = ∑ Xi can be approximatedusing Gaussian
NOISE
-3s 3s
Ramana Rao Kompella IMC 2004, Sicily, Italy
Additive Noise
ÿNoise due to benign connections is additivewith the attack flow sizeÿIf attack is SYN-FIN difference > T, then‡Choose a threshold T-3sigma‡PCF identifies any flow f >T w.h.p (1 - False
Negative Probability)‡PCF identifies a flow f, then f > T-6sigma w.h.p
(1 - False Positive Probability)
Ramana Rao Kompella IMC 2004, Sicily, Italy
Behavioral Aliasing: False Positives
SYN
SYN
SYN
FIN
SYN
SYN
Hash Buckets4
FALSE POSITIVE dueto mapping to attack flows
p = probability that a benign flowmaps to an attack flow’s bucket
pk = probability that a benign flow maps to an attack flow in all k stages
Ramana Rao Kompella IMC 2004, Sicily, Italy
Partial Completion Filter withmultiple stages
Field Extraction
Comparator
Comparator
Comparator
CountersHash 1
Hash 2
Hash 3
Stage 1
Stage 2
Stage 3
ALERT !If
all countersabove
threshold
Increment for SYN, Decrement for FIN
Ramana Rao Kompella IMC 2004, Sicily, Italy
Reducing False Positives Further
ÿAdd multiple stages with independent hashfunctionsÿMore stages reduces false positives exponentially‡A flow is selected only if buckets in all stages greater
than a thresholdÿBut false negatives increases linearly (approx.)ÿSo, choose number of stages so that false
positives reduced drastically while false negativesdon’t increase too much
Ramana Rao Kompella IMC 2004, Sicily, Italy
Measurement on Real Traces
ÿ Traces from two different ISPs A and B (CAIDA)‡ Named as ISPA Dir-0, Dir-1, ISPB Dir-0, Dir-1
ÿTuning the Parameters‡How to choose bucket sizes ?
• We use 5000 buckets
‡How big should the measurement interval be?• Small: fast detection but lacks clear signature to infer an attack.
• Large: increased time of detection
• We use 1 minute as the measurement interval
Ramana Rao Kompella IMC 2004, Sicily, Italy
How does SYN-FIN asymmetryaffect ?
Bias towards SYNscause slight shift inthe noise
Ramana Rao Kompella IMC 2004, Sicily, Italy
How many stages are good enough?
Tradeoff between False Positivesand Negatives
Ramana Rao Kompella IMC 2004, Sicily, Italy
Deployment
ÿ Syn Flood‡Forward Path of the attack traffic : PCF (SYN,FIN, <DIP,DP>)‡Reverse Path of the attack traffic : PCF (SYN, FIN, <SIP,SP>)
ÿ Port Scanning, Worm Scanning‡PCF (SYN, FIN, <SIP>)
ÿ Horizontal Port Scan‡PCF (SYN, FIN, <SIP,DP>)
ÿ Bidirectional deployment‡PCF (outgoing SYN, incoming FIN, <IP, Port>)
ÿ More details about spoofability in different deploymentoptions in the paper
Ramana Rao Kompella IMC 2004, Sicily, Italy
Experience of PCFs over largetimescales (1 day)
ÿTotal Destinations = 5.16 MillionÿTotal <DIP,DP> pairs = 30.36MillionÿThreshold = 150 (sigma=10)‡2.5 SYN-FIN imbalances per second (avg)
ÿTotal flows detected = 517ÿFalse Positives = 6ÿFalse Negatives = 0
ÿMore about actual characteristics of these flowssuch as duration and so on in the paper
Ramana Rao Kompella IMC 2004, Sicily, Italy
Conclusions
ÿ Scalable attack detection framework‡Behavioral aliasing and spoofing are key questions for any
scalable detection technique
ÿ PCF Data structure‡Provides a general tool to detect wide range of partial completion
attacks
ÿ Analysis‡ Gaussian Approximation to estimate behavioral aliasing.
ÿ Evaluation‡Validated against real network traces
ÿ Low Cost Implementation‡Using just 360KB of High speed memory