On the Difficulty of Scalably Detecting Network Attacks

Kirill Levchenkowith Ramamohan Paturi and George Varghese

•Which Intrusion Detection problems are hard (require per-flow state)?

• Counting distinct flows? Approximately?

• Detecting SYN Flooding? Port Scans?

• Are there efficient algorithms we don’t know yet?

•How to tell?

Detecting Attacks

•Abstract problem formulation

•Communication Complexity

•Source for NIDS problem reductions

•Establishes worst-case lower bounds

•Easy and fun to use

Theoretical Techniques

•Problem Motivation and Setting

•The Set Disjointness problem

•Application to SYN Flooding

•Implications

•Conclusion

Outline

Problem Setting

Idea: move the NIDS into the network, protect more systems!

Problem Setting

• Protect more hosts

• Single device to administer

• More informed position

•Challenges:• More hosts to protect

• Higher bandwidth links

•How do resource requirements scale?

•Are there fundamental limits?

Problem Setting

•Two packet sequences: traffic entering and leaving the network

•Goal: detect attack using packet sequences

Problem SettingOutbound

Inbound

•Formalize the algorithmic problem

• An algorithm on packet sequences

•Reduce one problem to another

• Show solving one problem solves the other

Algorithms & Reductions

•Solving the NIDS problem solves the Set Disjointness problem

•Use strong bounds on Set Disjointness

•Establishes space lower bound for the NIDS problem

Our ReductionSet Disjointness → NIDS Problem

Set Disjointness

Alice Bob

Disjoint?

Set Disjointness

Alice Bob

Communication:

(even randomized)

Let’s Try It!(on SYN Flooding)

SYN FloodingNormal Attack

Goal: detect unclosed connections.

SYN Flooding

1. Creates packet sequence corresponding to her set with SYN flag

2. Runs NIDS algorithm on input sequence

3. Suspends it after reading the last item

4. Sends the state of the algorithm to Bob

SYN Flooding

Yes/No1. Creates packet sequence corresponding to elements not in his set with FIN flag

2. Resumes NIDS algorithm on input sequence (using state sent by Alice)

3. Result indicates if sets intersect or not

SYN Flooding

•If A and B intersect, there is a SYN packet not followed by a FIN

•If A and B are disjoint, every SYN packet has a matching FIN

Algorithm sees:

1.Alice sent Bob the state of the NIDS algorithm:

2.Set Disjointness requires Ω(n) bits

∴The state of the algorithm is Ω(n) bits

SYN Flooding

no. of flows

•Port Scans

• per-host state required

•TCP connection hijacking

• per-flow state required

•Evasion by fragmentation

• re-assembly required

Other NIDS Problems

•Sometimes per-flow state is required, but:

• Examples are artificial (may not occur in practice)

• Problem semantics may be a great help

• System can fail gracefully when out of memory

•Additional information may be available!

Implications

•Additional information can be useful:

• Count outgoing SYN+ACK and FIN packets

• Works if protected network can be trusted

• Hop-count filtering or other fingerprinting

• Small UDP fragments are unusual

Implications

•Set Disjointness is a useful reduction source

• Even hard under randomization

• Permits relaxations (see paper)

•Exposes and formalizes the hardness of some NIDS problems

•Guides practical algorithm design to consider hard cases and failure modes

Conclusion