how to test an ids? - sigcomm · snort true pos. alerts snort false pos. alerts snort false...
TRANSCRIPT
![Page 1: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/1.jpg)
How to Test an IDS?GENESIDS: An Automated System for Generating Attack Traffic
WTMC 2018
Felix Erlacher, Falko Dressler
![Page 2: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/2.jpg)
Network Intrusion Detection Systems (NIDS)
Analyze network traffic for malicous activity
▶ Anomaly based NIDS▶ Have a model of ’normal’ traffic▶ Detect and alert deviations from ’normal’ traffic
▶ Signature based NIDS▶ Have rule-set of known attacks and incidents▶ Detect rule patterns in analyzed network traffic→ Example: Snort
Felix Erlacher: How to Test an IDS? GENESIDS 2
![Page 3: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/3.jpg)
How to test a NIDS?
▶ Real traffic?▶ hard to get▶ public traces: old, no payload▶ contains only very few attacks
▶ Manually creating attack traffic?▶ time intensive▶ cumbersome
SUMMARY: traces do not contain enough unique attacks
Felix Erlacher: How to Test an IDS? GENESIDS 3
![Page 4: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/4.jpg)
How to test a NIDS?
▶ Real traffic?▶ hard to get▶ public traces: old, no payload▶ contains only very few attacks
▶ Manually creating attack traffic?▶ time intensive▶ cumbersome
SUMMARY: traces do not contain enough unique attacks
Felix Erlacher: How to Test an IDS? GENESIDS 3
![Page 5: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/5.jpg)
How to test a NIDS?
▶ Real traffic?▶ hard to get▶ public traces: old, no payload▶ contains only very few attacks
▶ Manually creating attack traffic?▶ time intensive▶ cumbersome
SUMMARY: traces do not contain enough unique attacks
Felix Erlacher: How to Test an IDS? GENESIDS 3
![Page 6: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/6.jpg)
How to test a NIDS!
GENESIDS:Generating Events for Signature-based Intrusion Detection Systems
▶ INPUT: Set of attack descriptions▶ Snort syntax▶ HTTP attacks
▶ OUTPUT: Stateful network traffic containing attack patterns▶ One flow per attack▶ Annotated with an attack ID
Felix Erlacher: How to Test an IDS? GENESIDS 4
![Page 7: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/7.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 8: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/8.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 9: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/9.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 10: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/10.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 11: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/11.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 12: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/12.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 13: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/13.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 14: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/14.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 15: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/15.jpg)
Rule example:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
genesids -f example.rule -s example.com
Felix Erlacher: How to Test an IDS? GENESIDS 5
![Page 16: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/16.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 17: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/17.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 18: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/18.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 19: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/19.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 20: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/20.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST"; http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 21: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/21.jpg)
Example traffic in Wireshark:
alert tcp any any -> any any (msg:"This is an example rule";content:"POST";http_method;uricontent:"|2F|evil.jpg";pcre:"/AttackBody-V[0-9].*/P";sid:1234567; rev:0;)
Felix Erlacher: How to Test an IDS? GENESIDS 6
![Page 22: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/22.jpg)
GENESIDS Evaluation: Goals & Rules
▶ Ability to generate a variety of different attacks▶ Generated attacks trigger expected event
All supported Snort rules from:
▶ Snort.org subscriber rule-set▶ Snort.org community rule-set▶ Emerging Threats rule-set
TOTAL 8101 different rules
Felix Erlacher: How to Test an IDS? GENESIDS 7
![Page 23: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/23.jpg)
GENESIDS Evaluation: Goals & Rules
▶ Ability to generate a variety of different attacks▶ Generated attacks trigger expected event
All supported Snort rules from:
▶ Snort.org subscriber rule-set▶ Snort.org community rule-set▶ Emerging Threats rule-set
TOTAL 8101 different rules
Felix Erlacher: How to Test an IDS? GENESIDS 7
![Page 24: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/24.jpg)
GENESIDS Evaluation steps
SignaturesGENESIDS
TCP Connection
SignaturesHTTP Server
tcpdump
Step 1
Rules
00101100101
01001010010
00001110111
11100110100
10111010010
01010111111
NetworkTrace
Felix Erlacher: How to Test an IDS? GENESIDS 8
![Page 25: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/25.jpg)
GENESIDS Evaluation steps
Rules
SignaturesSnort
00101100101
01001010010
00001110111
11100110100
10111010010
01010111111
NetworkTrace
Alerts
Step 2
GENESIDS
TCP ConnectionHTTP Server
tcpdump
Step 1
Rules
00101100101
01001010010
00001110111
11100110100
10111010010
01010111111
NetworkTrace
Felix Erlacher: How to Test an IDS? GENESIDS 8
![Page 26: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/26.jpg)
Evaluation results: Generated attacks
0 20 40 60 80 1000
2000
4000
6000
8000
10000
Experiment Run
Att
ack
s
Attacks Sent
▶ GENESIDS: 8101 attacks generated (out of 8101 rules)
Felix Erlacher: How to Test an IDS? GENESIDS 9
![Page 27: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/27.jpg)
Evaluation results: True positives
0 20 40 60 80 1000
2000
4000
6000
8000
10000
Experiment Run
Att
ack
s
Attacks Sent
Snort True Pos. Alerts
▶ Snort: 7877 (avg) true positive alerts triggered (out of 8101)
Felix Erlacher: How to Test an IDS? GENESIDS 10
![Page 28: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/28.jpg)
Evaluation results: False positives
0 20 40 60 80 100100
200
500
1000
2000
5000
10000
Experiment Run
Att
ack
s
Attacks SentSnort True Pos. Alerts
Snort False Pos. Alerts
▶ Snort: 2847 (avg) false positive alerts triggered (62%triggered by 3 rules)
Felix Erlacher: How to Test an IDS? GENESIDS 11
![Page 29: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/29.jpg)
Evaluation results: False negatives
0 20 40 60 80 100100
200
500
1000
2000
5000
10000
Experiment Run
Att
ack
s
Attacks SentSnort True Pos. Alerts
Snort False Pos. Alerts
Snort False Negatives
▶ Snort: 223 (avg) false negatives (generated attacks that didnot trigger the corresponding alert)
▶ Total of 363 rules generated attack not triggering at leastonce (out of 100)
Felix Erlacher: How to Test an IDS? GENESIDS 12
![Page 30: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/30.jpg)
Conclusion
GENESIDS: Generating attack traffic for NIDS testing
▶ Accepting Snort syntax → thousands of up-to-date attackdefinitions
▶ 97% of generated attacks triggered corresponding alert
▶ Less than 3% failed to trigger corresponding alert
Felix Erlacher: How to Test an IDS? GENESIDS 13
![Page 31: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/31.jpg)
Software, configuration files, attack network traces:
www.ccs-labs.org/~erlacher/resources/
Reminder: GENESIDS Demo → Wednesday 14:10
Thank you for your attention
Felix Erlacher: How to Test an IDS? GENESIDS 14
![Page 32: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/32.jpg)
Software, configuration files, attack network traces:
www.ccs-labs.org/~erlacher/resources/
Reminder: GENESIDS Demo → Wednesday 14:10
Thank you for your attention
Felix Erlacher: How to Test an IDS? GENESIDS 14
![Page 33: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/33.jpg)
Mixed traffic with GENESIDS and TRex
Felix Erlacher: How to Test an IDS? GENESIDS 15
![Page 34: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/34.jpg)
False negatives: Closer look
363 different rules not triggering corresponding event over 100 runs
1. Rules never triggering alert (179)▶ Some require non-compliant HTTP (e.g. multiple \r\n\r\n)▶ Restricting strings with ^ and $▶ …
2. Rules failing to trigger at least once (in 100 runs) (184)▶ all of the rules contain a PCRE with random generation (.)
random generation produced unsupported character▶ …
Felix Erlacher: How to Test an IDS? GENESIDS 16
![Page 35: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/35.jpg)
TLS interception proxy
End-to-end cryptograpic service:
Interception Proxy:
01001001100
libpcap format
Felix Erlacher: How to Test an IDS? GENESIDS 17
![Page 36: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/36.jpg)
Typical monitoring scenario
01001001100
Internet
Felix Erlacher: How to Test an IDS? GENESIDS 18
![Page 37: How to Test an IDS? - SIGCOMM · Snort True Pos. Alerts Snort False Pos. Alerts Snort False Negatives Snort: 223 (avg) false negatives (generated attacks that did not trigger the](https://reader034.vdocument.in/reader034/viewer/2022042710/5f596375dda0ea2eda74cc2f/html5/thumbnails/37.jpg)
GENESIDS
SignaturesGENESIDS
TCP ConnectionHTTP Request
HTTP Response SignaturesHTTP Server
Rules
Loop through rules:1. parse rule2. generate patterns for HTTP request3. init TCP connection4. send HTTP request5. wait for response6. end TCP connection
repeatFelix Erlacher: How to Test an IDS? GENESIDS 19