27/8/2007 APAN 2007 - August 27, 2007 1

The Effects of Filtering Malicious Traffic under DoS Attacks

Chinawat Wongvivitkul

Sudsanguan Ngamsuriyaroj

Department of Computer Science, Faculty of Science

Mahidol University, Thailand

27/8/2007 APAN 2007 - August 27, 2007 2

Agenda

Introduction & Motivation Proposed Work Implementation Experiments & Results Conclusions and Future Work

27/8/2007 APAN 2007 - August 27, 2007 3

Introduction

DoS attacks have been well known for generating huge amount of adverse traffic to a target server and make the server unavailable for services.

Open Source IDS Software: Snort and Bro IDS

Signature detection: based on predefined rules

Anomaly detection: learn first and then classify statistical patterns of incoming traffic

27/8/2007 APAN 2007 - August 27, 2007 4

Motivation

Most studies used simulation tools, and only a few address the issues of server survivability under DoS attacks

Questions How to determine whether the incoming traffic is

malicious in real time How to create an anomaly detector using a simple

statistics How much traffic should be filtered out when the server

is under attacks to make the server survives No work does packet filtering interactively during the

attack

27/8/2007 APAN 2007 - August 27, 2007 5

Proposed Work

Normal output trafficDetection

Analysis

Input Traffic

Reducedoutput traffic

PacketControl

Traffic shaping

Detection Analysis Traffic Control

Drop malicious traffic

Dropped suspicious trafficPacketInformation

We propose a model to measure the effectiveness of filtering malicious traffic on the web server when under DoS attacks

27/8/2007 APAN 2007 - August 27, 2007 6

Proposed Work

Have two phases Detection Analysis

collect statistics of incoming traffic and classifies the status of the traffic.

Traffic Control redirect traffic according to its status, and also

filter traffic if the traffic is malicious

27/8/2007 APAN 2007 - August 27, 2007 7

Detection Analysis

Packet AnalysisInput Traffic

Sent to traffic control

Detection Analysis

Packet Recording

record readread

record

In_Packet Stat_Info

In_Packet keeps information of individual packets

Stat_Info keeps statistics of packets in In_Packet and classify the traffic according to its arrival rate

27/8/2007 APAN 2007 - August 27, 2007 8

Traffic Control

Normal OutputTraffic

Reducedoutput traffic

Traffic shaping

Traffic Control

Drop packets

Drop packets

Packets from Detection Analysis

Normal Traffic

Suspicious Traffic

Malicious Traffic

Read

Packet Control

Stat_Info

27/8/2007 APAN 2007 - August 27, 2007 9

Traffic Control

Normal Traffic sent to the target server with unlimited

bandwidth. Suspicious Traffic

sent to traffic shaping module so that their bandwidth is reduced before arriving at the target server.

Malicious Traffic is dropped before having a chance to attack the

target server

27/8/2007 APAN 2007 - August 27, 2007 10

Implementation

Focus on HTTP traffic only Modify Snort in-line for traffic classification,

traffic redirection, and traffic dropping

Web Server

Attacker

Legitimate USER

Modified Snort In-line

Hub

27/8/2007 APAN 2007 - August 27, 2007 11

Modified Snort In-Line Packet capture/decode engine

Do statistical analysis of each traffic stream Detection engine

Compute the arrival rate at every 30 packets of one traffic stream

Classify traffic into normal, suspicious and malicious according to its arrival rate

Control engine Add an extra module to redirect traffic to different paths

according to its status. Output engine

Perform traffic shaping by dropping suspicious and malicious traffic

27/8/2007 APAN 2007 - August 27, 2007 12

Packet capture/decode engine add Input_traffic function in “detect.c” file of Snort In-line.

Detection engine add the P_analysis function in “snort.c” file

Control engine add p_control function in “snort.c” file.

Output engine dropping the number of suspicious packets according to it

arrival rate

Example rule for dropping suspicious and malicious traffic drop tcp any any -> any 20000 (msg:"D=Http IDS Malicious access tcp

deny";) drop tcp any any -> any 40000 (msg:"D=Http IDS Suspicious access

tcp deny";)

Modified Snort In-Line

27/8/2007 APAN 2007 - August 27, 2007 13

Traffic Flows in Snort In-Line

Input Traffic Iptables(Send input traffic to Queuing)

Packets capture/decode Engine

Detection Engine

Control Engine

Output Engine Alerts/Logs

Snort-In-line

Output Traffic

Packet Type Traffic Rate Threshold (pps)

Normal < 65

Suspicious 65 - 1500

Malicious > 1500

27/8/2007 APAN 2007 - August 27, 2007 14

System Configuration for Experiments

Attacker sends malicious traffic to the web server for 5 minutes

No background traffic generated User makes a request to the server every 3 seconds

until there is a timeout since the server was down

27/8/2007 APAN 2007 - August 27, 2007 15

Experiment 1Server Timeout without Traffic Control

27/8/2007 APAN 2007 - August 27, 2007 16

Experiment 2Server Timeout with Traffic Control

One attacker and filtering rate is fixed at 1/1000

27/8/2007 APAN 2007 - August 27, 2007 17

Experiment 3Server Timeout with Traffic Control

One attacker and varying filtering rates of 1/100, 1/250, 1/500, 1/750, and 1/1000

27/8/2007 APAN 2007 - August 27, 2007 18

Experiment 4Server Timeout with Traffic Control

Three attackers and varying filtering rates of 1/100, 1/250, 1/500, 1/750, and 1/1000

27/8/2007 APAN 2007 - August 27, 2007 19

Conclusions

We show the effects of filtering malicious traffic to the survivability of the server under DoS attacks

We show that a simple and fast anomaly detection is possible by using the traffic arrival rate

Future work: make Snort adaptive and can respond to different arrival rates with adaptive filtering rate

27/8/2007 APAN 2007 - August 27, 2007 20

References

1. Atighetchi M., el.al., Adaptive Cyberdefense for Survival and Intrusion Tolerance, IEEE Internet Computing, Nov-Dec 2004

2. Deri L., Carbone R., and Suin S., Monitoring Networks Using ntop. Proceeding of the 2001 IEEE/IFIP International Symposium on Integrated Network Management, May 2001.

3. Houle K.J. and Weaver G.M., Trends in Denial of Services Attack Technology. CERT Coordination Center, Camegie Mellon University, October 2001.

4. Hwang K, Chen Y, and Liu H. Defending Distributed Systems Against Malicious Intrusions and Network Anomalies. Proceedings of 19th IEEE International Parallel and Distributed Processing Symposium, April 2005.

5. Kashiwa D, Chen E.Y. and Fuji H. Active Shaping: A Countermeasure Against DDoS Attacks. Proceedings of 2nd European Conference on Universal Multiservice Networks; April 2002.

6. Keromytis A., et.al., A Holistic Approach to Service Survivability, Proceedings of the ACM Workshop on Survivable and Self-Regenerative Systems, October 2003.

7. Lan K., Hussain A. and Dutta D., Effect of Malicious Traffic on the Network, Proceedings of Passive and Active Measurement Workshop, April 2003.

8. Lau F, Rubin S.H., Smith M.H. and Trajkovic L., Distributed Denial of Service Attacks. Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, October 2000.

27/8/2007 APAN 2007 - August 27, 2007 21

9. Lee W., Stolfo S.J., and Mok K., Mining in a Data-Flow Environment: Experience in Network Intrusion Detection, Proceedings of the 5th ACM SIGKDD, August 1999.

10. Lee W. and Stolfo S.J., A Framework for Constructing Features and Models for Intrusion Detection Systems, ACM Transactions in Information and System Security, 3(4), November 2000.

11. Long M., Wu C-H, and Hung J.Y., Denial of Service Attacks on Network-Based Control Systems: Impact and Mitigation, IEEE Transactions on Industrial Informatics, 1 (2), May 2005.

12. Mahoney M.V., Network Traffic Anomaly Detection Based on Packet Bytes. Proceedings of ACM Symposium on Applied Computing, March 2003.

13. Paxson V, Bro: A System for Detecting Network Intruders in Real-Time. Proceedings of the 7th USENIX Security Symposium; January 1998.

14. Roesch M, Snort–Lightweight Intrusion Detection for Networks. Proceedings of 13th LISA: Systems Administration Conference; November 1999.

15. Staniford S., Hoagland J.A. and McAlerney J.M., Practical Automated Detection of Stealthy Portscans. Journal of Computer Security, 1(1-2), 2002.

16. Sterne D., et. al., Autonomic Response to Distributed Denial of Service Attacks. Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection, October 2001.

17. Taylor C. and Alves-Foss J. NATE: Network Analysis of Anomalous Traffic Events – A Low-Cost Approach. Proceedings of the ACM workshop on New Security Paradigms, September 2001.

18. Xu J. and Lee W., Sustaining availability of Web Services under Distributed Denial of Service Attacks, IEEE Transactions on Computers, 52(2), February 2003.

References

27/8/2007 APAN 2007 - August 27, 2007 22

Thank You

Q & A