9/27/2013 1

FA-9550-12-1-0077 SEMANTIC APPROACH TO BEHAVIOR-BASED IDS AND ITS APPLICATIONS

PI: Victor Skormin vskormin@binghamton.edu

Dr. Andrey Dolgikh – Research Scientist

Zachary Birnbaum – PhD Student Patricia Moat – PhD Student

James Antonakos – PhD Student

BINGHAMTON U N I V E R S I T Y

Center Advanced

Information Technologies

mailto:vskormin@binghamton.edu

9/27/2013 2

SUBCONTRACT

Large Scale Detection of SSL Man-in-the-Middle(MITM) Vulnerabilities in Mobile Systems

Students: David Sounthiraraj Justin Sahs Garrett Greenwood

Professors: Dr. Zhiqiang Lin (Lead) Dr. Latifur Khan (Lead) Dr. Kevin Hamlen Dr. Bhavani Thuraisingham

Computer Science Department, University of Texas at Dallas

9/27/2013 3

Common anti viruses are inefficient

Against zero-day attacks, Against directed attacks, Against encrypted malware, Against multipartate malware, Against poly- and metamorphic malware, and Due to exponentially growing database of binary signatures of malware This justifies the behavioral IDS technologies that could be both anomaly detection and misuse detection

Information for malware analysis

and attack mitigation

Attacker

Information for malware analysis

and attack mitigation comes too late for

defending the high value target

Attacker

High value target

Traditional malware: Developed to create massive computer epidemics At the early stages of epidemics provides plenty of information for attack mitigation

Targeted attacks: Developed to create single attacks against high value target (industrial and government facilities) Does not create epidemics thus do not provide data for attack mitigation: the first instance of attack could also be the last instance The malware is highly specialized for particular environment of the target

9/27/2013 4

INTERNET

Central Computer

Control Electronics

(SCADA, PLC)

Servo mechanisms

Distorted control signals

Destructive Actions

Targeted attacks on SCADA systems

Functionality

Behavior

Primitive Actions

APIs

System calls NtOpenFile, NtDeviceIOControl …

ReadFile, Socket, Send …

Connect a socket, Open and Read File, Send a buffer via the socket

Send password.txt file via Socket Send password.txt file via Network Pipe … E-mail password txt.

Steal user credentials

Semantic pyramid of process behavior

9/27/2013 6

The lowest level is easy to monitor but it has low discriminating power for IDS

The highest level offers clear division between malicious and benign behavior but cannot be monitored

Our research works across the entire semantic pyramid of behavior

AGGREGATION OF SYSTEM CALLS INTO API FUNCTIONS

TOP OF THE SEMANTIC PYIRAMID MALICIOUS BENIGN

Theory and software tools

developed

9/27/2013 7

MONITORING SYSTEM CALL SEQUENCES

AGGREGATION OF API FUNCTIONS INTO FUNCTIONALITIES

9/27/2013 8

Monitoring System Calls

9/27/2013 9

Assembly of the Object Access Graph

9/27/2013 10

Compression of the Object Access Graph

9/27/2013 11

Compressed Graph Close-up (functionality CreateProcess)

9/27/2013 12

Representation of the Graph Components (Functionalities)

by Colored Petri Nets for Real-Time Detection

Call #8

11 5in outh h==

5 5,in outh h

11inh

Chain 5,11

1 5:in

fh h=

1 28 1 8 22

22

( ) & ( )

&( '*. ')

outf

in

h h h h

s exe

== ==

∈

1fh

Call #22

Functionality

1 28 8,h h

122 22,

inh s

11 56inh =

Call #11

11 134inh =

5 523, 56in outh h= =

Call #5

1 23fh =

111 11Call ( 56,...)h =

11Call (134,...)

556 Call (23,...)=

1 28 823, 90h h= =

22 2290, ' . 'out inh s cmd exe= =1 28 823, 12h h= =

2290=Call (' . ',...)cmd exe

8Call (23,12,...)

8Call (23,90,...)

2 22:in

fs s=

2 ' . 'fs cmd exe=

2 5 1

11 2

3 22

8 1 3

h =Call (h ,...)Call (h )h =Call (" . ",...)Call (h , h ,...)

cmd exe

Functionality:

9/27/2013 13

networked computers implementing a fixed set of legitimate programs

Syst

em c

all

data

Functionality extraction

Frequency,executions/minute

A B C D EExtracted functionalities

Frequency,executions/minute

A B D EExtracted functionalities

XC

Customized normalcy profile Abnormal profile indicativeof attack

IDS utilizing a customized normalcy profile

System call monitor

Recognition of the legitimate functionalities

Anomaly detection in legitimate functionalities

Detection of earlier unseen functionalities

Detection of abnormal execution frequencies

Computer System

Library of CPN representing customized

normalcy profile

Library of CPN representing

malicious functionalities

Alarm Block diagram of the system prototype

14

Detection of known malicious functionalities

9/27/2013 15

See Videos

https://docs.google.com/open?id=0B5QdGhtUiP_AYTNSaVlFbXh4ZXc https://docs.google.com/open?id=0B5QdGhtUiP_AUjJMeWFET2draTQ https://docs.google.com/open?id=0B5QdGhtUiP_ANXhXajQwUnY4cDA https://docs.google.com/open?id=0B5QdGhtUiP_ANmdFZmxXc25IV0U

https://docs.google.com/open?id=0B5QdGhtUiP_AYTNSaVlFbXh4ZXchttps://docs.google.com/open?id=0B5QdGhtUiP_AUjJMeWFET2draTQhttps://docs.google.com/open?id=0B5QdGhtUiP_ANXhXajQwUnY4cDAhttps://docs.google.com/open?id=0B5QdGhtUiP_ANmdFZmxXc25IV0U

9/27/2013 16

Next Steps

Theoretical: Further enhancement of the IDS procedures

Working without system calls Expanding the semantic approach to behavior analysis of other-

than-computer systems

9/27/2013 17

9/27/2013 18

9/27/2013 19

Checkpoints within Control Flow Graph (CFG) A

B

D

C

A

B

D

A

C

D

A

B

C

A

B

C D

Dynamic match

Dynamic mismatch

9/27/2013 20

Behavioral Analysis of UAV There are several different UAV attack vectors and could be classified as follows: •Attacks against Ground Control Stations (GCS) •Wireless Attacks •Attacks against onboard UAV Flight Control Computer (FCC) •Attacks against onboard UAV sensors Available data: •Control efforts •On-board sensors •Waypoint information

Diagnostic decisions: •Does the FCC functionality exhibit "extracurricular" activity? •Is the UAV behavior inconsistent with the assigned mission due to a cyber attack? •Is the UAV behavior modified by an incipient hardware failure?

9/27/2013 21

Next Steps

Applied: Application to SCADA systems Application to cloud systems Application to mobile phones

9/27/2013 22

Recent publications 1. Zachary Birnbaum, Andrey Dolgikh, Victor Skormin “Intrusion Detection using Object Access

Graphs” , submitted to The 8th International Conference on Malicious and Unwanted Software (Malware 2013), Fajardo, Puerto Rico, USA, October, 2013

2. Zachary Birnbaum, Andrey Dolgikh, Victor Skormin, Patricia Moat ”Intrusion Detection using n-grams of Object Access Graph components”, submitted to 2013 LASER Workshop, Arlington, Virginia, October, 2013

3. Andrey Dolgikh, Zachary Birnbaum, Victor Skormin, Customized Behavioral Normalcy Profiles for Critical Infrastructure Protection, Proceedings Annual Symposium on Information Assurance, ASIA, 2013

4. Andrey Dolgikh, Zachary Birnbaum, Yu Chen, Bingwei Liu, Victor Skormin, Proceedings Cloud Security Auditing based on Behavioral Modeling, Cloud Security Auditing Workshop at the IEEE Congress on Services, 2013

5. Andrey Dolgikh, Zachary Birnbaum, Yu Chen and Victor Skormin, Behavioral Modeling for Suspicious Process Detection in Cloud Computing Environments, Proceedings MDM mCloud 2013

6. A. Dolgikh, T. Nykodym, V. Skormin, and Z. Birnbaum, "Using Behavioral Modeling And Customized Normalcy Profiles As Protection Against Targeted Cyber-Attacks," Proceedings MMM-ACNS October 17, 2012, St. Petersburg, Russia.

7. V. Skormin, T. Nykodym, A. Dolgikh, J. Antonakos, "Customized Normalcy Profiles for the Detection of Targeted Attacks," Proceedings EvoStar'12, Computational Intelligence for Risk Management, Security and Defense Applications, Malaga, Spain, April 2012

9/27/2013 23

SUBCONTRACT

Large Scale Detection of SSL Man-in-the-Middle(MITM) Vulnerabilities in Mobile Systems

Students: David Sounthiraraj Justin Sahs Garrett Greenwood

Professors: Dr. Zhiqiang Lin (Lead) Dr. Latifur Khan (Lead) Dr. Kevin Hamlen Dr. Bhavani Thuraisingham

Computer Science Department, University of Texas at Dallas

Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23