detecting & preventing misuse of privilege (pmop)

Detecting & PreventingMisuse of Privilege

(PMOP)PI Meeting 7/13/05

Bob Balzer (Teknowledge)

Howie Shrobe (MIT)

• Updates since Kickoff• Updates since January

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

MITTeknowledge

Distinguishing AWDRAT & PMOP

• AWDRAT– Detecting misbehaving software

• Hijacks, overprivledged scripts, trap doors, faults

• PMOP– Detecting misbehaving operators

• Malicious intent, operator error

• For integrated SRS system need both capabilities

Progress Since 1/05 PI Meeting

• End-to-End working system• Instrumentation of Operator Actions• Expanded Scope

– Originally, application harm by application operator– Now, system harm by application operator

• System harm harm to OS objects (files, registry, etc.)• Integration of OS level Wrappers into PMOP architecture

– Next, system harm by OS operator

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What are we trying to do?• Block Harmful Operations

• Differentiate– Operator Error– Malicious Intent

Application Level^

Technical Challenges

• Modeling system to predict effect• Modeling Operator to differentiate

– Operator Error– Malicious Intent

• Applying security to the application layer– Creating application specific rule framework for defining

harm• Harm expressed orthogonally from OS objects

– Finding points in application to apply it– NOT complete coverage of application semantics

=> to be handled in Phase II for a specific high value system

Canned ComponentPublishes fixed output

Legacy ComponentCode Not Available

Table Lookup

MAF

CAF

Proposed MI

Approved MI

Targeting TNL

JEESEDC JW

CHWChem

Hazard

SPI TAP

CHI

Combat

Ops

AODB AS

LOC

Weather

Hazard

WH

WLC

ATO

EDC

CHW

Chem

Hazard

CHA

External JBI DemVal Dataflow(via Publish/Subscribe)

The Good – The Bad – The Ugly

Applying Security toApplication Layer

• CAF DemVal component– Builds Air Transport Plans– Publishes completely built Air Transport Plans– Edits partially built Air Transport Plans– Saves & Restores partially built Air Transport Plans

• Creating application specific rule framework for defining harm– Harm expressed orthogonally from OS objects– For CAF DemVal component

• Harm = publishing semantically malformed Air Transport Plan– What semantic knowledge and data is required to determine malformedness

• Finding points in application to apply it– For CAF DemVal component

• Commit = Publish Air Transport Plan

Limitations of Approach

• Harm defined as a Boolean No comparison metric

no comparison of alternatives (either predefined or generated) lesser of two or more evils not detected as beneficial suboptimal actions not detected as harmful

• Benefit is not defined Doing nothing or delaying action not detected as harmful

• Possible Phase II Improvements– Define Harm and Benefit as quantified dimensions– Provide operator performance model (generate alternatives)

• Minimal acceptable performance model enables– Detecting suboptimal choice Harm & Malicious Intent in such choices– Detecting delay/do-nothing as harmful & Malicious Intent in such choices

Insider Threat

• Space of Insider Attacks– Attacks through Application Software

– Attacks through OS GUI (Insider as OS user)

– Attacks through Insider’s software

– Physical attacks

Insider Threat

• Space of Insider Attacks– Attacks through Application Software

• General Framework: – Detect Harm (before it happens) & Block It

– Harm defined by application specific rules

• For JBI it was publishing malformed object

Built a framework for applying malformed rule to published object

Built a few “exemplar” malformed object rules

Limited by lack of domain knowledge & engineering funds

• For SaveAs GUI it was harm to JBI or System resources

Used SafeFamily wrapper & rule language

=> Coverage equals Rule Coverage

Insider Threat

• Space of Insider Attacks– Attacks through Application Software– Attacks through OS GUI (Insider as OS user)

• General Framework: – Detect Harm (before it happens) & Block It– Harm defined by application specific rules

• For OS GUI (Explorer process) it is harm to JBI or System resources Used SafeFamily wrapper & rule language=> Coverage equals Rule Coverage

Insider Threat

• Space of Insider Attacks– Attacks through Application Software

– Attacks through OS GUI (Insider as OS user)

– Attacks through Insider’s software• Problem for our approach because generic rule set must be used

• Planned for Option

– Physical attacks • Out of bounds

Insider Threat Coverage

• Space of Insider Attacks– Attacks through Application Software

• Coverage equals Rule Coverage

– Attacks through OS GUI (Insider as OS user)

• Coverage equals Rule Coverage

– Attacks through Insider’s software• Planned for Option

– Physical attacks • Out of bounds

% Coverable

80%

80%

0%

% Space

50%

25%

25%

% Thwartable

40%

20%

0%

===== 60%10%

How do we turn Coverable => Covered Thwartable => Thwarted

Metrics Issues

• How can we show rule framework is generic without covering all of application semantics?Use red team experiment

• Red team rules of engagement– Jointly define application semantics to be

defended– Require 100% coverage of that semantics

How do we turn Coverable => Covered Thwartable => Thwarted

• Force experiment to determine ability to thwart insider attack

– Three (proposed) Flags1. Harm application using only application GUI

(SaveAs GUI excluded)Using jointly defined subset of application semantics

2. Harm application using only SaveAs GUI

3. Harm application using OS GUI (Explorer process)(running other programs excluded)

How do we turn Coverable => Covered Thwartable => Thwarted

Red Team Experiment

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

How will you show success?• Block Harmful Operations

• Differentiate– Operator Error

– Malicious Intent

• Red-TeamExperiment

Block Harmful Operations

Differentiate– Operator Error

– Malicious Intent

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What are implicationsof success?

• Systems can be protectedfrom insider attacks

from operator error

from zero-day attacks

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is technical approach?• Observe effect of operator

action in system model• Match harmful

actions against– Errorful Operator Plans– Attack Plans

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is new?• Observe effect of operator

action in system model• Match harmful

actions against– Errorful Operator Plans– Attack Plans

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is hard?• Modeling System

to predict effect• Modeling Operator

to differentiate– Operator Error– Malicious Intent

What We’re Missing

• Realistic Rules (Domain Knowledgeable)– Would be created by SMEs in real deployment

• Comprehensive Rule Set– Would be created by SMEs in real deployment

• Instrumentation of the GUI actions– Just Mission Building/Editing methods currently

instrumented– GUI actions will be instrumented by 4/1/05

The Good – The Bad – The Ugly

Technology for SRS Integration

• Behavior Monitor/Authorizer– What code is doing– What human operator is doing

• Operational Models– Software Components

• Harm Detectors– Rule driven

• Application-Level• OS-Level

• Intent Determination

Backup (old slides)

Determining Malicious Intent

• Can the operator action be performed legally• Does the operator action cause harm

– Is there an alternative that doesn’t cause harm

– Is this the minimial harm alternative

• Does the operator action satisfy a requirement/goal• Is there a better way to accomplish the goal

– Should the operator have found this better way

MAF

CAF

Proposed MI

Approved MI

Targeting TNL

JEESEDC JW

CHWChem

Hazard

SPI TAP

CHI

Combat

Ops

AODB AS

LOC

Weather

Hazard

WH

WLC

ATO

EDC

CHW

Chem

Hazard

CHA

External

JBI DemVal Dataflow(via Publish/Subscribe)

What We’ve Got

• End-To-End Demonstration (demo shortly)– Working Prototypes of MOP components– Working models & rules of target application– Working integration of MOP components

The Good – The Bad – The Ugly

End-To-End Demonstration• Block Harmful Operations

• Differentiate– Operator Error

– Malicious Intent

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

JBIDemVal

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

Accommodations

• Java code base– Ported wrapper infrastructure

• Planning Application (harm is in future)– Defined Harm as publishing harmful plan

• Available JBI components to wrap– Detailed on next slide

The Good – The Bad – The Ugly

First SRS Tech Transition

• Architecture Visualizer used in HURT (IXO)– Animated Event Sequence Diagram– Animated Dataflow Architecture

M

M

Mediation Cocoon

M

M

JBIServer

MOP Execution Architecture

JBIClient

Harm Rules

Harm Detector

Scripted MOP Driven from History ScriptsNominalHarmful: Takeoff Before LandingHarmful: Missing Leg (landing not collocated with takeoff)

Visualizer

Scripts

Script Driver

History

ClientReconstitution

ArchitectureVisualizer

M

M

Mediation Cocoon

M

M

JBIServer

JBIClient

Mixed Initiative MOP• One Client Live (with human operator)• Others Scripted

DetectingHarmful Actions

Demo

Howie Slides Here

DetectingMalicious Intent

Demo