computer science systematic testing and verification of security policies tao xie department of...

Computer Science

Systematic Testing and Verification of Security Policies

Tao XieDepartment of Computer Science

North Carolina State Universityhttps://sites.google.com/site/asergrp/projects/policy

•Joint Work with Vincent Hu, Rick Khun, and ACTS group (NIST)•JeeHyun Hwang, Evan Martin (NCSU), Alex Liu (MSU)

Computer Science

Motivation

• Digital information is– Easy to access– Easy to search

• Sensitive information requires access control mechanisms

• Security policies are popularly in access control– Access control policies for applications– Firewall policies for networks

Computer Science

Motivation - cont.

• How to ensure the correct specification of security policies?– What you specify is what you get, but not

necessarily what you want

• Solution: systematic testing and verification of security policies

Computer Science

Example Access Control Policy

• Subjects: Student, Faculty• Actions: Assign, Receive• Resources: Grades

Rule 1: IF (faculty AND assign AND grades)Permit

Rule 2: IF (student AND receive AND grades)Permit

Rule 3: OTHERWISEDeny

Computer Science

Policy Verification

• Verify policy against specified property

What properties can you come up for this policy?




Computer Science

Policy Verification

Property: student can never assign grades




Violated with a counterexample request: faculty|student assign grades

Computer Science

Policy Verification

“when the specification language is sufficiently declarative, users have great difficulty providing a duplicate statement of behavior.” --- Shriram Krishnamurthi [RiseandRise 08]




Computer Science

Our Approaches

• Systematic policy verification– Property inference [POLICY 06, SSIRI 09, DBSec 10]

– Property-quality assessment [ACASC 08]– Properties derived from access control models [POLICY 10DE]

• Systematic policy testing– Structural coverage criteria [ICICS 06]

– Fault models/mutation testing [WWW 07]

– Test generation [SESS 07]

– Policy engine performance [SIGMETRICS 08, TC]

– Policy engine correctness [TAV-WEB 08]

– Firewall policy testing/fixing [SRDS 08/09, LISA 10]

•XACML policies

•XACML engines

•Firewall policies

Computer Science

XACML

• A standard access control policy language used to express access control policies– who can do what when

• A request/response language used to express– queries about whether access should be allowed

(requests) and – answers to those queries (responses)

•http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml

Computer Science

XACML Policy Structure

•http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml

policy

target

target

rule2

target

rule1

cond1 cond2

• A Policy Set holds other policies or policy sets.

• A policy is expressed as a set of rules.

• Rules have targets and a set of conditions that determine if the rule applies to a given request.

• Both rule and policy combining algorithms exist to reconcile conflicts.

Computer Science

A Simple Scenario

• A Subject who wishes to perform an Action on a Resource must do so through a PEP.

• The PEP forms the XACML request and sends it to the PDP.• The PDP checks the request against the Policy and returns an

XACML response.• The PEP either Permits or Denies access to the resource.

Computer Science

Software Testing

Expected Outputs

Test Outputs

Test Inputs

Program

Expected ResponsesResponsesRequests Policy

Software Testing

Policy Testing

Policy Testing

Computer Science

Research Problems and Solutions

• Test generation - Request generation– Policy Coverage Criteria– Random request generation– Request generation based on change-impact

analysis– Mutation testing to assess fault-detection

capability

• Test-result inspection - Response inspection– Request selection and minimization based on

structural coverage

Computer Science

Structural Policy Coverage Criteria

policy

target

target

rule2

target

rule1

cond1 cond2

policy covered if target matches

rule2 covered if target matches

Condition must evaluate to True and False to be covered entirely

Computer Science

Random Request Generation

• The example policy:– Subjects: Student, Faculty– Actions: Assign, Receive– Resources: Grades

• Model the set of attribute values as a vector of bits and randomize the bits

Student Faculty Assign Receive Grades

1 0 0 0 00 1 1 0 1

Computer Science

Cirg: Change-Impact Request Generation

policy

version synthesis

policyversions

change-impact analysis

3. requestgeneration

counterexamples

requests

Computer Science

IF (faculty AND assign AND grades)

ELSE IF (student AND receive AND grades)Permit

ELSEDeny

• Counter-example– faculty, assign, grades : Permit Deny

Cirg Example

PermitDeny

Computer Science

Rationale: synthesize two versions whose differences are coverage targets

• All-to-Empty• One-to-Empty• One-Increment• All-to-Minus-One• All-to-Change-One-Effect

Synthesized Versions

Computer Science

Margrave – Change-Impact Analysis Tool

• Faculty (f) can assign (a) grades (g)• Students (s) can receive (r) grades (g)

[Fisler et al. ICSE 05]

Multi-Terminal Decision Diagrams

Computer Science

Margrave Sample Output

1:/Subject, role, Faculty/ 2:/Subject, role, Student/ 3:/Resource, resource-class, ExternalGrades/ 4:/Resource, resource-class, InternalGrades/ 5:/Action, command, Assign/ 6:/Action, command, View/7:/Action, command, Receive/ 8:/Subject, role, TA/12345678{00010101 N->P00011001 N->P00100101 N->P00101001 N->P01010101 N->P01011001 N->P01100101 N->P01101001 N->P}

Computer Science

Software Mutation Testing

Test Outputs

Test Inputs

Program

Mutator

MutantProgram

Mutant Outputs

Differ?Mutant Killed!

Mutation Operators

Computer Science

Policy Mutation Testing

ResponsesRequests Policy

Mutator

MutantPolicy

Mutant Responses


Mutation Operators

Computer Science

Components of Mutation Testing Framework


Mutator

MutantPolicy

Mutant Responses


Mutation Operators

Computer Science

Research Questions

• Does test selection based on structural coverage criteria produce request sets with high fault-detection capability?

• What are the individual characteristics of each mutation operator? – Are some more difficult to kill than others? – Are some easily killed by request sets selected

based on structural coverage criteria?

Computer Science

Sample Policies

•continue: 51 policies, 56 rules

Computer Science

# of Requests Generated and Selected

0

5

10

15

20

25

30

35

40

45

50

codeA

codeB

codeC

codeD

conference

default-2

demo-11

demo-26

demo-5

mod-fedora

simple-policy

Nu

mb

er o

f R

equ

ests

Cirg NumReq NumRedReq

•continue: 373 (cirg), 500 (random), 32 (reduction)

Computer Science

Coverage Results

Random Request Set Cirg

Subject PolCov RuleCov CondCov PolCov RuleCov CondCov

codeA 100 100 N/A 100 100 N/A

codeB 100 100 N/A 100 100 N/A

codeC 100 100 N/A 100 100 N/A

codeD 100 100 N/A 100 100 N/A

conference 0 0 N/A 100 100 N/A

default-2 100 92.31 75 100 100 50

demo-11 100 100 75 100 100 50

demo-26 100 100 50 100 100 50

demo-5 100 100 75 100 100 50

mod-fedora 100 84.62 58.33 84.62 84.62 33.33

simple-policy 100 100 N/A 100 100 N/A

•continue: 32% RuleCov (random) vs. 98% RuleCov(cirg)

Computer Science

Mutation Operators, Mutation, and Equivalent Mutant Detection


Mutator

MutantPolicy

Mutant Response

s


Mutation Operators

Computer Science

Mutation Operators

• Each operator mutates a different policy element: policy set, policy, rule, condition, and/or their associated targets and effects.

Op Description

CPC Change Policy Combining algorithm

CRC Change Rule Combining algorithm

CRE Change Rule Effect

PSTF Policy Set Target False

PSTT Policy Set Target True

PTF Policy Target False

PTT Policy Target True

RCF Rule Condition False

RCT Rule Condition True

RMP Remove Policy

RMPS Remove Policy Set

RMR Remove Rule

RTF Rule Target False

RTT Rule Target True

Computer Science

IF (faculty AND assign AND grades)

ELSE IF (student AND receive AND grades)Permit

ELSEDeny

• The CRE mutation operator is performed on each rule and changes the decision effect (Permit Deny)

Change Rule Effect (CRE) Example

PermitDeny

Computer Science

Equivalent Mutant Detection

• An equivalent mutant is semantically equivalent although syntactically different than the original policy.

• They provide no value and waste resources.

• We use change-impact analysis to detect equivalent mutants and remove them.

Op # Mutants# Equivalent

CPC 1280 64

CRC 2316 32

CRE 968 0

PSTF 256 0

PSTT 256 0

PTF 579 0

PTT 579 0

RCF 670 0

RCT 18 0

RMP 579 0

RMPS 256 0

RMR 670 0

RTF 670 0

RTT 639 0

Computer Science

Request Evaluation and Mutant Detection


Mutator

MutantPolicy

Mutant Response

s


Mutation Operators

Computer Science

Sun’s XACML implementation

• An open source implementation of the XACML standard in Java

• Developed by Sun as part of an ongoing project on Internet Authorization in the Internet Security Research Group

•http://sunxacml.sourceforge.net/

Computer Science

Mutant-Killing Ratios by Subject

0

10

20

30

40

50

60

70

80

90

100

codeA

codeB

codeC

codeD

confe

rence

defau

lt-2

demo-1

1

demo-2

6

demo-5

mod-fe

dora

sim

ple-p

olicy

Mu

tan

t-K

illin

g R

atio

s

Cirg Random Selected Random

Computer Science

Mutant-Killing Ratio by Operator

0

10

20

30

40

50

60

70

80

90

100

PSTTPSTF

PTTPTF

RTTRTF

RCTRCF

CPCCRC

CRE

RMPS

RMP

RMR

Mu

tan

t-K

illin

g R

atio

s

Cirg Random Selected Random

Computer Science

Our Approaches



• Systematic policy testing– Structural coverage criteria [ICICS 06]– Fault models/mutation testing [WWW 07]– Test generation [SESS 07]– Policy engine performance [SIGMETRICS 08, TC]



•XACML policies

•XACML engines


Computer Science

Firewall Policy Structure• A Policy is expressed as a set of rules.• A Rule is represented as <predicate> → <decision>• <predicate> is a set of <clauses>

• An example firewall policy

•Rule r1’s <predicate>

•Rule r1’s <decision>•A range in each field refers <clause>

Computer Science

Structural Coverage Definition• Rationale: when the policy part with a fault is not

evaluated (i.e., “covered”), the fault is often not exposed.

• Rule coverage of a policy P by packets T = #rules evaluated by at least one packet in T

#rules in P• Predicate coverage of a policy P by packets T

#predicates evaluated to true or false by T at least once2 ×#predicates in P

• Clause coverage of a policy P by packets T #clauses evaluated to true or false by T at least once

2 ×#clauses in P

Computer Science

Test Packet Generation

Our objective: generating packets for achieving high structural coverage

1. Random Packet Generation – Randomly selects values for a packet

2. Packet Generation based on Local Constraint Solving– Considering individual rules in a policy

3. Packet Generation based on Global Constraint Solving– Considering multiple rules in a policy

Computer Science

Experiments (measuring coverage)• Test 14 firewall policies• Generate packets by our proposed three techniques• Measure structural coverage.

Computer Science

Experiments (measuring coverage)

Computer Science

Experiments (measuring fault detection capability)

• We also used reduced packet sets (maintaining the same level of structural coverage with the corresponding original packet set)

Computer Science

NCSU/NIST ACPT Architecture

•GUI

AC Model Templates

DataAcquisition

PolicyGeneratorStatic Verification

DynamicVerification

•Test inputs based•on structural or

•combinatorial coverage

•User,•attribute,•resource,

•role,•etc. data

•GUI allows specification of users, groups, attributes, roles, rules, policies, and resources

Generate enforceable

policies

.xml

Generate and evaluate test inputs

•Verify access control policies

•API/mechanism to consume/acquire external data related to policies

•Generate •test inputs

Test inputs with

their evaluated decisions

•Administrator

XACML

•http://www.nist.gov/itl/csd/set/acpt.cfm

Computer Science44

• Property specification in ACPT

ACPT

Computer Science45

• Verify the property against Policy A, the result return false with counterexample.

Static Verification

Computer Science46

• Verify the property against Policy B, the result return true.

Static Verification (cont.)

Computer Science47

Test Input Generation and Evaluation

Computer Science48

XACML Generation

Computer Science

Conclusion



• Systematic policy testing– Structural coverage criteria [ICICS 06]

– Fault models/mutation testing [WWW 07]

– Test generation [SESS 07]

– Policy engine performance [SIGMETRICS 08, TC]



•XACML policies

•XACML engines


Computer Science

Questions?

https://sites.google.com/site/asergrp/projects/policy

computer science systematic testing and verification of security policies tao xie department of...

Documents