randomized methods for network security games · robust control robust uncertain systems approach...

Randomized Methods for

Network Security Games

João P. Hespanha

Joint work with: S. Bopardikar, M. Prandini, A. Borri, M. D. Di Benedetto

The Geography of Control

1147.502295.003442.504590.00 cluster 1cluster 2cluster 3cluster 4cluster 5cluster 6cluster 7cluster 8cluster 9cluster 10cluster 11cluster 12cluster 13cluster 14cluster 15cluster 16cluster 17cluster 18cluster 19cluster 20

Number of document citations per cluster (20 document clusters)optimal control

optimaloptimal control problems

distributed parameter systemsboundary control

partial differential equations

commentset al

d

linear systemslinear

linear matrix inequalities

delaydelays

delay systems

parametersparameter estimation

parameter

robust controlrobust

uncertain systems

approachbasedusing

designdesigns

backsteppingstochastic systems

stochasticstochastic control

npm

usingconstantregion

controllercontrollers

controller designgeneralrespect

particularclass

functionclass nonlinear systems

polynomialspolynomialcoefficients

formvia

original

stabilitylyapunov functionsstability analysis

performanceworst case

l 1matrix

matriceseigenvalues

linear systemssynthesis

linear matrix inequalitiesproblems

convex optimizationconvex

robustnessperturbations

robustasymptotic stability

shownlimit cycles

conditionscondition

necessary sufficient conditions

descriptor systemssingular systems

given

exponential stabilitysingularly perturbed systems

fast

h infinity controlh infinity

h 2structure

multivariable systemsstructures

constraintsconstrainedconstraint

stablegivendual

pole placementpole assignmentzero dynamics

markov decision processesmarkov chaindistribution


linear matrix inequalitiesoptimizationoptimization problem

formulation

givenlinear systemsconstruction

systemsbehaviornotion

finite dimensionalfinite time

infinite dimensional systemstime

time delay systemstime varying

datainformation

based

givensuch

associatedmethodmethodsparallel

equivalentcriterionshown

continuous timesampled data systems

continuous time systemsoperator

infinite dimensional systemsoperators

approximationapproximate

approximations

output feedbackoutput

output regulation

nonlinear systemsglobal

global stabilizationestimation

filterkalman filter

computationcomplexitysequence

observabilityunderexact

solutionsolutions

riccati equation

state spacestate

state feedbackcontrollabilitycontrollability observability

study

feedbackfeedback control

stabilization

disturbance rejectionfrequency domain

frequency response12s

observerobservers

designalgorithmalgorithms

algorithm based

generalizedconcept

representationnoise

least squaresestimates

controlcontrolled

control systems

robot manipulatorsrobotrobots

discrete time systemsdiscrete time

discrete event systemshybrid systems

switched systemsswitched linear systems

passivity basedpassivitypassive

frameworksystemsstability

iterative learning controlconvergence

ilc

inputinput output

input state stabilitytransfer function

systemstransfer functions

localstatic output feedback

usingsystemssimple

minimum variance

algebraic riccati equationparametrizationparameterization

mechanical systemsmotion

nonholonomic systemsplantplantscontrol

identificationmodelsmodel

newnew approach

presented

dynamic systemsdynamic

dynamic programming

nonlinear systemsnonlinear

nonlinear controlcontrol

pid controlflexible structures

supervisory controldiscrete event systems

petri netsnetworknetworks

flow

sensitivity analysissensitivity

risk sensitivereduced order

orderreduction

closed loop systemclosed loop

gain schedulingusing

spacecraftattitude control

networked control systemschannelnetwork

controlfuzzy logic

fuzzy control

sliding mode controlvariable structure

sliding modepart

normal formimpulse response

signalssignal

compensationscheduling

policypolicies

usingpresentedself tuning

large scale systemsdecentralized control

interconnected systemsfault detectionfault diagnosis

fault detection isolation

multiplemanufacturing systems

production

systemsobtainedresults

high orderhigh performance

high gaintracking

tracking controltrajectoryreal time

implementationreal time systems

controlcontrol systemscontrol system

power systempower systems

control

controlinduction motorsinduction motor

dynamicsdynamical systems

synchronizationsensorstarget

mobile robots

responseusing

change

neural networksneural network

line

aircraftquantitative feedback theory

control

knownunknown

unknown inputsmodelingintegrated

models

multi agent systemsagents

consensus

developmenttechnologyresearch

processprocesses

process control

modelmodel predictive control

predictive control

controlenginefuel cell

vehiclecontrol

vehicles

problemstructured singular value

selection

extendedstandard

extended kalman filter

non linear systemsnon linear

non minimum phase systems

adaptive controladaptive

adaptation

optimalcontrol

adaptivecontrol

multi-agentsystems

robustcontrol

In co

llabo

ratio

n with

Stac

y Reb

ich H

espa

nha

The Geography of Control

1147.502295.003442.504590.00 cluster 1cluster 2cluster 3cluster 4cluster 5cluster 6cluster 7cluster 8cluster 9cluster 10cluster 11cluster 12cluster 13cluster 14cluster 15cluster 16cluster 17cluster 18cluster 19cluster 20

Number of document citations per cluster (20 document clusters)optimal control




commentset al

d



delaydelays

delay systems


parameter


uncertain systems

approachbasedusing

designdesigns



npm

usingconstantregion



particularclass



formvia

original



l 1matrix

matriceseigenvalues






shownlimit cycles

conditionscondition



given


fast


h 2structure



stablegivendual





formulation






datainformation

based

givensuch







approximations


output regulation



filterkalman filter



solutionsolutions

riccati equation

state spacestate


study


stabilization



observerobservers


algorithm based

generalizedconcept

representationnoise


controlcontrolled

control systems








ilc

inputinput output




usingsystemssimple

minimum variance





newnew approach

presented


dynamic programming






flow



orderreduction





controlfuzzy logic

fuzzy control


sliding modepart


signalssignal


policypolicies






production



high gaintracking





control




mobile robots

responseusing

change


line


control

knownunknown


models


consensus


processprocesses

process control


predictive control


vehiclecontrol

vehicles


selection

extendedstandard





adaptation

optimalcontrol

robustcontrol

adaptivecontrol

multi-agentsystems

delaystime-varying

roboticspassivityhybrid

systems

slidingmodes

normalforms

In co

llabo

ratio

n with

Stac

y Reb

ich H

espa

nha

165.5331496.5662 <=1

Distribution of Spong, M citationsoptimal control




commentset al

d



delaydelays

delay systems


parameter


uncertain systems

approachbasedusing

designdesigns



npm

usingconstantregion



particularclass



formvia

original



l 1matrix

matriceseigenvalues






shownlimit cycles

conditionscondition



given


fast


h 2structure



stablegivendual





formulation






datainformation

based

givensuch







approximations


output regulation



filterkalman filter



solutionsolutions

riccati equation

state spacestate


study


stabilization



observerobservers


algorithm based

generalizedconcept

representationnoise


controlcontrolled

control systems








ilc

inputinput output




usingsystemssimple

minimum variance





newnew approach

presented


dynamic programming






flow



orderreduction





controlfuzzy logic

fuzzy control


sliding modepart


signalssignal


policypolicies






production



high gaintracking





control




mobile robots

responseusing

change


line


control

knownunknown


models


consensus


processprocesses

process control


predictive control


vehiclecontrol

vehicles


selection

extendedstandard





adaptation

The Geography of Mark Spong

adaptivecontrol

multi-agentsystems

delaystime-varying

roboticspassivity

hybridsystems

slidingmodesnormal

forms

robustcontrol

In co

llabo

ratio

n with

Stac

y Reb

ich H

espa

nha

The Plan...

Motivation for large-scale games

Zero-Sum Games, Security Policies

Randomized Algorithms for Games (SSP Algorithm)

Case Study in Network Security (time permiting...)

Path Planning Games: Hide & Seek

?

?

?

?

?

?

??

?

??

What is the “best” way to hide a treasure among N

hiding places in the plane ?

Path Planning Games: Hide & Seek

?

?

?

?

?

?

??

?

??



Formulate problem as gameP1 selects hiding place (N locations)P2 selects robot path (N! paths)

Zero-sum gameP1 wants to maximize time to reach treasureP2 has opposite objective

Saddle-point is a mixed policy

P1 places treasure based on “optimal” distributionP2 selects path based on “optimal” distribution

Which???

Network Security

Cyber Situation Awareness ARO MURI:Protecting the cyber infrastructure that supports “missions”

processing an online purchase

monitoring a critical infrastructure

managing the admissions process

at a universityCyber Awareness Questions:

What is in the impact of a cyber-asset vulnerability (known or unknown)?What is in the impact of a particular counter measure (e.g., firewall control)?

International Capture the Flag (iCTF):Distributed, wide-area security exercise to test the security skills of the participantsHeld yearly since 2003, under the organization of Prof. Giovanni Vigna @ UCSB2010 edition involved 72 teams from around the world, over 900 participants

Cybaware

Service'1'

Internal'Network'

…'

VPN'server'

72'teams'from'16'countries,'900'students'playing'live:''the'UCSB'iCTF'is'the'world’s'largest'hacking'compeKKon!''

Firewall/IDS'

Service'2' …' Service'10' Botnet'C&C'

The'Bank'

ScoreBot'

Briber'

Flag'Submission'

.'.' .'

.'

1st'Prize:'$1,000'Sponsored'by''

Challenges'

ScoreBoard'

LityaLeaks'

iCTF*Architecture*

2010 iCTF Game

services S3, S7

services S4, S5

services S0, S2service S6

service S1

services S6, S2

service S9service S8

services S0, S1services S3, S9

service S2

service S0service S1services S3, S8

services S2, S3

Key ingredients1. Game objective & Actors

defender runs web server and collects revenue (when firewalls open) and bribes (when firewalls closed)attacker collects revenue by compromising active services

2. Players’ Actionsdefender controls firewallattacker explores service vulnerabilities

3. Information Structuredefender knows current status of several missions and services activeattacker has access to full state information (option I) or partial state information (option II)

Problem statistics:over 7800 distinct mission states (defender observations)over 2500 distinct observations available to the attackerdefender can choose among about 102527 distinct policiesattacker can choose among 10756 – 102616 distinct policies, depending on attacker's level of expertise

Randomized Search (H&S)

?

?

?

?

?

?

??

?

??



P1 selects robot path (N! paths)P2 selects hiding place (N locations)

Randomized algorithms:When the exhaustive exploration of a combinatorial decision tree is not possible…

explore a random subset of it

Motivation for this work:How can a player “protect” herself from an opponent engaged in random exploration?

2

6666664

...

. . . aij . . .

...

3

7777775

Zero-Sum Matrix Games

Two players:P1 - minimizerP2 - maximizer

Each player selects a policy:S1 - set of available policies for P1S2 - set of available policies for P2

All possible game outcomes can be encoded in a matrix (2D-array),jointly indexed by the actions of the players

P1‘s policy

P2‘s policy

game outcome whenP1 selects policy iP2 selects policy j

We are interested in games in which, at least one (possibly both)

dimensions of this matrix are very large.

Mixed security level for P1 (minimizer): V – min

ymax

zEraijs “ min

ymax

zy1Az

Randomized Search in Matrix Games

?

?

?

?

?

?

??

?

??

Suppose P2 (searcher) only considers an (unknown) random subset of the N! pathsand (somehow) selects a policy based on those...

Can P1 (hider) guarantee some minimum search time 1. without knowing which random subset was chosen2. with a small computational effort ?

P2

2

66664

3

77775P1‘s

policy

P2‘s policy

A2

Sampled-Security Policy Algorithm (SSP)

1. Player P1 randomly selects a submatrix of the overall game2

66664

3

77775

Player P1

2. P1 solves the corresponding subgame (as if it was the whole game) and computesmixed security level V1 corresponding security policy y*

A1

y⇤

3. P1 plays y* against P2’s policy z*

in very large games, submatrix A1 will likely not overlap the

matrix A2 used by P2

A2

2

66664

3

77775Player P2

P1‘s policy

P2‘s policy

Suppose P2 only considers an (unknown) random subset of its policies to compute a (mixed) policy z*...

Sampled-Security Policy Algorithm (SSP)

1. Player P1 randomly selects a submatrix of the overall game2

66664

3

77775

Player P1

2. P1 solves the corresponding subgame (as if it was the whole game) and computesmixed security level V1 corresponding security policy y*

A1

y⇤

3. P1 plays y* against P2’s policy z*

in very large games, submatrix A1 will likely not overlap the

matrix A2 used by P2

A2

2

66664

3

77775Player P2

Suppose P2 only considers an (unknown) random subset of its policies to compute a (mixed) policy z*...

Because of independent subsampling, a player can now be unpleasantly surprised:

outcome larger than minimizer expectedbased on its submatrix A1

Ey˚,z˚ raijs ° V1

Security with High Probability

Definition: The SSP algorithm is ϵ−secure for P1 (minimizer) with confidence 1−δ if

outcome larger than what P1 expected(by more than∊)

PpEy˚,z˚ raijs ° V1 ` ✏q § �

Probabilistic notion of security:probability of (unpleasant) surprises should be below a pre-specified boundwith more computational power, one can demand lower prob. of surprise

“Surprise” can arise because:our policy y* is actually badour security value V1 is overly optimisticP2 was lucky in the selection of z*






avoidable with very high probability

not so easily under our control

outcome larger than what P1 expected(by more than∊)




outcome larger than P1 expected(by more than∊)


Definition: A particular policy y* and value V1 is ϵ−secure for P1 (minimizer) with confidence 1−δ if

PpEy˚,z˚ raijs ° V1 ` ✏ | y˚, V1q § �

“Surprise” only because:P2 was lucky in the selection of z*

SSP Key Result

A2

2

66664

3

77775Player P2

m2 cols

n2 cols

Theorem: The SSP algorithm is ϵ = 0 − secure for P1 (minimizer) with confidence 1−δ, for

Conversely, to obtain desired confidence level δ, suffices to select

“fat” sampling for A1

⇓test more options for opponent than own

(by appropriate ratio)

2

66664

3

77775A1

Player P1

n1 cols

m1 cols

� =m1n2

n1

n1

m1• n2

�° 1

P1‘s policy

P2‘s policy

SSP Key Result

A2

2

66664

3

77775Player P2

m2 cols

n2 cols

2

66664

3

77775A1

Player P1

n1 cols

m1 cols

P1‘s policy

Theorem: The SSP algorithm is ϵ = 0 − secure for P1 (minimizer) with confidence 1−δ, for

Conversely, to obtain desired confidence level δ, suffices to select

“fat” sampling for A1

⇓test more options for opponent than own

(by appropriate ratio)

� =m1n2

n1

n1

m1• n2

�° 1

Game-independent boundsvalid for any gameindependent of the size of the game

Bounds on relative computationrequired size of my sample depends onsize of opponents’ sample (n2)the more I search for a good solution (large m1), the more options need to consider for opponent (large n1)

P2‘s policy

Security with High Probability (recall)


outcome larger than P1 expected(by more than∊)


Definition: A particular policy y* and value V1 is ϵ−secure for P1 (minimizer) with confidence 1−δ if

PpEy˚,z˚ raijs ° V1 ` ✏ | y˚, V1q § �

“Surprise” only because:P2 was lucky in the selection of z*


SSP Key Result (take 2)

A2

2

66664

3

77775Player P2

m2 cols

n2 cols

Theorem: With probability larger than 1−β, the SSP policy y* and value V1 are ϵ = 0 − secure for P1 (minimizer) with confidence 1−δ, for

2

66664

3

77775A1

Player P1

n1 cols

m1 cols

additional term


only with probability β (can be made extremely small ～ 10-9 with small computational cost)

with probability δ

n1 •´m1 `

cm1 ln

1

�2

¯n2

�

P1‘s policy

P2‘s policy

2010 iCTF Game

services S3, S7

services S4, S5


service S1

services S6, S2



service S2


services S2, S3

Key ingredients1. Game objective & Actors

defender runs web server and collects revenue (when firewalls open) and bribes (when firewalls closed)attacker collects revenue by compromising active services

2. Players’ Actionsdefender controls firewallattacker explores service vulnerabilities

3. Information Structuredefender knows current status of several missions and services activeattacker has access to full state information (option I) or partial state information (option II)

Problem statistics:over 7800 distinct mission states (defender observations)over 2500 distinct observations available to the attackerdefender can choose among about 102527 distinct policiesattacker can choose among 10756 – 102616 distinct policies, depending on attacker's level of expertise

SSP-based Analysis

services S3, S7

services S4, S5


service S1

services S6, S2



service S2


services S2, S3

BaselineNo attackFirewall always openAll revenue from bot

Defender receives avg. 314 units for completion of 4 missions

SSP-based Analysis

services S3, S7

services S4, S5


service S1

services S6, S2



service S2


services S2, S3


Attacks under Option I (ambiguous state information)Different levels of attacker sophisticationBribes not allowed

Level of attacker sophistication

# units received by Defender for 1 round of missions

[Option I, no bribes]

no service vulnerable (baseline) 314

S2 (vulnerable to 38 teams) 240

S2, S6, S9 (vulnerable to at least 6 team) 79

S0, S2, S4, S6, S7, S8, S9 (vulnerable to at least 1 team) 11

all services vulnerable 11

Increasing level of attacker sophistication Firewall closed

whenever a vulnerable service is active


SSP-based Analysis

services S3, S7

services S4, S5


service S1

services S6, S2



service S2


services S2, S3


Attacks under Option II (full state information)Different levels of attacker sophisticationBribes allowed


# units received by Litya for 1 round of missions


# units received by Litya for 1 round of missions[Option I, with bribes]

# units received by Litya for 1 round of missions[Option II, with bribes]

no service vulnerable (baseline) 314 314 314

S2 (vulnerable to 38 teams) 240 240 138

S2, S6, S9 (vulnerable to at least 6 team) 79 79 43

S0, S2, S4, S6, S7, S8, S9 (vulnerable to at least 1 team) 11 -738 -1327

all services vulnerable 11 -848 -1917

Increasing level of attacker sophistication


SSP-based Analysis

services S3, S7

services S4, S5


service S1

services S6, S2



service S2


services S2, S3


Attacks under Option II (full state information)Different levels of attacker sophisticationBribes allowed

Litya receives avg. 314 units for completion of all 4 missions


# units received by Litya for 1 round of missions


# units received by Litya for 1 round of missions[Option I, with bribes]

# units received by Litya for 1 round of missions[Option II, with bribes]

no service vulnerable (baseline) 314 314 314

S2 (vulnerable to 38 teams) 240 240 138

S2, S6, S9 (vulnerable to at least 6 team) 79 79 43

S0, S2, S4, S6, S7, S8, S9 (vulnerable to at least 1 team) 11 -738 -1327

all services vulnerable 11 -848 -1917

Increasing level of attacker sophistication

Provides Cyber-security office estimates of mission success Takes into account the effect of attacks & counter measuresResponses as a function of attacker sophisticationAbility to play what-if scenarios (vulnerabilities, information, etc.)

Conclusions

Large matrix games are fun!What I have covered:

Basic probability guarantees of randomized sampling for games

Case study in network security

What I have NOT covered:Can we determine the security level of an arbitrary policy obtained by a method other than SSP? Yes

Mistmatch between the players’ distributions if we sample “better” than opponent, no change in resultsif we sample “worse” than opponent, degradation in confidence level δ(but recoverable with more sampling)

Can we improve upon these bounds? Unclear

Multi-core parallel implementations

Other applications...

Conclusions

Thank You!

Large matrix games are fun!What I have covered:

Basic probability guarantees of randomized sampling for games

Case study in network security

What I have NOT covered:Can we determine the security level of an arbitrary policy obtained by a method other than SSP? Yes

Mistmatch between the players’ distributions if we sample “better” than opponent, no change in resultsif we sample “worse” than opponent, degradation in confidence level δ(but recoverable with more sampling)

Can we improve upon these bounds? Unclear

Multi-core parallel implementations

Other applications...

randomized methods for network security games · robust control robust uncertain systems approach...

Documents