robust adversarial risk analysis: a level-k approach
DESCRIPTION
Adversarial risk analysis is an active and important area of decision analytic research. Both single-actor decision analysis and multiple-actor game theory have been applied to this problem, with game theoretic methods being particularly popular. While game theory models do explicitly capture strategic interactions between attackers and defenders, two of the key assumptions—decision making based on subjective expected utility maximization and common knowledge of rationality—are known to be descriptively inaccurate in some situations. This paper addresses these shortcomings by proposing, formulating, and illustrating the application of robust optimization methodologies to a level-k game theory model for adversarial risk analysis. Level-k game theory provides a practical method for modeling bounded rationality. Robust optimization provides an alternative way to model the actions of conservative players facing “deep” uncertainties about their environment—uncertainties that are possible to bound but which are difficult or impossible to represent using probability distributions. Our approach thus combines level-k and robust optimization insights to provide a computationally tractable model of boundedly rational players who are faced with significant and difficult to quantify uncertainties.TRANSCRIPT
Robust Adversarial Risk Analysis: A level-k approach
LAURA MCLAY
Virginia Commonwealth UniversityStatistical Sciences & Operations
CASEY ROTHSCHILD
Wellesley CollegeEconomics
SETH GUIKEMAJohns Hopkins University
Geography and Environmental Engineering
Paper to appear in Decision Analysis
This paper was supported by research funded by DHS S&T under contract HSHQDC-10-C-00105.
Summary
• Robust optimization methodologies can be combined with Adversarial Risk Analysis (ARA) – Examines how an attacker is likely to process partial information
about defensive postures– Considers defenders with various levels of strategic information – Maintains computational efficiency of prior ARA algorithms
• Models actions of conservative players facing “deep” uncertainties about their environment– Uncertainties that are possible to bound – Uncertainties that are difficult or impossible to represent using
probability distributions • Apply our approach to a Defender-Attacker game
Motivation
• National Academy of Science 2008 report concerned that the Bioterrorism Risk Assessment program fails to adequately model adaptive U.S. adversaries
• Apply advanced analytical methods to model an intelligent adversary seeking to cause harm in situations characterized by imperfect information
• Expected utility maximization may not always be an appropriate choice paradigm to describe adversary behavior
Robust optimization (RO)
• Concerned with uncertain data in optimization models• RO finds the optimal solution that satisfies all constraints
given any possible distribution of constraint data• RO methods allow uncertainties in input parameters to be
distribution-free• RO models can be solved efficiently by standard algorithms
if uncertain inputs have a structure that does not make the problem harder– Reformulate semi-infinite RO models as standard math
programs• RO models are a game against nature
– Zero-sum, perfect and complete information
Robust players
• Robust players are more “conservative” • A robust paradigm is not a minimax paradigm
– Both minimax and robust paradigms are distribution-free and optimize over the worst-case scenario
– Minimax: player minimizes the maximum possible loss over both the uncertainty set and opponent actions
– Robust: allows player to optimally respond to opponent while robust against non-strategic uncertainties
• Game against nature, where nature sets worst possible values of uncertain inputs with perfect information
Robust games
• Multiple players, each of whom is a robust decision maker – “nature” is a notional player
• Can be modeled as 1. stochastic programming models that consider data uncertainty
within an optimization problem 2. stochastic games, where players determine the transition
probabilities• Equilibria exist for robust games under a wide range of assumptions• Rectilinearity of the uncertainty sets does not make robust games
“harder” than non-robust games– No dependencies between the uncertainty sets
Aghassi, M., D. Bertsimas. 2006. Robust game theory, Mathematical Programming 107, 231 – 273.Kardes, E. Ordóñez, F., Hall, R.W. 2011. Discounted robust stochastic games and an application to queuing
control, Operations Research 59(2), 365 – 382.
Level-k games
• Models players who are boundedly rational who play against a level-(k-1) opponent
• k = level of strategic sophistication– k = 0: player acts randomly – k = 1: player acts optimally but is not strategic– k = 2: player acts optimally by assuming a level-1 opponent– k = : player is rational
• Level-k games do not necessarily converge to a Nash equilibrium as k
• Level-k is a good heuristic– Accounts for behavior in a wide range of experiments
• Level-k have computational advantages– Solution is recursive
Approach
• Extend a level-k Bayesian ARA solution algorithm for a sequential Defender-Attacker model with imperfect observability of the Defender’s move*
• Two boundedly rational players:– Defender (Daphne) chooses– Attacker (Apollo) chooses
• Daphne assesses Apollo’s level of strategic sophistication k• Daphne’s beliefs of Apollo’s preferences
– CDF with expected utility• What Daphne believes Apollo believes about her preferences
– CDF with expected utility• Daphne’s true preferences
Rios Insua, I., J. Rios, and D. Banks. 2009. Adversarial risk analysis, Journal of the American Statistical Association 104, 841-854.Rothschild, C., L.A. McLay, S. Guikema. 2012. Adversarial risk analysis: A level-k approach. Risk Analysis (to appear).
DdAa
*D
A
D)( AAF
)( DDF
);,( AA da
( , ; )D Da d
Approach, cont’d.
• Uncertain information structure about Daphne’s decisions– Apollo’s signal with probabilities
• Daphne assesses Apollo’s decisions– Probability of selecting actions
• (Daphne estimates of) Apollo’s priors
• Daphne’s expected utility
)|(~ dq
Aa
DDDD dadapE );,()|(~ *
Dddp A ),()|( apD
Robust ARA• Robustness with respect to the (finite) information structure
– These are the decisions of a notional player (nature) who selects them to be as bad as possible for a player
• and denote the selections from Daphne and Apollo’s level-j decision problems, respectively– independent whenever
– Treats each player at each level as being independently robust
– Different levels of same player at different levels have different realizations of the uncertain parameters
• Interval parameter uncertainty:and lie in
– can be computed for each stage in the game by solving a simple linear program
)|(~ dq
( | )jDq d
)|(, dq jaA
),(),( mj
)|(),|( dUdL ( | )jDq d )|(, dq j
aA
ARA algorithm: a level-k approach7 steps
1. Assess preferences and basic perceptions for Daphne (and the information structure )
2. Assess Daphne’s perceptions of Apollo via the distribution
3. Assess Daphne’s beliefs about Apollo’s perceptions of Daphne via the distribution
4. Assess Daphne’s strategic reasoning level-k (with k > 1)
5. “Initiate” the recursion via uniform level-1 priors:a)
b)
*D
)|(~ dq )( AAF
)( DDF
D1
)(1 dpA Dd
A1
)|(1 apD
Aa
ARA algorithm, cont’d.6. Solve recursively for priors up to level-k: For j=1,…,k – 1
a) Solve for Daphne’s level-(j+1) priors:
• Randomly draw a large number of from
• For each draw, compute Apollo’s optimal action assuming Apollo is level-j:
• Estimate via the fraction of with
• Set
[non-robust case]
A AF
*
'
( ) ( | )( | ) arg max ( , ; ) ,( ') ( | ')
jA
j A a A Ajd D A
d D
p d q da a dp d q d
1( | ) ( | ), .jD Dp a p a
( | ), ,Dp a .)|(* aa A
A
,
,
* ,'( | ),
,
( ) ( | )( , ; )
( ') ( | ')( | ) arg max min
subject to ( | ) ( | ) ( | ),jA a
j jA A a
A Aj jd D A A a
j A a d Dq d d Dj
A a
p d q da d
p d q da
L d q d U d d D
ARA algorithm, cont’d.
6. b) Solve for Apollo’s level-(j+1) priors:
• Randomly draw a large number of from
• For each draw, compute Daphne’s optimal action assuming Daphne is level-j:
• Estimate via the fraction of the with
• Set
[non-robust case]
D DF
*( ) arg max ( | ) ( | ) ( , ; )jj D d D D D D
a Ad q d p a a d
)(~ dpA D dd D )(* 1( | ) ( | ), .j
D Dp a p a
( | ),
*
min ( | ) ( | ) ( , ; )
( ) arg max subject to ( | ) 1
( | ) ( | ) ( | ),
jD
j jD D D Dq d a A
jj D d D D
jD
q d p a a d
d q d
L d q d U d
ARA algorithm, cont’d.
7. Solve for (level-k) Daphne’s optimal action:
[non-robust case]
* *( ) arg max ( | ) ( | ) ( , ; )kk D d D D D D
a Ad q d p a a d
( | ),
*
min ( | ) ( | ) ( , ; )
( ) arg max subject to ( | ) 1
( | ) ( | ) ( | ),
kD
j kD D D Dq d a A
kk D d D D
kD
q d p a a d
d q d
L d q d U d
Example
• Apollo can either (actively try to) initiate a Smallpox attack (a1) or No Attack (a2) with A={a1,a2}
• Daphne can either install an array of smallpox Detectors (d1) or No Detectors (d2), with D={d1,d2}
• Information structure , and if
• The parameters qi are uncertain, with a range of possible values [0.3, 0.7]
Compare three cases:• Baseline robust model = both Daphne and Apollo are robust
decision-makers • Hybrid model = Daphne is a robust decision-maker but Apollo is not• Non-robust model = Neither Daphne nor Apollo are robust
decision-makers
1 2{ , , } ( | ) 1i iq d q ( | ) 0i jq d ( | )i i iq d q i j
Example: Utilities
• Daphne with real values– = cost of installing detectors
– = benefit of having Smallpox detectors in place if an attack is launched
– Daphne’s beliefs about Apollo’s beliefs about her preferences taken to be uniformly distributed on [0, 10] x [5, 15]
• Apollo– = direct cost from mounting an attack
– = additional cost if attack occurs in the presence of detectors
– Daphne’s beliefs are uniformly distributed on [0, 20] x [0,10]
None(2) (1)A A 10 (1)A
0 0
10
0
DetectorDetector None
Smallpox
No Attack
);,( DD da );,( AA da
(2) (1) 10D D
(1)D
)2(),1( DD * *( (1), (2)) (4,15)D D
(1)D
(2)D
(2)A
(1)A
(1), (2)A A
Results as a function of k
• Daphne’s optimal baseline robust decision is not to install detectors (k = 2, 3,…, 10 )• Non-robust model: install detectors (k = 4,5,…, 10 )• Hybrid model: install detectors (k = 2, 3,…, 10 )
• Daphne’s baseline robust expected utilities do not unilaterally decrease when compared to the non-robust base case• Solutions to all three models converge to a perfect Bayesian equilibrium with k.
Sensitivity of Daphne’s optimal level-4 decisions to her preference parameters
In baseline robust model, Daphne’s decisions are relatively insensitive to than in the other two models
– This is a result of having lower priors for a non-robust Apollo attackingDaphne’s decisions in the baseline robust and the non-robust models differ in 20
scenariosDaphne’s decisions in the hybrid and the non-robust models differ in 14 scenarios
3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 55
6
7
8
9
10
11
12
13
14
15
D*(1)
D*(
2)
3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 55
6
7
8
9
10
11
12
13
14
15
D*(1)
D*(
2)
3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 55
6
7
8
9
10
11
12
13
14
15
D*(1)
D*(
2)
Baseline robust Hybrid Non-robust
Gray = install detectors, white = do not install detectors
(2)D
Conclusions
• Extend level-k adversarial risk analysis model and algorithm to consider robust optimization methodologies
• A computational examples indicates that– optimal decisions can be significantly different when considering
robust decision-making as compared to non-robust decision-making– robust decision-making does not always unilaterally decrease the
expected utility of a given agent (vis a vis the parallel model with expected utility maximizers)
– uncertainty with regard to informational requirements set by nature may play out in different, unexpected ways between the players
• Posit that robust level-k approach offers promise for terrorism risk assessment and management– Addresses NAS concerns
• Could be used in a plural modeling approach, since no one model may be effective for modeling adversaries