discounting the future in systems theory chess review may 11, 2005 berkeley, ca luca de alfaro, uc...
Post on 18-Dec-2015
213 views
TRANSCRIPT
![Page 1: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/1.jpg)
Discounting the Future in Systems Theory
Chess ReviewMay 11, 2005Berkeley, CA
Luca de Alfaro, UC Santa Cruz
Tom Henzinger, UC Berkeley
Rupak Majumdar, UC Los Angeles
![Page 2: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/2.jpg)
a cb
A Graph Model of a System
![Page 3: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/3.jpg)
a cb
Property c ("eventually c")
![Page 4: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/4.jpg)
a cb
Property c ("eventually c")
c … some trace has the property c
![Page 5: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/5.jpg)
a cb
Property c ("eventually c")
c … some trace has the property c c … all traces have the property c
![Page 6: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/6.jpg)
graph
Richer Models
ADVERSARIAL CONCURRENCY:
game graph
FAIRNESS: -automaton
PROBABILITIES: Markov decision process
Stochastic game
Parity game
![Page 7: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/7.jpg)
a cb
1,1 2,2
1,2 2,1
1,1 1,2 2,2
2,1
Concurrent Game
player "left" player "right"
-for modeling open systems [Abramsky, Alur, Kupferman, Vardi, …] -for strategy synthesis ("control") [Ramadge, Wonham, Pnueli, Rosner]
![Page 8: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/8.jpg)
a cb
1,1 2,2
1,2 2,1
1,1 1,2 2,2
2,1
hhleftii c … player "left" has a strategy to enforce c
Property c
![Page 9: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/9.jpg)
a cb
1,1 2,2
1,2 2,1
1,11,2 2,2
2,1
hhleftii c … player "left" has a strategy to enforce c
left c … player “left" has a randomized strategy to enforce c
Property c
Pr(1): 0.5 Pr(2): 0.5
![Page 10: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/10.jpg)
Qualitative Models
Trace: sequence of observations
Property p: assigns a reward to each trace boolean rewards
Model m: generates a set of traces(game) graph
Value(p,m): defined from the rewards of the generated traces 9 or 8 (98)B
![Page 11: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/11.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Stochastic Game
![Page 12: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/12.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c ?
Property c
![Page 13: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/13.jpg)
Semi-Quantitative Models
Trace: sequence of observations
Property p: assigns a reward to each trace boolean rewards
Model m: generates a set of traces(game) graph
Value(p,m): defined from the rewards of the generated traces sup or inf (sup inf)[0,1] µ R
![Page 14: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/14.jpg)
Class of properties p over traces
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
A Systems Theory
![Page 15: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/15.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
-regular properties
-calculus bisimilarity
GRAPHS
A Systems Theory
Class of properties p over traces
![Page 16: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/16.jpg)
Transition Graph
Q states : Q 2Q transition relation
![Page 17: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/17.jpg)
Graph Regions
Q states : Q 2Q transition relation
= [ Q ! B ] regions 9pre, 8pre:
8pre(R)
9pre(R)
R µ Q
9
8
![Page 18: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/18.jpg)
Graph Property Values: Reachability
R
Given RµQ, find the states from which some trace leads to R.
R
![Page 19: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/19.jpg)
R
. . .
RR
[ pre(R)
R [ pre(
R) [ pre2(R
)
R = ( X) (R Ç 9pre(X))
Given RµQ, find the states from which some trace leads to R.
Graph Property Values: Reachability
![Page 20: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/20.jpg)
Q states l, r moves of both players : Q l r Q transition function
Concurrent Game
![Page 21: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/21.jpg)
Q states l, r moves of both players : Q l r Q transition function
= [ Q ! B ] regions lpre, rpre:
q lpre(R) iff (l l ) (r r) (q,l,r) R
Game Regions
lpre(R)R µ Q
1,1
2,*
1,2
![Page 22: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/22.jpg)
R
Game Property Values: Reachability
left R
Given RµQ, find the states from which player "left" has a strategy to force the game to R.
![Page 23: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/23.jpg)
R
. . .
left RR
[ lpre(R)
R [ lpre(R
) [ lpre2(R)
left R = ( X) (R Ç lpre(X))
Given RµQ, find the states from which player "left" has a strategy to force the game to R.
Game Property Values: Reachability
![Page 24: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/24.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
-regular properties
(lpre,rpre) fixpoint calculus alternating bisimilarity [Alur, H, Kupferman, Vardi]
GAME GRAPHS
An Open Systems Theory
Class of winning conditions p over traces
![Page 25: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/25.jpg)
Algorithm for computing
Value(p,m) over models m
-regular properties
(lpre,rpre) fixpoint calculus
GAME GRAPHS
An Open Systems Theory
Every deterministic fixpoint formula computes Value(p,m), where p is the linear interpretation [Vardi] of .
Class of winning conditions p over traces
hhleftiiR
( X) (R Ç lpre(X))
![Page 26: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/26.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
(lpre,rpre) fixpoint calculus
alternating bisimilarity
GAME GRAPHS
An Open Systems Theory
Two states agree on the values of all fixpoint formulas iff they are alternating bisimilar [Alur, H, Kupferman, Vardi].
![Page 27: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/27.jpg)
Q states l, r moves of both players : Q l r Dist(Q) probabilistic transition function
Stochastic Game
![Page 28: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/28.jpg)
Q states l, r moves of both players : Q l r Dist(Q) probabilistic transition function
= [ Q ! [0,1] ] quantitative regions
lpre, rpre:
lpre(R)(q) = (sup l l ) (inf r r) R((q,l,r))
Quantitative Game Regions
![Page 29: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/29.jpg)
Q states l, r moves of both players : Q l r Dist(Q) probabilistic transition function
= [ Q ! [0,1] ] quantitative regions
lpre, rpre:
lpre(R)(q) = (sup l l ) (inf r r) R((q,l,r))
Quantitative Game Regions
9 8
B
![Page 30: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/30.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c :
( X) (c Ç lpre(X))
0 10
Ç = pointwise max
![Page 31: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/31.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c :
( X) (c Ç lpre(X))
0 11
Ç = pointwise max
![Page 32: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/32.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c :
( X) (c Ç lpre(X))
0.8 11
Ç = pointwise max
![Page 33: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/33.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c :
( X) (c Ç lpre(X))
0.96 11
Ç = pointwise max
![Page 34: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/34.jpg)
a cb
1
1
2
2leftright
a: 0.6 b: 0.4
a: 0.1 b: 0.9
a: 0.5 b: 0.5
a: 0.2 b: 0.8
1
1
2
2leftright
a: 0.0 c: 1.0
a: 0.7 b: 0.3
a: 0.0 c: 1.0
a: 0.0 b: 1.0
Probability with which player "left" can enforce c :
( X) (c Ç lpre(X))
1 11
Ç = pointwise max
In the limit, the deterministic fixpoint formulas work for all -regular properties [de Alfaro, Majumdar].
![Page 35: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/35.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
-regular properties
quantitative fixpoint calculus quantitative bisimilarity [Desharnais, Gupta, Jagadeesan,
Panangaden]
MARKOV DECISION
PROCESSES
A Probabilistic Systems Theory
Class of properties p over traces
![Page 36: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/36.jpg)
Algorithm for computing
Value(p,m) over models m
quantitative -regular properties
quantitative fixpoint calculus
Class of properties p over traces
Every deterministic fixpoint formula computes expected Value(p,m), where p is the linear interpretation of .
A Probabilistic Systems Theory
MARKOV DECISION
PROCESSES
max expected value of satisfying R
( X) (R Ç 9pre(X))
![Page 37: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/37.jpg)
Qualitative Bisimilarity
e: Q2 ! {0,1} … equivalence relation
F … function on equivalences
F(e)(q,q') = 0 if q and q' disagree on observations = min { e(r,r’) | r2 9pre(q) Æ r’2 9pre(q’) }
else
Qualitative bisimilarity … greatest fixpoint of F
![Page 38: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/38.jpg)
Quantitative Bisimilarity
d: Q2 ! [0,1] … pseudo-metric ("distance")
F … function on pseudo-metrics
F(d)(q,q') = 1 if q and q' disagree on observations ¼ max of supl infr d((q,l,r),(q',l,r)) supr infl d((q,l,r),(q',l,r)) else
Quantitative bisimilarity … greatest fixpoint of F
Natural generalization of bisimilarity from binary relations to pseudo-metrics.
![Page 39: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/39.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
quantitative fixpoint calculus
quantitative bisimilarity
A Probabilistic Systems Theory
MARKOV DECISION
PROCESSES
Two states agree on the values of all quantitative fixpoint formulas iff their quantitative bisimilarity distance is 0.
![Page 40: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/40.jpg)
Great BUT …
1 The theory is too precise.
Even the smallest change in the probability of a transition can cause an arbitrarily large change in the value of a property.
2 The theory is not computational.
We cannot bound the rate of convergence for quantitative fixpoint formulas.
![Page 41: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/41.jpg)
Solution: Discounting
Economics:
A dollar today is better than a dollar tomorrow.
Value of $1.- today: 1 Tomorrow: for discount factor 0 < < 1
Day after tomorrow: 2
etc.
![Page 42: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/42.jpg)
Solution: Discounting
Economics:
A dollar today is better than a dollar tomorrow.
Value of $1.- today: 1 Tomorrow: for discount factor 0 < < 1
Day after tomorrow: 2
etc.
Engineering:
A bug today is worse than a bug tomorrow.
![Page 43: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/43.jpg)
Discounted Reachability
Reward ( c ) = k if c is first true after k transitions
0 if c is never true
The reward is proportional to how quickly c is satisfied.
![Page 44: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/44.jpg)
a cb
Discounted Property c
c
1
2
![Page 45: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/45.jpg)
a cb
Discounted Property c
c
1
2
Discounted fixpoint calculus: pre() ¢ pre()
![Page 46: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/46.jpg)
Fully Quantitative Models
Trace: sequence of observations
Property p: assigns a reward to each trace real reward
Model m: generates a set of traces(game) graph
Value(p,m): defined from the rewards of the generated traces sup or inf (sup inf)[0,1] µ R
![Page 47: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/47.jpg)
Discounted Bisimilarity
d: Q2 ! [0,1] … pseudo-metric ("distance")
F … function on pseudo-metrics
F(d)(q,q') = 1 if q and q' disagree on observations ¼ max of supl infr d((q,l,r),(q',l,r)) supr infl d((q,l,r),(q',l,r)) else
Quantitative bisimilarity … greatest fixpoint of F
¢
![Page 48: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/48.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
discounted-regular properties
discounted fixpoint calculus discounted bisimilarity
A Discounted Systems Theory
STOCHASTIC GAMES
Class of winning rewards p over traces
![Page 49: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/49.jpg)
Algorithm for computing
Value(p,m) over models m
discounted-regular properties
discounted fixpoint calculus
A Discounted Systems Theory
STOCHASTIC GAMES
Every discounted deterministic fixpoint formula computes Value(p,m), where p is the linear interpretation of .
Class of expected rewards p over traces
max expected reward R
achievable by left player
( X) (R Ç ¢ lpre(X))
![Page 50: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/50.jpg)
Algorithm for computing
Value(p,m) over models m
Distance between models w.r.t.
property values
discounted fixpoint calculus
discounted bisimilarity
A Discounted Systems Theory
STOCHASTIC GAMES
The difference between two states in the values of discounted fixpoint formulas is bounded by their discounted bisimilarity distance.
![Page 51: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/51.jpg)
Discounting is Robust
Continuity over Traces:
Every discounted fixpoint formula defines a reward function on traces that is continuous in the Cantor metric.
Continuity over Models:
If transition probabilities are perturbed by , then discounted bisimilarity distances change by at most f().
Discounting is robust against effects at infinity, and against numerical perturbations.
![Page 52: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/52.jpg)
Discounting is Computational
The iterative evaluation of an -discounted fixpoint formula converges geometrically in .
(So we can compute to any desired precision.)
![Page 53: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/53.jpg)
Discounting is Approximation
If the discount factor tends towards 1, then we recover the classical theory:
• lim! 1 -discounted interpretation of fixpoint formula = classical interpretation of
• lim! 1 -discounted bisimilarity = classical (alternating; quantitative) bisimilarity
![Page 54: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/54.jpg)
Further Work
• Exact computation of discounted values of temporal formulas over finite-state systems [de Alfaro, Faella, H, Majumdar, Stoelinga].
• Discounting real-time systems: continuous discounting of time delay rather than discrete discounting of number of steps [Prabhu].
![Page 55: Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,](https://reader031.vdocument.in/reader031/viewer/2022032800/56649d255503460f949fc2de/html5/thumbnails/55.jpg)
Conclusions
• Discounting provides a continuous and computational approximation theory of discrete and probabilistic processes.
• Discounting captures an important engineering intuition.
"In the long run, we're all dead." J.M. Keynes