solving games without determinization nir piterman École polytechnique fédéral de lausanne (epfl)...

Solving Games Without Determinization

Nir Piterman

École Polytechnique Fédéral de Lausanne (EPFL)

Switzerland

Joint work with Thomas A. Henzinger

Nondeterminizing NondeterministicAutomata

Nir Piterman

École Polytechnique Fédéral de Lausanne (EPFL)

Switzerland

Joint work with Thomas A. Henzinger

What?

• Get a nondeterministic automaton with n states.

• Construct a nondeterministic automaton with 2nn2n states.

• Why?

Plan of Talk

• Verification.

• Automata on Infinite Words.

• Synthesis.

• Design Synthesis in Action.

• Our solution.

Verification

• The normal process of development:– Write specifications (informally).– Develop design.– Test.

• Check that the system satisfies the specification.

Reactive Systems

• We are interested in systems that behave rather than compute (CPU, Operating system).

• Main complexity is in maintaining communication with a user / another program / the environment.

• The system has to be ready for every possible input.

• The system maintains behavior forever.

What is Behavior?

• The sequence of states the system passes along a computation.

• Nondeterministic systems / many possible inputs produce many possible behaviors.

• For reactive systems the behavior is infinite.

Automata Theoretic Approach to Verification

• Use automata to reason about systems and specifications.

• Questions like satisfiability and model checking reduce to emptiness of automata.

• Separates logical and algorithmic aspects of problems.

Automata on Infinite Words

• Introduced by Büchi, McNaughton, Elgot, Trakhtenbrot, Rabin, … in the 60s.

• Basically: take the same machine; run it on infinite words.

• In infinite runs there is no last state. Use the set of recurring states.

• Büchi acceptance: the set of recurring states intersects the set of accepting states.

Examples

q0 q1

Examples

q0 q1

Applications

• Satisfiability of S1S [Buc62] and linear time logics.– A linear time formula characterizes sets of

sequences.– Construct an automaton that accepts the set of

models of the formula.– Is the language of the automaton empty?

Applications

• Linear-time model checking [VW94].– A linear time formula characterizes sets of

sequences.– Construct an automaton that accepts all non-

models of the formula.– Consider the intersection of the automaton and the

system.– Is the intersection empty?

Verification

• The normal process of development:– Write specifications (informally).– Develop design.– Test.

• Check that the system satisfies the specification.

• We need a formal way to write specifications: temporal logic.

Specifications

• We formally write specifications using temporal logic.

• We use automata on infinite words as an intermediate tool to reason about specifications.

Synthesis

• Can’t we automatically produce the system from the specification?

• Produce systems that are ensured to work correctly.

Church’s Problem

In 1965 Church posed this problem as:

Given a circuit interface and a behavioral

specification, determine:

1. Does there exist an automaton (circuit) that realizes the specification?

2. Construct an implementing circuit.

Solutions

• Rabin develops the theory of automata on infinite trees [Rab69].

• Büchi and Landweber propose a reduction to infinite duration games [BL69].

• These are the main two solutions up till today.

Synthesis as a Game

• System controls internal variables. Environment controls input.

• Moves of system must match all possible future moves of environment.

• System plays against environment. – System tries to satisfy specification.– Environment tries to falsify specification.

• Success of system determined by the outcome of interaction.

Game Graphs• We represent games as directed graphs.

G=hV,V0,V1,E,v0i

• The vertices are partitioned to those of player 0 (system) and player 1 (environment).

• A play starts with a pebble on v0.

• If the pebble is on v2V0, player 0 chooses an outgoing edge and transfers the pebble.

• If the pebble is on v2V1, player 1 chooses the successor.

Winning Condition• An infinite play is an infinite sequence of states.

• Winning conditions:– Recurrence / persistence in terms of states of the game. – Linear temporal logic or automata on infinite words

over states of the game.

• Does there exist a winning strategy?

• Use the automaton to follow the play and determine the winner?

Use Automaton

• Add one pebble on the automaton.• Move the pebble on the automaton according to the

move in the game.• Decide acceptance according to the automaton.

Environment

System

Game Automaton

Simple Game

1 0 1

Visit finitely many 0’s

Environment

System

Nondeterminism is bad

1 0 1Environment

System

What’s the Problem?

• The opponent chooses between (infinitely) many different paths.

• A guess should match all possible paths.

• Deterministic automata don’t guess!

Determinization

• Need stronger acceptance conditions [Lan69].

• Starting with NBW with n states:– DRW with 22n states [McN66]. – DRW with (12)nn2n states and 2n index [Saf88].– DPW with n2n+2 states and 2n index [Pit06].

• Lower bound nO(n) [Mic88,Yan06]

Back to Games

• Games:– The opponent chooses between many different paths.

– A deterministic automaton enables monitoring the goal of the game.

• Games with LTL/NBW goals:– Convert LTL to NBW, convert NBW to DPW.

– Create product of game and DPW.

• Reasoning about general games reduces to reasoning about parity games.

The End?!

Not really …

In Practice

• Determinization is extremely complex.

Safra’s Construction

• Have a tree of subset constructions.

• Whenever a node (subset) visits F, create a new son with the states in F.

• If a node is removed – flash red light.

• If a node equals its sons – flash green light.

• The Rabin condition has a pair for every node. Node flashes red – bad. Node flashes green – good.

Deterministic State

• Ordered tree.

• Nodes are elements in {1,…,n}.

• Every node is labeled by a subset of the states.

• Every node is colored green, red, or white.

• Unused names are colored red.

Deterministic TransitionThe transition of d is the result of the following

transformations.

• Replace node label by labels of successors (subset construction).

• Spawn new sons with accepting states.

• Move states to ‘best’ nodes.

• Remove empty nodes.

• Nodes that equal their sons colored green.

0,1,3

3 1

1

42

0,1,3

3 1

1

421

1

3

5

0,1,3

3 1

1

4

15

20,1,3

3 1

1

4

0,1,3,4

4 12

1

4

What about your variant?

• Recently, improvement of Safra:– Safra: NBW(n) ! DRW(12nn2n,n)– Variant: NBW(n) ! DPW(n2n+2,2n)

• But: still trees, and everything else.

Or abcdefghij

In Practice

• Determinization is extremely complex.

• First implementation in CIAA05.

OmegaDet [STW05]

In Practice

• Determinization is extremely complex.

• First implementation in CIAA05.

• No way to implement symbolically.

• All or nothing.

• Resort to other solutions.

• Restrict attention to a subset of LTL. – Safety / reachability – linear time [RW89,AMPS98].– Recurrence / persistance – quadratic time [AMPS98].– Boolean combinations of safety / reachability [AT04].– Generalized Reactivity(1) – cubic time [PPS06].

Practical Solution 1

Practical Solution 2 [JGB05,HRS05]

• Heuristics that use the NBW.

• Works? Good.

• Does not work?

Nondeterminism

• Nondeterministic automata cannot be used for game monitoring.

• Or can they?

• They just have to be built correctly…

Good for Games Automata• Automata that can be controlled in a step-wise

fashion.

• Defined via a game on the structure of the automaton.

• Can be used for game monitoring.Environment

System

Game Automaton

Definition

• Define the monitor game played on the structure of the automaton:– Start from the initial state.– Opponent chooses a letter.– We choose successor.– We win if:

• The resulting word is not in the language

• The resulting run is accepting

• An automaton is GFG if we win from initial state.

1 1 1 1 1 1 1 · · · · 1 1 0 1 1 1 1 1 1 1 · · · ·

21 3

0,1

1 10,1

0

Use for Game Monitoring

• Given a GFG we combine the game with the GFG.

• Player 0 chooses how to advance the GFG.

Environment

System

Game Automaton

Where do I get one?

• Prove that an automaton is good for games if it fair-simulates another good for games.

• Deterministic automata are trivially good for games. So start from the deterministic automaton.

• We show how to construct one.

Construct a GFG Automaton

• Replace the tree structure by nondeterminism.• Follow nondeterministically n subsets of

states.• Ensure that all the runs followed by some

subset visit accepting states infinitely often.• Wrong guess? Change your mind!• Intuition:

- first set is the subset construction.- other n-1 sets follow subsets of first set.

Construct a GFG

• Let’s start with details on determinization.

Determinization in Detail

0,11

1a b aba

1

1

0

0

Subset Construction

• There are infinitely many runs that reach an accepting state a finite number of times.

• Somehow these runs have to be separated.

Determinization Construction

• Have a tree of subset constructions.

• Whenever a node (subset) visits F, create a new son with the states in F.

• If a node is removed – flash red light.

• If a node equals its sons – flash green light.

• The parity condition follows the minimal node that flashed red/green infinitely often.

What is a state

• A tree.

• Nodes are elements in {1,…,n}.

• Every node is labeled by a subset of the states.

• G2{1,...,n+1} - the least node colored green.

• R2{1,…,n+1} – the least node that got erased.

Transition

• Replace label by the set of successors (subset construction).

• Create youngest son with subset of accepting states.

• Move double states to older brothers.• If node equal to union of sons, remove sons

and color green.• Remove empty nodes.• Compact names.

0,1,3,4

4

0,3 0,3

b

0,1,3,4

4

subset construction

0,1,3,4

4 4,1

spawn sons

4

0,1,3,4

4

move to older sons

4

1

0,1,3,4

4

Handle full nodes

1

remove empty nodes

c

1

2

1

2

1

1

2

1

3

4 2

1

3

4 2

1

4

0,1,3

3 1

1

4

subset construction

2

0,1,3

3 1

1

42

spawn sons

1

1

3

5

0,1,3

3 1

1

4

15

move to older sons

2

0,1,3

3 1

1

4

Handle full nodes

a

subset construction

2

2

From OmegaDet [STW05]

1

0

1

0

1

10

0

Safra from a node’s point of view

• I follow some states.

• Some of them may disappear.

• If all visit acceptance set, I raise a green flag.

• If all disappear I die.

• After I die, I can be revived with a new set.

Our ConstructionA State

• Up to n subsets of the states of the NBW.

• Every state in a subset is either marked or unmarked.

• If a subset is empty all subsets above it are empty.

Our ConstructionA Transition

• Replace every set with a subset of the possible successors.

• Successors of marked states are marked; accepting states are marked.

• If all are marked, remove marking.

• An empty set can load a subset of the first set.

Advantages

• Very simple construction.

• Amenable to symbolic implementation.

• Natural incremental structure leading to complete solution.

A Range of Constructions

• We can get closer / further from the deterministic automaton.

• The number of states goes between n2n and n3n.

• It all depends on the symbolic implementation…

Incremental Construction

• We don’t always need n sets.

• An automaton with i+1 sets ‘monitors fully’ more games than an automaton with i sets.

• It depends on the game itself.

• It is not related (directly) to memory.

Summary• Replace deterministic automata by

nondeterministic automata.• Definition of GFG automata.• Construction of GFG automata. • Simple, amenable to symbolic implementation.• Incremental structure leading to the full solution.• Initial enumerative implementation.• Lower bound.

Safraless Decision Procedures [KV05]• Emptiness of alternating parity tree automata by

rank computation.

• Requires determinization for the upper bound.

• Reduces to Büchi games instead of parity.

• Complexity may be quadratically worse.

• Strategy may be exponentially worse.

• Enables solution of games with LTL winning conditions. Does not apply for NBW winning conditions. Does not apply to infinite structures.

Future Work

• Implementation.

• Reuse work done in increments.

• Understand better the incremental structure.

• Automata for the complement language.

• Lower bound on the index.

Going Both Ways

• It would be nice to find both winning and losing states fast.

• Starting from LTL it is easy.– Build NBW N for .– Build NBW N: for :.– Combine the game incrementally with GFG for N.– Combine the game incrementally with GFG for N: .

• Starting from NBW?– Build GFG for N.

– Build KV ranks for N.

Thank You