franck petit inria / lip lyon stabilization, synchronization of logical clocks and applications

Franck Petit INRIA / LIP

Lyon

Stabilization, Synchronization of Logical Clocks and Applications

o G=(V,E) is a connected networko V: set of n nodes/processeso E: set of m bidirectional linkso Np: set of neighboring nodes of p o Memory shared between neighboring nodes

2

Preliminaries

Bidirectional links

o In each computation step: According to its variables and the variables

of its neighbors, a node/process is either enabled to execute an action or not

Synchronous system Every enabled nodes execute an action atomically

Asynchronous system Some enabled nodes are chosen by an unfair

adversary The chosen nodes execute an action atomically

3

Preliminaries, Distributed Algorithm

Logical Clock Synchronization

o Synchronous/Asynchronous Environment

4

0 0 0 0 0 0 0 0 0

+4

+3

+2

+1

+0

1 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2 2

3 3 3 3 3 3 3 3 3

o Each node p maintains a logical clock register rp

rp := rp + 1



5

0 0 0 0 0 0 0 0 0

+4

+3

+2

+1

+0

o Each node p maintains a logical clock register rp

rp := rp + 1

0 0

1

0 0 0

1

0 00 0

2

0 0 0

2

0 0

+ Synchronizer

Global Synchronizer (asynchronous) distributed systems

o Wave Algorithms, e.g., Propagation of Information with Feedback

(PIF) Depth-First Token Circulation

o Global Synchronization, e.g., (Group) Mutual Exclusion Leader Election Reset Logical Clock Synchronization Rooted Spanning Tree …

6

In systems with unique IDs or a particular node (root, leader, main server…)?

r

Propagation of Information with Feedback


r

r

Leaf processors


r

A slow processor


r


r


o For One Puse:o Best Case: O(D)o Worst Case: O(N)Can we provide better complexities?



20

o Each node p maintains a phase clock register rp

0 0 0 0 0 0 0 0 00 0

1

0 0 0

1

0 00

1 1 1

0 0

1

0 00

1

2

1 1 1 1

0 00

1

2 2 2

1 1 1

0

1 1

2

3

2

1

2

1

0

2 2 2

3

2 2 2

1

0

2

3 3 3 3 3

2

1

0

2

3

4 4 4

3

2

1

0

+4

+3

+2

+1

+0

If ∀q N∈ p : rp ≤ rq then rp := rp + 1


21

o Each node maintains a phase clock registerr in [0,…,K-1]=ZK

o Safety: The gap between the phase clock of two neighboring nodes is at most equal to 1 (mod K).

o No starvation (Vivacity): Each phase clock r is incremented by 1 (mod K) infinitely often.

Unison


22

Unison

2

3

4

0

4

3

2

1 1

0

+4

+3

+2

+1

+0

K=5

0 0 0 0 0 0 0 0 00 0

1

0 0 0

1

0 00

1 1 1

0 0

1

0 00

1

2

1 1 1 1

0 00

1

2 2 2

1 1 1

0

1 1

2

3

2

1

2

1

0

2 2 2

3

2 2 2

1

0

2

3 3 3 3 3

2

1

0

2

3

4 4 4

3

2

1

0

Minimality of K ?


23

o K=2?o No possible order among the clocks

0 0 0 0 0 0 0 0 0

+1

+0

1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1

Synchronous

0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1


24

o K=2?o No possible order among the clocks

0 0 0 0 0 0 0 0 0

+1

+0

Asynchronous

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1


25

o Successor Function and Predecessor Function possible if K≥3

o K=3o Local total order ≤l over Z3={0,1,2}

a ≤l b iff 0≤ b-a mod 3 ≤ 1

0 ≤l 1 ; 1 ≤l2 ; 2 ≤l 0 ;


26

o K=3, Lifting

0

0

0 0

0 0

00

0

0

0

0

p.R, virtual register, it countsthe number of increments of p

0

0

State of the virtualp.R lifted from

Projection mod K

0

State of p.r


27

o K=3, Lifting

0

1

1 1

1 0

01

1

1

1

0

0 1

D0

0 1

D0


28

o K=3, Lifting

0

1

1 1

2 1

01

1

2

1

1

0 1 2

D0 D1

0 1 2

D0 D1


29

o K=3, Lifting

0

2

1 1

2 2

01

1

2

2

2

0 1 2…D0 D1 D2

0 1 2…D0 D1 D2


30

o K=3, Lifting

0

0

1 1

2 2

01

1

2

3

2

0 1 2…D0 D1 D2

0 1 2…D0 D1 D2


31

The lifting is compatible with the local ordering of ZK

0 1 2…D0 D1 D2

0 1 2…D0 D1 D2

q.r

p.rq.R

p.R

p.R ≤ q.R

p.r ≤l q.r

Lifted execution

Execution

The lifting defines a global preorder over the nodes

32

1

0K-1

K-2

Generalization over ZK

o (Local) total order ≤l over ZK={0,1,2,…,K}o Let M ≥ 1 and K ≥ 2M+1

a ≤l b iff 0≤ b-a mod K ≤ M

With M=2 and K=5, then for 14 ≤l 1 ; 0 ≤l1 ; 1 ≤l 1 ; 1 ≤l 2 ; 1 ≤l 3 ;

33

Complexities

o Space : O(1)o Time, for one pulse:

o Best Case: O(1)o Worst Case: O(D) ← [DELAY]

34

Fault Tolerance

Starting from an arbitrary configuration ?

Self-stabilization

35

A self-stabilizing system, regardless of its initial state, is guaranteed to converge to the intended behavior in finite time. [Dijkstra 74]


correct correctstabilization phase

Tra

nsi

ent

Fau

lt

Transient Faults

After a fault or an arbitrary initialization

36

o K=3

0 1 2…D0 D1 D2

0 1 2…D0 D1 D2

0

0

1 1

2 2


37

o K=3

0 1 2…D0 D1 D2

0 1 2…D0 D1 D2

0

0

2 1

1 2

01

5

4

3

2

DEADLOCK!

DEADLOCK!


38

o K=60

1

2

3

0

5

4

01

2

34

5

6


39

o K=61

1

2

3

0

5

4

01

2

34

5

6


40

o K=61

1

2

3

0

5

4

01

2

34

5

6

MUTUAL

EXCLUSION!MUTUAL

EXCLUSION!

41

What are the conditions to be able to (re)-synchronize the system ?


42

System Configurations

: The register values are arbitrary

3 0

1

1

21 10

No Deadlock

1 : The register of two neighboring process is less than or equal to 1

01 : There is no deadlock (there exists a compatible lifting to

the configuration)

1 3

1

2

3

1 2

1

2

0

K=4

43


o CG (CG n), the cyclomatic characteristic of

the graph: Equal to the size of the greatest cycle in one of the cycle basis of G where the size of the greatest cycle is minimum (equal to 2 if G is acyclic)[Boulinier, Petit, and Villain, PODC 2004]

THEOREM:

K>CG═> 1 =10




the configuration)

10/04/23 44

Cyclomatic characteristic of G

CG=4 in meches and tories

10/04/23 45

CG=6 in honeycombs


10/04/23 46

CG=4 in hypercubes


10/04/23 47

CG=n on rings


48


o To avoid deadlock due arbitrary initial values, K must be greater than CG (CG

n), the cyclomatic characteristic of the graph

1

0K-2

K>CG

Values of Register r

49


1

= 10

No Deadlock

?




the configuration)

Stabilization

o Global Reset (Stabilizing PIF), implies a root or IDso Local Reset (also woks in anonymous networks)

QUESTION: What is the motivation behind anonymity?

“Only a few amount of bits allows to distinguish a huge number of nodes!”

o Lack of (underlying) infrastructure

No unique identifier assignation or no central process

No maintenance of any distributed structure

o Economic advantages

o User privacy preserved [Delporte-Gallet, Fauconnier, Guerraoui, and Ruppert, OPODIS 2007]

o No one-to-one routing

51

Advantages of Anonymous Solutions

52

Self-Stabilizing Logical Clock Synchronization

o To avoid starvations due arbitrary initial values, K must be greater than CG (CG

n), the cyclomatic characteristic of the graph

o Safety eventually guaranteed using a reset mechanism: Register r is set in [-,…,0]

o To avoid starvations due arbitrary initial values, must be greater than than HG+1 (HG

n), the greatest chordless cycle of the graph [Boulinier, Petit, and Villain, PODC 2004]

-

K-1-1

-2

-3

> HG+1

1

0K-2

K>CG


53

Self-Stabilizing Logical Clock Synchronization

-

K-1-1

-2

-3

> HG+1

1

0K-2

K>CG


SSSyncAN: qNp: Correctp(q) <<application>>;rp:=(rp);AR: (qNp: Correctp(q) ) (rptail) rp:=-; //resetAC: rp init*( qNp: rqinit*rp≤(rq) rp:=(rp):

54

Asynchronous, Anonymous Logical Clock, Related Works

Gouda, Couvreur, Francez, 1992

# of states Stabilizing Time

O(n²) O(nd)

o SSSync(K,α,M)

O(nd) O(nd)α=0, K>M.CG, M>HG-2

O(n) O(n)α>HG-2, M=1, K>CG

Sur un arbre

Dolev, 2000 O(n²) O(d)

O(1) O(d)

55

Wave Algorithms in Anonymous Networks

o Anonymous global ( = Diam) waves for non fault-tolerant systems [Tel, Concurrency’88]

o All existing solutions for coordination problems (either global, local or at distance ) are based on: Existence of unique IDs (or a root) Underlying wave mechanism

QUESTION: Can we provide wave algorithms using neither IDS nor root, i.e., working in an anonymous system?

56

Logical Clock Synchronization in Asynchronous Settings

o Let us observe a possible execution of the self-stabilizing phase clock synchronization again

o The system (a chain of 4 nodes) is stabilizedo K=5

1

1

1

1

2

2

2

3

3

3

2

4

4

4

3 4

0

0

0

0

P0 P1 P2 P3 P4

0

0

time

During the execution, coherent phase cuts Pi are built defined by the same value i on each clock register.

57

Logical Clock Synchronization in Asynchronous Settings

o Let us observe a possible execution of the self-stabilizing phase clock synchronization again

o The system (a chain of 4 nodes) is stabilizedo K=5

1

1

1

1

2

2

2

3

3

3

2

4

4

4

3 4

0

0

0

0

P0 P1 P2 P3 P4

0

0

time

During the execution, coherent phase cuts Pi are built defined by the same value i on each clock register.

58

Phase Clock Synchronization in Asynchronous Settings

o Observe that:

1. After stabilization, the phase clock synchronization algorithm provides a wave stream

0

0

0

0 1

1

1

1

2

2

2

2 D

D

D

D

D+1

D+1

D+1

D+1

D+2

D+2

D+2

D+2

Stabilization(Convergence)

2. In the future of a coherent phase Pi , the dth phase clock incrementation causally depends on nodes at distance ≤d.

o When the clock register r is equal to a particular value, e.g., 0, add a particular event “DECIDE”

Self-Stabilizing -Wavelets

THEOREM: The self-stabilizing phase clock synchronization provides a self-stabilizing -wavelet mechanism

60

Self-Stabilizing -Wavelets

0

0

0

0

D

D

D

D

2D

2D

2D

2D

Propagation Feedback

o For instance, if =Diam, the above protocol provides a Propagation of Information with Feedback over an anonymous network

61

Self-Stabilizing -Local Infimum Computation

0

0

0

0

D

D

D

D

2D

2D

2D

2D


For instance, =

{a,e}

{b}

{a,b,c,f}

{d}

{a,b,c,d,e,f}

o An infimum over a set S is an associative, commutative, and idempotent (xx=x) binary operator.

62

Self-Stabilizing -Local Infimum Computation

0

0

0

0

D

D

D

D

2D

2D

2D

2D


For instance, = min

{9}

{2}

{6}

{0}

{0}

o An infimum over a set S is an associative, commutative, and idempotent (xx=x) binary operator.

63

Self-Stabilizing -Barrier Synchronization

o A two-layer phase clock algorithm

o The upper layer clock is driven by the lower layer clock

o In other words, every node moves the upper layer clock rUP from i to i+1 (mod KUP) each time the lower layer clock incremented the lower layer clock, rLOW, times

o The upper layer implements a self-stabilizing -Barrier Synchronization, i.e., Phase i+1 (mod KUP) is executed by each node only after all nodes at distance less than or equal to have completed Phase i (mod KUP)

64

Self-Stabilizing -Local Resource Allocation

o Let be a reflexive binary compatibility relationship over A, a set of resources.

o If (a,b) , then a and b are compatible, otherwise a and b are conflicting.

o The -Local Resource Allocation is specified as follows:

o Safety: If two nodes at distance less than or equal to execute their critical section simultaneously, then both are using compatible resources

o No starvation (Vivacity): If a node requests a resource to enter the critical section, then its request is eventually satisfied and it enters the critical section

65

An example : -Local Mutual Exclusion

o Let A be a set of IDs s.t. for every pair (a,b) in A, a (resp. b) is conflicting with b (resp.a) (i.e., =)

o Given two neighboring nodes P,Q, rP (resp. rQ) being the upper clock of P (resp. Q):

(rP ,IDP) (rQ ,IDQ)iff

(rP <(mod K) rQ) or ((rP =(mod K) rQ) and (IDP<IDQ))

o Mutual Exclusion at distance is provided by applying the order relation in each phase transition

o The stabilization time is in O(n) time unitso The space complexity is in O(log Diam)

Enabling Snap-Stabilization in Anonymous Networks

QUESTION: Can we design snap-stabilizing waves algorithms in anonymous networks?

Self-stabilization

67




Tra

nsi

ent

Fau

lt

Stabilization and Safety

A self-stabilizing system, regardless of the initial state of the processors, is guaranteed to converge to the intended behavior in finite time.

Dijkstra 1974

A priori, Safety is not guaranted during the faults

A snap-stabilizing system, regardless of the initial state of the processors, is guaranteed to converge to the intended behavior in 0 times.

Bui, Datta, Petit, Villain 1999

Safety is lost during faults and system stabilization.

Safety is lost during faults only.

User’s Point of View

X F(X)

Network

F(.)

X F(X) ?F(.)


1.X ≠ F(X)F(.)2.X ≠ F(X)3.X ≠ F(X)4.X ≠ F(X)N.X F(X)

Self-Stabilizing Systems


1.X F(X)F(.)

Snap-Stabilizing Systems


correctstabilization phase

illeg

itim

ate

leg

itim

ate

correctT

ran

sien

t F

ault

A general technique to deal with transient faults

Self-Stabilization


illeg

itim

ate

leg

itim

ate

stabilization time

enabled starting actions legitimate initial configuration

Request Request Request Request

Self-Stabilization

Tra

nsi

ent

Fau

lt

correct

abn

orm

aln

orm

al

first enabled starting action

always correct

Quality of Service

Request

can be abnormal

NORMAL initial configuration

Request

but service is ok

Snap-Stabilization

Tra

nsi

ent

Fau

lt

76

Coherent Cuts Detection

0 1 2

10 2

If I execute 3 consecutive normal

actions, my neighbors execute

at least one

77

If I execute x consecutive normal

actions, then my neighbors execute

at least x-2

0 1 2

10 2

0 1 2

10 n

n

n-12

Generalization


78

I can extend the phenomenon to my

neighbors

0 1

1

nn-1

n-1

2 n-2

3 n-3


79

10 11 12

11

20

19

12 18

13 17

14 15 16

13 14 15 16 17 18 19

15

15

15

[Cu, Cu+D]

16

16

16

14

14

14

After an enought long normal action sequence (NA), a coherent phase is guaranteed in the past.


80

11 0 -α

Reset1

2

…0

0 can mask a reset.If I want to know the

behavior of the network, I must avoid this value


81

Snap-Stabilization Wave Protocol for Anonynous Networks

After an enought long normal action sequence (NA), a coherent phase is guaranteed in the past.

A wave can computed after a sequence of 2D+3 normal actions passing through 0 and with an initial value greater than D.

82

A wave is requestedby an application

D < clock < K-2D-4

2D+3 consecutiveincrementations DECIDE

No, a reset occured

Yes

No

Yes

Snap-Stabilization Wave Protocol for Anonynous Networks

K > 3D+5

83

Enabling Snap-Stabilization in Anonymous Networks

Snap-Wave Toolbox

Underlayer Unison

Application layer

Request Answer

Some references

84

• UNISON and its applications• Boulinier, Petit, Villain [PODC 04]

When Graph Theory helps Self-Stabilization• Boulinier, Petit, Villain, [Algorithmica 08]

Synchronous vs. Asynchronous Unison• Boulinier, Petit, [APDCM 08]

Self-Stabilizing Wavelets and -Hop Coordination

• Boulinier, Levert, Petit : [ICDCN 08]Snap-Stabilizing Waves in Anonymous Networks

• Snap-Stabilization• Bui, Datta, Petit, Villain [Distributed Computing 07]

Snap-stabilization and PIF in Tree Networks

http://graal.ens-lyon.fr/~fpetit

franck petit inria / lip lyon stabilization, synchronization of logical clocks and applications

Documents

r p r q

logical clock register

d0d0 d0d0 slide

feedback slide

o safety

synchronizer slide

unison slide

phase clock register