communication & synchronization why do processes communicate in ds? –to exchange messages...

Communication & Synchronization

• Why do processes communicate in DS?– To exchange messages– To synchronize processes

• Why do processes synchronize in DS?– To coordinate access of shared resources– To order events

Time, Clocks and Clock Synchronization

• Time– Why is time important in DS?– E.g. UNIX make utility

• Clocks (Timer)– Physical clocks– Logical clocks (introduced by Leslie Lamport)– Vector clocks (introduced by Collin Fidge)

• Clock Synchronization– How do we synchronize clocks with real-world time?– How do we synchronize clocks with each other?

05 – 1 Distributed Algorithms/5.1 Clock Synchronization

Physical Clocks (1/3)

Problem: Clock Skew – clocks gradually get out of synch and give different values

Solution: Universal Coordinated Time (UTC):• Based on the number of transitions per second of the cesium 133

atom (very accurate).• At present, the real time is taken as the average of some 50

cesium-clocks around the world – International Atomic Time• Introduces a leap second from time to time to compensate that

days are getting longer.

UTC is broadcasted through short wave radio (with the accuracy of +/- 1 msec) and satellite (Geostationary Environment Operational Satellite, GEOS, with the accuracy of +/- 0.5 msec).

Question: Does this solve all our problems? Don’t we now have some global timing mechanism?



Problem: Suppose we have a distributed system with a UTC-receiver somewhere in it, we still have to distribute its time to each machine.

Basic principle:

• Every machine has a timer that generates an interrupt H (typically 60) times per second.

• There is a clock in machine p that ticks on each timer interrupt. Denote the value of that clock by Cp (t) , where t is UTC time.

• Ideally, we have that for each machine p, Cp (t) = t, or, in other words, dC/ dt = 1

• Theoretically, a timer with H=60 should generate 216,000 ticks per hour

• In practice, the relative error of modern timer chips is 10**-5 (or between 215,998 and 216,002 ticks per hour)



Goal: Never let two clocks in any system differ by more than time units => synchronize at least every 2seconds.


Where is the max. drift rate

•Principle I: Cristian Algorithm

• Every machine asks a time server for the accurate time.

• But you need an accurate measure of round trip delay, including interrupt handling and processing incoming messages.

•Principle II: The Berkeley Algorithm

•Let the time server scan all machines periodically, calculate an average, and inform each machine how it should adjust its time relative to its present time.

The time server polls periodically every machine for its time

The received times are averaged and each machine is notified of the amount of the time it should adjust

Note: you don’t even need to propagate UTC time


The Happened-Before Relationship

Problem: We first need to introduce a notion of ordering before we can order anything.

The happened-before relation on the set of events in a distributed system is the smallest relation satisfying:

• If a and b are two events in the same process, and a comes before b, then a b. (a happened before b)

• If a is the sending of a message, and b is the receipt of that message, then a b.

• If a b and b c, then a c. (transitive relation)

Note: if two events, x and y, happen in different processes that do not exchange messages, then they are said to be concurrent.

Note: this introduces a partial ordering of events in a system with concurrently operating processes.

05 – 6 Distributed Algorithms/5.2 Logical Clocks

Logical Clocks (1/2)

Problem: How do we maintain a global view on the system’s behavior that is consistent with the happened-before relation?

Solution: attach a timestamp C(e) to each event e, satisfying the following properties:

P1: If a and b are two events in the same process, and a b, then we demand that C (a) < C (b)

P2: If a corresponds to sending a message m, and b to the receipt of that message, then also C (a) < C (b)

Problem: How do we attach a timestamp to an event when there’s no global clock? maintain a consistent set of logical clocks, one per process.


Event counting example

• Three systems: P0, P1, P2

• Events a, b, c, …

• Local event counter on each system

• Systems occasionally communicate

Event counting example

a b

h i

k

P1

P2

P3

1 2

1 3

21

d f

g3

c

2

4 6

e5

j

Logical Clocks (2/2)

Each process Pi maintains a local counter Ci and adjusts this counter according to the following rules:

(1) For any two successive events that take place within Pi, Ci is incremented by 1.

(2) Each time a message m is sent by process Pi, the message receives a timestamp Tm = Ci.

(3) Whenever a message m is received by a process Pj, Pj adjusts its local counter Cj:

Property P1 is satisfied by (1); Property P2 by (2) and (3).

This is called the Lamport’s Algorithm


Logical Clocks – Example


Fig 5-7. (a) Three processes, each with its own clock. The clocks run at different rates. (b) Lamport’s algorithm corrects the clocks

• Assign the Lamport’s logical clock values for all the events in the above timing diagram. Assume that each process’s local clock is set to 0 initially.

a

b

c

d

e

f

gh

i

j

k

l

P1 P2 P3

From the above timing diagram, what can you say about the following events?

• between a and b: a b• between b and f : b f• between e and k: concurrent • between c and h: concurrent • between k and h: k h

a

b

c

d

e

f

gh

i

j

k

l

P1 P2 P3

1

2

3

4

1

3

4

5

6

1

2

3

Total Ordering with Logical Clocks

Problem: it can still occur that two events happen at the same time. Avoid this by attaching a process number to an event:

Pi timestamps event e with Ci(e).i

Then: Ci(a).i happened before Cj(b).j if and only if:

1: Ci(a) < Cj(a) ; or

2: Ci(a) = Cj(b) and i < j


Example: Totally-Ordered Multicast (1/2)

Problem: We sometimes need to guarantee that concurrent updates on a replicated database are seen in the same order everywhere:

• Process P1 adds $100 to an account (initial value: $1000)

• Process P2 increments account by 1%

• There are two replicas

Outcome: in absence of proper synchronization, replica #1 will end up with $1111, while replica #2 ends up with $1110.


Example: Totally-Ordered Multicast (2/2)

• Process Pi sends timestamped message msgi to all others. The message itself is put in a local queue queuei.

• Any incoming message at Pj is queued in queuej, according to its timestamp.

• Pj passes a message msgi to its application if:

(1) msgi is at the head of queuej

(2) for each process Pk, there is a message msgk in queuej with a larger timestamp.

Note: We are assuming that communication is reliable and FIFO ordered.


Fidge’s Logical Clocks

• with Lamport’s clocks, one cannot directly compare the timestamps of two events to determine their precedence relationship- if C(a) < C(b) then a b- if C(a) < C(b), it could be a b or a b- e.g., events e and b in the previous example Figure

* C(e) = 1 and C(b) = 2* thus C(e) < C(b) but e b

• the main problem is that a simple integer clock can not order both events within a process and events in different processes

• Collin Fidge developed an algorithm that overcomes this problem

• Fidge’s clock is represented as a vector [c1 , c 2 , …, cn] with an integer clock value for each process (ci contains the clock value of process i)

/ /

/

/

Fidge’s Algorithm

The Fidge’s logical clock is maintained as follows:

1: Initially all clock values are set to the smallest value.

2: The local clock value is incremented at least once before each primitive event in a process.

3: The current value of the entire logical clock vector is delivered to the receiver for every outgoing message.

4: Values in the timestamp vectors are never decremented.

5: Each time a process prepares to send a message, it increments its own logical clock in the vector by one and then sends its entire vector along with the message being sent.

6: Each time a process receives a message, it increments its own logical clock in the vector by one and updates each element in its vector by taking the maximum of the value in its own vector clock and the value in the vector in the received message (for every element).

Vector Clock

• Get r_vector from the received msg sent by process q;if l_vector [q] r_vector[q] then l_vector[q] : = r_vector[q] + 1;for i : = 1 to n do

l_vector[i] := max(l_vector[i], r_vector[i]);

• Timestamps attached to the events are compared as follows:• ep fq iff Tep [p] < Tfq [p]

• (where ep represents an event e occurring in process p, Tep

represents the timestamp vector of the event ep , and the ith element of Tep is denoted by Tep [i].)

• This means event ep happened before event fq if and only if process q received a direct or indirect message from p and that message was sent after ep had occurred. If ep and fq are in the same process (i,e., p = q), the local elements of their timestamps represent their occurrences in the process.

• Assign the Lamport’s and Fidge’s logical clock values for all the events in the above timing diagram. Assume that each process’s logical clock is set to 0 initially.

a

b

c

d

e

f

gh

i

j

k

l

P1 P2 P3

P1 P2 P3

a

b

c

d

e

f

gh

i

j

k

l

1

2

3

4

1

3

4

5

6

1

2

3

[1,0,0]

[2,0,0]

[3,0,0]

[4,0,0]

[0,1,0]

[3,2,0]

[3,3,3]

[3,4,3][5,5,3]

[0,0,3]

[0,0,2]

[0,0,1]

The above diagram shows both Lamport timestamps (an integer value ) and Fidge timestamps (a vector of integer values ) for each event.

– Lamport clocks: b h implies 2 < 5 3 < 4 but c g.

– Fidge clocks:f h since 2 < 4 is true, b h since 2 < 3 is true, h a since 4 < 0 is false, c h since (3 < 3) is false and (4 < 0) is false.

communication & synchronization why do processes communicate in ds? –to exchange messages...

Documents