logical clocks. topics logical clocks totally-ordered multicasting vector timestamps

55
Logical Clocks

Upload: marian-walker

Post on 18-Jan-2016

228 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Logical Clocks

Page 2: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Topics

Logical clocksTotally-Ordered MulticastingVector timestamps

Page 3: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

ReadingsVan Steen and Tanenbaum: 5.2Coulouris: 10.4L. Lamport, “Time, Clocks and the Ordering of

Events in Distributed Systems,” Communications of the ACM, Vol. 21, No. 7, July 1978, pp. 558-565.

C.J. Fidge, “Timestamps in Message-Passing Systems that Preserve the Partial Ordering”, Proceedings of the 11th Australian Computer Science Conference, Brisbane, pp. 56-66, February 1988.

Page 4: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Ordering of Events

For many applications, it is sufficient to be able to agree on the order that events occur and not the actual time of occurrence.

It is possible to use a logical clock to unambiguously order events

May be totally unrelated to real time.Lamport showed this is possible (1978).

Page 5: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

The Happened-Before Relation Lamport’s algorithm synchronizes logical clocks

and is based on the happened-before relation: a b is read as “a happened before b”

The definition of the happened-before relation: If a and b are events in the same process and a occurs before b, then

a b For any message m, send(m) send(m) rcv(m), where send(m) is the

event of sending the message and rcv(m) is event of receiving it. If a, b and c are events such that a b and b c then a c

Page 6: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

The Happened-Before RelationIf two events, x and y, happen in different

processes that do not exchange messages , then x y is not true, but neither is y x

The happened-before relation is sometimes referred to as causality.

Page 7: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

ExampleSay in process P1 you have

a code segment as follows:

1.1 x = 5;

1.2 y = 10*x;

1.3 send(y,P2);

Say in process P2 you have a code segment as follows:

2.1 a=8;

2.2 b=20*a;

2.3 rcv(y,P1);

2.4 b = b+y;

Let’s say that you start P1 and P2 at the same time. You know that 1.1 occurs before 1.2 which occurs before 1.3; You know that 2.1 occurs before 2.2 which occurs before 2.3 which is before 2.4.You do not know if 1.1 occurs before 2.1 or if 2.1 occurs before 1.1.You do know that 1.3 occurs before 2.3 and 2.4

Page 8: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Continuing from the example on the previous page –

The order of actual occurrence of operations is often not consistent from execution to execution. For example: Execution 1 (order of occurrence): 1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 2.4 Execution 2 (order of occurrence): 2.1,2.2,1.1,1.2,1.3, 2.3,2.4 Execution 3 (order of occurrence) 1.1, 2.1, 2.2, 1.2, 1.3, 2.3, 2.4

We can say that 1.1 “happens before” 2.3, but not that 1.1 “happens before” 2.2 or that 2.2 “happens before” 1.1.

Note that the above executions provide the same result.

Page 9: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Lamport’s Algorithm

We need a way of measuring time such that for every event a, we can assign it a time value C(a) on which all processes agree on the following: The clock time C must monotonically increase i.e., always

go forward. If a b then C(a) < C(b)

Each process, p, maintains a local counter Cp

The counter is adjusted based on the rules presented on the next page.

Page 10: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Lamport’s Algorithm

1. Cp is incremented before each event is issued at process p: Cp = Cp + 1

2. When p sends a message m, it piggybacks on m the value t=Cp

3. On receiving (m,t), process q computes Cq = max(Cq,t) and then applies the first rule before timestamping the event rcv(m).

Page 11: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example

a

b

P1 P2 P3

c

d

e

f

g

h

i

j

k

l

Assume that each process’s logical clock is set to 0

Page 12: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example

a

b

P1 P2 P3

c

d

e

f

g

h

i

j

k

l

Assume that each process’s logical clock is set to 0

1 112

323

4 4

5

6

3

Page 13: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example

From the timing diagram on the previous slide, what can you say about the following events? Between a and b: a b Between b and f: b f Between e and k: concurrent Between c and h: concurrent Between k and h: k h

Page 14: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Total Order

A timestamp of 1 is associated with events a, e, j in processes P1, P2, P3 respectively.

A timestamp of 2 is associated with events b, k in processes P1, P3 respectively.

The times may be the same but the events are distinct.

We would like to create a total order of all events i.e. for an event a, b we would like to say that either a b or b a

Page 15: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Total Order

Create total order by attaching a process number to an event.

Pi timestamps event e with Ci (e).i

We then say that Ci(a).i happens before Cj(b).j iff: Ci(a) < Cj(b); or

Ci(a) = Cj(b) and i < j

Page 16: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example (total order)

a

b

P1 P2 P3

c

d

e

f

g

h

i

j

k

l

Assume that each process’s logical clock is set to 0

1.1 1.21.32.1

3.22.33.1

4.1 4.2

5.2

6.2

3.3

Page 17: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally-Ordered Multicast Application of Lamport timestamps (with total

order) Scenario

Replicated accounts in New York(NY) and San Francisco(SF)

Two transactions occur at the same time and multicast Current balance: $1,000 Add $100 at SF Add interest of 1% at NY If not done in the same order at each site then one site

will record a total amount of $1,111 and the other records $1,110.

Page 18: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally-Ordered Multicasting

Updating a replicated database and leaving it in an inconsistent state.

Page 19: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally-Ordered Multicasting

We must ensure that the two update operations are performed in the same order at each copy.

Although it makes a difference whether the deposit is processed before the interest update or the other way around, it does matter which order is followed from the point of view of consistency.

We need totally-ordered multicast, that is a multicast operation by which all messages are delivered in the same order to each receiver. NOTE: Multicast refers to the sender sending a message to a collection

of receivers.

Page 20: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered MulticastAlgorithm

Update message is timestamped with sender’s logical time

Update message is multicast (including sender itself) When message is received

It is put into local queue Ordered according to timestamp, Multicast acknowledgement

Page 21: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example:Totally Ordered Multicast

Message is delivered to applications only when It is at head of queue It has been acknowledged by all involved processes Pi sends an acknowledgement to Pj if

Pi has not made an update request

Pi’s identifier is greater than Pj’s identifier

Pi’s update has been processed;

Lamport algorithm (extended for total order) ensures total ordering of events

Page 22: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

On the next slide m corresponds to “Add $100” and n corresponds to “Add interest of 1%”.

When sending an update message (e.g., m, n) the message will include the timestamp generated when the update was issued.

Page 23: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

San Francisco (P1)

1.1

2.1

New York (P2)

1.2

2.2

3.2

Issue m

Send m

Recv n

Issue n

Send n

Recv m3.1

Page 24: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

The sending of message m consists of sending the update operation and the time of issue which is 1.1

The sending of message n consists of sending the update operation and the time of issue which is 1.2

Messages are multicast to all processes in the group including itself. Assume that a message sent by a process to itself is received by the

process almost immediately. For other processes, there may be a delay.

Page 25: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast At this point, the queues have the following:

P1: (m,1.1), (n,1.2) P2: (m,1.1), (n,1.2)

P1 will multicast an acknowledgement for (m,1.1) but not (n,1.2). Why? P1’s identifier is higher then P2’s identifier and P1 has issued a

request 1.1 < 1.2

P2 will multicast an acknowledgement for (m,1.1) and (n,1.2) Why? P2’s identifier is not higher then P1’s identifier 1.1 < 1.2

Page 26: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

P1 does not issue an acknowledgement for (n,1.2) until operation m has been processed. 1< 2

Note: The actual receiving by P1 of message (n,1.2) is assigned a timestamp of 3.1.

Note: The actual receiving by P2 of message (m,1.1) is assigned a timestamp of 3.2

Page 27: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast If P2 gets (n,1.2) before (m,1.1) does it still multicast

an acknowledgement for (n,1.2)? Yes!

At this point, how does P2 know that there are other updates that should be done ahead of the one it issued? It doesn’t; It does not proceed to do the update specified in (n,1.2) until it gets an

acknowledgement from all other processes which in this case means P1.

Does P2 multicast an acknowledgement for (m,1.1) when it receives it? Yes, it does since 1 < 2

Page 28: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

San Francisco (P1)

1.1

2.1

3.1

5.1

New York (P2)

1.2

2.2

3.2

4.2

Issue m

Send m

Recv n

Issue n

Send n

Recv m

Send ack(m)Recv ack(m)

Note: The figure does not show a process sending a message to itself orthe multicast acks that it sends for the updates it issues.

Page 29: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

To summarize, the following messages have been sent: P1 and P2 have issued update operations. P1 has multicasted an acknowledgement message for

(m,1.1). P2 has multicasted acknowledgement messages for (m,1.1),

(n,1.2).

P1 and P2 have received an acknowledgement message from all processes for (m,1.1).

Hence, the update represented by m can proceed in both P1 and P2.

Page 30: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

San Francisco (P1)

1.1

2.1

3.1

5.1

New York (P2)

1.2

2.2

3.2

4.2

Issue m

Send m

Recv n

Issue n

Send n

Recv m

Send ack(m)Recv ack(m)

Process m

Note: The figure does not show the sending of messages it oneself

Process m

Page 31: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

When P1 has finished with m, it can then proceed to multicast an acknowledgement for (n,1.2).

When P1 and P2 both have received this acknowledgement, then it is the case that acknowledgements from all processes have been received for (n,1.2).

At this point, it is known that the update represented by n can proceed in both P1 and P2.

Page 32: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

San Francisco (P1)

1.1

2.1

3.1

5.1

New York (P2)

1.2

2.2

3.2

4.2

Issue m

Send m

Recv n

Issue n

Send n

Recv m

Send ack(m)

6.1Send ack(n)

Recv ack(m)

7.2 Recv ack(n)

Process m

Process nProcess n

Process m

Page 33: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example: Totally Ordered Multicast

What if there was a third process e.g., P3 that issued an update (call it o) at about the same time as P1 and P2.

The algorithm works as before. P1 will not multicast an acknowledgement for o until m has been done. P2 will not multicast an acknowledgement for o until n has been done.

Since an operation can’t proceed until acknowledgements for all processes have been received, o will not proceed until n and m have finished.

Page 34: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Problems with Lamport Clocks

Lamport timestamps do not capture causality.With Lamport’s clocks, one cannot directly

compare the timestamps of two events to determine their precedence relationship. If C(a) < C(b) is not true then a b is also not true. Knowing that C(a) < C(b) is true does not allow us to

conclude that a b is true. Example: In the first timing diagram, C(e) = 1 and C(b) =

2; thus C(e) < C(b) but it is not the case that e b

Page 35: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Problem with Lamport Clocks

The main problem is that a simple integer clock cannot order both events within a process and events in different processes.

C. Fidge developed an algorithm that overcomes this problem.

Fidge’s clock is represented as a vector [v1,v2,…,vn] with an integer clock value for each process (vi contains the clock value of process i). This is a vector timestamp.

Page 36: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Fidge’s Algorithm

Properties of vector timestampsvi [i] is the number of events that have

occurred so far at Pi

If vi [j] = k then Pi knows that k events have occurred at Pj

Page 37: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Fidge’s Algorithm

The Fidge’s logical clock is maintained as follows:

1. Initially all clock values are set to the smallest value (e.g., 0).

2. The local clock value is incremented at least once before each primitive event in a process i.e., vi[i] = vi[i] +1

3. The current value of the entire logical clock vector is delivered to the receiver for every outgoing message.

4. Values in the timestamp vectors are never decremented.

Page 38: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Fidge’s Algorithm

5. Upon receiving a message, the receiver sets the value of each entry in its local timestamp vector to the maximum of the two corresponding values in the local vector and in the remote vector received.

Let vq be piggybacked on the message sent by process q to process p; We then have: For i = 1 to n do

vp[i] = max(vp[i], vq [i] );

vp[p] = vp[p] + 1;

Page 39: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Fidge’s Algorithm

For two vector timestamps, va and vb

va is not equal to vb if there exists an i such that va[i] is not equal to vb[i]

va <= vb if for all i va[i] <= vb[i]

va < vb if for all i va[i] < = vb[i] AND va is not equal to vb

Events a and b are causally related if va < vb or vb< va .

Page 40: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example

P2

a

b

P1

c

d

e

f

g

h

i

P3

j

k

l

[1,0,0]

[2,0,0]

[3,0,0]

[4,0,0]

[0,1,0]

[2,2,0]

[2,3,2]

[2,4,2]

[4,5,2]

[0,0,1]

[0,0,2]

[0,0,3]

Page 41: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Vector Timestamps and Causality We have looked at total order of messages where all

messages are processed in the same order at each process.

It is possible to have any order where all you care about is that a message reaches all processes, but you don’t care about the order of execution.

Causal order is used when a message received by a process can potentially affect any subsequent message sent by that process. Those messages should be received in that order at all processes. Unrelated messages may be delivered in any order.

Page 42: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Causality and Modified Vector Timestamps

With a slight adjustment, vector timestamps can be used to guarantee causal message delivery.

We will illustrate this adjustment, the definition of causality and the motivation through an example.

Page 43: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application:Bulletin Board

The Internet’s electronic bulletin board service (network news)

Users (processes) join specific groups (discussion groups).

Postings, whether they are articles or reactions, are multicast to all group members.

Could use a totally-ordered multicasting scheme.

Page 44: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Display from a Bulletin Board Program

Users run bulletin board applications which multicast messages One multicast group per topic (e.g. os.interesting) Require reliable multicast - so that all members receive messages Ordering:

Bulletin board: os.interesting

Item From Subject

23 A.Hanlon Mach

24 G.Joseph Microkernels

25 A.Hanlon Re: Microkernels

26 T.L’Heureux RPC performance

27 M.Walker Re: Mach

end

Figure 11.13Colouris

total (makes the numbers the same at all sites)

FIFO (gives sender order)

causal (makes replies come after original message)

Page 45: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

A totally-ordered multicasting scheme does not imply that if message B is delivered after message A, that B is a reaction to A.

Totally-ordered multicasting is too strong in this case.

The receipt of an article causally precedes the posting of a reaction. The receipt of the reaction to an article should always follow the receipt of the article.

Page 46: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

If we look at the bulletin board example, it is allowed to have items 26 and 27 in different order at different sites.

Items 25 and 26 may be in different order at different sites.

Page 47: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

Vector timestamps can be used to guarantee causal message delivery. A slight variation of Fidge’s algorithm is used.

Each process Pi has an array Vi where Vi[j] denotes the number of events that process Pi knows have taken place.

Page 48: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

Vector timestamps are assumed to be updated only when posting or receiving articles i.e., when a message is sent or received.

Let Vq be piggybacked on the message sent by process q to process p; When p receives the message, then p does the following:

For i = 1 to n do

Vp[i] = max(Vp[i], Vq [i] );

When p sends a message, it does the following:Vp[p] = Vp[p] + 1;

Page 49: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

When a process Pi posts an article, it multicasts that article as a message with the vector timestamp. Let’s call this message a. Assume that the value of the timestamp is Vi

Process Pj posts a reaction. Let’s call this message r. Assume that the value of the timestamp is Vj

Note that Vj > Vi

Message r may arrive at Pk before message a.

Page 50: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

Pk will postpone delivery of r to the display of the bulletin board until all messages that causally precede r have been received as well.

Message r is delivered iff the following conditions are met: Vj[j] = Vk[j]+1

This states that r is the next message that Pk was expecting from process Pj

Vj[i] <= Vk[i] for all i not equal to j This states that Pk has seen at least as many messages as

seen by Pj when it sent message r.

Page 51: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board Initially Vj[i] = 0 and Vk[i] =0 for all i. This makes

sense since no messages have been sent. Let’s say that Pj sends a message where Vj[j] = 1. This

implies that this is the first message sent by Pj

Since Vk[j] is 0 then it will accept the message sent by Pj since it is expecting a first message from Pj.

Let’s say that Pj sent a message where Vj[j] = 5 (indicating a 5th message sent) and that Vk[j] is 3 which indicates Pk is expecting the 4th message sent by Pj . This indicates that the received message should be held back.

Page 52: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

Assume that Vj[i] <= Vk[i] is not true for some i. In other words, Vj[i] > Vk[i] for some i. This indicates that Pi sent a message that was received by Pj but not Pk .

In this case Pk will not deliver this message until it gets the missing message.

Page 53: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

P2

a

P1

c

d

P3

e

g

[1,0,0]

[1,0,0][1,0,0]

[1,0,1]

[1,0,1]

Post a

r: Reply a

Message a arrives at P2 before the reply r from P3 does

b

[1,0,1]

[0,0,0] [0,0,0] [0,0,0]

Page 54: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Example Application: Bulletin Board

P2

a

P1 P3

d

g

[1,0,0]

[1,0,0]

[1,0,1]

Post a

r: Reply a

Buffered

c[1,0,0]

The message a arrives at P2 after the reply from P3; The reply is not delivered right away.

b

[1,0,1]

[0,0,0] [0,0,0] [0,0,0]

Deliver r

Page 55: Logical Clocks. Topics Logical clocks Totally-Ordered Multicasting Vector timestamps

Summary

No notion of a globally shared clock.Local (physical) clocks must be synchronized

based on algorithms that take into account network latency.

Knowing the absolute time is not necessary.Logical clocks can be used for ordering

purposes.