Causal Delivery

(Thomas) Matt Guinn

CS523, Spring 2006

Research Basis

Causal Delivery Protocols in Real-time Systems: A Generic Model Paulo Verissimo University of Lisboa 1996

Available at library (not electronically) or from TA 004.3305 RE

Purpose – Multicast Reliability

Clock-drivenRequire a global time basis or referenceConsidered most suitable for real-time

systems Timer-driven

“Clock-less”Acknowledgement based relying on local

timers (e.g. relative references)Asynchrony led to doubts on applicability

Purpose (continued)

To show that clock-based protocols are not absolutely necessary to maintain a cause-effect ordering

Ensuring causal delivery in a timer-based protocol accomplishes this

Allows for more effective protocol development by providing more options in the search for a “best fit”

Outline

Basic definitions and assumptions Key concepts of causality Three theorems for proving causal

ordering

Causal Delivery – Defined

Given a distributed system where messages are being passed back and forth, causal delivery ensures that they are all delivered in precedence order

More formally: “Consider two messages, m1, m2, sent by p, resp q, to the same destination participant r. Causal delivery ensures that if sendp(m1) → sendq(m2), then deliverr(m1) → deliverr(m2), i.e. m1 is delivered to r before m2”

Assumptions and Notations Systems in question are synchronous

Known bounds on CPU speed, message transmission delay, clock rate drift

Events Basic event - ei

p – either internal or external, it is the “i-th” event of “participant” p

Action – ACTp(a) – participant p takes some external action (i.e. I/O or sending message)

Our “send” event Observation – OBSp(a) – participant p observes an external

event a such as an environmental observation, or delivery of a message sent by another participant

Our “deliver” event

Precedence

In order for an event to precede (→) another event, at least one of the following must hold true: for j > i, ei

p → ejp

for p != q, eip → ej

q if eip is an external event

ACTp(a) and ejq is an external event OBSq(a)

for p != q, eip → ej

q if there exists an event e such that ei

p → e and e → ejq

Previous work – Logical Ordering

Logical orderingCausality controlled by protocol itselfA message m1 logically precedes m2 if:

m1 is sent before m2, by the same participant

or m1 is delivered to the sender of m2 before it sends m2

or there exists a message m3 such that m1 → m3 and m3 → m2

Logical Ordering Continued Oddly similar…why cover again?

Logical ordering works naturally in environments where only that protocol is in use

Environments which include any type of data transmission which goes unnoticed or unaffected by the protocol can generate failures

These conditions are known as “hidden channels” or “clandestine” information

Almost all modern distributed environments have multiple protocols and any number of hidden channels

Would be represented in our system as an ACT or OBS in a chain of causally related events which do not correspond to sends or deliveries of a logical ordering protocol

Temporal Ordering Common sense tells us that in order for an

event to cause an effect in another, it must precede it in time

“a message m1 precedes m2 temporally if the send event of m1 physically occurs before that of m2, in a Newtownian time-frame” t(send(m1)) < t(send(m2))

Temporal Ordering Failings Time-stamping indeed helps maintain causal

order under optimal circumstances Fails to account for real-world timing constraints

The time required for data to travel from one participant to another, execution time at a node, etc

A minimum, finite amount of time is required in order for one event to cause an effect in another

δ is our notation for this minimum amount of time Only messages separated by at least this (δ)

amount of time can be considered “causally related”

Fixing Temporal Ordering To accommodate the shortcomings of temporal

ordering, we introduce three more concepts δ – precedence

An event e is said to δt-precede an event e` (e → e`), if t(e`) – t(e) > δt

Synchronism metrics Delivery time of a message, tp

d(m) is the interval between send(m) event and deliverp(m) event at p

t(dliverp(m)) – t(send(m))

Tpdmax = maximum delivery time for participant p

Tpdmin = minimum delivery time for participant p

Steadiness

Steadiness is the greatest difference across all possible participants between their maximum and minimum delivery times and is symbolized by σσ = maxp(Tp

dmax – Tpdmin)

We these fundamentals, we can now state the first of three theorems that support causal delivery

Theorem 1

A protocol with σ steadiness delivers messages m1 and m2, such that send(m1) -> send(m2), in temporal order

This proves precedence, but not necessarily causality since messages may still precede each other, but not be ordered causally due to the steadiness delay

Two additional concepts

Local granularity – The minimum delay between any two events in a given participantSymbolized as μt

Propagation Delay – The minimum delay between an Action event and its corresponding Observation event between two participantsSymbolized as μs

On hidden channels…

Adverse environments – “Those where channels external to the protocol can deliver information faster than the protocol itself” μs < minp(Tp

dmin) Propagation delay is now defined by fastest hidden

channel Favourable environments – “Those where the

channels external to the protocol are slower than or as fast as the latter, or do not exist…” μs = minp(Tp

dmin) Propagation delay is defined by the fastest execution

available from the protocol

Theorem 2 -Adverse and Favourable

A system with local granularity μt and propagation delay μs can ensure causal delivery if μt > maxμ(Tμ

dmax) - μs Valid for either adverse or favourable

environments Says that as long as your local time between

events is greater than the largest transmission time minus the propagation delay, you can ensure causal delivery

Theorem 3- Favourable

A system with local granularity μt, propagation delay μs, a protocol with steadiness σ and an absolute minimum delivery time of minμ(Tμ

dmin), can ensure causal delivery if μs = minμ(Tμ

dmin) and μt > σ

This is given since in favourable conditions, it lowers the allowed minimum value of μt

Conclusion Precedence, temporal order, synchronicity, δ –

precedence, steadiness, local granularity, and propagation delay are all quantities which must be known in order to have a provably causal system

Together they support the preceding three theorems which, when applied to a protocol in development, allow one to maintain the requirements for causal delivery if it is so required

Forms the basis for much future work which we see everyday in distributed applications

Thanks!

Direct questions to: tguinn2@uiuc.edu