1 lecture notes: concurrent and distributed programming © assaf schuster spring 2004 -itay maman-
Post on 19-Dec-2015
221 views
TRANSCRIPT
2
Outline (1/3) Virtual Time
Happened Before Relation Cuts Lamport Clock Vector Clock
Shared memory Sequential Consistency vs. Coherence Release Consistency
Release Consistency Implementations: Dash, Munin (Eager) Lazy Release Consistency
Interval Clock Vector Garbage Collection Optimization - Lazy diffing
Home Based Lazy Release Consistency
4
Happened Before Relation Partial order over events
An event is an atomic "instruction" in a program
Intuition: β < γ β is bound to precede γ in every execution
Definition: β < γ if either β, γ are on the same thread and β precedes γ β sends a message to γ exists λ such that β < λ, λ < γ
5
Cuts (1/2)
Intuition: In each thread, split the sequence of events into two subsets ("pre", "post"). Then, union all "pre" subsets.
Definition: Cut X over cut events e1,e2, .. en X(e1,e2, .. en) = {e in P1 | e <= e1} U {e in P2 | e <= e2} .. U {e in Pn | e <= en}
7
Lamport Clock A function L: Event -> Int Definition:
β, γ are two consecutive events on a thread λ is an event on another thread λ sends a message to γ: L(γ) = Max{ L(β), L(λ) } + 1 Otherwise: L(γ) = L(β) + 1
Usage: β < γ => L(β) < L(γ) L(β) < L(γ) => not(γ < β)
However, if β || γ every outcome is possible: L(β) < L(γ), or L(β) = L(γ), or L(β) > L(γ)
8
Vector Clock (1/3)
Intuition Each thread maintains a local vector
of approximated timestamps of all other threads.
When a message is received, the local vector is max-ed with the sender's vector
9
Vector Clock (2/3) Definition: Vector Clock function C: Event ->Intn
β, γ are two consecutive events on a thread j. λ is an event on another thread. Sj is a vector with 1 on the j-th position (rest is zeros). λ sends a message to γ: C(γ)=Max{ C(β), C(λ) } + Sj
Otherwise: C(γ) = C(β) + Sj
CP(m) is the current local vector of thread m. Invariant (for every k, j):
CP(j)[j] >= CP(k)[j], at any point in time.
10
Vector Clocks (3/3)
Theorems A cut X(e1, e2, .., en) is consistent max{ C(e1), C(e2), .., C(en) } = { CP(1)[1], CP(2)[2], .., CP(n)[n] } =>Different consistent cuts have different
timestamps C(β) < C(γ) β < γ C(β) || C(γ) β || γ C(β)[j] <= C(γ)[j] β < γ (where Pj is the thread of β)
11
Memory Models The model: Many threads, A single shared
memory Operations: r(x) => Read x, w(x,n) => Put n in
x All variables are initialized to 0
In reality: Many threads each having a local memory.
The memory model specifications determines how/when one thread "sees" the put operations of other threads
12
Memory Models - definitions:
Program order: The sequence of instructions in each thread.
Serialization: A "merge" of all program orders. Similar to the merging of two decks of
cards Legal Serialization:
r(x) = n Last put to x was w(x,n)
13
Sequential Consistency Vs. Coherence
Sequential Consistency (SC) Execution of a program is a legal serialization
Coherence P is a given execution (serialization) Let sub(p,v) denote the sub execution of P
having only r(v), w(v,n) operations. Coherence ensures that sub(p,v) is a legal
serialization for each variable v in the program p
14
Release Consistency (1/2)
Assumes the existence of Locks in the program acquire(l), release(l) The lock variables are Sequentially
Consistent Intuition: release(l) flushes all put
operations taking place since last acquire(l)
15
Release Consistency (2/2)
In the absence of release/acquire pairs, put operations may never propagate If Data Races exist we may "see" values in
an undefined order Theorem: A data race free program is
sequentially consistent under a Release Consistency Model
16
Release Consistency Implementations - Dash
DASH (original impl.) put(v,n) is immediately sent
(asynchronously) to all threads release(l) blocks until all previous put
requests have completed
17
Release Consistency Implementations - Munin (1/2)
Munin (AKA: Eager Release Consistency) All put operations are saved in a local
buffer release(l)
Eliminates duplications Sends Notification to all other threads of the
local modifications Waits till all threads have acknowledged.
18
Release Consistency Implementations - Munin(2/2)
Twinning & Diffing: The notification creation algorithm A Local "clean" copy (Twin) is created upon first put
to each page release: compare each page with its twin, create a
notification entry for each modified variable Two different notification strategies
Send actual "new" value- or - Send an invalidation message for the modified
variables (receiver will have to refetch on first read)
19
Lazy Release Consistency Yet another memory Model
A More relaxed version of Release Consistency Intuition: A thread needs to be notified only
when it acquires a lock Principles
Notifications are lazily sent: only to threads which actually acquire the released lock (possibly transitively)
Lock variables are sequentially consistent Uses the Invalidation strategy Actually
needed ??
20
Lazy Release Consistency Implementation (1/2)
Each thread maintains a Clock Vector A new interval begins at each synchronization (acq/rel). Thread Pj increments its position j each time it starts a
new interval. Local vector of acquirer is max-ed with vector of
releaser. VP(j,n) denotes the local vector of thread Pj when it
executed its n-th interval Invariant: VP(j,n)[j] = n Interval n of Pj covers interval x of Pk
if VP(j,n)[k] >= x
21
Lazy Release Consistency Implementation (2/2)
Each thread maintains a table of intervals One entry per interval Each entry lists all write notices + all diffs,
"produced" by the interval (including those accepted from other threads)
Acquire protocol:1. Acquirer attaches its local vector to the acquire
request 2. Releaser sends back all notices which belong to
intervals that are not covered by the acquirer.3. Acquirer invalidates all notified variables + Copies
accepted notices to the new interval's entry.
22
Lazy Release Consistency: Garbage Collection
A diff cannot be retained forever if a diff was already sent to all other
threads it can be discarded If a thread is running out of memory =>
"Stop the world" (All threads stop) All write notices are sent to everyone Now, All local diffs/notices can be discarded
23
Lazy Release Consistency: Optimization - Lazy diffing
A diff is not automatically computed The diff is created on the first time a
thread has asked for it Till then the Twin page must be kept
24
Home-based Lazy Release Consistency (HLRC)
Intuition: Each diff is applied only once. Each page has a designated owner (home)
Acquire protocol:1. Acquirer sends an acquire request2. Releaser sends the diff to the page's home 3. Home updates its "master" copy of the page +
sends acknowledgement back4. Releaser sends an invalidation message to the
acquirer
25
HLRC: Guaranteeing update completion (1/2)
The Challenge: the home thread must apply the updated diff before other threads can retrieve it
Solution 1: Flushing- acquirer waits till the home finished "digesting" the diff Problem 1.1: May cause the same page to be
fetched twice from the home (caused by two distinct notifications)
Solution 1.1: Each page has a version number. No fetching is carried out if local version matches the latest version (at the home)
26
HLRC: Guaranteeing update completion (2/2)
Solution 2: Clock Vectors Solves the problem of "waiting" on each acquire Each diff is sent with its clock vector Current vector is attached to each "page-fetch"
request The home waits till it receives a diff whose vector is
at least "as-recent-as" the request vector Summary:
Acquire does not block Fetch may block
27
Djit+ Intuition: Trace execution. Check clock
vector of each access against previously logged accesses
Pros: No false alarms No missed races (in the given scheduling)
Cons: Cannot prove a program is data-race free Requires lots of runs High Sensitivity to scheduling
28
Djit+: Local Time Frame (LTF)
An LTF is a scalar value Each release operation increments LTF
of the current thread (Interval starts) Each thread maintains an LTF vector Acquirer's vector is max-ed with
releaser's
29
Djit+ Algorithm
1st read and 1st write in each interval are recorded (per-thread)
A read operation is checked against all recorded writes A write operation is checked against all recorded reads
+ writes Checking predicate:
CL denotes the current LTF vector held by the "current" thread e denotes a previously recorded entry predicate(e,CL) = e.ltf ≥ CL[e.thread_id] TRUE => A data race
30
Lockset Intuition: Trace execution. Look for
violations of common locking schemes Pros:
Less sensitive to scheduling No missed races (in the given scheduling)
Cons: Cannot prove a program is data-race free Lots of false alarms Still scheduling depended
31
Lockset Algorithm
for each variable v init c(v) with { all-locks } On each access to a variable v
c(v) = c(v) ∩ { currently held locks } if c(v) = { } issue a race warning
Conflicting practices: Initialization: Often not protected Read Only Variable: No protection if no one writes
to it Multiple readers-single writer
32
MPI
Basic functions
MPI_Init(argc, argv);MPI_Finalize();MPI_Comm_rank(MPI_COMM_WORLD, &rank);MPI_Comm_size(MPI_COMM_WORLD, &count);MPI_Send(buf, n, MPI_INT, to, TAG1, MPI_COMM_WORLD);MPI_Recv(buf, n, MPI_INT, from, TAG1, MPI_COMM_WORLD, &st);
33
MPI Blocking Send
MPI_Send() - Wait till last byte sent size < threshold => Transmission starts on spot otherwise => Transmission starts when MPI_Recv() is
placed MPI_SSend() - Wait till MPI_Recv() is placed MPI_RSend() - Failure if no MPI_Recv() is pending
Failure = Program stops MPI_BSend() - Wait till the data is copied to a
client supplied buffer
34
MPI
Non Blocking Send MPI_ISend() - Returns immediately
size > threshold => Wait till MPI_IRecv() is placed
MPI_Request r; MPI_Status s; int f; // Flag
MPI_Isend(buf, n, MPI_INT, to, TAG1, MPI_COMM_WORLD, &r);MPI_Wait(&r, &s); // Wait till a request completesMPI_Test(&r, &f, &s); // Check status of a requestMPI_Probe(from, TAG1, MPI_COMM_WORLD, &s); // Wait for a message