Distributed Shared Memory

Presentation by

Deepthi Reddy

What is DSM?

DSM is an abstraction used for sharing data between computers that do not share physical memory.

Differences between DSM and Message Passing

DSM• Involved shared memory

directly no marshalling necessary.

• Can alter erroneously causing one another to fail.

• Synchronization is achieved through normal constructs shared memory programming such as locks and semaphores.

• Process can leave data in an agreed memory location for the other to examine when it runs.

• Underlying run-time ensures the transparency.

Message Passing• Variables have to be

marshalled and unmarshalled.

• Protected one another by having private address space.

• Synchronization is achieved through message passing primitives such as lock server.

• Process communicating must execute at the same time.

• All remote data access is explicit.

 

Implementation Approaches

• Hardware• Paged Virtual Memory• Middleware

Design and Implementation Issues

Structure of data held in DSM:• Syncronization model• DSM consistency model• Update options• Granularity• Thrashing

StructureDSM is a replication system in which each application process is presented some abstraction of a collection of

objects (but in this case collection looks more or less like memory). Different approaches to DSM vary in what they consider to be and ‘object’ and how the objects are addressed.

Byte-Oriented: Accessed as ordinary virtual memory – a contiguous array of bytes. Operations allowed on Objects are : read or LOAD and write or Store.

Example: Mether system, IVY. Object-oriented: Structured as a collection of language-level objects with higher-level semantics such as

stacks and dictionaries. Example: OrcaImmutable data: Viewed as collection of immutable data items that process can read, add and remove form. Examples: Agora, Linda, Tspaces and JavaSpaces. 

Synchronization model:  • Byte-Oriented: Accessed as ordinary virtual memory – a contiguous array of bytes. Operations allowed on

Objects are : read or LOAD and write or Store.Example: Mether system, IVY.  • Object-oriented: Structured as a collection of language-level objects with higher-level semantics such as

stacks and dictionaries.Example: Orca • Immutable data: Viewed as collection of immutable data items that process can read, add and remove

form.Examples: Agora, Linda, Tspaces and JavaSpaces

Synchronization model If a and b are 2 variables stored in DSM, the a

constraint might be that a=b always. If 2 or more process execute the following code:

  a:=a+1;b:=b+1;

 then an inconsistency arise.A Distributed synchronization need to be provided,

which includes familiar constructs such as locks and semaphores.

 

Consistency model • The local replica manager is implemented by a

combination of middleware and the kernel. Middleware performs the majority of DSM processing.

• Amortize communication costs by spreading them over multiple updates.

• Memory consistency: Specifies the consistency guarantees that a DSM makes about the values that process read from objects

• Example: process 1 process 2

br:=b; a:=a+1;ar:=a; b:=b+1;if(ar>=br) print(“OK”);

Consistency cotd..• Atomic Consistency (Linearizability): A replicated shared

object is said to be linearizable if for any execution there is some interleaving of the series of operations issued by all the clients that satisfies the following two criteria:

The interleaved sequence of operations meets the specification of a correct copy of the objects

The order of operations in the interleaving is consistent with the real times at which the operations occurred in the actual execution.

• Sequential Consistency: Linearizability is too strict for most practical purposes. This could be implemented by using a single server to hold all the shared data by making all process perform reads or writes by sending requests to the server.

• Coherence: Is an example of weaker form of consistency.

Weak Consistency: Dubois, et al define this model, called weak consistency, by saying that it has three properties:

Accesses to synchronization variables are sequentially consistent.

No access to a synchronization variable is allowed to be performed until all previous writes have completed everywhere.

No data access ( read or write ) is allowed to be performed until all previous accesses to synchronization variables have been performed.

Update OptionsTwo implementation Choices:• Write-update : The updates made by a process are made locally and

multicast to all other replica managers possessing a copy of the data item, which immediately modify the data read by local processes. Processes read the local copies of data items, without the need for communication. In addition to allowing multiple readers, several processes may write the same data item at the same time; this is known as multiple-reader-multiple-writer sharing.

• Write-invalidate: This protocol is commonly implemented in the form of multiple-reader-single-writer sharing. At any time, a data item may either be:

accessed in read-only mode by one or more processes

read and written by a single process

Granularity Granularity refers to the size of the shared memory unit. A page

size of that is a multiple of the size provided by the underlying hardware or the memory management system allows for the integration of DSM and the memory management systems. By integrating DSM with the underlying memory management system, a DSM system can take advantage of the built in protection mechanism to detect incoherent memory references, and use built in fault handlers to prevent and recover from inappropriate references.

Thrashing• A potential problem with Write-invalidate

protocols• Occur where the DSM run-time spends on inordinate

amount of time invalidating and transferring shared data compared with the time spent by application process doing useful work.

Sequential Consistency and IVY• Consistency model: sequential consistency • Granularity : virtual memory pages • Data location : distributed directory • Modifications : detected via page faults • Synchronization : not integrated (locks) • The system model: A collection of process shares a segment of

DSM which is mapped to the same range of addresses in each process.

• The problem of write update: Used with page-based implementations, where the page is left with write permissions after an initial page fault and several writes are allowed to occur before the updated page is propagated.

The write-invalidate: When one or more process

are reading the page, they have read-only permissions.

When one process is updating a page, all other process have no access permissions to the page.

A process with most up-to-date version of a page p is designated as its

owner – owner(p). The set of processes that have

a copy of a page p is called its copy set – copyset(p)

IVY contd..• Invalidation protocols:

Centralized manager algorithm: A single server called manager stores the location of owner(p) for every page p and the set copyset(p) is stored at owner(p) Using Multicast: When a process faults, it multicasts its page request to all the other process and only the owner of the page replies.

• Dynamic distributed manager algorithm: divides the overhead of locating pages between those computers

that access them. Every process keeps, for every page p, a hint as to the page’s current owner – the probable owner of p or probOwner(p). Hints are updates and requests are forwarded.

•Thrashing: Mirage extends the coherence protocol of the IVY system to control thrashing. memory page is transferred to a processor, the processor is allowed to keep the page for a duration d. If a processor receives an invalidation message or has to relinquish the page to some other writer, it checks to see whether d has expired. If not, the processor informs the control manager of the amount of time it must wait before the processor can honor the request. The control manager waits until d expires and then requests the page again.

Release Consistency and Munin• Consistency model: release consistency ("eager") - note:

multiple COHERENCY PROTOCOLS • Granularity : cross between variables and virtual memory

pages • Data location : distributed directory • Modifications : detected via page faults • Synchronization : integrated (locks, barriers, and condition

variables)• Memory Access: Two types –

Competing access: 2 types • Synchronization access

acquire release

• Non-synchronization access

non-competing access

Release Consistency: All modifications to

shared data made within a critical section must be made visible to other processors that might access the data prior to completion of the critical section.

A shared memory system is release consistent if it obeys the following rules

1. Before any ordinary read or write access to shared data may be performed, all previous acquires done by the process must have completed successfully.

2. Before a release operation may be performed, all previous ordinary reads and writes done by the process must have been performed.

3. Acquire and release operations must be performed in "processor consistency" order.

Munin• Implementation of release consistency:

Munin sends upadte or invalidation information as soon as lock is released.

programmers can take annotations that associate a lock with particular data items.

• Sharing Annotations:• Read-only• Migratory• Write shared• Producer-consumer• Reduction• Result• Conventional

Other Consistency models• Casual consistency• Processor consistency• Pipelined RAM• Entry Consistency• Scope consistency• Weak Consistency