chapter 16 parallel processing - yonsei...

YonseiYonsei UniversityUniversity

Chapter 16Chapter 16

Parallel ProcessingParallel Processing

YonseiYonsei UniversityUniversity16-2

ContentsContents• Multiple Processor Organizations• Symmetric Multiprocessors• Cache Coherence and the MESI Protocol• Clusters• Nonuniform Memory Access• Vector Computation


Types of Parallel ProcessorTypes of Parallel Processor Multiple Processor Multiple Processor OrganizationOrganization

• Types of Parallel Processor Systems– Single instruction, single data stream - SISD– Single instruction, multiple data stream - SIMD– Multiple instruction, single data stream - MISD– Multiple instruction, multiple data stream- MIMD


Taxonomy Of Parallel ProcessorTaxonomy Of Parallel Processor Multiple Processor Multiple Processor OrganizationOrganization


Alternative Computer OrganizationAlternative Computer Organization Multiple Processor Multiple Processor OrganizationOrganization

• SISD



• SIMD (with distributed memory)



• MIMD(with shared memory)



• MIMD (with distributed memory)


Design IssuesDesign Issues Multiple Processor Multiple Processor OrganizationOrganization

• Physical Organization• Interconnection Structures• Interprocessor Communication• Operating System Design• Application Software Techniques


SMP CharacteristicsSMP Characteristics Symmetric Symmetric MultiprocessorsMultiprocessors

• Two or more similar processors • Share the same main memory and I/O

facilities• All processors share access to I/O devices• All processors can perform the same

functions• The system is controlled by an integrated

OS


SMP AdvantagesSMP Advantages Symmetric Symmetric MultiprocessorsMultiprocessors

• Performance• availability• Incremental growth• Scaling


Multiprogramming & MultiprocessingMultiprogramming & Multiprocessing Symmetric Symmetric MultiprocessorsMultiprocessors

• Interleaving(multiprogramming)



• Interleaving and overlapping(multiprocessing)



• Interleaving(multiprogramming)


OrganizationOrganization• Time-shared or

common bus• Multiport memory• Central control unit

Symmetric Symmetric MultiprocessorsMultiprocessors


TimeTime--Shared BusShared Bus Symmetric Symmetric MultiprocessorsMultiprocessors

• To facilitate DMA transfers from I/O processors, the following features are provided– Addressing

• Distinguish modules on the bus to determine the source and destination of data

– Arbitration• I/O module can temporarily function as “master”

– Time sharing• When module is controlling the bus, other modules

are locked out and must, if necessary, suspend operation until bus access is achieved


Time Shared BusTime Shared Bus Symmetric Symmetric MultiprocessorsMultiprocessors


Bus OrganizationBus Organization’’s Advantagess Advantages Symmetric Symmetric MultiprocessorsMultiprocessors

• Simplicity– The simplest approach to multiprocessor

organization

• Flexibility– Easy to expand the system by attaching more

processors to the bus

• Reliability– The bus is essentially a passive medium


Bus OrganizationBus Organization’’s Drawbacks Drawback Symmetric Symmetric MultiprocessorsMultiprocessors

• Performance– The speed of the system is limited by the bus

cycle time– To equip each processor with a cache memory for

improving performance


MultiportMultiport MemoryMemory Symmetric Symmetric MultiprocessorsMultiprocessors

• The multiport memory allows the direct, independent access of main memory modules by each processor and I/O module

• Advantages– little or no modification is needed for either processor or

I/Omodules to accommodate multiport memory– better performance than bus approach– security

• To configure portions of memory as private to one or more processors and/or I/O module

• Disadvantage– Logic associated with memory is required for resolving

conflicts– more complex than the bus approach


Central Control UnitCentral Control Unit Symmetric Symmetric MultiprocessorsMultiprocessors

• To separate data streams back and forth between independent modules: processor, memory, I/O module

• The controller can buffer requests and perform arbitration and timing functions

• To pass status and control messages between processors

• To perform cache update alerting• Advantages

– flexibility and simplicity of interfacing of the bus approach

• Disadvantages– the control unit is quite complex– A potential performance bottleneck


Multiprocessor OS Multiprocessor OS DeisgnDeisgn Symmetric Symmetric MultiprocessorsMultiprocessors

• An SMP OS manages processor and other computer resources so that the user perceives a single OS controlling system resources

• The key design issues– Simultaneous concurrent processes– Scheduling– Synchronization– Memory management– Reliability and fault tolerance


Mainframe SMPMainframe SMP• IBM S/390 SMP Organization



Mainframe SMPMainframe SMP• PU

– CISC microprocessor– 64-KB L1 cache

• L2 cache– 384 KB– To be arranged in cluster of two– Each cluster supports three PUs

• Bus-switching network adapter(BSN)– to interconnect the L2 caches and the main

memory– To includes a level 3 (L3) cache (2 MB)

• Memory card



Switched InterconnectionSwitched Interconnection Symmetric Symmetric MultiprocessorsMultiprocessors

• The single bus becomes a bottlneck affecting the scalability

• The S/390 copes with this problem in two ways– Main memory is split into four separate cards,

each with its own storage controller that can handle memory accesses at high speeds

– The connection from processors to a single memory card is not in the form of a shared bus but rather point to point links, where each link connects a group of three processors via an L2 cache to a BSN


Shared L2 CachesShared L2 Caches Symmetric Symmetric MultiprocessorsMultiprocessors

• Needs of Shared L2 Caches– In moving from G3(generation 3) to G4, IBM

doubled the speed of the microprocessors. If the G3 organization was retained, a significant increase in bus traffic would occur. At the same time, it was desired to reuse as many G3 components as possible. Without a significant bus upgrade, the BSNs would become a bottleneck

– Analysis of typical S/390 workloads revealed a large degree of sharing of instructions and data among processors

• The use of one or more L2 caches, each of which was shared by multiple processors


L3 CacheL3 Cache Symmetric Symmetric MultiprocessorsMultiprocessors

• Each L3 cache provides a buffer between L2 cachees and one memory card

• To provides the data much more quickly than a main memory access if the requested cache line is already shared by other processors but was not recently used by the requesting processor

38 GB32Memory32 MB14L3 cache5256 KB5L2 cache

8932 KB1L1 cache

Hit Rate(%)Cache SizeAccess Penalty

(PU cycles)Memory

subsystem


Cache Coherence ProblemCache Coherence Problem• Write back

– Write operations are usually made only to the cache.

– Main memory is only updated when the corresponding cache line is flushed from the cache.

• Write through – All write operations are made to main memory as

well as to the cache, ensuring that main memory is always valid.

Cache coherence and Cache coherence and the the mesimesi protocolprotocol


Software SolutionsSoftware Solutions• Compiler-based coherence mechanisms :

analysis on the code to determine which data items may become unsafe for caching, and they mark those items accordingly

• Simplest approach : Prevent any shared data variables from being cached

• Efficient approach : Analyze the code to determine safe periods for shared variables



Hardware SolutionsHardware Solutions• Cache coherence protocols• Dynamic recognition at run time of

potential inconsistency conditions• Hardware schemes can be divided into

two categories– Directory protocols– Snoopy protocols



Directory ProtocolsDirectory Protocols• Collect and maintain information about

where copies of lines reside• Typically, there is a centralized controller

that is part of the main memory controller, and a directory that is stored in main memory

• The directory contains global state information about the contents of the various local cache



Snoopy ProtocolsSnoopy Protocols• Distribute the responsibility for maintaining

cache coherence among all of the cache controllers in a multiprocessor

• A cache must recognize when a line that it holds is shared with other caches

• When an update action is performed on a shared cache line, it must be announced to all other caches by a broadcast mechanism

• Each cache controller is able to “snoop” on the network to observe these broadcasted notifications and react accordingly

• Suited to a bus-based multiprocessor



Snoopy ProtocolsSnoopy Protocols• Two approach to the snoopy protocol

– Write-invalidate• Multiple readers but only one writer• When one of cache wants to write to the line, it

invalidates that line in the other cache and makes the line exclusive to the writing cache

– Write-update(Write-broadcast)• multiple writer as well as multiple reader• When a processor wishes to update a shared line, the

word to be updated is distributed to all others and caches containing that line can update it

• Performance depends on the number of local caches and pattern of memory reads and writes

• To adaptive protocols that employ both



MESI ProtocolMESI Protocol• Modified : The line in the cache has been

modified and is available only in this cache

• Exclusive : The line in the cache is the same as that in main memory and is not present in any other cache

• Shared : The line in the cache is the same as that in main memory and may be present in another cache

• Invalid : The line in the cache does not contain valid data



MESI State Transition DiagramMESI State Transition Diagram(a) Line in cache at initiating processor



MESI State Transition DiagramMESI State Transition Diagram(b) Line in snooping cache



MESI State Transition DiagramMESI State Transition Diagram Cache coherence and Cache coherence and the the mesimesi protocolprotocol


Read MissRead Miss• When a read miss occurs in the local

cache, the processor initiates a memory read to read the line of main memory containing the missing address

• The processor inserts a signal on the bus that alerts all other processor/cache units to snoop the transaction



Read HitRead Hit• When a read hit occurs on a line currently

in the local cache, the processor simply reads the required item

• There is no state change• The state remains modified, shared, or

exclusive



Write MissWrite Miss• When a write miss occurs in the local

cache, the processor initiates a memory read to read the line of main memory containing the missing address

• RWITM(read-with-intent-to-modify)• When the line is loaded, it is immediately

marked modified



Write HitWrite Hit• When a write hit occurs on a line currently

in the local cache, the effect depends on the current state of that line in the local cache:

• Shared• Exclusive• Modified



L1L1--L2 Cache ConsistencyL2 Cache Consistency• Needs to maintain data integrity across

both levels of cache and across all caches in the SMP configuration

• To extend the MESI protocol to the L1 caches– Each line in the L1 cache includes bits to indicate

the state

• Why?– To adopt the write-through policy in the L1 cache– The write through is to the L2 cache and not to the

memory



ClusteringClustering• Clustering is an alternative to symmetric

multiprocessing(SMP)• Cluster

– A group of interconnected, whole computers working together as a unified computing resource that can create the illusion of being one machine

• Whole Computer– A system that can run on its own, apart from the cluster

• Benefits– Absolute scalability– Incremental scalability– High availability– Superior price/performance

ClustersClusters


Cluster ConfigurationsCluster Configurations(a) Standby Server with No Shared Disk

ClustersClusters


Cluster ConfigurationsCluster Configurations(b) Shared Disk

ClustersClusters


Clustering MethodsClustering Methods ClustersClusters

Requires lock manager software. Usually used with disk mirroring or RAID technology

Low network and server overhead. Reduced risk of downtime caused by disk failure

Multiple servers simultaneously share access to disks

Servers share disks

Usually requires disk mirroring or RAID technology to compensate for risk of disk failure.

Reduced network and server overhead due to elimination of copying operations

Servers are cabled to the same disks, but each server owns its disks. If one server fails, its disks are taken over by the other server

Servers connected to disks

High network and server overhead due to copying operations

High availability

Separate servers have their own disks. Data is continuously copied from primary to secondary server

Separate servers

Increased complexityReduced cost because secondary servers can be used for processing

The secondary server is also used for processing tasks

Active secondary

High cost because the secondary server is unavailable for other processing tasks.

Easy to implementA secondary server takes over in case of primary server failure

Passive standby

LimitationsBenefitsDescriptionClustering Method


Operating System Design IssuesOperating System Design Issues• Failure Management

– Failover– Failback

• Load Balancing• Clusters versus SMP

ClustersClusters


Failure ManagementFailure Management• A highly available cluster

– A high probability that all resources will be in service

– If a failure does occur, the queries in progress are lost

– Any lost query will be serviced by a different computer in the cluster

• A fault-tolerant cluster– All resources are always available

– Achieved by the use of redundant shared disks and mechanisms for backing out uncommitted transactions

– Failover : Switching resources over from a failed to alternative system

– Failback : restoring resources to the original system

ClustersClusters


Load BalancingLoad Balancing• An effective capability for balancing the

load among available computer includes the requirement that the cluster be incrementally scalable

• Middleware mechanisms need to recognize that services can appear on different members of the cluster and may migrate form one member to another

ClustersClusters


Clusters Clusters vsvs SMPSMP• The advantages of the SMP approach

– SMP is easier to manage and configure than a cluster

– SMP usually takes up less physical space and draws less power than cluster

• The advantages of the cluster approach– To result in clusters dominating the high-

performance server market– Superior in terms of incremental and absolute

scalability– Superior in terms of availability

• All components of the system can readily be made highly redundant

ClustersClusters


NonuniformNonuniform Memory AccessMemory Access• Uniform memory access (UMA)

– All processors have access to all parts of main memory using loads and stores

– The memory access time of a processor to all regions of memory is the same

• Nonuniform memory access(NUMA)– All processors have access to all parts of main

memory using loads and stores– The memory access time of a processor differs

depending on which region of main memory is accessed

• Cache-coherent NUMA(CC-NUMA)– A NUMA system in which cache coherence is

maintained among the caches of the various processor

NonuniformNonuniform memory memory accessaccess


MotivationMotivation• A practical limit to the number of

processors that can be used– As the number of processors increase, bus traffic

also increases– The bus is used to exchange cache-coherence

signals, adding to the burden• An effective cache scheme reduces the

bus traffic between any one processor and main memory

• The objective with NUMA is to maintain a transparent system-wide memory while permitting multiple multiprocessor nodes, each with its own bus or other internal interconnect system



CCCC--NUMA OrganizationNUMA Organization NonuniformNonuniform memory memory accessaccess


OrganizationOrganization• Example

– Suppose the P2-3 requests a memory location 798, the memory of node 1

1. P2-3 issues read request on the snoopy bus of node 22. The directory on node 2 sees the request and recognizes

that the location is in lode13. Node 2’s directory sends a request to node 14. Node 1’s directory requests the contents of 7985. Node 1’s main memory responds by putting the requested

data on bus6. Node 1’s directory picks up the data from the bus7. The value is transferred back to node 2’s directory8. Node 2’s directory places the data back no node 2’bus9. The value is picked up and placed in P2-3’s cache and

delivered to P2-3



NUMA Pros and ConsNUMA Pros and Cons• The advantage of a CC-NUMA system

– Can deliver effective performance at higher levels of parallelism than SMP, without requiring major softwa4re changes

• The disadvantages of a CC-NUMA system– Not transparently look like an SMP

• Software changes will be required to move an operating system and applications from an SMP to a CC-NUMA system

– Availability



Vector ComputationVector Computation• The key to the design of a supercomputer

or array processor is to recognize that the main task is to perform arithmetic operations on arrays or vectors of floating-point numbers

• In general purpose computer, this will require iteration through each element of the array

• Vector processing• Parallel processing

Vector Vector computationcomputation


Example of Vector AdditionExample of Vector Addition Vector Vector computationcomputation


Matrix Multiplication (C = A x B)Matrix Multiplication (C = A x B) Vector Vector computationcomputation


OrganizationsOrganizations• Pipelined ALU• Parallel ALUs• Parallel processors



Approaches to Vector ComputationApproaches to Vector Computation(a) Pipelined ALU



Approaches to Vector ComputationApproaches to Vector Computation(b) Parallel ALUs



Pipelined ProcessingPipelined Processing Vector Vector computationcomputation


Ex. Computing C=(s x A) + BEx. Computing C=(s x A) + B1. Vector load A->Vector

Register(VR1)2. Vector load B->VR23. Vector multiply s x VR1->VR34. Vector add VR3 + VR2 -> VR45. Vector store VR4->C



Taxonomy of Computer OrganizationTaxonomy of Computer Organization Vector Vector computationcomputation


Taxonomy of Computer OrganizationTaxonomy of Computer Organization• Parallel processors

– The multiple processors can function cooperatively on a given task

• Vector processor– often equated with pipelined ALU organization– Also designed by a parallel ALU organization and a parallel

processor

• Array processing– Usually refers to a Parallel ALU– But any of the three organizations is optimized for the processing

of array– Usually refers to an auxiliary processor attached to a general-

purpose processor and used to perform vector computation

• The pipelined ALU organization dominates the marketplace

– less complex



IBM 3090 Vector FacilityIBM 3090 Vector Facility• IBM facility makes use of a number of

vector registers• Each register is actually a bank of scalar

registers• To compute vector sum C=A+B, the vectors

A and B are loaded into two vector registers• The data from these registers are passed

through the ALU as fast as possible• The computation overlap, and the loading of

the input data into the resisters in a block, results in a significant speeding up over an ordinary ALU operation

Vector computationVector computation


OrganizationOrganization• The fixed and predetermined structure of

vector data permits housekeeping instructions inside the loop to be replaced by faster internal machine operations

• Data-access and arithmetic operations on several successive vector elements can proceed concurrently by overlapping such operations in a pipelined design or by performing multiple-element operations in parallel

• The use of vector registers for intermediate results avoids additional storage reference



IBM 3090 with Vector FacilityIBM 3090 with Vector Facility Vector computationVector computation


RegistersRegisters• The IBM organization is referred to as

register-to-register, because the vector operands, both input and output, can be stored in vector registers

• The main disadvantage of the use of vectro registers is that the programmer or compiler must take them into account



Alternative ProgramsAlternative Programs Vector computationVector computation


Alternative ProgramsAlternative Programs(a) Storage to Storage



Alternative ProgramsAlternative Programs Vector computationVector computation

(b) Register to Register


Alternative ProgramsAlternative Programs(c) Storage to Register



Alternative ProgramsAlternative Programs(d) Compound Instructions



RegistersRegisters• The vector registers can also be coupled

to form 8 64-bit vector registers• The architecture specifies that each

register contains from 8 to 512 scalar elements

• Three additional registers are needed by the vector facility



Registers of the IBM 3090Registers of the IBM 3090 Vector computationVector computation


Compound InstructionCompound Instruction• Instruction execution can be overlapped

using chining to improve performance • The designers of the IBM vector facility

chose not to include this capability for several reasons

• Compound instruction do not require the use of additional registers for temporary storage of intermediate results and they require one less register



The Instruction SetThe Instruction Set• There are memory-to-register load and

register-to-memory store instructions • Many of instruction use a three operand

format



IBM 3090 Vector FacilityIBM 3090 Vector Facility Vector computationVector computation

chapter 16 parallel processing - yonsei...

Documents