Flynn’s Taxonomy

• SISD: Although instruction execution may be pipelined, computers in this category can decode only a single instruction in unit time

• SIMD: An array of processors all executing the same instruction but on different data

Flynn’s Taxonomy

• MISD: Some consider this category to be empty. However, some consider systolic arrays to fall in this category: – Data is pumped through processors, each

processor applying a different operation to the same data stream.

• MIMD: More than one CPU, each running its own program on its own data

Speedup & Efficiency

• Speedup = (Execution Time on N processors) / (Execution Time on 1 Single Processor)

• NB: Must be the most efficient serial algorithm

• Efficiency = Speedup / N

• Example: if sequential time = 8 sec, parallel time = 2 sec (on 5 processors), then Speedup = 4, Efficiency = 0.8

Speedup

• Parallelizability: Same as speedup but comparing against the parallel algorithm running on a 1-processor machine

• Can speedup be greater than 1?– Called superlinear speedup– Some say not possible because: a single processor

can always emulate thru time-slicing– Moreover, since parallelization always contains some

overhead, some argue it must be sublinear.

Superlinear Speedup?

• Some say it is possible because: – sometimes one of the parallel processors may

luck out.– Architecturally, emulation is never perfect

because of factors such as cache hit rate

Control and Data Parallelism

• Control Parallelism: Different operations are applied to different data elements simultaneously

• Data Parallelism: Same operations are applied to different different data

• Example: Sieve of Erastosthenes

Amdahl’s Law

• If f is an inherently sequential fraction of execution time, then

speedup 1 / [f + (1-f)/p]

• Corollary: Maximum speedup = 1/f