1

Tuesday, September 26, 2006

Wisdom consists of knowing when to avoid perfection.

- Horowitz

2

Quiz 2Assignment 1

3

Hypercube: log p dimensions with two nodes in each dimension

0-D hypercube

4

Hypercube: log p dimensions with two nodes in each dimension

1-D hypercube

0-D hypercube

5

Hypercube: log p dimensions with two nodes in each dimension

2-D hypercube

1-D hypercube

6

Hypercube: log p dimensions with two nodes in each dimension

3-D hypercube

2-D hypercube

7

Hypercube: log p dimensions with two nodes in each dimension

3-D hypercube

4-D hypercube

Each node is connected to d=log p other nodes

8

•Numbering

•Minimum distance between nodes

9

Diameter: Maximum distance between any two processing nodes in the network Ring 2-D Mesh Hypercube

10

Diameter: Maximum distance between any two processing nodes in the network Ring

• └p/2┘ 2-D Mesh

• 2(√p -1) no-wraparound• 2 └(√p /2) ┘ wraparound

Hypercube• log p

11

Connectivity: Multiplicity of paths Minimum arcs that need to be removed to disconnect

the network into twoRing

• 2 2-D Mesh

• 2 no-wraparound• 4 wraparound

Hypercube• d=log p

12

Bisection width: Minimum arcs that need to be removed to partition the

network into two equal halvesRing

• 2 2-D Mesh

• √p no-wraparound• 2√p wraparound

Hypercube• p/2

13

14

Domain Decomposition

In this type of partitioning, the data associated with a problem is decomposed. Each parallel task then works on a portion of the data.

15

Domain Decomposition

16

Functional Decomposition

17

Signal processing

18

Climate modeling

.

19

Examples of decomposition and task dependencies

20

Examples of decomposition and task dependencies.

21

Examples of decomposition and task dependencies.

22

Granularity

Fine vs. Coarse Decomposition in large number of small tasks

vs. small number of large tasks.

Maximum degree of concurrencyAverage degree of concurrencyConcurrency vs. Granularity?

23

Granularity

24

Granularity

Critical Path length: Longest directed path between any pair of start

and finish nodes is critical path

Average degree of concurrency: Ratio of total amount of work to the critical

path length

25

Granularity

•Another example

26

Granularity

Measure of the ratio of computation to communication.

Fine-grain Parallelism: Facilitates load balancing Implies high communication overhead and less

opportunity for performance enhancement Coarse-grain Parallelism:

High computation to communication ratio Implies more opportunity for performance increase Harder to load balance efficiently

27

Granularity

Example: Domain decompositions for a problem

involving a three-dimensional grid.