![Page 1: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/1.jpg)
Parallel Algorithms on Networks of Processors
Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill)School of Computing, Clemson University
![Page 2: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/2.jpg)
2
OutlineWhat are parallel algorithms? Why use them?
Challenges for parallel algorithm designers
Choosing a network
Partitioning the data
Designing the algorithm
Example
Recurrences (binary tree)
Analysis (Speedup and Efficiency)
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 3: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/3.jpg)
3
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 4: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/4.jpg)
4
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 5: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/5.jpg)
5
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 6: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/6.jpg)
6
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 7: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/7.jpg)
7
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 8: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/8.jpg)
8
Why Parallel Computation?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 9: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/9.jpg)
9
New ProcessorsFaster and Cheaper
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
January2011
![Page 10: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/10.jpg)
10
Partition the Data
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 11: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/11.jpg)
11
Organize the Processors
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 12: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/12.jpg)
12
Build a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 13: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/13.jpg)
13
Build a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 14: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/14.jpg)
14
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 15: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/15.jpg)
15
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 16: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/16.jpg)
16
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 17: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/17.jpg)
17
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 18: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/18.jpg)
18
Choosing a Network
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 19: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/19.jpg)
19
Are There Really Any Multiprocessing
Systems in Use Today?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 20: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/20.jpg)
20
Are There Really Any Multiprocessing
Systems in Use Today?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
HamburgJune 2011Top 500
![Page 21: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/21.jpg)
21
SupercomputersNEC/HP Tsubame
(Japan)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
1.192 petaflops ≈ 1.28 quadrillion floating point
operations per sec
73,278 Xeon cores
Infiniband grid network
![Page 22: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/22.jpg)
22
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersDawning Nebulae
(China)1.27 petaflops ≈ 1.36
quadrillion floating point operations per sec
9280 Intel 6-core Xeon processors = 55,680 cores
Infiniband grid network
![Page 23: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/23.jpg)
23
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersCray Jaguar (USA)
1.75 petaflops ≈ 1.876 quadrillion floating point
operations per sec
224,256 AMD cores
3D torus network
![Page 24: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/24.jpg)
24
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersNUDT Tianhe-1A
(China)2.566 petaflops ≈ 2.75
quadrillion floating point operations per sec
14336 CPUs
Undisclosed proprietary network
![Page 25: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/25.jpg)
25
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
SupercomputersFujitsu “K” (Japan)
K = “kei” = Japanese for 10 quadrillion
8.162 petaflops ≈ 9 quadrillion floating point operations per
sec
68,544 8-core SPARC64 processors = 548,352
cores
3-dimensional torus network called Tofu
![Page 26: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/26.jpg)
26
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
TOP500
Top 500 Computers in the World
![Page 27: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/27.jpg)
27
Where Does that Leave Us?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 28: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/28.jpg)
28
Where Does that Leave Us?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
![Page 29: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/29.jpg)
29
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 30: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/30.jpg)
30
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 31: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/31.jpg)
31
Where Does that Leave Us?
In a wonderful playground of mathematical algorithmic design where imagination and creativity are key!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 32: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/32.jpg)
32
Challenges for Parallel Algorithm
DesignersChoosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 33: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/33.jpg)
33
So Let’s Try It:
Choosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 34: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/34.jpg)
34
So Let’s Try It:Elliptic Partial Diff
EqnsChoosing a network
Partitioning the problem
Designing the parallel algorithm
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 35: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/35.jpg)
35
Elliptic PDEs
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplace’s or Poisson’s equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation.
![Page 36: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/36.jpg)
36
Designing the Algorithm
Why Solve Linear Recurrences?
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 37: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/37.jpg)
37
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 38: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/38.jpg)
38
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 39: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/39.jpg)
39
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 40: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/40.jpg)
40
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 41: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/41.jpg)
41
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences many many many times
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 42: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/42.jpg)
42
Designing the Algorithm
Why Solve Linear Recurrences?
The problem: Solving PDEs using the Method of Finite Differences leads to
Solving Block Tridiagonal Systems which leads to
Solving Tridiagonal Systems which leads to
Solving Linear Recurrences many many many times
Why Use a Binary Tree? Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 43: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/43.jpg)
43
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Key Idea: Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently.
![Page 44: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/44.jpg)
44
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Consider the following set of n equations:x0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
![Page 45: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/45.jpg)
45
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Consider the following set of n equations:x0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
Can we solve for xi in parallel?
![Page 46: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/46.jpg)
46
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
For uniformity:x0 = a0 + b0 x-1 b0=0, x-1=dummy variablex1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
![Page 47: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/47.jpg)
47
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
For uniformity:x0 = a0 + b0 x-1
x1 = a1 + b1 x0
x2 = a2 + b2 x1
...
xn-1 = an-1 + bn-1 xn-2
Observe, ifxi = a + b xj
xj = a’ + b’ xk
Thenxi = (a + ba’) +bb’ xk
= a” + b” xk
![Page 48: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/48.jpg)
48
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Notation changex0 = a0 + b0 x-1 C0,-1 = (a0,b0)x1 = a1 + b1 x0 C1,0 = (a1,b1)x2 = a2 + b2 x1 C2,1 = (a2,b2)...
xn-1 = an-1 + bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1)
Observe, ifxi = a + b xj
xj = a’ + b’ xk
Thenxi = (a + ba’) +bb’ xk
= a” + b” xk
![Page 49: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/49.jpg)
49
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Notation changex0 = a0 + b0 x-1 C0,-1 = (a0,b0)x1 = a1 + b1 x0 C1,0 = (a1,b1)x2 = a2 + b2 x1 C2,1 = (a2,b2)...
xn-1 = an-1 + bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1)
Observe, ifxi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
![Page 50: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/50.jpg)
50
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
![Page 51: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/51.jpg)
51
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:
![Page 52: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/52.jpg)
52
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved.
![Page 53: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/53.jpg)
53
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b)
![Page 54: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/54.jpg)
54
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b) means that b=0
![Page 55: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/55.jpg)
55
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
To summarize:xi = a + b xj Ci,j = (a,b)xj = a’ + b’ xk Cj,k = (a’,b’)
Thenxi = (a + ba’) +bb’ xk Ci,j Cj,k =
= a” + b” xk Ci,k = (a+ba’,bb’)
One last observation:If any variable is expressed in terms of
the dummy variable x-1 (e.g., x0 = a0 + b0 x-1) then that variable is solved. So Ci,-1 = (a,b) means that b=0 and that xi = a + b x-1 = a + 0 x-1 = a
![Page 56: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/56.jpg)
56
C0,-
1
C1,
0
C2,
1
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6 Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear Recurrences
![Page 57: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/57.jpg)
57
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
![Page 58: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/58.jpg)
58
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
![Page 59: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/59.jpg)
59
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
![Page 60: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/60.jpg)
60
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
![Page 61: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/61.jpg)
61
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
![Page 62: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/62.jpg)
62
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
![Page 63: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/63.jpg)
63
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
![Page 64: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/64.jpg)
64
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
Solved variable
![Page 65: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/65.jpg)
65
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
Solved variables
![Page 66: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/66.jpg)
66
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C6,5C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1C7,-1
How do we solve for the other variables?
![Page 67: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/67.jpg)
67
C6,5
C7,-1
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
C7,-
1
In the downsweep!
![Page 68: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/68.jpg)
68
C6,5
C7,-1
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
(x7,x-
1)
![Page 69: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/69.jpg)
69
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
C7,-
1
(x7,x-
1)
![Page 70: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/70.jpg)
70
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
C7,3 C3,-1
x3
(x7,x-
1)
![Page 71: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/71.jpg)
71
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C7,5 C5,3C3,1 C1,-1
C7,
3
C3,-
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
![Page 72: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/72.jpg)
72
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
C3,1 C1,-1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5C7,5 C5,3
x1
![Page 73: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/73.jpg)
73
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
C7,
5
C5,
3
C3,
1
C1,-
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x7,x5) (x5,x3) (x3,x1) (x1,x-1)
![Page 74: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/74.jpg)
74
(x7,x5)
C6,5
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-1C1,0C2,1C3,2C4,3
C5,4C7,6
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
![Page 75: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/75.jpg)
75
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
C0,-
1
C1,
0
C3,
2
C4,
3
C5,
4
C6,
5
C7,
6
C2,
1
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
![Page 76: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/76.jpg)
76
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
![Page 77: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/77.jpg)
77
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
Leaves contain solutions!
![Page 78: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/78.jpg)
78
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
But we can do better!
![Page 79: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/79.jpg)
79
(x7,x5)
(x7,x-1)
Linear Recurrences
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
x3
(x7,x-
1)
(x7,x3) (x3,x-1)
x5 x1
(x5,x3) (x3,x1) (x1,x-1)
x6 x4 x3 x1
(x7,x6) (x6,x5) (x5,x4)(x0,x-1)(x1,x0)(x4,x3) (x3,x2) (x2,x1)
(x7,x6) (x0,x-
1)(x6,x5) (x5,x4) (x4,x3) (x3,x2) (x2,x1) (x1,x0)
Pipelining the solution
![Page 80: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/80.jpg)
80
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 81: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/81.jpg)
81
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 82: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/82.jpg)
82
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 83: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/83.jpg)
83
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 84: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/84.jpg)
84
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 85: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/85.jpg)
85
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 86: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/86.jpg)
86
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 87: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/87.jpg)
87
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 88: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/88.jpg)
88
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 89: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/89.jpg)
89
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 90: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/90.jpg)
90
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Linear RecurrencesPipelining the solution
![Page 91: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/91.jpg)
91
AnalysisT1 = Time on one processor
Tn = Time on n processors
S = Speedup = T1/Tn (ideal: S = n)
E = Efficiency = S/n (ideal: E = 1)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 92: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/92.jpg)
92
Speedup and Efficiency
Single PassAssume 1 floating point operation requires 1 time unit
T1 = (n−1) (1 mult + 1 add) = 2n−2 time units
n leaves 2n processors T2n = (log2n)( 2 mults + 1 add) // upsweep
+ (log2n)(1 mult + 1 add) // downsweep= 5 log2n time units
S = Speedup = T1/T2n = (2n−2)/(5 log2n)
E = Efficiency = S/2n = (2n−2)/[ (5 log2n) (2n) ]
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 93: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/93.jpg)
93
Speedup
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0 200 400 600 800 1000 12000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Series1
Speedup and Efficiency
Single Pass
![Page 94: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/94.jpg)
94
Speedup Efficiency
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0 200 400 600 800 1000 12000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Series1
0 50 100 150 200 250 300 3500
0.01
0.02
0.03
0.04
0.05
0.06
Series1
Speedup and Efficiency
Single Pass
![Page 95: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/95.jpg)
95
With Pipelining. Assume kn equations for large k
T1 = (kn−1) (1 mult + 1 add) = 2kn−2 time units
n leaves 2n processors T2n = (log2n)( 2 mults + 1 add) // pipefill up
+ (k – 2 log 2n) (5) // pipeline on k−2log 2n waves
+ (log2n)( 1 mult + 1 add) // pipedrain
down= 5(k – log2n) time units
S = Speedup = T1/T2n = (2kn−2)/[5(k – log2n )]
E = Efficiency = S/2n = (2kn−2)/[5(k – log2n ) (2n)]
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With Pipelining
![Page 96: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/96.jpg)
96
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With PipeliningSpeedup
0 200 400 600 800 1000 12000
50
100
150
200
250
300
350
400
450
Series1
![Page 97: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/97.jpg)
97
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Speedup and Efficiency
With PipeliningSpeedup Efficiency
0 200 400 600 800 1000 12000
50
100
150
200
250
300
350
400
450
Series1
0 200 400 600 800 1000 12000.1999
0.2000
0.2001
0.2002
0.2003
Series1
![Page 98: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/98.jpg)
98
Technique Can Work For
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Second-order linear recurrencesx0 = a0
x1 = a1 + b1 x0
x2 = a2 + b2 x1 + c2 x0
...
xn-1 = an-1 + bn-1 xn-2 + cn-1 xn-3
Higher order linear recurrences
![Page 99: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/99.jpg)
99
Technique Can Work For
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Quotients of linear recurrencesx0 = a0
xi = (ai + bi xi-1)/(ci + di xi-1) i=1,2,…, n-1
Other recurrences
![Page 100: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/100.jpg)
100
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 101: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/101.jpg)
101
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future.
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 102: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/102.jpg)
102
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
![Page 103: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/103.jpg)
103
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 104: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/104.jpg)
104
Summary and Conclusions
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
The challenging world of parallel algorithmic design beckons and awaits creative minds.
![Page 105: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/105.jpg)
105
Summary and Conclusions
Chip technology is going to get even better/faster/cheaper for the foreseeable future. (Ignore the naysaying pundits!)
More massively parallel processing systems are going to be built and will become even better/faster/cheaper.
The challenging world of parallel algorithmic design beckons and awaits creative minds. Yes, this means you!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 106: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/106.jpg)
106
Where in the World is Clemson
University?
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
We are here!
Roy Pargas, Clemson University [email protected]
July 30, 2011
![Page 107: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/107.jpg)
107
Thank you for your kind attention!
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Questions?
![Page 108: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/108.jpg)
108
Extra Slides
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
![Page 109: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/109.jpg)
109
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Links
1. Fujitsu K Computer (K = “kei” = Japanese word for 10 quadrillion)
http://www.fujitsu.com/global/about/tech/k/http://en.wikipedia.org/wiki/K_computer
2. NUDT “Tianhe-1A” Computerhttp://blog.zorinaq.com/?e=36http://en.wikipedia.org/wiki/Tianhe-I
3. Cray Jaguarhttp://en.wikipedia.org/wiki/Jaguar_(computer)http://www.nccs.gov/jaguar/
4. Dawning Nebulaehttp://en.wikipedia.org/wiki/Dawning_Information_Industryhttp://www.theregister.co.uk/2010/05/31/top_500_supers_jun2010/
5. NEC/HP Tsubame 2.0 http://en.wikipedia.org/wiki/TOP500
![Page 110: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/110.jpg)
110
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
![Page 111: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/111.jpg)
111
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
![Page 112: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/112.jpg)
112
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
![Page 113: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/113.jpg)
113
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
![Page 114: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/114.jpg)
114
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
![Page 115: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/115.jpg)
115
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
![Page 116: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/116.jpg)
116
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
0 1
![Page 117: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/117.jpg)
117
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
0 1
0 1
![Page 118: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/118.jpg)
118
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
![Page 119: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/119.jpg)
119
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
![Page 120: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/120.jpg)
120
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
4-Degree
0-Degree
1-Degree
0 1
00
01
10
11
2-Degree
![Page 121: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/121.jpg)
121
Hypercube
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
3-Degree
4-Degree
5-Degree
![Page 122: Parallel Algorithms on Networks of Processors](https://reader038.vdocument.in/reader038/viewer/2022103101/56814315550346895daf68c2/html5/thumbnails/122.jpg)
122
All-to-All Communication
(Hypercube)
Roy Pargas, Clemson University [email protected]
July 30, 2011
50 Golden Years Ateneo Mathematics Program Quezon City, Philippines
Problem
Motivation