parallel programming concept dependency and loop parallelization
TRANSCRIPT
![Page 1: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/1.jpg)
PARALLEL PROGRAMMING CONCEPT DEPENDENCY AND LOOP PARALLELIZATION
•Parallel programming concept and their examples
•Dependency and their two types with examples
•Loop parallelism and their types with examples R.AISHWARYA
![Page 2: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/2.jpg)
Parallel programming???
![Page 3: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/3.jpg)
![Page 4: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/4.jpg)
• Simultaneous use of multiple compute resources to solve a computational problem
• Compute resources
• Primary reasons
• Best practices
• Goals
• Steps
![Page 5: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/5.jpg)
![Page 6: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/6.jpg)
![Page 7: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/7.jpg)
LOOP DEPENDENCY
![Page 8: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/8.jpg)
Statement 2: b=a+2; “5” Statement 1: a=5;
![Page 9: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/9.jpg)
•Find dependencies within iterations of a loop•Goal of determining different relationships between
statements.•To allow multiple processors to work on different
portions of the loop in parallel•First analyze the dependencies within individual
loops. •It help determine which statements in the loop need
to be completed before other statements can start.•Two general categories of dependencies: Data and
Control dependency
![Page 10: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/10.jpg)
function Dep(a,b)c:=a.bd:=2.cend function
(flow dependency)
function Nodep(a,b)c:=a.bd:=2.be:=a+bend function
(no dependency)
![Page 11: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/11.jpg)
DATA DEPENDENCY
TYPE
NOTATION DESCRIPTION
True (Flow) Dependence
S1 ->T S2
A true dependence between S1 and S2 means that S1 writes to a location later read from by S2
Anti Dependence S1 ->A S2
An anti-dependence between S1 and S2 means that S1 reads from a location later written to by S2.(before)
Output Dependence S1 ->I S2
An input dependence between S1 and S2 means that S1 and S2 read from the same location.
![Page 12: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/12.jpg)
EXAMPLESTrue dependenceS0: int a, b;S1: a = 2;S2: b = a + 40;S1 ->T S2, meaning that S1 has a true dependence on S2 because S1writes to the variable a, which S2 reads from.Anti-dependenceS0: int a, b = 40;S1: a = b - 38;S2: b = -1;S1 ->A S2, meaning that S1 has an anti-dependence on S2 because S1reads from the variable b before S2 writes to it.Output-dependenceS0: int a, b = 40;S1: a = b - 38;S2: a = 2;S1 ->O S2, meaning that S1 has an output dependence on S2 because both write to the variable a.
![Page 13: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/13.jpg)
CONTROL DEPENDENCYif(a == b)then{ c = “controlled”;}d=“not controlled”;
if(a == b)then{ }c = “controlled”;d=“not controlled”;
if(a == b)then{ c = “controlled”; d=“not controlled”;}
![Page 14: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/14.jpg)
DEPENDENCY IN LOOP
Loops can have two types of dependence:
•Loop-carried dependency•Loop-independent dependency
![Page 15: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/15.jpg)
LOOP CARRIED DEPENDENCY
• In loop-carried dependence, statements in an iteration of a loop depend on statements in another iteration of the loop.
for(i=0;i<4;i++){S1: b[i]=8;S2: a[i]=b[i-1] + 10;}
![Page 16: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/16.jpg)
LOOP INDEPENDENT DEPENDENCY
• In loop-independent dependence, loops have inter-iteration dependence, but do not have dependence between iterations.
• Each iteration may be treated as a block and performed in parallel without other synchronization efforts.
for (i=0;i<4;i++){S1: b[i] = 8;S2: a[i] =b[i] + 10;}
![Page 17: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/17.jpg)
for (i=1; i<4; i++)for (j=1; j<4; j++)S3: a[i][j] = a[i][j-1] + 1;
Node : Point in the iteration spaceDirected Edge: Dependency
Node: Point in the iteration spaceDirected Edge: next point that will be encounteredafter the current point is traversed
![Page 18: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/18.jpg)
LOOP PARALLELIZATION
![Page 19: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/19.jpg)
•Extraction parallel tasks from loops
•Data is stored in random access data structures
•A program exploiting loop-level parallelism will use multiple threads or processes which operate on same time
•It provides speedup
•Amdhal’s law
![Page 20: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/20.jpg)
Examples of Loop parallelization
for (int i = 0; i < n; i++) { S1: L[i] = L[i] + 10;}
for (int i = 1; i < n; i++){ S1: L[i] = L[i - 1] + 10;}
![Page 21: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/21.jpg)
Can the following Loop be made Parallel?
for (i=1;i<=100;i=i+1){A[i+1] = A[i] + C[i]; /*S1*/B[i+1] = B[i] + A[i+1]; /*S2*/}
![Page 22: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/22.jpg)
METHODOLOGIES FOR PARALLELIZING LOOPS
• DISTRIBUTED Loop
• DOALL Parallelism
• DOACROSS Parallelism
• HELIX • DOPIPE Parallelism
![Page 23: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/23.jpg)
DISTRIBUTED LOOP
for (int i = 1; i < n; i ++) { S1: a[i] = a[i -1] + b[i]; S2: c[i] = c[i] + d[i]; }
loop1: for (int i = 1; i < n; i ++) { S1: a[i] = a[i -1] + b[i]; }loop2: for (int i = 1; i < n; i ++) { S2: c[i] = c[i] + d[i];}
![Page 24: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/24.jpg)
DO ALL PARALLELISM
for (int i = 0; i < n; i++) { S1: a[i] = b[i] + c[i]; }
begin_parallelism();for (int i = 0; i < n; i++) { S1: a[i] = b[i] + c[i]; end_parallelism();}block();
![Page 25: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/25.jpg)
DO ACROSS PARALLELISM
for (int i = 1; i < n; i++) { a[i] = a[i - 1] + b[i] + 1;}
S1: int tmp = b[i] + 1;S2: a[i] = a[i - 1] + tmp;
post(0);for (int i = 1; i < n; i++) { S1: int tmp = b[i] + 1; wait(i - 1); S2: a[i] = a[i - 1] + tmp; post(i);}
![Page 26: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/26.jpg)
DO PIPE PARALLELISM
for (int i = 1; i < n; i++) { S1: a[i] = a[i - 1] + b[i]; S2: c[i] = c[i] + a[i];}
for (int i = 1; i < n; i++) { S1: a[i] = a[i - 1] + b[i]; post(i);} for (int i = 1; i < n; i++) { wait(i); S2: c[i] = c[i] + a[i];}
![Page 27: Parallel programming concept dependency and loop parallelization](https://reader035.vdocument.in/reader035/viewer/2022070520/58f9b7cb1a28ab0c238b45cd/html5/thumbnails/27.jpg)
THANK YOU