concurrency - software-lab.org€¦ · concurrency (part 1) 2 - 1 wake-up exercise what does this...
TRANSCRIPT
![Page 1: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/1.jpg)
1
Prof. Dr. Michael Pradel
Software Lab, University of StuttgartWinter 2019/2020
Programming Paradigms
Lecture 16:
Concurrency (Part 1)
![Page 2: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/2.jpg)
2 - 1
Wake-up Exercise
What does this Java code print?
https://ilias3.uni-stuttgart.de/vote/0ZT9
class Warmup {static boolean flag = false;static void raiseFlag() {flag = true;
}public static void main(String[] args)
throws Exception {ForkJoinPool.commonPool().execute(Warmup::raiseFlag);
while (!flag) {};System.out.println(flag);
}}
![Page 3: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/3.jpg)
2 - 2
Wake-up Exercise
What does this Java code print?
https://ilias3.uni-stuttgart.de/vote/0ZT9
class Warmup {static boolean flag = false;static void raiseFlag() {flag = true;
}public static void main(String[] args)
throws Exception {ForkJoinPool.commonPool().execute(Warmup::raiseFlag);
while (!flag) {};System.out.println(flag);
}}
raiseFlag:executed inconcurrentthread
![Page 4: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/4.jpg)
2 - 3
Wake-up Exercise
What does this Java code print?
https://ilias3.uni-stuttgart.de/vote/0ZT9
class Warmup {static boolean flag = false;static void raiseFlag() {flag = true;
}public static void main(String[] args)
throws Exception {ForkJoinPool.commonPool().execute(Warmup::raiseFlag);
while (!flag) {};System.out.println(flag);
}}
Shared variableaccessed bytwo threads
![Page 5: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/5.jpg)
2 - 4
Wake-up Exercise
What does this Java code print?
https://ilias3.uni-stuttgart.de/vote/0ZT9
class Warmup {static boolean flag = false;static void raiseFlag() {flag = true;
}public static void main(String[] args)
throws Exception {ForkJoinPool.commonPool().execute(Warmup::raiseFlag);
while (!flag) {};System.out.println(flag);
}}
Problem: Nosynchronization.Hence, mainthread may readold value
![Page 6: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/6.jpg)
2 - 5
Wake-up Exercise
What does this Java code print?
https://ilias3.uni-stuttgart.de/vote/0ZT9
class Warmup {static boolean flag = false;static void raiseFlag() {flag = true;
}public static void main(String[] args)
throws Exception {ForkJoinPool.commonPool().execute(Warmup::raiseFlag);
while (!flag) {};System.out.println(flag);
}}
Code may hang forever,print true, or print false!
![Page 7: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/7.jpg)
3
Overview
� Introduction
� Concurrent ProgrammingFundamentals
� Implementing Synchronization
� Language-level Constructs
![Page 8: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/8.jpg)
4
Motivation
Why do we care about concurrency?
� To capture the logical structure of a problem
� Inheritently concurrent problems, e.g., server
handling multiple requests
� To exploit parallel hardware for speed
� Since around 2005: Multi-core processors are the
norm
� To cope with physical distribution
� Local or global groups of interacting machines
![Page 9: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/9.jpg)
5
Terminology
� Concurrent� Two or more running tasks whose execution
may be at some unpredictable point
� Parallel� Two or more tasks are actively executing at the
same time
� Requires multiple processor cores
� Distributed� Physically separated processors
![Page 10: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/10.jpg)
6 - 1
Levels of Parallelism
� Signals propagating through circuitsand gates
� Instruction-level parallelism� E.g., load from memory while another
instruction executes
� Vector parallelism� E.g., GPUs execute a single instruction on a
vector of data
� Thread-level parallelism
![Page 11: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/11.jpg)
6 - 2
Levels of Parallelism
� Signals propagating through circuitsand gates
� Instruction-level parallelism� E.g., load from memory while another
instruction executes
� Vector parallelism� E.g., GPUs execute a single instruction on a
vector of data
� Thread-level parallelism
Handled implicitlyby hardware
![Page 12: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/12.jpg)
6 - 3
Levels of Parallelism
� Signals propagating through circuitsand gates
� Instruction-level parallelism� E.g., load from memory while another
instruction executes
� Vector parallelism� E.g., GPUs execute a single instruction on a
vector of data
� Thread-level parallelism
Specified byprogrammer in PL
![Page 13: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/13.jpg)
7 - 1
Example: Independent Tasks
// Task Parallel Library in C#Parallel.For(0, 100, i => {A[i] = foo(A[i]);
});
![Page 14: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/14.jpg)
7 - 2
Example: Independent Tasks
// Task Parallel Library in C#Parallel.For(0, 100, i => {A[i] = foo(A[i]);
});
Array ofdata
Function that updates eachelement independently
![Page 15: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/15.jpg)
7 - 3
Example: Independent Tasks
// Task Parallel Library in C#Parallel.For(0, 100, i => {A[i] = foo(A[i]);
});
Array ofdata
Function that updates eachelement independently
� No need to synchronize tasks
� Uses as many cores as possible (up to 100)
![Page 16: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/16.jpg)
8 - 1
Example: Dependent Tasks
// As before, but foo now is:int zero_count;public static int foo(int n) {int rtn = n - 1;if (rtn == 0) zero_count++;return rtn;
}
![Page 17: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/17.jpg)
8 - 2
Example: Dependent Tasks
// As before, but foo now is:int zero_count;public static int foo(int n) {int rtn = n - 1;if (rtn == 0) zero_count++;return rtn;
}
Count how many zeroswritten to the array
![Page 18: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/18.jpg)
1
![Page 19: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/19.jpg)
10
Data Races
� Definition of data race
� Two accesses to the same shared memory
location
� At least one access is a write
� Ordering of accesses is non-deterministic
![Page 20: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/20.jpg)
11
Overview
� Introduction
� Concurrent ProgrammingFundamentals
� Implementing Synchronization
� Language-level Constructs
![Page 21: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/21.jpg)
12 - 1
Processes, Threads, Tasks
� Process: Operating system constructthat may execute threads
� Thread: Active entity that theprogrammer thinks of as runningconcurrently with other threads
� Task: Unit of work that must beperformed by some thread
![Page 22: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/22.jpg)
12 - 2
Processes, Threads, Tasks
� Process: Operating system constructthat may execute threads
� Thread: Active entity that theprogrammer thinks of as runningconcurrently with other threads
� Task: Unit of work that must beperformed by some thread
OS level
PLlevel
Logicallevel
![Page 23: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/23.jpg)
12 - 3
Processes, Threads, Tasks
� Process: Operating system constructthat may execute threads
� Thread: Active entity that theprogrammer thinks of as runningconcurrently with other threads
� Task: Unit of work that must beperformed by some thread
� Terminology differs across PLs and systems
� More general than, e.g., Java’s “threads”
![Page 24: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/24.jpg)
13
Communication
� Constructs to pass informationbetween threads� Shared memory: Some variables accessible by
multiple threads
� Message passing: No shared state, but threadssend messages to each other
� Some PLs provide both
![Page 25: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/25.jpg)
14
Synchronization
� Mechanisms to control relative orderof operations in different threads
� Explicit in shared-memory model
� Must synchronize to ensure that variable read
sees newest value stored in the variable
� Implicit in message-passing model
� Sender receives message after it has been
sent
![Page 26: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/26.jpg)
15
Spinning vs. Blocking
� Two forms of synchronization
� Spinning (also: busy-waiting)
� Thread re-evaluates some condition until it
becomes true (because of some other thread)
� Blocking
� Waiting threads stops computation until some
condition becomes true
� Scheduler reactives the thread
![Page 27: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/27.jpg)
16
Examples
Sharedmemory
Messagepassing
Distributedcomputing
Language Java, C#,C/C++
Go Erlang
Extension OpenMP Remote pro-cedure call
Library pthreads,Windowsthreads
MPI Internetlibraries
![Page 28: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/28.jpg)
17
Thread Creation Syntax
� How to create a thread of execution?
� Five answers in popular PLs
� Co-begin
� Parallel loops
� Launch-at-elaboration
� Fork (with optional join)
� Implicit receipt
![Page 29: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/29.jpg)
18
Co-begin
� Compound statement where allstatements are executed concurrently
� Example (pseudo-code):
co-beginstmt_1stmt_2...stmt_n
end
![Page 30: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/30.jpg)
19 - 1
Example: C with OpenMP
#pragma omp sections{# pragma omp section
{ printf("thread 1 here\n"); }
# pragma omp section{ printf("thread 2 here\n"); }
}
![Page 31: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/31.jpg)
19 - 2
Example: C with OpenMP
#pragma omp sections{# pragma omp section
{ printf("thread 1 here\n"); }
# pragma omp section{ printf("thread 2 here\n"); }
}
Pragmas: Compiler directives(# sign must be in first column)
![Page 32: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/32.jpg)
20
Parallel Loops
� Loop whose iterations executeconcurrently instead of sequentially
� Ex. 1: C with OpenMP
� Ex. 2: C# with Task Parallel Library
#pragma omp parallel forfor (int i = 0; i < 3; i++) {printf("thread %d here\n", i);
}
Parallel.For(0, 3, i => {Console.WriteLine("Thread " + i + " here");
});
![Page 33: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/33.jpg)
21
Synchronization in Parallel Loops
� What about data races in parallelloops?
� Most PLs: Developer’s responsibility
� Some PLs: Implicit synchronization� E.g., forall loops in Fortran 95:
Synchronization on every assignment
• All reads on right-hand side are before writes on
the left-hand side
![Page 34: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/34.jpg)
22 - 1
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
![Page 35: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/35.jpg)
22 - 2
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
Reads and writesof array elements
![Page 36: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/36.jpg)
22 - 3
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
Reads and writesof array elements
Assignments: Implicitsynchronization points
![Page 37: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/37.jpg)
22 - 4
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
At first, all threadsread from B and C
![Page 38: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/38.jpg)
22 - 5
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
At first, all threadsread from B and C
Then, all writes toA(i) happen
![Page 39: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/39.jpg)
22 - 6
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
At first, all threadsread from B and C
Then, all writes toA(i) happen
Next, all threads readthe just writtenvalues from A
![Page 40: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/40.jpg)
22 - 7
Example: Fortran 95
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
At first, all threadsread from B and C
Then, all writes toA(i) happen
Next, all threads readthe just writtenvalues from A
Finally, the threadswrite updatedvalues to A(i+1)
![Page 41: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/41.jpg)
23
Quiz: Parallel Loops
What is the value of A after executing the loopwith these initial values:
� A is 0, 0, 0
� B is 1, 2, 3
� C is 3, 2, 3
� n is 3(Note: Arrays indices starts at 1 in Fortran)
forall (i=1:n-1)A(i) = B(i) + C(i)A(i+1) = A(i) + A(i+1)
end forall
https://ilias3.uni-stuttgart.de/vote/0ZT9
![Page 42: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/42.jpg)
1
![Page 43: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/43.jpg)
25
Data Sharing in Parallel Loops
� Some PLs: Can specify whichvariables are shared among threads
� E.g., OpenMP
� Shared data: All threads access same data
� Private data: Each thread has its own copy
� Reduction: Reduce a private variable across all
threads at end of loop
![Page 44: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/44.jpg)
26 - 1
Example: C with OpenMP
double A[N];double sum = 0;
#pragma omp parallel for \default(shared) reduction(+:sum)
for (int i = 0; i < N; i++) {sum += A[i];
}printf("sum: %f\n", sum);
![Page 45: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/45.jpg)
26 - 2
Example: C with OpenMP
double A[N];double sum = 0;
#pragma omp parallel for \default(shared) reduction(+:sum)
for (int i = 0; i < N; i++) {sum += A[i];
}printf("sum: %f\n", sum);
All variables (except for i)are shared by default
![Page 46: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/46.jpg)
26 - 3
Example: C with OpenMP
double A[N];double sum = 0;
#pragma omp parallel for \default(shared) reduction(+:sum)
for (int i = 0; i < N; i++) {sum += A[i];
}printf("sum: %f\n", sum);
All variables (except for i)are shared by default
Exception from default:
� Each thread has privatecopy of sum initializedbefore entering loop
� At end of loop, combineall copies with +
![Page 47: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/47.jpg)
27
Launch-at-Elaboration
� Associate a thread with a specificsubroutine
� Start thread when subroutine getscalled
� At end of subroutine, wait for thread tocomplete
� Thread shares local variables with thesubroutine
![Page 48: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/48.jpg)
28 - 1
Example: Ada
procedure P istask T isPut_Line ("In task T");
end T;beginPut_Line ("In default task of P");
end P;
![Page 49: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/49.jpg)
28 - 2
Example: Ada
procedure P istask T isPut_Line ("In task T");
end T;beginPut_Line ("In default task of P");
end P;
“Task” is Ada’s terminologyfor “thread”
Runs concurrentlywith (implicit) taskof P
![Page 50: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/50.jpg)
29
Fork/Join
� Fork: Explicit creation of a thread
� Join: Wait for a previously forkedthread to terminate
![Page 51: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/51.jpg)
30 - 1
Example: Java
class ImageRenderer extends Thread {ImageRenderer(someArg) { ... }public void run() {// code run by the thread
}}
// ...
ImageRenderer rend = new ImageRenderer(...);rend.start();// ...rend.join();
![Page 52: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/52.jpg)
30 - 2
Example: Java
class ImageRenderer extends Thread {ImageRenderer(someArg) { ... }public void run() {// code run by the thread
}}
// ...
ImageRenderer rend = new ImageRenderer(...);rend.start();// ...rend.join();
Threads: Subclasses of Thread
![Page 53: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/53.jpg)
30 - 3
Example: Java
class ImageRenderer extends Thread {ImageRenderer(someArg) { ... }public void run() {// code run by the thread
}}
// ...
ImageRenderer rend = new ImageRenderer(...);rend.start();// ...rend.join();
Threads: Subclasses of Thread
Share values withthread via arguments
![Page 54: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/54.jpg)
30 - 4
Example: Java
class ImageRenderer extends Thread {ImageRenderer(someArg) { ... }public void run() {// code run by the thread
}}
// ...
ImageRenderer rend = new ImageRenderer(...);rend.start();// ...rend.join();
Threads: Subclasses of Thread
Share values withthread via arguments
Lifetime of thread
![Page 55: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/55.jpg)
31 - 1
Example: C#
class ImageRenderer {public void Render() {// code to be run by the thread
}}
// ...
ImageRenderer rendObj = new ImageRenderer();Thread rend = new Thread(
new ThreadStart(rendObj.Render));rend.Start();// ...rend.Join();
![Page 56: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/56.jpg)
31 - 2
Example: C#
class ImageRenderer {public void Render() {// code to be run by the thread
}}
// ...
ImageRenderer rendObj = new ImageRenderer();Thread rend = new Thread(
new ThreadStart(rendObj.Render));rend.Start();// ...rend.Join();
Just a normal method ...
... turned into a thread
![Page 57: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/57.jpg)
32
Thread Pools
� Goal: Separate tasks to execute fromhow they are executed in threads
� Thread pool: Set of (idle) threads thatcan execute tasks
� Reduces cost of creating and starting threads
by reusing them
� Let pool implementation decide how exactly to
schedule tasks for execution
![Page 58: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/58.jpg)
33 - 1
Example: Java
class ImageRenderer implements Runnable {ImageRenderer(someArg) { ... }public void run() {// code run by this task
}}
// ...
Executor pool = Executors.newFixedThreadPool(4);pool.execute(new ImageRenderer(...));
![Page 59: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/59.jpg)
33 - 2
Example: Java
class ImageRenderer implements Runnable {ImageRenderer(someArg) { ... }public void run() {// code run by this task
}}
// ...
Executor pool = Executors.newFixedThreadPool(4);pool.execute(new ImageRenderer(...));
Not a Thread anymore,but only a Runnable
![Page 60: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/60.jpg)
33 - 3
Example: Java
class ImageRenderer implements Runnable {ImageRenderer(someArg) { ... }public void run() {// code run by this task
}}
// ...
Executor pool = Executors.newFixedThreadPool(4);pool.execute(new ImageRenderer(...));
Not a Thread anymore,but only a Runnable
Pool with 4 threads used toexecute any number of threads
![Page 61: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/61.jpg)
34
Spawn and Sync in Cilk
� Extension of C language
� Programmer expresses tasks and theirdependencies
� spawn calls a function to be executed as a
logically concurrent task
� sync joins all tasks spawned by the calling task
� Scheduler assigns tasks to processorcores through work stealing
![Page 62: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/62.jpg)
35 - 1
Example: Fibonacci
int fib (int n) {
if (n < 2) return 1;
else {
int res = 0;
res += fib (n - 1);
res += fib (n - 2);
return res;
}
}
Sequential implementation:
![Page 63: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/63.jpg)
35 - 2
Example: Fibonacci
cilk int fib (int n) {
if (n < 2) return 1;
else {
int res = 0;
res += spawn fib (n - 1);
res += spawn fib (n - 2);
sync;
return res;
}
}
Parallel implementation with Cilk:
![Page 64: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/64.jpg)
35 - 3
Example: Fibonacci
cilk int fib (int n) {
if (n < 2) return 1;
else {
int res = 0;
res += spawn fib (n - 1);
res += spawn fib (n - 2);
sync;
return res;
}
}
Parallel implementation with Cilk:
Execute in parallelwith parent
Wait until childrenhave returned
![Page 65: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/65.jpg)
36
Implicit Receipt
� In remote procedure call (RPC)-basedsystems
� Create thread in response to anincoming request from some otheraddress space
� E.g., from another machine
![Page 66: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/66.jpg)
37 - 1
Quiz: Concurrency
Which of the following sentences are true?
� Concurrency means different machines performcomputations at the same time.
� In OpenMP’s parallel loops, all data is private tothe respective thread.
� A thread in a thread pool may execute anynumber of tasks.
� The scheduler re-actives a busy-waiting thread.
https://ilias3.uni-stuttgart.de/vote/0ZT9
![Page 67: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/67.jpg)
37 - 2
Quiz: Concurrency
Which of the following sentences are true?
� Concurrency means different machines performcomputations at the same time.
� In OpenMP’s parallel loops, all data is private tothe respective thread.
� A thread in a thread pool may execute anynumber of tasks.
� The scheduler re-actives a busy-waiting thread.
https://ilias3.uni-stuttgart.de/vote/0ZT9
![Page 68: Concurrency - software-lab.org€¦ · Concurrency (Part 1) 2 - 1 Wake-up Exercise What does this Java code print? classWarmup {staticbooleanflag =false; staticvoidraiseFlag()](https://reader033.vdocument.in/reader033/viewer/2022050418/5f8e548d7e0132625033efaa/html5/thumbnails/68.jpg)
38
Overview
� Introduction
� Concurrent ProgrammingFundamentals
� Implementing Synchronization
� Language-level Constructs
4