gpu programming on cpu - using c++amp
Post on 29-Jun-2015
177 Views
Preview:
TRANSCRIPT
GPU Programming
on CPUs
Using C++AMP
Miller Lee
Outline
1. Introduction to C++AMP2. Introduction to Tiling3. tile_static4. barrier.wait and solutions
a. C++11 threadb. setjmp/longjmpc. ucontext
2
(Homogeneous coordinates)
(0, 0) (0, 1) (0, 2) (0, 3)
(1, 0) (1, 1) (1, 2) (1, 3)
(2, 0) (2, 1) (2, 2) (2, 3)
(3, 0) (3, 1) (3, 2) (3, 3)
X
0
1
2
3
Matrix A b
=
0
1
2
3
result
Computing example
● Simple matrix multiplication
3
C++ Version
1. int A[4][4];2. int b[4];3. int result[4];4. for (int i = 0; i < 4; i++) {5. result[i] = 0;6. for (int j = 0; j < 4; j++)7. result[i] += A[i][j] * b[j];8. }
4
C++AMP Version1. array_view<float, 2> A(4, 4);2. array_view<float, 1> b(4);3. array_view<float, 1> result(4);4. extent<1> ext(4);5. parallel_for_each(ext, [&](index<1> idx) restrict(amp)6. {7. result[idx[0]] = 0;8. for (int i = 0; i < 4; i++)9. result[idx[0]] += A(idx[0], i) * b(i);
10. });
5
memory access
0 1 2 3
P0 P1 P2 P3
global memory
b
100t
Total access time = 400t 6
shared memory
0 1 2 3
shared memory
10t
100t
Total access time = 130t
b
7
1. array_view<float, 2> A(4, 4);2. array_view<float, 1> b(4);3. array_view<float, 1> result(4);4. extent<1> ext(4);5. parallel_for_each(ext.tile<4>(), [&](tiled_index<4> tidx)
restrict(amp)6. {7. int local = tidx.local[0];8. int global = tidx.global[0];9. tile_statc int buf[4];
10. buf[local] = b[global];11. tidx.barrier.wait();12. result[idx[0]] = 0;13. for (int i = 0; i < 4; i++)14. result[idx[0]] += A[idx[0]][i] * buf[i];15. }); 8
barrier
9
Architecture
source: NVIDIA TESLA:AUNIFIED GRAPHICS AND COMPUTING ARCHITECTURE
shared memoryaccessible to all SPs
10
Goal
● Implement all the C++AMP function on CPU instead of GPU without any compiler modification.
11
tiled_static
● The limitation of C++ syntax leads to the following choices○ const, volatile○ __attribute__(...)○ static
● Choose static○ static memory can be shared among all the threads○ side effect: At most one thread group can be
executed at the same time.
#define tile_static static
12
Barrier.wait
● Threads in the same thread group will be waited at the point where “wait” is called.
● Program cana. perform real barrier actionb. jump out of current execution context
13
● True threading○ C++11 thread
● Fake threading(Coroutines)○ setjmp/longjmp○ makecontext/getcontext/swapcontext/setcontext
Approaches
14
C++11 thread
● launch hundreds of threads at a time.● implemente my own barrier by using C++11
mutex library.→ extremely slow.→ The data on static memory will be corrupted
15
setjmp/longjmp
● int setjmp(jmp_buf env)○ setjmp() saves the stack context/environment in env
for later use by longjmp.○ The stack context will be invalidated if the function
which called setjmp() returns.● void longjmp(jmp_buf env, int val);
○ longjmp() restores the environment saved by the last call of setjmp.
16
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf;4. void wait(void) {5. printf("wait\n"); // prints6. longjmp(buf,1); 7. }8. void first(void) {9. wait();
10. printf("first\n"); // does not print11. }12. int main() { 13. if (!setjmp(buf))14. first(); // when executed, setjmp returns 015. else // when longjmp jumps back, setjmp returns 116. printf("main\n"); // prints17. return 0;18. }
17
Pseudo code (1)void entry(){while(!finish) for(t : tasks) run(t)}
void fun(){… wait();...}
void fun(){… wait();...}
void entry(){while(!finish) for(t : tasks) run(t)}
void fun(){… wait();...}
void fun(){… wait();...}
18
Pseudo code (2)void entry(){while(!finish) for(t : tasks) run(t)}
void fun(){… wait();...}
void fun(){… wait();...}
void entry(){while(!finish) for(t : tasks) run(t)}
void fun(){… wait();...}
void fun(){… wait();...}
19
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf, b;4. void wait(void) {5. printf("wait\n");6. if (setjmp(b) == 0)7. longjmp(buf,1);8. }9. void first(void) {
10. wait();11. }12. int main() { 13. if (!setjmp(buf) )14. first();15. else {16. printf("main\n");17. longjmp(b, 10);18. }19. return 0;20. } 20
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf, b;4. void wait(void) {5. printf("wait\n");6. if (setjmp(b) == 0)7. longjmp(buf,1);8. }9. void first(void) {
10. wait();11. }12. int main() { 13. if (!setjmp(buf) )14. first();15. else {16. printf("main\n");17. longjmp(b, 10);18. }19. return 0;20. }
buf
21
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf, b;4. void wait(void) {5. printf("wait\n");6. if (setjmp(b) == 0)7. longjmp(buf,1);8. }9. void first(void) {
10. wait();11. }12. int main() { 13. if (!setjmp(buf) )14. first();15. else {16. printf("main\n");17. longjmp(b, 10);18. }19. return 0;20. }
ret address
buf
b
22
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf, b;4. void wait(void) {5. printf("wait\n");6. if (setjmp(b) == 0)7. longjmp(buf,1);8. }9. void first(void) {
10. wait();11. }12. int main() { 13. if (!setjmp(buf) )14. first();15. else {16. printf("main\n");17. longjmp(b, 10);18. }19. return 0;20. }
buf
b
23
1. #include <stdio.h>2. #include <setjmp.h>3. jmp_buf buf, b;4. void wait(void) {5. printf("wait\n");6. if (setjmp(b) == 0)7. longjmp(buf,1);8. }9. void first(void) {
10. wait();11. }12. int main() { 13. if (!setjmp(buf) )14. first();15. else {16. printf("main\n");17. longjmp(b, 10);18. }19. return 0;20. }
Cannot return？？？
？？？
？？？ buf
b
24
Problems
● Cannot return○ return address in the stack is destroyed
● Cannot use too many static variables○ will lost spilled registers
→ can be solved by using “alloca”http://www.codemud.net/~thinker/GinGin_CGI.py/show_id_doc/489
25
ucontext.h
● ucontext_t● getcontext● makecontest● swapcontext● setcontext
26
ucontext_ttypedef struct ucontext { struct ucontext *uc_link; sigset_t uc_sigmask; stack_t uc_stack; mcontext_t uc_mcontext; ...} ucontext_t;
● uc_link○ points to the context that will be resumed when the current context
terminates● uc_stack
○ the stack used by this context ● uc_mcontext
○ machine-specific representation of the saved context, that includes the calling thread's machine registers
27
Functions
● int getcontext(ucontext_t *ucp);○ initializes the structure pointed at by ucp.
● int setcontext(const ucontext_t *ucp);○ restores the user context pointed at by ucp
● int swapcontext(ucontext_t *oucp, const ucontext_t *ucp);○ saves the current context in the structure pointed to
by oucp, and then activates the context pointed to by ucp.
28
makecontext
● void makecontext(ucontext_t *ucp, void (*func)(), int argc, ...);○ glibc(x86_64) saves the arguments to registers
instead of pushing them on stack as AMD64 ABI said
○ The size of the arguments that passed to makecontext should be no less than sizeof(register)
29
1. #include <stdio.h>
2. #include <ucontext.h>
3. static ucontext_t ctx[2];
4. static void f1 (void) {
5. puts("start f1");
6. swapcontext(&ctx[1], &ctx[0]);
7. puts("finish f1");
8. }
9. int main (void)
10. {
11. char st1[8192];
12. getcontext(&ctx[1]);
13. ctx[1].uc_stack.ss_sp = st1;
14. ctx[1].uc_stack.ss_size = sizeof st1;
15. ctx[1].uc_link = &ctx[0];
16. makecontext(&ctx[1], f1, 0);
17. swapcontext(&ctx[0], &ctx[1]);
18. swapcontext(&ctx[0], &ctx[1]);
19. return 0;
20. } 30
1. #include <stdio.h>
2. #include <ucontext.h>
3. static ucontext_t ctx[3];
4. static void f1 (void) {
5. puts("start f1");
6. swapcontext(&ctx[1], &ctx
[0]);
7. puts("finish f1");
8. }
9. static void f2 (void)
10. {
11. puts("start f2");
12. swapcontext(&ctx[2], &ctx
[1]);
13. puts("finish f2");
14. }
1. int main (void)
2. {
3. char st1[8192], st2[8192];
4. getcontext(&ctx[1]);
5. ctx[1].uc_stack.ss_sp = st1;
6. ctx[1].uc_stack.ss_size = sizeof
st1;
7. ctx[1].uc_link = &ctx[0];
8. makecontext(&ctx[1], f1, 0);
9.
10. getcontext(&ctx[2]);
11. ctx[2].uc_stack.ss_sp = st2;
12. ctx[2].uc_stack.ss_size = sizeof
st2;
13. ctx[2].uc_link = &ctx[1];
14. makecontext(&ctx[2], f2, 0);
15. swapcontext(&ctx[0], &ctx[2]);
16. swapcontext(&ctx[0], &ctx[2]);
17. return 0;
18. }
31
Fake threading (yield)void entry(){ setup(fun, 2);while(!finish) switch_to();}
void fun(){… wait();...}
void fun(){… wait();...}
32
void entry(){ setup(fun, 2);while(!finish) switch_to();}
void fun(){… wait();...}
void fun(){… wait();...}
Problems
1. How to pass a lambda?○ makecontext(&ctx,
(void (*)(void))&Kernel::operator(), …);2. How to pass non-int arguments?
○ What if sizeof(Type) > sizeof(int)○ How about complex structure and class
33
Pass lambda
1. Use a wrapper function!!template <typename Ker, typename Arg>
void fun(Ker k, Arg arg)
{
k(arg);
}
template <typename Ker, typename Arg>
void makectx(Ker k, Arg arg)
{
makecontext(&ctx, (void (*)(void))fun<ker, Arg>, 2, k, arg);
}
34
Pass non-int arguments
2. Pass pointer instead!!template <typename Ker, typename Arg>
void fun(Ker *k, Arg *arg)
{
(*k)(*arg);
}
template <typename Ker, typename Arg>
void makectx(Ker k, Arg arg)
{
makecontext(&ctx, (void (*)(void))fun<ker, Arg>, 2, &k, &arg);
}
35
Additional
● Use a counter so that we can spawn coroutines dynamically
● Can it be multithreaded? Yes
36
true threading
barrier
There are 12 threads in one thread group
37
one thread
barrier
38
multithreading
barrier
Hardware Core = 4
39
barrierstruct bar_t { unsigned const count; std::atomic<unsigned> spaces; std::atomic<unsigned> generation; bar_t(unsigned count_) : count(count_), spaces(count_), generation(0) {} void wait() noexcept { unsigned const my_generation = generation; if (!--spaces) { spaces = count; ++generation; } else { while(generation == my_generation); } }}; source: C++ Concurrency in Action: Practical Multithreading 40
Summary
● It works fine on AMP right now● The importance of low level knowledge
41
42
top related