cs 380 - gpu and gpgpu programming lecture 24: additional ... · cs 380 - gpu and gpgpu programming...

94
CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST

Upload: others

Post on 18-Aug-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST

CS 380 - GPU and GPGPU ProgrammingLecture 24: Additional Stuff, Part 1

Markus Hadwiger, KAUST

Page 2: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST

2

Reading Assignment #14 (until May 11)

Read (required):

• Programming Massively Parallel Processors book,Chapter 10 (Sparse Matrix-Vector Multiplication)

• Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors, Nathan Bell and Michael Garlandhttp://www.nvidia.com/docs/IO/77944/sc09-spmv-throughput.pdf

Read (optional):

• CUSPARSE library description in the CUDA SDK

• CUSP library: http://cusplibrary.github.io/

• Incomplete-LU and Cholesky Preconditioned Iterative Methods Using CUSPARSE and CUBLAS, Maxim Naumovhttp://developer.nvidia.com/sites/default/files/akamai/cuda/files/

psts_white_paper_final.pdfthis is also included in the CUDA SDK!

Page 3: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST

Reading Assignment ++

“Modern GPU” library and links to other libraries at:• https://nvlabs.github.io/moderngpu/intro.html

About occupancy and latency• https://nvlabs.github.io/moderngpu/performance.html

Latency data obtained via micro benchmarking• http://lpgpu.org/wp/wp-content/uploads/2013/05/poster_andresch_acaces2014.pdf

Warp-aggregate atomics• http://devblogs.nvidia.com/parallelforall/cuda-pro-tip-optimized-filtering-warp-

aggregated-atomics/

Fast Histograms Using Shared Atomics on Maxwell• http://devblogs.nvidia.com/parallelforall/gpu-pro-tip-fast-histograms-using-shared-

atomics-maxwell/

CUDA 7 and C++11• http://devblogs.nvidia.com/parallelforall/power-cpp11-cuda-7/

PTX virtual assembly and SASS machine assembly• PTX in CUDA SDK: ptx_isa_4.2.pdf SASS in CUDA SDK: CUDA_Binary_Utilities.pdf

Markus Hadwiger, KAUST 3

Page 4: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 5: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 6: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 7: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 8: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 9: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 10: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 11: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 12: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 13: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 14: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 15: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 16: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 17: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 18: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 19: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 20: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 21: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 22: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 23: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 24: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 25: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 26: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 27: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 28: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 29: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 30: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 31: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 32: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 33: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 34: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 35: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 36: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 37: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 38: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 39: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 40: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 41: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 42: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 43: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 44: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 45: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 46: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 47: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 48: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 49: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 50: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 51: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 52: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 53: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 54: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 55: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 56: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 57: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 58: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 59: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 60: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 61: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 62: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 63: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 64: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 65: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 66: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 67: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 68: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 69: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 70: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 71: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 72: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 73: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 74: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 75: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 76: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 77: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 78: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 79: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 80: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 81: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 82: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 83: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 84: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 85: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 86: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 87: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 88: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 89: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 90: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 91: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 92: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 93: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST
Page 94: CS 380 - GPU and GPGPU Programming Lecture 24: Additional ... · CS 380 - GPU and GPGPU Programming Lecture 24: Additional Stuff, Part 1 Markus Hadwiger, KAUST