[harvard cs264] 14 - dynamic compilation for massively parallel processors (gregory diamos, georgia...

Dynamic Compilation for Massively Parallel

Processors

Gregory Diamos

PhD candidate

Georgia Institute of Technology and NVIDIA Research

April 14, 2011

Gregory Diamos CS264 - Dynamic Compilation 1/62

What is an execution model?

Goals of programming languages

Programming languages are designed for productivity.

Efficiency is measured in terms of:1 cost - hardware investment, power consumption, area requirement2 complexity - application development effort3 speed - amount of work performed per unit time

Goals of processor architecture

Hardware is designed for speed and efficiency.

Goals of processor architecture - 2

[1] - M. Koyanagi, T. Fukushima, and T. Tanaka. "High-Density Through Silicon Vias for 3-D LSIs" [2] - Novoselov et al. "Electric Field Effect in Atomically Thin Carbon Films." [3] - Intel Corp. 22nm test chip.

It is constrained by the limitations of physical devices.

Execution models bridge the gap

Goals of execution models

Execution models provide impedance matching between applications andhardware.

Goals:

leverage common optimizations across multiple applications.

limit the impact of hardware changes on software.

ISAs have traditionally been effective execution models.

Programming challenges of heterogeneity

The introduction of heterogeneous and multi-core processors changes thehardware/software interface:

Intel Nehalem IBM PowerEN AMD Fusion NVIDIA Fermi

1 multi-core creates multiple interfaces.2 heterogeneity creates different interfaces.3 these increase software complexity.

Program the entire processor, not individual cores.(new execution model abstractions are needed)

Emerging execution models

Bulk-synchronous parallel (BSP)

[1] - Leslie Valiant. A bridging model for parallel computing.

The Parallel Thread eXecution (PTX) Model

PTX defines a kernel as a 2-level grid of bulk-synchronous tasks.

Dynamically translating PTX

Dynamic compilers can transform this parallelism to fit the hardware.

Beyond PTX - Data distributions

Beyond PTX - Memory hierarchies

[1] - Leslie Valiant. A bridging model for multi-core.[2] Fatahalian et al. Sequoia: Programming the memory hierarchy.

Dynamic compilation/binarytranslation

Binary translation

Binary translators are everywhere

If you are running a browser, you are using dynamic compilation.

x86 binary translation

Low Level Virtual Machines

Compile all programs to a common virtual machine representation (LLVMIR), keep this around.

Perform common optimizations on this IR.

Target various machines by lowering it to an ISA.

Statically or via JIT compilation.

Execution model translation

Extend binary translation to execution model translation.

Dynamic compilers can map threads/tasks to the HW.

Different core architectures

Can we target these from the same execution model.

What about efficiency?

Ocelot

Enables thread-aware compiler transformations.

Mapping CTAs to cores - thread fusion

Scheduler Block

Restore Registers

Spill Registers

Barrier

Original PTX Code Transformed PTX Code

Transform threads into loops over the program.

Distribute loops to handle barriers.

Mapping CTAs to cores - vectorization

Pack adjacent threads into vector instructions.

Speculate that divergence never occurs, check in case it does.

Mapping CTAs to cores - multiple instruction streams

T0 T1 T2 T3

Instructions from different threads are independent.

merge instruction streams and statically schedule on functional units.

PTX analysis

Divergence analysis

Subkernels

subkernel

Thread frontier analysis

Supporting control flow on SIMD processors requires finding divergentbranches and potential re-converge points.

if((cond1() || cond2()) && (cond3() || cond4())){ ...}

bra cond1() bra cond3()

bra cond2() bra cond4()....

compound conditionals

short circuit control flow

bra cond1()

bra cond3()

bra cond2()

bra cond4()

Thread FrontiersBlock Id

{B2 - B3}

{B3 - Exit}

{B4 - Exit}

{B5 - Exit}

T0 T1 T2 T3 T0 T1 T2 T3

thread-frontier reconvergence

post dominatorreconvergenceof T1 and T3

Push B3 on T0

Push Exit on T1

Push B5 on T2

re-convergence at thread frontiers

immediate post-dominator re-convergence

Push Exit on T4

Pop stack Exit on T4

Pop stack switch to B5 on T2

Pop stack switch to B3 on T0

post dominatorreconvergence

of T1, T2, and T3

Pop stack Exit on T1

Compiler analysis can identify immediate post donimators orthread-frontiers as re-convergence points.

Consequences of architecturedifferences

Degraded performance portability

0 1000 2000 3000 4000 5000 6000

Fermi SGEMM

AMD SGEMM

0 1000 2000 3000 4000 5000 6000

Fermi SGEMM

AMD SGEMM

Performance of two OpenCL applications, one tuned for AMD, the otherfor NVIDIA.

Memory traversal patterns

Warp(4) Cycle 1 Warp(4) Cycle 2

Warp(1) Cycle 1 Warp(1) Cycle 2

Thread loops change row major memory accesses to column majoraccesses.

Reduced memory bandwidth on CPUs

Optimized for SIMD (GPU)

Optimized for single-threaded CPU

This reduces memory bandwidth by 10x for a memory microbenchmarkrunning on a 4-core CPU.

The good news

Scaling across three decades of processors

Many existing applications still scale.

280GTX has 40x more peak flops than a Phenom, 480x more than anAtom.

Questions?

Databases on GPUs

Who cares about databases?

What do applications look like?

Gobs of data

Distributed systems

Lots of parallelism

What do CPU algorithms look like?

What do cpu algorithms looklike?

Btrees

Sequential algorithms

relation 1

relation 2

result

It doesn’t look good

Outlook not so good...

Or does it?

Where is the parallelism?

Flattened trees

Relational algebra

Inner Join

A Case Study: Inner Join

1. Recursive partitioning

2. Block streaming

Blocking into pages, shared memory buffers, and transaction sizedchunks makes memory accesses efficient.

3. Shared memory merging network

A network for join can be constructed, similar to a sorting network.

4. Data chunking

Stream compaction packs result data into chunks that can be streamedout of shared memory efficiently.

Operator fusion

Will it blend?

Yes it blends.

Operator NVIDIA C2050 Phenom 9570

inner-join 26.4-32.3 GB/s 0.11-0.63 GB/sselect 104.2 GB/s 2.55 GB/sset operators 45.8 GB/s 0.72 GB/sprojection 54.3 GB/s 2.34 GB/scross product 98.8 GB/s 2.67 GB/s

Questions?

Conclusions

Emerging heterogeneous architectures need matching execution modelabstractions.

dynamic compilation can enable portability.

When writing massively parallel codes, consider:

data structures and algorithms.

mapping onto the execution model.

transformations in the compiler/runtime.

processor micro-architecture.

Thoughts on open source software

Questions?

Contact Me:

gregory.diamos@gatech.edu

Contribute to Harmony, Ocelot, and Vanaheimr:

http://code.google.com/p/harmonyruntime/

http://code.google.com/p/gpuocelot/

http://code.google.com/p/vanaheimr/

[harvard cs264] 14 - dynamic compilation for massively parallel processors (gregory diamos, georgia...

Education

massively parallel logic simulation

[harvard cs264] 11a - programming the memory hierarchy with...

[harvard cs264] 02 - parallel thinking, architecture, theory...

cisco massively scalable data center

massively maintained accessibility: wordpress

massively parellel signature sequencing

massively distributed database systems

[harvard cs264] 03 - introduction to gpu computing, cuda...

the massively multiplayer gaming...

massively multiplayer journalism

massively broadband wireless communications and...

mimblewimble: private, massively-prunable...

[harvard cs264] 01 - introduction

mpc for mpc: secure computation on a massively parallel...

[harvard cs264] 07 - gpu cluster programming (mpi & zeromq)

groups massively unemployed/underemployed

massively parallel computing

massively sharded mysql - o'reilly...

massively parallel...

persistent rnns - (stashing recurrent weights...