Thread Contracts for Safe Parallelism

Rajesh Karmani, P Madhusudan, Brandon MooreUniversity of Illinois at Urbana-Champaign

PPoPP 2011Thread Contracts for Safe ParallelismNSF

Whats the problem?Data-races in parallel programs is just plain wrong.For example, C++ gives no semantics for programs with races!

But no mechanism currently for building large data-race-free programs, without significantly changing the programming style

Proposal of this paper:An annotation mechanism for data-parallel programs that helps building large data-race-free programs Annotation augments existing codeAnnotation Race-freedom (automatically checked)Program Annotation (tested)

#Data-race and language semantics simultaneous accesses to a memory location by two different threads, where one of them is a write.

Memory models for programming languagesSpecifies what exactly will be ensured for a read instruction in the programJava memory model [Manson et al., POPL 05]A complex , buggy model for programs with races C++ (new version) [Boehm, Adve, PLDI 08]No semantics for programs with races!

Consensus Data-Race-Free (DRF) Guarantee: Memory model assures that Data-race-free programs sequentially consistent#ACCORD (Annotations for Concurrent Co-ORDination)Light-weight annotation that help develop parallel softwareFormally express the coordination strategy that the programmer has in mindCheck the strategy (for data-race-freedom) and its implementation

Annotations have been successful in avoiding memory errors in sequential programsEiffel, JML, Microsoft Code Contracts, Cofoja

NOT a type system to express memory regions or program synchronization

#4In this workAnnotations for data race freedom in

data-parallel programs that access arrays, vectors, matrices

with fork-join synchronization and locks

Typical for OpenMP applications

Annotations express the sharing strategy

#foreach loop: semantically creates p threads; one for each iteration of the loop implicit barrier at end of foreach loop void mm (int [m,n] A, int[n,p] B) { for (int i:=0; i < m; i := i + 1) for (int j:=0; j < n; j := j + 1) foreach (int k:=0; k < p; k := k+1) { C[i,k] := C[i,k] + (A[i,j] * B[j,k]) ; }}

Illustration: Parallel Matrix Multiplication#Illustration: Parallel Matrix MultiplicationAnnotation specifies the read and write sets for kth thread, for each k. Sets can be an over-approximation.Annotation utilizes current values of the program variables (i,j) and thread-id (k)void mm (int [m,n] A, int[n,p] B) { for (int i:=0; i < m; i := i + 1) for (int j:=0; j < n; j := j + 1) foreach (int k:=0; k < p; k := k+1) reads A[i,j], B[j,k], C[i,k] writes C[i,k] { C[i,k] := C[i,k] + (A[i,j] * B[j,k]) ; }}

ACCORDannotation#Accord annotations in RedFully Parallel Matrix MultiplicationAux variables ($x,$y) are quantified over specified ranges where clause: arithmetic and logical constraints void mm ( int [m,n] A, int[n,p] B ) { foreach (int i:=0; i < m; i := i + 1) reads A[i,$x], B[$x, $y], C[i, $y] writes C[i, $y] where 0