issues in translation of high performance fortran
Post on 02-Jan-2016
27 Views
Preview:
DESCRIPTION
TRANSCRIPT
Issues in Translation of High Performance Fortran
Bryan Carpenter
NPAC at Syracuse UniversitySyracuse, NY 13244dbc@npac.syr.edu
Goals of this lecture Discuss translation of some
elementary HPF examples to MPI code. Illustrate the need for a Distributed Array Descriptor (DAD).
Develop an abstract model of a DAD, and show how it can be used to translate simple codes.
Contents of Lecture Introduction.
Translation of simple HPF fragment to SPMD. The problem of procedures.
Requirements for an array descriptor. Groups.
Process grids. Restricted groups.
Range objects. A DAD
A simple HPF program Here is a simple HPF program:
!HPF$ PROCESSORS P(4)
REAL A(50) !HPF$ DISTRIBUTE A(BLOCK) ONTO P
FORALL (I = 1:50) A(I) = 1.0 * I
We want to translate this to an MPI program.
Translation of simple program
INTEGER W_RANK, W_SIZE, ERRCODE
INTEGER BLK_SIZEPARAMETER (BLK_SIZE = (50 + 3)/4)
REAL A(BLK_SIZE)
INTEGER BLK_START, BLK_COUNTINTEGER L, I
CALL MPI_INIT(ERRCODE)
CALL MPI_COMM_RANK(MPI_COMM_WORLD, W_RANK, ERRCODE)CALL MPI_COMM_SIZE(MPI_COMM_WORLD, W_SIZE, ERRCODE)
IF (W_RANK < 4) THEN BLK_START = W_RANK * BLK_SIZE IF (50 – BLK_START >= BLK_SIZE) THEN BLK_COUNT = BLK_SIZE ELSEIF (50 – BLK_START > 0) THEN BLK_COUNT = 50 – BLK_START ELSE BLK_COUNT = 0 ENDIF
DO L = 1, BLK_COUNT I = BLK_START + L A(L) = 1.0 * I ENDDOENDIF
CALL MPI_FINALIZE(ERRCODE)
Setting up the environment
Associated code:
INTEGER W_RANK, W_SIZE, ERRCODE. . .
CALL MPI_INIT(ERRCODE)
CALL MPI_COMM_SIZE(MPI_COMM_WORLD, W_RANK, ERRCODE)CALL MPI_COMM_RANK(MPI_COMM_WORLD, W_RANK, ERRCODE). . .
CALL MPI_FINALIZE(ERRCODE)
Allocating segment of the distributed array Associated statements are:
INTEGER BLK_SIZE PARAMETER (BLK_SIZE = (50 + 3)/4)
REAL A(BLK_SIZE)
Segment size is 50/4
Testing this processor holds a segment Associated code is:
IF (W_RANK < 4) THEN . . . ENDIF
Assumes number of MPI processes is at least the size of the largest processor arrangement of HPF program.
Computing parameters of locally held segment
Associated code: INTEGER BLK_START, BLK_COUNT . . . BLK_START = W_RANK * BLK_SIZE IF (50 – BLK_START >= BLK_SIZE) THEN BLK_COUNT = BLK_SIZE ELSEIF (50 – BLK_START > 0) THEN BLK_COUNT = 50 – BLK_START ELSE BLK_COUNT = 0 ENDIF
BLK_START—position in global index space. BLK_COUNT—elements in segment.
Loop over local elements Associated code:
INTEGER L, I . . .
DO L = 1, BLK_COUNT I = BLK_START + L A(L) = 1.0 * I ENDDO
An HPF procedure Superficially similar program:
SUBROUTINE INIT(D)
REAL D(50) !HPF$ INHERIT D
FORALL (I = 1:50) D(I) = 1.0 * I END
INHERIT directive means mapping of dummy should be same as actual, whatever that is.
Procedure call with block-distributed actual
!HPF$ PROCESSORS P(4)
REAL A(50)!HPF$ DISTRIBUTE A(BLOCK) ONTO P
CALL INIT(A)
Mapping of D:
Procedure call with cyclically distributed actual
!HPF$ PROCESSORS P(4)
REAL A(50)!HPF$ DISTRIBUTE A(CYCLIC) ONTO P
CALL INIT(A)
Mapping of D:
Procedure call with strided alignment of actual
!HPF$ PROCESSORS P(4)
REAL A(100)!HPF$ DISTRIBUTE A(BLOCK) ONTO P
CALL INIT(A(1:100:2))
Mapping of D:
Procedure call with row-aligned actual
!HPF$ PROCESSORS Q(2, 2)
REAL A(6, 50)!HPF$ DISTRIBUTE A(BLOCK, BLOCK) ONTO Q
CALL INIT(A(2, :)) Mapping of D:
The problem Somehow INIT must be translated to deal
with data having any of these decompositions, or any legal HPF mapping. Actual mapping not known until run-time.
Not an artificial example. Libraries that operate on distributed arrays (eg the communication libraries discussed later) must deal with exactly this situation.
Requirements for an array descriptor Seems that to translate procedure
calls, need some non-trivial data structure to describe layout of actual argument.
The Distributed Array Descriptor (DAD). Want to understand requirements and
best organization of a DAD. Adopt object-oriented principles to
build an abstract design.
Distributed array dimensions Obvious structural feature of HPF array:
multidimensional. Each dimension mapped independently as:
Collapsed (serial), Simple block distribution, Simple cyclic distribution, Block cyclic distribution, General block distribution (HPF 2.0), Linear alignment to any of above.
Converting block distribution to cyclic distribution
BLK_SIZE = (N + NP – 1) / NP. . .
BLK_START = R * BLK_SIZE. . .
IF (N – BLK_START >= BLK_SIZE) THEN
BLK_COUNT = BLK_SIZEELSEIF (N – BLK_START > 0) THEN BLK_COUNT = N – BLK_STARTELSE BLK_COUNT = 0ENDIF. . .
I = BLK_START + L
BLK_SIZE = (N + NP – 1) / NP. . .
BLK_START = R. . .
BLK_COUNT = (N – R + NP – 1) / NP
. . .
I = BLK_START + NP * (L - 1) + 1
Distributed ranges Have different kinds of array dimension
(distribution format). Each kind of dimension has a different
set of formulae for segment layout, index computation, etc.
OO interpretation: virtual functions on a class hierarchy.
Implement as the Range hierarchy. DAD for rank-r array will contain r Range
objects, one per dimension.
Dealing with “hidden” dimensions of sections Array may be mapped to slice of grid:
Rank-1 section only has one range object. Need some other structure to represent embedding in subgrid.
DAD groups Need a group concept similar to
MPI_Group. Want lightweight structure for
representing arbitrary slices of process grids.
Object representing grid itself needs multidimensional structure (cf Cartesian Communicator in MPI).
Representing processor arrangements In OO runtime descriptor, expect entity
like processor arrangement becomes an object.
Use C++ for definiteness: !HPF$ PROCESSORS P(4)
becomes Procs1 p(4);
and !HPF$ PROCESSORS Q(2, 2)
becomes Procs2 q(2, 2);
Hierarchy of process grids
Interface of Procs and Dimension
class Procs {public: int member() const;
Dimension dim(const int d) const; . . .};
class Dimension {public: int size() const;
int crd() const; . . .};
Using Procs in translation
INTEGER W_RANK, . . .
CALL MPI_COMM_RANK(MPI_COMM_WORLD, W_RANK, ERRCODE)
. . .
IF (W_RANK < 4) THEN BLK_START = W_RANK * BLK_SIZE . . .ENDIF
Procs1 p(4);. . .
if (p.member()) { blk_start = p.dim(0).crd() *
blk_size; . . .}
Becomes:
Restricted process groups Slice of process grid to which array
section may be mapped. Portion of grid selected by
specifying subset of dimension coordinates.
Lightweight representation. Use bitmask to represent dimension set.
Example restricted groups in 2-dimensional grid
Representation of subgrids
example dimension lead tuple set process
a) {dim(0), dim(1)} 0 (p, 11 , 0) 2
b) {dim(0)} 8 (p, 10 , 8) 2
c) {dim(1)} 1 (p, 01 , 1) 2
d) {} 6 (p, 00 , 6) 2
The Group classclass Group {public: Group(const Procs& p);
void restrict(Dimension d, const int coord);
int member() const; . . .}
Lightweight—implementation in about 3 words. Can freely copy and discard. DAD contains a Group object.
Ranges In DAD, range object describes
extent and distribution format of one array dimension.
Expect a class hierarchy of ranges. Each subclass corresponds to a
different kind of distribution format for an array dimension.
A hierarchy of ranges
Interface of the Range class
Class range {public: int size() const;
Dimension dim() const;
int volume() const;
Range subrng(const int extent, const int base, const int stride = 1) const;
void block(Block* blk, const int crd) const;
void location(Location* loc, const int glb) const; . . .};
Translating simple HPF program to C++
Source:
!HPF$ PROCESSORS P(4)
REAL A(50)!HPF$ DISTRIBUTE A(BLOCK) ONTO
P
FORALL (I = 1:50) A(I) = 1.0 * I
Translation:
Procs1 p(4);
BlockRange x(50, p.dim(0));
float* a = new float [x.volume()];
if (p.member()) { Block b; x.block(&b, p.dim(0).crd());
for (int l = 0; l < b.count; l++) { const int i = b.glb_bas + b.glb_stp * l
+ 1; a [b.sub_bas + b.sub_stp * l] = 1.0 * i; }}
Features of C++ translation Arguments of BlockRange constructor are
process dimension and extent of range. Fields of Block define count of local loop
and base and step for local subscript and global index.
If distribution directive is changed to: !HPF$ DISTRIBUTE A(CYCLIC) ONTO P
only change is x declaration becomes: CyclicRange x(50, p.dim(0));
—apparently making progress toward writing code that works for any distribution.
The Block and Location structures
struct Block { int count;
int glb_bas; int glb_stp;
int sub_bas; int sub_stp;};
struct Location {
int sub; int crd; . . .};
Memory strides
Fortran 90 program:REAL B(100, 100). . .CALL FOO(B(1, :))
SUBROUTINE FOO(C )REAL C(:). . .END
First dimension of D most-rapidly-varying in memory.
Second dimension has memory stride 100—inherited by C.
Fortran compilers normally pass a dope vector containing r extents and r strides for rank-r argument.
Stride not really a property of the distributed range. Store separately in DAD.
A DAD
Abstract DAD for a rank-r array is an object containing: A distribution group, and r range objects, and r integer strides.
Interface of the DAD class
Struct DAD { DAD(const int _rank, const Group& _group, Map _maps []);
const Group& grp() const;
Range rng(const int d) const;
int str(const int d) const; . . .};
Map structure
struct Map { Map(Range _range, const int
_stride);
Range range; int stride;};
Translating HPF program with inherited mapping
Source:
SUBROUTINE INIT(D)
REAL D(50)!HPF$ INHERIT D
FORALL (I = 1:50) D(I) = 1.0 * I
END
Translation:
void init(float* d, DAD* d_dad) {
Group p = d_dad->grp(); if (p.member()) { Range x = d_dad->rng(0); int s = d_dad->str(0);
Block b; x.block(&b, p.dim(0).crd());
for (int l = 0; l < b.count; l++) { const int i = b.glb_bas + b.glb_stp * l +
1; d [s * (b.sub_bas + b.sub_stp * l)] = 1.0
* i; } }}
Translation of call with block-distributed actual
Source:
!HPF$ PROCESSORS P(4)
REAL A(50)!HPF$ DISTRIBUTE A(BLOCK) ONTO
P
CALL INIT(A)
Translation:
Procs1 p(4);
BlockRange x(50, p.dim(0));
float* a = new float [x.volume()];
Map maps [1];maps [0] = Map(x, 1);
DAD dad(1, p, maps);
init(a, &dad);
Translation of call with cyclically distributed actual
Source:
!HPF$ PROCESSORS P(4)
REAL A(50)!HPF$ DISTRIBUTE A(CYCLIC) ONTO P
CALL INIT(A)
Translation:
Procs1 p(4);
CyclicRange x(50, p.dim(0));
float* a = new float [x.volume()];
Map maps [1];maps [0] = Map(x, 1);
DAD dad(1, p, maps);
init(a, &dad);
Translation of call with strided alignment of actual
Source:
!HPF$ PROCESSORS P(4)
REAL A(100)!HPF$ DISTRIBUTE A(BLOCK) ONTO
P
CALL INIT(A(1:100:2))
Translation:
Procs1 p(4);
BlockRange x(100, p.dim(0));
float* a = new float [x.volume()];
// Create DAD for section a(::2)
Range x2 = x.subrng(50, 0, 2);
Map maps [1];maps [0] = Map(x2, 1);
DAD dad(1, p, maps);
init(a, &dad);
Translation of call with row-aligned actual
Source:
!HPF$ PROCESSORS Q(2, 2)
REAL A(6, 50)!HPF$ DISTRIBUTE A(BLOCK, BLOCK) ONTO Q
CALL INIT(A(2, :))
Translation:Procs2 q(2, 2);
BlockRange x(6, q.dim(0)), y(50, q.dim(1));
float* a = new float [x.volume() * y.volume()];
// Create DAD for section a(1, :)
Location i;x.location(&i, 1);
Group p = q;p.restrict(q.dim(0), i.crd);
Map maps [1];maps [0] = Map(y, x.volume());
DAD dad(1, p, maps);
init(a + i.sub, &dad);
Other features of the Adlib DAD Support for block-cyclic distributions.
Local loops traversing distributed data need outer loop over set of local blocks. LocBlocksIndex iterator class. offset() method computes overall memory offset.
Support for ghost extension, other memory layouts. Shift in memory for ghost region not in local subscript (universal—memory-layout-independent). disp(), offset(), step() methods applied to local subscript.
Other features of the Adlib DAD, II Support for loops over subranges.
Additional block() methods take triplet arguments—directly traverse subranges. crds() methods define ranges of coordinates where local blocks actually exist.
Other feature to support communication library. AllBlocksIndex.
Miscellaneous inquires and predicates. Useful in general libraries, and for runtime checking programs for correctness.
Next Lecture: Communication in Data Parallel
Languages Patterns of communication needed to
implement language constructs. Libraries that support these
communication patterns.
top related