1 "workshop 31: developing a hands-on undergraduate parallel programming course with pattern...
TRANSCRIPT
1
"Workshop 31: Developing a Hands-on Undergraduate Parallel Programming
Course with Pattern Programming
SIGCSE 2013 - The 44th ACM Technical Symposium on Computer Science Education
Saturday March 9, 2013, 3:00 pm - 6:00 pm
Dr. Barry WilkinsonUniversity of North Carolina Charlotte
Dr. Clayton FernerUniversity of North Carolina
Wilmington
© 2013 B. Wilkinson/Clayton Ferner SIGCSE 2013 Workshop 31 Intro.ppt Modification date: March 1, 2013
Problem Addressed
• To make parallel programming more useable and scalable.
• Parallel programming -- writing programs using multiple computers and processors collectively to solve problems -- has a very long history but still a challenge.
2
Traditional approach
• Traditional approach
• Explicitly specifying message-passing (MPI), and
• Low-level threads APIs (Pthreads, Java
threads, OpenMP, …).
• Need a better structured approach.3
Pattern Programming Concept
Programmer begins by constructing his program using established computational or algorithmic “patterns” that provide a structure.
4
“Design patterns” part of software engineering for many years:
• Reusable solutions to commonly occurring problems *• Patterns provide guide to “best practices”, not a final
implementation• Provides good scalable design structure• Can reason more easier about programs• Potential for automatic conversion into executable code
avoiding low-level programming – We do that here.• Particularly useful for the complexities of
parallel/distributed computing* http://en.wikipedia.org/wiki/Design_pattern_(computer_science)
5
In Parallel/Distributed computingWhat patterns are we talking about?
• Low-level algorithmic patterns that might be embedded into a program such as fork-join, broadcast/scatter/gather.
• Higher level algorithm patterns for forming a complete program such as workpool, pipeline, stencil, map-reduce.
We concentrate upon higher-level “computational/algorithm ” level patterns rather than lower level patterns.
Some Patterns
6
Workers
Workpool
Master Two-way connection
Compute node
Source/sink
7
Workers
Pipeline
MasterTwo-way connection
Compute node
Source/sink
One-way connection
Stage 1 Stage 3Stage 2
8
Divide and Conquer
DivideTwo-way connection
Compute node
Source/sink
Merge
9
All-to-All
Two-way connection
Compute node
Source/sink
10
Stencil
Two-way connection
Compute node
Source/sink
Usually a synchronous computation
- Performs number of iterations to converge on solutione.g. for solving Laplace’s/heat equation
On each iteration, each node communicates with neighbors to get stored computed values
Parallel Patterns• Advantages
• Possible to create parallel code from the pattern specification automatically – see later.
• Abstracts/hides underlying computing environment• Generally avoids deadlocks and race conditions• Reduces source code size (lines of code)• Hierarchical designs with patterns embedded into
patterns, and pattern operators to combine patterns
• Disadvantages• New approach to learn• Takes away some of the freedom from programmer• Performance reduced slightly(but compare using high
level languages instead of assembly language)11
Previous/Existing Work
• Patterns explored in several projects.• Industrial efforts
– Intel
Threading Building Blocks (TBB), Intel Cilk plus, Intel Array Building Blocks (ArBB).
Focus on very low level patterns such as fork-join
• Universities:– University of Illinois at Urbana-Champaign and University of
California, Berkeley– University of Torino/Università di Pisa Italy
12
Our approachFocuses on a few higher level patterns of wide applicability (e.g. workpool, synchronous all-to-all, pipelined, stencil).
Software framework developed called “Seeds” to easily construct an application from established patterns without need to write low level message passing or thread based code.
Will to automatically distribute code across processor cores, computers, or geographical distributed computers and execute the parallel code.
13
Acknowledgements
Extending work to teaching environment supported by the National Science Foundation under grant "Collaborative Research: Teaching Multicore and Many-Core Programming at a Higher Level of Abstraction" #1141005/1141006 (2012-2015).
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Work initiated by Jeremy Villalobos in his PhD thesis “Running Parallel Applications on a Heterogeneous Environment with Accessible Development Practices and Automatic Scalability,” UNC-Charlotte, 2011. Jeremy developed “Seeds” pattern programming software.
Next
Session 1Seeds Framework and Workpool
PatternMonte Carlo computation
15
Questions