PAPI Tool Evaluation

Bryan Golden1/4/2004

HCS Research LaboratoryUniversity of Florida

2

Basic Information

Name: Performance Application Programming Interface (PAPI)

Developer: University of Tennessee Knoxville Current version:

PAPI 3.0.7 Website: http://icl.cs.utk.edu/papi/ Contact: Phil Mucci

3

Evaluation (1) Available metrics (5 – standard set as well as comprehensive set)

HW counters Cost (5)

Free Documentation quality (5 – extensive user guide)

Clear document? Yes Helpful document? Yes Useful documents:

PAPI 3 User’s Guide, Programmer’s Reference, Conversion Cookbook, and Software Specification

Extendibility (5 – Many tools leverage the functionality of PAPI) Not as well developed as MDL from Paradyne

Filtering and Aggregation (N/A)

4

PAPI Implementation

Java Monitor GUI

PAPI Low LevelPAPI High Level

Hardware Performance Counter

Operating System

Kernel Extension

PAPI Machine Dependent SubstrateMachine

Specific Layer

Portable Layer

5

Evaluation (2)

Hardware support (5)

Heterogeneity support (4) Yes – Code is highly

portable, yet semantic differences may be troublesome

Hardware Operating System Requirements

POWER3, 604, 604e AIX 4.3.xpmtoolkit from IBM alphaWorks

POWER4, POWER3, 604, 604e

AIX 5.x install bos.pmapi

Cray T3E Unicos None

Cray X1 Unicos/mp None

AMD Athlon/Opteron, Intel thru Pentium III

Linux 2.2,2.4,2.6 Mikael Pettersson's PerfCtr kernel patch for Linux (included)

Intel thru Pentium III Windows NT, 2000, XP

adminstrator privilege for installation

Intel Pentium IV Linux 2.2,2.4,2.6 Mikael Pettersson's PerfCtr kernel patch for Linux (included)

Intel Itanium I & II Linux 2.4,2.6 Stephane Eranian's PerfMon kernel patch for Linux

(included)

UltraSparc I, II & III Solaris 2.8,2.9 None

MIPS R10K, R12K, R14K

IRIX None

Alpha Tru64 Unix Contact paul.drongowski@hp.com for required system

software Alpha Linux IProbe patch

(included)

6

Evaluation (3) Installation (3 – Requires kernel patch in some

cases) How to get software

Download source from their website How hard to install

Simple with user guide Applying the patch can be a problem for some

Components needed Everything is included with the download

Estimate number of hours needed for installation Few hours with help from support

Interoperability (5) Other hardware counters: PCL Other General tools: TAU, SCALEA, vprof, SvPablo,

PeformanceBench, DynaProf

7

Evaluation (4) Learning curve (3 –Very low level, but to be expected)

Very tedious but highly portable The graphical interface, perfometer, is fairly well

documented, however, it is not meant to be used as a tool (just to show off PAPI’s functionality)

Manual overhead (2 – Function calls are almost like using printf, however, the naming conventions are intuitive) Manual instrumentation

Source code instrumentation with instrumentation API No automatic instrumentation

8

Evaluation (5) Measurement accuracy (5)

Evaluation of the measuring technique: The literature on the PAPI website provides validation of their techniques. However, due to the nature of HW counter data, this is almost trivial.

Multiple analyses (1) None

Multiple executions (1 - PAPI is not designed for multiple executions) No support

Multiple views (1) Views

Profiling: perfometer has only one view, however, other tool which use PAPI may be better

Intuitive? Yes, however, perfometer is not meant to be a tool Performance bottleneck identification (1)

No native support for multiple views Performance bottleneck identification (1)

None

9

Evaluation (6) Profiling / tracing support (Various)

PAPI has no native tracing support, however many tools that use it do

Profiling? Tracing? Profiling only

Trace format? N/A

Trace strategy? N/A

Mechanism for turning on and off tracing N/A

Searching (Various) Dependent on the tool used to call the PAPI library routines

Software support (N/A) PAPI is a library

10

Evaluation (7) Source code correlation (4)

Able to correlate performance data to source code: Can correlate back to the function name level but not to line numbers

Response time (4) No noticeable delays

System stability (3) Crash rate: Varies on different platforms, Windows XP crashes often,

Linux is fairly stable Technical support (5)

Time to get a response from developer: Fast Usefulness of messages the system provides: Good, messages are

informative and thorough Comments

De facto standard for accessing HW counters Works well with many other tools Provides a great deal of information Tool is fairly popular