grid io apis william gropp mathematics and computer science division
TRANSCRIPT
Grid IO APIs
William GroppMathematics and Computer Science Division
University of Chicago Department of Energy
Issues in Grid IO
• Latency• Bandwidth• Fault tolerance• Maintaining transparency to the
user• APIs that capture the user’s intent
while preserving well defined semantics and performance
University of Chicago Department of Energy
One User Model
• Parallel application expresses IO using higher-level library (HDF) or parallel IO interface (MPI-IO)
• For many apps, files are not bytes streams Sequences of objects such as solution arrays
• Files are either read or written, not both at the same time Caching of data on client or server side can be used
AppHDF5
MPI-IO ADIOGlobusIO
UDP
University of Chicago Department of Energy
Grid IO with ROMIO
• Implementation strategy: Add a new “filesystem” type: RIO (remote I/O)
• Use ADIO as generic parallel file interface
ADIO
MPI
PVFS NFS Unix Others
ADIOnetw
ork
RIO
RIOD
University of Chicago Department of Energy
Extending ROMIO
MPI_File_read_all
ADIO_ReadStridedColl
ADIOI_PVFS_ReadStridedCollADIO_PVFS_ReadContig
……
• Relatively easy to define new devices
• Layered definition makes simple ports possible
…
University of Chicago Department of Energy
Exploiting Collective Operations
Write_all
Write_all
Write_all
Write_all
RIO RIOD
Aggregated data written
Must preserve collective I/O operation to retain performance
# of servers # processes
University of Chicago Department of Energy
Caching for Noncollective I/O
Read
Read
Read
Read
RIO RIOD
Note that caching is nearly impossible if full POSIX semantics retained
Replicas a related, higher-level approach already supported by grid toolkits like Globus
University of Chicago Department of Energy
Optimizing WAN Data Transfers
• TCP: Stream interface (in-order delivery) Implementations use window to optimize for
occasional out-of-order delivery
• File ops: Commonly read/write an object, not a stream Object may be large (MB to TB) Exploit by deferring/aggregating acks and retries on
an object basis (one kind of fault tolerance) Stripe data paths (GridFTP uses just this)
• Preliminary work is promising (Dickens et al; PDPTA 2001) Exploits user-intent at MPI level
University of Chicago Department of Energy
Quality of Service for Data Transfers
• What is the user API?• One approach: MPI Attributes
Attributes are a (key,value) pair; value is a pointer May be attached to communicators, datatypes, and MPI
memory windows Used in MPICH-GQ (Globus-enabled version of MPICH
with QoS)
• MPI File objects do not have attributes But do have “info”: (key,value) pairs, key and value
both a string
• Allows the use of communicator attributes to implement the underlying communication and info hints on the MPI File as the user API
University of Chicago Department of Energy
Issues for Grid IO APIs
• A Grid I/O API must support Collective I/O
• Performance• Latency
Object-based transfer completion• WAN optimizations• Appropriate atomicity
• Exploit opportunities for extensions in standards Use existing mechanisms (APIs) to transfer QoS, intent,
other information
• Use Grid infrastructure E.g., Globus Access to Secondary Storage (GASS)