mochi: composable lightweight data services for …...– provide a reusable toolkit of components...
Post on 03-Sep-2020
3 Views
Preview:
TRANSCRIPT
6TH JLESC WORKSHOP
MOCHI: COMPOSABLELIGHTWEIGHT DATA SERVICES FOR HPC
erhtjhtyhy
PHIL CARNSMathematics andComputer Science DivisionArgonne National Laboratory
November 30, 2016
Kobe, Japan
MOTIVATING TRENDS IN HPC DATA AND STORAGE
HARDWARE TRENDS
More layers are being added to the storage hierarchy
Tradeoffs in price, maturity, capacity, longevity, performance, and density
MemoryMemory
Burst BufferBurst Buffer
Parallel File SystemParallel File System
Campaign StorageCampaign Storage
ArchiveArchive
MemoryMemory
Parallel File SystemParallel File System
ArchiveArchive
HPC Before 2016
HPC After 20161-2 PB/sec Residence – hours Overwritten – continuous
4-6 TB/sec Residence – hours Overwritten – hours
1-2 TB/sec Residence – days/weeks Flushed – weeks
100-300 GB/sec Residence – months-year Flushed – months-year
10s GB/sec (parallel tape Residence – forever
HPSS
Parallel
Tape
Lustre
Parallel File
System
DRAM
Slide adapted from
Gary Grider (LANL)
SOFTWARETRENDS
4
Application Data
Application
CheckpointsExecutables
and Libraries
Intermediate
Data
Products
SPINDLE SCR
FTIDataWarp
MDHIMKelpie
� Alternative to one-size-fits-all “parallel file system for data management” model
� Data services are provisioned on demand
� Co-designed with applications in some cases
Data services are
specializing to provide
application and domain-
specific functionality
WHAT’S NEXT?Enabling the evolution of data-intensive scientific services
5
� Support emerging hardware (e.g., NVM), storage hierarchies, and deployment
scenarios
� Lower the barrier to entry for new specialized services
– Avoid building new services from the ground up
– Provide a reusable toolkit of components and “microservices” for critical
functionality
– Compose to as needed for the task at hand
� Example microservices: key/value storage, group membership, replication
� Example composed scientific service: “Genomics Query Service”
ENABLING DATA SERVICES
ROB ROSS, PHILIP CARNS, KEVIN HARMS,
JOHN JENKINS, AND SHANE SNYDER
GARTH GIBSON, CHUCK CRANOR,
QING ZHENG, AND GEORGE AMVROSIADIS
JEROME SOUMAGNE AND JOE LEE
GALEN SHIPMAN AND BRAD SETTLEMYER
Argonne National Laboratory
Carnegie Mellon University
The HDF Group
Los Alamos National Laboratory
EXAMPLE DATA SERVICEEarly proof of concept: lightweight object storage service
7
� RPC framework: Mercury
� Concurrency: Argobots
� Network layer: CCI
� Storage access:
NVML/libpmem
ACCESS LATENCY
8
� Can we retain the performance
characteristics of NVM across
remote/distributed services?
� Must also be resource-friendly (no busy-
polling on network, limited core usage)
Evaluation on InfiniBandnetwork with RAM storage
ACCESS LATENCY
9
� Can we retain the performance
characteristics of NVM across
remote/distributed services?
� Must also be resource-friendly (no busy-
polling on network, limited core usage)
Protocol modes:
� Eager mode, data is packed into
RPC msg
Evaluation on InfiniBandnetwork with RAM storage
ACCESS LATENCY
10
� Can we retain the performance
characteristics of NVM across
remote/distributed services?
� Must also be resource-friendly (no busy-
polling on network, limited core usage)
Protocol modes:
� Eager mode, data is packed into
RPC msg
� Data is copied to/from pre-registered
RDMA buffers
Evaluation on InfiniBandnetwork with RAM storage
ACCESS LATENCY
11
� Can we retain the performance
characteristics of NVM across
remote/distributed services?
� Must also be resource-friendly (no busy-
polling on network, limited core usage)
Protocol modes:
� Eager mode, data is packed into
RPC msg
� Data is copied to/from pre-registered
RDMA buffers
� RDMA “in place” by registering
memory on demand
Evaluation on InfiniBandnetwork with RAM storage
12
� Objective: Provide computation cache as a
service to minimize fine-scale model executions
� Approach:
– Start with a key/value store
– Distributed approximate nearest-neighbor
query for interpolation candidates
� Status:
– Mercury-based in-memory DB service
– Investigating distributed, incremental
nearest-neighbor indexing
– Applying technique to multiple applicationsImport/export
DB instances
Distributed DB
Application domain
Query 6D space for
nearest neighbors Co-designing a fine-scale model database
EXAMPLE DATA SERVICE
DATA RESEARCH AT ANL
� Mochi:
– Specialized data services
– http://www.mcs.anl.gov/research/
projects/mochi/
� Darshan:
– Large scale I/O characterization
– http://www.mcs.anl.gov/research/
projects/darshan/
� CODES:
– Discrete event simulation of HPC
storage and networks
– http://www.mcs.anl.gov/research/
projects/codes/ 13
Observe:
Darshan
Implement:
Mochi
Model:
CODES
Mochi is just one component
www.anl.gov
THANK YOU!
THIS WORK WAS SUPPORTED BY THE U.S. DEPARTMENT OF ENERGY, OFFICE OF
SCIENCE, ADVANCED SCIENTIFIC COMPUTING RESEARCH, UNDER CONTRACT
DE-AC02-06CH11357.
WE ARE HIRING! THE MATHEMATICS AND COMPUTER SCIENCE DIVISION OF
ARGONNE NATIONAL LABORATORY IS SEEKING OUTSTANDING PEOPLE AT
MULTIPLE CAREER LEVELS.
HTTP://BIT.LY/2G8MOPV
(COMPUTER SCIENCE RESEARCH)
HTTP://BIT.LY/2FSPJW5
(SOFTWARE DEVELOPMENT)
top related