intel technologies’ features and benefits depend on system … · intel® software tools intel®...
TRANSCRIPT
All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps
http://www.intel.com/performance.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at http://www.intel.com/content/www/us/en/software/intel-solutions-for-lustre-software.html.
You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
3D XPoint, Intel, the Intel logo, Intel Core, Intel Xeon Phi, Optane and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
* Other names and brands may be claimed as the property of others.
© 2016 Intel Corporation
3
Introductions & Welcome Bret Costelow
State of Lustre* in HPC Today Brent Gorda
The New OpenSFS* and Why Join? Peter Jones
Intel® Solutions for Lustre* software Roadmap Micah Bhakti
The Future of HPC Data Management Jessica Popp
Intel® HPC Orchestrator Overview / Update Figen Ulgen
Lustre at SC16 & Closing Bret Costelow
Bret CostelowDirector, Sales/Global Business developmentHigh Performance Data Division
November 2016: Intel’s Analysis of Top 100 Systems (top100.org)
9 of Top10 Sites
75% of Top100
Most Adopted PFS
Most Scalable PFS
Open Source GPL v2
Commercial Packaging
Vibrant Community
6
75%
19%1% 7%
Lustre GPFS Other Unknown
LIFE SCIENCE Oil & GasArtificial Intelligence Financial Services
7
Precision MedicineGenomic Data AnalysisModeling & Simulations
Autonomous CarsMachine / Deep Learning
Economic & Stock Market Forecasting, Simulation.
Financial Analysis. portfolio management
Seismic analysis, oil services, reservoir
modeling
Visit ‘Events’ Page.Subscribe HPDD Newsletter. www.intel.com/lustre
8
Entry Level Small Form Factor Lustre Solution
Data Migration with Intel® EE for Lustre* software
O&G Case StudyGenomics Case Study
9
www.intel.com/lustre-training
About 1000 Users have accessed Training
4 Courses / 19 Modules.
Fundamentals to Advanced Topics.
Training Refresh – Q1 ‘17
Brent GordaGeneral ManagerHigh Performance Data Division
Intel® Scalable System Framework
1
Small Clusters Through Supercomputers
Compute and Data-Centric Computing
Standards-Based Programmability
On-Premise and Cloud-Based
Intel® Xeon® Processors
Intel® Xeon Phi™ Processors
Intel® FPGAs and Server Solutions
Intel® Solutions for Lustre*
Intel® Optane™ Technology
3D XPoint™ Technology
Intel® SSDs
Intel® Omni-Path Architecture
Intel® Silicon Photonics
Intel® Ethernet
Intel® HPC Orchestrator
Intel® Software Tools
Intel® Cluster Ready Program
Intel Supported SDVis
ComputeFabric
Memory / Storage
Software
A Holistic Solution for All HPC Needs
13
Statistics courtesy of Dustin Leverman(ORNL)Source: http://git.whamcloud.com/fs/lustre-release.git.
14
ANU 14 Atos 7 CEA 8
Clogeny 1
Cray
32
DDN 20
Fujitsu 1
Intel 359
IU 29
LLNL 32
ORNL 37
Other 11
Purdue 1
Seagate 82
SGI 1 Intel remains committedLustre is a key ingredient in systems
Intel support for OpenSFS
Intel & HPC SW
HPDD & community releases
Highlights
Uni Hamburg + German Client Research Centre (DKRZ)
Client-side data compression
Adaptive optimized ZFS data compression
15
GSI Helmholtz Centre for Heavy Ion Research
TSM* HSM copytool
Johannes Gutenberg University Mainz
Global adaptive IO scheduler
Lawrence Berkeley National Laboratory
Spark* and Hadoop* on Lustre
University of California Santa Cruz
Automated client-side load balancing
Intel® Parallel Computing Centers
Peter JonesDirector of Lustre EngineeringHigh Performance Data Division
OpenSFS Updates
Stephen Simms
President
OpenSFS
LUG 2017 !
Indiana
University in
Bloomington
Indiana
May 30-June 2
2017
What Is OpenSFS?
• OpenSFS is a vendor neutral, member supported non-profit organization
bringing together the open source file system community for the high
performance computing sector.
• Our mission is to aggregate community resources and be the center of
collaborative activities to ensure efficient coordination of technology
advancement, development and education.
• The end goal is the continued evolution of robust open source file systems for
the HPC community.
19
OpenSFS Reorganization
• Why?
Establish total user community control over OpenSFS
Broaden the membership base
Increase participation from the members
Distribute responsibility equally among the members
New OpenSFS Structure
• Two levels of membership
Members
– Lustre end users
Participants
– Lustre vendors
• User community controlled and driven
Board members can only be selected from the Members
Members vote to elect the Board and on changes to OpenSFS
• Flat and low annual membership fee
$1000 – Members
$5000 – Participants
OpenSFS Responsibilities
• Organize LUG
• Collect feature and development requests from the Members
• Produce an annual document summarizing the requests for Participants
• Provide a unified voice for the user community to Lustre vendors
New OpenSFS Structure
• Why join? http://opensfs.org/join/
• Bylaws are available here
http://cdn.opensfs.org/wp-content/uploads/2016/09/Open-SFS-Amendment-and-
Restated-Bylaws_Final_091516.pdf
• 5 Board members
Will be elected by the Members
Two-year terms
Currently governed by an appointed Board
Current Board & Upcoming Elections
• Current Board (appointed by Previous Board)
Stephen Simms – President - Indiana University
Sarp Oral – Vice President – Oak Ridge National Laboratory
Steve Monk – Treasurer – Sandia National Laboratory
Rick Wagner – Secretary – Globus
Shawn Hall – At Large – BP
• Next elections will be held at LUG’17 (Indiana University May 30th – June 1st)
Nominations Committee will nominate the candidates for Board Positions
• Nominations Committee Chair
Mark Gary – Lawrence Livermore National Laboratory
• Nominations also by petition
Summary
• Lustre’s Roadmap continues to advance to meet the evolving needs of HPC
• OpenSFS is an organization that represents the users of Lustre
• You can find out more about
Lustre at www.lustre.org
OpenSFS at www.opensfs.org
• For details on how to join OpenSFS contact [email protected]
• Save the date – LUG 2017 @ Indiana University May 31st – June 2nd
www.opensfs.org
Open Scalable File Systems, Inc.3855 SW 153rd Drive
Beaverton, OR 97003
Ph: 503-619-0561
Fax: 503-644-6708
Thank you!
Copyright © 2016 OpenSFS.
LUG 2017 !
Indiana
University in
Bloomington
Indiana
May 30-June 2
2017
Micah BhaktiProduct ManagerHigh Performance Data Division
Product placement not representative of final launch date within the specified quarter
Q4 ’16 Q1 ’17 Q2’17 Q3 ’17 Q4 ’17
EnterpriseHPC, commercial technical computing, and analytics Client m’data perf.
OpenZFS perf.SnapshotsSELinux, KerberosOPA supportRHEL 7OpenZFS 0.6.5.3
EE 3.0
Lustre* 2.7.x
DNE 2Client m’data perf.LFSCK perf.SELinux, Kerberos
FE 2.8
Lustre 2.8.x
CloudHPC, commercial technical computing and analytics delivered on cloud services
FoundationCommunity Lustresoftware coupled with Intel support
Expanded OpenZFS support in IMLIML graphical UI enhancementsBulk IO Performance (16MB IO)OpenZFS metadata performance Subdirectory mounts
Enterprise Edition 3.1
Lustre 2.7.x
DNE 2Multi-rail LNETIML asset mgmt. ZED updates
EE 4.0
Lustre <TBA>
RHEL 7 Server SupportLustre release update
Cloud Edition 1.3
Lustre 2.8.x
SW updatesNew instances
CE 1.5
Lustre <TBA>
Updated Lustre and base OS buildsNew instance support
Cloud Edition 1.4
Lustre 2.9.x
UID/GID MappingShared key cryptoLarge Block IOSubdirectory mounts
Foundation Edition 2.9
Lustre 2.9.x
OpenZFS SnapshotsMulti-rail LNETProject quotas
Foundation Edition 2.10
Lustre 2.10.x
Shipping
Development
Planning
28
29
OpenZFS Storage Enhancements
Managed Mode / Metadata Performance Increased
Lustre Manager Updates
Drill Down Interface / Expanded Job Stats
New Lustre Core Features
Subdirectory Mounts / Bulk IO Performance
30
Metadata Striping
Directory Scaling Across Metadata Servers
Multi-rail LNET
Multiple Network Interfaces for Bandwidth / Resilience
OpenZFS Fault Management
Fault Management Architecture for OpenZFS
31
Rapid Storage Deployment
Easily Spin Up Storage Clusters in Under 10 Min
Scale Up Performance
Storage Performance Scales With Storage Instances
Secure Storage Encryption
Encrypted Data Over-the-wire and at Rest
32
Remote Mapping & Crypto
Lightweight Remote Client / Authentication & Encryption (IU, OpenSFS)
Application IO Hinting
API & Client Interface for Performance IO Handling (DDN)
New Lustre Core Features
Subdirectory Mounts (DDN) / Bulk IO Performance
Ongoing Feature Engineering
33
HSM Data Mover
Decouples HSM Agent and Data Mover Components
Modular Storage Interfaces
Data on MDT
Optimize performance of small file IO
Small files (as defined by administrator) are stored on MDT
Client MDSOpen, write, attributes
layout, lock, attributes, read
HSMAgent
POSIX
RPC
S3
Lib
lustre
.ap
i
Jessica PoppDirector of EngineeringHigh Performance Data Division
36
Intel Developers
LustreCommunity
LustreResellers
Lustre has provided a compelling story for Technical Computing for
many years
Leader in performance and scalability for POSIX environments
Focused on supporting next-gen fabric, silicon, and solid state storage
Has a healthy and growing ecosystem of users and contributors
Statistics courtesy of Lawrence Livermore National Laboratories (Chris Morrone)Source: http://git.whamcloud.com/fs/lustre_release.git/shortlog/refs/heads/b2_8
37
ANUAtos
Canonical
CEACray
DDN
Fujitsu
GSI
Intel
IULLNL
ORNL
Seagate
SGI
Clogeny
Other41
55
42
52
69 7076
65
92
17
1013
1915 14 15 17
0
10
20
30
40
50
60
70
80
90
100
1.8.0 2.1.0 2.2.0 2.3.0 2.4.0 2.5.0 2.6.0 2.7.0 2.8.0
Unique Developer and Organization Count
Developers Organizations Lustre 2.8 Commits
38P
erf
orm
an
ce/S
cala
bil
ity
Availability
• Enterprise
• Cloud
• HPC/Tech Computing • Next Generation Storage
Factors Driving Change:
The rapid growth of datasets is demanding unprecedented scale
Mission critical data & decreased MTTF require data availability and resilience
Advances in memory and fabric technologies are straining legacy storage software stacks
39
Lustre is POSIXA strong & growing ecosystem
Broad application support
Long-term product roadmap advances core feature set
DAOS is object Leverages next-gen HW technologies
New levels of scalability and resiliency
Pure object storage model exposes new capabilities for scalable consistency
Pool
Pool
DAOS Tier
Container
Object
DKey
Akey[i]
Intel is driving continued investment in Lustre to advance core features and maintain a dominant position in the Technical Computing segment:
40
Performance Resiliency Security IntegrationMetadata Directory Scaling
OpenZFS Performance Enhancements
Multi-rail LNET
Improved Small File Performance
OpenZFS Fault Monitoring Framework
Progressive File Layouts
File Level Replication
Tiered Storage Enhancements
Over-the-Wire Encryption
On-disk Encryption for Data at Rest
Isolation for Subdirectory Mounts
Mandatory Access Control using SELinux*
Upstream Lustre Client
Support for Next-Generation Fabrics
Optimizations for GPGPU Architectures
DAOS - Distributed Async Object Storage Framework
Reliable, fast, flexible object storage
DAOS API
Allows applications to directly access
native Open Source Object Storage APIs
New Applications
HDF5*, NetCDF*
Modern HPC data management framework makes porting apps easy
Existing Applications
Hadoop,* Spark*
HDFS could access DAOS Storage easily to support popular analytics tools
AnalyticsFrameworks
POSIX-lite
Full POSIX via Lustre*
Posix-compliant distributed file system for legacy application access
Legacy ApplicationsTop Level API
Domain-specific APIs and high-level data models
Simplifies integration and application support
DAOS API
New applications can write directly to DAOS API
Allows direct integration with DAOS functionality
DAOS Storage
Distributed storage provides caching and resilience
Fine-grained versioning and global consistency
DAOS provides a flexible and scalable object storage system
42
Multiple Top-level APIs
Domain-specific APIs: HDF5*, ADIOS*, PnetCDF*
High-level data models: HDFS*, Spark*
Caching and Tiering
Transparent caching within storage system
Efficient and transparent data staging
Sharding and Resilience
Throughput scales across storage nodes
Erasure coding & replication for availability
Memory class object storage
Ultra-low latency / fine grain I/O
– Direct NVRAM load/store access
– Bypasses OS & traditional I/O stack
Fine-grain versioning & global consistency
Applications
Top-level APIs
Tools
DAOS-SR
DAOS-M
DAOS-CT
43
DoE FastForward Program delivered PoC DAOS code over Lustre (2012-2014)
DAOS Design developed for DOE CORAL program based on FastForward learnings (2015)
DoE ESSIO program developing a pre-production model of HDF5 & DAOSwhich tests DoE application integration (2016-2017)
Additional projects in the pipeline to continue to advance DAOS functionality
Additional work is needed to fully implement the stackand is targeted for the Exascale timeframe
Figen UlgenGeneral ManagerHPC Platform Software & Cloud
Fragmented efforts across the ecosystem, with minimal commercial investment
A desire to get exascale performance & speed up
software adoption of
HW innovation
New complex
workloads (ML, Big
Data, etc.) drive more complexity
into the software
stack
THE REALITY: We will not be able to get where we want to go without a major change in system software development
“Everyone building their own solution”
The HPC Software ecosystem challenge we’re solving
Intel® HPC OrchestratorIntroducingHPC modular system software platform that
simplifies integration, validation, & maintenance.
Reduces complexity and duplicated effort & includes Intel Technical Support
Launching at SC’16!
Intel® HPC Orchestrator
Intel® HPC Orchestrator
SC’15Project Launched
ISC’16Product Announced
SC’16Product Launch
Participation in OpenHPC as of November 2016
• Argonne National Laboratory • Center for Research in Extreme Scale Technologies – Indiana University
29 Members
OpenHPC is a Linux Foundation* Project which gained wide participation right away
The goal is to collaboratively advance the state of the software ecosystem
Governing board is composed of Platinum members plus reps from Silver , Academic, Technical committees
WWW.OpenHPC.CommunityCourtesy of OpenHPC*
*Other names and brands may be claimed as the property of others.
• Intra-stack APIs to allow for customization/differentiation (OEM enabling)• Defined external APIs for consistency across versions (ISVs)
Node-specific OS Kernel(s)
Linux* Distro Runtime Libraries
Overlay & Pub-sub Networks, Identity
User Space Utilities
SW Development
ToolchainCompiler &
Programming Model
Runtimes
High Performance
Parallel LibrariesScalable
Debugging & Perf
Analysis Tools
Optimized I/O
Libraries
I/O Services
Data Collection
And System
Monitors
Workload Manager
Resource Mgmnt
Runtimes
DB Schema
Scalable DBS
yst
em
Ma
na
ge
me
nt
(Co
nfi
, In
ve
nto
ry)
Pro
visio
nin
g
Sy
stem
Dia
gn
ostics
Fa
bric M
gm
nt
Operator Interface
ISV Applications
Hardware
Intel® Enterprise Edition for Lustre client pre-integrated**
** - Intel® Enterprise Edition for Lustre* support contract purchased separately.
Intel® HPC Orchestrator Modular View
Intel® HPC Orchestrator
• Open Source community under Linux Foundation*
• Ecosystem innovation building a consistent HPC SW Platform
• Platform agnostic
• 29 global members
• End users / OEM / ISV mix
Intel® Scalable System Framework A Holistic Design Solution for All HPC
• Intel’s distribution of OpenHPC*
• 3 products to serve differing HPC user segment needs
• Intel HW optimized
• Advanced testing, premium features
• Product technical support & updates provided by Intel
• Provides software integration and validation of Intel SSF elements
• Optimization for Intel SSF configurations
• On-Premise and Cloud-Based
SC’16 announcements
Intel® HPC Orchestrator Target end user segment Key Features
“ADVANCED” (available) Top500: research & enterprise - Sufficient performance and scalability
- Ease of Install
“TURNKEY” (future) Enterprise, workload specific - Ease of use & administration
- Auto-configuration
“CUSTOM” (future) Top50: large scale, high perf. - Performance & scalability- Energy efficiency
Available through two initial OEM partners
Q1’17 New Feature: PBS Pro* added as workload manager option with Technical SupportAdding to currently available SLURM Technical Support
*Other names and brands may be claimed as the property of others.
*Additional partner announcements expected shortly
Intel® HPC Orchestrator system software is GA !
Bret Costelow
Intel @SC16 – intel.com/sc16Lustre@SC16 – intel.com/lustre/sc16