ddn update 2019 - npo tsukuba oss technical support...
TRANSCRIPT
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
1
DDN Storage | © 2019 DDN Storage © 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
DDN Storage | © 2019 DDN Storage
DDN Update 2019Gfarm Workshop 2019
2019/2/1
DataDirect Networks Japan, Inc.
橋爪信明 ([email protected])
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
2
DDN Storage | © 2019 DDN Storage 2
2018年主な出来事@DDN
• Declustered RAID(DCR)対応製品へラインナップ刷新
► SFAOS11世代のHWへの切り替え元年
► SFA200/400NV, SFA7990, SFA18K
• 6月 Intel社よりLustre部門を買収
► DDNのWhamcloudディビジョンとして活動
► Open Source Community Edition開発
► Lustre L3サポート
• 9月 Tintri社を買収
► DDN内の独立したTintriディビジョンとして活動
►仮想化・マルチクラウド自動最適化フラッシュストレージ
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
3
DDN Storage | © 2019 DDN Storage 3
DDN SFA
ALL-FLASH AND HYBRID BLOCK STORAGE PLATFORMS
200NV 400NV 7990 14KX 18K
20GB/s
1M IOP/s
40GB/s
2M IOP/s
20GB/s
1M IOP/s
60GB/s
4M IOP/s
90GB/s
3.2M IOP/s
24 NVME Slots 24 NVME Slots
Up to 450
SSD/HDD
48 NVMe Slots
Up to 1872
SSD/HDD
48 NVMe Slots
Up to 1872
SSD/HDD
EDR IB (4), OPA (2)
FC32 (8), FC (8)
EDR IB (8), OPA (4) EDR IB (4), OPA (2)
FC16 (8)
EDR IB (12|8)
OPA (4), FC16 (24)
EDR IB (16), OPA (8)
COMING 2019COMING 2019 NEW 2018NEW 2018 To be EOS
2019年より全グレードで新ラインナップを展開
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
4
DDN Storage | © 2019 DDN Storage 4
DDN SFA | 製品名と位置付け
位置付け 用途 200NV 400NV 7990 18K
総称 全体をまとめて呼ぶ場合 SFA200NV SFA400NV SFA7990 SFA18K
ブロックストレージ ファイルサーバに接続 SFA200NV SFA400NV SFA7990 SFA18K
エンベデッドアプライアンス コントローラ上にファイルサーバを実装
SFA200NVE SFA400NVE SFA7990E SFA18KE
ExaScalerアプライアンス ExaScaler(Lustre)エンベデッド ES200NV ES400NV ES7990 ES18K
GridScalerアプライアンス GridScaler(GPFS)エンベデッド GS200NV GS400NV GS7990 GS18K
A3Iアプライアンス AI向けエンベデッド AI200 AI400 AI7990
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
5
DDN Storage | © 2019 DDN Storage © 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
DDN Storage | © 2019 DDN Storage
SFA18K
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
6
DDN Storage | © 2019 DDN Storage 6
MAXIMUM SCALABILITY WITH NVMe + SAS
* Cached
SFA18K
FOR CAPACITY & BANDWIDTH WITH DRIVE TYPE FLEXABILITY
Block 90GB/s
Embedded 78GB/s
• Start in as little as 4U and Scale up & Scale out;
Up to 1872 drives per system:
o 72 Slots in “head”, 48 NVMe/SAS + 24 SAS
o Add up to 20x SS9012 Enclosures (90 SAS slots)
• Flexible Access for File: IB or Ethernet (16 ports), OPA
(8 ports)
Integrates high-performance, low-latency, RDMA capable
networks. EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet and OPA
• Access for Block: IB (16 ports)
Block systems attach via EDR IB to external servers
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
7
DDN Storage | © 2019 DDN Storage 7
SFA18K Overview
Storage Fusion FabricHighly-Over-Provisioned Backend RAIDed
Fabric Withstands Failures of Drives,
Enclosures, Cables, etc
Data IntegrityReal-Time Data Integrity Verification for every I/O
Flash and NVMe Enabled in
‘head’Up To 48 NVMe devices
Up to 72 SAS SSD
Broad Connectivity Options
EDR IB
40/100 GbE
OmniPath
High Density ArrayUp to 972 drives in 44U rack.
SS9012 enclosure is 90 HDDs / SSDs in
4U
ReACT Cache ManagementReal-Time I/O Data-Aware Cache Optimizer,
Small IOs Go to Cache; Large & Streaming
I/Os Bypass Cache
Real-Time, Multi-CPU RAID EngineFull Implementation of Real-Time Storage OS,
Interrupt-Free; Massively Parallel I/O Delivery
System
SFA Embedded Low-Latency VirtIO data path for DMA IO access
to local VM
0 500 1000
8+2 rebuild vs DCR rebuild in 112 disk pool and 210 disk pool
DCR 210 DCR 112 Legacy 8+2
DeClustered RAID
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
8
DDN Storage | © 2019 DDN Storage 8
SFA18K Controller
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
9
DDN Storage | © 2019 DDN Storage 9
ConfigSFA18K
Only+ 1 SS9012 +2 SS9012 +4 SS9012 +5 SS9012 +6 SS9012 +8 SS9012 +10 SS9012 +16 SS9012 +20 SS9012
SFA (BLOCK)
ES
GS
2.5” NVMe or
SAS SSD
Slots
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
72
(NVMe:48)
3.5” HDD or
SAS SSD
Slots
0 90 180 360 450 540 720 900 1440 1800
Enclosure
ConnectionNA Direct Direct Direct Direct Direct Direct Direct
Direct
Daisy ChainDaisy Chain
最大物理容量 NA 1.26PB 2.52PB 5.04PB 6.30PB 7.56PB 10.08PB 12.6PB 20.16PB 25.20PB
Expansion
Options
SFA18K CONFIGURATIONS
※最大物理容量は14TB NL-SAS HDDをSS9012に実装した場合
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
10
DDN Storage | © 2019 DDN Storage 10
EXAScaler ES18K Early Performance - IOR 1MB FPP
• EXAScaler IOR Client Benchmarking with an ES18K EDR with 408 NL-SAS
Drives as 8 DeClustered RAID Pools of 51 drives.
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
Ban
dw
idth
(M
B/s
ec)
ES18K IOR(FPP) Write: 408x NL-SAS, 8 x51/MR=1
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
Ba
nd
wid
th (
MB
/se
c)
ES18K IOR(FPP) Read: 408x NL-SAS, 8 x51/MR=1
61GB/s
Write58GB/
s Read
Tuning is
still in
progress
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
11
DDN Storage | © 2019 DDN Storage © 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
DDN Storage | © 2019 DDN Storage
SFA7990
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
12
DDN Storage | © 2019 DDN Storage 12
SFA7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
* Cached
SFA7990
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU
Block 20GB/s
Embedded 20GB/s
• Start at 4U and Scale Up or Scale Out
Mix Disk and Flash SSD for Hybrid Performance
• Flexible Access for File: (IB, Ethernet, OPA)
Integrates high-performance, low-latency, RDMA capable
networks. Both EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet plus OPA
• Flexible Access for Block: (IB, FC)
Block systems attach via FC or IB to external servers
DDN’s ES7990 and GS7990, EXAScaler® and GRIDScaler® appliances,
offer scalable, high performance parallel file systems in an integrated
package. Designed, deployed and supported by the experts in data intensive
workloads, these appliances are the gold standard for eliminating bottlenecks and
maximizing application performance.
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
13
DDN Storage | © 2019 DDN Storage 13
SFA7990 OVERVIEW
Redundantly Connected
EnclosuresAll configurations feature directly
attached, redundant links to
SFA7990
Flexible ConnectivityBlock Target: 4x EDR IB OR 8x FC16
Embedded File System: 4 x EDR IB
/40/100 GbE
OR 2 x OPA
Full SAS Performance for all
configurationsBalanced SAS Architecture provides
saturation level performance for all
expansion options
High Density HDD and SAS SSDSS9012 Enclosures and SFA7990 house
up to 90 3.5” Devices.
High Performance SASBackend Bandwidth supported by
12Gbps SAS Links from all
enclosures to SFA7990
SFAOS
Mirrored, battery-backed write-back cache
ReACT Cache Management
Fully Declustered Data Placement for rapid rebuilds
EMBEDDED FILESYSTEM APPLIANCES
EXAScaler®
GRIDScaler®
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
14
DDN Storage | © 2019 DDN Storage 14
Configuration SFA7990 Only + 1 SS9012 +2 SS9012 +4 SS9012
Supported
Solutions
SFA (BLOCK)
ES7990
GS7990
3.5” HDD or SAS
SSD Slots90 180 270 450
最大物理容量 1.26PB 2.52PB 3.78PB 6.30PB
Expansion
Options
SFA7990 CONFIGURATIONS
※最大物理容量は14TB NL-SAS HDDをSS9012に実装した場合
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
15
DDN Storage | © 2019 DDN Storage 15
ES7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
ES7990
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU
20GB/s
0
5000
10000
15000
20000
25000
80xNL-SAS 160xNL-SAS
Thro
ugh
pu
t (M
B/s
ec)
ES7990 Sequential IO Performance
Write Read
0
2
4
6
8
10
12
14
16
18
20
0
100
200
300
400
500
600
700
800
900
4KB 8KB 16KB 32KB 64KB 128KB
Thro
ugh
pu
t (M
B/s
ec)
IOP
S (o
ps/
sec)
IO Size (Byte)
ES7990 Random IO Performance (10xSSD)
IOPS B/W
780K IOPS@4KB
13.7GB/s Write
205MB/s Per Disk
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
16
DDN Storage | © 2019 DDN Storage 16
GS7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
* Cached
GS7990
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU
20GB/s
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
17
DDN Storage | © 2019 DDN Storage © 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
DDN Storage | © 2019 DDN Storage
SFA200/400NV
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
18
DDN Storage | © 2019 DDN Storage 18
WORLD’S MOST EFFICIENT HPC FLASH
* Cached
SFA200NV
OPTIMIZED FOR NVMe IOPS, MULTICORE & MULTI GPU
Block 20GB/s
Embedded 20GB/s
• Start at 2U and Scale-out
All NVMe Flash, high density, extreme
performance
• Flexible Access for File: (IB, Ethernet,
OPA)
Integrates high-performance, low-latency,
RDMA capable networks. EDR InfiniBand and
100Gbps (or 40Gbps) Ethernet and OPA
• Flexible Access for Block: (IB, FC)
Block systems attach via FC or IB to external
servers
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
19
DDN Storage | © 2019 DDN Storage 19
WORLD’S MOST EFFICIENT HPC FLASH
* Cached
SFA400NVBlock 40GB/s
Embedded 40GB/s
• Start at 2U and Scale-out
All NVMe Flash, high density, extreme performance
• Flexible Access for File: (IB, Ethernet, OPA)
Integrates high-performance, low-latency, RDMA capable
networks. EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet and OPA
OPTIMIZED FOR NVMe IOPS, MULTICORE & MULTI GPU
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
20
DDN Storage | © 2019 DDN Storage 20
GS200NV : SCALABLE FILE PLATFORMS
• Saturation level random read
performance with small I/O
transfers
• Ideal for Analytics platforms
requiring Enterprise Features
OVER 23GB/s
THROUGHPUT EVEN
FOR RANDOM READ
WORKLOADS
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
21
DDN Storage | © 2019 DDN Storage 21
0
200
400
600
800
1000
1200
1400
Raw
Capacity (
TB
)
SFA200NV and SFA400NV Capacities
SFA200NV AND SFA400NV | SCALABLE FILE PLATFORMS
• Choose your starting option
from as small as 38TB or up to
152TB in just 2 RU
• Scale-out with NVMe to
Petabytes of Flash and TB/s of
performance
Start with a complete All
NVMe Flash, Parallel File
System from DDN in just
2RU
S M L 2L 4L 8L38TB 76TB 152TB 304TB 608TB 1.2PB
Coming soon
15.36TB NVMe
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
22
DDN Storage | © 2019 DDN Storage © 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
DDN Storage | © 2019 DDN Storage
A3IAccelerated, Any-scale AI Solutions From DDN
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
23
DDN Storage | © 2019 DDN Storage 23
AI7990 AI200
All NVME parallel file storage appliance
Optimized for the most intensive workloads
30TB, 60TB, 120TB in 2RU
23GB/s, 395K IOPS • 100Gb Ethernet or EDR InfiniBand • Start with 1, Scale out and mix
Hybrid parallel file storage appliance
Optimized for capacity, intermix with flash
1PB in 4RU
DDN PARALLEL FILE STORAGE APPLIANCES FOR AI AND DL
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
24
DDN Storage | © 2019 DDN Storage 24
®
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
25
DDN Storage | © 2019 DDN Storage 25
®
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
26
DDN Storage | © 2019 DDN Storage 26
A3I
wit
h s
ing
le D
GX
-1
A3I
wit
h q
ua
d D
GX
-1
A3I
wit
h D
GX
-1 P
OD
DDN A3I WITH DGX-1 SCALABLE REFERENCE ARCHITECTURES
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
27
DDN Storage | © 2019 DDN Storage 27
Over 100K IOPs and
10GB/s to 1 Container
EXAScaler client demonstrates over 10GB
per second to a single container and over
100K IOPs.
Typical Deep Learning Codes will perform
IOs around 128K IOs – and see around
6GB/s with this pattern.
DDN A3I Solutions: Performance to a single container
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
28
DDN Storage | © 2019 DDN Storage 28
60% Faster -
Consistent, Linear AI
Performance
AI benchmarking of AI Frameworks with
popular test sets demonstrates images per
second rates 60% higher than competing
systems
DDN A3I Solutions: Fully-optimized with widely-used DL frameworks
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
29
DDN Storage | © 2019 DDN Storage 29
AI Benchmarking: DDN enables and accelerates
1796
4344
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Imag
es P
er
Seco
nd
CAFFE GOOGLENET TRAINING THROUGHPUT2.4X MORE IMAGES WITH DDN STORAGE
DDN Storage Other Storage Products
379
197
0 50 100 150 200 250 300 350 400
Tim
e (i
nm
inu
tes)
CAFFE GOOGLENET TRAINING TIME2X FASTER WITH DDN STORAGE
DDN Storage Other Storage Products
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
30
DDN Storage | © 2019 DDN Storage 30
Full GPU Saturation,
Maximum Productivity
DDN AI200 delivers a steady high-
throughput, low-latency stream of data to
the DL training application
Achieving full GPU saturation ensures
every compute cycle is put to productive
use
DDN true end-to-end parallelism enables
acceleration at scale for distributed
computing with multiple multi-GPU nodes
AI Benchmarking: IO during TensorFlow training
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
31
DDN Storage | © 2019 DDN Storage 31
AI Benchmarking: Optimized TensorFlow Data Sets
Optimized Data Sets,
Accelerated Training
Convert discrete data and metadata asset
collections into series of streamlined binary files for
TensorFlow
Maximize DL training and validation efficiency while
minimizing data management and preparation
overhead
Achieve more accurate results by eliminating
manual data shuffle bias
67
23
0 10 20 30 40 50 60 70 80
Tim
e (i
n m
inu
tes)
TFRECORD CONVERSION BENCHMARK3X FASTER WITH DDN STORAGE
DDN Storage Other Storage Products
1614
671
0 200 400 600 800 1000 1200 1400 1600 1800
Tim
e (i
nse
con
ds)
TENSORFLOW RESNET-50 TRAINING PERFORMANCE
2.5X FASTER WITH TFRECORDS
TFRecord Data Set Discrete Data Set
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
32
DDN Storage | © 2019 DDN Storage 32
DDN PARALLEL ARCHITECTURE AND PROTOCOL vs NFS
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
GPU GPU
FULL GPU SATURATION
MAXIMUM AI PRODUCTIVITY
GPU STARVATION
CRIPPLED AI PERFORMANCE
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
33
DDN Storage | © 2019 DDN Storage 33
DDN FULL GPU
SATURATION
MAX GPU
UTILIZATION
DGX-1 GPU UTILIZATION (%) AI200 READ (MB/S)
DGX-1 GPU UTILIZATION (%) NFS STORAGE READ
(MB/S)
2X FASTER
Consistent DL performance
Achieves full GPU utilization
Efficient loading of data set
NFS GPU
STARVATION
MAX GPU
UTILIZATION
DDN PARALLEL ARCHITECTURE AND PROTOCOL vs NFS
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
34
DDN Storage | © 2019 DDN Storage 34
DDN A3I SOLUTIONS PERFORMANCE ENHANCEMENTS
AI200 and AI400
Optimizations for AI
Performance improvements for
Random Read up delivers up to to
around 750K IOPs per RU for AI400
and over 22GB/s per RU
0
10
20
30
40
50
60
0
200
400
600
800
1000
1200
1400
1600
1800
4KB 8KB 16KB 32KB 64KB 128KB
Thro
uh
gpu
t (G
B/s
ec)
IOP
S (K
op
s/se
c)
IO Size(Byte)
AI200 and AI400 (Random Read IOPS and Throughput)
AI200-Throughput
AI400-Throughput
AI200-IOPS
AI400-IOPS
© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
35
DDN Storage | © 2019 DDN Storage
Tokyu Bancho Bldg. 8F
6-2 Yonbancho Chiyoda-ku,
Tokyo 102-0081
+81-3-3261-9101
+81-3-3261-9140
company/datadirect-networks
@ddn_limitless
Thank You!Keep in touch with us