Introduce Whamcloud Lustre Product – Integrated Manager for Lustre

WanHee, Kim | kimwh@isbc.co.kr | ISBC Inc 2020

1

2020-03-06

• Lustre* is an object based, open source, distributed, parallel, clustered file system– Runs externally from compute cluster

– Accessed by clients over network (Ethernet, InfiniBand)

– Up to 512 PB file system size, 32 PB per file

– Production file systems have exceeded 2TB/sec

• Designed for maximum performance at massive scale

• POSIX compliant

• Global, shared name space

• All clients can access all data

• Very resource efficient and cost effective

ISBC Korea2

What is Lustre*

ISBC Korea3

HISTORY of lustre , A journey into innovation and freedom

2001

1.0 1.6

1.8 2.0

2012

2.1

201520032007

20092010 2011

2013

2.3 2.5 2.7

1999

Lustre was born as a research project by Peter Braamat CMU

2018

2.10

2019

2.12

ISBC Korea4

Lustre User Community Group Survey - Usage

ISBC Korea5

Lustre Version Roadmap

2018 2019 2020 2021

4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q

Feature Release

LTSUpdate

2.12.0 2.13.02.14

2.15

2.10.x LTS (IML 4.x)

2.12.x LTS (IML 5.x)

2.10.6 2.10.7 2.10.8

2.12.1 2.12.2 2.12.3 2.12.4

2.12

- Lazy size on MDT- LNET Health- DNE Dir Restriping- Improve ZFS HA- Support EL8 Client

2.13

- Persistent Client Cache- Multi-Rail Routing- OverStriping- File Heat

2.14 (PLAN – TBD)

- FLR Erasure Coding- Pool Quota- DNE Auto Restriping- Enhance HSM

2.15 (PLAN – TBD)

- Client Encryption- Writeback Cache (LU-10938)- Adaptive Compression

2.14.x LTS (IML 6.x)

2.12.0

2.14.0

ISBC Korea6

Whamcloud Lustre VersionEnterprise HPC, commercial technical computing, and analytics

Whamcloud OS Lustre

V5.0.0.0 RHEL/CentOS7.6 V2.12.1

V4.0.10.2 RHEL/CentOS7.6 V2.10.8.0

V4.0.10.1 RHEL/CentOS7.6 V2.10.7.0

V3.1.1.0 RHEL/CentOS7.3 V2.7.19.10

V2.4.2.5 RHEL/CentOS6.8 V2.5.42.23

V2.2.0.2 RHEL/CentOS6.6 V2.5.34

V2.0.1.2 RHEL/CentOS6.5 V2.5.29

ISBC Korea7

Integrated Manager for Lustre - IML

ISBC Korea8

LLNL Sequoia Lustre Architecture

OSS 서버 2대에, 12G SAS HBA 4장 구성

최소 구성

ISBC Korea9

Performance Sample Design #1

IML

Lustre Clients

MetadataServers

Object StorageServers

AD ServerCIFS GW

Old Storage Write 4~5 GB/sec

New StorageWrite 6~8 GB/sec

4 OSS , 12G SAS HBA 8EA Design

OSS 2 + Storage Group Extension , Use Old and New Storage with Mixed Design

ISBC Korea10

Performance Sample Design #2

IML

Lustre Clients

MetadataServers

Object StorageServers

Object StorageServers

AD ServerCIFS GW

Old Storage Write 6~8 GB/sec

New StorageWrite 12 ~ 16GB/sec

ISBC Korea11

IML 4.10 Screenshot (Based on CentOS7.6)

스케쥴러와 연동 하여 모니터링 가능

스케쥴러에 등록된 JOB ID를 기준으로수행되는 동안의 사용량 모니터링

수행이 끝난 Job 은 모두 0 으로 보입니다.

ISBC Korea12

ISBC Korea13

Key Customers in different markets are using Enterprise Lustre today