Download - Ceph Day LA: Adventures in Ceph & ISCSI
ADVENTURES IN iSCSI
Paul Evansprincipal architect
daystrom technology grouppaul at daystrom dot com
los angelesceph day
SQUEEZING WAY-COOL STORAGE TECH INTO A LEGACY MODEL
+Tu Holmesstorage lead
electronic artstu at ea dot com
WHAT’S IN THIS TALK
• Why (add) iSCSI?
• the How of iSCSI + Ceph
• the Good & Bad
• Lessons (hopefully) Learned
CEPH: THE FUTURE OF STORAGEWHY SADDLE IT WITH OLD (SCSI) TECH?
Reasons for iSCSI
IncompatibilitiesVirtualization Databases
EA i SCS I REASONS…
✓ Kernel
(and Desire to use VolMgr)
Incompatibility
CE
PH
+ I
SC
SI
x12 x12 x12 x12 x12 x12
OSD Host MON Host iSCSI GTWY Host
Main Network
H H H H
DES IRED i SCS I FEATURES
Maximum Throughput
Good Support
HA Options
CEPH+ iSCS I OPT IONSLIO
(targetcli) SCST TGT iET
Maintainer Datera (rt)Red Hat (fb) SCST Ltd. Community
(FUJITA Tomonori) Community
Lastest Stable 2.1.0 (rt)2.1.fb41(fb) 3.0.1 1.0.60 1.4.20
Latest Commit
Jan 2015 (rt)Jun 2015 (fb) June 2015 Jul 2015 Jun 2014
Next Release 3.0 (?) 3.1 ? ?
Mainline Kernel Yes No No No
Supppor ted Fea tu re s
LIO SCST STGT iET
Kernel Yes Yes User Space Split
RBD Yes Yes Yes No
iSER Yes Pre-Release Yes No
SRP (IB) Yes Yes No No
ALUA Yes Yes No No
EA
CL
US
TE
R
CU
RR
EN
TOSD Service
MON Service
iSCSI GTWY Service
H StorNext Host
StorNext Backup HostVol Mgr
x12 x12 x12 x12 x12 x12 x12 x12
H H H HH H
EA
CL
US
TE
R
PO
OL
CO
NFI
GCEPH
iSCSI
VolMgrBACKUP HOST
ISCSI GATEWAYS
CEPH CLUSTER
EC Tier
Cache Tier
RBD RBD RBD RBD RBD RBD
HW
SP
EC
S
SCALE-OUT CLUSTER OF STORAGEOBJECT+FILE+BLOCK ACCESS
HIGH AVAILABILITYSELF-HEALING
SIM
PLE
LOW
-CO
ST
SCAL
E-O
UT S
TORA
GE
THE SIMPLE SOLUTION FOR THE STORAGE OF EVERYTHING
The storageFOUNDRY Nautilus system is the smart choice for organizations with diverse storage needs and explosive data growth. Nautilus is a massively scalable, open-source storage solution developed from the ground-up to provide object, block and file-system access in one self-managing, self-healing platform. Using a reliable non-stop architecture, this multi-streamed approach to storage grids is the flexible solution for Data Automation, High-Throughput Media or Analytics, and OpenStack Clouds. The Nautilus C100 series are fast and dense 1U storage nodes for delivering from 48TB to 120TB in a single rack unit, with a low-service architecture to minimize in-field support needs. The C100 nodes provide a range of protocols and speeds, from 1G Ethernet to 56G InfiniBand, enabling it to create stand-alone C-series Cloud deployments, or drop into applicationtuned roles for 4K-Media, GS-Genomics and E-series deployments. Nautilus answers the call for multi-protocol, cost-effective and scalable storage that serves up classic Enterprise Storage I/O and well as innovative direct-access data, making it the simple choice for today’s unpredictable computing needs and THE STORAGE OF EVERYTHING.
NFS SMB FTP HTTP iSCSI 1/10/40G ETH 56G InfiniBand 8G/16G FC
foundrystorage
C100/C140/C150SCALE-OUT STORAGE NODES
5600NS
NAUTILUS
NAUTILUSobject storage
8x C100-48T 1U Nodes
• 12 x 4TB SATA HDD• 1 x 400G SATA SSD• 1 x E3-1200 4Core CPU
• 32G ECC DRAM• 1 x 10GE / 4 x 1GE
foun
dry
stora
ge
www.storageFOUNDRY.net
Nautilus C100-G3Nautilus C300-G3
Nautilus 4K
GO
OD
& B
AD
✓ Up to 1GB/sec Burst✓ 600MB Sustained WritePerformance
Scalability
Availability
ReliabilityB
✓ Add cluster (Ceph) storage on-the-fly✓ Dynamically Expand/Add LUNs
✓ Ceph is naturally Highly Available✓ iSCSI HA via Active/Passive Gateways
✓ Current config: resource constrained✓ Difficult to recover from LIO faults
RE
LIA
BIL
ITY
✓ Limit Scrubbing to Low-Load Times
✓ Limit Background Maintenance/Repair
✓ Limit PGs to maximize resources
✓ Monitor/Track occurrences of Blocked IO
ReliabilityBTunings
EA
CL
US
TE
RNEXT-GEN
x12 x12 x12 x12 x12 x12 x12 x12
OSD Service
MON Service
iSCSI GTWY Service
H StorNext Host
StorNext Backup HostVol Mgr
H H H HH H
INT
O T
HE
FU
TU
RE
Active-Active (i)SCSI Support is comingI
➡ Updates to both LIO & Ceph occurring
➡ ‘Support’ for VMware / Persistent Reservations
➡ Timeline: 60-180 days… YMMV
Futures…
the Three Lessons Learned
Resource Management
Resource Management
Resource Management
It’s mostly about…
thank you!
Paul Evansprincipal architect
paul at daystrom dot com
technology grouptechnology group
Tu HolmesStorage Lead
tu at ea dot com