introduction to ceph and architectural overviewpeople.redhat.com/~flucifre/talks/blu - intro to ceph...
TRANSCRIPT
![Page 1: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/1.jpg)
Introduction to Ceph and Architectural OverviewFederico LucifrediProduct Management Director, Ceph StorageBoston, December 16th, 2015
![Page 2: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/2.jpg)
2
CLOUD SERVICES
COMPUTE NETWORK STORAGE
the future of storage™
![Page 3: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/3.jpg)
3
HUMANHUMAN COMPUTERCOMPUTER TAPETAPE
HUMANHUMAN ROCKROCK
HUMANHUMAN
INKINK
PAPERPAPER
![Page 4: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/4.jpg)
4
HUMANHUMAN COMPUTERCOMPUTER TAPETAPE
![Page 5: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/5.jpg)
5
YOUYOU TECHNOLOGYTECHNOLOGY YOUR DATAYOUR DATA
![Page 6: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/6.jpg)
6
How Much Store Things All Human History?!writing
paper
computers
distributed storage
cloud computing
gaaaaaaaaahhhh!!!!!!
carving
![Page 7: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/7.jpg)
7
HUMANHUMAN COMPUTERCOMPUTER DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
HUMANHUMAN
HUMANHUMAN
![Page 8: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/8.jpg)
8
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
COMPUTERCOMPUTER
![Page 9: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/9.jpg)
9
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
DISKDISK
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMANHUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
GIANT SPENDY
COMPUTER
GIANT SPENDY
COMPUTER
![Page 10: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/10.jpg)
10
DISKDISKCOMPUTERCOMPUTER
HUMANHUMAN
HUMANHUMAN
HUMANHUMANDISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
![Page 11: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/11.jpg)
11
HUMANHUMAN
HUMANHUMAN
HUMANHUMAN
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
![Page 12: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/12.jpg)
12
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
“STORAGE APPLIANCE”
![Page 13: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/13.jpg)
Storage ApplianceMichael Moll, Wikipedia / CC BY-SA 2.0 13
![Page 14: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/14.jpg)
SUPPORT AND MAINTENANCESUPPORT AND MAINTENANCE
PROPRIETARY SOFTWARE
PROPRIETARY SOFTWARE
14
PROPRIETARY HARDWARE
PROPRIETARY HARDWARE
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
34% of revenue(5.2 billion dollars)
1.1 billion in R&DSpent in a year
1.6 million square feetof manufacturing space
![Page 15: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/15.jpg)
15
1010100110
1010110011
1001100101
1001101011
1001100111
1001010011
THE CLOUD
![Page 16: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/16.jpg)
SUPPORT AND MAINTENANCESUPPORT AND MAINTENANCE
PROPRIETARY SOFTWARE
PROPRIETARY SOFTWARE
16
PROPRIETARY HARDWARE
PROPRIETARY HARDWARE
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
STANDARD HARDWARESTANDARD HARDWARE
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
DISKDISKCOMPUTERCOMPUTER
OPEN SOURCE SOFTWARE
OPEN SOURCE SOFTWARE
ENTERPRISE SUBSCRIPTION
ENTERPRISE SUBSCRIPTION
(optional)
![Page 17: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/17.jpg)
17
![Page 18: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/18.jpg)
18
OPEN SOURCEOPEN SOURCE
COMMUNITY-FOCUSEDCOMMUNITY-FOCUSED
SCALABLESCALABLE
NO SINGLE POINT OF FAILURENO SINGLE POINT OF FAILURE
SOFTWARE BASEDSOFTWARE BASED
SELF-MANAGINGSELF-MANAGING
philosophy design
![Page 19: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/19.jpg)
19
8 years & 20,000 commits later…
![Page 20: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/20.jpg)
20
![Page 21: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/21.jpg)
21
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
![Page 22: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/22.jpg)
22
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
![Page 23: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/23.jpg)
23
DISKDISK
FSFS
DISKDISK DISKDISK
OSDOSD
DISKDISK DISKDISK
OSDOSD OSDOSD OSDOSD OSDOSD
FSFS FSFS FSFSFSFS btrfsxfsext4
MMMMMM
![Page 24: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/24.jpg)
24
MM
MM
MM
HUMANHUMAN
![Page 25: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/25.jpg)
25
Monitors:• Maintain cluster membership
and state• Provide consensus for
distributed decision-making• Small, odd number• These do not serve stored
objects to clients
MM
OSDs:• 10s to 10000s in a cluster• One per disk• (or one per SSD, RAID group…)
• Serve stored objects to clients• Intelligently peer to perform
replication and recovery tasks
![Page 26: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/26.jpg)
26
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
![Page 27: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/27.jpg)
LIBRADOSLIBRADOS
MM
MM
MM
27
APPAPP
socket
![Page 28: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/28.jpg)
LLLIBRADOS• Provides direct access to
RADOS for applications• C, C++, Python, PHP, Java,
Erlang• Direct access to storage nodes• No HTTP overhead
![Page 29: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/29.jpg)
29
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
![Page 30: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/30.jpg)
30
MM
MM
MM
LIBRADOSLIBRADOS
RADOSGWRADOSGW
APPAPP
socket
REST
![Page 31: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/31.jpg)
31
RADOS Gateway:• REST-based object storage
proxy• Uses RADOS to store objects• API supports buckets,
accounts• Usage accounting for billing• Compatible with S3 and
Swift applications
![Page 32: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/32.jpg)
32
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
![Page 33: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/33.jpg)
33
MM
MM
MM
VMVM
LIBRADOSLIBRADOSLIBRBDLIBRBD
VIRTUALIZATION CONTAINERVIRTUALIZATION CONTAINER
![Page 34: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/34.jpg)
LIBRADOSLIBRADOS
34
MM
MM
MM
LIBRBDLIBRBD
CONTAINERCONTAINER
LIBRADOSLIBRADOSLIBRBDLIBRBD
CONTAINERCONTAINERVMVM
![Page 35: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/35.jpg)
LIBRADOSLIBRADOS
35
MM
MM
MM
KRBD (KERNEL MODULE)KRBD (KERNEL MODULE)
HOSTHOST
![Page 36: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/36.jpg)
36
RADOS Block Device:• Storage of disk images in
RADOS• Decouples VMs from host• Images are striped across the
cluster (pool)• Snapshots• Copy-on-write clones• Support in:• Mainline Linux Kernel (2.6.39+)• Qemu/KVM• OpenStack, CloudStack
![Page 37: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/37.jpg)
37
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
LIBRADOS
A library allowingapps to directlyaccess RADOS,with support forC, C++, Java,Python, Ruby,and PHP
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
RBD
A reliable and fully-distributed block device, with a Linux kernel client and a QEMU/KVM driver
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
CEPH FS
A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
RADOSGW
A bucket-based REST gateway, compatible with S3 and Swift
APPAPP APPAPP HOST/VMHOST/VM CLIENTCLIENT
![Page 38: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/38.jpg)
38
MM
MM
MM
CLIENTCLIENT
01100110
datametadata
![Page 39: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/39.jpg)
39
Metadata Server• Manages metadata for a POSIX-
compliant shared filesystem• Directory hierarchy• File metadata (owner,
timestamps, mode, etc.)
• Stores metadata in RADOS• Does not serve file data to
clients• Only required for shared
filesystem
![Page 40: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/40.jpg)
What Makes Ceph Unique?Part one: CRUSH
40
![Page 41: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/41.jpg)
41
APPAPP??
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
![Page 42: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/42.jpg)
How Long Did It Take You To Find Your Keys This Morning?azmeen, Flickr / CC BY 2.0 42
![Page 43: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/43.jpg)
43
APPAPP
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
![Page 44: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/44.jpg)
Dear Diary: Today I Put My Keys on the Kitchen CounterBarnaby, Flickr / CC BY 2.0 44
![Page 45: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/45.jpg)
45
APPAPP
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
DDCC
A-G
H-N
O-T
U-Z
F*F*
![Page 46: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/46.jpg)
I Always Put My Keys on the Hook By the Doorvitamindave, Flickr / CC BY 2.0 46
![Page 47: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/47.jpg)
HOW DO YOUFIND YOUR KEYS
WHEN YOUR HOUSEIS
INFINITELY BIGAND
ALWAYS CHANGING?
47
![Page 48: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/48.jpg)
The Answer: CRUSH!!!!!pasukaru76, Flickr / CC SA 2.0 48
![Page 49: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/49.jpg)
49
10 10 01 01 10 10 01 11 01 1010 10 01 01 10 10 01 11 01 10
1010 1010 0101 0101 1010 1010 0101 1111 0101 1010
hash(object name) % num pg
CRUSH(pg, cluster state, rule set)
![Page 50: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/50.jpg)
50
10 10 01 01 10 10 01 11 01 1010 10 01 01 10 10 01 11 01 10
1010 1010 0101 0101 1010 1010 0101 1111 0101 1010
![Page 51: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/51.jpg)
51
CRUSH• Pseudo-random placement
algorithm• Fast calculation, no lookup• Repeatable, deterministic
• Statistically uniform distribution• Stable mapping• Limited data migration on change
• Rule-based configuration• Infrastructure topology aware• Adjustable replication• Weighting
![Page 52: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/52.jpg)
52
CLIENTCLIENT
??
![Page 53: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/53.jpg)
53
NAME: "foo"POOL: "bar"
0101 11111001 00111010 11010011 1011 "bar" = 3
hash("foo") % 256 = 0x23
OBJECT PLACEMENT GROUP
243
12
CRUSH TARGET OSDsPLACEMENT GROUP
3.23
3.23
![Page 54: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/54.jpg)
54
![Page 55: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/55.jpg)
55
![Page 56: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/56.jpg)
56
CLIENTCLIENT
??
![Page 57: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/57.jpg)
What Makes Ceph UniquePart two: thin provisioning
57
![Page 58: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/58.jpg)
LIBRADOSLIBRADOS
58
MM
MM
MM
VMVM
LIBRBDLIBRBD
VIRTUALIZATION CONTAINERVIRTUALIZATION CONTAINER
![Page 59: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/59.jpg)
HOW DO YOUSPIN UP
THOUSANDS OF VMsINSTANTLY
ANDEFFICIENTLY?
59
![Page 60: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/60.jpg)
144144
60
00 00 00 00
instant copy
= 144
![Page 61: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/61.jpg)
44144144
61
CLIENTCLIENT
write
write
write
= 148
write
![Page 62: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/62.jpg)
44144144
62
CLIENTCLIENTread
read
read
= 148
![Page 63: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/63.jpg)
What Makes Ceph Unique?Part three: clustered metadata
63
![Page 64: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/64.jpg)
POSIX Filesystem MetadataBarnaby, Flickr / CC BY 2.0 64
![Page 65: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/65.jpg)
65
MM
MM
MM
CLIENTCLIENT
01100110
![Page 66: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/66.jpg)
66
MM
MM
MM
![Page 67: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/67.jpg)
67
one tree
three metadata servers
??
![Page 68: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/68.jpg)
68
![Page 69: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/69.jpg)
69
![Page 70: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/70.jpg)
70
![Page 71: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/71.jpg)
71
![Page 72: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/72.jpg)
72
DYNAMIC SUBTREE PARTITIONING
![Page 73: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/73.jpg)
Getting Started With Ceph
Read about the latest version of Ceph.• The latest stuff is always at http://ceph.com/get
Deploy a test cluster using ceph-deploy.• Read the quick-start guide at http://ceph.com/qsg
Deploy a test cluster on the AWS free-tier using Juju.• Read the guide at http://ceph.com/juju
Read the rest of the docs!• Find docs for the latest release at http://ceph.com/docs
73
Have a working cluster up quickly.
![Page 74: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/74.jpg)
Getting Involved With Ceph
Most project discussion happens on the mailing list.• Join or view archives at http://ceph.com/list
IRC is a great place to get help (or help others!)• Find details and historical logs at http://ceph.com/irc
The tracker manages our bugs and feature requests.• Register and start looking around at http://ceph.com/tracker
Doc updates and suggestions are always welcome.• Learn how to contribute docs at http://ceph.com/docwriting
74
Help build the best storage system around!
![Page 75: Introduction to Ceph and Architectural Overviewpeople.redhat.com/~flucifre/talks/BLU - Intro to Ceph and... · 2016-03-08 · Introduction to Ceph and Architectural Overview Federico](https://reader036.vdocument.in/reader036/viewer/2022062915/5e97363ca8a9ab01ab22aeb9/html5/thumbnails/75.jpg)
Ceph Hammer (v0.94.x)
1. Rados Performance enhancements: All Flash environments2. Simplified RGW deployment3. RGW Object Versioning and Bucket Sharding4. RBD Mandatory Locking, Object Maps, Copy on Read5. CephFS Snapshot improvements
and many more. See https://ceph.com/releases/v0-94-hammer-released/
75
Best Ceph ever.