turning openstack swift into a vm storage platform
TRANSCRIPT
OpenStack Meetup: Turning OpenStack Swift into a VM storage platform
Wim Provoost (@wimpers_be)
Open vStorage (@openvstorage)
CloudFounders
• CloudFounders (2007)– 120 people in the US, Belgium, Romania and India
• Products– Open vStorage (open source under an Apache 2.0 license)
• Ultra reliable, high performance, open storage layer for virtual machines.
• Compatible with all major hypervisors.• Supports OpenStack
– vRun = Converged Infrastructure• Software that transforms server hardware to converged infrastructure
with unparalleled performance & flexibility.• Software can run on any x86 server or as a virtual machine
(VMware/OpenStack)• Uses Open vStorage as basis• Competes with Nutanix, Simplivity and upcoming EVO:RAIL
Object storage: the future of scale out storage
Object storage is the future for storing massive amounts of data:
• PRO– Build scale-out storage by adding more disk/servers when needed
– High reliability by distributing content across nodes
– Easy to manage (no knowledge needed about RAID or volumes)
– Standard API as all solutions support S3 API, no vendor lock-in
– Cost-effictive: use low cost, large capacity SATA disks, increase performance by adding spindles/gateways
• CON– NO support for Virtual Machine Disk Storage (see next slide)
Object storage challenges as VM storage
• Eventual consistency (the CAP Theorem)
• Latency & performance– VMs require low latency and high performance
– Object stores are developed to contain lots of data
(large disks, low performance)
– Additional latency as Object Store is on the Local LAN instead of attached to the host like DAS
• Different Management Paradigms– Object Stores understand Objects <> Hypervisors understand blocks, files
What is needed is a technology whereby Virtual Machines can use object stores instead of a SAN and get the benefits of the low cost
and scale-out capabilities of object stores
Traditional OpenStack Setup
NovaInstance
Management
SwiftObject Storage
CinderBlock Storage
GlanceImage store
VM
Provides
volume for
Provisions
Stores
image in
Stores backups in
Provides
image for
Ceph, SAN, NAS, ...
Provides
disk space
2 storage platforms?!
The answer: Open vStorage
NovaInstance
Management
SwiftObject Storage
GlanceImage store
VM
Provides
volume for
Provisions
Stores
image in
Stores backups in
Provides
image for
Open vStorage
CinderBlock Storage
Provides
disk space
Converts Object storage
into Block storage
What is Open vStorage
Open vStorage is an open source “Storage Router“ and is installed on a host or a cluster of hosts to create a VM-centric, clustered, reliable,
scale out and high performance storage system for virtual machines.
Open vStorage Intro
• +40.000 IOPS per Hypervisor
• Made for virtual machines
• Unified Namespace
• Ultra Reliable
• Unlimited Snapshots
• Endless Scalable
• Lowest Management Cost In Market
SAN / NASS3 compatibleObject Based
Storage
Hypervisor
Open v Storage
Hypervisor
Open v Storage
Hypervisor
Open v Storage
Hypervisor
Open v Storage
Distributed File Systems
Ultra Scalable Storage For Virtual Machines
KVM
Scale-outVM VM
VM VM
SSDSSD
OpenvStorage
KVM
VM VM
VM VM
SSDSSD
OpenvStorage
KVM
VM VM
VM VM
SSDSSD
OpenvStorage
Virtual File System interface
Solving Eventual Consistency using time based approach
SSD or PCI FlashLBA 1: 4k block 1
LBA 2: 4k block 2
LBA 3: 4k block 3
LBA 4: 4k block 4
LBA 5: 4k block 5
LBA 1: 4k block 6
LBA 1: 4k block 7
LBA 3: 4k block 8
LBA 6: 4k block 9
LBA 7: 4k block 10
LBA 8: 4k block 11
LBA 2: 4k block 12
LBA 9: 4k block 13
LBA 10: 4k block 14
New writes
SCO 1
4k block 1
4k block 2
4k block 3
4k block 4
4k block 5
4k block 6
4k block 7
SCO 2
4k block 8
4k block 9
4k block 10
4k block 11
4k block 12
4k block 13
4k block 14
LBA 5: 4k block 15
LBA 10: 4k block 16
4k block 15
4k block 16
SCO 3
New writes
SCO1
SCO2Transfer SCOs once they are full (4MB)to the Storage Backend at slow pace
Each write is appended
to the current Storage
Container Object (SCO)
Open vStorage <> distributed file system
VSA 1 VSA 2 VSA 3
Arakoon – (config params, metadata, ...)
vDisk1
vDisk2
InternalBucket
vDisk3
VFS2 VFS3
xml
VOLDRV
VM
VOLDRV
Object Router
FILEDRV
FILEDRV
VOLDRV
Object Router
FILEDRV
KVM1 KVM2 KVM3
VFS1
Object Router
Live Motion – In depth (Phase 1)
VSA 1 VSA 2 VSA 3
Arakoon – (config params, metadata, ...)
vDisk1
vDisk2
InternalBucket
vDisk3
VFS3
vmx
VOLDRV
VOLDRV
Object Router
FILEDRV
FILEDRV
VM
VOLDRV
Object Router
FILEDRV
KVM1 KVM2 KVM3
VFS1
VMLive Motion
Object Router
VFS2
Live Motion – In depth (Phase 2)
VSA 1 VSA 2 VSA 3
Arakoon – (config params, metadata, ...)
vDisk1
vDisk2
InternalBucket
vDisk3
VFS2 VFS3
xml
VOLDRV
VOLDRV
FILEDRV
FILEDRV
VM
VOLDRV
Object Router
FILEDRV
KVM1 KVM2 KVM3
VFS1
VMLive Motion
Object Router Handover Object Router
How does Open vStorage solve the problem
• Open vStorage is a middleware layer in between the hypervisor and the object store. (Converts object storage into block storage)– On the host: location based storage (block storage).
– On the backend: time based storage (ideal for objects stores).
– Open vStorage turns a volume into a single bucket.
• OpenStack Cinder Plugin for easy integration (snapshots, ...).
• Distributed file systems don’t work! Open vStorage is not a distributed file sysem!– All hosts ‘think’ they see the same virtual file systems.
– Volume is ‘live’ on 1 host instead of all hosts.
– Only the virtual file system metadata is distributed.
• Caching inside the host fixes impedance mismatch between slow, high latency backend and fast, low latency requirement of Virtual Machines.
The benefits
• Single distributed storage platform (Swift) to maintain. This platform is very performant and almost infinitely scalable.
• Better performance due to caching inside the host where the Nova instance runs.
• VM-centric management on storage level.
– Thin-cloning, zero-copy snapshots, ...
• Add storage which isn’t compatible with Cinder.
Supported OpenStack actions
• Volumes
– Add / Remove
– Create / Delete snapshot
– Clone from snapshot
– Clone volume from volume
– Import image to volume
– Export volume to image
• Instances
– Live Migration
– Evacuate
Deployments
• Our internal environments are now running Open vStorage + swift
• Open vStorage is available in beta
– Certified OpenStack Cinder plugin is expected with Kilo release
– Feel free to use in test environments and provide feedback
– Supported production release Q1
Next steps
• The code: https://bitbucket.org/openvstorage/openvstorage/
• Download the packages and deploy in your OpenStack environment
• Documentation: http://doc.openvstorage.com
• Support: https://groups.google.com/forum/#!forum/open-vstorage
• Follow us on Twitter (@OpenvStorage) or read our blog (http://blog.openvstorage.com)
OpenStack Paris
We will be there! Look out for the “Cloudfounders” booth.
November, 3-7