www.it-novum.com© it-novum GmbH
Seite 1
Building a flexible and large-scale software-defined storage platform for OpenStack
OPENSTACK DAY ItalyMay 30th 2014, Milan
Michael Kienle, CEOit-novum GmbH
www.it-novum.com© it-novum GmbH
Seite 2
Source: RED HAT
Open Clouds are picking up right now!
www.it-novum.com© it-novum GmbH
Seite 3
Open Stack: Open cloud operating system supported by theOpenStack Foundation backed up by ### well-known companies
Want to learn more: listen to the other tracks
Major Design Guidelines are OPEN:Open source (apache), Open design (blueprints), Open development(branches), Open community (tech leads)
Although OpenStack is purposely open it partly relies on underlying proprietary storage products.
To build your cloud upon a truly OPEN software-defined storage platform (SDS) you might achieve greater flexibility and $$$$-savings.
openATTIC is a SDS cloud storage platform optimized for OpenStack'sCinder component.
What is this all about?
www.it-novum.com© it-novum GmbH
Seite 4
Introduction & AgendaOpenStack
Focus on Storage within OpenStack, esp. CinderRequirements & Optimization of current approaches
SDS – software-defined storageHow it worksSDS as part of the software-defined datacenter (SDDC)Why now?Advantages
openATTICHow to bring SDS and OpenStack togetherVideo/Examples
Agenda
www.it-novum.com© it-novum GmbH
Seite 5
Leading provider of open source business solutions75 employees currently; approx. 20% growth p.a.Headquarters in Fulda/DE; Branch office in Vienna/ATHundreds of mid-sized and large corporate clientsOperating public listed parent company‘s data center for 20+ yearsand migrated most of this to open source
Mission: Help customers to build and run a 100% open-source software-defined datacenter
Proven Expertise in Business Open Source Infrastructure and ApplicationsSAP Extended Business Partner (for integrating SAP into open sourcesolutions)
Open Business Solutions. Delivered!
www.it-novum.com© it-novum GmbH
Seite 6
Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, commonAPIs
Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.)
Support the whole IaaS stack (Storage, Security, Networking etc.)
Scale far beyond the limitations of an „old virtualized landscape“
Enable the software-defined datacenter (SDDC)
OpenStack: Value Proposition
www.it-novum.com© it-novum GmbH
Seite 7
Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, commonAPIs
Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.)
Support the whole IaaS stack (Storage, Security, Networking etc.)
Scale far beyond the limitations of an „old virtualized landscape“
Enable the software-defined datacenter (SDDC)
OpenStack: Value Proposition
www.it-novum.com© it-novum GmbH
Seite 8
Storage costs optimization ($$$$)Data growth of >>40% p.a. needs to be managed automatically and waycheaper
Flexibility and reduced vendor lock-in (open)Incumbent players are perfect in securing their huge marginsUsing innovative technology without renewing existing assets
Cloud Computing has characterized the next generation IT delivery model, easy orchestration/automation of (storage) services is essential
need scale-up/scale-out of underlying storage at lesser costsSoftware-Defined DataCenter (SDDC) becomes a major disruptive IT trend(SDS beeing part of this)
Standardisation enables automation & abstraction of software from hardware-> Commodity hardware and SDS gained strength
Customers‘ Perspective on Storage
www.it-novum.com© it-novum GmbH
Seite 9
Cinder / Block Storage Swift / Object StorageObjectives • Storage for running VM disk
volumes on a host• Ideal for performance
sensititve apps
• Ideal for cost effective,scale-out storage
• Backup & Archive• Dropbox-like services
Use Cases • Production Applications• Traditional IT Systems• Database-driven Apps• Dev-/Test Landscape
• ISO Image- and VM-Templates
• Snapshots• Backup & Archive• Repository
Workloads • High chance content• Smaller, Random R/W• Higher / Bursty IO
• More static content• Larger, sequential r/w• Lower IOPS
Storage in OpenStack
www.it-novum.com© it-novum GmbH
Seite 10
Architecture to provide block-level resources to OpenStack („block storage asa service“)Persistent block-level storage volumes for use with OpenStack Nova compute instances (e.g. iSCSI target)Manages creation, attach/detach, clone of these storage volumensIntention of Cinder: keep you from beeing locked-in:
Ability to change the backend (the storage system) easilyMigrate data & expect the same base level of functionality
Many vendor/storage plug-ins for Cinder available;the choice between functionality and $$$$
Cinder - Details
LegendPresistent volume controlPresistent volume data
VM instance
Note that iSCSI isjust an example –several additional
protocols aresupported (e.g.,
FC, NFS)
Nova
/dev/vda
iSCSIinitiator
Cinder
KVM
iSCSItarget
www.it-novum.com© it-novum GmbH
Seite 11
Limitations of Commodity Storage (like a simple Linux server orcomparable) because of lacking features like:
node redundancy & clustering & HA, scalingQoS, Deduplication/Compression, ReplicationExample: Image Cloning (e.g. image from Glance) could be fasterthan block-based copy&paste…
Limitations of Enterprise Storage (the big names):Vendor lock-in$$$$$
Consider Open Source & SDS !
Choose your underlying storage system!
www.it-novum.com© it-novum GmbH
Seite 12
SDS is the abstraction of the software from the hardware.
Benefits:Independence from big-name commercial developer of proprietarysystems (bundle of hardware and storage os) for better pricing andfreedom of choiceHardware is treated as a pool, scalable across multiple storagesystemsInnovative functionality (e.g. top-down policy driven), easier toadministrate, faster to implement (software can be extended andmaintained more easily than hardware)Part of the Software-defined DataCenter (SDDC)
By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner, “Tech Go-to-Market: Storage Providers Should Focus on Three Growth Opportunities Through 2016”, Nov 2013), by 2018 OpenSource Storage will gain 20% of the market , approx. 10$B (Gartner, “2014 Strategic Roadmap for Storage”, March 2014).
Software-defined Storage (SDS)
www.it-novum.com© it-novum GmbH
Seite 13
ConfigurationManagement
System (CMS)
Service Transition Service OperationService Design
Monitoring
Event Correlation
User
Oth
erM
onito
rs
CapacityManagement
RiskManagement
Cloud Self Service
Sys
tem
App
licat
ion
End
-2-E
nd
Proc
ess
ChangeManagement
ProblemManagement
SAP
ApplicationsService DeskNetwork Operation Center
ReleaseManagement
Cloud Management
IncidentManagement
Infrastructure
Software-defined Storage Virtualization
CMDB
Discovery
Rep
ortin
gSe
rvic
e Le
vel M
anag
emen
t
The Big Picture: SDDC with 100% open sourceThe Big Picture: SDDC with 100% open source
www.it-novum.com© it-novum GmbH
Seite 14
ITSMi-doit & OTRS
Umbrella MonitoringopenITCOCKPIT
Cloud ManagementOpenStack
ReportingPentaho
Billing, Backup,...SAP, SEP sesam
ConfigurationDeployment
Services & Tools
API
Web
CLI
Apps
StatusChecking
ApplicationManagement
AccessSamba, NFS, AFP, iSCSI, FC
AvailabilityDRBD, Pacemaker,
Corosync, ...
EfficiencyZFS, LVM, BTFRS, XFS
BackupSnapshotting, Tape-
Backup, ...
AnalysisopenITCOCKPIT,
Nagios, ...
Back end
Core
Front end
openATTIC Framework - Architecture
Interfaces
www.it-novum.com© it-novum GmbH
Seite 15
MirroringBlock- & File-basedSynchronous & asynchronousObject-based
ClusteringAutomated Failover Multi Node Support
Support & ServicesTraining & OptimizationIntegration & Development Guaranteed Response Time Guaranteed Restore Time(Appliance only)
ProtectionIntegration Tape-SystemsSnapshottingConsistent Snapshotting
EfficiencyDeduplicationCompressionPerformance Config-Set Config-Set for Virtualization
Unified and versatileSAN / Block Storage (FC, iSCSI, FCoE)NAS / File-based (CIFS, NFS)Object Storage
Central ConfigurationWeb FrontendCommand Line Interface
Dynamic MonitoringAvailabilityPerformanceCapacity
Cloud SupportOpenStack/CinderopenQRM
Umbrella ManagementMaster/Slave Architecture
Current FeaturesOpen API, Scalable, Cinder-ready
www.it-novum.com© it-novum GmbH
Seite 16
Current FeaturesBenchmarkExample: Read/write performance using ZFS Pool
www.it-novum.com© it-novum GmbH
Seite 17
All access protocols, block (FC,iSCSI, FCoE) and files storage systems (CIFS, NFS) in every single openATTIC systemIn its latest definition Unified Storage contains object storageas well (Ceph, Swift). Object storage architecture manages data as objects and allows inexpensive, scalable and self-healing retention of massive amounts of unstructured DataopenATTIC can use Ceph for Object Storage and enables the following
Extending storage capacity and performance seamlessly on demand (multi-node scalibility)No upfront investment costs for eventual higher requirements in the futureScale out capacity and performance without migrate the existing infrastructure
Unified StorageExpample: Scale out with Object Storage
www.it-novum.com© it-novum GmbH
Seite 18
More Cloud Support/Integration: openstack Swift, CloudStack, Eucalyptus…QoS for top-down orchestration of services and policy enforcementBusiness-aware/application-aware SnapAppsSnapLock: Permanent and compliant long-term storage for meetingregulatory requirementsFlash/SSD Support for AppliancesBetter Security Hardening and Performance Optimization forAppliancesBetter support for easy scale-up/scale-out Deeper Integration of Big Data solutions: Hadoop, SparkEye-Candy GUI front-end Better documentation and more video tutorials….
Product Roadmap
www.it-novum.com© it-novum GmbH
Seite 19
Volume create/deleteSnapshot create/deleteVolume attach/detach (called by Nova)Create Volume from SnapshotCopy Image to Volume / Volume to ImageGet Volume Stats, Clone VolumeExtend Volume (mandatory since Icehouse)
openATTIC Cinder plug-in: VideoWe show you some examples how this works…
How it works: Cinder API (Icehouse)
www.it-novum.com© it-novum GmbH
Seite 20
Unit testsCinder driver certification fileRelease hopefully next one…
More Info: www.openattic.org/enIntegration in other cloud platforms has been done already
OpenStack Cinder / openAttic: Status
www.it-novum.com© it-novum GmbH
Seite 21
Open and comprehensive API achivies great flexibility forintegration into different cloud platformsOptimization of huge data through intelligent openATTICfunctionalities
Consistent snapshots relieve backup, accelerate desaster recoveryand reduce downtimesDeduplication, Compression, efficient mirroring and redundancy
Huge cost savings:openATTIC runs on commodity hardwarereducing vendor lock-in Open Source approach saves license costs
Are there limitations of software-defined storage / software-defineddatacenter?
Summary: openATTIC within OpenStack
www.it-novum.com© it-novum GmbH
Seite 22
Open and comprehensive API achivies great flexibility forintegration into different cloud platformsOptimization of huge data through intelligent openATTICfunctionalities
Consistent snapshots relieve backup, accelerate desaster recoveryand reduce downtimesDeduplication, Compression, efficient mirroring and redundancy
Huge cost savings:openATTIC runs on commodity hardwarereducing vendor lock-in Open Source approach saves license costs
However, the bad news is: even software-defined datacenters still need some underlying hardware…
Summary: openATTIC within OpenStack
www.it-novum.com© it-novum GmbH
Seite 23
Thank you!Questions? - More than welcome!
Cloud Storage
Intelligent SDS
Unified Storage
Power your cloudPower your cloud Rock-solid storageRock-solid storage Scale out data center Scale out data center
Power your cloud & scale out your data center with rock-solid Software-Defined Storage (SDS) truly on open source, join the community & more
about the openATTIC project at http://openattic.org/en
Power your cloud & scale out your data center with rock-solid Software-Defined Storage (SDS) truly on open source, join the community & more
about the openATTIC project at http://openattic.org/en