opennebula at bit.nl
DESCRIPTION
How OpenNebula is being used at BIT.nl.TRANSCRIPT
OpenNebula @BITStefan Kooman (UNIX engineer)
(A) BIT (MORE)
● BIT is a business to business internet service provider specialized in colocation and managed hosting
● BIT delivers to quality aware customers the backbone of their IT and internet infrastructure
● Reliability is the focus of BIT’s services (redundancy is keyword (n+1))
● No costs upfront for startups (no need to buy iron, support contracts, hardware management, etc.)
● Short timetodeliver● Flexible (grow / shrink storage, VM capacity, # VM's)● Infrastructure as a Service (IaaS)
Why choose “BIT” instead of $PUBLIC_CLOUD?● Customers want to know were their data gets stored (compliancy)● Alternative for shared hosting (customers with special requirements)● Hybrid Solutions: bare metal & BIT VM's possible● ISO 27001
Why we do have a “cloud” @ BIT
Project Requirements
● Remote console / control (stop, start, reboot)● Customer should be able to change VM
capacity (resource pool aka Virtual DataCenter)● VM speed should be on par with bare metal● API (provisioning, billing, integration)
Why did we choose OpenNebula?
● Simple but powerful / flexible● Works out of the box● Easy to maintain / upgrade● (API) Interface(s)● OSS● Great community / development organization (C12G)
Customer Cloud setup @BIT
● Ubuntu with KVM● Openvswitch (ready for SDN)● Cisco Switches● NetApp MetroCluster Storage (NFS)● Fujitsu servers
Customer Cloud setup @BIT
● Oned: Active – standby “OpenNebula” (manual failover) located in seperate DataCenters
(mysql mastermaster repl., Pacemaker/Corosync)● Apache with Phusion Passenger (Ruby app server)● Loadbalanced “frontend” (Sunstone, OCCI)● Cfengine2 configuration Management
(Particular) Nice OpenNebula Features
● Hooks● Multiple Datastores functionality● Requirements on VM provisioning (filtering)● Virtual Data Centers (VDC) in ONE 4.6
What do we use hooks for?
● XMLRPC call to BIT Administration & Configuration Engine (BACE)
● Live migrate VM's (on hypervisors) as well as in BACE (nearrealtime)
Multiple Datastores
● NetApp Qtree (NFS)
→ Provide seperation (“partitioning”)
→ Billing (IOPS / Disk Space)
→ Tiering possible (SAS, SSD/SATA)● Future?
→ Distributed Object Storage (CEPH, Gluster)
Multiple datastores (continued)
Datastore seperation (security, performance)● Production datastores @filer1, volume1
→ Customer A
→ Customer B● Test / Development @filer2, volume2
→ Test VM's
Provisioning Requirements (filtering)
● DATASTORES● CLUSTERS● HOSTS
→ Custom Attributes (awesome \o/)
Examples:SCHED_REQUIREMENTS="WINDOWSLICENSED=\"TRUE\""
SCHED_REQUIREMENTS="DATACENTER=\”BIT1\”"
SCHED_DS_REQUIREMENTS="NAME=system_ds_1_kvm_cluster"
Virtual Data Center
● Virtual Resource abstraction● Pool of resources, flexibility for endusers● Possibilities for Resellers
Integration with BIT: provisioning
Command Line Tool● Vmadm
For now funtcionality is limited:
→ stop, start, reboot, list
Future:
→ full fledged provisioning:
create, destroy, clone, resize capacity, etc.
Integration with BIT: provisioning
Command Line Tool (interfaces with OCCI)● Vmadm
For now only:
→ stop, start, reboot, list
Future:
→ full fledged provisioning
create, destroy, clone, resize capacity, etc.
Integration with BIT: Portal
Customer Portal Interface to securely manage all services (DNS, MAIL, VM's, MONITORING, BILLING, etc.)
● For now only:
→ stop, start, reboot
→ Out of band management: Console access (KVM)
Future:
→ full fledged provisioning
create, destroy, clone, resize capacity, etc.
Integration with BIT: Developer perspective
So far, all good news!!!● Good documentation (and lot's of it)● Hooks (again :))
Virtualization @BIT (currently)
Currently● Ad hoc managed KVM hypervisors (virt
manager, virsh)● VM creation process relatively labour intensive● Live migration of VM's takes a long time (but
could be scripted / automated)● No Integration with BACE
Virtualization @BIT (Future)
OpenNebula Cloud dedicated for BIT services
Benefits:● Integration with BACE● New features available to do cool stuff
→ OneFlow
→ CloudFederation (BIT3, Partner DC)
→ Internal AppMarket (ready to go BIT modified vms to play / test /develop)
→ CCS / R&D self service (no need to ask the UNIX engineers)
Virtualization @BIT (Future)
OpenNebula Cloud dedicated for BIT services
Benefits:● Integration with BACE● New features available to do cool stuff
→ OneFlow
→ CloudFederation (BIT3, Partner DC)
→ Internal AppMarket (ready to go BIT modified vms to play / test /develop)
→ CCS / R&D self service (no need to ask the UNIX engineers)
Challenges
● Network Integration (SDN) (ipallocations, vlan provisioning, routing, firewalling, loadbalancing, etc.)
● DNS automation (PTR's, Forwards)
→ Probably going to solve with hooks● Snapshot integration (NetApp)
BIT and OpenNebula
What you might expect from us● More feature requests ;)● More bug reports (hopefully not ;))● Support on mailinglist● Translate Sunstone into Dutch (to be completed in two
days from now)● Zerocopy driver functionality (NetApp)
→ faster provisioning, less host IO