Virtual Machine Allocation in Current CloudComputing Middleware

GWDG eScience GroupGesellschaft fur wissenschaftliche Datenverarbeitung mbH Gottingen, Germany

AbstractThis is a technical report for the virtual machine(VM) allocation status in current cloud computing middleware.The middleware that we are going to research are Open CloudComputing Interface (OCCI), Opennebula (ONE) and Openstack(OSA). We concern 4 aspects in this report: 1, our requirements;2, our utility functions; 3, the data; 4, the algorithms formaximizing the utility functions.

Index TermsCloud computing, Virtual machine allocation,OCCI, ONE, OSA.

I. REQUIREMENTS

By considering the current cloud environment at GWDG,we are going to optimize our own utility functions and deploythe strategies into various cloud middleware we are using.

Least energy consumption. VM consolidatioin, less car-bon footprint, green computing.

Minimize the number of the active servers and VMs livemigrations, with performance constraints.

Reasonable resource utilization. Not over provisioned, notunder provisioned.

Load balancing among servers. Higher customer satisfaction, accepting rate. Avoid SLA violation, by monitoring, keep QoS guaran-

tees. higher service reliability. Higher profit.

II. ALGORITHMS IN OPENSTACK AND OPENNEBULA

The followings are the current ONE/OSA default VMallocation strategies.

A. OpenStack

OpenStack is updating its functionalities from Cactusto Diablo to the most recent fifth release Essex. Themost important features cover user UI, namely OpenStackDashboard, OpenStack Compute (Nova), OpenStack Ob-ject Storage (Swift), OpenStack Identity and OpenStackImage Service (Glance). There are many components, theycommunicate with each other through RabbitMessageQueueProtocol and RPC, for example Nova-api will process all theVM requests from the Queue. The communication with outsideis through HTTP. For VM scheduling, the most importantcomponent is nova-scheduler.

In OpenStack, The nova-scheduler maps the nova-API callsto the appropriate openstack components. It runs as a daemonnamed nova-schedule and picks up a compute/network/volumeserver from a pool of available resources depending uponthe scheduling algorithm in place. A scheduler can base its

decisions on various factors such as load, memory, physicaldistance of the availability zone, CPU architecture, etc. Sincethe data centers, on which the OpenStack is running, aredifferent from each other. Therefore, nova-schedule shouldbe customized according to factor in current state of theentire cloud infrastructure and apply complicated algorithm toensure efficient usage. To that end, nova-schedule implementsa pluggable architecture that lets you choose (or write) yourown algorithm for scheduling. The nova scheduler implementsa pluggable architecture. All schedulers should inherit fromclass Scheduler in driver.py.

Currently the nova-scheduler implements a few basicscheduling algorithms:

chance.py: inherits driver.Scheduler, in this method,a compute host is chosen randomly across availabilityzones.

simple.py, inherits chance.ChanceScheduler, hostswhose load is least are chosen to run the instance. Theload information may be fetched from a load balancer.

multi.py: inherits driver.Scheduler, a mapping of meth-ods to topics so we can figure out which driver to use.For compute, points to filter scheduler.FilterScheduler,for volume (binding extra storage to VM instance), pointto chance.ChanceScheduler.

filter scheduler, supports filtering and weighting to makeinformed decisions on where a new instance shouldbe created. This Scheduler supports only working withCompute Nodes [2]. See Figures 1, 2. First, the properhosts will be filtered and selected according to some pre-defined filters, e.g., AllHostFilter, AvailabilityZoneFilter,ComputeFilter, CoreFilter, IsolatedHostFilter, JsonFilter,RamFilter, SimpleCIDRAffinityFilter, DifferentHostFil-ter, SameHostFilter and so on. Different filters are de-signed for different purposes. For instance, JsonFilterprovides the opportunity to write complicated queries forthe hosts capabilities filtering, based on simple JSON-like syntax. When we get all the proper hosts, thenwe have to weight them and select the one with leastcosts. Therefore, if we want to define our own strategyfrom scratch, new filter and new cost functions areneeded.

manager.py as a scheduling manager has to set the defaultscheduling algorithm. In Nova2011.2 Version,

flags.DEFINE string(scheduler driver,nova.scheduler.chance.ChanceScheduler, Driver to use

Fig. 1. Hosts filtering

Fig. 2. Host weighting

for the scheduler)In Nova2011.3 Version,flags.DEFINE string(scheduler driver,

nova.scheduler.multi.MultiScheduler, Default driver touse for the scheduler)

In Nova2012.1 Version,flags.DEFINE string(scheduler driver,

nova.scheduler.multi.MultiScheduler, Default driver touse for the scheduler)

Therefore, there is no change in scheduling.

B. OCCI with OpenStack Features

OpenStack is one of the major players in the cloud com-munity at the moment. Still it currently lacks an standardizedInterface which can be used to manage VMs. This changesnow! During the Cloud Plugfest OCCI has been demoed ontop of OpenStack [4].

Status: General support for VM management is available. First set of tests available.

Aligned with the coding Standards of OpenStack. OCCI Compliant.Features: Deployment and Management of VMs and Volumes. Scaleup (Resize), Rebuild, Imaging of VMs. Integrated as a nova-wsgi WSGI service.At the moment, fetching infrastructure configuration

through OCCI is not supported and depending on how andwhat OCCI of OpenStack wants to achieve.

C. OpenNebulaIn [5], the Scheduler 3.4 module is in charge of the

assignment between pending VMs and known Hosts. Open-Nebulas architecture defines this module as a separate processthat can be started independently of oned. The OpenNebulascheduling framework is designed in a generic way, so it ishighly modifiable and can be easily replaced by third-partydevelopments.

Very similar to Openstacks filtering and weighting strate-gies. The Match-making Scheduler in OpenNebula comes witha match making scheduler (mm sched) that implements theRank Scheduling Policy. The goal of this policy is to prioritizethose resources more suitable for the VM.

The match-making algorithm works as follows: First those hosts that do not meet the VM requirements

(see the REQUIREMENTS attribute) and do not haveenough resources (available CPU and memory) to run theVM are filtered out.

The RANK expression is evaluated upon this list usingthe information gathered by the monitor drivers.

Any variable reported by the monitor driver can beincluded in the rank expression.

Those resources with a higher rank are used first toallocate VMs.

It is true that this scheduler algorithm easily allows theimplementation of several placement heuristics (see below)depending of the RANK expression used.

There are three default placement policies in ONE, namely:Packing Policy Target: Minimize the number of cluster nodes in use Heuristic: Pack the VMs in the cluster nodes to reduce

VM fragmentation Implementation: Use those nodes with more VMs running

firstStriping Policy Target: Maximize the resources available to VMs in a

node Heuristic: Spread the VMs in the cluster nodes Implementation: Use those nodes with less VMs running

firstLoad-aware Policy Target: Maximize the resources available to VMs in a

node Heuristic: Use those nodes with less load Implementation: Use those nodes with more FREECPU

first

III. RELATED WORKIn [18], authors propose a joint-VM provisioning approach

in which multiple VMs are consolidated and provisionedtogether, based on an estimate of their aggregate capacityneeds. This approach exploits statistical multiplexing amongthe workload patterns of multiple VMs, i.e., the peaks andvalleys in one workload pattern do not necessarily coincidewith the others. A performance constraint describing the ca-pacity need of a VM for achieving a certain level of applicationperformance.

In [19], authors introduce a novel technique called Back-ward Speculative Placement (BSP) that projects the pastdemand behavior of a VM to a candidate target host as a basefor handling the deployment requests or reallocation decisions.Collect the past VM demands for future prediction. Besides,authors also restrict the maximum number of migration. Theoptimization component is defined such that the deferencebetween the most and the least loaded hosts is minimized.However, the resource that BSP is considering is only focusedon CPU.

In [20], authors present an application-centric energy-awarestrategy for VM allocation by considering energy consump-tion, performance (response time of application) and SLAviolation. They evaluate the optimal combination of the appli-cations with different type (CPU, memory and I/O intensive)through VM consolidation to maximize the resource utilizationand energy consumption. However, this work still doesnt dis-cuss much in detail how do they concern multi-resources typesin dealing with resource over-provision and under-provision.VM migration is not considered as well.

In [21], authors analyzed the key challenge for providingthe cloud service by meeting the SLA and capacity constraints.VM migration is a technology that moves a VM from one nodeto another node, it is able to on the one hand save as muchenergy as possible, on the other hand keep the performanceof the VM as same as it is expected. However, based onthe impact of VM migration in [22], although live-migrationis introduced recently, for instance in OpenStack, no visibledowntime and no transaction loss is only the ideal goal. Thenumber of migration should controled and minimized, as ithas impact on the performance of the other running VMs inthe source and destination hosts, which is mainly proportionalto the amount of memory allocated to the VM.

In [23], authors present a dynamic resource allocationsolution that is implemented in OpenStack, the objectives ofthe work is to balance load, minimizing power consumptionand differentiation among different classes of cloud services.Finally the trade-off between the objectives and cost of VMlive-migration is discussed.

In [24] [25] [26], authors present a comprehensive frame-work and solution (algorithms) for handling the energy con-cern in VM allocation and VM consolidation of data centers.The proposed approach is going to be implemented in Open-Stack.

In [27], authors present a dynamic server VM live-migrationand consolidation algorithm for managing SLA violations.

In [28], authors present a VM consolidation strategy thatis implemented on OpenStack cloud by considering SLAviolation.

In [29], authors present a power consumption evaluationon the effects of live migration of VMs. Results show thatthe power overhead of migration is much less in the scenarioof employing the strategy of consolidation than the regulardeployment without using consolidation. Our results are basedon the typical physical server, the power of which is linearmodel of CPU utilization percentage.

IV. UTILITY

The utility functions, namely our targets we want to achieve.

A. Power and Energy Cost

Energy-aware VM scheduling in IaaS clouds. See the fol-lowing figure, power meter is able to provide the actual powerconsumed by the System Under Test (SUT) [1].

Fig. 3. Energy collection

B. Optimal Resource Utilization

[TBD]

C. Customers Satisfaction

[TBD]

D. QoS and Other Operation Costs

[TBD]

E. Profit

[TBD]

V. DATA COLLECTION

What values can we get? Where to get them? How to getthem?

Monitoring information from livert, cgroups, openstdb. Service history information, if any. Infrastructure configuration information (Openstack can

retrieve them). Service requests. Energy consumption.

VI. NEXT STEPS

More SotA analysis is required. Next step is to reviewthe following papers: [6] [7] [9] [8] [10] [11] [12] [13][14] [15] [16] [17].

REFERENCES

[1] Qingwen Chen Towards energy-aware VM scheduling in IaaS cloudsthrough empirical studies.

[2] (2012) Filtering and weighting in Openstack, [Online]. Available:http://nova.openstack.org/devref/filter scheduler.html

[3] (2011) Proposals in Openstack, [Online]. Available: http://www.mail-archive.com/netstack@lists.launchpad.net/msg00298.html

[4] (2012) OCCI and Openstack, [Online]. Available: http://occi-wg.org/tag/openstack/

[5] (2012) Match-making in Opennebula, [Online]. Available:http://opennebula.org/documentation:rel3.4:schg

[6] C. Abhishek, G. Pawan, S. Prashant Quantifying the benefits of resourcemultiplexing in on-demand data centers.

[7] W. Guiyi, V. Athanasios, Z. Yao, X. Naixue A game-theoretic methodof fair resource allocation for cloud computing services.

[8] C. Sivadon Optimal Virtual Machine Placement accross Multiple CloudProviders.

[9] A. Janki Negotiation for resource allocation in IaaS Cloud.[10] A. Marcoss Evaluating the cost-benefit of using cloud computing to

extend the capacity of clusters.[11] A. Bo Automated negotiation with decommitment for dynamic resource

allocation in cloud computing.[12] L. Ignacio On the management of virtual machines for cloud infras-

tructures.[13] J. Pramod Usage management in cloud computing.[14] G. Hadi Multi-dimensional SLA-based resource allocation for multi-

tier cloud computing systems.[15] S. Shekhar Energy aware consolidation for cloud computing.[16] G. Hadi Maximizing Profit in Cloud Computing System via Resource

Allocation.[17] L. Jim Performance Model Driven QoS Guarantees and Optimization

in Clouds.[18] X. Q. Meng Efficient Resource Provisioning in Compute Clouds via

VM Multiplexing in Proc !CAC2010.[19] N. M. Calcavecchia VM Placement Stratrgies for Cloud Scenarios in

Proc IEEE 5th International Coference on Cloud Computing.[20] H. Viswanathan, Energy-Aware Application-Centric VM Allocation for

HPC Workloads[21] S. Das, Faster and Efficient VM Migrations for Improving SLA and

ROI in Cloud Infrastructures[22] F. Hermenier, entropy a consolidation manager for clusters[23] F. Wuhib, Dynamic Resource Allocation with Management Objectives

Implementation for an OpenStack Cloud[24] A. Beloglazov, energy-aware resource allocation heuristics for efficient

management of data centers for cloud computing[25] A. Beloglazov, OpenStack Neat: A Framework for Dynamic Consoli-

dation of Virtual Machines in OpenStack Clouds A Blueprint[26] A. Beloglazov, Optimal Online Deterministic Algorithms and Adaptive

Heuristics for Energy and Performance Efficient Dynamic Consolidationof Virtual Machines in Cloud Data Centers

[27] N. Bobroff, Dynamic Placement of VMs for Managing SLA Violations[28] A. Corradi, VM consolidatio: A real case based on OpenStack Cloud

FGCS 2012.

[29] Q. Huang, Power Consumption of VM Live-Migration in Clouds CMC2011.