backfill_ccgrid_2011_2

8/6/2019 backfill_ccgrid_2011_2

1/10

Improving Utilization of Infrastructure Clouds

Paul Marshall

Department of Computer Science

University of Colorado at Boulder

Boulder, CO USA

[email protected]

Kate Keahey1,2

and Tim Freeman1,2

1Computation Institute, University of Chicago

2Argonne National Laboratory

Chicago, IL USA

[email protected] and [email protected]

Abstract A key advantage of Infrastructure-as-a-Service (IaaS)

clouds is providing users on-demand access to resources.

However, to provide on-demand access, cloud providers must

either significantly overprovision their infrastructure (and pay a

high price for operating resources with low utilization) or reject a

large proportion of user requests (in which case the access is no

longer on-demand). At the same time, not all users require truly

on-demand access to resources. Many applications and

workflows are designed for recoverable systems where

interruptions in service are expected. For instance, many

scientists utilize High Throughput Computing (HTC)-enabled

resources, such as Condor, where jobs are dispatched to availableresources and terminated when the resource is no longer

available.

We propose a cloud infrastructure that combines on-demand

allocation of resources with opportunistic provisioning of cycles

from idle cloud nodes to other processes by deploying backfill

Virtual Machines (VMs). For demonstration and experimental

evaluation, we extend the Nimbus cloud computing toolkit to

deploy backfill VMs on idle cloud nodes for processing an HTC

workload. Initial tests show an increase in IaaS cloud utilization

from 37.5% to 100% during a portion of the evaluation trace but

only 6.39% overhead cost for processing the HTC workload.

We demonstrate that a shared infrastructure between IaaS cloud

providers and an HTC job management system can be highlybeneficial to both the IaaS cloud provider and HTC users by

increasing the utilization of the cloud infrastructure (thereby

decreasing the overall cost) and contributing cycles that would

otherwise be idle to processing HTC jobs.

Cloud computing, Infrastructure-as-a-Service, High Throughput

Computing

I. INTRODUCTIONIn the recent years, Infrastructure-as-a-Service (IaaS) cloud

computing [14] has emerged as an attractive alternative to theacquisition and management of physical resources. The on-demand provisioning it supports allows users to elastically

expand and contract the resource base available to them basedon an immediate need a pattern that enables a quickturnaround time when dealing with emergencies, workingtowards deadlines, or growing an institutional resource base.This pattern makes it convenient for institutions to configure private clouds that allow their users a seamless or nearseamless transition to community or commercial cloudssupporting compatible VM images and cloud interfaces. Suchprivate clouds are typically configured using open source IaaSimplementations such as Nimbus [16] or Eucalyptus [17].

However, such private cloud installations also face autilization problem. In order to ensure on-demand availability a provider needs to overprovision: keep a large proportion ofnodes idle so that they can be used to satisfy an on-demand

request, which could come at any time. The need to keep allthese nodes idle leads to low utilization. The only way toimprove it is to keep fewer nodes idle. But this meanspotentially rejecting a higher proportion of requests to a pointat which a provider no longer provides on-demand computing.This situation is particularly hard to accept in the world ofscientific computing where the use of batch schedulerstypically ensures high utilization [25] and thus much betterresource amortization. Thus, potential low utilizationconstitutes a significant potential obstacle to the adoption ofcloud computing in the scientific world.

At the same time, while the on-demand capabilities of IaaSclouds are ideal for many scientific use cases, there are others

that do not necessarily require on-demand access to resources.Many systems, specifically volunteer computing systems suchas SETI@Home [4] and Folding@Home [11], are capable oftaking advantage of resources available opportunistically andare also preemptible, i.e., designed as failure resilient systemswhere interruptions in service can be handled withoutcompromising the integrity of computation. One example in thescientific community is the use of high throughput computing(HTC), as implemented by e.g., the Condor [24] system whereusers employ HTC-enabled resources to process their

Figure 1. Example Backfill Deployment

!"#$"%&'""(&

)*+,-.&!("-$&

/*01&2&

/*01&3&

4"%5.6781&/1%9*81&

:;;&)"$1.&

;7.01%&

4"%51%.&

4"%51%.&

&&

:;;&?&

=.1%&

:;&

:;;&@&

=.1%&

:;&

=.1%&

:;&

:;;&A&

=.1%&

:;&

:;;&B&

2785C((&

:;&

D#*E701&"%&F1%+*#701&

4"%5.6781.&

/-,+*0&G",&&

2785C((&

:;&

H*.6708I&F7.5.&

H*.6708I&F7.5&

2785C((&

:;&

G"*#&'""(&

D#*E701&"%&

F1%+*#701&

4"%5.6781.&

J7-#8I&2785C((&

)"$1.&K#L$1+7#$&

=.1%&

MF!&=.1%&


2/10

workloads. These applications are designed to scavengeunused resource cycles: for example, when a user stops usingtheir desktop, the screensaver might use the resource to run avolunteer computing program. The job may then be preemptedwhen the resource becomes unavailable (i.e., the user is using itagain), in which case the job is typically requeued andrescheduled on another available resource by the HTC systemthat manages it.

We propose a cloud infrastructure that combines on-demand allocation of resources with opportunistic provisioningof cycles from idle cloud nodes to other processes, such asHTC, by deploying backfill VMs. Backfill VMs are deployedon idle cloud nodes and can be configured to perform anydesired function. A backfill VM is terminated when theresource is needed to satisfy an on-demand request. If we canensure that the computation occurring in backfill VMs isresilient to such sudden termination, the time that wouldotherwise be idle can be profitably spent. Furthermore, cyclesvia backfill VMs can be provided to users at a lower cost thanon-demand VMs because of the cloud providers ability toterminate the instances when needed, thus for users that workwith HTC resources and possibly expect such behavior already,

backfill VMs would provide a less expensive option whenmoving their workloads to the cloud. Overall, this designachieves two goals: for cloud providers, it offers a path tohigher utilized clouds; for cloud users, it offers another type ofresource lease, potentially cheaper than on-demand, non-preemptible resource.

In our work we extend the Nimbus toolkit [16] to deploy backfill VMs on idle Virtual Machine Monitor (VMM) nodes[21]. Nimbus is an open source toolkit for deploying IaaSclouds, designed with extensibility in mind, which makes itparticularly suitable for projects such as the one described here.To illustrate how the system works, we configure the backfillVMs as Condor workers that integrate with a Condor pool to

process HTC jobs. We evaluate the ability of the system toincrease utilization of the IaaS cloud infrastructure withoutsacrificing the ability of the IaaS cloud to provision resourceson-demand. We also evaluate the ability of the system tocontribute cycles that would otherwise be idle to processingHTC jobs.

We find that during certain portions of our experimentalevaluation backfill VMs contribute to an increase in theutilization of the IaaS cloud infrastructure from 37.5% to 100%with only 6.39% overhead cost for processing the HTCworkload. Additionally, backfill VMs process the entireCondor workload using what would have otherwise been idlecycles.

The remainder of the paper is organized as follows. InSection II we examine the general approach of deploying backfill VMs on idle IaaS cloud nodes, in Section III wediscuss our specific extensions to the Nimbus toolkit in order todeploy backfill VMs on idle cloud nodes. In Section IV weevaluate our implementation, in Section V we cover the relatedwork in the field, and in Section VI we discuss our directionsfor future work. We conclude in Section VII.

II. APPROACHA compute infrastructure cloud operates by allowing a user

to make leases against its pool of resources; an infrastructurelease makes a resource available to the user based on set oflease terms defining the availability, capacity and generalconditions of a lease. In our system we focus on investigatingtwo types of leases:

On-demand, non-preemptible and flexible leases give auser access to a resource within interactive time ofmaking the request and make the resource available foran agreed-upon period of time. The user can deployany VM compatible with the system.

Opportunistic, preemptible and pre-set leases give auser access to a resource at an indeterminate time andmake the resource available to the user for anindeterminate amount of time. Further, this resource ispre-defined for the user by the cloud administrator, i.e.the user cannot provide his or her own VM.

In the rest of the paper we will be referring to them as on-demand leases and preemptible leases respectively. We define

the following roles in our system. An on-demand useris a userthat requests on-demand VMs/leases from an IaaS cloud. Anon-demand VMis an IaaS VM that has been provisioned via onon-demand lease for a specific user. A backfill VMis a VM thathas been deployed automatically by the IaaS cloud manager onan idle IaaS node using a preemptible lease. An IaaS cloudadministrator is the person or persons responsible forconfiguring and managing the IaaS cloud resource. An HTCuser is a user that submits jobs to an HTC queue and an HTCworkeris a system that process jobs from the HTC queue.

In the context of IaaS clouds, backfill VMs are genericVMs deployed on IaaS resources using a preemptible lease thatmay be configured to perform any function. Backfill VMs havetwo major constraints. First, backfill VMs may be terminatedsuddenly in order to free up space for the IaaS cloud managerto service an on-demand lease. Second, because of theunpredictable timing of on-demand leases (from any number ofunique users), a variable number of backfill VMs may beavailable at any given time. Thus, we assume that applicationsexecuting inside backfill VMs are designed to handleenvironments that contain a variable number of workers thatmay join or leave the system at any time.

Certain applications are not well suited for these volatileenvironments, for example, parallel applications that require all processes to be present for the duration of the applicationsexecution and lack checkpoint/restart capabilities. An exampleof a system that is well-suited for backfill VMs are High

Throughput Computing (HTC) workloads [24]. HTCworkloads typically consist of a large number of jobs thateventually need to be processed. An assumption of HTCworkloads is that they do not have an immediate deadline andtherefore do not need resources to be available at a particulartime. In addition, terminating individual HTC jobs in themiddle of execution and requeuing them for later execution isan acceptable action as long as the system is eventually able toprocess the workload.


3/10

This work further assumes that backfill VM images anddeployments are managed by the IaaS cloud administrator;backfill VMs are not directly managed by remote cloud users.This work also assumes that the HTC scheduler is not aware ofthe IaaS cloud scheduler details and vice versa. Thus, thescheduling of HTC jobs on HTC workers (in backfill VMs) isseparate from the launching and terminating of user VMs onthe IaaS cloud.

A.ArchitectureFigure 1 is a simple example deployment consisting of an

IaaS Nimbus cloud using backfill VMs to run Condor HTCjobs in order to increase cloud infrastructure utilization. In thisexample the HTC user submits 3 individual tasks to the Condormaster, which is able to schedule 1 task immediately on a localworker. However, because the remaining resources in site BsCondor pool are busy, Condor leverages the cycles provided bysite As backfill VMs to launch the other 2 tasks immediately.Condor is an ideal candidate for such a deployment because ofits original design as a cycle-scavenger where idle desktopmachines execute jobs until the systems user returns, afterwhich the job is either migrated to another system or prematurely terminated and re-launched on another system. A backfill deployment could be much more complex then theexample depicted in Figure 1. For instance, backfill VMs couldhost a number of different backfill processes in addition toCondor workers. The Condor workers could also be configuredto execute jobs from multiple Condor pools or sites instead of asingle pool, as shown in the figure.

IaaS cloud administrators must consider a variety ofdifferent backfill configurations when deploying a backfillsolution on an IaaS cloud. The configuration is influenced bythe characteristics of the underlying physical IaaS resources,the users of on-demand leases, and the users of preemptibleleases.

First, appropriate backfill applications and workflowsshould be identified. These applications should accommodatethe constraints discussed previously, namely, they should beable to utilize a variable number of nodes that may join or leavethe system at any time.

Second, if the IaaS Virtual Machine Monitor (VMM) nodeshave multiple cores, the IaaS cloud administrator mustdetermine the granularity with which to deploy backfill VMs.One possible solution is to deploy a single backfill VM foreach core on a VMM node. This approach allows the IaaScloud manager to have fine-grained control over VMdeployment. For example, a node with 8 cores may becomprised of 3 user VMs and 5 backfill VMs. However, thedisadvantage of this approach is the increased overhead

introduced by running additional VMs. An alternative solutionis to deploy a single backfill VM per VMM node, utilizing allof the available cores in the backfill VM. This approachreduces the virtualization overhead on the VMM node sinceonly a single backfill VM would be running, however, if thecloud receives an on-demand user request for a single core it ispossible that the entire backfill VM will need to be terminatedto satisfy the on-demand request. This would leave theadditional VMM cores idle. Alternatively, the administratormay wish to configure VM deployments based on allocation of

other resources, such as RAM, instead of (or in addition to)CPU cores.

Third, the IaaS cloud administrator must determine theapproximate size of the backfill deployment, relative to the sizeof the IaaS cloud. The administrator may allow the backfilldeployment to utilize all available nodes, however, theadministrator should consider the additional overhead requiredto terminate backfill nodes in order to fulfill on-demand user

requests. Depending on the method used to terminate backfillVMs (e.g. immediately trash the VM or cleanly shut it down)and the number of backfill nodes being terminated, thisoverhead may be significant. Consequently, IaaS cloudadministrators may configure backfill deployments to onlyutilize a percentage of idle nodes or never allow backfilldeployments exceed a preset and static number of idle nodes.

Finally, the IaaS cloud administrator must determine the backfill VM image deployment method. A fresh backfill VMimage could potentially be transferred from the VM imagerepository for every backfill VM deployment (this process isreferred to as VM image propagation and is a typical patternthat remote users expect for their deployments); however, this

introduces network contention and may slow the deployment ofon-demand user VMs. Another solution is to propagate the backfill image to the node and cache it on the node, onlypropagating it again if the image is updated or is removed fromthe cache on the node. A third and simple solution is tomanually place the backfill image on each VMM node. Thus,when a backfill VM boots, the image is already on the VMMnode and doesnt require an image to be copied. This approachreduces network contention (since it is only performed at selecttimes) and reduces launch time for the VM (since the backfillimage doesnt need to be copied across the network). However,any changes to the backfill image require that the IaaS cloudadministrator push it out to all VMM nodes.

B.Backfill Termination PoliciesBackfill VMs are deployed on idle VMM nodes and

terminated whenever space is needed to service an on-demandlease. However, the specific backfill VMs that are selected fortermination may impact the services, applications, orworkflows executing inside those VMs. For this reason, ideallythe backfill VM termination policies should consider theapplications and services running inside those VMs as well asgeneric factors such as the need for clean shutdown, the abilityof an application to checkpoint/restart, etc. We discuss belowtwo simple policies that do not integrate such hints andhighlight their shortcomings.

One simple policy is to select the backfill VMs to terminateat random. This approach ignores a number of factors that

could impact the services and applications running insidebackfill VMs. In particular, if the random backfill VM selectedfor termination is the only backfill VM performing a usefultask then its work may be lost while idle backfill VMs continuerunning.

Another policy is to select the backfill VM that has beenrunning for the least amount of time. This policy makes theassumption that the backfill VMs running the longest also runlong-running jobs that have performed the most work and


4/10

therefore will lose the most work when terminated. Theintuition behind this policy is that a workload consistingentirely of short running jobs (e.g. less then a few minutes) willonly be slightly impacted by the termination of any backfillVM however, workloads that consist of long running jobs (or acombination of long and short jobs) will be impacted more bythe termination of a VM that has been deployed for anextended period of time. In this case all of the VM's work will be lost unless the application supports checkpoint/restart ormigration. It is, however, possible that the backfill VM runningthe longest will not always be the backfill VM with the mostwork to lose; without tighter integration between the HTC jobmanager and backfill termination policy this information is notreadily accessible.

More advanced backfill termination policies will requirehints from the IaaS scheduler in the form of a more complete picture of cloud VM placement or integration with theapplications and services running inside of the backfill VMs.For instance, if the backfill termination policy is aware of VMplacement on individual VMM nodes, it may be able to selectthe fewest number of backfill VMs for termination in order tosatisfy an on-demand lease instead of blindly terminating

backfill VMs until the request can be fulfilled. Alternatively, ifthe backfill termination policy integrates with the applicationsand services running inside of backfill VMs then it may be ableto identify which backfill VMs are performing useful tasks andavoid terminating those VMs.

III. IMPLEMENTATIONWe extend the open source Nimbus cloud computing

toolkit, which provides on-demand access to resources (in theform of VMs), to support the deployment of preemptible leaseson idle cloud nodes, also referred to as backfill. We make anumber of simplifying assumptions in our currentimplementation. First, the Nimbus administrator mustconfigure backfill. On-demand cloud users cannot elect todeploy backfill VMs. Second, the current implementation iscapable of using only a single backfill VM image per VMMnode. Different backfill VM images could be deployed ondifferent VMM nodes, allowing multiple backfill VM imagesto operate within the same IaaS cloud, each performingdifferent functions.

Unless the user specifies the max number of backfillinstances, backfill automatically attempts to deploy as many backfill VMs as possible when it is enabled. Initially, wesupport two termination policies for selecting backfill VMs fortermination in order to fulfill an on-demand lease. The first policy simply selects a random backfill VM. The second policy, and the default, terminates the most recently deployed

backfill VM in an attempt to minimize the amount of worklost by the premature termination of backfill VMs. In futurework we hope to add additional backfill termination policies.

Finally, our backfill implementation cleanly shuts down the backfill VM. Clean shutdown requires additional time overtrashing the VM, however, performing a clean shutdownnotifies services and applications running inside the backfillVM that the VM will be terminated, allowing them to respond

appropriately (e.g. notify a central manager to reschedulecurrently running jobs).

A.Backfill Configuration OptionsOnly the Nimbus cloud administrator can configure

Backfill. The main configuration options are specified in a backfill.conf file on the Nimbus service node, allowingadministrators to easily configure and deploy backfill VMs onNimbus clouds. The backfill.conf options include:

Backfill.disabled: This option specifies whether backfill is enabled or disabled for the specific cloud.The default is disabled.

Max.instances: This option specifies the maximumnumber of backfill VMs to launch (assuming there areenough idle nodes). The default, 0, launches as manyas possible.

Disk.image: This option specifies the full path to the backfill VM image on the VMM nodes. This optionassumes that the backfill VM image has already been pushed out to the VMM node, the Nimbus servicedoes not automatically transfer it. The image must be

in the same location on every VMM node.

Memory.MB: This option specifies the amount ofmemory (RAM) to use for backfill VMs. The defaultis 64 MB.

VCPUs: This option specifies the number of VCPUsto use for backfill VMs. The default is 1.

Duration.seconds: This option specifies the amount oftime (in seconds) backfill VMs should run before being terminated (currently Nimbus doesnt supportinfinite length VM deployments). The default is oneweek.

Termination.policy: This option specifies thetermination policy to use when terminating backfillVMs. The policies currently supported include amost recent policy that first terminates backfill VMsrunning for the least amount of time and an any policy that simply terminates a random backfill VM.The default is the most recent policy.

Retry.period: This option specifies the duration (inseconds) that the backfill timer waits in betweenattempts to deploy backfill VMs on idle VMM nodes.The default is 300 seconds.

Network: This option allows the IaaS cloudadministrator to specify whether the backfill VMs

should use the public network or private.

B.Extensions to the Nimbus Workspace ServiceWe modified the Nimbus workspace service [16] to support

backfill VM deployments. The workspace service isresponsible for managing the VMM nodes and servicing on-demand user requests for VMs. In particular, we added a backfill Java class that contains the majority of the backfillimplementation code. The backfill configuration file is readwhen the Nimbus workspace service is started; if backfill is


5/10

enabled then the service attempts to launch backfill VMs untilthe request for resources is denied or the maximum number ofbackfill instances is reached (as specified in backfill.conf). Theservice also starts a backfill timer that continually loops,sleeping for the duration specified by duration.seconds in backfill.conf, and attempts to launch backfill VMs when itwakes (until the request for resources is denied or themaximum number of backfill instances have been launched).

As part of our modifications to the Nimbus workspaceservice, we also modified its scheduler to detect any rejectedrequests for on-demand user VMs. If we detect a rejected on-demand user request and backfill is enabled, we attempt toterminate the appropriate number of backfill VMs so that theuser request can be fulfilled. After the backfill VMs areterminated we attempt to service the user request again. If weare not able to service the request, we continue terminating backfill VMs until we are able to service the request or allbackfill VMs are terminated. If all backfill VMs are terminatedand we are still unable to service the request, then the request isrejected. We also modified the scheduler to detect when on-demand users are terminating VMs. In this case backfillattempts to re-launch backfill VMs (if backfill is enabled)

without waiting for the backfill timer to expire.

In general this is an acceptable approach, however, there isa design flaw in this initial implementation. It is possible thatall backfill VMs could be terminated and yet the on-demandrequest could still be rejected. In this case the ideal solutionwould be to recognize, upfront, that the IaaS cloud is unable tofulfill the on-demand request and, therefore, the on-demandrequest should be rejected immediately before terminating any backfill VMs. However, recognizing this upfront requires acomplete picture of the VM placement and the location ofindividual VM deployments on VMM nodes.

IV. EVALUATIONOur evaluation examines an IaaS backfill deployment from

two perspectives. First, we consider the ability of the system toincrease utilization of the IaaS cloud infrastructure withoutsacrificing the ability of the cloud to provision resources on-demand. Second, we consider the ability of the system tocontribute otherwise idle cycles to process HTC jobs using backfill VMs. We use Condor as the HTC job manager,leveraging its ability to requeue jobs that are interrupted duringexecution.

For the evaluation we deploy a backfill-enabled version ofNimbus 2.6 on FutureGrid [8]. Nimbus runs on a cluster of 16VMM nodes with 2.4 GHz 8-core Intel Xeon processors and 24GB of RAM with 20 GB allocated for user VMs, allowing for atotal of 128 single-core VMs. The Nimbus workspace servicenode runs on an additional node. The Nimbus workspaceservice listens for incoming on-demand user requests for VMsand launches or terminates the VMs on the VMM nodes. Thisnode also hosts the user VM image repository.

In our experiments, we assume that a single backfill VMutilizes the entire VMM node (all 8 cores). We choose thislevel of granularity in order to reduce virtualization overheadfor backfill VMs and avoid additional network contentioncaused by transferring a backfill VM image over the network

each time it was deployed on an idle cloud node; instead thebackfill VM images were manually copied to the VMM nodes before the evaluation began. Backfill VMs are configured asCondor worker nodes, preconfigured (at boot) to join ourCondor master running on our evaluation node. The Condorpool does not contain any additional worker nodes.

We use an additional two nodes (identical to the VMMnodes described above) to generate the workload. One node is

used to host the Condor master and queues the Condor jobs.The second node executes the workspace service workload,requesting on-demand user VMs. On-demand user requestsonly request a single core.

For all of the evaluations involving backfill we use the mostrecent backfill termination policy. The most recent backfilltermination policy first terminates the backfill VMs that have been running for the least amount of time. The backfill VMsare terminated using clean shutdown. Cleanly shutting downbackfill VMs enables the Condor workers running inside of thebackfill VMs to notify the Condor master to reschedule its jobs.If clean shutdown is not used with Condor and the backfill VMis simply trashed, then Condor relies on timeouts before

rescheduling jobs, which can take up to two hours. (As of thetime of this writing Condor has an experimental feature toreverse the direction of its pings that determine the status ofworker nodes, this would eliminate the long timeout period andthe need to cleanly shutdown the backfill VMs. We enabled thefeature, however, we did not observe the system behaving asexpected. Interrupted jobs were still experiencing prohibitivelylong delays before being resubmitted to the Condor queue.Therefore, we did not use this feature for the evaluation,instead we terminate the backfill VMs using clean shutdown.)

For the evaluation we define the following metrics:

Utilization is the percentage of user cyclesconsumed by CPU cores on the VMM nodes in the

IaaS cloud that are either running an HTC job orrunning an on-demand user VM. Because backfilllaunches VMs on any idle VMM node, regardlessof the presence of HTC jobs, it is possible for theentire IaaS infrastructure to be running backfillVMs on all VMM nodes but still have 0%utilization. For our evaluation backfill VMs must be running Condor jobs for them to contribute tothe overall utilization of the infrastructure.

First queued time is the amount of time that elapses between the time when a Condor job is submittedand when it first begins executing.

Last queued time is the amount of time that elapsesbetween the time the Condor job is first submittedand the time the Condor job finally beginsexecuting for the last time before completingsuccessfully. We note that it is possible for backfillVMs to be terminated by the deployment of on-demand user VMs, preempting Condor jobsexecuting in backfill VMs, and thus requiring theirresubmission. While this may happen to a Condorjob any number of times, it is presumed that the job


6/10

will eventually be able to execute successfully tocompletion.

User VM service response time is the amount oftime it takes the Nimbus service to respond to anon-demand user request, i.e., the time betweenwhen the service first receives the request and thetime it determines whether a VM will be launchedor that the request will be rejected. This time doesnot include the amount of time that it takes toactually boot the on-demand user VM or propagatethe VM image, only the amount of time it takes theservice to determine whether or not the request willbe handled. If backfill is enabled and backfill VMsneed to be terminated to deploy an on-demand userVM, the user VM service response time willinclude the necessary time to terminate backfillVMs.

A. Workload TracesThe workloads we selected are based on real workload

traces, modified to fit the size of our environment. The Condorworkload used for the evaluation consists of a Condor tracefrom the Condor Log Analyzer at the University of NotreDame [6]. The workload contains 748 serial jobs that eachsleep for differing amounts of time, with a minimum of 1second, a maximum of 2089 seconds, and a standard deviationof 533.2. The Condor trace submits 400 jobs to the Condorqueue immediately, followed by an additional 348 jobs 2573seconds later.

Along with the Condor workload we consider an on-demand

IaaS cloud workload that we selected from the University ofChicago (UC) Nimbus science cloud [16]. We chose this particular workload trace because, despite its lack ofdynamism, it is generally characteristic of the traces weobserved on the UC Nimbus cloud. We did not observe the UCNimbus cloud to be highly dynamic over relatively short time periods (e.g., a few hours). User requests were typically for astatic set of instances over a long period of time (e.g. 6 VMsfor 24 hours). In cases where user requests overlapped, therequests often overlapped for extended periods of time (e.g. 6

hours). Additionally, we selected this trace because itdemonstrates the expected behavior of an overprovisionedcloud infrastructure that is the focus of this work, i.e., itcontains many idle VMM nodes available to service on-demand requests. Although there are an infinite number of possible on-demand and HTC workload scenarios that wecould have generated for our evaluation, many which may haveartificially highlighted the usefulness of backfill to either theon-demand user community or the HTC user community, weinstead chose to base our evaluation off of two realistic

workload traces. By choosing two realistic workload traces weare able to demonstrate and evaluate the usefulness of backfillto both communities given at least one possible scenario.(Furthermore, we selected an on-demand trace from theconsiderably smaller UC Nimbus science cloud then a largerand possibly more dynamic cloud provider, such as Amazon orthe Magellan cloud at Argonne National Laboratory [13],because of the lack of availability of such traces at the time ofthis work.)

Figure 2. On-demand user VM workload (no backfill). Figure 3. Condor workload without on-demand user requests. All 16VMM nodes are used exclusively to process the Condor jobs.

Figure 4. Condor workload with on-demand user requests, using the most

recent backfill termination policy. The 16 VMM nodes run both on-demand VMs and backfill VMs to process the Condor workload.


7/10

Because the University of Chicago Nimbus cloud onlycontains a total of 16 cores and our evaluation environmentcontains 128 cores we multiplied the workloads by 8 so that 16individual requests for the University of Chicago cloud (16cores) would be 128 individual requests for the entire 128 coresin the evaluation environment. Thus, an individual request for asingle core on the University of Chicago cloud is 8 individualrequests, each for a single core, in our evaluation environment.The on-demand user workload requests a total of 56 individualVMs over the course of the evaluation. Finally, we terminatethe evaluation shortly after the overlapping Condor tracecompletes.

Both workloads submit individual and independent requests;each request is for a single core. In the Condor workload the jobs simply consist of a program that sleeps for the desiredamount of time. In the on-demand workload VMs are startedand run for the appropriate duration. Backfill VMs are capableof executing 8 jobs concurrently across the 8 cores in a backfillVM, while individual on-demand user requests are single-coreVMs. RAM is divided evenly among the VMs.

B. Understanding System BehaviorTo understand the system behavior we compare three

different scenarios. The first scenario only considers the on-demand user workload; the number of cores used in thisworkload is shown in Figure 2. In this case the IaaS cloudachieves an average utilization of 36.36%, shown in Figure 5,with a minimum utilization of 0% and a maximum utilizationof 43.75%.

The second scenario simply involves running the Condorworkload on all 16 VMMs (128 cores) without the on-demand

user workload. In this case the entire Condor workloadcompletes in approximately 84 minutes (5042 seconds), asshown in Figure 3.

In the third scenario the Condor workload is overlaid withthe on-demand user workload. The Condor workload takes anadditional 11 minutes and 45 seconds over the case whereCondor has exclusive access to the resources, completing inapproximately 96 minutes (5747 seconds), as shown in Figure4. However, the utilization of the cloud infrastructure, shown in

Figure 5. IaaS cloud infrastructure utilization without backfill VMs, onlyrunning on-demand user VMs.

Figure 6. IaaS cloud infrastructure utilization with on-demand user VMsand backfill VMs processing Condor jobs.

Figure 7. Condor job queued time when the job first begins executing,using the most recent backfill termination policy. Figure 8. Condor job queued time when the job begins executing for thelast time before successful completion, using the most recent backfilltermination policy.


8/10

Figure 6, increases to an average utilization of 83.82% with aminimum utilization of 0% and a maximum of 100%. As theCondor jobs complete (just before 6000 seconds in theevaluation) utilization again drops because the IaaS cloud is nolonger running Condor jobs in addition to on-demand userVMs.

The large increase in utilization is due to the fact that thecloud infrastructure is no longer solely dedicated to servicingon-demand user VM requests, instead the cloud infrastructureis also able to process jobs from a Condor workload withoutcompromising its ability to service on-demand VM requests.The increase in utilization is dependent upon the amount ofwork in the HTC workload. Naturally, longer and more HTCjobs will translate into higher utilization.

While increased utilization certainly benefits the cloudprovider, Figure 4 also demonstrates that it is advantageous toHTC workloads. The workload, which originally takesapproximately 85 minutes on the same dedicated hardware(Figure 3), is only delayed by 11 minutes and 45 seconds(completing in just under 96 minutes) when on-demand userVMs are introduced into the system as shown in Figure 4.However, presumably the cost of utilizing backfill nodes would be lower than utilizing dedicated on-demand user VMs sincebackfill VMs may be reclaimed by the cloud provider withoutwarning.

C. Understanding System PerformanceTo understand how the IaaS cloud environment and backfill

solution impacts on-demand users and HTC users we againconsider the three different scenarios. The first scenarioinvolves the on-demand user workload. The second scenarioinvolves Condor jobs running on the 16 VMM nodes withoutinterruption from on-demand user VMs and the third scenariooverlays the first two.

In Figure 7 we can see that the Condor first queued time issmallest when no user VMs are present, i.e., if Condor isallowed exclusive access to its own hardware for executing

jobs. Enabling backfill and introducing user VMs causes anincrease in the Condor first queued time because there arefewer backfill VMs processing Condor jobs since on-demanduser VMs are also running.

When backfill is enabled there is a noticeable increase in theamount of time that Condor jobs are delayed until they finally begin executing before successful completion, as seen by thenumerous spikes for individual Condor jobs in Figure 8 (of

which there are a total of 48). These 48 jobs actually first beginexecuting much earlier, as seen by the absence of spikes inFigure 7. These jobs are delayed because of the arrival of theon-demand VMs, which cause the termination of backfill VMs, preempting the running Condor jobs. Of the 48 jobs that arepreempted the average amount of additional time these jobs aredelayed (before they begin executing for the final time) is 627seconds with a standard deviation of 76.78 seconds; theminimum amount of extra time that a job is delayed is 273seconds and the maximum is 714 seconds. The 48 preempted jobs spent a total of 22,716 CPU seconds processing theCondor workload before they were preempted. The entireCondor workload required a total of 355,245 CPU seconds.Thus, for our experimental traces, the use of a backfill-enabled

IaaS cloud resulted in an additional 6.39% of overhead for theCondor workload.

Figure 9 demonstrates the impact that backfill has on on-demand user requests. When backfill is disabled all on-demanduser requests are handled in 2 seconds or less. However, when backfill is enabled the amount of time to respond to an on-demand user request can be as high as 13 seconds, though themajority more closely match the case where backfill isdisabled. The large delay in response time is when the Nimbusservice must terminate (via clean shutdown) backfill VMs inorder to service the on-demand user request. Additionally, because the evaluation environment consists of 8-core nodeswith backfill VMs consuming all 8 cores, whenever a backfill

VM is terminated to free space for an on-demand user VM(even if the on-demand user request is only for a single core),the remaining cores on the VMM node remain idle and freelyavailable for future on-demand user VMs.

While this evaluation is based on two real workload traces,one can imagine that under some of the possible workloads, backfill VMs may be more or less beneficial to IaaS cloud providers and HTC users. Certain workloads, environmentcharacteristics, and backfill termination policies willundoubtedly lend themselves as more beneficial to onecommunity over the other. This is something we will considerin future work. However, our backfill solution and evaluationdemonstrates that when considering a realistic on-demand userworkload trace and a realistic Condor workload trace, a shared

infrastructure between IaaS cloud providers and an HTC jobmanagement system can be highly beneficial to both IaaS cloud provider and HTC users by increasing the utilization of thecloud infrastructure (thereby decreasing the overall cost) andcontributing cycles that would otherwise be idle to processingHTC jobs.

Figure 9. Nimbus service response time for on-demand user VM

requests. This is the time from when the Nimbus service first receives therequest until the service responds. It includes the time required to

terminate backfill VMs (when applicable), however, it does not includethe time for a user VM to boot.


9/10

V. RELATED WORKAlthough our work utilizes backfill to achieve high

utilization of an IaaS infrastructure, it is different from workthat uses backfill scheduling to increase the utilization of largesupercomputers [7]. Scheduling on supercomputers does nottypically assume that backfill jobs will be preempted by an on-demand request, seeking to immediately access the resources,while our work assumes this to be the default case. Instead,

these backfill scheduling algorithms only attempt to backfillunused resources with requests that match the available slotsboth in their resource needs as well as their expected runtime.There are, however, preemption based backfill solutions [22]that share many similar characteristics to our work. The majorexception is their focus on queue-based supercomputers andour focus on IaaS cloud infrastructures.

Volunteer computing systems, such as BOINC [2], harvestcycles from idle systems distributed across the Internet. Majorexamples of volunteer applications include SETI@Home [2]and Folding@Home [11]. These applications are designed toaccommodate interruptions in service since widely distributedcomputers, operated by a seemingly infinite number of

disparate users, cannot provide any guarantee of service. In thecase of volunteer computing systems interruptions in serviceare usually the result of users returning to their systems to dowork, systems crashing, or systems becoming disconnectedfrom the Internet. Much research on volunteer computingfocuses on the usefulness, efficiency, and failure prediction ofthese volatile environments [1], [3], [18], [19]. Our workfocuses on providing cycles within an IaaS infrastructure thatwould have otherwise been idle to other processes, such asHTC or volunteer computing, where the services may beinterrupted by the arrival of requests for on-demand VMs.Applications that leverage volunteer computing systems wouldbe ideal candidates for backfill VMs because of their ability tohandle unexpected failures in service.

In [23] we also leverage recovery techniques, specificallysuspending and resuming VMs, to achieve high utilization ofIaaS cloud infrastructures. While the goal of maintaining highutilization via introducing different types of leases is the sameas the work described here, the leases themselves as well as therecovery technique used, specifically that of suspending andresuming VMs, is different from the focus in our work. Insteadof using suspend/resume to support advanced reservations weleverage a recovery system that uses resubmission (Condor) toensure that high utilization is achieved and no work is lost.

Another area that shares related themes to our work is spotpricing, as exemplified by Amazon [2]. With spot pricing users place bids for instances and the cloud provider periodically

adjusts the price of spot instances, terminating the spotinstances with bids that fall below the new spot price andlaunching instances that meet or exceed the spot price. Ourwork uses the current demand for on-demand user VMs todetermine the availability for backfill VMs while Amazonbases availability of spot instances on a spot price.

VI. FUTURE WORKThe backfill implementation used in this paper was an

initial prototype created to demonstrate of the usefulness of

combining IaaS cloud infrastructure resources with other purposes, such as HTC, through backfill VMs. The prototypeimplementation used in this work is publicly available onGitHub [15]. The 2.7 release of the Nimbus toolkit [16]includes the official release of the backfill implementation. Inthe 2.7 release backfill instances are essentially zero-cost spotinstances that have a lower priority than on-demand instancesand spot instances. Therefore, backfill instances arepreemptible by both on-demand requests and spot requests.

The future work opens up the opportunity to exploredifferent variants of the policies described in Section II. Forinstance, exploring finer granularity with which to deployVMs, optimizing the backfill image deployment method, aswell as termination policies. Another possible area for futurework is suspending backfill VMs instead of terminating them.Such a solution may be ideal for a backfill application that doesnot leverage resubmission as its recovery mechanism.

Another set of challenges arises if we broaden the definitionof the preemptible lease, e.g., by removing the assumption thatonly one type of backfill VMs may be used or that only theadministrator can configure backfill VMs. One simple

refinement would be for the administrator to define multiple backfill VMs and have policies on how backfill resources areshared among them (e.g., what percentage of available cyclesshould be devoted to each). However, if users are to submit backfill VMs (i.e., the preemptible lease as defined in thispaper would no longer be fixed) some arbitration mechanismneeds to be defined for deciding between various user/instancerequests. For example, AWS uses auctions to make suchdecisions (i.e., spot instances) but many other mechanismscould also be explored. Additionally, we could also considerdifferent types of leases, e.g., to provide for the impact of backfill VMs on parallel jobs where all processes for a singleparallel job must be available.

Another set of challenges arises out of exploring various

aspects of resource utilization, energy savings, cost and pricing.An assumption throughout this paper has been that improvingutilization is advantageous because it leads to better resourceamortization and thus lower costs per computation cycle. Thisneed not necessarily be so: green computing techniquesallowing providers to power down a proportion of resources[12] may be a better option in some cases, and prices obtainedby auction need not necessarily be sufficient to amortize cost.A more thorough model taking into accounts these factorswould be needed.

VII. CONCLUSIONSIn this paper we propose a cloud infrastructure that

combines on-demand allocation of resources with opportunisticprovisioning of cycles from idle cloud nodes to other processes,such as HTC, by deploying backfill VMs. We extend the opensource Nimbus IaaS toolkit to deploy backfill VMs on idlecloud nodes.

We evaluate the backfill solution using an on-demand userworkload and an HTC workload. We find backfill VMscontribute to an increase of the utilization of the IaaS cloudinfrastructure from 37.5% to 100% during a portion of theevaluation trace but result in only 6.39% additional overhead


10/10

for processing the HTC workload. Additionally, backfill VMsmake available cycles that would have otherwise been idle toassist in processing HTC jobs. In particular, a Condor workloadthat originally completes in approximately 85 minutes ondedicated hardware is only delayed by 11 minutes and 45seconds (completing in just under 96 minutes) when on-demand user VMs are introduced into the system.

ACKNOWLEDGMENTS

We would like to thank David LaBissoniere for his helpand advice deploying and evaluating our system on FutureGrid.

This material is based upon work supported in part by NSFSDCI grant No. 0721867 and, in part, the NSF Grant No.0910812 to Indiana University for "FutureGrid: AnExperimental, High-Performance Grid Test-bed." Partners inthe FutureGrid project include U. Chicago, U. Florida, SanDiego Supercomputer Center - UC San Diego, U. SouthernCalifornia, U. Texas at Austin, U. Tennessee at Knoxville, U.of Virginia, Purdue I., and T-U. Dresden.

REFERENCES

[1]

Acharya A, Edjlali G, and Saltz J. The Utility of Exploiting IdleWorkstations for Parallel Computation, SIGMETRICS 97, pp. 225-34.

[2] Amazon Web Services. Amazon.com, Inc. [Online]. RetreivedDecember 6, 2010, from: http://www.amazon.com/aws/

[3] Anderson D and Fedak G. The Computational and Storage Potential ofVolunteer Computing, CCGRID06, 2006, p. 73-80.

[4] Anderson DP, Cobb J, Korpela E, Lebofsky M, Werthimer D.SETI@home: An Experiment in Public-Resource Computing.Communications of the ACM, 45(11), November 2002, 56-61.

[5] Anderson, D. BOINC: A System for Public-Resource Computing andStorage, 5th IEEE/ACM Workshop on Grid Computing, Nov. 2004.

[6] Douglas Thain, David Cieslak, and Nitesh Chawla, "Condor LogAnalyzer", http://condorlog.cse.nd.edu, 2009.

[7] Feitelson DG, Rudolph L. Parallel job scheduling: Issues andapproaches. Lecture Notes in Computer Science: Job SchedulingStrategies for Parallel Processing, 949, 1995.

[8] FutureGrid. [Online]. Retreived December 6, 2010, from:http://futuregrid.org/

[9] Internet Retailer Magazine. [Online]. Retreived December 6, 2010,from: http://www.internetretailer.com/top500/list/

[10] Keahey, K., I. Foster, T. Freeman, and X. Zhang. Virtual Workspaces:Achieving Quality of Service and Quality of Life in the Grid. Scientific

Programming Journal, vol 13, No. 4, 2005, Special Issue: DynamicGrids and Worldwide Computing, pp. 265-276.

[11] Larson SM, Snow CD, Shirts M, Pande VS. Folding@Home andGenome@Home: Using distributed computing to tackle previouslyintractable problems in computational biology. ComputationalGenomics, Horizon Press, 2002.

[12] Lefe`vre, L. and Orgerie, AC. Designing and evaluating an energyefficient cloud, The Journal of Supercomputing, vol. 51, pp. 352373,2010,

[13] Magellan. [Online]. Retreived Feburary 11, 2011, from:http://magellan.alcf.anl.gov/

[14] Michael A, et al., Above the Clouds: A Berkeley View of CloudComputing, EECS Department, University of California, Berkeley,Tech. Rep. UCB/EECS-2009-28, Feb 2009.

[15]Nimbus 2.6 Backfill (prototype). GitHub. [Online]. Retreived December6, 2010, from: https://github.com/pdmars/nimbus/tree/backfill-2.6

[16] Nimbus. [Online]. Retreived December 6, 2010, fromhttp://www.nimbusproject.org/

[17]Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L,and Zagorodnov D. Eucalyptus opensource cloud-computing system. In

CCA08: Cloud Computing and Its Applications, 2008.

[18] Ren X, Lee S, Eigenmann R, and Bagchi S. Resource FailurePrediction in Fine-Grained Cycle Sharing System, HPDC 06, 2006.

[19] Ryu K and Hollingsworth J. "Unobtrusiveness and Efficiency in IdleCycle Stealing for PC Grids," in Proceedings of IPDPS04, 2004, p. 62a.

[20] Science Clouds. [Online]. Retreived December 6, 2010, from:http://www.scienceclouds.org/

[21] Smith, JE. and Nair, R. Virtual machines: versatile platforms for systemsand processes. Morgan Kaufmann Publishers, San Francisco, CA, USA,2005.

[22] Snell Q, Clement M, and Jackson D. Preemption based backfill. InFeitelson, Rudolph, and Schwiegelshohn, editors, Job SchedulingStrategies for Parallel Processing, pages 2437. Springer Verlag, 2002.Lect. Notes Comput. Sci. vol. 2537.

[23] Sotomayor B, Keahey K, Foster I. Combining Batch Execution andLeasing Using Virtual Machines. the 17th International ACMSymposium on High-Performance Parallel and Distributed Computing(HPDC) Boston, MA. June 2008.

[24] Tannenbaum T, Wright D, Miller K, Livny M. Condor - A DistributedJob Scheduler, in Thomas Sterling, editor, Beowulf Cluster Computingwith Linux, The MIT Press, 2002.

[25] Woitaszek, M. and Tufo, H., Developing a cloud computing chargingmodel for high-performance computing resources, in Computer andInformation Technology (CIT), 2010 IEEE 10th InternationalConference on, July 2010, pp. 210217.

backfill_ccgrid_2011_2

Documents