UNIVERSITI PUTRA MALAYSIA

NASRIN AKHTER

FSKTM 2015 27

ENERGY AND PERFORMANCE EFFICIENT RESOURCE ALLOCATION FOR CLOUD-BASED DATA CENTERS

© COPYRIG

HT UPM

ENERGY AND PERFORMANCE EFFICIENT RESOURCEALLOCATION FOR CLOUD-BASED DATA CENTERS

By

NASRIN AKHTER

Thesis Submitted to the School of Graduate Studies, Universiti PutraMalaysia, in Fulfilment of the Requirements for the Degree of Master of

Science

July 2015

© COPYRIG

HT UPM

© COPYRIG

HT UPM

All material contained within the thesis, including without limitation text, lo-gos, icons, photographs and all other artwork, is copyright material of Uni-versiti Putra Malaysia unless otherwise stated. Use may be made of anymaterial contained within the thesis for non-commercial purposes from thecopyright holder. Commercial use of material may only be made with theexpress, prior, written permission of Universiti Putra Malaysia.

Copyright c© Universiti Putra Malaysia

© COPYRIG

HT UPM

DEDICATIONS

To my belovedMother and Father

© COPYRIG

HT UPM

Abstract of thesis presented to the Senate of Universiti Putra Malaysia infulfilment of the requirement for the degree of Master of Science

ENERGY AND PERFORMANCE EFFICIENT RESOURCEALLOCATION FOR CLOUD-BASED DATA CENTERS

By

NASRIN AKHTER

July 2015

Chairman: Professor Mohamed Othman, PhDFaculty: Computer Science and Information Technology

Cloud computing provides computing as a service form, due to this moreand more users migrated into the cloud instead of maintaining own physicalinfrastructure. It’s offering hardware, software, infrastructure as a serviceform to the users. Users have to pay as much as they use which also knownas pay as you go model. Cloud computing facilitates sharing resources overthe Internet. It is also a technology revolution, offering flexible IT usage ina cost efficient and pay-per-use manner. Cloud computing is a model forenabling convenient, on-demand network access to a shared pool of config-urable computing resources such as, networks, servers, storage, applications,and services. These resources can be rapidly provisioned and released withminimal management effort. The underlying concept of cloud computing isthe separation of applications from the operating systems and the hardwareon which they run. The Cloud computing proliferation has resulted in theestablishment of large-scale data centers around the world containing thou-sands of computing nodes. But data centers consumed erroneous electricalenergy for its huge hardware infrastructures which also responsible for car-bon emission.

This thesis demonstrates that the dynamic consolidations of virtual machinesin cloud data centers by proposing energy efficient algorithms and policies.The aim of this research is to propose a few new host overload detection andVM selection algorithms in order to minimize energy consumption. Recently,researchers have shown an increased interest to reduce energy consumptionin cloud data centers by consolidation of Virtual Machines. The objective isto develop the quality of service constraints by the under workload for re-

i

© COPYRIG

HT UPM

ducing energy consumption and to improve the application of computingresources. Dynamic VM consolidation leverages fine grained fluctuationsin the application workloads, and continuously reallocates VMs using livemigration to minimize the number of active physical nodes. Energy con-sumption is reduced by dynamically deactivating and reactivating physi-cal nodes to meet the current resource demand. The proposed approachis distributed, scalable, and efficient in managing the energy and perfor-mance trade-off. This research clarifies the concerns about energy savingsin cloud computing, analyzing the factors that make the reduction of energyconsumption. Energy consumption, energy and service level agreement vio-lation, service level agreement violation and number of virtual machine mi-gration was measured to ensure service performance while energy consump-tion reduced. In this study, we found that our proposed algorithms able toreduce energy consumption by 3% and 19% for host overload detection andVM selection algorithm respectively.

ii

© COPYRIG

HT UPM

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysiasebagai memenuhi keperluan untuk ijazah Master Sains

PRESTASI SUMBER DAN TENAGA PERUNTUKAN CEKAPBERASASKAN PUSAT DATA AWAN

Oleh

NASRIN AKHTER

Julai 2015

Pengerusi: Profesor Mohamed Othman, PhDFakulti: Sains Komputer dan Teknologi Maklumat

Pengkomputeran awan menyediakan pengkomputeran sebagai satu bentukperkhidmatan, oleh itu ini lebih ramai pengguna berhijrah ke pengkom-puteran awan dan bukannya mengekalkan infrastruktur fizikal sendiri. Iamenawarkan perkakasan, perisian, infrastruktur sebagai satu bentuk perkhid-matan kepada pengguna. Pengguna perlu membayar sebanyak yang merekagunakan yang juga dikenali sebagai ”pay-as-you-go” model. Pengkomput-eran awan memudahkan perkongsian sumber di Internet. Ia juga meru-pakan satu revolusi teknologi yang menawarkan penggunaan IT yang fleksi-bel dengan cara yang cekap dan kos gaji per penggunaan. Model pengkom-puteran awan adalah untuk memudahkan, permintaan akses rangkaian un-tuk berkongsi sumber pengkomputeran yang dikonfigurasikan seperti,rangkaian, pelayan, penyimpanan, aplikasi dan perkhidmatan. Ia bolehdiperuntukkan dengan cepat dan dibebaskan dengan usaha penguru-san yang minimum. Konsep asas pengkomputeran awan adalah pemisa-han aplikasi dari sistem pengkomputeran dan perkakasan di mana ia di-laksamakan. Dalam kepesatan pengkomputeran awan telah menyebabkanpenubuhan pusat-pusat data secara besar-besaran di seluruh dunia yangmengandungi beribu-ribu nod pengiraan. Tetapi pusat-pusat data yang di-gunakan tenaga elektrik yang salah untuk infrastruktur perkakasan yang be-sar yang juga bertanggungjawab untuk pelepasan karbon.

Tesis ini menunjukkan bahawa penyatuan dinamik Mesin Maya (VM) dipusat-pusat data awan dengan mencadangkan algoritma cekap tenaga dandasar. Baru-baru ini, para penyelidik telah menunjukkan peningkatan minatuntuk mengurangkan penggunaan tenaga di pusat data awan menggunakanpenyatuan Mesin Maya. Matlamatnya adalah untuk membangunkan kekan-gan kualiti perkhidmatan di bawah beban kerja untuk mengurangkan peng-gunaan tenaga dan untuk meningkatkan penggunaan sumber komputer. Di-

iii

© COPYRIG

HT UPM

namik VM penyatuan peringkat umur- permohonan dan peruntukan berteru-san denda secara terperinci bagi turun naik dalam beban kerja VM meng-gunakan penghijrahan secara langsung untuk mengurangkan bilangan nodfizikal aktif. Penggunaan tenaga dikurangkan dengan dinamik Menyahak-tifkan dan mengaktifkan semula nod fizikal bagi memenuhi permintaan sum-ber semasa. Pendekatan yang disyorkan diedarkan secara berskala, dancekap dalam menguruskan tenaga prestasi pertimbangan. Kajian ini menje-laskan kebimbangan tertap penjimatan tenaga dalam perkomputeran awan,menganalisis faktor-faktor yang membuat pengurangan penggunaan tenaga.Penggunaan tenaga, tahap perkhidmatan tenaga dan perjanjian pengaba-ian, perkhidmatan perjanjian tahap pengabaian dan beberapa penghijrahanVM diukur untuk memastikan prestasi perkhidmatan manakala penggu-naan tenaga dikurangkan. Dalam kajian ini, kami mendapati bahawa algo-ritma yang dicadangan dapat mengurangkan penggunaan tenaga masing-masing sebanyak 3% dan 19% untuk mengesan beban hos dan VM algoritmapemilihan.

iv

© COPYRIG

HT UPM

ACKNOWLEDGEMENTS

Research is a beautiful journey, and it delights us with the taste of innova-tion. I would like to thank all of them who help me to complete a successfuljourney. At the outset, I like thanks to the almighty to give the strength tocomplete my Master study. I also thank to all of my lab mates, friends atFSKTM and all staffs at UPM for their helps, suggestions, knowledge shar-ing and great cooperation throughout my research works.

Thanks to the Universiti Putra Malaysia for giving me the change to studyhere and give me the opportunity to meet with diverse Malaysian and In-ternational education and research playground. Specially, I would like tothanks to the chairman of my supervisory committee Professor Dr. Mo-hamed Othman for his proper guidance, suggestions and constructive com-ments to improve my works. I am grateful to him for his coordination andinspiration to complete this research works.

I also thankful to my supervisory committee member Dr. Masnida Hussinfor her valuable comments on my works. I am thankful to all FSKTM facultymembers, lab mates and UPM staffs for their nice support throughout myresearch. I am heartily thankful to my parents, sister, brothers and all otherfamily members for their nice support at all times. Finally, I thank my hus-band for his patience, cooperation, inspiration and supporting me during thestudy period.

v

© COPYRIG

HT UPM

© COPYRIG

HT UPM

This thesis was submitted to the Senate of Universiti Putra Malaysia and hasbeen accepted as fulfilment of the requirement for the degree of Master ofScience.

The members of the Supervisory Committee were as follows:

Mohamed Othman, Ph.D.ProfessorFaculty of Computer Science and Information TechnologyUniversiti Putra Malaysia(Chairperson)

Masnida Hussin, Ph.D.Senior LecturerFaculty of Computer Science and Information TechnologyUniversiti Putra Malaysia(Member)

BUJANG BIN KIM HUAT, Ph.D.Professor and DeanSchool of Graduate StudiesUniversiti Putra Malaysia

Date:

vii

© COPYRIG

HT UPM

© COPYRIG

HT UPM

© COPYRIG

HT UPM

TABLE OF CONTENTS

Page

ABSTRACT i

ABSTRAK iii

ACKNOWLEDGEMENTS v

APPROVAL vi

DECLARATION viii

LIST OF TABLES xiii

LIST OF FIGURES xiv

LIST OF ABBREVIATIONS xvi

CHAPTER

1 INTRODUCTION 11.1 General Overview 11.2 Energy Savings in Cloud Computing 1

1.2.1 Energy-aware Data Centres 21.2.2 Energy Savings in Networks and Protocols 21.2.3 The Effect of Internet Applications 2

1.3 Problem Statement 21.4 Research Objectives 31.5 Research Scope and Delimitation 31.6 Contributions 41.7 Thesis Organization 4

2 LITERATURE REVIEW 62.1 Introduction 62.2 Basic Concepts of Cloud Computing 7

2.2.1 Cloud Computing Architecture 72.2.2 Cloud Computing Reference Models 82.2.3 Cloud Types 9

2.3 Relationship Between Energy and Power 102.4 Related Works 10

2.4.1 Energy Aware System Architecture for Cloud 102.4.2 Energy Efficiency in Traditional and Virtual Data Center 142.4.3 Energy Aware Resource Allocation for Cloud Data Cen-

ter 192.4.4 VM Allocation and Overload Detection Algorithms 222.4.5 VM Selection Algorithms 252.4.6 CPU frequency scaling for energy saving 26

2.5 Power Performance Issues 272.6 Energy saving by using renewable energy 372.7 Green Cloud Computing Model 38

x

© COPYRIG

HT UPM

2.8 Resource Optimization 402.9 Problem of Single VM Migration 412.10 Problem of Dynamic VM Consolidation 412.11 Heuristics for VM Consolidation 422.12 Host Underload and Overload Detection 422.13 Challenges and Future Issues 422.14 Summary 43

3 METHODOLOGY 443.1 Introduction 443.2 Research Framework 443.3 Experimental Setup and Simulation Setup 44

3.3.1 System Model 463.3.2 Experimental design and Overall Framework of Pro-

posed Algorithms 473.3.3 System Power Utilization Model 473.3.4 Workload Characterization 483.3.5 Experimental Setup 493.3.6 Simulation Scenario 49

3.4 Performance Metrics 493.4.1 Energy Consumption 493.4.2 SLA Violation 503.4.3 Number of VM Migrations 513.4.4 Energy and SLA Violations 51

3.5 Physical Resources 513.6 Network Behaviour Modeling 513.7 Summary 52

4 HEURISTICS BASED HOST OVERLOADING DETECTION 534.1 Introduction 534.2 Statistical Methods Used for Energy Savings in Data Center 534.3 Proposed Methods for Defining Host Utilization Threshold 54

4.3.1 Utilization Threshold Using Mean Absolute Deviation 554.3.2 Utilization Threshold Using Midhinge 55

4.4 Algorithm for Host Overloading Detection 564.5 Evaluation of VM Consolidation 57

4.5.1 Simulation Setup of Benchmark Work 574.5.2 Benchmark Work Experimental Results 57

4.6 Simulation with Proposed Host Overloading Detection Algo-rithms 624.6.1 Simulation Setup 624.6.2 Simulation Results and Analysis 63

4.7 Summary 69

xi

© COPYRIG

HT UPM

5 VM SELECTION ALGORITHM 705.1 Introduction 705.2 VM Placement and Selection in Cloud 705.3 The Maximum Migration Time Method 70

5.3.1 General Description 715.3.2 Problem Formulation 715.3.3 Methods for VM Selection for Migration 71

5.4 Algorithm for Virtual Machine Selection 725.5 Simulation Results and Discussions 72

5.5.1 Energy Consumption 735.5.2 SLA Time Per Active Host 785.5.3 Virtual Machine Migration 81

5.6 Summary 84

6 CONCLUSION AND FUTURE RESEARCH DIRECTIONS 856.1 Conclusion 856.2 Future Works 86

REFERENCES 87

BIODATA OF STUDENT 98

LIST OF PUBLICATIONS 100

xii

© COPYRIG

HT UPM

LIST OF TABLES

Table Page

2.1 Approach of some key related literature on energy-efficiencyin Traditional and Virtual Data Centers 18

2.2 List of CPU governors 262.3 Summary of the works on energy aware cloud architecture

and algorithms. 302.4 Research summary of research addressed DCs powering with

renewable energy 39

3.1 Power consumption of the servers at 0% to 50% load level(kWh) (Beloglazov and Buyya, 2012). 48

3.2 Power consumption of the servers at 60% to 100% load level(kWh) (Beloglazov and Buyya, 2012). 48

3.3 CPU utilization of workload data (Beloglazov and Buyya, 2012). 483.4 Configuration of instances. 49

4.1 Comparison of various statistical methods. 544.2 Host Overloading Detection. 56

5.1 Comparison of various methods for VM selection. 725.2 Energy Aware VM Selection (EAVMS). 73

xiii

© COPYRIG

HT UPM

LIST OF FIGURES

Figure Page

2.1 CO2 emission of Data center, Networks and Peripherals by2020Webb (2008). 6

2.2 Schematic definition of cloud computing (Khorshed et al., 2012). 72.3 Architecture of Cloud Computing (Buyya et al., 2013). 82.4 The system Architecture of the resource management system

for Cloud data centers (Beloglazov and Buyya, 2010). 112.5 High-level Market-oriented cloud Architecture (Buyya et al.,

2009). 122.6 The high-level energy aware system architecture for resource

allocation for cloud data centers (Beloglazov et al., 2012). 132.7 Taxonomy of energy aware resource allocation in cloud data

center. 152.8 Architecture of energy consumption analysis tool (Chen et al.,

2012). 212.9 System architecture for VM allocation (Cao et al., 2012). 232.10 VM allocation model (Rodero et al., 2012). 242.11 Integrated green cloud architecture (Hulkury and Doomun,

2013). 40

3.1 Framework of the Research. 453.2 The System Model. 463.3 Framework of proposed algorithms. 473.4 Scenario of the Simulation. 503.5 Flow of network communication. 51

4.1 Energy Consumption (kWh) of Existing Algorithms. 584.2 Energy and SLA Violation of Existing Algorithms. 584.3 Performance Degradation due to Migration of Existing Algo-

rithms. 604.4 SLA Time Per Active Host of Existing Algorithms. 604.5 SLA Violation of Existing Algorithms. 614.6 VM Migration of Existing Algorithms. 624.7 Result Comparison of Mean Absolute Deviation (a) Energy

Consumption (b) Energy and SLA Violation. 644.8 Result Comparison of Mean Absolute Deviation (a) SLA Vio-

lation (b) SLA Time Per Active Host (c) Performance Degrada-tion due to Migration (d) VM Migration. 65

4.9 Result Comparison of Midhinge (a) Energy Consumption (b)Energy and SLA Violation. 67

4.10 Result Comparison of Midhinge (a) SLA Violation (b) SLATime Per Active Host (c) Performance Degradation due to Mi-gration (d) VM Migration. 68

xiv

© COPYRIG

HT UPM

5.1 Result Comparison of Energy Consumption (a) Maximum Mi-gration Time with IQR (b) Maximum Migration Time with LRR. 74

5.2 Result Comparison of Energy Consumption (a) Maximum Mi-gration Time with MAD (b) Maximum Migration Time withTHR 0.8. 75

5.3 Result Comparison of Energy Consumption (a) Maximum Mi-gration Time with THR 1.0 (b) Maximum Migration Time withLR. 77

5.4 Result Comparison of SLA Time per Active Host (a) Maxi-mum Migration Time with IQR (b) Maximum Migration Timewith LRR (c)Maximum Migration Time with MAD (d) Maxi-mum Migration Time with THR 0.8. 79

5.5 Result Comparison of SLA Time per Active Host (a) Maxi-mum Migration Time with THR 1.0 (b) Maximum MigrationTime with LR. 80

5.6 Result Comparison of VM Migration (a) Maximum MigrationTime with IQR (b) Maximum Migration Time with LRR (c)MaximumMigration Time with MAD (d) Maximum Migration Time withTHR 0.8. 82

5.7 Result Comparison of VM Migration (a) Maximum MigrationTime with THR 1.0 (b) Maximum Migration Time with LR. 83

xv

© COPYRIG

HT UPM

LIST OF ABBREVIATIONS

BFD Best Fit DecreasingCPU Central Processing UnitDVFS Dynamic Voltage and Frequency ScalingDVFS Dynamic Voltage and Frequency ScalingDVS Dynamic Voltage ScalingEATM Energy Aware Thermal ManagementEAVMS Energy Aware VM SelectionEC Energy Congruent strategyECI Energy Congruent IndexESV Energy and SLA ViolationFF First-FitGbPS Giga bit Per SecondHPC High Performance ComputingIaaS Infrastructure as a ServiceIQR Interquartile RangeIT Information TechnologykWh Kilo Watt Per HourLR Local RegressionLRR Local Regression RobustMAD Median Absolute DeviationMBFD Modified Best Fit DecreasingMC Maximum CorrelationMeAD Mean Absolute DeviationMH MidhingeMIPS Millions Instruction Per Second

xvi

© COPYRIG

HT UPM

MMT Minimum Migration TimeMPI Message Passing InterfaceMxMT Maximum Migration TimeNAS Network Attached StorageNPA Non Power AwareOS Operating SystemPABFD Power Aware Best Fit DecreasingPDM Performance Degradation due to MigrationQoS Quality of ServiceRAILS Redundant Array for Inexpensive Load SharingRC Random ChoiceRnd RandomROI Return on InvestmentRS Random SelectionSLA Service Level AgreementSLATAH SLA Time per Active HostSLAV SLA ViolationTA Temperature AwareTHR ThresholdVM Virtual MachineVMM Virtual Machine MonitorVOVO Vary On Vary OffW WattWh Watt per hour

xvii

© COPYRIG

HT UPM

© COPYRIG

HT UPM

CHAPTER 1

INTRODUCTION

1.1 General Overview

The cloud computing provide user to use computing resources as their de-mand. This pay-as-you-go model saves upfront costs of purchasing IT in-frastructure. Organizations can outsource their computational needs to thecloud and it can reduce hardware, software and maintenance cost. The enor-mous growing of cloud computing resulted in the establishment of large-scale data centers containing thousands of computing nodes. These datacenter consume huge electrical energy. An average data center consumes asmuch energy as 25,000 households (Kaplan et al., 2008).

Energy consumption of data center could be increased due to the lack ofhardware power efficiency. However, inefficient usage of hardware resourcesalso leads to high energy consumption. Generally most of the time serversoperate at 10-50% of their full capacity (Barroso and Holzle, 2007). Even incompletely idle servers still consume about 70% of their peak power (Fanet al., 2007). Hence, servers could not be underutilized if concerned abouthigh energy consumption.

Virtualization technology has the capability to resolve energy inefficiency is-sue. Through virtualization technology it is possible to create several VirtualMachines (VMs) instances into a single physical machine. In this way re-source utilization greatly improved, energy consumption reduced and profitmargin of the cloud service providers increased. Following this techniqueunderutilized servers could be consolidate and keeps unutilized servers insleep or hibernate mode. Instead of using all servers, consolidation of VMundertaken as their system resource needs dynamically and ensure mini-mum number of physical resource usage.

1.2 Energy Savings in Cloud Computing

Cloud computing is trending deployment in a business environment andrunning other many popular sites. Reducing energy consumption is themost challenging research problem in the world of cloud computing. It isnecessary to implement energy efficient method for all system layers in or-der to reduce energy consumption of a system. Services are running dependon various demand of users. Beside faster and reliable service, users mayconcern about energy efficiency. A management system could be developedin order to track energy-efficient service for future load distribution (Mooreet al., 2005).

© COPYRIG

HT UPM

1.2.1 Energy-aware Data Centres

Nowadays virtualization is an important technology which enables energy-efficient services for servers in data centres. Idle servers in data centers put insleep mode is one of the way to save energy. Every operative host producesheat. But if hardware consolidation initiate and migrate to another under-loaded host then it is possible to reduce energy consumption and deliverbetter energy efficiency. Cooling system of data center also responsible forincurring higher energy consumption cost. Places may be chosen with lowertemperature in a region to build data center or develop new techniques forcooling system of data center for reducing power consumption.

1.2.2 Energy Savings in Networks and Protocols

A particular amount of energy consumed by network communication whichshown by the research. The energy consumption of network communica-tion must be improved by deal with QoS, performance and energy savingtrade-off (Gelenbe and Silvestri, 2009). The energy-efficient process of net-work elements may be increased if a way found which could be redesign oroptimized of network protocols. The network devices are allocated to assignservices to such devices that are energy efficient and these devices are work-ing while other devices are going to sleep mode. In other words, networkservice allocation could be transfer from the energy inefficient devices to en-ergy efficient devices. However, out-of-band signalling should be consideredin order to develop protocols with energy awareness.

1.2.3 The Effect of Internet Applications

One large application region is Information circulation in the Internet. Butnow this time most portion of Internet traffic are ruled by web based, video-on-demand, peer-to-peer and web services and consistently involved morethan 85% of the Internet traffic for several years (Berl et al., 2010). Internettraffic will raise up more significantly when the cloud computing will bea major platform for creating and retrieving information. The ideal opera-tion for replication of content and propagation algorithms will be energy, asprimary factor. Therefore, energy efficiency storage, computation, commu-nication and performance, energy trade-offs should be reconsidered with therespect of cloud computing.

1.3 Problem Statement

Flexible user requirements and on demand access to the resources makescloud computing acceptable to the various types of consumer. Moving tocloud save infrastructure cost and eliminate system maintenance technicaleffort for the users. Cloud providers have to ensure service flexibility andmaintain different types of user requirements. Providers also required tohides underlying infrastructure from the cloud users. So, deliver best perfor-mance is main concern for the cloud infrastructure deployments. Fortunately

2

© COPYRIG

HT UPM

providers are able to achieve this goal without considering energy issues. Asa result energy consumption vastly increased as the cloud domain expands.

Paying attention on energy consumption sustain the growth of cloud com-puting in near future. Otherwise underlying cloud infrastructure causedhigher energy consumption. Resources of data center must be maintainedby energy efficient manner. Solution of the stated problem will lead to theGreen cloud computing dream.

Several works had done for the cluster and the cloud for improving energyefficiency. Load balancing and unbalancing decisions using double thresh-old advances energy efficiency for the cluster (Pinheiro et al., 2001). How-ever, it has lack of power, performance trade-off and experiments with thereal life workload. VM allocation and selection algorithms for the cloud wereproposed by Beloglazov et al. (2012). These algorithms were experimentedin real life workload, but suggested for further development of these algo-rithms to achieve more energy efficiency as well as investigation with morecomplex workloads like Markov Chains also suggested.

In this research, we are going to propose energy efficient resource manage-ment algorithm for cloud data center by applying various statistical meth-ods. We work with host overload detection and VM selection algorithms forthe cloud data center.

1.4 Research Objectives

The core objective of this research is to propose a heuristics energy and per-formance aware resource allocation algorithms. These algorithms will helpto reduce energy consumption by utilizing previous resource allocation his-tory. Beside this, proposed algorithms will defence with performance degra-dation while energy consumption minimized. Proposed algorithms are moreenergy efficient comparing existing algorithms and it also minimized Returnof Investment (ROI). To fulfil our aim we achieve following objectives:

1. Propose host overload detection algorithm by using dynamic VM con-solidation technique to improve performance and energy efficiency.

2. Propose VM selection algorithm to minimize energy consumption whichimprove energy efficiency by using heuristic based efficient VM selec-tion.

1.5 Research Scope and Delimitation

Many factors are related to energy efficient resource allocation such as en-ergy consumption of memory, virtual network topology, safe operating tem-perature of system hardware, application allocation for efficient usage of sys-tem resources etc. Because power consumption of cloud data center varied

3

© COPYRIG

HT UPM

depends on these factors. The proper decision of VM allocation and selec-tion helps to reduce energy consumption by changing power state (sleep orturn off) of unallocated or partially allocated physical server. Both allocationand selection factors are responsible for this process. Employment of thisresearch is with VM allocation and selection algorithms in cloud based datacenter.

Energy efficiency of hardware is not considered in this work. However, itwas consider resource management of cloud data center in energy and per-formance efficient manner. Proposed algorithms are evaluated in simulationenvironment. Implementation of proposed algorithms in live cloud is thebeyond of this research scope. But, simulation based study ensures efficientmanagement of cloud infrastructure (Calheiros et al., 2011). It was assumedthat no hardware failure occurred during the system operation. Modern datacenter equipped with redundant system resources so hardware failure doesnot take into place and hardware failure could be considered as not signifi-cant.

1.6 Contributions

Randomize online algorithm shows better performance comparing with thedeterministic algorithm (Ben-David et al., 1994). We proposed heuristic basedresource allocation algorithms for new policy to minimize energy consump-tion and enhance performance. After reviewing previously proposed re-source allocation algorithms, we propose host overload detection and VMselection algorithms (Pinheiro et al., 2001; Beloglazov and Buyya, 2010; Bel-oglazov et al., 2012; Beloglazov and Buyya, 2012).

We tested proposed algorithms in a simulated environment by using Plan-etLab workload because PlanetLab workload cosidered as real life work-load collected from thousands servers all over the world (Beloglazov andBuyya, 2012). Proposed host overload detection algorithms improve the per-formance of resource allocation while proposed VM selection algorithm min-imizes energy consumption. The main contributions of this research are thefollowings:

1. Proposed host overload detection algorithms which enhance perfor-mance on energy efficiency comparing with previously proposed algo-rithms.

2. Proposed VM selection algorithm which reduces energy consumptionof cloud data center comparing with previously proposed algorithms

1.7 Thesis Organization

A general overview of the energy efficient resource allocation problem presentsin Chapter 1. Objectives and contribution of this research also described in

4

© COPYRIG

HT UPM

the same chapter.

In chapter 2, literature review of the resource allocation of cloud data centeris presented. Cloud computing and energy efficiency of cloud also describedin this chapter. We also investigated some energy efficient system architec-ture and resource allocation algorithms in order to identify challenges andresearch issues.

Simulation environments and experimental setup for resource allocation al-gorithm is presents in chapter 3. Methodology of our research with somekey performance metrics presented in this chapter. Four physical machinesused for the experiments, detail configuration of these physical machines aredescribe in this chapter.

To consolidate VMs, overloaded host have to find after that, some VM fromoverloaded host will be migrated to the underloaded hosts. Host overloaddetection algorithm proposed in chapter 4 with experimental results andanalysis. This algorithm considered as first finding of this research.

Chapter 5 proposes energy efficient VM selection algorithm. Formulation ofthe technique of proposed algorithm described in this chapter. Simulationresults and the analysis of obtained results are describes at the end of thischapter.

Conclusion of our research and further research direction discuss in Chapter6. References, student bio-data and list of publications are included at theend of this thesis.

5

© COPYRIG

HT UPM

REFERENCES

Alahmadi, A., Che, D., Khaleel, M., Zhu, M. M., and Ghodous, P. (2015). Aninnovative energy-aware cloud task scheduling framework. In CloudComputing (CLOUD), 2015 IEEE 8th International Conference on, pages493–500. IEEE.

Amazon (2013). Amazon Elastic Compute Cloud.https://aws.amazon.com/ec2/pricing/ [Last Accesed: 28 Dec 2013].

Andrew, L. L., Lin, M., and Wierman, A. (2010). Optimality, fairness, androbustness in speed scaling designs. In ACM SIGMETRICS PerformanceEvaluation Review, volume 38, pages 37–48. ACM.

Appleby, K., Fakhouri, S., Fong, L., Goldszmidt, G., Kalantar, M., Krishnaku-mar, S., Pazel, D. P., Pershing, J., and Rochwerger, B. (2001). Oceano-slabased management of a computing utility. In Integrated Network Manage-ment Proceedings, 2001 IEEE/IFIP International Symposium on, pages 855–868. IEEE.

Arjmandi, M., Balouchzahi, N. M., Raahemifar, K., Fathy, M., and Akbari, A.(2016). Reduction of power consumption in cloud data centers via dy-namic migration of virtual machines. International Journal of Informationand Education Technology, 6(4):286–290.

Aupy, G., Benoit, A., and Robert, Y. (2012). Energy-aware scheduling un-der reliability and makespan constraints. In High Performance Computing(HiPC), 2012 19th International Conference on, pages 1–10. IEEE.

Ayguade, E. and Torres, J. (2009). Creating power-aware middleware forenergy-efficient data centers. ERCIM News, 79:27–28.

Barak, A. and La’adan, O. (1998). The mosix multicomputer operating sys-tem for high performance cluster computing. Future Generation ComputerSystems, 13(45):361–372.

Barford, P. and Crovella, M. (1998). Generating representative web work-loads for network and server performance evaluation. SIGMETRICSPerform. Eval. Rev., 26(1):151–160.

Barroso, L. A. and Holzle, U. (2007). The case for energy-proportional com-puting. IEEE computer, 40(12):33–37.

Baskaran, S. and Thambidurai, P. (2011). A Dynamic Slack Management Tech-nique for Real-Time System with Precedence and Resource Constraints, vol-ume 198 of Communications in Computer and Information Science, chap-ter 38, pages 365–374. Springer Berlin Heidelberg.

Beitelmal, A. and Fabris, D. (2014). Servers and data centers energy perfor-mance metrics. Energy and Buildings, 80:562–569.

87

© COPYRIG

HT UPM

Beloglazov, A., Abawajy, J., and Buyya, R. (2012). Energy-aware resourceallocation heuristics for efficient management of data centers for cloudcomputing. Future Generation Computer Systems, 28(5):755–768.

Beloglazov, A. and Buyya, R. (2010). Energy efficient resource manage-ment in virtualized cloud data centers. In Proceedings of the 2010 10thIEEE/ACM International Conference on Cluster, Cloud and Grid Computing,pages 826–831. IEEE Computer Society.

Beloglazov, A. and Buyya, R. (2012). Optimal online deterministic algorithmsand adaptive heuristics for energy and performance efficient dynamicconsolidation of virtual machines in cloud data centers. Concurrency andComputation: Practice and Experience, 24(13):1397–1420.

Beloglazov, A., Buyya, R., Lee, Y. C., and Zomaya, A. (2011). A taxonomy andsurvey of energy-efficient data centers and cloud computing systems.Advances in Computers, 82:47–111.

Ben-David, S., Borodin, A., Karp, R., Tardos, G., and Wigderson, A. (1990).On the power of randomization in online algorithms. In Proceedings ofthe twenty-second annual ACM symposium on Theory of computing, pages379–386. ACM.

Ben-David, S., Borodin, A., Karp, R., Tardos, G., and Wigderson, A. (1994).On the power of randomization in on-line algorithms. Algorithmica,11(1):2–14.

Berl, A., Gelenbe, E., Di Girolamo, M., Giuliani, G., De Meer, H., Dang, M. Q.,and Pentikousis, K. (2010). Energy-efficient cloud computing. The com-puter journal, 53(7):1045–1051.

Berral, J. L., Goiri, i., Nou, R., Juli, F., Guitart, J., Gavald, R., and Torres,J. (2010). Towards energy-aware scheduling in data centers using ma-chine learning. In Proceedings of the 1st International Conference on energy-Efficient Computing and Networking, pages 215–224. ACM.

Bird, S., Achuthan, A., Maatallah, O. A., Hu, W., Janoyan, K., Kwasinski,A., Matthews, J., Mayhew, D., Owen, J., and Marzocca, P. (2014). Dis-tributed (green) data centers: A new concept for energy, computing, andtelecommunications. Energy for Sustainable Development, 19:83–91.

Borodin, A. and El-Yaniv, R. (1998). Online computation and competitive analy-sis. Cambridge University Press.

Buyya, R., Vecchiola, C., and Selvi, S. T. (2013). Mastering Cloud Computing,Chapter 2 - Principles of Parallel and Distributed Computing. Morgan Kauf-mann, Boston.

Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., and Brandic, I. (2009). Cloudcomputing and emerging it platforms: Vision, hype, and reality for de-livering computing as the 5th utility. Future Generation Computer Systems,25(6):599–616.

88

© COPYRIG

HT UPM

Calheiros, R. N., Buyya, R., and De Rose, C. A. (2009). A heuristic for map-ping virtual machines and links in emulation testbeds. In Parallel Pro-cessing, 2009. ICPP’09. International Conference on, pages 518–525. IEEE.

Calheiros, R. N., Ranjan, R., Beloglazov, A., De Rose, C. A., and Buyya,R. (2011). Cloudsim: a toolkit for modeling and simulation of cloudcomputing environments and evaluation of resource provisioning algo-rithms. Software: Practice and Experience, 41(1):23–50.

Cao, J., Wu, Y., and Li, M. (2012). Energy Efficient Allocation of Virtual Machinesin Cloud Computing Environments Based on Demand Forecast, volume 7296of Lecture Notes in Computer Science, chapter 12, pages 137–151. SpringerBerlin Heidelberg.

Cardosa, M., Korupolu, M. R., and Singh, A. (2009). Shares and utilities basedpower consolidation in virtualized server environments. In IntegratedNetwork Management, 2009. IM’09. IFIP/IEEE International Symposium on,pages 327–334. IEEE.

Carli, T., Henriot, S., Cohen, J., and Tomasik, J. (2016). A packing problemapproach to energy-aware load distribution in clouds. Sustainable Com-puting: Informatics and Systems, 9:20–32.

CE (2013). Center of expertise for energy efficiency in data centers. Retrievedfrom http://datacenters.lbl.gov/ [Last Accesed: 25 Dec 2013].

Chase, J. S., Anderson, D. C., Thakar, P. N., Vahdat, A. M., and Doyle,R. P. (2001). Managing energy and server resources in hosting centers.In ACM SIGOPS Operating Systems Review, volume 35, pages 103–116.ACM.

Chen, F., Schneider, J.-G., Yang, Y., Grundy, J., and He, Q. (2012). An en-ergy consumption model and analysis tool for cloud computing envi-ronments. In Green and Sustainable Software (GREENS), 2012 First Inter-national Workshop on, pages 45–50. IEEE.

Chen, S., Irving, S., and Peng, L. (2015). Operational cost optimization forcloud computing data centers using renewable energy. IEEE SystemsJournal, pages 1–12.

Chiaraviglio, L. and Matta, I. (2010). Greencoop: cooperative green rout-ing with energy-efficient servers. In Proceedings of the 1st InternationalConference on Energy-Efficient Computing and Networking, pages 191–194.ACM.

CoMon (2014). CoMon Project. http://comon.cs.princeton.edu/ [Last Ac-cesed: 23 Dec 2013].

Cordeschi, N., Shojafar, M., and Baccarelli, E. (2013). Energy-saving self-configuring networked data centers. Computer Networks, 57(17):3479–3491.

89

© COPYRIG

HT UPM

Cupertino, L., Da Costa, G., Oleksiak, A., Pia, W., Pierson, J.-M., Salom, J.,Siso, L., Stolf, P., Sun, H., Zilio, T., et al. (2015). Energy-efficient, thermal-aware modeling and simulation of data centers: The coolemall approachand evaluation results. Ad Hoc Networks, 25:535–553.

Da Costa, G., De Assuncao, M. D., Gelas, J.-P., Georgiou, Y., Lefevre, L., Org-erie, A.-C., Pierson, J.-M., Richard, O., and Sayah, A. (2010). Multi-facetapproach to reduce energy consumption in clouds and grids: the green-net framework. In Proceedings of the 1st international conference on energy-efficient computing and networking, pages 95–104. ACM.

Depoorter, V., Oro, E., and Salom, J. (2015). The location as an energy effi-ciency and renewable energy supply measure for data centres in europe.Applied Energy, 140:338–349.

Dodonov, E. and de Mello, R. F. (2010). A novel approach for distributedapplication scheduling based on prediction of communication events.Future Generation Computer Systems, 26(5):740–752.

Doh, I. H., Kim, Y. J., Kim, E., Choi, J., Lee, D., and Noh, S. H. (2013). Towardsgreener data centers with storage class memory. Future Generation Com-puter Systems, 29(8):1969–1980.

Drozdowski, M. (2009). Scheduling with Communication Delays, chapter 6,pages 209–299. Computer Communications and Networks. SpringerLondon.

Elnozahy, E. M., Kistler, M., and Rajamony, R. (2003). Energy-efficient serverclusters, pages 179–197. Springer.

Fan, X., Weber, W.-D., and Barroso, L. A. (2007). Power provisioning fora warehouse-sized computer. In ACM SIGARCH Computer ArchitectureNews, volume 35, pages 13–23. ACM.

Feitelson, D. (2002). Workload Modeling for Performance Evaluation, volume2459 of Lecture Notes in Computer Science, chapter 6, pages 114–141.Springer Berlin Heidelberg.

Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Lee, G., Patterson,D., Rabkin, A., and Stoica, I. (2009). Above the clouds: A berkeley viewof cloud computing. Dept. Electrical Eng. and Comput. Sciences, Universityof California, Berkeley, Rep. UCB/EECS, 28.

Galizia, A. and Quarati, A. (2012). Job allocation strategies for energy-aware and efficient grid infrastructures. Journal of Systems and Software,85(7):1588–1606.

Gandhi, A., Harchol-Balter, M., Das, R., and Lefurgy, C. (2009). Optimalpower allocation in server farms. SIGMETRICS Perform. Eval. Rev.,37(1):157–168.

90

© COPYRIG

HT UPM

Gelenbe, E. and Silvestri, S. (2009). Reducing power consumption in wirednetworks. In Computer and Information Sciences, 2009. ISCIS 2009. 24thInternational Symposium on, pages 292–297. IEEE.

Ghafari, S. M., Fazeli, M., Patooghy, A., and Rikhtechi, L. (2013). Bee-mmt:A load balancing method for power consumption management in cloudcomputing. In Contemporary Computing (IC3), 2013 Sixth InternationalConference on, pages 76–80. IEEE.

Ghamkhari, M. and Mohsenian-Rad, H. (2013). Energy and performancemanagement of green data centers: A profit maximization approach.Smart Grid, IEEE Transactions on, 4(2):1017–1025.

Goiri, I., Katsak, W., Le, K., Nguyen, T. D., and Bianchini, R. (2013). Parasoland greenswitch: Managing datacenters powered by renewable energy.In ACM SIGARCH Computer Architecture News, volume 41, pages 51–64.ACM.

Goiri, I., Katsak, W., Le, K., Nguyen, T. D., and Bianchini, R. (2014). De-signing and managing datacenters powered by renewable energy. IEEEMicro, (3):8–16.

Gomaa, M., Powell, M. D., and Vijaykumar, T. N. (2004). Heat-and-run:leveraging smt and cmp to manage power density through the oper-ating system. SIGARCH Comput. Archit. News, 32(5):260–270.

Greenberg, A., Hamilton, J., Maltz, D. A., and Patel, P. (2008). The cost of acloud: research problems in data center networks. SIGCOMM Comput.Commun. Rev., 39(1):68–73.

Guo, C., Lu, G., Wang, H. J., Yang, S., Kong, C., Sun, P., Wu, W., and Zhang, Y.(2010). Secondnet: a data center network virtualization architecture withbandwidth guarantees. In Proceedings of the 6th International Conference,pages 15:1–15:12. ACM.

Gupta, M. and Singh, S. (2003). Greening of the internet. In Proceedings of the2003 conference on Applications, technologies, architectures, and protocols forcomputer communications, pages 19–26. ACM.

Gyarmati, L. and Trinh, T. A. (2010). How can architecture help to reduceenergy consumption in data center networking? In Proceedings of the1st International Conference on Energy-Efficient Computing and Networking,pages 183–186. ACM.

Heo, S., Barr, K., and Asanovic, K. (2003). Reducing power density throughactivity migration. In Low Power Electronics and Design, 2003. ISLPED’03.Proceedings of the 2003 International Symposium on, pages 217–222. IEEE.

Hulkury, M. and Doomun, M. (2013). Integrated green cloud computingarchitecture. pages 269–274.

91

© COPYRIG

HT UPM

Ibrahim, S., Phan, T.-D., Carpen-Amarie, A., Chihoub, H.-E., Moise, D., andAntoniu, G. (2016). Governing energy consumption in hadoop throughcpu frequency scaling: an analysis. Future Generation Computer Systems,54:219–232.

Jung, G., Joshi, K. R., Hiltunen, M. A., Schlichting, R. D., and Pu, C. (2008).Generating adaptation policies for multi-tier applications in consoli-dated server environments. In Autonomic Computing, 2008. ICAC’08. In-ternational Conference on, pages 23–32. IEEE.

Jung, G., Joshi, K. R., Hiltunen, M. A., Schlichting, R. D., and Pu, C. (2009).A cost-sensitive adaptation engine for server consolidation of multitier applica-tions, pages 163–183. Springer.

Kaplan, J. M., Forrest, W., and Kindler, N. (2008). Revolutionizing data cen-ter energy efficiency. Technical report, Technical report, McKinsey &Company.

Kecskemeti, G. (2015). Dissect-cf: a simulator to foster energy-awarescheduling in infrastructure clouds. Simulation Modelling Practice andTheory, 58:188–218.

Khan, S. U. and Ahmad, I. (2009). A cooperative game theoretical techniquefor joint optimization of energy consumption and response time in com-putational grids. Parallel and Distributed Systems, IEEE Transactions on,20(3):346–360.

Khorshed, M. T., Ali, A. B. M. S., and Wasimi, S. A. (2012). A survey ongaps, threat remediation challenges and some thoughts for proactive at-tack detection in cloud computing. Future Generation Computer Systems,28(6):833–851.

Klingert, S., Niedermeier, F., Dupont, C., Giuliani, G., Schulze, T., andde Meer, H. (2015). Renewable energy-aware data centre operations forsmart cities the dc4cities approach. In Smart Cities and Green ICT Systems(SMARTGREENS), 2015 International Conference on, pages 1–9. IEEE.

Ko, Y. M. and Cho, Y. (2014). A distributed speed scaling and load balanc-ing algorithm for energy efficient data centers. Performance Evaluation,79:120–133.

Koseoglu, M. and Karasan, E. (2010). Joint resource and network schedul-ing with adaptive offset determination for optical burst switched grids.Future Generation Computer Systems, 26(4):576–589.

Kuehn, P. J. and Mashaly, M. E. (2015). Automatic energy efficiency manage-ment of data center resources by load-dependent server activation andsleep modes. Ad Hoc Networks, 25:497–504.

Kumar, M. R. V. and Raghunathan, S. (2016). Heterogeneity and thermalaware adaptive heuristics for energy efficient consolidation of virtual

92

© COPYRIG

HT UPM

machines in infrastructure clouds. Journal of Computer and System Sci-ences, 82(2):191–212.

Kumar, S., Talwar, V., Kumar, V., Ranganathan, P., and Schwan, K. (2009).vmanage: loosely coupled platform and virtualization management indata centers. In Proceedings of the 6th international conference on Autonomiccomputing, pages 127–136. ACM.

Kusic, D., Kephart, J., Hanson, J., Kandasamy, N., and Jiang, G. (2009). Powerand performance management of virtualized computing environmentsvia lookahead control. Cluster Computing, 12(1):1–15.

Lajevardi, B., Haapala, K. R., and Junker, J. F. (2015). Real-time monitoringand evaluation of energy efficiency and thermal management of datacenters. Journal of Manufacturing Systems, 37:511–516.

Lee, Y. C. and Zomaya, A. Y. (2011). Energy conscious scheduling for dis-tributed computing systems under different operating conditions. Par-allel and Distributed Systems, IEEE Transactions on, 22(8):1374–1381.

Leon, X. and Navarro, L. (2013). A stackelberg game to derive the limits ofenergy savings for the allocation of data center resources. Future Gener-ation Computer Systems, 29(1):74–83.

Li, H. (2009). Workload dynamics on clusters and grids. The Journal of Super-computing, 47(1):1–20.

Li, K. (2012). Optimal configuration of a multicore server processor for man-aging the power and performance tradeoff. The Journal of Supercomput-ing, 61(1):189–214.

Liu, W., Duan, Y., and Du, W. (2012a). An energy efficient clustering-basedscheduling algorithm for parallel tasks on homogeneous dvs-enabledclusters. In Computer Supported Cooperative Work in Design (CSCWD),2012 IEEE 16th International Conference on, pages 575–582. IEEE.

Liu, Z., Chen, Y., Bash, C., Wierman, A., Gmach, D., Wang, Z., Marwah, M.,and Hyser, C. (2012b). Renewable and cooling aware workload man-agement for sustainable data centers. In ACM SIGMETRICS PerformanceEvaluation Review, volume 40, pages 175–186. ACM.

Luo, J.-p., Li, X., and Chen, M.-r. (2014). Hybrid shuffled frog leaping algo-rithm for energy-efficient dynamic consolidation of virtual machines incloud data centers. Expert Systems with Applications, 41(13):5804–5816.

Luo, L., Wu, W., Tsai, W., Di, D., and Zhang, F. (2013). Simulation of powerconsumption of cloud data centers. Simulation Modelling Practice andTheory, 39:152–171.

Malviya, P., Agrawal, S., and Singh, S. (2014). An effective approach for al-locating vms to reduce the power consumption of virtualized cloud en-vironment. In Communication Systems and Network Technologies (CSNT),2014 Fourth International Conference on, pages 573–577. IEEE.

93

© COPYRIG

HT UPM

Markoff, J. and Hansell, S. (2006). Hiding in plain sight, google seeks morepower. New York Times, 14:1–2.

Mateo, J., Vilaplana, J., Pla, L. M., Lerida, J. L., and Solsona, F. (2014). A greenstrategy for federated and heterogeneous clouds with communicatingworkloads. The Scientific World Journal, 2014:1–9.

Medina, A., Lakhina, A., Matta, I., and Byers, J. (2001). Brite: An approachto universal topology generation. In Modeling, Analysis and Simulation ofComputer and Telecommunication Systems, 2001. Proceedings. Ninth Interna-tional Symposium on, pages 346–353. IEEE.

Meisner, D., Gold, B. T., and Wenisch, T. F. (2009). Powernap: eliminatingserver idle power. SIGPLAN Not., 44(3):205–216.

Minas, L. and Ellison, B. (2009). Energy efficiency for information technology:How to reduce power consumption in servers and data centers. Intel Press.

Moore, J. D., Chase, J. S., Ranganathan, P., and Sharma, R. K. (2005). Mak-ing scheduling” cool”: Temperature-aware workload placement in datacenters. In USENIX annual technical conference, General Track, pages 61–75.

Nathuji, R. and Schwan, K. (2007). Virtualpower: coordinated power man-agement in virtualized enterprise systems. SIGOPS Oper. Syst. Rev.,41(6):265–278.

Neugebauer, R. and McAuley, D. (2001). Energy is just another resource:Energy accounting and energy pricing in the nemesis os. In Hot Topicsin Operating Systems, 2001. Proceedings of the Eighth Workshop on, pages67–72. IEEE.

Newman, P. and Kotonya, G. (2015). A resource-aware framework forresource-constrained service-oriented systems. Future Generation Com-puter Systems, 47:161–175.

Nguyen, K.-K., Cheriet, M., Lemay, M., Savoie, M., and Ho, B. (2013). Pow-ering a data center network via renewable energy: A green testbed. In-ternet Computing, IEEE, 17(1):40–49.

Nikolopoulos, D. S. (2009). Green building blocks-software stacks for energy-efficient clusters and data centers. ERCIM News, 79:29–30.

NRDC (2013). America’s data centers consuming and wasting grow-ing amounts of energy. http://www.nrdc.org/energy/data-center-efficiency-assessment.asp [Last Accesed: 28 Oct 2015].

Oro, E., Depoorter, V., Pflugradt, N., and Salom, J. (2015). Overview of directair free cooling and thermal energy storage potential energy savings indata centres. Applied Thermal Engineering, 85:100–110.

94

© COPYRIG

HT UPM

Palis, M. A., Jing-Chiou, L., and Wei, D. S. L. (1996). Task clustering andscheduling for distributed memory parallel architectures. Parallel andDistributed Systems, IEEE Transactions on, 7(1):46–55.

Panarello, C., Lombardo, A., Schembra, G., Chiaraviglio, L., and Mellia, M.(2010). Energy saving and network performance: a trade-off approach.In Proceedings of the 1st International Conference on Energy-Efficient Com-puting and Networking, pages 41–50. ACM.

Park, K. and Pai, V. S. (2006). Comon: a mostly-scalable monitoring systemfor planetlab. ACM SIGOPS Operating Systems Review, 40(1):65–74.

Pelley, S., Meisner, D., Wenisch, T. F., and VanGilder, J. W. (2009). Under-standing and abstracting total data center power. In Workshop on Energy-Efficient Design, pages 1–6. Austin, Texas.

Piao, J. T. and Yan, J. (2010). A network-aware virtual machine placementand migration approach in cloud computing. In Grid and CooperativeComputing (GCC), 2010 9th International Conference on, pages 87–92. IEEE.

Pinheiro, E. and Bianchini, R. (1999). Nomad: a scalable operating system forclusters of uni-and multiprocessors. In Cluster Computing, 1999. Proceed-ings. 1st IEEE Computer Society International Workshop on, pages 247–254.IEEE.

Pinheiro, E., Bianchini, R., Carrera, E. V., and Heath, T. (2001). Load bal-ancing and unbalancing for power and performance in cluster-basedsystems. In Workshop on compilers and operating systems for low power,volume 180, pages 182–195. Barcelona, Spain.

Quan, D. M., Mezza, F., Sannenli, D., and Giafreda, R. (2012). T-alloc: a prac-tical energy efficient resource allocation algorithm for traditional datacenters. Future Generation Computer Systems, 28(5):791–800.

Raghavendra, R., Ranganathan, P., Talwar, V., Wang, Z., and Zhu, X. (2008).No ”power” struggles: coordinated multi-level power management forthe data center. SIGOPS Oper. Syst. Rev., 42(2):48–59.

Ranganathan, P., Leech, P., Irwin, D., and Chase, J. (2006). Ensemble-levelpower management for dense blade servers. SIGARCH Comput. Archit.News, 34(2):66–77.

Raycroft, P., Jansen, R., Jarus, M., and Brenner, P. R. (2014). Performancebounded energy efficient virtual machine allocation in the global cloud.Sustainable Computing: Informatics and Systems, 4(1):1–9.

Rethinagiri, S. K., Palomar, O., Sobe, A., Yalcin, G., Knauth, T., Gil, R. T.,Prieto, P., Schneegaß, M., Cristal, A., Unsal, O., et al. (2015). Paradime:Parallel distributed infrastructure for minimization of energy for datacenters. Microprocessors and Microsystems, 39(8):1174–1189.

95

© COPYRIG

HT UPM

Rodero, I., Viswanathan, H., Lee, E., Gamell, M., Pompili, D., and Parashar,M. (2012). Energy-efficient thermal-aware autonomic managementof virtualized hpc cloud infrastructure. Journal of Grid Computing,10(3):447–473.

Rodero-Merino, L., Vaquero, L. M., Gil, V., Galn, F., Fontn, J., Montero, R. S.,and Llorente, I. M. (2010). From infrastructure delivery to service man-agement in clouds. Future Generation Computer Systems, 26(8):1226–1240.

Sharma, M., Arunachalam, K., and Sharma, D. (2015). Analyzing the datacenter efficiency by using pue to make data centers more energy efficientby reducing the electrical consumption and exploring new strategies.Procedia Computer Science, 48:142–148.

Sharma, R. K., Bash, C. E., Patel, C. D., Friedrich, R. J., and Chase, J. S. (2005).Balance of power: dynamic thermal management for internet data cen-ters. Internet Computing, IEEE, 9(1):42–49.

Sheikh, H. F. and Ahmad, I. (2012). Simultaneous optimization of perfor-mance, energy and temperature for dag scheduling in multi-core pro-cessors. In Green Computing Conference (IGCC), 2012 International, pages1–6. IEEE.

Shidik, G. F., Mustofa, K., et al. (2015). Evaluation of selection policy withvarious virtual machine instances in dynamic vm consolidation for en-ergy efficient at cloud data centers. Journal of Networks, 10(7):397–406.

Song, B., Ernemann, C., and Yahyapour, R. (2005). Parallel Computer WorkloadModeling with Markov Chains, volume 3277 of Lecture Notes in ComputerScience, chapter 3, pages 47–62. Springer Berlin Heidelberg.

Srikantaiah, S., Kansal, A., and Zhao, F. (2008). Energy aware consolidationfor cloud computing. In Proceedings of the 2008 conference on Power awarecomputing and systems, volume 10, pages 1–5. San Diego, California.

Sullivan, D. G. and Seltzer, M. I. (2000). Isolation with flexibility: A re-source management framework for central servers. In Proceedings of theUSENIX Annual Technical Conference, pages 337–350.

Sun, H., Cao, Y., and Hsu, W.-J. (2009). Non-clairvoyant speed scaling forbatched parallel jobs on multiprocessors. In Proceedings of the 6th ACMconference on Computing frontiers, pages 99–108. ACM.

Sun, H., Stolf, P., Pierson, J.-M., and Da Costa, G. (2014). Energy-efficientand thermal-aware resource management for heterogeneous datacen-ters. Sustainable Computing: Informatics and Systems, 4(4):292–306.

Theja, P. R. and Babu, S. K. (2015). An evolutionary computing based energyefficient vm consolidation scheme for optimal resource utilization andqos assurance. Indian Journal of Science and Technology, 8(26):1–11.

96

© COPYRIG

HT UPM

Toms, L., Caminero, A. C., Carrin, C., and Caminero, B. (2011). Network-aware meta-scheduling in advance with autonomous self-tuning sys-tem. Future Generation Computer Systems, 27(5):486–497.

Uddin, M. and Rahman, A. A. (2012). Energy efficiency and low carbonenabler green it framework for data centers considering green metrics.Renewable and Sustainable Energy Reviews, 16(6):4078–4094.

Vasic, N. and Kostic, D. (2010). Energy-aware traffic engineering. In Proceed-ings of the 1st International Conference on Energy-Efficient Computing andNetworking, pages 169–178. ACM.

Verghese, B., Gupta, A., and Rosenblum, M. (1998). Performance isolation:sharing and isolation in shared-memory multiprocessors. SIGOPS Oper.Syst. Rev., 32(5):181–192.

Verma, A., Ahuja, P., and Neogi, A. (2008). pmapper: power and migrationcost aware application placement in virtualized systems. In Middleware2008, pages 243–264. Springer.

Verma, A., Dasgupta, G., Nayak, T. K., De, P., and Kothari, R. (2009). Serverworkload analysis for power minimization using consolidation. In Pro-ceedings of the 2009 conference on USENIX Annual technical conference,pages 28–42. USENIX Association.

Von Laszewski, G., Wang, L., Younge, A. J., and He, X. (2009). Power-awarescheduling of virtual machines in dvfs-enabled clusters. In Cluster Com-puting and Workshops, 2009. CLUSTER’09. IEEE International Conferenceon, pages 1–10. IEEE.

Vrbsky, S. V., Galloway, M., Carr, R., Nori, R., and Grubic, D. (2013). Decreas-ing power consumption with energy efficient data aware strategies. Fu-ture Generation Computer Systems, 29(5):1152–1163.

Wang, L., Tao, J., von Laszewski, G., and Chen, D. (2010). Power awarescheduling for parallel tasks via task clustering. In Parallel and Dis-tributed Systems (ICPADS), 2010 IEEE 16th International Conference on,pages 629–634. IEEE.

Webb, M. (2008). Smart 2020: enabling the low carbon economy in the in-formation age, a report by the climate group on behalf of the global es-ustainability initiative (gesi). Global eSustainability Initiative (GeSI), Tech.Rep.

Wolke, A., Tsend-Ayush, B., Pfeiffer, C., and Bichler, M. (2015). More than binpacking: Dynamic resource allocation strategies in cloud data centers.Information Systems, 52:83–95.

Yakhchi, M., Ghafari, S. M., Yakhchi, S., Fazeli, M., and Patooghi, A. (2015a).Proposing a load balancing method based on cuckoo optimization algo-rithm for energy management in cloud computing infrastructures. In

97

© COPYRIG

HT UPM

Modeling, Simulation, and Applied Optimization (ICMSAO), 2015 6th Inter-national Conference on, pages 1–5. IEEE.

Yakhchi, S., Ghafari, S., Yakhchi, M., Fazeli, M., and Patooghy, A. (2015b).Ica-mmt: A load balancing method in cloud computing environment.In Web Applications and Networking (WSWAN), 2015 2nd World Symposiumon, pages 1–7. IEEE.

Zhang, Q., Metri, G., Raghavan, S., and Shi, W. (2014). Rescue: An energy-aware scheduler for cloud environments. Sustainable Computing: Infor-matics and Systems, 4(4):215–224.

Zhang, X., Lu, J.-J., Qin, X., and Zhao, X.-N. (2013). A high-level energy con-sumption model for heterogeneous data centers. Simulation ModellingPractice and Theory, 39:41–55.

Zheng, X. and Cai, Y. (2011). Energy-aware load dispatching in geographi-cally located internet data centers. Sustainable Computing: Informatics andSystems, 1(4):275–285.

Zhu, X., Young, D., Watson, B. J., Wang, Z., Rolia, J., Singhal, S., McKee, B.,Hyser, C., Gmach, D., Gardner, R., et al. (2008). 1000 islands: Integratedcapacity and workload management for the next generation data cen-ter. In Autonomic Computing, 2008. ICAC’08. International Conference on,pages 172–181. IEEE.

98