Adaptive software in cloud computing

Marin Litoiumlitoiu@yorku.caYork UniversityCanada

© Marin Litoiu, EU-Canada Future Internet Workshop

© Marin Litoiu, EU-Canada Future Internet Workshop

Content

Elasticity

Business Driven Elasticity

Architectures

– Model based feedback loops

– Strategy trees based feedback loops

Open Issues

© Marin Litoiu, EU-Canada Future Internet Workshop

Elasticity Traditionally, we sized applications for typical workloads

Wasted capacity

(Cost)

Lost revenue

Lost customers

(lost revenue)

*Berkley report The promise: adapt at runtime

© Marin Litoiu, EU-Canada Future Internet Workshop

Business Driven Elasticity

Elasticity achieves business goals– Minimize cost (and/or increase revenue)

– Includes energy savings– Meet SLO

By monitoring the goals through sensors

Response times, utilization, profit, cost etc..

And by changing control inputs (actuators)

– Hardware resource allocation ( CPUs, memory, storage…)

– Software resources (licences, threads, replicas…)

Subject to policies (constraints)

– Hardware and software capacity limits

5

© Marin Litoiu, EU-Canada Future Internet Workshop

Adaptive Feedback Loops

© Marin Litoiu, EU-Canada Future Internet Workshop

Cloud Landscape

SaaS Management

PaaS Management

IaaS Management

SaaS

(Salesforce, Google, IBM)

PaaS

(Google, IBM)

IaaS

(Amazon, IBM)

CPU

Hardware

Execution EnvironmentProgramming Environment

ApplicationSimple services

(OpenID)

Virtual

machineCPU

AccessServices

Service Models: IaaS, PaaS, SaaS

Deployment Models: private, community, hybrid, public

© Marin Litoiu, EU-Canada Future Internet Workshop

…hence Layered Adaptation

PaaSPaaS

IaaSIaaS

SaaS SaaS

Sensors•Processors , memory, disk utilization•Processors throughput and response

times

Actuators•VM to processor allocation

•VM settings ( memory, CPU ratio)•VM storage

•Network bandwidth

Major Goals:•hardware utilization

Constraints: •capacity

Sensors•VM Utilization, response time,

throughput•Container utilizations, throughput,

response times....

Actuators•Number of VMs, licenses

•Allocation of containers to VMs•Container settings (threads,

caching)...

Major Goals:•Reduce cost

•Increase revenue Constraints:

•capacity

Sensors:- Service (application) QoS

-response time -throughput

Actuators•Deployment topology

•Parameter tuning

Major Goals•QoS (response time,

throughput)

Constraints :•SLA

© Marin Litoiu, EU-Canada Future Internet Workshop

1. Model Based Adaptive Loops

Performance model

Optimization & Control

State estimator

Monitoring

Services

Goals & Policies

Control Change(uc)

Disturbances(pc)

yc,uc,pc,xc

Workload classifier

Cloud Layer (xc,yc)

A

ctuato

rs Sensors

Mo

del id

entificatio

n

© Marin Litoiu, EU-Canada Future Internet Workshop

Adaptation at PaaS Layer

Goal: Profit = Revenue-Cost

– Revenue = proportional with the number of applications

– Cost = Price per VM running + price licences

• thus at a given moment the goal is minimum cost for the given applications

Constraints: SLAs, capacity limits, etc

PaaSPaaSSensors•VM Utilization, response

time, throughput•Container utilizations, throughput, response

times....

Goals:

•platform profit

Actuators•Number of VMs, licenses

•Allocation of containers to VMs

•Container settings (threads, caching)...

© Marin Litoiu, EU-Canada Future Internet Workshop

PaaS Optimization and Control….

sctsht ,,

minCOST =

H

h

T

t hthC1 1

(3)

subject to:

Service level agreement: for each c, fc fc,SLA.

Host capacity: for each h, ht h

t T

. To limit the

maximum processor utilization to h <1 replace Ωh by hΩh .

Flow balance: ht ts

h H s S

(for all t); ts sc

t T c C

(for

all s); sc c scf d (for all s and c)

Nonnegative flows: for all h, t, s, ht ≥ 0, ts ≥ 0, γsc ≥ 0.

Minimize COST in PaaS

-Across N applications

-Subject to

-SLA

-application integrity

-processing capacity

-memory

-licence constraints

By asking IaaS to slice the physical resources into

virtual resources

© Marin Litoiu, EU-Canada Future Internet Workshop

Results (1): SaaS

The cost is low when traffic is low

Response time is kept below a

target

The application uses less physical

machines when traffic is low

multitier interactive applications

© Marin Litoiu, EU-Canada Future Internet Workshop

Results(2): PaaS Results are for multi tier applications, that can scale horizontally and vertically

FO: full optimization

IO: incremental optimization

It is more efficient to redeploy ALL applications periodically (similar to disk defragmentation)

© Marin Litoiu, EU-Canada Future Internet Workshop

2. Policy Based Adaptive Loops

© Marin Litoiu, EU-Canada Future Internet Workshop

Further Challenges

Centralized versus decentralized adaptation

– Geographically distributed clouds

Coordination among different layers

– Sensors and actuators

Global versus local optimization

Accurate models for different layers

© Marin Litoiu, EU-Canada Future Internet Workshop

References

Litoiu M, Woodside M., Wong J., Ng J., Iszlai G., “A Business Driven Cloud Optimization Architecture”, Proceedings of ACM SAC 2010, Sierre, Switzerland, March 24-29, 2010 

Simmons B., Litoiu M., “Towards a Cloud Optimization Architecture  with Strategy Trees,” Proceedings of IEEE I2TS 2010,  Rio de Janeiro, Brazil, Dec 2010.

Ghanbari H., Litoiu M., Simmons B., Barna C., “Feedback-based Optimization of a Private Cloud,” IEEE Conference on Utility and Cloud Computing ( UCC 2010), December, Chennai, India, 2010.

Zheng T., Litoiu M., Woodside M., “Integrated Estimation and Tracking of Performance Model Parameters and their Trends,” 2nd  ACM/Spec International Conference on Performance Engineering, Karlsruhe, March 14-16, 2011.