using on-premise iaas clouds to rapidly accelerate the dev/test lifecycle

© 2012 Eucalyptus Systems, Inc.

Using On-premise IaaS Clouds to Rapidly Accelerate the Dev/Test Lifecycle

Rich Wolski, CTO & Co-founder


Agenda

• What is Self-service IT?

• Dev/Test Defined

• Why is Dev/Test Difficult?

• The On-premise Dev/Test Cloud

• Eucalyptus 3.1

• Case Studies

• Next Steps

3


What is Self-service IT?

• eCommerce style access to infrastructure resources – Shop on the web for the infrastructure necessary to run applications

• Transactional and asynchronous – Each user request is a transaction

– Service delivery is asynchronous

• Bundled QoS – Each “purchase” is for a bundle of resources with specified QoS

• User and resource scalable – many users using many resources

• User isolation – Each user purchase is independent

• Scale out and scale up

4


Dev/Test Defined

• Software product development consists of sequenced development and testing phases

• Development – Source code management

– Software dependency management

– Infrastructure configuration management

• Unit testing – Test management

– Debugging and IDE integration

• Integration testing – Functional and regression

• Package testing – Build and dependency management

5


Why is Dev/Test Difficult?

• Lots of independent activities with time-varying resource requirements sharing common infrastructure

– Configuration, infrastructure, and code management

• Bursty workloads

– Dev/test cycle is predictable but highly varying

• Accounting

– Products must share resources but account separately

– Isolation is a question of accounting, not security

• Resource decommissioning

– “When is a dev/test cycle over?”

6


Dev/Test Infrastructure Usage

• Frequent setup/teardown of resources

– Hard to manage

• Dev/test function do not necessarily share configs

– No standardization

• Lots of idle/dormant capacity

– Low utilization of infrastructure

• Isolated & quota controlled usage

– Dev/test phases cannot interfere Infrastructure usage over a period

De

ma

nd

fo

r re

so

urc

es

Peak

development

Unit

testing

Integration

testing

7


The On-premises Dev/Test Cloud

• User-controlled self-service infrastructure

– Web interface for acquiring quota controlled resources with QoS

– Lease mechanism for “garbage collection”

• Images as software containers

– Customizable for specific functionality and configuration

– Version controlled

– Cataloged

• Access control, quotas, and isolation

– Separate users must not be able to interfere with each other

– Groups of users and dev phases must operate independently

• High-availability

– Can’t lose long-running test output due to resource failure

8


Eucalyptus 3.1

• Implements the AWS cloud APIs

– Images, storage, configuration management, QoS, identity management

• Extends the API with on-premises features necessary for dev/test

– Quotas: users and groups can be quota limited

– ID management integration

– Leases

• Infrastructure agnostic

– Eucalyptus platform exposes resources to users as QoS capacity

– Dev/test can leverage different infrastructure components and virtualization technologies through a common set of cloud APIs

• Platform HA

– No single point of failure and no data loss

• Open source

9


Functional Decomposition

10


Graphical User Interface (soon)

• Intuitive self-service UI

• Key management

• Shell/RDP access

• Elastic Block Storage

• Remote logging

• Virtual Machine monitoring

• Budgeting & chargeback

11

Eucalyptus Cloud Manager


Structure Dev/Test Cloud

Dev Zone 1

WEB App DB

WEB App DB

WEB App DB

Test Zone 1

WEB App DB

WEB App DB

WEB App DB

WEB App DB

WEB App DB

WEB App DB

Dev Zone 2

WEB App DB

WEB App DB

WEB App DB

WEB App DB

Prod Zone 1

WEB App DB

WEB App DB

Shared Infrastructure

• Isolation zones on shared infrastructure

• Scale-out and scale-in elasticity

12


SCM/Repo

Images as Containers

Linux

JEE

Hibernate

Spring

Log4J

Proprietary

code

App Server

Web Server

Various Agents

EMI

Linux

Various Agents

JEE

App Server

Web Server

Proprietary code

Log4J

Spring

Hibernate Convert

Existing

stack to

portable

Container

Walrus

Self-Service or

Dev/Test

Workflow system

VM

VM

VM

13


Dev/Test with Eucalyptus

• Integrate cloud provisioning with dev tool chain

– Build tools

– SCM systems

– Application servers

– Testing and code analysis tools

• Process & utilization reporting

• Dev configuration sharing

Orchestration through Cloud Self-service API

14


The AWS and Eucalyptus Ecosystem

Physical Resource Management

Virtual Resource Management

Cloud Resources

IaaS Web Services

Third Party

Management

SaaS / PaaS Providers

15


Case Studies

Paul Weiss, Technical Marketing

16


Case Study #1: Eucalyptus QA System

17


Eucalyptus QA System

18


Commitment to Quality

19

Source: http://eucatest.wordpress.com/2012/06/23/quality-flow-in-eucalyptus/

500,000+ Instances

http://eucatest.wordpress.com/2012/06/23/quality-flow-in-eucalyptus/








Case Study #2: Global Telco Provider

20


The Client

• Global telecommunications provider

• Over $15B in annual revenue

• 70,000+ employees

• Operations in nearly 150 countries

21


The Challenges

• Market Forces

– Fierce competition

– Mobile market rapidly moves

• Internal Forces

– Lengthy fulfillment times

– Inconsistent dev/test/prod environments

– Widely distributed development teams

• Financial Factors

– Hardware costs

– Labor costs

– Expensive customer acquisition

22


Legacy Environment

23

Security

Monitoring

DNS

Firewall

IT Services

QA

Compute Compute

Storage Network Storage Network

Fulfillment can take days or even weeks

Acquiring new hardware extends the delay

Provisioning tends to be manual

Validating newly provisioned resources takes time

Misconfiguration places additional burden on IT due to troubleshooting

Difficult to properly configure each environment to be secure and manageable by Operations

Differences between dev / test setups are common

Development


The Solution

24

Eucalyptus

Image

Library

Security

Self

Provisioning

Compute Network Storage

Monitoring

DNS

Firewall

IT Services IaaS Core

Build / Test

Automation

Availability

Zone

Availability

Zone

Self service interface for rapid fulfillment

Connectivity to IT services is standardized for all users

Build / Test automation systems are integral to the solution

Development and QA configurations may be sync’d

Resources are shared by multiple teams

Isolation is inherent in the system and enhanced by setting up multiple availability zones

Infrastructure upgrades may be less disruptive


Benefits Realized

• Increased delivery speed & capacity

• Better development efficiency

• Improved product quality

• Lowered demand for operations support

• Decreased hardware costs

25

26

Questions?


Next Steps

• Read “7 Steps to Success with an Automated Dev/Test Cloud”

– http://sl.eucalyptus.com/devtestcloud

• Register for a 30 Day Free Trial

– Self-service or sales-supported

– For VMware, XEN, KVM

– All major Linux distros

– http://sl.eucalyptus.com/freetrial

27

http://sl.eucalyptus.com/devtestcloud

http://sl.eucalyptus.com/freetrial

http://sl.eucalyptus.com/freetrial


Thank you.

using on-premise iaas clouds to rapidly accelerate the dev/test lifecycle

Technology

eucalyptus systems

code devtest

infrastructure resources

eucalyptus qa system17

eucalyptus qa system18

devtest quotas

devtest difficult

resources devtest function