451 group analysis - skytap.com group originally published this report as an overview of ......

451 Group Analyst Research:Cloud Automation

analyst research

II THE 451 GROUP: CLOUD AUTOMATION

SUMMARYSkytap is a cloud automation solution that provides users with self-service access to on-demand virtual machines and data centers, while enabling IT to maintain complete control over security policies and usage.

Skytap is ideal for dynamic workloads such as application development, testing, ERP migration, virtual training, and sales demonstrations. Skytap can be securely connected to existing data centers using virtual private cloud technology and can reduce costs by 50% or more.

This 451 Group report provides an overview of cloud automation, including the business case, technical requirements and use cases for cloud automation solutions. 451 Group originally published this report as an overview of ‘virtual lab automation’. These technologies are now commonly referred to as ‘cloud automation’ solutions and this report has been updated to reflect this change with permission from 451 Group.

© THE 451 GROUP. ALL RIGHTS RESERVED I

ABOUT THE 451 GROUP The 451 Group is a technology analyst company. We publish market analysis focused on innovation in enterprise IT, and support our clients through a range of syndicated research and advisory services. Clients of the company — at vendor, investor, service-provider and end-user organizations — rely on 451 insights to do business better.

ABOUT TIER1 RESEARCH Tier1 Research covers consumer, enterprise and carrier IT services, particularly hosting, colocation, content delivery, Internet services, software-as-a-service and enterprise services. Tier1’s focus is on the movement of services to the Internet — what they are, how they are delivered and where they are going.

Please note that the following 451 report is copyright protected and is being provided to you on a limited, licensed basis. By viewing this document, you consent to and agree to abide by the terms of this license and the general Terms of Use (below) for users of services of The 451 Group. Only authorized, licensed users may access this and other content from The 451 Group.

If you have any questions about this license or terms of use for your organization, please contact your account manager directly. Alternately, you can contact a general representative of The 451 Group directly via phone at 212-505-3030 or via mail at 20 West 37th Street, 6th Floor, New York, N.Y. 10018.

(See Terms of Use for more information)

Analyzing the Business of Enterprise IT Innovation

London37-41 Gower StreetLondon, UK WC1E 6HHPhone: +44 (0)20.7299.7765Fax: +44 (0)20.7299.7799

Boston52 Broad Street, 2nd FloorBoston, MA 02109Phone: 617.261.0699Fax: 617.261.0688

New York20 West 37th Street, 6th FloorNew York, NY 10018Phone: 212.505.3030Fax: 212.505.2630

San Francisco140 Geary Street, 9th FloorSan Francisco, CA 94108Phone: 415.989.1555Fax: 415.989.1558

II THE 451 GROUP: CLOUD AUTOMATION

TABLE OF CONTENTS

EXECUTIVE SUMMARY 1

KEY FINDINGS. . . . . . . . . . . . . . . . . . . . . . . . . . 2

DEFINITIONS OF KEY TERMS . . . . . . . . . . . . . . . . . . . . 3

SECTION 1: INTRODUCTION: A VIEW INTO THE BLACK BOX 5

1.1 SOFTWARE DELIVERY PROCESSES: AN UNVARNISHED LOOK . . . . . . 5

1.2 INEFFICIENCIES IN SOFTWARE DEVELOPMENT ORGANIZATIONS . . . . . 7

1.2.1 Manual Provisioning of Preproduction Hardware . . . . . . . . 7

1.2.2 Ad Hoc Management of Preproduction Hardware . . . . . . . . 7

1.2.3 Lack of Repeatability. . . . . . . . . . . . . . . . . . 8

SECTION 2: CLOUD AUTOMATION 9

2.1 WHAT IS IT? . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 THE BUSINESS CASE FOR CLOUD AUTOMATION . . . . . . . . . . . 9

2.2.1 Reduced Time to Market . . . . . . . . . . . . . . . . 9

2.2.2 Efficient Resource Management . . . . . . . . . . . . . .10

2.2.3 Improved Testing Reliability . . . . . . . . . . . . . . . 11

2.2.4 Reuse of Automated Provisioning Know-How in Production . . . . 11

2.3 TECHNICAL REQUIREMENTS FOR CLOUD AUTOMATION . . . . . . . . 12

2.3.1 A Library of Configuration Templates . . . . . . . . . . . .12

2.3.2 Complex Configurations. . . . . . . . . . . . . . . . .12

2.3.3 Self-Service Provisioning . . . . . . . . . . . . . . . .13

2.3.4 User Management. . . . . . . . . . . . . . . . . . .13

2.3.5 Governance and Reporting . . . . . . . . . . . . . . . .13

2.4 USE CASES . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4.1 Development and Testing . . . . . . . . . . . . . . . .14

2.4.2 Software Demonstrations and Training . . . . . . . . . . .15

2.4.3 IT Operations . . . . . . . . . . . . . . . . . . . .15

© THE 451 GROUP. ALL RIGHTS RESERVED III

SECTION 3: DISRUPTORS IN THE MARKET 17

3.1 HYPERVISOR COMMODIFICATION . . . . . . . . . . . . . . . . 18

3.2 CLOUD COMPUTING. . . . . . . . . . . . . . . . . . . . . . 19

SECTION 4: CONCLUSION 21

ADDITIONAL RESEARCH 21

TERMS OF USE 22

© THE 451 GROUP. ALL RIGHTS RESERVED 1

EXECUTIVE SUMMARYSoftware development organizations within ISVs and large enterprises have mainly been left to their own devices to administer and maintain their development and test environments. Due to their technically self-sufficient user base, their essentially dynamic nature and the sheer cost and labor of organizing them, development and test environments have escaped the rigor with which production environments are scrutinized and tuned.

Virtualization technology stands to radically change this situation for the better and provide time, cost and personnel savings to software development organizations. This report focuses on test lab automation, which uses virtualization technology to bring manageability to development, test and other preproduction environments.

2 THE 451 GROUP: CLOUD AUTOMATION

KEY FINDINGS• As software development is mounting in scale and complexity, tolerance for software

failures is falling. Software development organizations, which are often globally and organizationally distributed, are compelled to validate the software they produce against a growing battery of tests.

• Preproduction environments are necessarily more dynamic than production environments, and require more flexible management practices than those applied to production environments.

• Practices that may have been used to manage and scale preproduction environments in the past are ill-suited to the dynamism of present-day preproduction environments and result in inefficient utilization of preproduction environments and resource wastage.

• Virtualization drastically improves the economics of creating and destroying machine instances. Cloud automation leverages these changed economics to enable self-provisioning complex lab configurations, which can greatly reduce time to market for software development.

• Complex lab configurations can be used for development and test purposes, training and pre-sales demonstrations. When organizations can automatically provision even very complex machine configurations, they rapidly realize that virtual machines as secure, transportable containers for code are applicable in production as well. Test lab automation may enable workloads to be transferred directly from staging areas to internal or external clouds.

• Deploying cloud automation technology is likely to make developers and testers much less averse to running complex test suites on their code. Several hard-to-reproduce problems that can arise from slip-ups during complex deployments are no longer hypothetical and can now be recreated easily. Testing for backward compatibility and across hardware and software platforms becomes considerably easier than before.

• By eliminating unnecessary overhead, cloud automation technology can reduce resource wastage and pave the way for greater visibility into and more efficient use of preproduction environments.

• The increasing level of comfort with virtualization generally, coupled with the commoditization of low-level virtualization technology, makes the test lab automation market a key segment of innovation in virtualization management technology and an attractive locus for M&A activity.


DEFINITIONS OF KEY TERMSVirtualization – An umbrella term for the abstraction of computer resources, typically for the purposes of partitioning, emulating or aggregating those resources. In doing so, virtualization should also make resources easier to manage.

Preproduction environments – An internal-facing environment where applications are developed, tested and staged before deploying them toward line-of-business activities. The term preproduction encompasses development and quality-assurance (QA) functions and stands distinct from production environments, which are generally more locked down and subject to stricter change controls.

Provisioning – The process by which machine configuration templates are converted into running instances of physical and virtual machines for development and test purposes.

Cloud – An abstraction layer that simplifies access to, and imposes retail discipline on, computing infrastructure. Cloud computing is essentially IT as a service.

Hypervisor – An abstraction layer or virtualization platform that can host multiple operating systems on a single physical resource.

Administrators – IT staff whose duties include procuring and provisioning the physical preproduction environment and supporting developers and testers. Their role lies outside the core software development organization, and they have minimal direct involvement in development and test activities.

Developers – Professionals within the core software development organization whose duties include writing new software, fixing defects in previously deployed software, and executing and managing software builds.

Testers – Professionals within the core software development organization whose duties include creating test harnesses, recording test scripts and subjecting software to additional validation after it has passed basic checks from developers.

Agile development – A class of software development techniques that mitigates the risk that software projects fail by adopting testing and validation techniques early in the application development lifecycle and emphasizing working code over feature checklists. There are several kinds of agile development, but they all trace their origins to the Agile Manifesto, a seminal document from 2001 that lays out broad principles for agile methods.

Unit test – A simple test that validates no more than a few functional aspects of a software component. Usually written by a developer rather than a tester.


This Page Intentionally Blank


SECTION 1:Introduction: A View Into the Black Box

1.1 SOFTWARE DELIVERY PROCESSES: AN UNVARNISHED LOOK

Application development lifecycles within most software development organizations are complex balancing acts that must account for a variety of people and contingencies. Although the particulars of application creation vary according to specific processes used within an organization, we can look upon the post-design phase of a software project as consisting of three functional phases: development, testing and release. Although these phases usually mark increasing stability for a software project, they also mean a growing number of items against which the project must be validated.

During the development phase of the project, software developers write new code implementing new or enhanced functionality. They are also frequently responsible for unit tests that exercise well-defined portions of their code. The test phase of the project is carried out by testers (or quality-assurance engineers, as they are known in some organizations), who create test scripts and test harnesses from specifications and earlier iterations, if available. The release phase comes near the end of the project lifecycle, when release engineers formulate the final packaging of a release.

As a project moves closer to completion, growing portions of the code base must be validated with integration tests. In the test phase, testers must validate cross-cutting concerns such as security and user interface flow. If a release is adding support for a new platform, release engineers must document any subtly different behaviors of the software on the new platform. If the release is an update rather than the start of a new line of releases, release engineers must document all the ways in which it breaks compatibility or differs in function from previous releases. In addition to growing batteries of functional tests prior to deployment, security testing is also being pushed progressively earlier into the application development lifecycle. Running through all these validation points is time-consuming and also imposes much overhead in terms of setting up and managing complex hardware and software configurations.

Software development organizations manage some of this complexity by using isolated environments for development, staging and production. More often than not, however, procurement, asset management and provisioning process discipline is lax in preproduction environments because they are subject to less strict rules. Some of this lax regime is by design, because preproduction environments are necessarily more fluid than locked-down, production environments. Development teams, however, often end up making unwarranted assumptions (say, about dependency fulfillment at runtime) that aren’t borne out in production environments, which in turn leads to software failures.


An already involved process of software development and validation acquires many more layers of complexity when we take into account two contemporary realities of software production: open source components and geographically distributed teams. Open source components are widely prevalent, if not ubiquitous, in software written for both internal uses and external distribution. These components are often drawn from multiple open source projects, which may not always follow coordinated change cycles. The different paces of development in the different open source projects used in a software project result in a multitude of software configurations that must be tested.

Similarly with geographically distributed teams, the task of shepherding software through development and complex validation becomes much harder. Any number of links in the complex chain of development and testing within a large enterprise may be outsourced to other organizations or other countries. These external sites may not always have environments that closely match those where the software will eventually run. Shared development tools, such as source control repositories, do not by themselves encapsulate the knowledge necessary to build and test software in preproduction environments. Outsourced and offshore software development may make it possible for a team to function through most of a 24-hour period, but if there are miscommunications between different cliques in a team, valuable work hours are lost.

All the above inefficiencies have concrete effects on software development organizations in both production and preproduction environments. Heavy validation workloads for developers and testers, along with lax enforcement of essential discipline in preproduction environments, result in software failures. These failures have a measurable impact on revenue. The business side of an organization has historically had little time or inclination to look into how software development processes that support the business actually work. All too often, the preproduction side of a software development organization has been treated as a black box that produces mostly working software most of the time.

Software development teams are hostile to heavy supervision in most cases, but unobtrusively gaining visibility into how they work confers several benefits. For one, software failures can become less mysterious and more easily diagnosable. Clearer insight into software development processes can empower companies to set more measurable performance milestones for their technical employees. Organizations can manage software development risk at finer-grained levels if they actually know a thing or two about the inscrutable software supply chain. Additionally, business units can influence their software development teams to eliminate any glaring inefficiencies that may run rampant in preproduction environments. We will examine these inefficiencies in greater detail in the next section.


1.2 INEFFICIENCIES IN SOFTWARE DEVELOPMENT ORGANIZATIONS

Software development as it is carried out today in several organizations has several process inefficiencies and behaviors that may make software failures more likely and harder to diagnose. We will examine some of these inefficiencies in greater detail in this section.

1.2.1 MANUAL PROVISIONING OF PREPRODUCTION HARDWARE

When developers or testers need to test out a hypothesis or reproduce an error condition, they may not always have the requisite computing resources at hand. If necessary, they must request development and test servers from an internal IT support organization geared toward developer needs. This requisitioning process may take several days, if not weeks, to complete, and it introduces delays into the cycle of fixing a bug or adding a feature. By the time the requisitioning process completes, the developer or tester has most probably encountered several other issues that need attention.

As test-driven, agile development methodologies take root in software devel-opment teams, it is increasingly important for developers to be able to provi-sion and instantiate complex hardware and software configurations. The more onerous the provisioning of computing resources, the less amenable devel-opers and testers are to trying out new ideas that could improve their product. Alternatively, a developer or tester may choose to sidestep a time-consuming manual provisioning process and repurpose existing hardware ad hoc for new uses. Eventually, with team member churn and quite simply the fallibility of human and institutional memory, such ad hoc management will break down.

1.2.2 AD HOC MANAGEMENT OF PREPRODUCTION HARDWARE

As we mentioned earlier, software development teams, most often by design, aren’t subject to stringent hardware and software asset management policies because of the fluidity of preproduction environments. Once development and test servers have been provisioned, development teams often end up accumulating significant pools of hardware and software bound only by loosely enforced conventions and ad hoc rules.

These management practices for the preproduction environment are not always well thought through, and may result in a lot of wasted hardware and software assets. Administrators may not know if previously provisioned hardware is in use, abandoned or no longer within specification. Within software development organizations, developers and testers often rely on individual servers tucked away under their desks. There is little department-wide awareness of or accountability for all this computing power.


In many situations, it may make sense for development teams assembling complex computing configurations to simply pool together existing hardware rather than requisition a custom-built test bed. Some development teams over-come this problem to some extent by maintaining loosely structured lists of shared development resources. Even when these shared resources are well known within a team, it is possible for team members to inadvertently step on configurations other team members have set up.

1.2.3 LACK OF REPEATABILITYDespite best efforts to maintain configuration hygiene on shared development and test lab resources, a slip-up in ad hoc management practices can pollute and adversely affect development and test environments belonging to multiple team members. Shared development and test servers usually don’t sufficiently isolate team members from one another. One of the biggest frustrations developers and testers report after incidents in production is the inability to reliably reproduce a problem. Subtle differences in server configuration and execution environments may be the reason production issues are difficult to reproduce reliably. The frustrations arising from these differences only get exacerbated when they straddle organizational boundaries between vendors and their customers.

Currently, when attempting to reproduce and fix a production issue, developers and testers must go through the time-consuming effort of scrubbing existing preproduction hardware clean or setting up new preproduction hardware to match the production configurations on which the issue occurred, because administrators don’t always have the application-specific knowledge necessary to do this work. All this effort on the part of developers and testers to duplicate configurations down to the finest detail adds to the time needed to fix an issue — time which could have been saved by leveraging a repository of known configurations.


SECTION 2:Cloud AutomationIn Section 1, we looked at development and test practices in typical software devel-opment organizations, and identified some inefficiencies in these processes. This section of the report defines cloud automation and makes business and technical cases for the benefits it can bring to development and test environments.

2.1 WHAT IS IT?Cloud automation brings virtualization management technology to the application development lifecycle by enabling the creation and instantiation of computing configurations for development and test purposes.

Basic virtualization technology first got its foothold in preproduction environments, but these environments did not make significant use of automation. Now, as virtualization makes inroads into production environments and hypervisors continue to become commodified, vendors seek to create value by moving ‘up the stack’ to virtualization management technology and using it to mediate a variety of IT processes.

Cloud automation offerings are appropriate for several situations — test management, development work, training, pre-sales proofs of concept and improving the resiliency of IT operations, to name a few — but they all use virtualization technology to manage complex configurations of machines. By making it possible to provision and start up an isolated, self-sufficient machine instance in short order with little outside assistance, test lab automation offerings rely on virtualization technology to enable a new kind of economics for development and test labs.

2.2 THE BUSINESS CASE FOR CLOUD AUTOMATIONThe fundamentally changed economics that cloud automation technology brings to development and test environments lays the groundwork for several quantifiable business benefits.

2.2.1 REDUCED TIME TO MARKETManual provisioning of development and test hardware by administrators of preproduction environments takes an inordinately long time and is a poor fit for the inherently dynamic nature of preproduction environments. Cloud automation sensibly devolves some control of preproduction machines to developers and testers themselves. Rather than route every setup and tear-down request for complex lab configuration through an administrator, developers and testers can become self-sufficient to a large degree. Administrators still retain control over major changes to the preproduction environment, such as ordering physical servers to host virtual machines.


Today, developers and testers must coordinate access to shared resources with their teammates and work with administrators to receive rights to work on them. With a cloud automation product, however, developers and testers can themselves set up and use complex configurations of fully isolated virtual machines on shared hardware and still have full control over them without much intervention from administrators. Software engineering managers will recognize this as an application of an oft-stated maxim of software engineering productivity: ‘Give your developers everything they need, and then get out of the way.’

One medium-sized publicly traded technology vendor we spoke with reported that its cloud automation deployment had cut provisioning turnaround times from two weeks to a few hours. The benefits of quick turnaround times for environment provisioning cannot be overstated: developers and testers become much less averse to running complex test suites on their code, and several hypothetical conditions arising from complex deployments can be created and tested with ease. Together, these benefits raise the quality of code that is deployed into production and could reduce the severity of software failures, if not their frequency.

2.2.2 EFFICIENT RESOURCE MANAGEMENTCloud automation products are built around a database of complex configurations. These may be purely virtual or hybrid physical-and-virtual configurations. This repository of configurations acts as a set of templates to spin up and tear down development and test lab setups. Because cloud automation products represent a central portal for provisioning computing resources for development and test needs, obsolete or abandoned servers that may be hiding under the desks of developers and testers can be eliminated for the most part. Because of the additional layer of abstraction that virtualization technology offers, administrators of preproduction environments can consolidate development and test servers and keep a close watch on actual resource utiliza-tion, which could mean fewer instances of blatant over- or under-provisioning.

This information can be used for allocating shared resources effectively, such as by using a scheduling and reservation system to use lab configurations. Software licenses meant specifically for preproduction purposes can be tracked more effectively and be safely retired when they have not been used for some time.

The lack of a preproduction resource management system can particularly hurt software development teams that are distributed around the world because it is uncommon for administrators of preproduction environments to be distributed around the world as well. Cloud automation is one way of keeping resource costs from spiraling out of control.


2.2.3 IMPROVED TESTING RELIABILITYWith a cloud automation offering, software developers and testers can provi-sion and spin up clean copies of configurations used in their software deploy-ments with much less effort and time. The reduced cost of running simple unit tests and complex integration tests alike means that they will likely be run more often. Because virtual machine configurations can be cloned very easily, the effects of even the most subtle changes in an application’s environment can be measured and documented. Testing for backward compatibility and across hardware and software platforms also becomes considerably easier than before.

Many of the cloud automation offerings profiled in this report provide the ability to take snapshots of complex configurations in any state of execution. These snapshots are a great way of communicating among members of a distributed software development team and are concrete artifacts to attach to issue-tracking processes within a team.

2.2.4 REUSE OF AUTOMATED PROVISIONING KNOW-HOW IN PRODUCTION

Spinning up and bringing down complex configurations of hardware and software on demand has uses that extend well beyond the application development lifecycle. End users of cloud automation may view it as a proving ground for automating provisioning and deployment on the production side of their organizations.

Several users of cloud automation offerings we spoke with reported that they are used for augmenting capacity during peak development and test workloads, such as those around a software release. The lessons learned from these deploy-ments can be applied to the problem of handling peak user loads on mission-critical applications with complex configurations, particularly at companies looking to stay ahead of their competition by experimenting with emerging models of public and private cloud computing.

Automated setup and tear-down features combined with basic user management features mean that external, licensed applications can be provisioned and deprovisioned complete with the machine on which they are running. A virtual machine can essentially act as a full-fledged capsule for an application and can be provisioned at a fine-grained level. Complex configurations of virtual machines that can be easily cloned and live-migrated to other physical machines can form the basis for speedy disaster recovery and more fully formed business continuity plans.


2.3 TECHNICAL REQUIREMENTS FOR CLOUD AUTOMATIONThis section contains an overview of the minimum set of features a cloud automation product should support in order to realize the business case for cloud automation outlined above. The vendors profiled in this report target their offerings at several different uses, but all of them support the requirements detailed in this section to varying degrees.

These requirements are stated in the abstract as guidance to end users looking to deploy cloud automation products within their environments; a subsequent section in the report contains a detailed matrix that compares the features of products from vendors actually profiled in this report.

2.3.1 A LIBRARY OF CONFIGURATION TEMPLATESCloud automation offerings must support a shared, centralized library of configuration templates. These templates must be more than just a list of configuration instructions; they must be actionable, in that a developer or tester must be able to provision underlying hardware resources for them and instantiate them into running configurations with little external help come build or test time.

In addition to serving as the conceptual heart of a test lab automation product, this library acts as a durable repository of knowledge about configurations over time and is a valuable medium of communication between the administrators and users of a preproduction environment. Accordingly, such a library should be searchable and support custom annotations that enable the library to be adapted to an organization’s preproduction environment.

To serve as an effective medium of communication among geographically dispersed teams with well-differentiated job descriptions, cloud automation products that explicitly target development and test functions must also have the ability to securely store development and test assets in one location. These assets and other library items can then be shared as ordinary links over the usual communication channels, including email and instant messaging.

2.3.2 COMPLEX CONFIGURATIONSCloud automation offerings must support the ability to create complex config-urations of machines to run builds and tests. These configurations should support not only configurations with virtual machines only, but also those hybrid ones composed of physical and virtual machines. Support for hybrid configurations is key for prospective customers looking to adopt cloud automa-tion incrementally, as well as for those that must include physical components such as storage area networks, network switches and other hard-to-virtualize hardware in their lab configurations.


As the hypervisor continues to be commodified, customers should be able to use whichever hypervisor they want. Given the number of vendors that are releasing hypervisors or incorporating them into their product portfolios in some form, pure-play cloud automation vendors that support multiple hyper-visor platforms will find themselves best placed to attract customers or even for an acquisition by a large vendor.

2.3.3 SELF-SERVICE PROVISIONINGSelf-service provisioning can deliver huge cost savings to users of cloud auto-mation products. Administrators no longer need to mediate every transaction between developers or testers and their working environment. Simpler, lower-touch transactions can be run directly by developers and testers on isolated virtual machine instances on which they could have full administrative access, if necessary. Self-service provisioning should also be attractive to organizations evaluating cloud automation as a test bed for provi-sioning automation for production applications.

2.3.4 USER MANAGEMENTOne of the biggest wins from cloud automation is the possibility for adminis-trators to reduce some of their low-touch workload by devolving reasonable controls of the preproduction environment to its users — namely, developers and testers. In geographically and organizationally distributed software development teams, a sophisticated, fine-grained user management system that enables authorization to be enforced at the level of individual users, functional groups and projects is indeed the bedrock of such devolution.

Ideally, the policy enforcement point for cloud automation products should be able to integrate with larger, organizationally mandated authentication and policy decision points such as LDAP or Active Directory. It is possible to guard against inappropriate use of preproduction resources by distributed software teams by instituting fine-grained control over user actions and setting up the ability to instantly grant and revoke permissions to users and teams in accordance with enterprise-wide policies.

2.3.5 GOVERNANCE AND REPORTINGThe ability to manage and report events that occur at several different levels — a project, a user or user group and at the system — is another key requirement for trusted devolution of control of the preproduction environment to its users. A durable record of actions associated with each artifact in the system enables audit trails to be generated and stored. Administrators and other relevant team members need to know about significant events such as physical hardware failures, or users or user groups overstepping previously allocated development licenses or other resources.


A sophisticated event management system would also enable developers and testers to specify actions to take when a serious error occurs on a shared preproduction resource. Such actions might include ignoring the error, suspending the currently running configuration, canceling subsequent tasks in the current job and so on.

2.4 USE CASES As isolated, transportable containers for application logic, cloud automation products are appropriate for a number of use cases within organizations that develop software for internal use or for packaging and deployment outside the firewall. This section contains a summary of customer experiences with cloud automation.

2.4.1 DEVELOPMENT AND TESTINGWell before server virtualization appeared on the horizon, virtualization on the x86 platform was used in development and test environments. Developers and testers are by no means unfamiliar with the idea of using virtual machines as isolation mechanisms for executing code. What is new with the cloud automation offerings surveyed in this report is the degree of automation and the remarkable time and cost savings that they bring to preproduction environments, particularly globally and organizationally distributed ones. Development and testing is by far the dominant use case for cloud automation offerings.

One user of cloud automation technology we spoke with is a midsized ISV with a product development team of 80 members spread across the US, India and Australia. This company reported provisioning times being cut down from two weeks to a less than a day. From the standpoint of administrative costs, a pool of 800 physical preproduction servers and 200 virtual ones is being managed by a team of four employees. The cloud automation offering it used also dove-tailed with the internal adoption of agile and test-driven development techniques such as Scrum. Consolidating development and test servers on physical servers also enabled this company to manage about 1,000 development and test servers for its global development team with a support staff of four.

Cloud automation offerings are well suited for functional testing at all levels — unit tests, integration tests and regression tests — but not for load testing. Today, fully virtualized workloads occupy a minority of the average enter-prise datacenter’s capacity. The complex configurations in virtual lab environ-ments may be functional replicas of production setups, but they don’t exactly replicate the performance characteristics of a production environment. For this reason, results of load tests run on virtual replicas of physical production infrastructure will be less indicative of performance in production than those of functional tests. For best results, cloud automation offerings for


this use case should integrate with developer tools in general, and build and test automation tools in particular.

2.4.2 SOFTWARE DEMONSTRATIONS AND TRAININGVirtual labs are manifestly appropriate as quick application delivery mechanisms in trusted settings such as an instructional lab or in the context of a pre-sales engagement. Such uses typically cross organizational boundaries and are held in varying environments. Therefore, in addition to strong configuration and user management features, a cloud automation product targeting this use case should support constructing complex, hybrid configurations from a broad selection of operating systems. Scripting support and integration with build and test automation tools are not necessary, but collaboration features that support desktop sharing among multiple users are crucial.

Although software product cycles have fallen dramatically in deference to greater user expectations, software sales cycles have not shrunk correspondingly. Proofs of concept and other pre-sales engagements are still fairly high-touch affairs that require significant investments of time and effort from vendors and prospective customers. With hosted cloud automation technology, a large ISV we spoke with creates a virtual machine configuration on publicly acces-sible infrastructure and simply sends a URL to a prospective customer who wishes to try it out. Recipients of these software demonstrations no longer need to provision hardware and software specifically for software evaluation purposes, but still receive the benefit of a demo into which they may be able to import live data.

From the software vendor’s point of view, the demos are time-limited and easy to set up and tear down. The software vendor can service many more leads and deploy valuable sales engineering resources only to well-qualified leads that have already seen some success with remote proofs of concept.

2.4.3 IT OPERATIONSThe complex simulation capabilities of cloud automation products are useful for bringing greater transparency to the process of deploying applications into production. When rolling out a change to a mission-critical production applica-tion, development, quality-assurance and IT staff try to certify it and anticipate possible outcomes from it using complex infrastructural validation techniques. Infrastructure testing is time-consuming and not always reliable because of differences between preproduction and production environments that are easy to miss. Although this kind of testing is carried out on a best-effort basis by well-trained, cross-functional teams, lack of adequate support from testing tools could lead to unexpected failures.


Cloud automation enables operations staff to clone production environments easily and thus keep preproduction and production configurations better in sync with each other. As true virtualized workloads come to be used more and more in the datacenter, replicas of production setups will be higher-fidelity copies of production environments.


SECTION 3:Disruptors in the MarketThis section contains an overview of disruptors that cloud automation vendors must keep in mind when formulating product strategies and roadmaps. In addition, we also present our opinion of how the vendors profiled in this report will respond to these disruptions. We define a market disruption as any key market condition, present or emergent, that challenges the status quo and stands to affect vendors from both a technical and a business-model point of view.

We visualize vendor responses to each market disruption in the form of a Disruption Cone for that disruption. Each Disruption Cone captures two parameters of a vendor’s response to a particular disruption:

• Stance – If a vendor is plotted in the upper, darker half of a Disruption Cone, we believe that vendor’s stance toward that disruption is proactive; a proactive stance toward a disruption would enable a vendor to anticipate that disruption more effectively. If a vendor is plotted in the lower, lighter half of a Disruption Cone, we believe that vendor to have a reactive stance toward that disruption; a reactive stance toward a disruption could mean that a vendor is affected adversely by that disruption. Vertical positioning in the proactive and reactive regions is done for layout reasons only and does not carry any special meaning. Proactive-reactive assessments apply in the context of a particular disruption only; the same vendor may well be proactive in the context of one market disruption and reactive in the context of another.

• Technology – If a vendor is plotted toward the right (mouth) of the Disruption Cone, we believe its technology represents a revolutionary way of dealing with the disruption relative to its peers in the Disruption Cone. If a vendor is plotted near the left (apex) of the Disruption Cone, we believe its technology to be an incremental, evolutionary step toward dealing with the disruption, relative to its technology prior to the disruption and to its peers in the Disruption Cone.

Readers must note that Disruption Cones are concise representations of a qualitative, ultimately subjective opinion — ours. It is not our aim to pit a disparate set of vendors against one another wholesale and anoint some as leaders and others as followers. Disruption Cones are an attempt at capturing our complex understanding of the disruptive forces operating in a market and how vendors are responding to them.


3.1 HYPERVISOR COMMODIFICATIONSeveral large hardware and software vendors have taken steps to account for virtualization in their product roadmaps. The hypervisor is fast becoming commodifi ed as a number of different fundamental virtualization technologies emerge. Prospective customers for cloud automation products usually have installed preproduction environments with signifi cant legacy components. Pure-play cloud automation vendors would be best positioned to acquire customers and eventually angle for an M&A exit if they pursue broad-based support for a variety of hypervisors and virtualization technologies.

The cloud automation vendors best suited to deal with this disruption are VMLogix, for its broad support of hypervisor platforms; Skytap, for its fully hosted approach; and Surgient, which has considerable experience hosting its offering.


3.2 CLOUD COMPUTINGCloud computing is a model of thinking about computing that combines an elastic pool of computing resources with a retail discipline. Cloud computing’s pay-as-you-go philosophy represents a major shift in the way environments — preproduction and production alike — are provisioned and allocated. Preproduction environments in particular are prime candidates for folding into a cloud computing model because of their lower-risk profi le relative to production environments and because of the option to avoid any up-front investment on them. Cloud automation vendors that offer hosted versions of their product are best placed to respond to customer demand for cloud-like economic models.

Skytap, naturally, is best placed to take on this disruption because of its cloud orientation from the ground up. Technologically, Surgient and Hatsize are also favorably disposed toward this disruption because of their experience hosting their products. Cloud computing, however, is both a technical and economic organizing principle. Skytap has priced its service in a manner attractive to those actively considering taking a cloud computing approach.


This Page Intentionally Blank


SECTION 4:ConclusionBasic virtualization technology got its first foothold in development and testing, and has moved into production workloads since. Now, another wave of innovation driven by cloud automation technology is finding adherents in the development-and-testing world. The immediate benefits of cloud automation technology to software development organizations in the form of better resource management and reduction in overhead should undoubtedly be obvious.

Beyond just these, however, development and testing uses of cloud automation technology will act as a proving ground for its deployment to the broader problem of datacenter management and automation. As virtualization continues to make inroads into production workloads and makes it much easier to create and deploy machines, effective automation and judicious resource management enabled by cloud automation technology will be key in combating virtual machine sprawl. The pot of gold at the end of the datacenter automation rainbow is huge — we are watching with bated breath to see who walks away with it.

Additional ResearchThis report is part of The 451 Group’s Infrastructure Computing for the Enterprise (ICE) practice. Closely related reports include: Virtualization: Reinventing Desktop Computing (July 2008)Managing the Virtual Revolution (December 2007)Blue-Sky Thinking About Cloud Computing (June 2008)

Future reports of interest include Virtually Automatic, a report on datacenter automation, expected to be released in November 2008.


TERMS OF USEAll research, analysis, data and any other information produced by The 451 Group (the “Group”), in any form, is proprietary to the Group and is protected by U.S. and foreign laws governing intellectual property. All such information published by the Group or presented by its employees, in any form, is copyright protected, inclusive of material appearing in a hard-copy format, electronically, on the Group’s website or via any other means. Since applicable laws do not require a copyright notice, the omission of the copyright notice by the Group does not invalidate copyright protection, and it does not indicate that the Group authorizes the reproduction of such proprietary material at any time.

Individual Named User Access: Access to the Group’s services is restricted to individual named users employed by a paid client organization (“Paid Client(s)”) and those individuals within orga-nizations to which the Group offers privileged, limited-time free access for the explicit purposes of evaluation only (“Trial Client(s)”), outlined below. In the instance of a Paid Client relation-ship between the Group and an organization, access to any and all material by that organization’s employees across any and all of the Group’s delivery formats is limited to the individual named users authorized by a Paid Client to make use of its account in accordance with the Terms of Use, outlined herein, and in accordance with the number of individual named users allowed in that Paid Client’s license to access specific services from the Group.

Trial Client Access: In the instance of a Trial Client relationship — i.e., an organization utilizing the Group’s copyrighted material on an unpaid basis and for the explicit purposes of evaluation only — usage is limited to individual named users within that Trial Client organization, except in instances where a broader evaluation is explicitly authorized for a Trial Client by its account manager at the Group. Should a Trial Client, at the conclusion of its trial period, choose not to become a Paid Client of the Group, then that Trial Client shall remove any and all copyrighted materials of the Group residing in electronic format on that Trial Client’s computer systems and storage devices, as well as destroying any and all copyrighted materials of the Group existing in printed hard-copy format in the possession of any and all employees of the Trial Client organization.

FORWARDING AND SHARING OF COPYRIGHTED MATERIALS: NO INDIVIDUAL NAMED USER WITHIN A PAID CLIENT ORGANIZATION OR A TRIAL CLIENT ORGANIZATION MAY FORWARD OR OTHERWISE REDISTRIBUTE COPYRIGHTED MATERIALS OF THE GROUP, VIA ANY MEANS, TO ANY THIRD-PARTY INDIVIDUALS AT ANY TIME WITHOUT THE EXPRESSED WRITTEN PERMISSION OF THE GROUP, EXCEPT AS PERMITTED, BELOW, VIA THE ‘EMAIL A COLLEAGUE’ FEATURE.

Analyzing the Business of Enterprise IT Innovation


Among Paid Client organizations and/or Trial Client organizations, limited forwarding of mate-rials from individual named users to individual, third-party non-licensed users may be permitted under certain circumstances via the Email a Colleague feature, which is built into the Group’s content delivery system. Permissible forwarding within an individual organization shall not exceed, per individual user account, on a monthly basis, a total of more than five (5) forwards in the 451 Market Insight Service, and three (3) in 451 TechDealmaker, sent each to a single recip-ient; further, permissible forwarding shall only occur via the Email a Colleague feature. A forward shall be defined as a single report sent to a single recipient. (For example, for the 451 Market Insight Service, this could be one (1) report sent to five (5) individuals, five (5) reports sent to one (1) individual or any other combinations in between.)

Exceptions to limitations on permissible forwarding within Paid Client organizations or Trial Client organizations may be made to accommodate promotional activities, offered from time to time, by an account manager of the Group (“Promotional Offers”) and at that account manag-er’s sole discretion. At no time shall Promotional Offers limit the copyright of the Group, nor shall they be construed to limit the Terms of Use outlined herein. Promotional Offers are inherently limited in time and scope, and may be changed, updated or rescinded at any time without notice, at which point limitations on permissible forwarding shall revert back to the basic terms outlined above.

There shall be no limit, via the permissible forwarding terms outlined above, to the number of times an individual named user within a Paid Client organization or a Trial Client organization may use the Email a Colleague feature to send reports to himself/herself only, for the purposes of offline viewing of reports or otherwise for his/her sole convenience; however, doing so shall only be permitted under the conditions that the reports must be forwarded to the email address connected to the individual’s user account, that the reports may not be forwarded to any third-party individuals at any time, under any circumstances, and that should the individual named user’s account be discontinued at any point, in consideration of the fact the (s)he is no longer a licensed user, (s)he must immediately delete any and all copyrighted materials of the Group residing in electronic format on his/her computer systems and storage devices, and destroy any and all copyrighted materials of the Group existing in printed hard-copy format.

The exception outlined above shall not include 451 Special Reports or any Adoption Research Services, including the 451 Grid Adoption Research Service, which are available only to indi-vidual named users with no exceptions, nor shall it include any other services, named or unnamed, provided by the Group to a client organization at any time. Should any Paid Client organization wish to have additional employees, beyond its current group of licensed, named, individual users, trial the service for the purposes of adding user licenses, then the Group may grant on a provisional basis additional individual trial access to these employees of the Paid Client organization.

At no time may materials produced by the Group be forwarded to email aliases — i.e., email addresses existing for the purpose of redistribution to a broader group of interested individuals.

Authorized Reproduction: Public reproduction or republication — including citations from analysts’ reports — of the Group’s material(s) is forbidden, except with the expressed written permission of the Group. To inquire about authorized reproduction, individual named users should contact their respective account manager(s).

451 group analysis - skytap.com group originally published this report as an overview of ......

Documents