Continuous Delivery Leveraging HP ALM &UFT and Skytap

Orasi Software at a Glance

Our Company ∙ Focus on enterprise

software quality management

∙ 1800+ Customers ∙ Partners with Mercury/HP

since 1994

Our Technology ∙ HP Test Management and Automation ∙ HP Enterprise Security Platform ∙ Perfecto Mobile MobileCloud ∙ Shunra, SAP, Blueprint

2013 HP Solution Partner of the Year (US) 2009, 2011 & 2012 HP Support Partner of the Year

2011 HP Software Partner of the Year (US)

Our Services ∙ Implementation Services

∙ Install and Mentoring ∙ Mobile, SAP, Security

and Performance Testing ∙ Onsite, Rural or Offshore ∙ Software Support

Business Demands Faster delivery of applications and new features

Better collaboration and alignment with business users

No decrease in quality

Mobility

Continuous Delivery

Challenges for Dev/Test Teams

With quicker project cycles, how can you accomplish all the necessary testing?

Do you have the appropriate infrastructure needed for testing at each phase and at quick intervals?

Test Automation

Unit testing & code coverage analysis

Functional testing the UI and API

Mobile platforms

Performance, load, scalability

Service virtualization

Agile Tricks with Test Automation

Time must be explicitly allocated for test automation in each sprint

In sprint retrospectives, automation goals should be considered as vital as the

development objectives

Automation should begin as early as possible and should not lag by more

than one sprint

Multi-layered testing allows for test automation in advance of GUI stability,

and furthers test coverage

1 2

3 4

Inefficiencies in the SDLC

Slow to order, slow to provision, low on IT’s priority list

Difficult to change or recycle hardware once ordered

Costly to duplicate production configurations

Impractical to share environments with customers or remote teams

Costly to maintain

Test Environments/Infrastructure

Skytap Cloud Test Environments Multi-VM environments with software defined networking

Spin up test environments on demand and in under a minute

Create environments only when needed, pay for what you use

CREATE CUSTOM ENVIRONMENTS

FLEXIBILITY

DEBUGABILITY

REPEATABILITY

COLLABORATION

Create golden templates of build & test environments

Create fresh instances of environments from golden templates

Create multiple copies of the environment in parallel

Save failures frozen in time templates for further investigation

Run application components in Skytap Cloud or on premise

Connect components together using self-service VPN connections

Use projects to enable teams to work together

Use one-click Publish URLs to share resources outside your organization

Demonstration

Jenkins

Detects that code was checked in and kicks off a build

Tells Skytap to start up the desired test environment

Deploys the new app to the Skytap test environment

Tells HP ALM to kick off the automated smoke test set

Pushes build results to HP ALM (ala ALI)

Developer Implements a user story in Eclipse and checks the code in to Subversion.

Test Engineer Simultaneously automates tests with HP UFT for the last sprint’s features and checks them into HP ALM.

Resources Webinar: Optimizing Test Teams for Continuous Delivery

http://www.orasi.com/news/Documents/2014-03-26Optimizing_Test_Teams_for_CD.wmv

Webinar: Cloud Enabled ALM: Continuous Integration & Automated Testing http://www.skytap.com/news-events/webinars/cloud-enabled-alm-continuous-integration-automated-testing-1

White Paper: Agile’s Essential Ingredient https://h20229.www2.hp.com/partner/protected/assets/pdf/1225_CC_HPAgileAutomation_WP_D3.pdf

Plugin: Skyap plugin for Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Skytap+Cloud+CI+Plugin

Plugin: HP Application Automation plugin for Jenkins https://wiki.jenkins-ci.org/display/JENKINS/HP+Application+Automation+Tools

Thank you!

Back-up Slides

+ + +

Jenkins Continuous Integration harness

Builds triggered by code check-in

Skytap plugin

HP Automation Tools plugin

Skytap Cloud Build, use, tear down configured test

environments on demand

Test environment deployed as a post-build step

HP Test Automation HP UFT and HP LR tests, stored in ALM or on file system

Invoked as a post-build step

Jenkins Plugin Capabilities

Orasi/Skytap Demo Architecture

Two Skytap Configurations

DEVELOPMENT ENVIRONMENT

Eclipse

Subversion Jenkins

Ant HP ALM

DEVCLIENT

DEVSERVER

SMOKE TEST ENVIRONMENT

HP UFT HP Sprinter

Online Banking

TESTSERVER

TESTCLIENT

10.0.0.4

10.0.0.5

9.0.0.1

9.0.0.2

• Uses 12 SVMs (out of 50) on Orasi Skytap account

• Two networks can talk to one another

Integration Testing - Continuous Integration (Breaking the Build)

Skytap Capability Value Delivered

• Fast environment creation

• Build failures are detected earlier, ensuring test teams do not get blocked

• Builds and integration happens continuously avoiding long periods of debug/fix time

• Integrations with Microsoft TFS and Jenkins for CI.

One of the main initial tests of incoming changes is whether or not a check-in will break a build. If the build breaks, developers need to determine how the build broke and why. Understanding how the build was broken and how long it takes to perform a complete system build can lead to hours or days of lost testing. Continuous integration addresses this problem by building and doing smoke tests after every check-in. Creating an entire environment quickly is generally not possible. There is typically an environment used for this purpose. This is inefficient as these resources sit idle most of the time and it serializes the check-in/build/test cycle across developers. Additionally it requires engineering work to automate the continuous integration lifecycle.

Systems Testing – Environment Contention

Skytap Capability Value Delivered

• Fast environment creation

• Build failures are detected earlier, ensuring test teams do not get blocked

• Builds and integration happens continuously avoiding long periods of debug/fix time

• Removal of contention across testing teams

Systems testing often occurs near the final stages of deployment when all components of the system are brought together and tested as an entire system. Test environments at this stage require more infrastructure and can be smaller than what is running in production. Test teams are limited to fewer environments leading to contention. Large test teams need to schedule time on test labs for their specific tests. This slows the SDLC as new features enter a queue for final testing before production release. IT will often push back on additional capex/time to setup and maintain environments for peak load testing and they often sit idle particularly in waterfall development.

Automated Regression Testing

Skytap Capability Value Delivered

• Fast environment creation

• On-demand ‘clean” environments for testing

• Automate regression test cycles and detect problems faster

• Reduce capex and waste by eliminating idle test environments

Regression testing is performed to ensure that none of the new code has broken any pre-existing use cases/features. This is typically done by running all of the prior automated tests to ensure they pass. While automated nightly builds will help ensure the build was not broken, doing automated regression testing on a clean “golden” test environment helps ensure that new code is not impacting the features customers have come to depend on. Automating regression testing allows your testing teams to focus on testing new capabilities vs. ensuring existing capabilities are not accidentally impact.

Systems Testing – Environment Decay/Contamination

Skytap Capability Value Delivered

• Fast environment creation

• On-demand ‘clean” environments for testing based on golden template environments

• Reduction of days to minutes to re-image and ‘disinfect’ environments

Environment decay/contamination is a large problem for teams that share a static set of test environments. As each individual or team takes their turn using an environment they may install software or an agent to help debug an issue. They may also make modifications to a configuration setting to see if it yields better performance for the application under test. Overtime the environment can become contaminated. Reproducing bugs in one environment may not be easily recreated in another environment.

Systems Testing – Capture Environments on Failure

Skytap Capability Value Delivered

• Fast copy/clone of environments

• Rapid creation and sharing of environments

• Higher productivity, sooner for new developers

When a system test reveals a problem, the test team enters a bug and a developer will attempt to reproduce the bug. If the developer cannot reproduce the bug, they are eventually granted access to the actual test environment where the bug was discovered. The testing team effectively becomes blocked while the developer debugs and resolves the issue in the test environment. This causes testing delays and slows progress.