planning your testing & automation environment

PLANNING YOUR TESTING & AUTOMATION ENVIRONMENTViranand Strout, MANDIANT

(NOVATAIG 7/13/2011)

A Little About Me…

Shameless Recruiting

Software Testers (Automation ++) Capacity and Performance Engineers Software Engineers …and much more!

Alexandria (HQ) and a new Reston office (Summer 2011)

is hiring!(http://jobs.mandiant.com)

What’s the problem?

The environment drives the test coverage.

How about this?

The tests drive the environment.

Head Scratching

I like software but that network and machine configuration stuff is boring.

What’s the deal with it happening in production but not in QA?

I’m the only one who can run my automation

Developer who? You can use that machine… but don’t

mess it up! It only happens on my machine

Where Do I Start?

• Test Requirements

• PriorityTest

Requirements

• Budget• Resources• Priority

Environment Requirement

s • Who?• What?• How?

Process

• Tie it together

• Test• Refine

Build

Test Requirements

• Test Requirements

• PriorityTest

Requirements

• Budget• Resources• Priority

Environment Requirement

s • Who?• What?• How?

Process

• Tie it together

• Test• Refine

Build

Breaking it Down

Installation/Deployment Functional System Configuration Mgmt Scale/Performance Security/Communication Automation Continuous Integration Others…

Example: Configuration Management

I need to be able to test my software on 10 operating systems in any given build

How do I quickly pull up those configurations and make sure that they are ready for test?

What I need: I need x bare metal environments and and x virtual environments at my disposal. These environments need to be stood up and reverted in a clean way.

Environment Requirements

• Test Requirements• Priority

Test Requirements

What is the SA Worried About?

Virtualization Firewalls Network Servers Backup/Restore Uptime Throughput Environment

Bloat

Example: Network Architecture Subnets, routing Active Directory Firewall traffic Need to know: How do we anticipate

different components within a system to be installed in production or on a customer site and how do they communicate to each other?

Process

• Test Requirements• Priority

Test Requirements

We Don’t Need No Stinkin’ Process

Who will use it? How often? How will they use it?

Ease of use Accountability Recoverability Speed

Example: Deployment Strategy

Backup

test environment

Initialize environmen

t

Run deployment

Validate

Restore

Build the Environment

• Test Requirements• Priority

Test Requirements

Tie it Together

Installation/DeploymentFunctional

SystemConfiguration MgmtScale/Performance

Security/CommunicationAutomation

Continuous Integration

VirtualizationFirewallsNetworkServers

Backup/RestoreUptime

ThroughputEnvironment Bloat

QA SA

Pretty Pictures

Sample Software Test Environment

Security and Network

Subnet 1 Subnet 2

Subnet 3

Port 1433 Port 80

Templates and Cloning

Template Environment

Initialization Scripts

Virtualized Environment

Templates and Cloning

Template Environment

Virtualized Environment

QA Environment

Development EnvironmentAutomation Environment

Automation and Continuous Integration

Template Environment

Virtualized Environment

Automation Environment

Virtualization API

CI Server

Test AutomationServer

Templates and Cloning

Template Environment

Virtualized Environment

QA Enviro Config 1QA Enviro Config 2

Virtualization API

Test AutomationServer

Viranand StroutViranand.Strout@Mandiant.com

Thank You!