automated testing in continuous change management

#

Marc HornbeekSr. Solutions ArchitectSpirent Communications

Experiences of Automated Testing in Continuous Change Management

#

Marc HornbeekSenior Solutions ArchitectSpirent Communications

Marc Hornbeek is Sr. Solutions Architect of automated test management solutions at Spirent Communications, Infrastructure Test Optimization (ITO) BU. He recently managed SCM and DevOps at Spirent. As primary architect of test automation tools and champion of test automation for firms ranging from start-ups to large multi-national companies he published more than 30 articles and has been a speaker at numerous international conferences and user forums.

#

• Network test systems and solutions• Global engineering teams• Transitions:

– Perforce to GitFusion/DVCS– SCM to DevOps– Large Waterfall to Agile Modular Releases– Train Model: “Freight train” > “Rapid Transit” > “Movers”

Spirent Communications

#

Continuous Change Management

Continuous Testing provides continuouscontrol for DevOps CCM

CI CT CD

#

Changes drive builds and tests

Continuous Test System

Test results drive changes

Change Management

System

Report and TrackingSystem

Build Systems

(Real or Virtual) Test Systems(Real or Virtual)

Artifacts Repository

SUT Systems(Real or Virtual)

Process Automation Tool

Poll

Start

Checkout

StartRun

PullPush

Reports And Logs

PullPull

Change Request

Run

#

Agile Train Model CTThis presentation describes Spirent’s journey of testing solutions and tools for faster and more modular trains.

#

Train Model Release Velocity

Large infrequent releases

Monthlysegment releases

Continuouspackagereleases

Separatetest setups

Networked DevOps

Central DevOps

Freight Train Rapid Transit Modular Mover

#

Continuous ControlEntiresystem

Segmentspecific

Modularpackages

Systemdecisions

Modularreverts

Integration stage tests and reverts

#

Build / Test System Cycle VelocityLarge infrequent releases

Monthlysegment releases

Continuouspackagereleases

Full build& regress96 hour

Incr. build & regress1 to 8 hours

Incr. build full regress 32 hours

#

Lower Test Cost, Higher CoveragePhysical labs

Virtuallabs

Fixed multiple ofsingle lab

Shared resource

Cloudlabs

On-demandresource

#

Test Version ManagementSeparate agents

Centralagent

Distributedagents

Singledepot

GitFusion / DVCS

Databasecontrolled

#

Test SuitesEntire SUT

SUT Segment

Modularpackages

Fixedsuites

Modular sequencedsuites

Taggedsuites

#

Test AnalysisEntire SUT

Segmentspecific

Modularpackages

Fixedreport

Analysis network

Segmentanalysis

#

CT Admin / % AvailabilitySeparate agents

Centralagent

Distributedagents

Various admin tools<90% Avail.

Central admin tools> 99% Avail

Central admin tools > 95% Avail

#

Engineering Process and CultureSeparate Processes

CommonProcess

DistributedProcess

Distinctcultures

Culture alignments

Cultureadjustments

#

• Culture changes • Solution architecture• Modular systems• Change-driven

automation• ROI driven• Incremental budget

Pitfalls and Lessons Learned

#

• Change-driven configuration• Scalability desktop to global lab• Mixed virtual and real SUT • KPI Dashboard• Novice to expert user interfaces• Services for test creation (Create), test set-up

(Connect), test execution (Play) and test results analysis.

CT Toolkit Checklist

#

• Seamless integration, rich APIs and plug-ins.• Pre-packaged tests• Variable skill level test IDE• Automated results

CT Toolkit Checklist (continued)

#

• CT automation journey is challenging but ROI is high: velocity, stability, coverage

• Each CT capability level achieved enables the next so keep the train moving!

• Choose CT automation solutions and tools with change-driven control capabilities

Summary

##

Thank you!Marc Hornbeekmarc.hornbeek@spirent.com

#

RESOURCESCD Report: info.perforce.com/cd-report

White Paper: perforce.com/white-paper-link

Template: perforce.com/template