john rhodes - devops automated testing

DevOps Automation Best Practices

Agenda

• Why Development Automation and Agility

• Automated Build Best Practices

• Automated Test Best Practices

– Database compare tool

What is DevOps

DevOps

A recent global study, commissioned by CA Technologies,

which canvassed 1,300 senior IT decision-makers

worldwide, revealed respondents had seen a 17% to

23% improvement with regards to increased

earnings, shorter time-to-market, heightened

competitiveness and greater customer satisfaction

due to their DevOps initiatives.

Mode 1 vs Mode 2 Organizations

Two Types of Organizations: Mode 1 “linear” vs Mode 2 “non-linear”.

Is one better? Not necessarily.Does DevOps Apply to both? Yes!

Gartner

By 2020, 80% of the practices associated with DevOps

and Mode 2 “Agile” management will be adopted by

Mode 1 organizations.

DevOps is a Bridge

Key message: Automate as Much as Possible,

regardless of your star status!

One way to get there

7

• DILBERT © 2007 Scott Adams. Used By permission of UNIVERSAL UCLICK. All rightsreserved.

Best Practices of Agile Management

• Manage Agile Projects

– Manage user stories/tickets/use cases -> Backlog

– Plan, prioritize, order, specify, assign resources

– Execute sprints, work on sprint backlog items

– Monitor progress

– Deploy changes

• Transparency

– Reports

Best Practices Release Automation

SCRUM – an agile framework

Sprints

User Stories

Kanban Board

OPEN DEVELOPMENT TEST COMPLETED2007697

Check out process

Story Points: 0

2007693

Add price/discount information

Story Points: 6

2007692

Update product detail information

Story Points: 2

2007674

Create Welcome Page

Story Points: 2

Login Process2007698

Enter credit card information

Story Points: 0

2007695

Browse product catalogue

Story Points: 0

2007694

Upoad product pictures

Story Points: 3

2007675

Create new login user

Story Points: 4

Customer Management2007699

Track shipment

Story Points: 0

2007696

Select product for shopping cart

Story Points: 0

2007676

Login with user name and password

Story Points: 5

Product Management2007700

View Order History

Story Points: 0

2007677

Request new password

Story Points: 4

Product Order Process2007701

Report order summary per month

Story Points: 0

2007678

Update security questions

Story Points: 3

Reporting2007702

Report cancelled orders YTD

Story Points: 0

2007679

Disable login user

Story Points: 2

2007703

Report order backlog

Story Points: 0

2007685

Set up Infrastructure for AgileX development

Story Points: 4

2007704

Cancel order

Story Points: 0

2007686

Load backlog for AgileX development

Story Points: 3

2007705

Report orders by product group

Story Points: 0

2007691

Create product

Story Points: 4

KANBAN BOARD

Story Mapping

AgileX

│

LOGIN PROCESS CUSTOMER MANAGEMENT PRODUCT MANAGEMENT PRODUCT ORDER PROCESS REPORTING

2007674

Create Welcome Page

Story Points: 2

2007681

Create new customer

Story Points: 6

2007690

Create product groups

Story Points: 1

2007696

Select product for shopping cart

Story Points: 0

2007701

Report order summary per month

Story Points: 0

2007675

Create new login user

Story Points: 4

2007682

Update customer information

Story Points: 3

2007691

Create product

Story Points: 4

2007697

Check out process

Story Points: 0

2007702

Report cancelled orders YTD

Story Points: 0

2007676

Login with user name and password

Story Points: 5

2007683

Disable customer from ordering products

Story Points: 1

2007692

Update product detail information

Story Points: 2

2007698

Enter credit card information

Story Points: 0

2007703

Report order backlog

Story Points: 0

2007677

Request new password

Story Points: 4

2007684

Integrate google maps for customer location

Story Points: 3

2007693

Add price/discount information

Story Points: 6

2007699

Track shipment

Story Points: 0

2007705

Report orders by product group

Story Points: 0

2007678

Update security questions

Story Points: 3

2007695

Browse product catalogue

Story Points: 0

2007694

Upoad product pictures

Story Points: 3

2007700

View Order History

Story Points: 0

2007679

Disable login user

Story Points: 2

2007704

Cancel order

Story Points: 0

2007680

View login history

Story Points: 3

STORY MAPPING

Burn Down Charts

Sprint 0 From 09/30/2013 To 10/04/2013 Estimated Story Points: 7

Date Day Story Points Burn Down Ideal Burn Down

9/30/2013 1 0 7 7

10/01/2013 2 3 4 5

10/02/2013 3 0 4 4

10/03/2013 4 0 4 2

10/04/2013 5 4 0 0

Sprint 1 From 10/07/2013 To 10/18/2013 Estimated Story Points: 23

Date Day Story Points Burn Down Ideal Burn Down

10/7/2013 1 0 23 23

10/08/2013 2 2 21 20

10/09/2013 3 0 21 18

10/10/2013 4 4 17 15

10/11/2013 5 0 17 13

10/14/2013 6 9 8 10

10/15/2013 7 0 8 8

10/16/2013 8 0 8 5

10/17/2013 9 3 5 3

10/18/2013 10 2 3 0

BURN DOWN CHARTS

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10

Sprint 1 From 10/07/2013 To 10/18/2013

Burn Down

Ideal Burn Down

Sprint 2 From 10/21/2013 To 11/01/2013

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9 10

Sprint 0 From 09/30/2013 To 10/04/2013

Burn Down

Ideal Burn Down

What has changed?

CM First Group. 15

CA Plex CA 2EIBM i

LibraryFilesystem

SVN Repository

Automated Deployment

CM First Group. 16

Plex Function.dll

.pnl

.dll

Plex FunctionPlex FunctionPlex FunctionPlex FunctionPlex FunctionPlex FunctionPlex Functions in PKG and Code

library

IBM I

Java WebClient

.NET

.warPlex FunctionPlex FunctionPlex FunctionPlex FunctionPlex FunctionPlex FunctionPlex Functions in

PKG

Plex Function & DBRPGIV

LF and PF

WinC

Update a Model

15.06.2016 Page 17

The extracted/checked in models can be selected for update into the group model (depending on configuration)The Model Manager unlocks the group model and opens up the localmodel.

Summary

• Moving towards Agile / DevOps

– Short Cycles

– Transparency, Inspection, Adaption

• Use Tooling

– AUTOMATION is crucial for Agile/DevOPs

Best Practices in Automated Testing

CM First Group. 19

Cost of Defects

1:10:100

Tortoise and the Hare

Slo

w c

yle

Fast

Cyc

leHigh Defect Low Defect

Promise of Test Automation

• Speed

– Automated tests fast to run

• Flexibility

– Can be done off hours

• Accuracy

– Not prone to human error

• Reusability

– Without boredom

• Maximizing People

– QA modifies and perfects the process

Tortoise and Hare Reprise

Slo

w c

yle

Fast

Cyc

leHigh Defect Low Defect

Payback Times

Cu

mu

lati

ve $

$$

Time / Test Cycles

Naïve Automation

Manual

Next Gen Automation

Types of Test Automation

• Functional Testing

– Unit Tests

– Integration Tests

– Regression Tests

• API Testing

• Load Testing

• Database Compare

• Code Test Coverage

• Vendors

– Worksoft

• 5250/3270, SAP (Certify)

– SmartBear

• Web, Windows (TestComplete, LoadComplete, SOAPUI)

Best Practices to Get Started

• Choose a tool that can do end to end testing across

technologies

• Look at existing testing closely for actual coverage

– Considering using coverage tools

• Change the way you think about time

• Don’t try to “boil the ocean”. Pick your 25 top processes.

Certify Process

Users and Groups

Applications and Projects

Application Maps

Versioning Maps

Versioning Maps

Requirements

Processes

Variables and Datasets

Executing Processes

Executing Processes

Executing Processes

Executing Processes