DevOps & Quality

Dave van Herpen (@daveherpen)

| 2

|

Context: Scrum

3

quality

quality

|

Context: Enterprise Agility

4

2. Dealing with

size & complexity

3. Collaboration

throughout the

lifecycle

1. Portfolio, risk &

strategy alignment

Fit for future

Fit for enterprise

Fit for lifecycle

4. Integrating

external parties

Fit for integration

quality

quality

quality

quality

|

DevOps: value

5

INTEGRATED

QUALITY

QUALITY

AT SPEED

CONTINUOUS

QUALITY

QUALITY

OF USE Customer Satisfaction

Optimal value & risk

Short TTM

Efficient operations

Business value driven

Feedback loops

Fast flow

Multi- disciplinary

teams

Generic business driver DevOps principles Value

1

2

3

4

| 6

1) Business value driven (QUALITY OF USE)

Agile Development

fixes this Lean Startup

fixes this

DevOps Lite

fixes this

DevOps creates end-to-end Agility and Value creation.

|

2) Feedback loops (CONTINUOUS QUALITY)

7

|

3) Fast flow (QUALITY AT SPEED)

8

|

4) Multidisciplinary teams (INTEGRATED QUALITY)

9

T-shaped &

x-functional

|

Related themes & methodologies

DevOps

Agile

Lean

ALM

Cloud

CD

ITIL/

TMap/

….

|

…an organizational mindset

…for continuously improving value

from the digital value chain

…by enabling cross-functional collaboration

…on process, technology and behavior level

So…DevOps is…

|

CALMS

12

Culture Automation

Metrics Sharing

Lean

|

Sample DevOps metrics

Customer Satisfaction Rate (NPS)

Employee Satisfaction (Happiness Index)

Velocity (# story points per sprint put in production)

Quality (# defects per story point)

Mean Time To Value/Lead Time (days/hours)

Mean Time to Recover (minutes)

Deployment Rate (# deploys per day)

Release Predictability Measure (%)

13

|

Key quality challenges with DevOps (WQR)

1. Insufficient speed in testing due to insufficient levels of test automation (39%)

2. Difficulty in identifying the right areas on which test should focus (33%)

3. Lack of appropriate test environments and data (31%)

4. Lack of professional test and quality expertise in teams (31%)

5. Lack of good testing approach that fits with DevOps approach (29%)

6. Difficulty to re-use and repeat tests across iterations (29%)

DevOps Quality

Approach

Key Solution Components

1

|

1) Integrated DevOps & Quality approach

Key principles include:

15

Zero-touch

testing Continuous quality

monitoring Quality is a team

responsibility

Understanding

value, priority & risk

|

Key quality challenges with DevOps (WQR)

1. Insufficient speed in testing due to insufficient levels of test automation (39%)

2. Difficulty in identifying the right areas on which test should focus (33%)

3. Lack of appropriate test environments and data (31%)

4. Lack of professional test and quality expertise in teams (31%)

5. Lack of good testing approach that fits with DevOps approach (29%)

6. Difficulty to re-use and repeat tests across iterations (29%)

DevOps Quality

Approach

DevOps Quality

Automation

Framework

Key Solution Components

1 2

|

2) Zero-touch QA Automation Platform

17

The DevOps Quality

Automation backbone:

BDD & TDD enabling tools

Configuration Management

Continuous Integration

Build Automation

Deployment

Defect management

Functional Testing

Exploratory Testing

Performance Testing

Service Virtualization

Environment Virtualization

Test Data Management

Continuous Quality Monitoring

Predictive Analysis

Version Control

TFS

SVN

GIT

Mercurial

Build/Unit Test

IDE

Maven

Gradle

JUnit/Nunit

Mockito

Code Quality

Sonar

PMD

Findbugs

Corbetura

Continuous

Integration

Jenkins

Teamcity

Cruise Control

Functional

Testing

Selenium

QTP/UFT

Cucumber

Capybara

Source Code Management

DevelopmentQualityAssurance

Deploy To TestIntegration Testing

Feature Toggling

Togglz

Non Functional

testing

Performance• Jmeter, LoadRunner,

StormRunner, NeoLoad

Security• IBM Appscan, HP

Fortify Paros Proxy, Fiddler

Deployment Tools

Capistrano

Urban Code

HPE Codar

Configure Servers

Chef

Binary Version

Control

Nexus

Artifactory

Deploy toProduction

UserAcceptance

Deploy to PreProduction

InfrastructureProvisioning

Packaging &Archiving

Monitoring (Nagios, Munin, Dynatrace)

|

Key quality challenges with DevOps (WQR)

1. Insufficient speed in testing due to insufficient levels of test automation (39%)

2. Difficulty in identifying the right areas on which test should focus (33%)

3. Lack of appropriate test environments and data (31%)

4. Lack of professional test and quality expertise in teams (31%)

5. Lack of good testing approach that fits with DevOps approach (29%)

6. Difficulty to re-use and repeat tests across iterations (29%)

DevOps Quality

Approach

DevOps Quality

Automation

Framework

DevOps Quality

Engineer

DevOps Quality

Environment

Key Solution Components

DevOps Quality

Metrics Model

1 2 3 4 5

|

19

Monitoring

Monitoring

Monitoring

Continuous

Delivery

Test

Automation

(zero-touch)

Empathy

Blameless

Failure

Git, GitHub, Chef, Puppet,

Selenium, JUnit, Vagrant, Maven, Jenkins, Elastic,

UrbanCode, JIRA, OpenShift,TFS,

Splunk, Docker,

AWS, Azure, BlueMix

Built-in

quality,

security &

resilience

Value

Metrics

& Flow

PSP

CS X-funct

behavior

Cloud, Microservices & Containers

Agile & Lean

|

Moving beyond Agile