the shift to quality engineering - accenture€¦ · qa is asked to delivery transparency into root...

The Shift to Quality Engineering

Bart Gourley

2016 Accenture Testing Symposium

February 2016

2Copyright © 2016 Accenture All rights reserved.

Today’s

Discussion Industry/market pressures and the quality

challenge

Trending responses for testing organizations• Structural transformations (enabling a quality

factory)

• Rollout of new roles

• Shifts to new/different tools

What’s next?

Industry change in is disrupting internal quality process, putting some testing organization at risk if they fail to adapt.

Industry/Market Pressures And the Quality Challenge

Challenges for Testing OrganizationsIndustry Drivers of Change

Growth of new and potentially disruptive

B2C IT products in historically stable

markets

Increasing business agility requirement pressures from dynamic client demands

Continued downward cost pressures

• Stakeholders: Integration with development, business, and

customer partners within the quality workflow often requires

continuous and active collaboration through non-IT

traditional methods (e.g. waterfall vs. agile/lean)

• Methodologies: Structured multi-speed IT test

methodologies (e.g., internal industrialized agile, waterfall,

extreme, hybrid approaches) are required where they often

do not exist.

• Focus: Engineering ‘for’ quality is replacing testing for

quality. QA is asked to delivery transparency into root

causes of quality impacts both up and downstream from

testing (e.g., shift left and shift right).

• Skills: New/Non-traditional QA roles and skills (e.g., SDET,

Techno testers, Quality Czar, etc.) are required.

• Value: The challenging economic environment retains a

focus on higher value SDLC activities. However, innovation

has been slow to respond to creating more value at lower

costs.Crucible for Transformation

• Uber and Shared Car

Services

• Cable and Content

Streaming Services

• Theaters and Direct to

On Demand

• Market feedback

rates have

increased (app

store, real-time

surveys, etc.)• Budget alignment

with product market

performance and

value

Copyright © 2016 Accenture All rights reserved. 3

Quality organizations are seizing the moment to evolve — not simply to survive, but to elevate their value.

Trending Responses for testing organizations


Challenges What actions are others taking? What benefits are they seeing?

• Stakeholders

• Methodologies

• Focus

• Skills

• Value

• Test organizations are moving away from being

external to the development team and towards

being part of the engineering team through the

staffing of roles with new skills (e.g., SDET, QA

engineer, etc.)

• Test organizations are shifting left and right to

influence and to sometimes control quality

across the delivery.

• Centralized test organizations are shifting away

from Testing Center of Excellences towards

Quality Center of Excellences and are held

accountable for more than defect leakage

• Test organizations often need to reorganize

from a structure and capability perspective to

support the new engineering models and/or

requirements

• Tighter integration with internal

and external (market)

stakeholders

• Increased perceived value by

stakeholders

• Decreases in UAT defect leakage

and improvement in market

feedback

• Increased transparency into the

true drivers of quality (e.g.,

development code readiness

‘green builds’, requirement

readiness, and crowd acceptance)

Industry is redefining how software is developed, tested, & delivered. Lean engineering is rapidly evolving the QA space.

Trending Responses for testing organizationsStructural Transformations > Enabling Quality Engineering


Core Concepts• Quality is a by-product of the engineering process, not a point-in-time

function

• Quality activities are deeply embedded into release engineering

(DevOps), product performance (business/market definition and

success monitoring), and technical development

• Quality organizations and their activities are lean at the core and scale

variability for execution

Common Traits of Mature Quality Factories• Technical level quality activities (TDD, BDD, etc.) are formalized and

SDETs (or the equivalent function) are prolific supporting these

functions

• Quality oversight and control is injected into each part of the SDLC

• Leaders in the space envision the SDLC as a lean

process that, like any other manufacturing process, has

waste that should be reduced to optimize for cost, time to

market, and market relevance.

• Shifting left and right removes traditional test stages

and gates and blends quality measures into each function

of the engineering process.

The intersection of DevOps, Traditional QA, Development, and Market Feedback

A holistic quality engineering approach places more testing and automation left in the delivery workflow.

Trending Responses for testing organizationsStructural Transformations > Example Run State

6Copyright © 2016 Accenture All rights reserved.

Forming Norming & Storming Performing

• The SDET term is applied to any

non-development/test resource

that is providing testing within the

development team.

• Typical Environmental

Factors/Prerequisites: None

• The SDET term is applied to any engineering

resource that is providing low-level automation

and test coverage that is stored within the source

code repository.

• Typical Environmental Factors/Prerequisites:

Development and development integration

leveraged in testing

• The SDET term is applied to any engineering resource that

is providing low-level automation and test coverage that is

stored within the source code repository, providing

coverage and execution integration into the continuous

delivery solution, that collaborates with integration level

validations from external teams and that delivers within an

agile methodology structure.

• Typical Environmental Factors/Prerequisites: Continuous

Integration solution is leveraged

Establishing a Quality Czar and SDET capability is a journey that typically starts with identifying muti-functional/’utility’

resources and ends with those resources engineering portions of the production solution*.

Trending Responses for testing organizationsStructural Transformations > ‘New’ Role Definition


* Conceptual and for discussion only

- Quality Czar: SDLC experienced resource that has a mixed skill background of PM, QA, Development, and

Release Engineering. The resource provides governance and input into all engineering activities to drive for quality

outcomes

- SDET: technical testing resource that frequently provides low level automation and manual testing services within a

sprint team.

- QA Analyst: functional testing resource that responsible to provide behavior and/or business transaction validation

of the solution within an integration team.

SDET Journey Example

Example role functions within an DevOps driven development workflow

Trending Responses for testing organizationsStructural Transformations > ‘New’ Role Definition


SDET Activity

QA Analyst Activity

Developer Activity

Automated Activity

Test driven development:

Developer cuts feature branch,

writes test case, tests, writes

code, tests and submits for review

Gulp provides

Automated task

scripting

Bamboo begins

automated build &

deploy process

JS Hint, JSRC, HTMLLINT rand

other code quality run and

perform analysis

Feedback is passed to

developers

QA Testers test on multiple devices

(Automated Regression (selenium) +

Manual Functional)

Once code review is approved, Git

with Atlassian Stash is used to

merge the code update with

development branch

Cod e artifacts, including applications are

stored/retrieved in NPM and Bower JSON

used to integrate with Rest API

Gulp runs automated Karma

behavior with Jasmine assertions

and test reporting with Istanbul

Gherkin

Behavioral driven development:Automation tester writes Gherkins scripts that become inputs for drafting unit test cases.

Istanbul

Test results are sent to

SonarQube server for analysis

Automated Smoke Testing (Build Validation)

Code is packaged & readied for

deploy by Gulp

Code is deployed to environments by

Bamboo via Docker

Analysis of testing

results and

determination of follow-

up / action items

Creates automated and manual tests for

continuous validation activities

Creation and execution of E2E, full

integration, performance, crowd,

and other testing.

Creates and integrates unit tests into

code base

Quality Czar

Tooling needs to support a quality engineering workflow blend functional traditional and technical development tools

Trending Responses for testing organizationsStructural Transformations > Shifts to new/different tools


* Conceptual and for discussion only

Tooling focus is on supporting the

continuous and often automated

delivery and analysis of quality

measurements.

Continuous

Delivery

Architecture

Relevant

Tooling landscapes are evolving to

support a structure of core+. The

core is centrally supported where

as the + is supported only within

the scrum or application stack.

Cloud Enabled

Tooling is more frequently on-

demand and cloud supported either

through SaaS or frameworks that

leverage dynamic infrastructure

Tooling scope and breath looks at

the entire SDLC and promotes

adoption of requirement capture

and traceability to deployment

performance. Example: BDD

Cucumber & JBehave.

Workflow

Questions?

the shift to quality engineering - accenture€¦ · qa is asked to delivery transparency into root...

Documents