tricentis continuous testing platform - kms solutionstricentis continuous testing platform mike...

32
1 Tricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement

Upload: others

Post on 28-May-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

1

Tricentis Continuous Testing Platform

Mike Schlabinger, VP Global Sales Enablement

Page 2: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

4

What is Continuous Testing?

Continuous Testing is the process of executing automated tests as part of

the software delivery pipeline in order to obtain feedback on the business

risks associated with a software release as rapidly as possible.

• Right Feedback, Right Time, for the Right Stakeholder

• Can happen at any point in the SDLC

Page 3: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

5

Cloud-based performance labs

at the fingertips of every

developer and tester for on-

demand load testing.

DistributedLoad Testing

Resilient regression testing

across any architecture or

application stack at the speed

of change.

AutomatedContinuous Testing

Scalable, in-sprint test

management for open source

test automation, exploratory

testing and BDD.

Agile Dev Testing

Software Testing Reinvented for Agile and DevOps

Tricentis Continuous Testing Platform

Page 4: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

6

Cloud-based performance

testing labs on-demand

DistributedLoad Testing

Resilient regression testing at

the speed of change

AutomatedContinuous Testing

Open source testing and agile

test management

Agile Dev Testing

Software Testing Reinvented for Agile and DevOps

Tricentis Continuous Testing Platform

Page 5: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

7

Cloud-based performance

testing labs on-demand

DistributedLoad Testing

Resilient regression testing at

the speed of change

AutomatedContinuous Testing

Open source testing and agile

test management

Agile Dev Testing

Cross Project and Portfolio Visibility

Tricentis Analytics

Tricentis Continuous Testing Platform

Page 6: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

8

DistributedLoad Testing

AutomatedContinuous Testing

Agile Dev Testing

2xRisk Coverage

30-40%Cost Reduction

2-3x Release Speed

10xPerformance

Compared to other testing tools

Tricentis Continuous Testing Platform

Page 7: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

9

Tricentis AnalyticsMetrics

Joint Analytics & Reporting

Cross-Project Reporting

CustomDashboard

AnalyticsDashboard

ReportingDatabase

ToscaProjects

qTestProjects

Page 8: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

10

Integration Roadmap

Risk Based Prioritization

Test Case DesignOptimize

AutomatedContinuous Testing

UI Automation

API TestingAutomate

Active Test Data Management

Test Driven Service VirtualizationProvide

Exploratory Testing

Manage

Test Management

Manual Testing integrate

2

Presented at Tricentis Accelerate Conference in Vienna

12.0, Oct 2018

12.1, Q1 2019 Public availability

1 2

12.2, Q2 2019

3

Public availability

Risk-based Testing and Test Case Design shared qTest and Tosca

Unified Dashboard

1Analytics

Agile Dev Testing

Risk Based Prioritization

Test Case DesignOptimize

3

Active Test Data Management

4

13.0, Q3/Q4 2019

4

Public availability

Active TDM shared qTest and Tosca

Provide

Internal, Subject to Change

Page 9: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

11

Expansive Technology SupportO

PE

N

SO

UR

CE

RPG

TE

CH

NO

LO

GY

SU

PP

OR

TD

EV

OP

SS

UP

PO

RT

Page 10: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

12

Software Testing Reinvented for DevOps

Make open source tools more productive• Central, agile test management

• Orchestrate execution across tools

• Consolidated analytics and reporting

Page 11: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

13

Software Testing Reinvented for DevOps

Language and legacytechnology support

• Single interface for multiple languages

• Model-Based Test Automation for rapid creation, management and execution

• Consolidated analytics and reporting

RPG

Page 12: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

14

Software Testing Reinvented for DevOps

Industry’s broadest support for Packaged Applications

• Business focused interface for ease of use

• Model-Based Test Automation for rapid creation, management and execution

• Consolidated analytics and reporting

Page 13: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

15

Software Testing Reinvented for DevOps

Complete support for SAP

• Single, business interface for all SAP UIs

• Model-Based Test Automation for rapid creation, management and execution

• Resilient end-to-end tests across technologies

SAP WinGUI SAP Fiori

Page 14: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

17

Software Testing Reinvented for DevOps

Agile Dev Testing

• Agile Test Management by providing manual and automated test traceability to Jira issues, bugs, and releases in real-time

• Shift Quality Left with open source test automation management, test script scheduling, and support for BDD

• Predictive Test Analytics to gauge release readiness through executive dashboards & portfolio level reporting

Page 15: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

18

Software Testing Reinvented for DevOps

AutomatedContinuous Testing

• Continuous Testing with model-based test automation and support for open source tools throughout the DevTest process

• Central Platform with seamless integration into DevOps toolchains for developers, testers and business users

• High Speed Releases by optimizing and orchestrating end-to-end tests across teams, projects and applications

Page 16: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

19

Software Testing Reinvented for DevOps

Distributed LoadTesting

• Easy Test Creation with ability to create tests in open source and proprietary tools for performance testers of all skill levels

• Flexible, Massive Scale by leveraging the cloud through shared or private AWS and Azure integrations that fits into your DevOps pipeline

• Rich Bottleneck Analysis that highlights areas of risk for quick resolution of issues

Page 17: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

23

Integrate

Prioritize

Design

Plan Code & Test Design Version Build Test in Sandbox Deploy to Stage

Monitor & RPA

Provision

Integrate

Virtualize

Automate

Configure

Integrate

Explore

Manage

Integrate

Distribute

Execute

Report,

Automate & Integrate

PLAN DESIGN BUILD CONTINUOUS TESTING DEPLOY MONITOR

Page 18: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

24

A Sprint in a Tester’s Life

Refinement Sprint

Epic / User Stories created

User Stories (de-)refined

Acceptance criteria (ATDD/BDD) aligned

Model based SV – Test

Model-based API Test

Model based UI Test, Web Desktop & Mobile

Test Data Management

Load Test

Exploratory Testing

As UI-User I want to retrieve customer data via json service (Middleware)

REPORTINGRisk & TCD

Provide Test Artifacts as “Self-Service“ in & across teams

DEVs

QA

PO

GUI Load Test

As a User I want to view customer data (UI)

Page 19: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

26

About Tricentis

Page 20: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

30

Top Analyst Recognitions

“It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agilecontinuous automation practices.”

“Tricentis‘ Tosca Testsuite makesautomation easy with model-basedautomation. Via Tosca Testsuite, Tricentisprovides top test automation andoptimization design capabilities, test assetreuse and combined automation.“

Page 21: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

31

Dec 2018

Jan 2011

Aug 2014

Dec 2015

Nov 2017

Page 22: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

32

Dec 2018

Jan 2011

Aug 2014

Dec 2015

Nov 2017

“It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agile continuous automation practices.”

Page 23: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

33

#44

Awards & Accolades

Page 24: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

34

Agile testing for core insurance system,

4.5M lines of code

19K tests → 3.5K tests(-82%)

46% → 93%(2x)

4 weeks → 3 days(9x)

2K tests → 0.6K tests(-70%)

Continuous testing for online banking system

30% → 85%(2.8x)

120 hours → 30 min (240x)

Continuous Testing through a Digital TCoE,

all core systems

6 weeks → 5 days(8.4x)

12K tests → 4.2K tests(-65%)

43% → 92%(2.2x)

Page 25: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

35

Bringing standardization to 600+ agile teams with

30+ automation tools

Providing cutting edge digital experiences for

65 million active users

Doubled QA licenses of qTest in 2nd year to

support global teams in Japan and California

Launched PlayStation Store by centralizing QA results

with integrating to Jira

Standardized all hardware and digital

app testing across qTest

More than 400+ test automation agents

connected for a single view of application health

Centralized reporting across 30+ automation tools and frameworks

Clear visibility into manual to automated progress for executives to see QA value

Moving from Legacy QC/ALM to modern agile

practices

Scalable users across multiple NetApp divisions will millions of test results from home grown test

automation

Best of breed approach for agile transform using Jira

Software + qTest

Migrated 25 large scale projects including test cases,

plans, linked traceability

Page 26: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

36

Transformed TCOE from Legacy ALM to modern

CI/CD model

Scaling extreme test automation for API, SDK,

UI, & performance

Providing digital customer experiences across web, and mobile

Compliance through centralizing QA results

and integrating with Jira

Scaled open source test automation from hundreds

to thousands

Decreased average regression testing time from 2 days to 2 hours

Reduce test maintenance time through trend

analysis

Substantially reduced critical production defects

from 15 avg to 0

Increased test automation to 83%

Scaled monthly test activity to 153 platforms

tested, 422 builds & 300,000 test runs

Scaled number of test runs executed from 45 to

7,000 per minute

Reduced a test run reporting from hours to

minutes

Page 27: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

100%

ManualTesting

0%

AutomatedTesting

TestingPresent

Customer Case Study

There’s a way to do it better. Find it!

Project Charter

11Manual

Testers

4755Manual

Test Cases

?Unknown

Coverage

10Weeks

Execution

Page 28: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

Risk Coverage Optimization Test Data Management

48Hours

Execution

Single Agent

8Hours

Execution

Multiple Agents

Customer Case Study

11Manual

Testers

4755Manual

Test Cases

?Unknown

Coverage

10Weeks

Execution

89%Business Risk

Coverage

1193Manual

Test Cases

37%Business Risk

5Weeks

Execution

2.5Weeks

Execution

92%Automated

Test Cases

53%Business Risk

Coverage

Smoke Testing

34Minutes

Execution

Smoke Testing

Test Case Count 7%

Automation (UI & API) & OSV Continuous Integration & Distributed Execution

75% Redundancy 50% Effort Test Data

64% API Tests

Page 29: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

8Hours

Execution

Multiple Agents

Customer Case Study

72%Critical Defect Reduction

Production Defects

89%Business Risk

Coverage

1193Manual

Test Cases

5Weeks

Execution

2.5Weeks

Execution

92%Automated

Test Cases

34Minutes

Execution

Smoke Testing

53%Business Risk

Coverage

Smoke Testing

Test Case Count 7%

Test Case Design & Exploratory Testing

64% API Tests

Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution

Page 30: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,
Page 31: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

SAPWithin five months a team of three implemented the coreregression suite, including 333 business scenarios across various business functions such as Financial Processes (Accounts Payable, Accounts Receivable, General Ledger), Procurement, Controlling, HR, Project related Processes, and automatic Invoice Processing, covering 34,237 test actions covering 6,734 verification points. The total execution of the SAP automated regression suite takes less than 24 hours; this would take 518 hours to execute manually. By running the automation suite the total effort is reduced by 96%.

RMS Case Study

Page 32: Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike Schlabinger, VP Global Sales Enablement. 4 ... Accelerate Conference in Vienna 12.0,

42

Questions?