Automated Testing in PipelinesIntegrating Tools and Concepts

Layla Franke, Vlad Ilie, Alina Ionescu

dev.haufe.com

github.com/Haufe-Lexware

@HaufeDev

-Lexware

Scope of this Talk

• Coded UI for UI, Postman + Newman for API (LicenseManager – Layla)

• Testing Framework for Web UI in Java with Selenium (Umantis – Vlad)

• Framework in Java for API testing, Soap UI for API testing (Foundational Services – Alina)

1 LicenseManager - Layla

Coded UI for UI, Postman + Newman for API

License Manager

• C# Web Application

• Inhouse Tool for Management of Serial Numbers

• Existing Continuous Delivery Pipeline in Go.CD

GUI Tests

Coded UI Tests in Visual Studio:

• Recording via Internet Explorer and built-in recorder

• Execution of tests from Commandline with MSTest

• Playback of Tests in various browsers

Build Pipeline

Compile

Deploy Application

Coded UI Test

Internet Explorer

View Test Reports

Do other stuff

Pulish App

QualityDev

Coded UI Tests – Pipeline integration

• Compile your code normally

• Deploy you application on test machine

• Start Coded UI Tests

You can use MSTest.exe on Go Agents

• Use parameterized URL and Browser

Define environment variables in GoCD

• Test Replay runs on Go Agent

• Coded UI Tests Finished

View HTML Test Reports and Coverage Report in Pipeline

Coded UI Test

Chrome

Coded UI Test

Firefox

Go Agent

Go Agent

Go Agent

Coded UI Test Reports - Overview

Coded UI Test Report - Failed

Coded UI Test Report - Passed

Create REST API Tests using Postman

Use Postman to create REST API Tests:

• Export Tests as a „Postman Collection“

• Have a variable URL using a „ Postman Environment”

Newman – Pipeline integration

• Deploy application on App Server

• Execute Newman Tests

Newman is a command line tool used for

automatic execution of tests created with

Postman

• View Test Results in Pipeline

Build Pipeline Go Agent

Newman Test

QualityDev

Pulish App

App Server

View Result

Compile

Deploy App to Server

CD pipeline integration

How to integrate in CD Pipeline?

Visual Studio Coded UI Tests

• Test DLL is created in Compile Job

• Tests run after deployment on test environment in stage QualityDev

• Tests for different browsers can be executed in parallel

Newman REST API Tests

• Can be executed after deployment on test environment

2 Umantis - Vlad

Testing Framework for Web UI in Java with Selenium

About Umantis

• Web project (perl)

• 3 applications

(with stable and beta customers)

• Bugfix release every 2 weeks

• Feature release every 3 months

• Prio1 releases

Test Project Structure

• Java project

• Built using maven

• Selenium web driver

• Page Objects

• testNG

• Git/gerrit

• Jenkins

IDE

• Intellij

• Implement

• Run

• Debug

Test Implementation

• Start from test

• Implement steps as they are

needed

• Checks after actions

@Test(groups = {"smoke.extended", "applications"})public void SMK049_test_delete_application_from_internal_as_HR() {

new ApplicationsPage().openAndLoginAs(hrUser.getLogin(), hrUser.getPassword()).selectApplicantByName(jobApplicantAniston.getFirstName()).deleteApplicationLinkClick().deleteApplicationSubmit().searchLeftForApplicantByName(jobApplicantAniston.getFirstName()).checkApplicantDoesNotExist(jobApplicantAniston.getFirstName());

}

Page Objects

• Keep UI elements private

(don’t expose them to tests)

• Expose user actions

• Use data structures instead

of multiple parameters

public class JobApplicationExternalFormPage extends PageBase {

@FindBy(id = "value_153712")WebElement dataReleaseRadioButtonGroup;..............................@FindBy(id = "Button_1161113")private WebElement submitFinalApplicationButton;

@FindBy(id = "Button_112465")private WebElement continueButton;

@FindBy(id = "errorcollectionlabel_3606")private WebElement errorMessage;.............................private void fillJobApplicationForm(User applicant) {

UIActions.waitForElement(titleRadioButtonGroup);UIActions.setValue(new RadioButtonGroup(titleRadioButtonGroup),

applicant.getTitleAsValue());UIActions.setValue(firstNameEditBox, applicant.getFirstName());......................................................................UIActions.setValue(letterOfApplication, applicant.getLetterPath());UIActions.setValue(resume, applicant.getCvPath());UIActions.setValue(document1, applicant.getDoc1Path());try {

UIActions.setValue(new RadioButtonGroup(dataReleaseRadioButtonGroup),RadioButtonValue.DATA_NOT_AGREE.getValue());

} catch (NoSuchElementException ignored) {}

UIActions.click(iAgreeCheckBox);}.............................

Page Objects - examples

• PageBase.java

• CreateNewEmployeePage.java

• AddPartialEventPage

UIActions

• Static Class

• Entry point for all ui actions

• setValue() and checkValue() for

different types of controls

Test Execution

• Jenkins – a job for each product and branch

• Local – using intelliJ

• On each commit (continuous integration) using gerrit and jenkins

• Maven

• Command line (can be used with any CI or CD tool):

java –jar umantis_test.jar ...

Test Environment

• Selenium Grid using docker containers

• Build test project container (based on maven image)

• Test execution container (based on jdk image)

• Selenium grid (based on hub and node selenium images)

Test Results - overview

Test Results – Detail

• Test failure with history

• Exception with stack trace

• URL, screenshot and html

where the failure occured

3 Foundational Services - Alina

Framework in Java for API testing, Soap UI for API testing

What we started from:

•Atlantic – the monolith

15 + clients…

Splitting the testing:

• 1. DEV Tests – Junit tests

• 2. System Tests - Junit tests

• 3. SOAP UI tests

• 4. Security Tests

• 5. Load Tests

• 6. Rollout (Docker) tests

• 7. UI Automated tests

• 8. Exploratory (manual)

The Testing Framework

Junit Test Framework

Atlantic

CodeClients

Helpers

Tests

SoapUI Tests

DEV

Unit Tests

Security Tests

Load Tests

Rollout

Docker TestsAtlantic

Client

Atlantic

Client

Atlantic

Client

HP ALM

UI Tests

Manual

Tests

All the endpoints:

Tests examples:

@Testpublic void get_userWithUserId() {

UserDto user1 = UserClient.createNewPendingActiveUser();Map<String, String> queryParams = new HashMap();queryParams.put("userid", user1.getUserID().toString());com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto result =

RestUtilForOldResources.get(com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto.class, headers, queryParams, PathsHelper.umPaths().users());

assertUserValues(user1, result, true);

Where are they triggered from:

• After deployment, from GoCD there are triggered 2 Docker compose files:

• 1 with Maven for the Junit tests

• 1 with SoapUI for the SoapUI tests

They run in parallel.

Results in Rocket.Chat:

Things we learned

• Achieve “good enough” coverage

• Prioritize

• Take/create shortcuts if you can

• Bypass Logins if possible

• Write small “straight to target” tests (no end-to-end anymore)

• Use more code and API less UI

• Communication is key to good collaboration

Goal?

Faster, more reliable releases.

4 Thank you!

Layla, Vlad, Alina

@HAUFE

Dev Ops