1CONFIDENTIAL

Testing of the backend systemOctober 15, 2016

FROM CHAOS TO TEST AUTOMATION

2CONFIDENTIAL

ROMAN SOROKA• Lead test automation engineer in EPAM

• Activist of COMAQA.by community• Over 7 years in IT.

• Relocated to Minsk from Moscow

• Roles in the career: developer, software engineer in test, QA lead, delivery manager.

3CONFIDENTIAL

I am from Sarov

4CONFIDENTIAL

Why Minsk

Minsk =

= Moscow

5CONFIDENTIAL

Why Minsk

Minsk =

= Moscow

6CONFIDENTIAL

Why Minsk

Minsk =

= Moscow

7CONFIDENTIAL

Why Minsk

Minsk =

= Moscow

8CONFIDENTIAL

Why Minsk

Minsk =

9CONFIDENTIAL

Why Minsk

MinskEPAM =

>4000epamers

10CONFIDENTIAL

Roman Sorokarnsoroka@gmail.com

11CONFIDENTIAL

The ground rules

25 minutes 15

minutes

12CONFIDENTIAL

1 2 3 4

Agenda

The first steps on the project

Dealing with chaos Selling the test automation

The framework architecture inside

out

13CONFIDENTIAL

Intro

14CONFIDENTIAL

Intro

15CONFIDENTIAL

The first steps on the project

16CONFIDENTIAL

Expectation vs Reality

17CONFIDENTIAL

Expectation vs Reality

18CONFIDENTIAL

EXPECTATION REALITY

Expectation vs Reality

Clear workflow

Specified acceptance criteriaUsable documentation

Good working autotests

19CONFIDENTIAL

•Ask questions•Confirm answers or their absence•Fill the knowledge base

Solutions

20CONFIDENTIAL

Knowledge transfer

Find out what was expected from your predecessor!Make sure that Customer knows how it was done!

21CONFIDENTIAL

Ask questions

22CONFIDENTIAL

Get confirmation

23CONFIDENTIAL

Ask questions again

24CONFIDENTIAL

Summary for the first steps

•Gather knowledge as many as possible!

•There are no too many stupid question to ask!

25CONFIDENTIAL

Kb.e

pam

.co mThink

about yourself in the future and other people!

Fill the knowledge base

26CONFIDENTIAL

Dealing with chaos

27CONFIDENTIAL

•Earn trust and respect•Make friends•Find out the politics behind•Start offering incremental improvements

Solutions

28CONFIDENTIAL

Earn respect

Do some simple and required task much better then it was done before!Use it as an example of good work!

29CONFIDENTIAL

Earn respect

Get known for your knowledge and understanding of the job

30CONFIDENTIAL

Earn trust

Help out people if you can!It will earn their trust.

31CONFIDENTIAL

Make friends

Invest in good relationships! They help in achieving all goals!

32CONFIDENTIAL

Politics behind

33CONFIDENTIAL

Find allies

34CONFIDENTIAL

Find allies

Find ones who can be trusted.

35CONFIDENTIAL

Find out why the things are in the current state of affairs!

Find out reasons

36CONFIDENTIAL

Find out reasons

We need to show the Client that we can achieve the expected results!

37CONFIDENTIAL

Incremental improvements

Propose to improve some small thing which will give the meaningful value!

38CONFIDENTIAL

Incremental improvements

39CONFIDENTIAL

Incremental improvements

40CONFIDENTIAL

Selling test automation

41CONFIDENTIAL

•Ask advices•Gather facts•Calculate benefits•Present the proposition to the customer

Solutions

42CONFIDENTIAL

Ask advices

43CONFIDENTIAL

• The number of defects detected in production in Jan-Jul 2016 is twice higher than in the same period 2015.

• The ratio of defects detected in production amounts to 31.5%. (The industry recommended value is less than 10%)

• The total sanity test coverage amounts to 34%. (The value recommended by customer standards is above 80%.)

• The immediate test automation technical debt is more than 1000h.

• The development team spends up to 30% on the bug fixing per iteration.

Gather facts

44CONFIDENTIAL

Calculate benefits

  Time to implement the suggested TAF

  Elimination of the technical debt

  Time savings with the new TAF 

1 key tester works 64 hours per iteration in the ideal time in both cases for the elimination of the debt only

Existing TAF                       25

i1 i3 i5 i7 i9 i11 i13 i15 i17 i19 i21 i23           

Suggested TAF    5        7        

  13

i1 i3 i5 i7 i9 i11  i12  i13 i15 i17 i19 i21 i23

45CONFIDENTIAL

Present proposition to the customer

46CONFIDENTIAL

Be flexible

47CONFIDENTIAL

The framework architecture inside out

48CONFIDENTIAL

BACK-END. What’s that?

49CONFIDENTIAL

•Allow the computer to do the math for you•Collaborate with developers•Use the cases as the documentation•Make it easy for other people to use•Create easy readable results of testing

Solutions

50CONFIDENTIAL

Allow the computer to do the math for you

Create a sanity test caseCreate all positive requests with

required fields

Create all negative requests without required fields

Create all positive requests within boundary conditions

Create all negative requests outside boundary conditions

Create all positive requests with correct type of arguments

…

51CONFIDENTIAL

Use TAF to generate all available permutations of test cases.People don’t need to think about each available combination it should be done automatically once the requirements are specified.

1

Soltuion#1

52CONFIDENTIAL

Collaborate with developers

53CONFIDENTIAL

Take the functional created by developers don’t reinvent the wheel. Use development code data base as well.

2

Soltuion#2

54CONFIDENTIAL

Use the cases as the documentation

55CONFIDENTIAL

Use TAF to generate the most actual and comprehensive documentation from your test cases.It will contain working request and real responses and be always up to date and be complied with the acceptance criteria for the task.

3

Soltuion#2

56CONFIDENTIAL

Make it easy for other people to use

Once the framework is created anyone without automation knowledge should be able to add test cases to it!

57CONFIDENTIAL

Create TAF to allow adding cases even to business people if they want it.Adding test cases should not require knowledge beyond the average computer user.

4

Soltuion#3

58CONFIDENTIAL

Create easy readable results of testingCreate easy readable results of testing

59CONFIDENTIAL

Write in the main log only really useful information on the state of affairs.Logs and results should not contain all the information, but the necessary one which allows to quickly identify the root cause of the problem.

5

Soltuion#4

60CONFIDENTIAL

THANK YOU FOR ATTENTION!

??? QUESTIONS ??

?

61CONFIDENTIAL

Create easy readable results of testingTHE END