challenges and approaches to automating testing of software

David Montero - Challenges and Approaces to Automation Testing of Software

1

Challenges and Approaches to Automating Testing of Software

Reading, October 7th 2016

07/10/2016


2

JSON, XML

SeleniumSikuli

Approaches…

07/10/2016


3

Unit tests: maximum granularity◦ Assert 2=sum(1,1)

Quick Same codebase Immediate feedback Need knowledge of the code Minimal functional meaning

Unit Test Frameworks

07/10/2016

TDD


4

Collection of related Unit Tests Still need to have codebase knowledge Enhanced functional meaning… but not

complete! Still quick and immediate feedback

◦ TestCalculator(a,b,x,y,z…)◦ Assert x=sum(a,b)◦ Assert y=prod(a,b)◦ Assert z=div(a,b)◦ ….

Component Test Frameworks

07/10/2016


5

Contract based Requirements come into play: functional

tests Inputs and outputs to full system (E2E) /

components / single component Different approaches:

◦ Large inputs: Data Driven Framework◦ Behaviour Driven Framework: cucumber, jbehave◦ Keyword Driven Framework

Integration and API tests

ALL OF THEM COMBINED!

07/10/2016


6

Full system End to End, functional tests Costly: time, tools, development Must be kept SIMPLE Flaky! Combined with Behaviour, Data Driven or

Keyword driven frameworks

07/10/2016

Automated GUI Tests


7

How many tests?

07/10/2016

Challenges?

“AS MANY AS NEEDED, AS LITTLE

AS NECESSARY”


8

Decision making process:◦ What is the impact of not having a test?◦ What is the cost of building it?◦ Time constraints?◦ Tools needed?

07/10/2016

How many? Risk based approach

£


9

“Why did the automation not detect this?”

“Will it be 100% automated?”

“We won’t need manual test”

07/10/2016

Common misconceptions

EXPLORATORY TESTING!!


10

Test environments challenge◦ Use of DOCKER

Performance testing◦ Gatling

Continuous Integration challenge◦ Continuous testing◦ Automated release build◦ Speed up of releases

07/10/2016

Other Challenges to consider


11

NLP Frameworks? Machine Learning Self correcting software?

07/10/2016

The Future! (present?)


12

Counting goals!

07/10/2016

Practical case


13

Risk? Approaches:

◦ Unit tests◦ Data driven◦ Behaviour driven

07/10/2016

Counting goals£1,000,000 !


14

Unit tests :assertEquals(“1-3”, scoreGoal(HOME))… All the possible cases? Scope? Input, output

07/10/2016

Counting goals


15

Data driven framework Run the last 20? Man U – Arsenal games Assert scores Is data complete? Penalties, extra time…

07/10/2016

Counting goals


16

Behaviour Driven framework Team involvement – business! All speak the same language? Ambiguity? Duplicity?

07/10/2016

Counting goals


17

Keyword driven framework Similar to behaviour driven Free “syntax”

07/10/2016

Counting goals


1807/10/2016

Thanks for your attention!

Q&A

challenges and approaches to automating testing of software

Documents