automation testing in agile project

Hien Nguyen @AgileVietnam #MonthlyMeetup Aug 2015

§ Agile testing manifesto§ Our project & approach§ Automation test with continous integration§ Pitfalls & best practices

2

4

Collaborative ownershipDetached objectivity

Targeted automationWidespread anti-regression

Defect preventionDefect detection

Exploratory testingPredetermined scripting

over

5Source: Anand Bagmar – ThoughtWorks

Connect with 3rd

party systems

Internal use

Tool (Website)

6

test, test, test…Automate as most as possible

TEAM responsibility for BUILT-IN quality

7

less repeatable works = more VALUE

Testing mindset trainer

Test plan & execution manager

Rollout gateway

8

Automation

Deployment

Integration

Test

9

Unit

• Single module

Integration

• Cross modules

• Contact data layer

Performance

• Scalable environment

UI

• Simple UI operations

10

11Source: ThoughtWorks

§ MUST HAVE & first step in automation testing chain

§ Best practices:§ Naming convention§ Batch

12

§ Best practices§ Grouping§ Isolation§ Keep it simple

§ Issues: Test data§ Pre-defined data§ Generated data

13

§ Best practices§ Test scripts under source control

§ Solution:§ Octopus deployment

§ Website: IIS§ Clients as run-once Windows service à invoke Jmeter

§ Scalable clients with Hyper-V

§ Trigger:§ After integration test§ Deployment

§ Test server§ Clients

14

15

Hyper-V

TeamCity Octopus

pull

deploy

§ Best practices:§ Keep it simple§ Not (too) complex test case§ Naming convention

§ Solution:§ Selenium test code§ Integrate to build pipeline

16

§ Automated test data / code / plan.. generation?

§ Indication: § When project’s structure is being stable for some sprints§ Modules’ structures look similar § Test code structures look similar

§ But please careful§ We need manual works to maintain the automated works§ Today it reduces work but tomorrow…

17

Our objectives

§ Transparent: Team can real-time know project status

§ Improvement: Team can improve mindset and (testing, coding) skills

§ Fewer manual works, fewer people, fewer breaks…by smarter way J

19

§ Cost reduction, ROI increase

§ No longer or not frequently maintain needs

§ (Too) love UI or system tests

§ Only QA responsibility for automation test

20

Need

• objectives• team

Implement

• as soon as possible

• more unit test, less UI test

• team

Continuous

• refactor• execute

Don’t

• need to test everything

• worry to disable tests SOMETIMES

Remember

• You have MANUALworks for AUTOMATEDworks

• Trade-off

21

Automation test is not a tool that let us work from Monday to Thursday and let the machine do repeatable works in Friday. It’s a method to let us work from Monday to Friday safety by knowing our issues and preventing bugs everyday.

We have to do the manual works for automated test, includes the testing for it. So more automated testing could be done, more manual work need to be done.

The total cost usually doesn’t change. But whenever we want to execute large set of test cases in a short time, we can – that we normally cannot do by using people.

22

§ We are hiring a tester for mobile team @Planday

23

24

25

Certified ScrumMaster®September 21-22, 2015Hanoi

Certified Scrum Product Owner®September 21-22, 2015Ho Chi Minh city