workshop agile testing table tennis player and coach improve it services bv 4 www....

45
A Practical Perspective Erik van Veenendaal www.erikvanveenendaal.nl Workshop Agile Testing

Upload: others

Post on 15-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

A Practical Perspective

Erik van Veenendaalwww.erikvanveenendaal.nl

Workshop Agile Testing

Page 2: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Workshop Agile Testing

� Introduction, ISTQB and CTQB

�What is Agile (Testing)?

�Problems and Challenges

�Agile Testing Practices

Improve IT Services B.V. 2

Page 3: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Know Your Audience

� Do you know / Have experience with:

- Agile, SCRUM, ISTQB, TMap

- Others…………?

� What role do you have?

� Any testing experience?

- How many years?

� In which domain do you work?

� Which lifecycle do you use?

Improve IT Services B.V. 3

Page 4: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Erik van Veenendaal

� In testing since 1989 working for many different clients and in many different roles

� Author TMap, ISTQB Foundation, TMMi and many other books and papers

� Former Vice-President International Software Testing Qualifications Board (ISTQB)

� CEO TMMi Foundation � Regular keynote and tutorial speaker� Winner Testing Excellence Awards� Living in Bonaire (Caribbean island)� Table tennis player and coach

Improve IT Services BV 4

www. erikvanveenendaal.nl

Page 5: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

� World’s leading organization for certification of professionals in software testing

• Benefits include- raising industry standards

- promoting career development

- providing competitive edge for employers

� More than 700.000 exams, more than 500.000 certificates

� 3rd largest IT certification scheme

� Foundation, Advanced and Expert certificates

ISTQB Certification

Improve IT Services B.V. 5

Page 6: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

World -wide Coverage

Improve IT Services BV 6

� Countries covered by Member Boards (57 Member Boards covering 81 countries, representing over 90% of the world-wide GDP) and Global Exam Providers

Page 7: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

ISTQB Structure and Syllabi

Improve IT Services BV 7

Page 8: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Now also in Curaçao (region)

Improve IT Services BV 8

To contribute to the quality of software products being delivered in Curaçao and its region, through better testing to be enabled by enhancing the knowledge and skills of test professionals and to

support the further development of the testing profession.

www.ctqb.org

Page 9: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Agile Manifesto

Improve IT Services BV 9

Page 10: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Whole -Team Approach

Team should include:

� Everyone with the knowledge and skills necessary to ensure project success

� Representatives from the customer and other business stakeholders who determine product features

Benefits include:

� Enhanced communication and collaboration within the team

� Leveraging the various skill sets within the team to the benefit of the project

� Quality is everyone’s responsibility

Improve IT Services BV 10

Page 11: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Early and Frequent Feedback

� Avoid requirements misunderstandings

� Clarify customer feature requests, available for customer use early

� Discover, isolate and resolve quality problems early

� Provide information to team regarding productivity and ability to deliver

� Promote consistent project momentum (flow)

Improve IT Services BV 11

Page 12: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

12

SCRUM

Improve IT Services B.V.

Page 13: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

13

Agile Key Characteristics

� Iterative development, “build a little, test a little”

� A tangible product available earlier in the life cycle

� Strong business involvement

� Early validation (and verification)

� Collaborative approach, e.g., during the definition of requirements and test design

� No signed off requirements; based on user needs

� Regression testing increasingly important

Improve IT Services B.V.

Page 14: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Impr

ove

IT S

ervi

ces

B.V

.14

and

a pi

ctur

e …

Live

Ope

ratio

n

Define

Develop

Build

Test

Implement

Define

Implement

Define

Develop

Build

Test

Implement

Develop

Build

Test

Spr

int 1

Spr

int 2

Spr

int 3

Page 15: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Top 5 Agile Benefits

1. Ability to manage changing priorities

2. Increased team productivity

3. Improved project visibility

4. Increased team motivation

5. Better delivery predictability

Improve IT Services BV 15

Source: 10th Annual State of Agile™ Report

Page 16: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Agile ≠ Silver Bullit

16

1

10

100

1000

10000

10 100 1000

Project Size (KLOC)

Def

ects

“despite continuous growth in adoption of agile, many organizations are still facing the same challenges as in previous years”, World Quality Report

Page 17: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Tester survey, Is Agile working?

Improve IT Services BV 17

23.3

43.3

33.3

NoPartiallyYes

Page 18: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Agile -like / Fragile

Improve IT Services BV 18

It’s simple, but not easy!!

Page 19: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Risk Assessment

� Be aware, Be ready

� Risks: Factors that could have negative influence on the Agile testing implementation project

� Based on past experiences- Using a checklist

� Likelihood and Impact� Discuss the risks thoroughly with stakeholders

- What can be done to mitigate the risks

- Track actions and assign risk owners!

Improve IT Services BV 19

Page 20: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Exercise – 3/5 minutes only

� In small teams discuss possible risks when implementing Agile testing – make a list!

� Things you could consider …..- Organization- Processes- People- Team- Methods & Technique- Environment- Automation & tooling- ……

Improve IT Services BV 20

Page 21: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

21

Agile Testing Challenges (1)

� Test automation is not that easy

� Re-usable test scripts for regression testing

� Maintain product overview

� No full requirements specification (scope creep)

� End-to-end testing is done late, if at all

� Communication with users (business language) and developers (technical language)

� Time boxing

� Audit trail from requirements to code / test cases

Improve IT Services B.V.

“Agile implementation risk list”

Page 22: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

22

Agile Testing Challenges (2)

� Involvement of business and real-users

� How to define acceptance criteria

� Maturity level of the organization

� Testing still responsible for quality

� How much documentation is enough

� Understanding of Agile manifesto

� Technical and business knowledge of testers

� Testing knowledge of developers

� Testing non-functional’s within a sprint

Improve IT Services B.V.

“Agile implementation risk list”

Page 23: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Agile Testing Best Practices

Improve IT Services BV 23

Page 24: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Getting the most out of it

� Learn by doing (and not just by listening)

� Make your mind-map for these best practices!

Improve IT Services BV 24

Page 25: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

� High level description of test levels / types for an Agile project, and testing to be performed within each level.

− What testing is done in iterations, e.g., what is unit testing?

− Contains description of test levels− Addresses the generic product risks− Coherent with non-sprint test levels− May well take (partly) over from test plan & test process

Practice #1: Test Strategy

Improve IT Services BV 25

Objective, ResponsibilityLevel of independenceEntry and exit criteria

process and deliverablesTest design techniques

Test StrategyGeneric Risks Blueprint

Page 26: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Tool: Agile testing Quadrants

Improve IT Services BV 26

Page 27: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Practice # 2: Risk -Based

� Product Risk Session

� Lightweight: Risk Poker

� Risk-Based Test Approach

Define Clear Testing Priorities

Testing is risk-based

Communication, Customer CollaborationWorking software & Responding to change

27Improve IT Services BV

Page 28: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Setting Priorities

�Product risk identification

- User story based, brainstorming, failure history

�Product risk analysis

- Risk = impact x likelihood

� Impact, e.g., safety damage, usage intensity, external visibility, ….

�Likelihood, e.g., new technology, complexity, interfacing, ….

28Improve IT Services BV

Page 29: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

HL

L

Product Risk Matrix

29

II

IV

I

III

H

M

M

xx

x

x

x

Likelihood

Impact

Must Test

Should Test

Could Test

Would Test

R2

R1

Improve IT Services BV

Must test

…… Test Approach X…..

Should test

…… Test Approach Y…..

Could Test

…... Test Approach Z…..

Would Test

…... Test Approach A …..

Page 30: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Improve IT Services BV 30

Practice #3: Acceptance Criteria

- User Story: what is needed and has to be built(and tested)

As a student

I want to buy a parking pass

So that I can get to school quickly

rationale, to prevent “gold plating”

Page 31: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

31

Acceptance Criteria

� We have to be able to tell whether a solution completely satisfies, or fits, a requirement

� Refinement or grooming sessions

� To make user stories testable

� Also very important for non-functional user stories

� It is usually necessary to discuss acceptance criteria with the stakeholder, e.g.,

• 90% of the customers must be able to get the correct ticket from the product in no more than 25 seconds

involve tester here

Improve IT Services B.V.

given <context> when <the event occurs> then <expected outcome>

Page 32: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

User Story & Acceptance CriteriaAs a ISACA conference attendee, I want to be able to register with ISACA Curacao online, so I can register quickly and cut down on paperwork.

Questions by the tester might include� What information needs to be collected to allow a user to register?

� Where does this information need to be collected/delivered?

� Can the user pay online as part of the registration process?

� Does the user need to be sent an acknowledgment?

Acceptance Criteria� A user cannot submit a form without completing all the mandatory fields.

� Information from the form is stored in the registrations database.

� Protection against spam is working.

� Payment can be made via credit card.

� An acknowledgment email is sent to the user after submitting the form.Improve IT Services BV 32

Page 33: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Practice # 4: Unit Testing

� Code Coverage

� Buddy Testing (XP)

� Test Driven Development

Get Developers doing Unit Testing

33

Testing requires independence

Working software over comprehensive documentation

Improve IT Services BV

Page 34: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

34

Unit Testing Process

design

code TEST

CoverageCheck !

testcode

Remember: most high level testing will only achieve 30 - 40% code coverage

Improve IT Services BV

Page 35: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Practice # 5: Exploratory Testing

� Apply Test Design

� Use Cases

� Exploratory Testing

Use Test Techniques as Tools

Testing is finding defects

Responding to change, Customercollaboration and No comphrehensive doc’s

35Improve IT Services BV

Page 36: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Improve IT Services BV 36

Exploratory Testing

� Very different approach- But also a different mindset andobjective

� An interactive test process

- Using the information gained whiletesting to design new and bettertests

� NOT error-guessing as there is a formal process defined !!

- tasks, objectives and deliverables

tests

PRODUCT

TEST

Page 37: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Exploratory Testing Process

Keep track ofbug vs. fix ratio!

Charter

Sessions- Exploration- Design- Execution

37

Session-Based Testing

Debriefing

Improve IT Services BV

Heuristics

- Boundaries- CRUD- configurations- Interruptions- ....

Page 38: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

38

Practice #6: People

� Training (on-the-job)

� Team Dynamics

� Not only for Testers!!

Build Experienced & Skilled Testers

Testing is an extremely complex, intellectually challenging task

Individuals and interactionsover processes and tools

Improve IT Services BV 38

Page 39: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Improve IT Services BV 39

People Skills

Test knowledge- Test principles

- Techniques

- Tools, etc.

IT knowledge- Scripting / Automation

- Requirements Eng.

- Configuration mgt.

Domainknowledge- Business process

- User characteristics

Soft skills- Communication

- Critical mindset

- Presentation & reporting

via (exchanging) Practical Experiences, Coaching

and (formal) Training

Best practices and presentations

- Values

Page 40: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Team Dynamics

Effective teams don’t just happen!

� Right experience and skills

� Right attitude & respect

� Common purpose

� Team management

� The right team mixture, e.g., Belbin profiles

Balanced teams are more effective

Nobody is perfect but a team can be ……..

40Improve IT Services BV

Page 41: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

Uhhhhh …. Test Automation

� The objective is to manage the Regression Risk

� Test Automation is NOT an objective

Things to consider

- Costs ….

- Skills ….

- Test Level ….

- Traceability ….

- Defects found ….

- Maturity ….Improve IT Services BV 41

Page 42: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

The Agile Testing Tool Box

�This will make you survive in most Agile projects

42Improve IT Services BV

Page 43: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

But don’t forget …..

Improve IT Services BV 43

It’s simple, but not easy!

Page 44: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

44

www.erikvanveenendaal.nl

[email protected]

@ErikvVeenendaal

Improve IT Services B.V.

Page 45: Workshop Agile Testing Table tennis player and coach Improve IT Services BV 4 www. erikvanveenendaal.nl. World’s leading organization for certification of professionals in software

45Improve IT Services B.V.