test automation principles

Test Automation and xUnitTest Automation and xUnit

Author:Carlo Ibarra

Author:Carlo Ibarra

ScopeScope

Does not cover TDD.Test automation, principles and best practices.

xUnit basics.

Does not cover TDD.Test automation, principles and best practices.

xUnit basics.

TerminologyTerminology

SUT – system under test.

DOC – depended on component.

Fixture – preconditions to a test; everything a test needs to set up before assertion.

SUT – system under test.

DOC – depended on component.

Fixture – preconditions to a test; everything a test needs to set up before assertion.

Test AutomationTest Automation

“What I need is a list of specific unknown problems we will

encounter.”

- a Dilbert Quotes contest entry

Goals of Test AutomationGoals of Test Automation Tests should help us improve quality. Tests should help us improve quality.

Tests should help us understand the SUT. Tests should help us understand the SUT.

Tests should reduce (and not introduce) risks. Tests should reduce (and not introduce) risks.

Tests should be easy to run. Tests should be easy to run.

Tests should be easy to write. Tests should be easy to write.

Tests should require minimal maintenance as the system evolves around them. Tests should require minimal maintenance as the system evolves around them.

Four-Phase TestFour-Phase Test

1. Fixture Setup- test data, mocks, expected results constants

1. Fixture Setup- test data, mocks, expected results constants

2. Exercise SUT- Interact with the SUT to exercise desired behavior

2. Exercise SUT- Interact with the SUT to exercise desired behavior

3. Result Verification- Determine if expected outcome is obtained

3. Result Verification- Determine if expected outcome is obtained

4. Teardown- Restore state back to the state before testing

4. Teardown- Restore state back to the state before testing

The “Fragile Test” ProblemThe “Fragile Test” Problem

Behavior sensitivity- Change in requirements, functionality

Interface sensitivity- Change in UI

Data sensitivity- Change in source structures and state

Context sensitivity - Change in workflow or external DOCs

Behavior sensitivity- Change in requirements, functionality

Interface sensitivity- Change in UI

Data sensitivity- Change in source structures and state

Context sensitivity - Change in workflow or external DOCs

Principles of Test Automation

“Quality is free, but only to those

who are willing to pay heavily for it.”

- Lister, Demarco: “Peopleware”

Principle:

Write the Tests First.

Principle:

Use the Front Door First.

Principle:

Communicate Intent.

Principle:

Don’t Modify the SUT.

Principle:

Keep the Tests Independent.

Principle:

Isolate the SUT.

Principle:

Minimize Test Overlap.

Principle:

Minimize Untestable Code.

Principle:

Keep Test Logic Out of Production.

Principle:

Verify One Condition Per Test.

Principle:

Test Concerns Separately.

Principle:

Ensure Commensurate Effort

and Responsibility.

xUnit BasicsxUnit Basics

“In order to make an apple pie from scratch,you must first create the universe.”

- Carl Sagan

“In order to make an apple pie from scratch,you must first create the universe.”

- Carl Sagan

Introducing xUnitIntroducing xUnit Make it easy for developers to write test without

learning a new language.

Make it easy to test individual classes and objects without needing to have the rest of the application available.

Make it easy to run test or many test in a single action.

Minimize the cost of running tests so programmers aren’t discourage from running existing tests.

Make it easy for developers to write test without learning a new language.

Make it easy to test individual classes and objects without needing to have the rest of the application available.

Make it easy to run test or many test in a single action.

Minimize the cost of running tests so programmers aren’t discourage from running existing tests.

Common FeaturesCommon Features

Specify a test as a Test Method.

Specify the expected results within the test method in the form of calls to assertion methods.

Aggregate the tests into test suites that can be run as a single operation.

Run one or more tests to get a report on the results of a test run.

Specify a test as a Test Method.

Specify the expected results within the test method in the form of calls to assertion methods.

Aggregate the tests into test suites that can be run as a single operation.

Run one or more tests to get a report on the results of a test run.

Q&AQ&A