odd testing

Obstacle Driven DevelopmentODD Testing

Testing in History 1

Testing ideas is implicit to science and technology.

• Testing implemented on products for many years

• Ideas are assumptions without sufficient testing

• Todays technology is a result of centuries of tests

21/06/2016 ©odd.enterprises 2

Testing in History 2

Testing implemented on certain products for many years.

• Tests must replicate real world conditions

• Armour designed to be bullet proof is tested

• Non standard components required this approach


Cost of Failure

Cost of failure is often greater than associated costs related to a successful development.

• Undetected errors may become very costly

• Increased use of specification can reduce costs overall

• Cost increases exponentially for each stage a bug is undetected


Fail Early, Fail Often

Achieving success with ODD is through identifying, correcting and preventing failure.

• Undiscovered errors cost 10x more to fix by next stage

• Errors become expensive to solve

• 2 stages missed ≈ 100x

• 3 stages missed ≈ 1000x


Verification and Validation 1

Verification and validation occurs between stages with appropriate adaptions.

Verification and validation concern the questions:

• Verification

Is it built in the right way?

• Validation

Is it built right?


Verification and Validation 2

Verification and validation are adapted for each stage.

• Specification– Verification and

validation (of behaviours)

• Solution– Testing and design

• Production– Quality assurance and

control

• Analysis– Utilisation and

elicitation


ODD Elements

ODD Elements are single generic parts of a development.

• Divided into stage, system and abstraction level

• Higher level elements will consist of combined lower levels

• Each stage contains different and distinct elements


Analysis Elements

Analysis links Production and Specification stages with tests solved and created by each level.

• Analysis links Production by elicitation of customers

• Analysis links Specification by verification of behaviours


Specification Elements

Specification links Analysis and Solution stages with tests solved and created by each level.

• Specification links Analysis through solving of tests

• Specification links Solution through creation of tests


Solution Elements

Solution links Specification and Production stages with tests solved and created by each level.

• Solution links Specification by design according to tests

• Solution links Production by quality assurance tests according to solution


Production Elements

Production links Solution and Analysis stages with tests solved and created by each level.

• Production links Solution through quality control according to tests

• Production links Analysis through utilisation of products features


ODD Flow Chart

Flow chart to demonstrate a generic ODD process.

Problems or obstacles to be overcome are divided into 4 stages with appropriate testing.

• Analysis

• Specification

• Solution

• Production


Linking Tests 1

Tests link behaviours with solutions through testing and design.

• Solutions designed according to tests from behaviours

• Each solution is a single element of a product

• Unit testing is applied• Test suite created and ran when

changes occur


Linking Tests 2

Testing and design concerns solutions created from behaviours of a specification.

• Each solution implements 1 or more behaviours

• Tests suite ran for any changes or additions

• Created as with Test Driven Development


ODD Test Suites

Test suites implemented to create a solution for software and identify errors.

• Test suites contain individual and combined unit tests

• Test suites are intended to be implemented between all stages

• TDD process extended throughout development to create ODD


Linking Behaviours to Situations 1

Decision tree adapts to any situation and linked to a specification by creating tests.

• Any situation can be described by a decision tree

• Diagram appropriate for failure mode effects and analysis

• Linked to a behaviour which covers the situation


Linking Behaviours to Situations 2

Each branch of a decision tree results in a situation to be covered by one or more behaviours.

• Each situation is tested to ensure coverage by specification

• Ensures situations are covered before creation of solution

• All expected situations should have an associated behaviour


Linking Solution to Production 1

Solution must be produced with consistent quality and often for very large quantities.

• Solution assures and controls quality of related production

• Tests are created for quality assurance

• Tests passed give measure for quality control


Linking Solution to Production 2

Each element of a solution creates a quality assurance test which a production element must pass.

• Quality assurance and control based on probability

• Assurance will determine an acceptable failure rate

• Control ensures failure rate is acceptable


Linking Production to Analysis 1

Linking production to analysis ensures product features are utilised and elicited.

• Features of product utilised in practical situations by customers

• Elicitation after utilisation of feature to find requirements

• Verifies current solutions and identifies new obstacles


Linking Production to Analysis 2

Linking production to analysis has product features used as this is how a customer sees a product.

• New feature may cover requirement but create another

• Each product element and level investigated separately


ODD is 3D


ODD without Tests


ODD with Tests


ODD with Passed Tests


ODD Generic Flow Chart

Each stage of ODD is an adaption of this generic flow chart.

Flow chart is adapted to provide:

• Analysis - Utilisation and Elicitation

• Specification – Verification and Validation

• Solution - Testing and Design

• Production – Quality Assurance and Control


Creating Tests 1

Creation of solutions inspired by Behaviour Driven Development.

• Tests created by rewriting a behaviour and designing test

• Design of solution according to tests reduces debugging

• Testing and design may continue until all behaviours implemented


Creating Tests 2

Full test suite created using each behaviour in specification.

• Creating a test first ensures an objective is understood

• Design according to passing tests reduces ambiguity

• Passing a test ensures behaviour is implemented


Flow Chart for Solution

1. Select behaviour which a solution has to perform.

2. Test created to ensure solution performs behaviour.

3. Solution designed to pass test.

4. Repeat until solution passes test.

5. Repeat until behaviours tested and solutions designed.

6. Production stage begins.


Flowchart for Production

1. Select solution which production is to create.

2. Quality assurance test ensures production creates solution.

3. Production begins.

4. Repeat until quality is controlled.

5. Repeat until all production is quality controlled.

6. Analysis stage begins.


Flowchart for Analysis

1. Select feature to be elicited for analysis.

2. Utilisation test is created for feature.

3. Product is utilised by customers.

4. Repeat elicitation until sufficient feedback obtained.

5. Repeat until all features in production are tested.

6. Specification stage begins.


Flowchart for Specification

1. Requirement selected to be covered by behaviour.

2. Verification test is created.

3. Behaviour is specified according to the test.

4. Repeat until behaviour is validated.

5. Repeat until all requirements are covered by behaviours.

6. Solution stage begins.


ODD Materials

ODD is explained in further presentations.

• Obstacle Driven

Development

• ODD: Requirements Analysis

• ODD: Extending a

Specification

• ODD: Extending TDD

• ODD: Extending V-models

• ODD Is Not Agile or Waterfall

ODD Is Not Agile or

Waterfall

Obstacle Driven Development

ODD: Requirements

Analysis

ODD: Extending a Specification

ODD: Extending V-models

ODD: Extending TDD


http://www.odd.enterprises/obstacle-driven-development.html

http://www.odd.enterprises/odd-requirements-analysis.html

http://www.odd.enterprises/odd-specification.html

http://www.odd.enterprises/odd-extending-tdd.html

http://www.odd.enterprises/odd-extending-v-models.html

http://www.odd.enterprises/odd-is-not-agile.html

Further Information and Questions

• Website

• Presentations

• Facebook

• Twitter

• Email


http://odd.enterprises/

http://www.odd.enterprises/odd-presentations.html

https://www.facebook.com/odd.enterprises

https://twitter.com/odd_enterprises

mailto:[email protected]

Legal Stuff

ReferencesTest Driven Development for Embedded C

James Grenning, 2011

Test Driven Development

http://en.wikipedia.org/wiki/Test-driven development

Behaviour Driven Development

http://en.wikipedia.org/wiki/Behavior-driven development

Unit Testing

http://en.wikipedia.org/wiki/Unit testing

Contact us for more information on sources and references.

DisclaimerThe ODD M-model and associated processes are provided by odd.enterprises and may be used for any purpose whatsoever.

The names odd.enterprises and associated logos should not be used in any representation, advertising, publicity or other manner whatsoever to endorse or promote any entity that adopts or uses the model and/or associated processes.

odd.enterprises does not guarantee to provide support, consulting, training or assistance of any kind with regards to the use of the model and/or processes including any updates.

You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees against any claim or demand including reasonable solicitors fees, related to your use, reliance or adoption of the model and/or processes for any purpose whatsoever.

The model is provided by odd.enterprises “as is” and any express or implied warranties, included but not limited to the implied warranties of merchantability and fitness for a particular purpose are expressly disclaimed.

In no event shall odd.enterprises be liable for any damages whatsoever, including but not limited to claims associated with the loss of data or profits, which may result from any action in contract, negligence or other tortious claim that arises out of or in connection with the use or performance of the model.


http://en.wikipedia.org/wiki/Test-driven_development

http://en.wikipedia.org/wiki/Behavior-driven_development

http://en.wikipedia.org/wiki/Unit_testing

odd testing

Engineering