Model-Based Testing with SmartestingJean-Pierre Schoch

Sogeti’s Second Testing Academy

29 April 2009

Agenda

• Model-Based Testing and Smartesting

• A Technical Overview

• Benefits and Conclusion

Model-Based Testing: What and Why?

• The problems of software testingo Time is limitedo Applications can be complexo Requirements are changing

• Limitations of manual testingo Time-Consumingo Can rarely go deep enough

• Limitations of test automationo Test scripts may be challenging to designo Maintenance can be difficult and

costly

• Model-Based Testing (MBT)o “Model-based testing is software testing in which test cases are derived in

whole or in part from a model that describes some (usually functional) aspects of the system under test (SUT).” [Wikipedia]

Model System

Abstract tests

Executable tests

can be run against

are derived from

is a partial description of

are abstract versions of

Model-Based Testing with Smartesting

• Smartesting provides:

o A Model-Based Testing solution for functional testing

o … For the automatic generation of test cases

o … From a UML modeling of the requirements

• But who are we … ?

o Independent Software Vendor, founded in 2003, privately held

o Spin-off of a Computer Science Lab in France (CNRS / INRIA)

o 35 people, R&D Center in Besançon, offices in Paris, Amsterdam and Bangalore

o Several innovation awardso Supported by venture capital

The Smartesting Solution

• Modeling requirements to automatically generate test cases

• Functional testing

• Integrates with existing tools (requirement management, test management and execution, etc.)

• Eclipse-based

TestExecution

Test Management

SmartestingAutomated Test

Design

Business NeedsAnalysis

Agenda

• Model-Based Testing and Smartesting

• A Technical Overview

• Benefits and Conclusion

Smartesting Center™

Test Generation ProcessITERATIVEPROCESS

ManualTest

TestAutomation

7

TestAutomation

Engineer

Testers

Test Analyst

Requirement LinksRequirement Links

Test scenariosTest scenarios

Adaptation LayerAdaptation Layer(Keyword Library)(Keyword Library)

Requirement LinksRequirement Links

GeneratedGeneratedTestsTests

Business Needs

Test Designer™

Business Analyst

TestTestresultsresults

Test ModelTest ModelRepositoryRepository

Test Requirements

Third party environment

Test repository

Test Model and UML Diagrams

• Class diagramso To represent the application to test (system

under test)o To represent the business entities

associated with the application

• State Machine diagramso To model the dynamic behavior of the

application

• Object (or instance) diagrams o To model test data (in a generic way)

• And also …o Graphical constructs are not enougho Business rules are captured using OCL

(Object Constraint Lanquage)

Class and State Machine Diagrams: An Example

9

Class diagrams capture the definition of the “system under test” (not shown here) and the business entities of the application

UML constraints (and class structures) capture business rules

State machine diagrams capture the behavior of the system

This constraint is used to evaluate the result of the submit transition

(static)

(dynamic)

The Reference Application

Modeling with Smartesting

Generating Test Cases

Publishing Test Cases to Test Management Environment

HP Quality Center

Agenda

• Model-Based Testing and Smartesting

• A Technical Overview

• Benefits and Conclusion

Requirement Validation and Early Test Generation• Test Model = Model of the requirements

o Detailed levelo Independent of codeo Can (should?) exist before code is writteno Test cases derived from the model

“Unreachable” test cases may be an indication of requirement incompatibilities, errors, etc.

Changes to the state of the system can be observed step by stepTool provides a simulation mode at

the model level

• Benefitso Requirements validated before

code is writteno Facilitates communication with

business expertso Can be used as an input to

development

Requirement Traceability and Test Coverage• Test Cases published to the Test Management tool with

requirement linkso Test model marked with requirement ID tagso Synchronization between generated test cases and requirements entirely

automaticNo manual, error-prone intervention

o Publication will update the test status for modified test caseso Leverages the analysis and reporting capabilities in the test management

tool

• Benefitso Reliable traceability links and test

coverage informationo Guarantee that the generated test

cases are relevant

Improved Productivity and Maintenance

• “Normal” benefits from automationo Significant reduction on execution time

In particular during regression testingo Effectiveness of automation vs. manual approach

• Separation of concernso Business logic (captured in test model) separated from implementation

technology (testing environment and “adaptation layer”)

• Impact of functional changeso Changes to the modeled requirements automatically applied to test cases

The number of affected test cases has no consequence on schedule

• Impact of implementation changeso No effect on the test model

For instance, the look and feel can evolve separatelySignificantly facilitates application migration

Smartesting and SOGETI

• We’re here at the Software Testing Academy

• SOGETI NLo Proof of Concept (PoC) the Solution and Innovation Department

Now looking for a “first” projectGoal: To implement Smartesting as a standard tool for the STAAS test line

o Smartesting initiative to document integration of Smartesting with TMap

• SOGETI Franceo Contribution of François Darphin, Testing National Practice Manager SOGETI

France, to the book “Industrialiser le test fonctionnel”, by Bruno Legeard, Smartesting CTO

o Business development in progress (identification of first project)o Participation of SOGETI France to a Smartesting customer event coming

soon…

• And also with Capgeminio PoC realized for Capgemini France in Bordeauxo Business development in progress (identification of first project)

“Practical Model-based Testing” – December 2006By Bruno Legeard, CTO Smartesting

Smartesting Case Study – November 2008

“Industrialiser le test fonctionnel” – February 2009By Bruno Legeard, CTO Smartesting

Soon in English…

Sogeti’s 2nd Testing Academy29 April 2009www.sogeti.com