international telecommunication union itu-t study group 17, geneva, 5-14 october 2005 an...

International Telecommunication Union

ITU-T Study Group 17, Geneva, 5-14 October 2005

An IntroductionAn Introductiontoto version version

33

Dr. Michael EbnerUniversity Göttingen, Germany

[email protected]

212 October 2005

ITU-T


About the speaker

o Is researcher in the telematics group ofProf. Dr. D. Hogrefe at the Institute for Informatics, University of Göttingen, Germany

o Is active in the research area of test automation, model-based testing, and protocol modeling

o Working with TTCN-3 since 2000• Influenced data types and synchronous

communication of TTCN-3

312 October 2005

ITU-T


An Introduction toInstitute for Informatics,University of Göttingen

George-August-Universität Göttingen

Department of Mathematics

Institute for Informatics

Gauß, Weber,Hilbert, …

Center for InformaticsOther 12 departments

Founded in 1737 by King George II (England), who also founded Princeton U (1746) and Columbia U (1754)

01/2003~

Telematics GroupProf. Hogrefe

Distributed Systems Group, Prof. Switzer

Theory Group Prof. Waack

Softw. Eng. Group Prof. Grabowski

Database GroupProf. May

2000~

Practical Informatics Group Prof. Neumair

(GWDG)

Service Eng. GroupProf. Margaria

44 Nobel Prize-winner live(d) and work(ed) here

412 October 2005

ITU-T


Table of contents

1. Overall view of TTCN-32. TTCN-3 en detail

a. Types and valuesb. Behavior specification

3. Implementation of TTCN-34. Presentation formats5. Final remarks

512 October 2005

ITU-T


Why using a standardised test language? 1(3)

Developer

Heterogeneityincreases

Testingthroughout the

processIntegrator

SystemsIntegrator

612 October 2005

ITU-T



Testing tight to Services

TTCN-3

Testing tightto Development

e.g. JUnit

Developer

Integrator

SystemsIntegrator

712 October 2005

ITU-T



o Standardised test language• improves communication

• among developers• with the customer

• improves transparency• one test language for all departments• No proprietary test languages

• reduces costs• costs of training• usage of commercial test equipments

(tools, test suites, etc.)

812 October 2005

ITU-T


What is TTCN-3 ?

o The standardised (black-box) test specification and test implementation language.

o Developed• by the European Telecommunications Standards

Institute (ETSI) from 1999 to 2001.• based on the experiences from previous TTCN

versions.o Applicable for all kinds of black-box testing for

reactive and distributed systems, e.g.• Telecom systems (ISDN, ATM);• Mobile (telecom) systems (GSM, UMTS);• Internet (has been applied to IPv6);• CORBA based systems.

912 October 2005

ITU-T


TTCN stands for …???

o Tree and Tabular Combined Notation• For TTCN-1 and TTCN-2

o Testing and Test Control Notation• Since TTCN-3

1012 October 2005

ITU-T


The TTCN-3 series of standards1(3)

TTCN-3 User

Graphical Format

Tabular Format

ASN.1 Types & Values

Other Types & Values n

Other Types & Values 2

TTCN-3CoreNotation

Presentation Format n


1112 October 2005

ITU-T



TTCN-3 User

Graphical Format

Tabular Format




TTCN-3CoreNotation



msc mi_synch1_conc1

mtc ISAP1 MSAP2

UML Testing Profile

IDL

XML

C/C++

:testcase myTestcase () runs on MTCType system TSIType{ mydefault := activate (OtherwiseFail);

verdict.set(pass);

:connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);:map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);:PTC_ISAP1.start(func_PTC_ISAP1());PTC_MSAP2.start(func_PTC_MSAP2());Synchronization(); all component.done;log(”Correct Termination”);

}

:

1212 October 2005

ITU-T



o European Standard (ES) in 6 parts

• ES 201 873-1: TTCN-3 Core Language• ES 201 873-2: TTCN-3 Tabular Presentation

Format (TFT)• ES 201 873-3: TTCN-3 Graphical Presentation

Format (GFT)

• ES 201 873-4: TTCN-3 Operational Semantics• ES 201 873-5: TTCN-3 Runtime Interface (TRI)• ES 201 873-6: TTCN-3 Control Interface (TCI)

• ES 201 873-7: Using ASN.1 with TTCN-3• ES 201 873-8: Using IDL with TTCN-3• ES 201 873-9: Using XML with TTCN-3• ES 201 873-10: Using C/C++ with TTCN-3

(planned)

Z.140Z.141

Z.142

Z.143Z.144Z.145

Z.146

ITU-T Recommendation (SG17 / WP3 / Q14)

New edition 3.1.1 of TTCN-3 since June 2005

1312 October 2005

ITU-T


Concepts

o Black-Box Testing with TTCN-3o Test Configurationo Test Componentso Communication Portso Test Verdictso Ingredients of TTCN-3o Structure of TTCN-3 specso Re-use of existing Code

1412 October 2005

ITU-T


Black-box testing with TTCN-3

TTCN-3 Test Case

Port.send(Stimulus) Port.receive(Response)

System Under Test

Port

• Assignmentof aTest Verdict

1512 October 2005

ITU-T


TTCN-3 Test Case

Test configuration 1(2)

create

create

MTC

start

start

create

TCs

TC

TC

start

SUT

1612 October 2005

ITU-T


Test configuration 2(2)

SUT

Real Test System Interface

Test System

Abstract Test System Interface

TC2TC1

IN OUT

OUT IN

Connected Ports

INOUTMapped Ports

INOUT

Real Test System Interface

1712 October 2005

ITU-T


Test components

o There are three ‘kinds’ of components• Abstract Test System Interface defined as component• MTC (Main Test Component)• PTC (Parallel Test Component)

Test System

Real test system connected to the SUT

Abstract Test System Interface

MTC PTCnPTC1 PTC2

1812 October 2005

ITU-T


Communication ports 1(2)

o Test components communicate via portso A test port is modeled as an infinite FIFO queueo Ports have direction (in, out, inout)o There are three types of port

• message-based, procedure-based or mixed

TC1 TC2

P1.send (Msg) P2.receive (Msg)

P2 (in)P1 (out)

1912 October 2005

ITU-T


Communication ports 2(2)

call getcall

reply orraise exception

Caller Callee

getreply orcatch exception

blocking blocking

LanguageLanguage

2012 October 2005

ITU-T


PIF

Test verdicts

o Test verdicts: none < pass < inconc < fail < error

o Each test component has its own local verdict, which can be set (setverdict) and read (getverdict).

o A test case returns a global verdict

MTC PTC1 PTCN

Verdict returned by the test case when it terminates

I

setverdict(inconc)setverdict(fail)

F P

setverdict(pass)

2112 October 2005

ITU-T


Ingredients of TTCN-3

o Built-in and user-defined generic data types (e.g., to define messages, service primitives, information elements, PDUs).

o Actual test data transmitted/received during testing.

o Definition of the components and communication ports that are used to build various testing configurations.

o Specification of the dynamic test system behavior.

Test Behavior

Test Configuration

TTCN-3

Test Data

Data Types

2212 October 2005

ITU-T


Structure of TTCN-3 specs – TTCN-3 Modules

o Modules are the building blocks of all TTCN-3 test specifications.

o A test suite is a module.

o A module has a definitions part and an (optional) control part.

o Modules can be parameterized.

o Modules can import definitions from other modules.

Module

Module Control

Module Definitions

2312 October 2005

ITU-T


Structure of TTCN-3 specs –Module definitions part

o Definitions are global to the entire module.

o Data Type definitions are based on TTCN-3 predefined and structured types.

o Templates define the test data.

o Ports and Components are used in Test Configurations.

o Functions, Altsteps and Test Cases define behavior.

Test Cases

Altsteps

Functions

Test Components

Communication Ports

Signature Templates

Data Templates

Signatures

Constants

Data Types

2412 October 2005

ITU-T


Structure of TTCN-3 specs – Module control part 1(2)

Module

Module Control

Module Definitions

o Module control is the ‘dynamic’ part of a TTCN-3 specification where the test cases are executed.

o Local declarations, such as variables and timers may be made in the control part.

o Basic programming statements may be used to select and control the execution of the test cases.

2512 October 2005

ITU-T


Structure of TTCN-3 specs – Module control part 2(2)

module … {… control{ var integer count;

if(execute(SIP_UA_REC_V_001()) == pass) {

// Execute test case 10 times count := 0; while( count <= 10) {

execute(SIP_UA_REC_V_002());count := count + 1;

} // end while

} // end if

} // end control} // end module

2612 October 2005

ITU-T


Reuse of existing code –The import mechanism 1(2)

o Main Module contains the control part, which specifies test suite execution.

o Modules may reuse definitions from other (library) modules.

o Implicit import of definitions via chains of imports is not allowed, i.e., an explicit import has to be added.• Reason: A module

shall know all modules which it depends on.

MainModule

LibraryModule 1

import

LibraryModule 2

import

LibraryModule 3

import

implicit importof a definition required

explicitimport

2712 October 2005

ITU-T


Reuse of existing code –The import mechanism 2(2)

o Import allows to import• single definitions• definitions of a certain

kind, and• groups of definitions

from other modules.

o Definitions may be imported recursively.

o If several definitions are addressed, certain definitions can be excluded by using an except directive.

import form ModuleOne { modulepar ModPar2; type RecordType_T2}

import from ModuleTworecursive { testcase T_case}

import from ModuleThreeall except { template all}

import form ModuleOne { modulepar ModPar2; type RecordType_T2}

import from ModuleTworecursive { testcase T_case}

import from ModuleThreeall except { template all}

2812 October 2005

ITU-T


Use of non–TTCN-3 code – Language mapping

o Importing non TTCN-3 definitions requires a language mapping onto TTCN-3.

o The language mapping defines the meaning of non TTCN-3 definitions in TTCN-3 modules.

o The language mapping may provide TTCN-3 additional features for imported definitions (e.g., operations for ASN.1 data types).

TTCN-3Module

importASN.1 Spec.

XML Spec.

:

XYZ Spec.

:

TTCN-3

ASN.1

:

XML

XYZ

:

languagemapping

2912 October 2005

ITU-T


Table of contents




3012 October 2005

ITU-T


LanguageLanguage Data types and test data –Constants, types, templates 1(2)

o TTCN-3 data types:• integer, char, universal char,

float, boolean, objid• enumerated• bitstring, hexstring,

octetstring, charstring, universal charstring,

• record, record of, set, set of, union

• anytype• verdicttype• address

o Useful types• IEEE754float, long, byte, …• Iso8859string, utf16string, …Test Cases

Altsteps

Functions

Test Components

Communication Ports

Signature Templates

Data Templates

Signatures

Constants

Data Types

3112 October 2005

ITU-T


LanguageLanguage Data types and test data –Constants, types, templates 2(2)

const integer int_Const := 7; // Normal constantexternal const boolean bool_Const; // External constant

type record Request { // Structured type definition RequestLine requestLine, ReqMessageHeader reqMessageHeader optional, charstring crlf, charstring messageBody optional}

template Request Invite := { // template for the Request type requestLine := Request_Line("INVITE"), reqMessageHeader := Req_Mes_Header("INVITE"), crlf := ?, messageBody := omit}

3212 October 2005

ITU-T


LanguageLanguage Encoding of Messages –Attributes 1(2)

o Attributes can be assigned to all kinds of definitions, groups and modules.

o Kinds of attributes• Encoding information

• encode – attribute• variant – attribute• Values are standardized for ASN.1

encoding• Presentation information

• display – attribute• Values are standardized for the

graphical and tabular presentation format.

• User-defined• extension - attribute

Attributes

Language Element

3312 October 2005

ITU-T


Encoding of Messages –Attributes 2(2)

group MyPDUs {

type record MyPDU1 { … }

group MySpecialPDUs {


with {extension "MyVerySpecialRule"}


}

with {extension "MySpecialRule"}

}

with {

display "PDU";

extension "MyRule"

}

LanguageLanguage

3412 October 2005

ITU-T


Table of contents




3512 October 2005

ITU-T


LanguageLanguage Specifying test behavior –Alternative behavior 1(2)

o … has to be specified whenever test component is ready to take a response from the SUT or a timeout.

o … is typically defined by several alternatives, which

• are evaluated according to their appearance

• may be guarded

• can be part of an altstep, which may be called explicitly or activated as default.

o … forks the test behavior (the typical „tree“), but in TTCN-3 alternatives can be joined again after the end of an alternative behavior.

o All other cases can be handled in an else branch.

3612 October 2005

ITU-T


LanguageLanguage Specifying test behavior –Alternative behavior 2(2)

alt {

[] L1.receive(MyMessage1) {

: // do something

}

[x>1] L2.receive(MyMessage2) {}

[x<=1] L2.receive(MyMessage3) {}

[] MyAltstep; // call of a altstep

[else] stop // else branch

}

3712 October 2005

ITU-T


o Shorthand for all alternative permutations

• All messages must be received, but in any order

interleave {[] PCO1.receive(MyMessageOne) {}[] PCO1.receive(MyMessageTwo) {}[] PCO1.receive(MyMessageThree) {}

}

Specifying test behavior –Interleaved behavior

LanguageLanguage

3812 October 2005

ITU-T


Table of contents




3912 October 2005

ITU-T


Overview –The TTCN-3 test system

o TE – TTCN-3 Executableo SA – System Adaptero PA – Platform Adaptero CD – Coder/Decodero TM – Test Managemento CH – Component

Handlingo SUT – System Under Test

TE

Test System User

SUT

TCI

TRI

CH

CD

SA PA

TM

ETSI ES 201 873-1: TTCN-3 Core LanguageETSI ES 201 873-5: TTCN-3 Runtime Interface (TRI)ETSI ES 201 873-6: TTCN-3 Control Interfaces (TCI)

4012 October 2005

ITU-T


Overview –Implementing TTCN-3

Abstract Test Suitecompile

TE

TTCN-3Executable

TTCN-3 Runtime System

+

SUT

Executable Test Suite

build

4112 October 2005

ITU-T


Table of contents




4212 October 2005

ITU-T


TTCN-3 presentation formats – Overall view

TTCN-3 User

Graphical Format

Tabular Format




TTCN-3CoreNotation



UML Testing Profile

IDL

XML

C/C++

4312 October 2005

ITU-T


The graphical presentation format of TTCN-3

testcase MyTestCase (in boolean internetService, inout integer nrPass) runs on MtcType system TestSystemtype

MtcType

mtcmPCOtype

P1mCPtype

CP

map(self:P1,System:mPCO)

if (internetservice)

var default def := activate (MyDefault())

newInternetPTC()

var reportType report

Page 1(2)

4412 October 2005

ITU-T


sd invalidPIN

storeCardData(current)

«sut»atm

hwe

display(”Enter PIN”)

isPinCorrect(invalidPIN)

isPinCorrect : false

current

{readOnly} Integer invalidPIN; { current.isPinCorrect(invalidPIN) == false }

isPinCorrect(invalidPIN)

display(”Invalid PIN”)

display(”Enter PIN again”)

isPinCorrect : false

t1(2.0)

t1

{0 .. 3}

«validationAction»pass

U2TP – A test caseData partition

Setting arbitrated

verdict

Stimulus

Observation

Setting a timer

Stopping a timer

Duration constraint

Function return value

4512 October 2005

ITU-T


Table of contents




4612 October 2005

ITU-T


Managing TTCN-3 projects –Potential usage 1(2)

o Applicability for different test phases• Host/target tests• System integration and

system test• User acceptance test

o Features required for a test automation solution• Test design (low-level,

high-level, graphical)• Scheduling of test runs• Centralized/distributed

test runs• Log/trace analysis• Test case management• Support of standards

Legend for the table on next slide … feature must be

supported in this test phase

• ()… feature should be supported

•fully colored field… feature supported by TTCN-3

•hatched colored field… feature partly supported by TTCN-3

4712 October 2005

ITU-T


Managing TTCN-3 projects –Potential usage 2(2)

RequirementHost Test

Target Test

System

Integr.

System Test

User Accept

.

Low-level test design (programming level)

() ()

High-level test design (textual)

Graphical test design and test result analysis

Scheduling of test runs / test resource management

()

Centralized test execution ()

Distributed test execution

Log / trace file analysis

Test case management (including version control)

Support of ASN.1 and standardized test suites

()

4812 October 2005

ITU-T


TTCN-3 extensionsWork items in standardization

o Language extensions mechanisms• Packages and profiles

o Extended communication mechanisms• Broadcast / multicast

(edition 3.0.0)• Synchronization /

coordination (edition 3.0.0)

o Real-time extensions• Absolute time support • Time-constrained

operations

o Better performance testing support• Implicit test

configuration• Implicit communication• Performance

measurement

4912 October 2005

ITU-T


Current TTCN-3 tool providers

o Tool providers• Danet• DaVinci

Communication• Open TTCN• Telelogic• Testing Technologies• Strategic Test Solutions

o Internal tools used by• Nokia• Ericsson• Motorola

o Existing test devices with TTCN-3 support(for telecommunication applications)• Alcatel A1100• Navtel InterWatch• Nethawk• Tektronix G20

5012 October 2005

ITU-T


Conclusions

o TTCN-3 finds its way into practiceo Lots of interest from industry and

academiao Mature TTCN-3 standardo Stimulates further research &

development• New application domains, e.g. automotive• Real-time and performance testing• Test patterns

o Still possibilities to shape the future TTCN-3

5112 October 2005

ITU-T


Acknowledgement

o Slides are based on a TTCN-3 Tutorial from Grabowski and Ulrich

o Prof. Dr. Jens Grabowski• Institute for Informatics, University of

Göttingen• Is in the TTCN-3 business since the

beginningo Dr. Andreas Ulrich

• Is a Principal Engineer at Siemens’ CorporateTechnology Division in München, Germany

• Member of the ETSI TTCN-3 maintenance group



An IntroductionAn Introductiontoto version 3version 3

Dr. Michael EbnerUniversity Göttingen, Germany

[email protected]



Literature on Literature on TTCN-3TTCN-3Standard documents

Overview articlesGraphical presentation format

TTCN-3 control and runtime interfaceIDL to TTCN-3 mapping

TTCN-3 real-time extensionsUML Testing Profile

5412 October 2005

ITU-T


Literature on TTCN-3 1(6)

o Standard documents: (can be found on the TTCN-3 homepage: http://www.ttcn3.org/ or http://www.etsi.org/ptcc/ptccttcn3.htm)• ES 201 873-1: TTCN-3 Core Language• ES 201 873-2: TTCN-3 Tabular Presentation Format (TFT)• ES 201 873-3: TTCN-3 Graphical Presentation Format (GFT)• ES 201 873-4: TTCN-3 Operational Semantics• ES 201 873-5: TTCN-3 Runtime Interface (TRI)• ES 201 873-6: TTCN-3 Control Interface (TCI)• ES 201 873-7: Using ASN.1 with TTCN-3• ES 201 873-8: Using IDL with TTCN-3• ES 201 873-9: Using XML with TTCN-3• ES 201 873-10: Using C/C++ with TTCN-3 (planned)

o Example test suites:• See: http://www.etsi.org/ptcc/ptccttcn3.htm • E.g., http://www.etsi.org/ptcc/ptccsip_osp.htm

5512 October 2005

ITU-T



o Overview articles• Jens Grabowski, Dieter Hogrefe, György Réthy, Ina

Schieferdecker, Anthony Wiles, Colin Willcock. An introduction into the testing and test control notation (TTCN-3). Computer Networks, Volume 42, Issue 3, Elsevier, Amsterdam, Juni 2003, 375-403.

• Jens Grabowski, Anthony Wiles, Colin Willcock, Dieter Hogrefe. On the Design of the new Testing Language TTCN-3. '13th IFIP International Workshop on Testing Communicating Systems' (Testcom 2000), Ottawa, 29.8.2000-1.9.2000, Kluwer Academic Publishers, August 2000.

o Graphical presentation format• P. Baker, E. Rudolph, I. Schieferdecker. Graphical Test

Specification - The Graphical Format of TTCN-3. Proc. of the 10th SDL Forum 2001, Copenhagen, June 2001.

• E. Rudolph, I. Schieferdecker, J. Grabowski: HyperMSC - a Graphical Representation of TTCN. Proc. of the 2nd Workshop of the SDL Forum, Society on SDL and MSC, Grenoble, June 2000.

5612 October 2005

ITU-T



o TTCN-3 control and runtime interface• S. Schulz, T. Vassiliou-Gioles. Implementation of TTCN-3 Test

Systems using the TRI. Testing Internet Technologies and Services - The IFIP 14th International Conference on Testing of Communicating Systems March, 19th - 22nd, 2002, Berlin, Kluwer Academic Publishers, March 2002.

• I. Schieferdecker, T. Vassiliou-Gioles. Realizing distributed TTCN-3 test systems with TCI. IFIP 15th Intern. Conf. on Testing Communicating Systems - TestCom 2003, Cannes, France, May 2003.

o IDL to TTCN-3 mapping• M. Ebner, A. Yin, M. Li. A Definition and Utilisation of OMG IDL to

TTCN-3 Mappings. TestCom 2002: Testing Internet Technologies and Services - The IFIP 14th International Conference on Testing of Communicating Systems March, 19th - 22nd, 2002, Berlin, Kluwer Academic Publishers, March 2002.

o XML to TTCN-3 mapping• I. Schieferdecker, B. Stepien. Automated Testing of XML/SOAP based

Web Services. Proceedings of the 13. Fachkonferenz der Gesellschaft für Informatik (GI) Fachgruppe "Kommunikation in verteilten Systemen" (KiVS), Leipzig (Germnay), Febr. 26th-28th, 2003.

5712 October 2005

ITU-T



o TTCN-3 real-time extensions• Zhen Ru Dai, Jens Grabowski, Helmut Neukirchen. Timed

TTCN-3 - A Real-Time Extension for TTCN-3. TestCom 2002: Testing Internet Technologies and Services -- The IFIP 14th International Conference on Testing of Communicating Systems March, 19th - 22nd, 2002, Berlin, Kluwer Academic Publishers, March 2002.

• Zhen Ru Dai, Jens Grabowski, Helmut Neukirchen. TimedTTCN-3 Based Graphical Real-Time Test Specification. Testing of Communicating Systems (Editors: D. Hogrefe, A. Wiles). Proceedings of the 15th IFIP International Conference on Testing of Communicating Systems, Sophia Antipolis, France, May 2003. LNCS 2644, Springer, May 2003.

• Helmut Neukirchen, Zhen Ru Dai, Jens Grabowski. Communication Patterns for Expressing Real-Time Requirements Using MSC and their Application to Testing. Testing of Communicating Systems. Proceedings of the 16th IFIP International Conference on Testing of Communicating Systems, Oxford, UK, March 2004. LNCS 2978, Springer, March 2004.

5812 October 2005

ITU-T



o UML Testing Profile• Online Resources: http://www.fokus.gmd.de/u2tp/• U2TP consortium. UML 2.0 Testing Profile Specification. OMG

Adopted Specification ptc/03-08-03. (for download see: http://www.fokus.gmd.de/u2tp/)

• Zhen Ru Dai, Jens Grabowski, Helmut Neukirchen, Holger Pals.From Design to Test with UML - Applied to a Roaming Algorithm for Bluetooth Devices. Testing of Communicating Systems. Proceedings of the 16th IFIP International Conference on Testing of Communicating Systems (TestCom2004), Oxford, United Kingdom, March 2004. LNCS 2978, Springer, March 2004.

• Ina Schieferdecker, Zhen Ru Dai, Jens Grabowski, Axel Rennoch.The UML 2.0 Testing Profile and its Relation to TTCN-3. Testing of Communicating Systems (Editors: D. Hogrefe, A. Wiles). Proceedings of the 15th IFIP International Conference on Testing of Communicating Systems (TestCom2003), Sophia Antipolis, France, May 2003. LNCS 2644, Springer, May 2003, pp. 79-94.

5912 October 2005

ITU-T



o UML Testing Profile (cont.)• Paul Baker, Zhen Ru Dai, Jens Grabowski, Øystein Haugen,

Serge Lucio, Eric Samuelsson, Ina Schieferdecker, Clay Williams. The UML Testing Profile. Proceedings of the 'Conference on Quality Engineering in Software Technology 2004' (CONQUEST 2004) in Nuremberg (Germany), September 22-24, 2004 .

o Further resources• More can be found in the proceedings of the

TestCom conferences and on the homepages of the TTCN-3 team members (e.g., I. Schieferdecker and J. Grabowski) and the TTCN-3 tool providers.

international telecommunication union itu-t study group 17, geneva, 5-14 october 2005 an...

Documents

itut itut study group

implementation of ttcn

o structure of ttcn

services ttcn

o testing

telematics group of

previous ttcn versions

overall view of ttcn