Geneva, 9(pm)-10 February 2009

InternationalTelecommunicationUnion

Providing testability for ITU Recommendations

Ostap Monkewich,OMCI

ITU-T Workshop on“New challenges for Telecommunication

Security Standardizations"

Geneva, 9(pm)-10 February 2009

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 2

Theme

“What is missing from your Recommendations that is needed for

testing?”

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 3

Why do we need to test?

To show the customer that the product meets the requirementsTo show that the product is likely to interoperate with other vendorsTo demonstrate that Recommendations were implemented inside the productTo improve the quality of ITU-T Recommendations

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 4

Need high-quality test results

Repeatable by any competent test laboratoryCan be used to compare with test results obtained for similar productsRecognized by all and accepted in all geographical market regions

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 5

What kind of testing?

Conformance Testingverifies if the implementation does what the Recommendation says it is supposed to do

Product Test Suite (Gold Standard)

Interoperability Testingchecks if two implementation communicate at functionalities level

• N = 6 products or 15 pairs• Each product is tested 15 times • N = 100 or ~5000 pairs• Each product is tested 5000

times

• Functionalities• Every pair

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 6

Frequently Asked Question

Why not do only interoperability testing?Answer: We don’t know what to do when two implementations do not interoperate:

If we change something – are we closer to the Recommendation or farther from it?

Non-conforming changes destroy interoperability with other vendors

Serious problems will not be discovered when observing functionalities only

example: turning lights on and off in a new house is a good “interoperability” or functionality test. The house may burn down at a later time because the wiring did not conform to the standard

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 7

Sources of Interoperability Problems

RecommendationsErrors in RecommendationsAmbiguities in natural languageUnverified or invalid behaviours described

Implementers’ different interpretations of text

Requirements in text, more than one meaningNo standardized questionnaire for supplier

Incompatible Recommendations/implementationsDifferent choices of optionsDevice incompatibilitiesDifferent host system configurations

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 8

In addition to Base RecommendationRequirements Clause

Extracted from text, need no interpretationImplementation Conformance Statement (ICS) Proforma

Supplier declares what pieces were implemented from Recommendation

Implementation eXtra Information for Testing (IXIT) Proforma

Identifies non-standardized items, how architecture, interfaces are packaged

Test Suite StructureLogically groups test cases

Test Purposesone test purpose/verdict per test case

Abstract Test SuiteSet of tests that covers the Recommendation

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 9

Recommendations to support testing

Bas

e R

ecom

men

dat

ion R1

R2

R3

Rn

.

.

.

TP1

TP2

TP3

TPn

.

.

.

TC1

TC2

TC3

TCn

.

.

.

I

CS

An

swer

s

TC1

TC3

TCn

.

.

.

RequirementsClause

TSS &TP ATSExecutionTest Cases

ICS

TSS & TP - Test Suite Structure and Test Purposes TP - Test PurposeATS: - Abstract Test Suite TC - Test CaseICS: - Implementation Conformance Statement R - Requirement

IX

IT I

nfo

rmat

ion

IXIT

IC

S Q

ues

tion

s

ICSProforma

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 10

Requirements – NGN Draft Rec. Y.2702

(R-40) Each user/subscriber associated with an application service subscription is required to be uniquely

addressable

(R-41) It is required that it be possible for the end user to access a service simultaneously multiple times and/or from multiple devices.

(R-42) It is required that it be possible to support multiple

subscription profiles for an individual end-user.

(O-1) It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the end user to uniquely identify the NGN provider.

(O-2) It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the user to

authenticate and authorize the NGN provider.

Sections 7.2.3 and 8.2.1 – NGN User authentication for service access and network attachment

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 11

Implementation Conformance Statement (ICS) Proforma

Index Text Status Ref. Val. Support

3.5.1

3.5.2

3.5.3

When User Equipment (UE) sends HTTP Request does Service Provider (SP) return http Response with <lib:AuthnRequest >

M

M

O

IV.2.1.2

1)

IV.2.1.2

2)

IV.2.1.2

x)

_Yes _No

_Yes _No

_Yes _No

On receipt of HTTP Request from UE does SP send a redirect http Response with <lib:AuthnRequest > in the URL

When UE sends HTTP Request does SP inform the Identity Provider (IdP) of the receipt

Appendix IV to NGN Draft Rec. Y.2702 on Identity management (IdM)

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 12

Examples of Test Purposes inSIP Security Testing

Ensure that the IUT, after sending the initial REGISTER request to the Registrar, ignores Registrar OK response by sending a second REGISTER request Ensure that the IUT re-sends the initial REGISTER request on receipt from the Registrar of a 401 Unauthorized response in which WWW-Authenticate header does not contain the nonce= field

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 13

Implementation Conformance Statement (ICS)

ICS = ICS Proforma with answersA list of requirements and options claimed to have been implemented

Used forShopping for products to match for interoperabilityselecting Test Cases from test suite for test execution

ICS

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 14

Test Suite Structure

Test Suite

Test Case Test CaseTest CaseTest Case

Test Group

Test GroupTest Group

Test Group

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 15

Test Case Structure

Test Case

Test Event Test EventTest EventTest Event

Test Step

Test StepTest Step

Test Step

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 16

Conclusion

First, Conformance testing then Interoperability testingHigh-quality Recommendations are neededBase Recommendations require additional parts to produce widely accepted test resultsMethodology Recommendations for testing are available – attached charts

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 17

Additional Slides

All you need to develop high-qualityRecommendations and Test Specifications

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 18

Conformance and Interoperability Testing Methodology Recommendations

X.290 - General ConceptsX.291 - Abstract Test Suite SpecificationX.292 - (Superseded by Z.160/170 series)X.293 - Test RealizationX.294 - Requirements on Test Laboratories and ClientsX.295 - Protocol Profile Test SpecificationX.296 - Implementation Conformance StatementsSupplement 1 to X.290 series - Generic approach to interoperability testingSupplement 2 to X.290 series - Interoperability testing framework and methodology

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 19

Testing and Test Control Notation (TTCN-3)

Z.161 (Z.140 Revised): Core Language Z.162 (Z.141 Revised): Tabular FormatZ.163 (Z.142 Revised): Graphical Format Z.164 (Z.143 Revised): Operational semanticsZ.165 (Z.144 Revised): Runtime interfaceZ.166 (Z.145 Revised): Control interfaceZ.167 (New): Using ASN.1 with TTCN-3Z.168 (New): The IDL to TTCN-3 mapping Z.169 (New): Using XML Schema with TTCN-3Z.170 (New): TTCN-3 documentation tags

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 20

Specification and Description Language

Z.100 Overview of SDL-2008Z.101 Basic SDL-2008Z.102 Comprehensive SDL-2008Z.103 Shorthand notation and annotation in SDL-2008Z.104 Data and action language in SDL-2008Z.105 SDL-2008 combined with ASN.1 modulesZ.106 Common Interchange Format for SDL-2008

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 21

Abstract Syntax Notation One (ASN.1)X.680 (07/02)  Information technology - Abstract Syntax Notation One (ASN.1): Specification of basic notation

X.680-X.693 (07/02) Information Technology - Abstract Syntax Notation One (ASN.1) & ASN.1 encoding rules

X.681 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Information object specification

X.682 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Constraint specification

X.683 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Parameterization of ASN.1 specifications

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 22

Abstract Syntax Notation One (ASN.1)

X.690 (07/02)Information technology - ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)

X.691 (07/02) Information technology - ASN.1 encoding rules: Specification of Packed Encoding Rules (PER)

X.692 (03/02)  Information technology - ASN.1 encoding rules: Specification of Encoding Control Notation (ECN)  

X.693 (12/01) Information technology - ASN.1 encoding rules: XML encoding rules

Geneva, 9(pm)-10 February 2009InternationalTelecommunicationUnion 23

Supporting Recommendations

Z.120 Message Sequence Chart (MSC)Z.150 User Requirements Notation - Language Requirements and FrameworkZ.151 User Requirements Notation - Language Definition

Z.110 Application of Formal Description TechniquesZ.450 Quality aspects of protocol-related Recommendations