MASTER TEST SPECIFICATIONFORSIMPLE RAILROADCOMMAND PROTOCOL(SRCP)

28 JUNE 2010

Tester,

Ankit Singh

Masters High Integrity System

University of Applied SciencesFrankfurt am Main

OVERVIEW

SRCP Version 0.8.3

SRCP is an internet protocol for controlling and programming (digital) model railway system.

Communication: TCP Network connection between Client & Server

SRCP Provides Three Modes of operations: Handshake Mode Command Mode Info Mode

PROJECT SCHEDULE

STRATEGIES: TO SELECT THE FINAL TEST CASES

Important Network Establishment between a client and a server

1. Level of priority a. MUST/MUST NOT tagged requirements b. SHALL/RECOMMEND tagged requirements c. CAN/OPERATIONAL tagged requirements

2. Valid/Invalid requirements

3. General flow of operations requirements

TEST METRICS USED

Test metric Definition Purpose How to calculate

Defect severity The severity level of a defect indicates the potential business impact for the end user (business impact = effect on the end user x frequency of occurrence).

Provides indications about the quality of the product under test. High-severity defects means low product quality, and vice versa. At the end of this phase, this information is useful to make the release decision based on the number of defects and theirseverity levels.

Every defect has severity levels attached to it.Broadly, these are Critical, Serious, Medium and Low.

Test coverage Defined as the extent to which testing covers the products complete functionality.

This metric is an indication of the completeness of the testing. It does not indicate anything about the effectiveness of the testing. This can be used as a criterion to stop testing.

Coverage could be with respect to requirements,functional topic list, business flows, use cases,etc. It can be calculated based on the number of items that were covered vs. the total number of items.

BLACK BOX TESTING

Equivalence partitioning

Boundary-value analysis

Error guessing

WRONG TEST SPECIFICATION (BLACK BOX TESTING)

Test Case ID.No Test Area Command Expected Output Behavior Description

Severity Level

3_1 Command set ConnEctionMode Srcp Command

410 ERROR unknown command

Fail : <TimeStamp>202 OK CONNECTIONMODE

Low

3_2 Feedback SET 1 FB 0x111111 0 100 INFO 1 FB 0x111111 0

Fail: <TimeStamp> 422 ERROR unsupported device group

Critical

3_3 Session GET 0 SESSION 2 100 INFO 0 SESSION 2

Fail: <TimeStamp> 422 ERROR unsupporteddevice group

Critical

3_4 Server RESET 0 SERVER 101 INFO 0 SERVER Fail : <TimeStamp> 423 ERROR unsupportedoperation

Crititcal

3_5 Time TERM 0 TIME 102 INFO 0 TIME Fail: <TimeStamp> 422 ERROR unsupported device group

Critical

WHITE BOX TESTING

Statement coverage

Decision coverage

Condition coverage

WHITE BOX TESTING REPORTSAFE CODING GUIDELINES

Function Bug Suggestion

enqueueInfoFB 1000, Message length hardcoded

Should be passed any specified message, and calculate sizedynamically

queue_reset_fb usleep not checked for incorrect result

check the result returned by usleep()

infoFB msg overflow can occur, so message length

should be dynamically checked

Check for the overflow

FINDINGS TCP-port 4303 assigned by IANA but not

mentioned in specification

Incomplete data in specification leads to confusion

Insufficient details in Specification

In some cases commands are not working on certain device groups

Differences between Specification & srcpd implementations.

Proper Return checks are not done in the source codes which can lead to crashing of the system.

IMPROVEMENTS

Perform stress tests (eg. high number of simultaneous connections)

Increase code coverage

Increase control over environment (OS settings, real time debugging info)

Improve overall clarity about specification

Server Crash testing (Server crash and recovery time)

CONCLUSION

Full code coverage using black-box testing isn’t feasible

White-box in combination with black-box testing considerably increases code coverage

Increased code coverage = Increased code safety

REFERENCES

Lecture notes and slides of Dr. Shoenfelder.

The Art of Software Testing, Second Edition, Glenford J. Myers

http://en.wikipedia.org/wiki/Test_case

Thank You

For

Listening patiently&

Asking No Questions!!!