Automatic Software Testing Toolfor Computer Networks

ARD Presentation

Adi ShacharYaniv CohenDudi Patimer

www.cs.bgu.ac.il/~adishach

Introduction

Access Layers' flagship product, Portnox, is a natural extension of existing security policies and methodologies, allowing network administrators to configure access parameters for physical network ports, and proceed to monitor, control, and manage LAN entities, including devices, switches, ports, access slots, and more.Portnox represents a new approach to LAN protection, providing complete online control of end-point access to the corporate network.Portnox allows networks administrators to determine which devices are allowed LAN access.

Vision

The tool will provide the ability for Access Layers testers to perform tests on their product easily and efficiently by changing settings and execute actions from a central computer over a group of end-stations.Moreover, the tool will enable them to perform tests that they could not perform before, such as simultaneously executing actions on group of end-stations or by a given schedule.

The Problem Domain

When Access Layers QA's testers execute tests on their software they need to change the settings of an end-stations and to perform different actions, for example: turning on/off the computer, changing the IP address, enable/disable Ethernet card, changes in the network settings of the end-station, login/logoff with users, etc.Nowadays, in order to execute the actions and the setting changes, they should execute all the necessary actions manually in the end-stations.

Solution (1)

Our project allows performing all the above actions automatically from a central station with simple GUI, and in addition to view the status of the end-stations at any step. Furthermore, our tool will be able to add and execute new scripts that were written by the testers on the end-stations, manage the scheduling of these actions, meaning to execute actions in chosen order, and to schedule the execution order of the end-stations.

Solution (2)

The tool will be composed of the following components:

• TSC (Test Set & Configuration) – it contains a set of actions that the user chooses to run and also configuration settings such as choosing the end-stations for the test and scheduling the execution of the tests.

• Agents – this component will be placed on the end-stations; its responsibilities are to receive and execute commands on the end-station, and to send results and end-station state back to the ATS.

Solution (3)

• ATS (Automatic Testing Server) - the main component of the project; its responsibilities are to:o Establish connection to the end-stations.o Remote executing commands/TSC on the end-stations, receiving

results and end-stations state.o Create, modify and save TSC.o Define and validate the pre-conditions and the post-conditions for

each action in the TSC.o Generate reports and logs.

Solution (4)

• GUI – friendly and easy to use interface.

• Database – this component stores the following data:o Basic actions – definition of the pre-conditions, the post-condition

and the commands/script for the execution for each basic action. The basic actions are depending on the operating system, meaning the scripts should use the commands of the operating system.

o Logs – events that occur due to the use of the tool. o Additional actions – scripts that are written by the user, their pre-

conditions and post-conditions. o TSC.o Executions results, which contains the state of the end-stations

before and after execution of each action.

High-Level Overview

ATS

Database

End-Stations

TSC

GUI

Functional Requirements (1)

Establish Connection & Getting General Information

• The Agent will be able to get general information from the end-station, such as information about the operating system, Ethernet card, state of the station, etc.

• The ATS will be able to receive general information from the Agent.

• The ATS will be able to establish connection to Agents in the end stations by IP address or by physical address.

Functional Requirements (2)

Assembling a TSC (1)

• The user will be able to create a new TSC composed of the following components:

o Basic actions (built-in actions).o Additional actions (scripts).o TSC.

• The user will be able to configure the following settings in the TSC:o The end-stations which the user selected to run the TSC on.o The scheduling policy of the components in the test set.o The scheduling policy of the execution of tests between the end-

stations that was selected (parallel, the order of the end-stations, the time gap between each pair of end stations, etc).

o The number of times to run this TSC.

Functional Requirements (3)

Assembling a TSC (2)

• The ATS will be able to generate a script from a TSC, which will include the pre-conditions and post-conditions scripts before and after each action in the test set, the scheduling policy and more.

• The user will be able to save/load/modify/delete the TSC to the database.

Functional Requirements (4)

Creating Additional Actions

• The user will be able to add actions to the system.

• The additional actions will be composed of pre-conditions, post-conditions scripts and the script of the actions.

• The user will be able to load scripts from file or to write/edit script using the editor in the ATS.

• After additional action was added to the ATS, the user will be able to assemble TSC that contains this new action.

• The user will be able to delete saved additional action.

Functional Requirements (5)

Executing TSC or Single Action

• The ATS will check the state of the end-stations before executing the TSC/single action.

• The user will be able to execute the TSC/single action on the selected end-stations.

• The ATS will process the results and will act according to the results.

• The user has the ability to view the state of each end-station at any step of the execution of the TSC.

Functional Requirements (6)

Generating Reports & Logs

• The user will be able to get reports of the last execution or other executions that were saved on the database.

• The ATS should log the results and other activities for debugging and monitoring the behavior of the system, moreover to overcome inconsistency in the database.

Non-Functional Requirements (1)

Speed, Capacity & Throughput

• The ATS should be able to handle as much as possible end-stations.

• The execution of TSC/single action should take at most as the timeout defined in the TSC/single action (the timeout defines the worst case).

• Generating reports from the results will complete in less than 1 second.

• All the GUI interactions, except actions that need communication, will complete in less than 1 second.

Non-Functional Requirements (2)

Reliability

• The ATS will check the state of each end-station before executing TSC on them, and will notify the user if the state isn't appropriate to execute TSC.

• The system relies on the end-stations while executing TSC, if one end-station fails on executing an action the Agent will inform the ATS about the failure.

• The system will define timeout for each TSC. If an end-station disconnects during executing TSC, the system will detect it and send a message to the user.

Non-Functional Requirements (3)

Safety & Security

• The system is an internal organization system. The data does not need to be encrypted. No cryptographic protocols will be used.

Portability

• The system will work on all of Microsoft Windows versions; the design will support extension to communicate and execute commands/actions on other operating systems in the future.

• All the components in the system should be on the same LAN.

Non-Functional Requirements (4)

Usability

• The system will be easy to use and the user will require several hours to learn how to operate the system.

• The user will have to be familiar with the script languages being used in the system in order to create test descriptions via scripts.

Major Use-Cases (1)

Major Use-Cases (2)SectionPurpose

NameAssembling a TSC.

DescriptionThe creation process of a new TSC.

Pre-ConditionThere exists at least one basic action or additional action on the database.

Post-ConditionA new TSC is created and stored on the database.

Basic Course of Action1. The user requests to create new TSC.2. The system displays the 'new TSC' screen.3. The user selects the basic actions he wants to include in the test.4. The user selects the additional actions he wants to include in the test.5. The user selects the TSC's he wants to include in the test.6. The system loads the selected TSC's from the database.7. The user selects the end-stations on which to run the TSC.8. The user selects the scheduling policy of the execution between the

end-stations.9. The user selects the order of execution for the actions on the TSC.10. The user requests to save the TSC.11. The system saves the TSC on the database.12. The system informs to the user about successfully creation of TSC and

presents information on the new TSC (estimated time of execution, max time and more).

Major Use-Cases (3)SectionPurpose

NameCreate additional action.

DescriptionCreating new additional action.

Pre-Condition-

Post-ConditionThe additional action added to the system and stored in the DB.

Basic Course of Action1. The user requests to create additional action.2. The system displays the 'New Additional Action' screen.3. The user selects a name for the new additional action.4. The user writes the pre-condition script for the additional action.5. The user writes the script for the additional action.6. The user writes the post-condition script for the additional action.7. The system stores the new additional action to the DB.

Major Use-Cases (4)SectionPurpose

NameExecute a single action (1).

DescriptionExecuting a single action on selected end-station\s and getting results and states.

Pre-ConditionThe ATS is connected to the end-stations that were selected.

Post-Condition1. The single action was executed on all of the selected end-stations.2. The ATS received the results of the execution and the new state of the end-

station\s.

Basic Course of Action

1. The user requests to execute a single action.2. The system displays the 'Single Action Execution' screen.3. The user selects action to execute from a list of all actions.4. The user selects the end-stations to execute the action on.5. The user requests to execute the action.6. The system sends state request to the selected end-station\s.7. The Agent sends the state back to the ATS.8. The system receives the state of the end-station\s.9. The system validates the state of each end-station\s.10. The system sends the action to the Agents on the end-station\s.

Major Use-Cases (5)SectionPurpose

NameExecute a single action (2).

Basic Course of Action

11. The Agent runs the action on the end-station.12. The Agent sends the results and the state of the end-station to the ATS.13. The system receives the execution results and state.14. The system generates report.15. The system presents the report to the user.16. The system stores the execution results and the states of the end-stations on the

database.

Major Use-Cases (6)SectionPurpose

NameExecution of TSC (1).

DescriptionExecuting a TSC on selected end-station\s and getting results.

Pre-Condition1. The TSC exists.2. The ATS is connected to the end-stations that are defined on the TSC.

Post-Condition1. The TSC was executed on all of the selected end-stations.2. The ATS received the results of the execution.

Basic Course of Action1. The user requests to execute a TSC.2. The system displays the 'TSC Execution' screen.3. The user selects TSC to execute.4. The system loads the selected TSC.5. (Optional) The user selects the end-stations to execute the TSC on (If

the user skip on this step the end-stations to execute the TSC on are as defined at the TSC).

6. The user requests to execute the TSC.7. The system sends state request to the selected end-station\s.8. The Agent sends the state back to the ATS.9. The system receives the state of the end-station\s.10. The system validates the state of each end-station\s.

Major Use-Cases (7)SectionPurpose

NameExecution of TSC (2).

Basic Course of Action11. The system sends the next action to the Agents on the end-station\s.12. The Agent runs the action according to the scheduling defined in the TSC.13. The Agent sends the results and the state of the end-station to the ATS.14. The system receives the execution results and states.15. The system validates the results and the states.16. Return to step 11 until all the actions in the TSC done.17. The system generates report.18. The system presents the report to the user.19. The system stores the execution results and the states of the end-stations

on the database.

Risks

• Handling large number of end-stations, identifying and managing the results for each end-station.

The End