capture & replay
DESCRIPTION
Capture & Replay. TTCN-3 Test Case Generation from arbitrary traces. Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies). Agenda. The Motivation The Technical Challenges Capture & Replay for TTworkbench Demo Q & A. Motivation(s). TTCN-3 Users. - PowerPoint PPT PresentationTRANSCRIPT
TTCN-3 Test Case Generation from arbitrary traces
Capture & Replay
Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Agenda
The MotivationThe Technical ChallengesCapture & Replay for TTworkbench DemoQ & A
2
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Motivation(s)
TTCN-3 UsersWe have a TTCN-3
framework, where do the test cases come from?
IssuesManual test case creation tedious taskError proneDomain Knowledge required
Non-TTCN-3 UsersWe see a scenario and we would like to replay it later
IssuesWhat means replay w.r.t. testing?What is the test system? What is the system under test?
3
The desire to capture observed communication in test scripts suitable for test automation
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
The Idea
4
Automatically generate
test case out of traces
Reproduce & Analyse Situation
from the Field (w/o SUT)
Automate existing
manual tests
Fast IOP Trace Analysis
(IMS/TETRA/ITS)
Codec tests
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
What is a Trace?
5
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
What is a Trace?
The observed communication is captured (stored) in a trace (file)Trace contains
Communication between two or more entitiesMessage exchangeOne or more protocols
6
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
A Generic Communication Setup
7
A1
A2
B1
B2
.
.
.
.
.
.
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
The SetupBut Who is Who?
A1 B1
A1 SUT
SUT B1
IOP1 IOP2
SIM1 SIM2
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
The Setup From Message to Templates
A1 B1
INVITE sip:[email protected] SIP/2.0Call-ID: [email protected]: <sip:[email protected]:5061;transport=UDP>Content-Disposition: sessionContent-Length: 150Content-Type: application/sdpCSeq: 2 INVITEFrom: "TESTER1" <sip:[email protected]>;tag=1169Max-Forwards: 70To: <sip:[email protected]>Via: SIP/2.0/UDP 127.0.0.1:5061;branch=z9hG4bK5704
...
template INVITE_Request T1 := { requestLine := { method := “INVITE”, requestUri := “sip:[email protected]”, sipVersion := “SIP/2.0”}, msgHeader := { callId := “[email protected]”, contact := …, cSeq := “2 INVITE”, fromField := "TESTER1 …" , toField := <sip:[email protected]>, via := …., maxForwards := { fieldName := MAX_FORWARDS_E,
forwards := 70 }, .... allow := omit, ...
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
The Technical Challenge
It is somehow easy to generate TTCN-3 templates from each message, butthe challenge is to obtain usable and maintainable test data and test scripts
Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend
Variable but constant informationShould be configurable before execution
White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution
10
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
The Technical Challenge
It is somehow easy to generate TTCN-3 templates from each message, butthe challenge is to obtain usable and maintainable test data and test scripts
Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend
Variable but constant informationShould be configurable before execution
White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution
11
Protocol and/or domain knowledgehas to be included!
The SolutionAutomated TTCN-3 Test Generation and Execution
12
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Example Scenario
13
SIP
User A calls User B
User B accept call
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Example Scenario
14
SIP
Use a common Monitor Tool to capture messages e.g. Wireshark
Trace File
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Example Scenario
15
Trace File TTworkbench Execution
Put the Trace File into TTworkbench ….
Generate the captured Test Scenarios…
Replay your previously captured Test Scenarios full automatically
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Example Scenario
16
Trace File TTworkbench Execution
Put the Trace File into TTworkbench ….
Generate the captured Test Scenarios…
Replay your previously captured Test Scenarios full automatically
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Know-How Integration
Manual definitions in a configuration file, toExtract existing data and use the same values in the message flow (sequence numbers, call IDs, …)Define uninteresting fields for ignoring at matching time
Automatically extract similar data from templates to create “better” templates and structuresReuse existing
Type systems and codecsTest adapters and component definitionsTest functions and Alt-Steps
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
DEMO!
18
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Key Benefits
Speed up functional and interoperability testingBy reducing manual task
Automate testingGenerate your test, instead of implementing it manuallyExecute them full automatically Get your detailed test reports
Quality Optimization Test case analyzer reduces errors in test cases Fast reusability and easy extension of already defined tests
19
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Supported Capturing Sources & Requirements
Wireshark The most popular network analysis and capturing tool
Jpcap Service The remote capturing service provides capturing on remote machines
Pcap Files Trace files created beforehand
TTworkbench logs (TLZ)Analyze the logs without an SUT
Capturing Console Input and Output
Capturing of data via serial connections (RS232, USB) SSH, TELNET Local Console
WindowsWinPCAP
Linux/UnixLibpcap at least version 1.0
Recommendation: Install Wireshark
Includes the installation of WinPCAP and LibPCAP
20
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Further information
Testing Technologiestestingtech.comfacebook.com -> Testing-Technologies
Capture & Replaytestingtech.com/products/ttworkbench.phptestingtech.com/download/datasheets/CaptureAndReplay.pdf
21
Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©
Contact
22
Bogdan Stanca-Kaposta
Testing [email protected]
om
Theo VassiliouTesting Technologies