improving batch-process testing techniques with a domain-specific language
DESCRIPTION
Slides used at the JavaOne 2011 session.TRANSCRIPT
Improving Batch-Process Testing Techniqueswith a Domain-Specific Language
Alberto Lemos (Dr. Spock)Senior Software Architect
SpockNET
Danival Taffarel CalegariMATERA Systems Architect
Globalcode Instructor
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
Objective
“Show a simple way to improve batch process tests
with a Domain-Specific Language”
Motivation
• Test automation contributes to the project success
• It is hard and time consuming to write and automate tests
• It is more complex and hard to write tests for batch process
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
Principles for a good Test Automation
• Automated tests must be:
• Easy to write
• Easy to run
• Self-contained
• Run quickly
• Always updated
Challenges to test a Batch Process
• How to test Batch Process?
• How to control when it starts or stops?
• How to prevent concurrent run?
• The preparation and validation steps require lots of complex SQLs
Domain-Specific Language
• Why to use DSL to test?
• A good DSL must be:
• Expressive and natural
• Easy to use and learn
• Hard to misuse
• Extensible
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
Making test writing easier with DSL
• Define a language specifically to write the tests
• The language must be closer to the business than to the technology
• The easier the DSL, the greater the chance to it will be used
Making test writing easier with DSL
• The DSL implementation strategy in 2 steps:
• 1st step: Use Internal (embedded) DSL
• 2nd step: Use external DSL
Internal (embedded) DSL
• It is easier to write
• It has tool support
• It provides a good fluency for those who know the underlining programming language.
External DSL
• It has less technological limitations
• It adds more valueper line of code
• It can be used by a layman
Steps for DSL Setup
Create a basic infrastructure to run the processes and verify the results.
Create a DSL for each functionalityWrite the test for the functionality
Steps for DSL Setup
Steps for DSL Setup• How can I run a batch in an isolated form?
Steps for DSL Setup• How can I run a batch in an isolated form?
• How should the test code to run the SQL to verify the results?
Steps for DSL Setup• How can I run a batch in an isolated form?
• How should the test code to run the SQL to verify the results?
• How to make the DSL open a browser, do login and press buttons?
Steps for DSL Setup
Steps for DSL Setup• Use the builder pattern
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
• Provide an abstract base class to be extended
Steps for DSL Setup• Use the builder pattern
• Provide default values or they will be automatically generated
• Provide an abstract base class to be extended
• Reuse similar functionalities
Agenda
• Principles & Proposals
• Why to use a DSL to test?
• 2 Case Studies
Case studies
• Application of the techniques in two global bank systems with strong batch processing.
• Test automation for an already existing system.
• Test automation for an system built from scratch.
First case study
• A system responsible for receiving buy and sell orders from international clients, processing those orders and sending them to brazilian stock exchange systems.
• It’s a very critical system.
First case studyInternational Clients
WebLogic 8.1
WLI
App
InboundQueue
Oracle DB
SQL Server
MQSeries
First case study
• Challenges
• Run the system tests isolated.
• Post messages into WebLogic JMS queues.
• To determine when a process finishes.
Run the system isolated
WebLogic 8.1
International Clients
WLI
App
InboundQueue
MQSeries
Oracle DB
SQL Server
Fedora 4 VM
Windows VM
Run the system isolated
WebLogic 8.1
WLI
App
InboundQueue
Oracle DB
SQL Server
MQSeries
Simplifying test setup• A web application was built to run inside
the test server.
• It acts as a “spy” on the server.
• It can access the server database connections
• It can access the server message queues.
Simplifying test setup
WebLogic 8.1
WLI
App
InboundQueue
Fedora 4 VM
Windows VM
Oracle DB
SQL Server
MQSeries
Simplifying test setup
WebLogic 8.1
WLI
App
InboundQueue
Fedora 4 VM
Windows VM
Oracle DB
SQL Server
MQSeriesTest Spy
Simplifying test setup
WebLogic 8.1
WLI
App
Test SpySelenium Server
Selenium RC
Test cases
Test environment
Simplifying test writing
• A DSL has been made using Selenium to access the test application server.
• The completion for each command is checked by issue SQL commands querying control tables until the results appear or a timeout occurs.
Simplifying test writing
Pros & Cons• The test execution is very “visual”.
• Testers with basic programming skills wrote more than 400 test cases.
• It doesn’t leave an open backdoor in the production environment.
• It was very difficult to put in a continuous integration test environment.
Second case study• A system responsible for receiving
allocation commands for contracts in future markets (BM&F).
• It monitors the allocation status and displays alerts in a monitor.
• System failures and delays can cause fines and image damages to the bank.
Second case studyAllocation commands
Tomcat 6
App
Oracle DB
Tables SP
Second case study• Challenges
• Run the system tests isolated.
• To control the execution of the stored procedure.
• How to check in and out parameters for the stored procedure?
Run the system isolatedAllocation commands
Tomcat 6
App
Oracle DB
Tables SP
Ubuntu Server VM
Run the system isolated
Tomcat 6
AppOracle DB
Tables SPRemoteControl
Ubuntu Server VM
SP control
Tomcat 6
App
Oracle DB
Tables SP*
Tables*RemoteControl
Simplifying test setup
Tomcat 6
App
RemoteControl
Test Spy
Test environment
Spring Remoting
Test cases
Simplifying test writing
• A DSL has been made using Spring Remoting to access the test application server.
• The completion for each command is checked by the remote control put inside the application.
Pros & Cons• It’s faster than Selenium solution.
• The batch execution control is more precise.
• Linux based VM is more portable.
• It demanded such effort to prepare the environment (VM, mock SP, semaphores)
Thank you!
• Alberto Lemos (Dr. Spock)
@drspockbr
http://about.me/drspockbr
• Danival T. Calegari
@danivaltc