bdd and test automation in evalutionary product suite
TRANSCRIPT
BDD and Test Automation in EvolutionaryProduct Suite
Lasantha Ranaweera
Software ArchitecteBuilder Technology Center
Colombo
Colombo Java Meetup 2015
Outline
IntroductioneBuilderQuality Story
Same Problem, New SolutionBit of TheoryImplementationChallengesFuture DirectionsKey Take Aways
eBuilder
At a Glance
An Evolutionary Product Suite
One Core Layer, Many ProductsCustomer Care Orchestration (CCO)
An Evolutionary Product Suite
One Core Layer, Many ProductsCustomer Care Orchestration (CCO)
An Evolutionary Product Suite
One Core Layer, Many Products
Customer Care Orchestration (CCO)
An Evolutionary Product Suite
One Core Layer, Many Products
Customer Care Orchestration (CCO)
Key Properties
BusinessI Software as a Service.I Integration is a Key Part.
Core PlatformI Legacy => JEE Based, Started Over 12 Years Ago.I CCO => Based on Microservices.
ProductI Legacy => Running on JBoss AS.
Quality Story
Long Story in Short
I No Automated Unit Testing from Inception.I Automation Varied on Product Level.I Existing Automation Tools Old/Not Reliable.I Several Automation Attempts Ended with Failure.I Long QA Cycles / More Slip Through.I Story Continues ....
Bit of Theory
TDD vs BDD
Language and Scopehttps://www.youtube.com/watch?v=mT8QDNNhExg
Language
TDD
assertequals(counter,5);
BDD
counter equals to 5;
Scope
Scope
Scope
Scope
Scope
Testing Pyramid
http://martinfowler.com/bliki/TestPyramid.html
Is TDD Dead ?
http://martinfowler.com/articles/is-tdd-dead/
Just Say No to More End-to-End Tests
Unit E2E
Fast
Reliable
Isolates Failures
Simulates a Real User
http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-more-end-to-end-tests.html
Step 1
Step 2
Step 3
Step 4
Step 5
Implementation
Picard
Time Line
I Started 2013 Q2.I Started for One Critical Product with 0% Automation.I Extensive Support Given by Top Management.I Identify Potential as a Common Framework.I Roll-out Carried with Help of Product QA Teams.I Currently Using 5 eBuilder Products.
In the Start
ConsiderationsI End User.I Acceptable Time Limit.I Many Types of Tests.
DecisionsI End User => Testers in Scrum Teams.I Semi Technical Language Suitable for Testers.I Stories Independent from Each Other.I Developing a Framework for All Domains in eBuilder.I Support Atlassian Bamboo CI Server.
In the Start
ConsiderationsI End User.I Acceptable Time Limit.I Many Types of Tests.
DecisionsI End User => Testers in Scrum Teams.I Semi Technical Language Suitable for Testers.I Stories Independent from Each Other.I Developing a Framework for All Domains in eBuilder.I Support Atlassian Bamboo CI Server.
Different Skills and People
Testers DevelopersTesters DevelopersTesters DevelopersTesters Developers
Behavior Suite Picard
Different Skills and People
Testers Developers
Testers DevelopersTesters DevelopersTesters Developers
Behavior Suite Picard
Different Skills and People
Testers Developers
Testers Developers
Testers DevelopersTesters Developers
Behavior Suite Picard
Different Skills and People
Testers DevelopersTesters Developers
Testers Developers
Testers Developers
Behavior Suite Picard
Different Skills and People
Testers DevelopersTesters DevelopersTesters Developers
Testers Developers
Behavior Suite Picard
Separation of Work
Behavior SuiteI JBehave + XML + XPath and Property Files.I One Central Configuration File.I Many Story Suites
1. Smoke Suite2. Regression Suite
PicardI Picard has over 99% Java Code.I Spring based XML configurations.I Apache Maven to Run Stories.
Separation of Work
Behavior SuiteI JBehave + XML + XPath and Property Files.I One Central Configuration File.I Many Story Suites
1. Smoke Suite2. Regression Suite
PicardI Picard has over 99% Java Code.I Spring based XML configurations.I Apache Maven to Run Stories.
Bit More Details..
High Level Story
Base 1 Base 2 Base 3 Base 4
Mapping Layer
Web Browser XML FTP Web Service
JBehave Code Generator
I Long Running Scenarios.I Model Reusable Base
Stories.
Story Generator
High Level Base
A High Level Story
A High Level Story
A Base Story
Main Configuration File
Page Mapping File
Step Mapping
Demo
Demonstration
Browser Testing
FactsI Supports Struts, JQuery, ExtJS and Bootstrap.I FluentSelenium with XPath are used.I Non-ambiguous Step Naming was a Challenge.I Page Names Given as Configurations.
Cross Browser
Browser Testing
FactsI Supports Struts, JQuery, ExtJS and Bootstrap.I FluentSelenium with XPath are used.I Non-ambiguous Step Naming was a Challenge.I Page Names Given as Configurations.
Cross Browser
Integration Testing
I XML, Excel and EDI File Handling.I Support for both FTP and SFTP Protocols.I Web Service Testing
1. Traditional2. RESTFul
Bamboo Server Integration
I Production Environment for Test Suite.I Customized Reporting Module.I Extended Reporting Module Provided by Enthusiast.I Both Parallel and Sequential Test Execution with Streams.
Test Stream Execution
Test Stream Execution
Test Stream Execution
Test Stream Execution
Ease of Use
I JBehave + Maven Provide Good Tooling.I Easy Start with Base Stories.I Readability of Step Mappings
1. Good Naming Convention2. Logical Grouping
Challenges
Reliability
I Proven Base Stories.I Understand UI Behavior.
Unit E2E Picard
Time Limit
I Stream Implementation.I No Long Wait Times.
Unit E2E Picard
Isolate Failures
I Screen Capture for UI Failures.I Proper Logging for Step Implementations.
Unit E2E Picard
Future Directions
Using Amazon EC2 For Test Automation
I Scalable CI server with Elastic Bamboo.I Heterogeneous Environment Testing.
Microservices Testing
I E2E Test Automation Not Easy with Microservices.I Automating Unit Tests Easier.I REST Heavily Used in Inter Service Communication.I Planning to Support Consumer-Driven Test Framework.I Mobile UI Testing Another Potential Avenue.
Key Take Aways
I Different Skills + BDD => Better Quality.
I No Survival Without Test Automation.I Unit Testing Number 1 Priority.I There are Situations E2E Testing Better Suited.I Good Results Can Achieve Even with E2E Testing.
Key Take Aways
I Different Skills + BDD => Better Quality.I No Survival Without Test Automation.
I Unit Testing Number 1 Priority.I There are Situations E2E Testing Better Suited.I Good Results Can Achieve Even with E2E Testing.
Key Take Aways
I Different Skills + BDD => Better Quality.I No Survival Without Test Automation.I Unit Testing Number 1 Priority.
I There are Situations E2E Testing Better Suited.I Good Results Can Achieve Even with E2E Testing.
Key Take Aways
I Different Skills + BDD => Better Quality.I No Survival Without Test Automation.I Unit Testing Number 1 Priority.I There are Situations E2E Testing Better Suited.
I Good Results Can Achieve Even with E2E Testing.
Key Take Aways
I Different Skills + BDD => Better Quality.I No Survival Without Test Automation.I Unit Testing Number 1 Priority.I There are Situations E2E Testing Better Suited.I Good Results Can Achieve Even with E2E Testing.
Key Take Aways
I Different Skills + BDD => Better Quality.I No Survival Without Test Automation.I Unit Testing Number 1 Priority.I There are Situations E2E Testing Better Suited.I Good Results Can Achieve Even with E2E Testing.
Thank You