End-to-End Agile Testing using Incremental Approach for a Leading EIM Solution Provider
ATTENTION. ALWAYS.
Our Customer is one of the global leaders in Enterprise Information Management (EIM) solutions disrupting the ECM market by eliminating information silos and providing quick and easy access to the right content from any core business system and device. They have offices in United States, United Kingdom, Finland, France, Germany, Sweden and Australia and they have about 5,00,000 global users across 100 countries supporting in 25 different languages.
• The product supports desktop, web, mobile and cloud and had about 25000 tests which resulted in time consuming manual activities & increased costs
• Ensuring each build release adhered to the high-quality standards
• Many custom applications built on the base product and each application has its own release schedule.
Business Challenges:
ABOUT THE CUSTOMER
TECHNOLOGY
THE CHALLENGE
Test Complete & Jscript.
C# and NUnit
Selenium, Appium & Java
Selenium, Java and TestNG; Selenium, C# and NUnit.
Desktop, QMS & CK Automation:
Web Automation:
Mobile Automation:
Connector API Automation:
• Frequent releases caused versioning issues that ultimately affected a wide range of customers
• Environment compatibility issues against different operating systems (windows 7/8/10) and browsers like Internet explorer 11, Firefox, Chrome, Safari & Opera and platforms (Windows & MAC).
• Application Server & Client Compatibility issues were challenging to test with the given timeframe especially during releases.
• Multi-language support and compatibility across different applications in the product.
• High flow of regression issues with release builds.
Technical Challenges:
THE SOLUTION Aspire provided end-to-end product testing including manual and automation services for UI Dektop, Web UI, Mobile UI and Connector API.
Aspire executed automation tests to ensure high quality and provide quicker feedback.
1. UI DESKTOP AUTOMATION
• Implemented shift-left testing and encouraged running test automation for each developer change before they merged to a stable branch.
• Implemented Incremental test automation approach and using AFTA (Aspire’s Framework for test Automation) covered most of the repetitive manual testing processes by automating the test cases. Along with this, Functional, Compatibility, API, Performance and Security test automation were also implemented successfully.
• Integrated the CI pipeline using Jenkins and TeamCity for continuous execution of test automation for each and every build eliminating manual steps.
• Developed around 6580 test cases and was also responsible for retesting and follow up of bugs which were created in earlier builds until they were fixed.
• On every stable build, almost 9000 tests with the duration of 405 hrs were running against Windows 7, Windows 8 and Windows 10 using 64 bit & 32 bit operating systems. Also cross browser testing for Chrome, Firefox, IE11, Edge, Mac Safari browsers, Different versions of Android and IOS mobile and tablets were executed.
• Provided release documentation for multiple languages - English, Finnish, French, and Dutch.
Automation execution was run for almost 3-4 times for every build to ensure quality
Smoke Testing was done for compatibility testing in different language builds.
Performed post-release checks.
Highlights
End-to-End automation using continuous integration tools like Jenkins and TeamCity
Identified regression issues at an early stage that lead to a smooth release.
Executed cross-browser testing
Provided impressive Dashboard reports through open source reporting plugin
Highlights
2. WEB UI AUTOMATION
Aspire provided web automation testing services to the customer for their end-to-end product testing. Open source automation tool was used which is feasible for most of the product requirement to reduce the software costs.
High test coverage through Smoke, User Acceptance, Regression & ad hoc tests at different levels
Covered on-Premises & Cloud test environments
Ensured zero defect leakage
Manual regression testing was done for every sprint based on the user story.
Ad hoc testing on pre-release builds after system testing to increase the test coverage.
Highlights
3. WEB & MOBILE MANUAL TESTING
Aspire extended their support on ad-hoc testing on the builds to identify any defects which are not covered as part of the sprint scope. We developed 11000+ test cases in web & 2000 cases in mobile and on every sprint and identified an average of 50-60 defects as part of testing services. Also tested bugs which were created in earlier builds to makes sure the bug fixes are working as expected.
Solution and approach:
4. MOBILE UI AUTOMATION
Aspire provided mobile automation testing services to the customer for their native application testing consisting of 2000+ tests of the product.
Open source automation tool was used to reduce the software costs
Web automation framework was used and hence less effort was required to scale up scripts across platforms like android & iOS product.
Supported different versions of Android & IoS, Mobile & Tablet, and Portrait & Landscape views.
Compatibility testing was executed for different OS versions of Android and IOS, Server and Mobile applications.
Highlights
5. CONNECTOR API AUTOMATION
Aspire provides API automation testing services to the customer for their UI-Extension applications that helps connecting various external repositories. We are also in the process of automating the tests consisting of 1000+ tests of the product with 10 connector applications and there will be 10 more connectors developed each year
Open source automation tool used to reduce the software costs
End to End automation, Integrated with CI tools
Early identification of compatibility issues
Basic performance test reports to measure the performance between different UI-Extension applications.
Extent reports are used for reporting
Aspire’s end to end testing helped the customer streamline their testing process and accelerate the product quality with faster market releases.
Desktop UI Automation
Web UI Automation
Mobile UI Automation
Compliance Kit Automation
Connector API Automation
Full Regression testing
Full Regression testing
Compatibility testing
Sanity testing
Sanity testing
Smoke testing
Smoke testing
Smoke testing
Regression
Regression
Regression
8000
8000
2000
1493
937
254
166
108
2000
1000
734
5300
2670
1476
1206
937
254
166
108
212
610
324
884
445
246
201
156
42
28
18
35
102
54
75
15
96
7
6
3
1
3
5
6
3
809
430
150
194
150
39
27
15
30
96
51
Every two weeks
Monthly
Every two weeks
Daily
Daily
Daily
Daily
Weekly
Monthly
Weekly
Daily
Project Test Suites#Test Cases
Manual Effort
(Hours)
Automation Effort
(Hours)Frequency Saved
HoursAutomated
Cases
More environment test coverage through test Automation
Faster feedback using shift left and continuous testing.
Reduced manual effort spent by executing automation at many places, so that QA members could focus more on the User story & UAT tests for a quicker and promising product release.
THE RESULT
Aspire Systems is a global technology services firm serving as a trusted technology partner for our customers. We work with some of the world’s most innovative enterprises and independent software vendors, helping them leverage technology and outsourcing in our specific areas of expertise. Our core philosophy of “Attention. Always.” communicates our belief in lavishing care and attention on our customer and employees.
ABOUT ASPIRE
www.aspiresys.com
ATTENTION. ALWAYS.
For more info contact [email protected] or visit www.aspiresys.com
SINGAPORE+65 3163 3050
NORTH AMERICA+1 630 368 0970
EUROPE+44 203 170 6115
INDIA+91 44 6740 4000
MIDDLE EAST+971 50 658 8831