mobile test automation for a fortune 500 us retail conglomerate using calabash
DESCRIPTION
Case Study: Reduced Test cycle days from 10 to 1 for Retail conglomerate using Calabash. Aspire was chosen for its prior experience in working under Acceptance Test Driven Development projects.TRANSCRIPT
1
Reduced Test cycle days from 10 to 1 for a F500 US Retail conglomerate using Aspire’s Mobile Testing / Test Automation CoE
Our Customer – Regarded as one of the premier retailers in US, our customer has a presence over almost 850 stores in 45 states. With almost 180,000
employees, its annual revenue stands at around $27 billion.
Challenges
With a larger chunk of its revenue coming through online shopping, our customer felt it should be extremely critical to facilitate end-users with a smooth and easy-to-useapplication for e-shopping. They already developed an application for iOS since most of the end-users (around 85%) were on this platform. This app could be used by the end-user to search for any particular item across different categories and place an order.
Our customer faced the following challenges with their testing arm.Our customer faced the following challenges with their testing arm.
Reduced customer sasfacon due to loads of bugs and it kept on increasing with the subsequent releases
Unable to idenfy a partner who worked in BDD and ATDD Development / Tesng model
Lack of in house experse in Mobile funconal tesng and Test Automaon
Exponenal increase in tesng life cycle me in every release
The applicaons had to be funconally tested on different mobile devices: iPad 2, iPad 3, iPhone 4, iPhone 4s and iPhone 5iPhone 4, iPhone 4s and iPhone 5
1
Solution
Aer rigorous vendor evaluaon, Aspire was chosen as a QA partner for tesng their mobile applicaon based on our experse in Calabash and prior experience in working under ATDD (Acceptance Test Driven Development) scenario. TTo begin with, Aspire set up a tesng CoE (Center of Excellence) for our customer and started funconal tesng with near zero test cases situaon. We kept adding newer test cases and the regression test suite grew larger with each release. Also, Aspire’s team worked on achieving the following matrices while performing regression/ funconal tesng.
Assured execuon of 60 test cases per day TTest cases designed based on requirements would be 70% first me right
80% of defects idenfied could be traced back to the test cases
Aer the 9th release and over 900+ test cases on stack, Test Automaon was considered for implementaon. Considering the factor that our customer wanted to implement ATDD (Acceptance Test Driven Development), we chose Calabash as the automaon tool for the mobile app. Since Calabash could perfectly sit on top of Cucumber, it seemed to be a beer fit for the behavior driven framework. It also provides APIs that are specialized to nave apps running on touch screen devices.
WWe implemented Mobile Test Automaon using Calabash to our customers in record 2 months’ me where we converted 600 test cases into test scripts. Our customer was impressed with our work and they asked us to do web services level performance tesng using Fiddler.
Technology Java
Test Management Tool Test link
Automaon tool Calabash
Approach Behavior Driven Tesng/ Acceptance Test Driven Tesng
Framework Cucumber + JRuby
Performance Tesng Fiddler
EnEngagement Model Tesng as a Service (TaaS)
SNAPSHOT
BENEFITS
Reduced the test cycle from 10 days to just 2 days
Almost 50% cost reduction of the overall QA process
Reduced number of defects with 30%
Good returns on investment due to open source tools
Increased stability of the application
40% resource productivity improvement due to automation
ABOUT ASPIRE SYSTEMSAspireAspire Systems is a global technology services firm serving as a trusted technology partner for
our customers. We work with some of the world's most innovative enterprises and independent
software vendors, helping them leverage technology and outsourcing in our specific areas of
expertise. Our services include Product Engineering, Enterprise Transformation, Independent
Testing Services and IT Infrastructure Support services. Our core philosophy of “Attention. Always." communicates our belief in lavishing care and attention on our customers and employees.
Aspire Systems, Inc.
1735 Technology Drive
Suite 260, San Jose, CA - 95110, USA
Tel: +1-408-260-2076, +1-408-260-2090
Fax: +1-408-904-4591
E-mail: [email protected]
Aspire Systems
1, Lyric Square, Hammersmith
London - W6 0NB, UK
Tel: +44 203 170 6115
E-mail: [email protected]
USA
INDIAAspire Systems (India) Pvt. Ltd.
1/D-1, SIPCOT IT PARK
Siruseri, Chennai - 603 103, Tamil Nadu, India
Tel: +91-44-6740 4000
Fax: +91-44-6740 4234
E-mail: [email protected]
UK
1