managed testing services - systems plus …...managed testing services systems plus transformations...
TRANSCRIPT
Managed Testing Services “Helps you Run the Business, Change the Business Optimally”
Author: C.V. Narayana
Managed Testing Services
Systems Plus Transformations Confidential Page 1
Contents
Introduction ..................................................................................................................................... 2
MTS is Critical for Enterprises .......................................................................................................... 4
Key Challenges ................................................................................................................................. 4
MTS Overview .................................................................................................................................. 6
Governance ..................................................................................................................................... 8
MTS Roadmap ................................................................................................................................. 9
Value Proposition .......................................................................................................................... 11
Strategic Benefits ................................................................................................. 11
Operational Benefits ............................................................................................ 11
Managed Testing Services
Systems Plus Transformations Confidential Page 2
Introduction
In an increasingly complex business environment, superior customer satisfaction, efficient service delivery
and operational excellence are imperative for enterprises to stay competitive.
IT management teams are focusing their attention on scalable performance and ensuring minimal
application downtime to “Run the Business” as usual. From an application quality perspective, this would
mean the need to ensure rigorous quality checks to identify and eliminate performance bottlenecks as well
as minimise the impact of any new patches or application updates on the production environment. Lack of
consistent QA processes and highly skilled teams lead to deterioration of application quality resulting in
lower customer satisfaction levels, unhappy customers and eroding market share. Keeping the Lights On
(KTLO) though may be a repetitive process, is essential for the survival of a business.
The proliferation of business events like mergers & acquisitions, new technology paradigms (such as social
platforms, mobility, and data proliferation) new growth markets etc. are driving enterprises to “Change the
Business” continuously to stay competitive in the new competitive map. Enterprises are stretched to adopt
newer sophisticated IT systems within short time cycles to get to market faster with differentiated products
and services, and expand to newer high-growth geographies. This puts the spotlight firmly on the QA
organization, adding tremendous pressure to accelerate the testing cycles for new applications even while
ensuring high quality at an optimal cost. All of these also lead to an exponential increase in the complexity
of the IT landscape, which becomes unmanageable in the absence of consistent processes and tools.
Managed Testing Services (MTS) powered by business event-based testing model helps enterprises ‘Run
the Business’ and ‘Change the Business’ at will, without compromising on the quality of service delivery.
Managed Testing Services
Systems Plus Transformations Confidential Page 3
MTS behaves as an extension of the customer organisation by infusing the best of breed of centralised
teams and localised test team. MTS team is an independent verification and validation arm for all
application streams, providing end-to-end testing services - right from Test Strategy to Test Planning, Test
Execution, Reporting and Optimisation. This translates to an ownership for ensuring the completeness,
availability and change management of your business application landscape, managed by SLAs.
Managed Testing Services
Systems Plus Transformations Confidential Page 4
MTS is Critical for Enterprises
Typical enterprise applications fall under the 3 large buckets as given above. There are specific teams that
address web, bespoke (home grown applications), legacy, and packaged applications. Each of these teams
have scope specific to their needs. In addition, each of these teams will have specific process, tools and
standards. These structures lead to following challenges:
Key Challenges:
Spiraling testing costs
o License proliferation
o Underutilised teams
o Duplication of efforts
o Low absorption of best practices and maturity models
o Costs that add because of variable teams
o Knowledge management
o End to end testing visibility and hence issues in production and loss of business
Managed Testing Services
Systems Plus Transformations Confidential Page 5
o Insufficient automation to address the repeated tasks in testing
Product quality
Transparency in testing
Lesser no of cycles of testing
Overloading specialist departments with testing duties
No standardisation of testing activities to align with maturity models
Higher consequential costs caused by errors during live operation such as loss of business down
time, productivity and response times to customers
To address the above challenges, the IT management in an enterprise, have multiple options such as add
teams onsite at customer place or work with a near site team or an offshore team or a combination of
these. This is where Managed Testing Services (MTS) by a specialised vendor can offer onshore/ near shore
and offshore can provide a quality, value and cost effective solution.
Managed Testing Services
Systems Plus Transformations Confidential Page 6
MTS Overview
Managed Testing Services from Systems Plus helps customers, in terms of taking the responsibility of
testing of applications across the business processes in a defined and predictable manner built around SLAs
(Service Level Agreements.)
The pillars of MTS as defined in the diagram are:
1. Test execution process addressing
Alignment with Test Maturity Model
Metrics / KPI to measure the performance of the MTS and provide a quantitative approach to
improvements
Test management to address the scope of testing and plan based on SLAs.
Defect management to ensure the stated DDE (defect detection efficiency) is achieved from inception
to transition of IT applications.
Continuous improvements are carried out both at the operational level based on KPIs (Key
Performance Indicators) and engagement level based on SLAs.
Managed Testing Services
Systems Plus Transformations Confidential Page 7
2. Bouquet of services addressing functional and NFR testing based on ETVX paradigm, templates and
standard operating procedure which can be aligned with the customer processes. We offer unique
testing experience which is business oriented (Business process testing) built on branded or open-
source tools. The testing services provided are:
Test Process Consulting
To define the approach to MTS based on TMMi processes, analysis of existing test assets,
structures and challenges in enterprise
Functional Testing
Based on test cases, test data and test design which lend itself for reusability. Extension to
the service is also available to address Business process Testing (BPT).
Performance Testing
Services address the enterprises investment dilemma by allowing for Opex (operation
expenditure) instead of CAPEX (capital expenditure). This service also reduces the cost of
performance testing by judicious mix of open source and branded licenses to reduce the
cost of performance testing.
Security Testing
Services address the application exposure to threats on the web. There are reusable test
cases, methodology and tools to address the OWASP compliance based on STRIDE
(Spoofing, Tampering data, Repudiation, Information disclosure, Denial of Service,
Elevation of service) classification of threats and DREAD (Damage potential,
Reproducibility, Exploitability, Affected users, Discoverability) rating.
Automation Testing
Built on keyword driven assets which can clearly define ROI for the manual testing with
specific period for the ROI. This service is usually preceded by Automation consulting,
which clearly spells out the business benefits of automation and the expected period for
ROI.
Accessibility Testing (I18N)
Addresses the need for differently abled individuals to access the website and in
compliance to WCAG 2.0 standards
ERP Testing
Which can cover specific business process oriented testing reducing the manual effort and
specialists’ involvement, every time the ERP product is modified.
Managed Testing Services
Systems Plus Transformations Confidential Page 8
Governance
The governance of a Global delivery model is essential to creating a performing engagement on a
structured basis. This is addressed at 3 levels
Operational review addresses the Test plan and actual and the reasons for any deviations and
corrections necessary to address the deviations such as schedule, quality of deliverables. This is carried
out on a weekly basis
Strategic review addresses the engagement issues such as CSAT (customer satisfaction), overall
performance and change management. In steady state this is carried out once a quarter. SLA review is
one of components of the strategic review which looks at the committed SLAs such as DDE, ROI etc and
the compliance and corrective actions
Project teams exchange project data on a daily basis to make sure the project goals are met
Tools and reusable components
Test operation tools. We have extensive experience in setting up Test center of excellence (TCOE) using
open source tools and branded tools depending on the customer needs
Managed Testing Services
Systems Plus Transformations Confidential Page 9
Automation (keyword driven automation). This can reduce the number of licenses for automation, time
to automate and can support open source tools like Selenium and HP QTP. We also have libraries to
migrate the scripts from one platform to another
Business process testing components helps business users to define the expected outcome of a
business process such as order to cash or procure to pay etc., The components built on the keyword
driven library can build the test assets on top of the functional screens by abstracting the manual
process of entering the data for each of the functional screen.
OWASP compliant security test assets
Performance testing framework which converts the capex to opex
WCAG compliant accessibility test assets
MTS Roadmap
MTS roadmap is broadly classified into 3 stages with each stage clearly establishing a stage gate and
deliverables.
Managed Testing Services
Systems Plus Transformations Confidential Page 10
MTS setup: During this phase a small team travels onsite to the customer location to understand the
business processes, IT applications supporting the same, the test team structure, tools and technologies
and the challenges faced by the testing team. Based on the assessment, the team presents the
understanding of the challenges and a roadmap for a managed testing process. As a part of the knowledge
transition, the team will demonstrate the understanding of the testing process and application
understanding by participating in onsite mock office. Offshore facility setup, testing process, transition
roadmap for each of the applications, communication, parallel testing will be the key deliverables of this
setup phase.
Service optimisation comprises of 2 sub stages namely optimised delivery model and resource optimisation.
As part of the optimised delivery, the applications are grouped under web, legacy applications, packaged
products and custom software applications. This enables the human capital to be pooled or optimised
based on the type of technology and hence re-use of technology assets and the team. This increases the
test coverage and reduction in defect detection efficiency leading to reduction in Cost of Quality. Optimal
blended delivery model with an onsite local partner and an offshore delivery partner will provide the most
optimal delivery in terms of customer communication and execution efficiency and cost.
As a part of Resource pooling/ Resource optimisation sub-stage, the NFR reusable assets such as
Automation test assets, Security, performance and accessibility test assets will be used to reduce the cost
of quality and increase the test coverage. As a part of this stage all the stable metrics such as schedule
adherence, defect detection efficiency, y-o-y business benefits, productivity, and demand management will
be baselined
Steady State (SLA based) stage addresses the end to end testing of the applications. During this stage based
on the baselined metrics of the service optimization phase, commercial and operational SLAs are signed.
The team structure will have a fixed and variable team depending on the nature of work such as “Run the
business” or “Change the business”. Sample SLAs include
Y-o-Y reduction in cost of testing
Defect deduction efficiency
Test coverage
Schedule adherence
Managed Testing Services
Systems Plus Transformations Confidential Page 11
Value Proposition
Strategic Benefits
Business event based testing; focus on meeting business needs
Faster turnaround times for go to market initiatives
Drastically reduced management overheads
Overall improvement in end-user satisfaction
Cost reduction of 30-50 per cent by global delivery model
o Introduction of maturity models
o Reusable assets for NFR testing such as automation, security testing, performance testing
and accessibility testing
o Increased level of test automation
o Consolidation of licenses and capex to opex model
Quality of deliverables with increase in defect detection efficiency (DDE) and test coverage
Removal of the burden on specialist departments
Ability to manage fixed and variable teams effectively
With SLA management, the outcomes are more predictable leading to better quality
Operational Benefits
Lower resource- & team-cost, achieved through effective utilization & sharing of resources
Unbiased and objective testing from the business users’ perspective
Streamlined testing processes across multiple application streams as well as reusage of test
artifacts
Defect analysis and feedback to the development team for error reduction and early error
detection
Reusable assets such as automation assets, security and accessibility test assets
Standardized processes
Flexibility in processes and team ramp-up