agile testing - build a truly agile software organization
Post on 20-Jan-2015
1.271 Views
Preview:
DESCRIPTION
TRANSCRIPT
Agile Testing BUILD A TRULY AGILE SOFTWARE ORGANIZATION
1/30/14
2 ©Alliance Global Services 2014
Today’s Presenter
As Vice President, Solution Engineering, Sreekanth has more than 14 years of senior technology leadership experience and leads Alliance’s Solution Engineering organization in developing cutting edge solutions. He is a respected technology thought leader with a focus in testing Agile projects, test process automation and metrics-based development approaches and utilizing lean techniques in software development.
Today’s Host
As Director of Marketing, Sharon Lee heads the marketing strategy and brand messaging for Alliance. With over 12 years of experience in both digital and traditional marketing, she is focused on the effective use of media for compelling brand messaging and creating successful marketing programs with measurable results that impact revenue. Sharon holds a B.A. from the University of Pennsylvania.
3 ©Alliance Global Services 2014
AGENDA
4 ©Alliance Global Services 2014
Agenda
• The case for QA in an Agile World
• Changing role of QA in an Agile World
• Role of QA in Agile organizations
• Profile of Testers and QA Analysts for these
organizations
• Automation best practices
• Metrics and measurements
5 ©Alliance Global Services 2014
The Case for QA in an Agile World
6 ©Alliance Global Services 2014
The Promise of Agile
Develop iteratively as business evolves
Deliver working software rapidly
Flexibility to change with business needs
Incorporate end-user feedback
Highly productive technical teams
7 ©Alliance Global Services 2014
Common pitfalls of Agile projects
Functionality/defect treadmill
High regression defect seepage
“Epic” failures
Non-functional requirements fail
Low effective team throughput
8 ©Alliance Global Services 2014
What Agile Assumes
Communication
Collaboration
Access to Stakeholders
Cross Functional Teams
9 ©Alliance Global Services 2014
Role of Testing in Agile Teams
Some Agile proponents did not see (or argued against) the need for QA specialists on an assembled team.
If you use TDD, Unit Testing and “Everyone Tests their own Stuff,” do we need a dedicated QA team?
10 ©Alliance Global Services 2014
Agile Layers of Testing
Prep
Development Start Wrapper Sprint 2 or 4 Week Sprints
JumpStart 24h
24h
Harden, Docs, Release
Vis
ion
Pro
du
ct B
ackl
og
Sprint Backlog Tested, Working
Software Increment
Product X
Sprint Review Sprint
Planning
Release Planning
Sprint 1 Sprint 2 Sprint n
3 - 5 Sprints
Prototyping Reqs Testing
Unit Testing Regression
Integration Compatibility Performance
Security
Quality Assurance
Acceptance Functional
Deployment
11 ©Alliance Global Services 2014
Role of QA in an Agile World
12 ©Alliance Global Services 2014
Organization empowers health plans to transform the performance of their provider networks, while significantly reducing medical, administrative and IT costs and simplifies the design, maintenance, reimbursement and performance management of provider networks, while facilitating provider-patient collaboration.
Development Objectives:
• Rapid release of new solutions and rich new features of their core Platform into market to take advantage of increased health care IT spending in the payer market due to health insurance reform (PPACA)
• Ensure high level of customer satisfaction dependent on user experience, maintainability, scalability and platform performance
• Ensure the highest levels of security and confidential data access meeting all regulatory needs and customer specific needs
Development Organization:
• TDD approach for Developers engaged in 2-week sprint cycles
• Enterprise development technologies
• Interfaces with several in-house and 3rd party development teams and applications
• SCRUM, Kanban and Lean techniques
Organization Profile
13 ©Alliance Global Services 2014
Agile Layers of Testing
Prep
Development Start Wrapper Sprint 2 or 4 Week Sprints
JumpStart 24h
24h
Harden, Docs, Release
Vis
ion
Pro
du
ct B
ackl
og
Sprint Backlog Tested, Working
Software Increment
Product X
Sprint Review Sprint
Planning
Release Planning
Sprint 1 Sprint 2 Sprint n
3 - 5 Sprints
Prototyping Reqs Testing
Unit Testing Regression
Integration Compatibility Performance
Security
Quality Assurance
Functional
Deployment
Acceptance
14 ©Alliance Global Services 2014
Testing Activities
Activity Developers Testers
Requirements Testing
Acceptance Unit Testing Functional/Exploratory Testing
Regression Testing
Compatibility Testing
Performance Testing
Security Testing
Deployment Testing
15 ©Alliance Global Services 2014
Quality Assurance
Activity Developers Testers
Test Strategy
Test Planning Process Compliance Root Cause Analysis
Metrics & Reporting
16 ©Alliance Global Services 2014
Automation in Agile
17 ©Alliance Global Services 2014
Automation Objectives
18 ©Alliance Global Services 2014
Layers of Automation Layered Automation Testing Strategy for end to end coverage, automated solutions for
each tier and testing on actual devices
UX Testing
Services & API Testing
Unit Testing
19 ©Alliance Global Services 2014
Automation - Best Practices
Automation is development
Automation Architecture
Lagging sprints
Non-functional
Integration with CI
20 ©Alliance Global Services 2014
Environments
Test Management
CI Integration
Reporting
Infrastructure
21 ©Alliance Global Services 2014
• Re-usable automated test suites with minimal maintenance effort
• Cross application E2E automated testing
• Excel based test suite
• Detailed test summary reports along with step wise description and screen shots
• Test scripts, test reports integrated with QMS
• Automated 55% of regression scenarios which includes 70% of business critical scenarios
Test Automation Framework
22 ©Alliance Global Services 2014
Leveraging:
An Agile test automation platform that drives quality and efficiency
Webdriver / Robotium QTP TestComplete Sikuli
Scriptless Testing
Automation Framework to develop rapidly
across the enterprise creating highly
maintainable suites
Automation Coverage for All
Popular UI and Service Interfaces
InsideEdge AFT utilizes a revolutionary plug-in
architecture that leverages multiple execution
engines to obtain access to UI and Web
Services
Collaboratively Develop Automation
Business Analysts, Functional Testers and Automation Scripters can
collaborate to develop automation as an integral part of testing – not
a separate and specialized activity
23 ©Alliance Global Services 2014
Metrics
24 ©Alliance Global Services 2014
Metrics
25 ©Alliance Global Services 2014
Measuring Effectiveness Automation initiatives need metrics reporting and analysis as an integral part of the initiative to ensure success of the initiative.
• Metrics should capture both business level and execution level values to ensure efficiency is accurately captured and reported
• Metrics should not be left for the end of the sprint/month/project to capture and track
• Metrics measurements allow implementation level course corrections or expectations management easier
• Business Objectives outlined should be directly tied to the metrics
program and measured through out the initiative
• Assumptions and approximations can be made at the beginning of the initiative
26 ©Alliance Global Services 2014
Key Metrics for Agile Projects
• Use Validation points to evaluate application’s map with requirements
• Communicate to all stakeholders the coverage provided by formal testing
Coverage Metrics
• Number of defects identified in different stages of software lifecycle
• Number of defects created in different stages of software lifecycle
Defect Identification Efficiency
• Quality of software code
• Impact of technical or architectural choices Technical Debt
• Demonstrate the confidence of the testing team in the testing efforts executed on a release with the resources provided for testing Confidence Level
27 ©Alliance Global Services 2014
Leveraging:
Shortening time-to-market and enabling predictable delivery of software that works every time
Continuous Integration
Baked-In Quality: script-less automated
functional testing (xAFT), unit tests, continuous
integration (CI), automated builds, and
Clean Code analysis ensure ready to deploy
software without the wait
InsideView™ Toolbox
Growing collection of reusable tools and
components to solve common problems and
speed develop of robust web and mobile
applications
Application Lifecycle
Management (ALM)
Integrated Dashboard for managing user
stories (requirements), tasks, issues, backlogs,
burndowns, defects, quality & productivity
metrics give a one-stop view of all project
activity, progress, and results
Knowledge Management
Formal knowledge acquisition and evaluation
framework that focuses on knowledge
synthesis, assessment and retention so that
the full team understands users, features,
goals & processes and “gets it”
28 ©Alliance Global Services 2014
Automated Code Quality Analysis
29 ©Alliance Global Services 2014
Thank You Sreekanth Singaraju VP of Solution Engineering ssingaraju@allianceglobalservices.com www.allianceglobalservices.com
30 ©Alliance Global Services 2014
Coverage Metrics
Principle Communicate to all stakeholders the coverage provided by formal testing Dimensions • Use Validation points to evaluate application’s compliance with requirements • Weight them by business criticality Definition Designed Coverage = Designed Validation Points / All Validation Points (Estimated) Executed Coverage = Executed Validation Points / All Validation Points (Estimated) Best Practices • Keep coverage metrics simple • Introduce granularity to address business needs • Trend to show progress (or declines – great case for additional resources)
31 ©Alliance Global Services 2014
Defect Identification Efficiency
Principle Identify how defects are identified and created. Evaluate compliance of Agile principles. Dimensions • Number of defects identified in different stages of software lifecycle • Number of defects created in different stages of software lifecycle • Use key Software stages to draw valuable inferences
Definition Proportion of defects identified by each stage of a lifecycle – Ex – Testing – 56%, UAT – 18%, Production - 4%, other - 22% Best Practices • Start with defects identified first – then introduce defects created • Use “Other” to catch exceptions and introduce granularity incrementally • Factor in defect severity if that information is readily available
32 ©Alliance Global Services 2014
Technical Debt
Principle Provide a financial impact of technical choices on the overall product resulting from technology, process and implementation choices made during development Dimensions • Quality of software code • Unresolved defects that are business or quality critical • Impact of technical or architectural choices
Definition Cost estimates for redressing most critical (Business perspective) defects or technical choices Best Practices • Explosive- Handle and communicate with care • Be conservative – use a very strong criteria to evaluate “criticality” • Use several available tools to evaluate software code quality
33 ©Alliance Global Services 2014
Confidence Level
Principle Demonstrate the confidence of the testing team in the testing efforts executed on a release with the resources provided for testing. Informational • Test Execution Coverage • Expertise of resources to test the application • Quantitative results of defects including Defect identification efficiency
Definition Based on the evaluation of the 3 dimensions above rank each of them on a scale of 0(Poor) – 1(High). Overall confidence = (3.Coverage.Resources.Results)/(Coverage+Resources+Results)
Best Practices • Be prepared with supporting facts • Define clearly laid out criteria for ranking
top related