software development qa best practices

24
Software Development Software Development QA Best Practices QA Best Practices May 20, 2010 Suzette Hackl, CSM Senior Project Manager Skyline Technologies, Inc.

Upload: kedem

Post on 09-Jan-2016

33 views

Category:

Documents


1 download

DESCRIPTION

Software Development QA Best Practices. Suzette Hackl, CSM Senior Project Manager Skyline Technologies, Inc. May 20, 2010. Agenda. Definition of “Best Practices” Software Quality Assurance as it relates to: Process People Technology Application to SCRUM Questions?. “Best Practices”. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Software Development QA Best Practices

Software DevelopmentSoftware DevelopmentQA Best PracticesQA Best Practices

May 20, 2010

Suzette Hackl, CSMSenior Project Manager

Skyline Technologies, Inc.

Page 2: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”Definition of “Best Practices”• Software Quality Assurance as it relates to:

– Process– People– Technology

• Application to SCRUM• Questions?

Page 3: Software Development QA Best Practices

“Best Practices”• A collection of Software Quality “What Works Now” practices. • Suzette’s collection of practices come from many sources and span across industries,

sectors, and experiences.• Recognizable, Obvious, Debatable…..• The list of practices in the QA space is large. A large laundry list is hard to conceptualize

and translate into implementation. I have grouped my culled down list by People, Process, and Technology and categorized them by:

– Basic: The basic are exactly that. They are the training wheels you need to get started and when you take them off, it is evident that you know how to ride. When you eventually take off your training wheels and ride that does not mean that you always remember how to ride. Each one of these basic practices is “needed” for successful quality assurance. Removing or discontinuing one of these basic practices will degrade quality of your systems and potentially put business at risk.

– Foundational: The foundational practices are the rock and mortar that will hold your “Quality” program together. They have significant value adds to an organization. Unlike the basic, they are probably not as well known and therefore need implementation help.

– Incremental: The incremental practices are to be judiciously applied in special conditions and bring many returns in terms of value. Individually they do not provide broad gains across the board of testing because of their specialized nature.

Page 4: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:

– ProcessProcess– People– Technology

• Application to SCRUM• Questions?

Page 5: Software Development QA Best Practices

QA ProcessesStandard & Consistent Requirements for Test Planning

One of the roles of software testing is to ensure that the product meets the requirements of the client. Capturing the requirements therefore becomes an essential part not only to help develop but to create test plans that can be used to gauge if the developed application is likely to meet the customers’ needs. Therefore, requirement management and its translation to produce test plans is an important step. The premise of this practice is that requirements are consistently and clearly documented so as to be interpreted correctly by the development and QA team members.

Standard & Consistent Functional Specifications or Solution Design Documentation Functional Specifications are a key part of many development processes. While it is a development process

aspect, it is critically necessary for software functional testing. The advantage of having a functional specification is the test generation activities could happen in parallel with the development of the code. This is ideal from several dimensions. Firstly, it gains parallelism in execution, removing a serious serialization bottleneck in the development process. By the time the software code is ready, the test cases are also ready to be ran against the code. Secondly, if forces a degree of clarity from the perspective of a designer and an architect so essential for the overall efficiencies of development. Thirdly, the functional specifications become documentation that can be shared with the business to gain an additional perspective on what is being developed.

Common Glossary of QA Terminology This may seem to be the most basic of all concepts but it is one of the most important. Having clearly defined

terms communicated across the organization will go a long way in the acceptance and engagement of Quality Assurance

Page 6: Software Development QA Best Practices

QA Processes - continuedReviews & Inspection

This best practice includes code reviews, Internal and external QA Audits, and Project Audits.

Test Monitoring & Governance The premise of test monitoring is to ensure quality key performance indicators are being met. This is done thru

verification across project phases and deliverables. By test monitoring, QA assures software completeness and readiness for delivery.

Page 7: Software Development QA Best Practices

QA Processes - continuedFormal Entry and Exit Criteria The idea is that every process step has a precise entry and precise exit criteria. These are defined by the development

process and are watched by management to gate the movement from one stage to another. The preciseness of these criteria is determined by each individual project leadership. However, this practice allows much more careful management of the software development process.

Functional Test - Variations Most functional tests are written as black box tests working off a functional specification. The numbers of test cases that are generated usually are variations on the input space coupled with visiting the output conditions. Functional testing involves writing down different variations to cover as much of the state space as one deems necessary for a program. The best practice involves understanding how to write variations and gain coverage which is adequate enough to thoroughly test the function. Given that there is no measure of coverage for functional tests, the practice of writing variations does involve an element of art

User Validation and Acceptance The idea is to release an application to a limited number of users and get feedback to fix problems before implemented to the entire user community. This best practice has everything to do with getting functionality in the hands of users as early in the process so as to reduce cost / business interruptions and post implementation issues

Page 8: Software Development QA Best Practices

QA Processes - continued

Automated Test Execution The use of automated testing to support rapid testing, capturing issues earlier in the development lifecycle, and providing detailed logs of frequent test results (sometimes through nightly builds). The automated test environment must be carefully setup to ensure accurate and consistent testing outcomes. Automatic testing should be run at code check-in to minimize changes of disrupting a working build. The goal of automated test execution is to minimize the amount of manual work involved in test execution and gain higher coverage with a larger number of test cases. The automated test execution has a significant impact on both the tool sets for test execution and also the way tests are designed. Integral to automated testing is the verification of current operations and logging of test failures with diagnosis information

“Nightly” Builds & Continuous Testing The concept captures frequent builds from changes that are being promoted into the change control

system. The advantage is firstly, that if a major regression occurs because of errors recently generated, they are captured quickly. Secondly, regression tests can be run in the background and cycled when it makes sense. Thirdly, the newer releases of software are available to developers and testers sooner. Continuous testing uses excess cycles on a developer's workstation to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It reduces the time and energy required to keep code well-tested, and prevents regression errors from persisting uncaught for long periods of time.

Page 9: Software Development QA Best Practices

QA Processes - continuedIntegration Testing with User Scenarios As we integrate multiple software products and create end user applications that invoke one or multiple

systems, the task of testing the end user features gets complicated. One of the viable methods of testing is to develop user scenarios that exercise the functionality of the applications. These are called user scenarios. The advantage of the user scenario is that it tests systems in the ways that most likely reflect a user’s everyday usage

Software Development Lifecycle & Incorporating QA Quality Assurance encompasses the entire software development process, which includes processes such as software design, coding, source code control, code reviews, change Management, configuration management, and release management. There are specific phase activities that should be conducted during the software life cycle. At the conclusion of each phase there is a key gate check and management decision that the existing phase has been completed and approval to proceed to the next phase. Below is an example of how this applies to SCRUM.

Page 10: Software Development QA Best Practices

QA Processes - continuedQuality Assurance Benchmarks and Performance Indicators

Establish and collect useful metrics that will facilitate decision making. Benchmarks and Performance Indicators (PI´s) cannot be meaningfully formulated unless there is a direct connection between them and the related policies and goals.

Usability Testing Usability testing means a way to measure how people (intended/end user) find it (easy, moderate or

hard) to interact with and use the system keeping its purpose in mind. It is a standard statement that "Usability testing measures the usability of the system". Some would argue that application usability perceived by the user community is the utmost factor that determines success. Usability testing needs to not only assess how usable a system is but also provide feedback on methods to improve the user experience and thereby gain a positive quality image.

Test Case Quality and Re-usability This practice is two-fold. Firstly, the valuation of a test case’s quality. Secondly, the value of re-using

test cases where applicable. Secondly, the re-usability of test cases.

Pairing There are various methods of pairing skilled individuals of a project team to produce higher quality of

work more efficiently; pairing developers, teaming testers with developers, etc. It’s been recognized for a long time that the close coupling of testers with developers improves both the test cases and the code that is developed

Page 11: Software Development QA Best Practices

QA Processes - continued

Risk Management and Confidence The notion of this practice is to focus testing on risky areas and to find the important bugs; you have to practice some kind of Risk Management. There is a factor of confidence used here. The practice of accessing a risk factor to tests can also help with resource modeling. For example; testing of high risk, low confidence functionalities make more sense to be staffed by those QA Team members with high domain knowledge while low risk, high confidence functionalities could be staff by lower priced resources.

Page 12: Software Development QA Best Practices

QA Processes - continuedBug-Tracking and Resolution with Root Cause Analysis

Estimating

Traceability The linkages between test cases, their parent design elements, and their parent

requirements are maintained through a technique termed Requirements Traceability. In essence, it is necessary to be able to trace the linkage from a test case all the way through to a grandparent goal.

INPUTS OUTPUT Requirements WBS Project Charter /Other Project Resource Plan Documents Schedule System Docs

Budget (total project ETC)

User/Customer Interviews Architecture Diagrams

Estimating Approaches:

1 – Top Down

Start at the system level and assess the overall system

functionality and how this is delivered through sub-systems

2 – Bottom Up

Start at the component level and estimate the effort

required for each component. Add these efforts to reach a

final estimate

Page 13: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:

– Process– People– Technology

• Application to SCRUM• Questions?

Page 14: Software Development QA Best Practices

PeopleSoftware Quality Ownership The premise of this practice is that business owns the task of determining the quality expectations

and KPIs of an application. Quality Control owns the task of assuring that software meets those quality guidelines.

Clearly Defined QA Roles & Expectations QA Professional/Senior Lead (QA Analyst III) QA Lead (QA Analyst II ) QA Analyst (QA Analyst I) SCRUM Team Member

QA Analyst Domain Knowledge Testers require domain knowledge and have to be at least as skilled as an expert user/power user.

Good Writing Skills are Mandatory Writing understandable and repeatable test scripts, which also contains information about the idea

and the intention of the test, the expected results and technical background, isn’t easy. There's also a fine line between too detailed test scripts, which are difficult to maintain, and too superficial ones. This is truly an “art” and not a science and takes experience. A good test case is one which brings you closer to your goal. There’s no simple formula or prescription for generating “good” test cases.

Page 15: Software Development QA Best Practices

People - continued

Software Delivery & Capacity Planning A unit of work represents both the functionality being delivered and the quality of the functionality. We see that each work unit has both functionality and a quality component. We don’t want to intentionally plan to deliver functionality without quality

QA Members have broad and comprehensive knowledge in IT

The premise of this best practice is that QA folks mustn’t be experts or gurus, but they should be able to warm up fast to many technologies. For example different operating systems, networks, Internet, databases, program languages, etc. Quality Assurance is not only closely related to development but also support, technical writing and field service. Therefore QA engineers should have some knowledge and experience in at least one of these areas. There shouldn't also be any borders or constraints that prevent or discourage QA people to switch to development, technical writing, field service or support - and vice versa.

Page 16: Software Development QA Best Practices

People - continuedQA people need to be good communicators and team players They

must be diplomatic (but sometimes tough), convincing and able to present.

QA Job Satisfaction - Paradigm shift Awareness of the value of Quality Assurance to an organization greatly impacts job satisfaction for a

QA professional. This includes the understanding and acceptance of testing limitations.

Page 17: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”• Software Quality Assurance as it relates to:Software Quality Assurance as it relates to:

– Process– People– TechnologyTechnology

• Application to SCRUM• Questions?

Page 18: Software Development QA Best Practices

Technology

System Intellectual Intelligence This practice is related to the system knowledge held internally by an organization and its owning

group. This knowledge ranges from architectural design to code practices and standards used in the application. This is a control measure for protection. Augmenting development teams with consultants is a common practice as a means of meeting surges and recessions in business demands. Nevertheless, retaining an underlying knowledge level is important for continuity and risk mitigation. The level of intellectual intelligence retained is unique to the specific software application, what business functions, and the value of these functions.

Application Ownership We have already discussed the concept of quality ownership. The practice is the answer to the

question “Who is the person that has the ultimate ownership of the application?” It is the one single ring able neck and product owner as described in SCRUM. IT has the responsibility for ensuring the system is maintained, sustained, and operational.

Automation Testing Tools Unit test automation, System or Functional test automation, Performance test automation, Regression test automation

Technical Infrastructure and Support Test Environments & Supporting applications and job scheduling tools

Page 19: Software Development QA Best Practices

TechnologyUtilization of single platform to support QA Processes The concept of a single platform to support multiple QA processes is not new. There are studies that

show how quality attributes can be lost between systems resulting in delivery of a product which is not what the client expected. The use of a single platform for work/project requests, requirements, test plans, test cases, design, development, test execution is very useful. It reduces the potential for loss of data and redundancy as well as provides one source for metric reporting.

Refactoring Refactoring is the concept of cleaning up, improving the inner working of a system. All system

functionality and outcomes are to remain unaffected given the same inputs. The amount of refactoring is dependent on the “swamp” or “ball of spaghetti” inside the applications. Lack of architecture direction and strategic planning of application growth can often result in this situation. Refactoring neither fixes bugs nor adds new functionality, though it might precede either activity. Rather, it improves the understandability of the code, changes its internal structure and design, and removes dead code. The amount of refactoring is based on the technical team’s willingness and speed to technical debt repayment plan.

Code Coverage Analysis/Tool The process of finding areas of a program not exercised by a set of test cases, creating additional test

cases to increase coverage, and determining a quantitative measure of code coverage, which is an indirect measure of quality. An optional aspect of code coverage analysis is identifying redundant test cases that do not increase coverage. Analysis of code coverage (or test coverage analysis) is a technique whereby test cases are created to exercise all parts of the software.

Page 20: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”• Software Quality Assurance as it relates to:

– Process– People– Technology

• Application to SCRUMApplication to SCRUM• Questions?

Page 21: Software Development QA Best Practices

The Scrum Framework

Potential Deployment

Sprint Review

Product & Team Backlog Formation

Sprint Planning

2 Parts: Selection and Decomp

Daily Scrum

Sprint 2-4 Weeks

Team Retrospective

Page 22: Software Development QA Best Practices

Agenda

• Definition of “Best Practices”• Software Quality Assurance as it relates to:

– Process– People– Technology

• Application to SCRUM• Questions?Questions?

Page 23: Software Development QA Best Practices

Questions?

Page 24: Software Development QA Best Practices

Thank You

For questions or more information, please contact:

Suzette Hackl, Senior Project Manager

[email protected]